Compare commits

...

87 Commits

Author SHA1 Message Date
alt-glitch add160fd1b chore: remove dead _save_oversized_tool_result after merge
Superseded by maybe_persist_tool_result from tools/tool_result_storage.
Function had zero call sites — only its own test suite referenced it.
2026-04-07 01:35:31 +00:00
alt-glitch 51cf4e0bbc fix: address PR review — alias expansion + L3 budget enforcement
- Add `shopt -s expand_aliases` to snapshot so aliases captured by
  `alias -p` actually work under `bash -c` (review comment #2)
- Pass threshold=0 in enforce_turn_budget() so L3 can force-persist
  results below the 50K default when aggregate budget is exceeded
  (review comment #3)
- Add regression test: 6x42K results (each under 50K) exceeding 200K
  budget are now correctly persisted
2026-04-07 01:33:05 +00:00
alt-glitch 72bd14e09d perf(environments): reduce per-command overhead in _before_execute hooks
- Daytona: skip refresh_data() API call unless sandbox was interrupted/errored
- Docker: cache _build_forward_env_args() to avoid re-reading .env every command
- All remote backends: TTL-based sync skip (5s) to avoid redundant dir walks
2026-04-07 01:33:05 +00:00
alt-glitch 2fe8fd8720 docs: replace L2/L3 jargon labels with descriptive comments
Expanded tool_result_storage.py module docstring to document the
three-level architecture. Replaced opaque L2/L3 labels at call
sites with self-describing comments.
2026-04-07 01:33:05 +00:00
alt-glitch ab35753c52 refactor(managed_modal): replace sentinel-key dicts with _ExecStartResult dataclass 2026-04-07 01:33:05 +00:00
alt-glitch 51bd4aecff fix: update stale _ModalProcessHandle/_DaytonaProcessHandle references in docstrings
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:33:05 +00:00
alt-glitch bead55bbcb refactor(environments): extract _ThreadedProcessHandle base class
Eliminates ~50 lines of duplicated pipe+thread+poll boilerplate between
_ModalProcessHandle and _DaytonaProcessHandle. Both now use closures
passed to the shared _ThreadedProcessHandle in base.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:33:05 +00:00
alt-glitch 5c491734f8 chore: reset uv.lock to main to fix nix build
The lockfile had drifted from main (debugpy, exa-py, version bump)
causing atomicwrites to fail building in the nix sandbox due to
missing setuptools in pyproject-build-systems.
2026-04-07 01:32:47 +00:00
alt-glitch 454edc7771 perf(ssh): add mtime-based caching to file sync
SSH _before_execute() ran rsync unconditionally before every command,
adding ~2.3s overhead even when zero bytes were transferred. This was
80% of per-command latency (actual execution: ~0.6s).

Add (mtime, size) caching — matching the pattern Modal and Daytona
already use — to skip rsync when local files haven't changed:

- Per-file mtime+size check for credential files
- Directory fingerprint (set of relpath/mtime/size tuples) for skills
- --delete flag on skills rsync to prune uninstalled skills
- Track created remote dirs to avoid redundant mkdir -p calls
- Cache invalidation on rsync failure (remote may have been wiped)
- force=True parameter as escape hatch for debugging

Before: ~3s per SSH command (2.3s rsync + 0.6s execution)
After:  ~0.6s per SSH command (mtime check + execution)
SSH test suite: 134s → 50s
2026-04-07 01:32:47 +00:00
alt-glitch 49d1390b40 fix(environments): move CWD tracking from remote file to in-band stdout
Previously, _wrap_command() wrote pwd to a file on the remote (container,
sandbox, SSH host), then _update_cwd_from_file() read it back via another
_run_bash() call. On Modal/Daytona this was a full API round-trip just to
read 20 bytes.

Now the wrapping template echoes the cwd to stdout with markers:
  printf '\n__HERMES_CWD__%s__HERMES_CWD__\n' "$(pwd -P)"

_extract_cwd_from_output() parses it from the output already in memory.
Zero extra round-trips on any backend. The cwdfile, _read_file_in_env(),
and per-backend overrides are all deleted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:32:47 +00:00
alt-glitch 7046d9b834 feat(agent_loop): add L2+L3 tool result persistence to eval path
- Add _tool_result_storage_dir to HermesAgentLoop.__init__
- Apply maybe_persist_tool_result() before tool message append
- Add enforce_turn_budget() after all tool calls in a turn
- Both wrapped in try/except (best-effort in eval path)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:32:47 +00:00
alt-glitch acbbdc05d4 feat(run_agent): replace 100K truncation with persist-to-disk (L2+L3)
Layer 2: Replace destructive head-truncation with maybe_persist_tool_result()
in both concurrent and sequential tool execution paths. Large results are
now written to ~/.hermes/sessions/{id}/tool-results/ with a 2KB preview
in context. Model can read_file the persisted path for full content.

Layer 3: Add enforce_turn_budget() after all tool results in a turn.
If aggregate exceeds 200K chars, persist largest results first until
under budget. Runs after concurrent futures.wait() (single-threaded).

Callbacks still receive full untruncated results before persistence.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:32:47 +00:00
alt-glitch 9cbfa1309b feat(tools): add per-tool result size thresholds and search_files output cap
Declares max_result_size_chars on each tool registration so the persistence
layer can apply per-tool limits instead of the global 50K default. Adds a
Layer 1 output cap inside search_tool() to prevent context overflow, and
adds a schema maximum of 10000 to the search_files limit parameter.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 01:31:48 +00:00
alt-glitch 3dfce74099 feat(tools): add tool result persistence module + registry support
Add tools/tool_result_storage.py implementing Layer 2 (per-result) and
Layer 3 (per-turn budget) persistence for large tool outputs. Results
exceeding thresholds are written to disk with a <persisted-output>
preview block replacing the inline content. Extend ToolEntry and
ToolRegistry with max_result_size_chars for per-tool threshold control.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:31:48 +00:00
alt-glitch 431262f9a5 test(daytona): update unit tests for unified execution model
- Update execute tests to account for init_session during __init__
- Fix CWD resolution tests for cwdfile reads
- Patch is_interrupted at base module level (where _wait_for_process uses it)
- Update stdin heredoc test for new call pattern
- 27/27 Daytona unit tests passing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:31:48 +00:00
alt-glitch ec43496d5a refactor(environments): delete persistent_shell.py + dead code — Phase 8
- DELETE persistent_shell.py entirely (277 lines removed)
- Remove _SHELL_NOISE_SUBSTRINGS, _clean_shell_noise, _extract_fenced_output
  from local.py (unused after fence marker removal)
- Adapt ManagedModalEnvironment to use BaseEnvironment + _wrap_command()
  while keeping its own HTTP-based execute()
- Remove _OUTPUT_FENCE constant

42/42 tests passing across all testable backends.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:31:48 +00:00
alt-glitch ef8a985ee3 refactor(environments): migrate Modal + Daytona to unified model — Phase 5+6
ModalEnvironment:
- Create _ModalProcessHandle adapter (async SDK → ProcessHandle protocol)
- Routes async sandbox.exec through thread + OS pipe for stdout
- Remove BaseModalExecutionEnvironment inheritance, use BaseEnvironment
- Remove _start_modal_exec/_poll_modal_exec/_cancel_modal_exec
- Move file sync to _before_execute hook
- Preserve _AsyncWorker, sandbox lifecycle, snapshot management

DaytonaEnvironment:
- Create _DaytonaProcessHandle adapter (blocking SDK → ProcessHandle)
- Preserves shell timeout wrapper (SDK timeout unreliable)
- Add _run_bash with heredoc stdin embedding
- Move file sync to _before_execute hook
- Preserve sandbox lifecycle, persistent resume logic

ManagedModal left unchanged (HTTP-based, keeps modal_common.py dep).
42/42 local tests passing. SDK backends untested (require credentials).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:31:41 +00:00
alt-glitch 0482133559 refactor(terminal_tool): remove persistent shell params from factory
- Remove persistent= from LocalEnvironment creation
- Remove persistent= from SSHEnvironment creation
- Container persistent_filesystem params unchanged (different concept)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:31:34 +00:00
alt-glitch 637bbbee7e refactor(environments): migrate Docker + SSH to unified model — Phase 2+4
DockerEnvironment:
- Add _run_bash/_run_bash_login, extract _build_forward_env_args()
- Remove execute() override with duplicate timeout/interrupt loop
- Remove -w flag (CWD handled by wrapping template)
- Call init_session() after container creation
- 42/42 tests passing on debian:bookworm-slim (21.7s)

SSHEnvironment:
- Remove PersistentShellMixin inheritance entirely
- Remove all IPC methods: _read_temp_files, _kill_shell_children,
  _cleanup_temp_files, _spawn_shell_process, _execute_oneshot
- Add _run_bash/_run_bash_login with shlex.quote for SSH transport
- Override _read_file_in_env with capture_output=True to suppress
  SSH connection warnings (post-quantum key exchange etc.)
- Move _sync_skills_and_credentials to _before_execute hook
- Remove persistent parameter
- 42/42 tests passing on SSH (173s, no more polling overhead)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:31:34 +00:00
alt-glitch cd814392ae refactor(environments): migrate SingularityEnvironment to unified model — Phase 3
- Add _run_bash/_run_bash_login, remove execute() override
- Remove --pwd flag (CWD handled by wrapping template)
- Remove is_interrupted import (handled by BaseEnvironment)
- Call init_session() after instance start

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:31:29 +00:00
alt-glitch fe2e1c16c6 refactor(environments): unify execution layer — Phase 0+1 (base + local)
Add spawn-per-call execution model to BaseEnvironment:
- ProcessHandle protocol for backend abstraction
- Shell snapshot creation (bash -l once, capture env to file)
- Command wrapping template (source snapshot + cd + eval + pwd tracking)
- Unified _wait_for_process with interrupt/timeout handling
- CWD tracking via cwdfile read after process exit

Migrate LocalEnvironment to unified model:
- Remove PersistentShellMixin inheritance
- Implement _run_bash (Popen + process group kill)
- Remove fence markers, oneshot method, shell noise cleanup
- Session snapshot captures env vars, functions, aliases

New test capabilities validated:
- CWD persists across execute() calls (was 37% manual cd prefix)
- stdin_data piping works
- Exit codes preserved through wrapper
- Single quotes survive eval escaping
- Snapshot fallback works when creation fails

42/42 tests passing on local backend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 01:31:08 +00:00
Teknium 0e336b0e71 fix: backfill codex stream output from output_item.done events (#5689)
Salvages the core fix from PR #5673 (egerev) onto current main.

The chatgpt.com/backend-api/codex endpoint streams valid output items
via response.output_item.done events, but the OpenAI SDK's
get_final_response() returns an empty output list. This caused every
Codex response to be rejected as invalid.

Fix: collect output_item.done events during streaming and backfill
response.output when get_final_response() returns empty. Falls back
to synthesizing from text deltas when no done events were received.

Also moves the synthesis logic from the validation loop (too late, from
#5681) into _run_codex_stream() (before the response leaves the
streaming function), and simplifies the validation to just log
diagnostics since recovery now happens upstream.

Co-authored-by: Egor <egerev@users.noreply.github.com>
2026-04-06 18:19:30 -07:00
Grateful Dave e5aaa38ca7 fix: sync openai-codex pool entry from ~/.codex/auth.json on exhaustion (#5610)
OpenAI OAuth refresh tokens are single-use and rotate on every refresh.
When the Codex CLI (or another Hermes profile) refreshes its token, the
pool entry's refresh_token becomes stale. Subsequent refresh attempts
fail with invalid_grant, and the entry enters a 24-hour exhaustion
cooldown with no recovery path.

This mirrors the existing _sync_anthropic_entry_from_credentials_file()
pattern: when an openai-codex entry is exhausted, compare its
refresh_token against ~/.codex/auth.json and sync the fresh pair if
they differ.

Fixes the common scenario where users run 'codex login' to refresh
their token externally and Hermes never picks it up.

Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai>
2026-04-06 18:16:56 -07:00
Teknium dc4c07ed9d fix: codex OAuth credential pool disconnect + expired token import (#5681)
Three bugs causing OpenAI Codex sessions to fail silently:

1. Credential pool vs legacy store disconnect: hermes auth and hermes
   model store device_code tokens in the credential pool, but
   get_codex_auth_status(), resolve_codex_runtime_credentials(), and
   _model_flow_openai_codex() only read from the legacy provider state.
   Fresh pool tokens were invisible to the auth status checks and model
   selection flow.

2. _import_codex_cli_tokens() imported expired tokens from ~/.codex/
   without checking JWT expiry. Combined with _login_openai_codex()
   saying 'Login successful!' for expired credentials, users got stuck
   in a loop of dead tokens being recycled.

3. _login_openai_codex() accepted expired tokens from
   resolve_codex_runtime_credentials() without validating expiry before
   telling the user login succeeded.

Fixes:
- get_codex_auth_status() now checks credential pool first, falls back
  to legacy provider state
- _model_flow_openai_codex() uses pool-aware auth status for token
  retrieval when fetching model lists
- _import_codex_cli_tokens() validates JWT exp claim, rejects expired
- _login_openai_codex() verifies resolved token isn't expiring before
  accepting existing credentials
- _run_codex_stream() logs response.incomplete/failed terminal events
  with status and incomplete_details for diagnostics
- Codex empty output recovery: captures streamed text during streaming
  and synthesizes a response when get_final_response() returns empty
  output (handles chatgpt.com backend-api edge cases)
2026-04-06 18:10:33 -07:00
Teknium 8cf013ecd9 fix: replace stale 'hermes login' refs with 'hermes auth' + fix credential removal re-seeding (#5670)
Two fixes:

1. Replace all stale 'hermes login' references with 'hermes auth' across
   auth.py, auxiliary_client.py, delegate_tool.py, config.py, run_agent.py,
   and documentation. The 'hermes login' command was deprecated; 'hermes auth'
   now handles OAuth credential management.

2. Fix credential removal not persisting for singleton-sourced credentials
   (device_code for openai-codex/nous, hermes_pkce for anthropic).
   auth_remove_command already cleared env vars for env-sourced credentials,
   but singleton credentials stored in the auth store were re-seeded by
   _seed_from_singletons() on the next load_pool() call. Now clears the
   underlying auth store entry when removing singleton-sourced credentials.
2026-04-06 17:17:57 -07:00
Teknium adb418fb53 fix: cross-platform browser test path separators
Use os.path.join for Windows install path so test passes on Linux
(os.path.join uses / on Linux, \ on Windows).
2026-04-06 16:54:16 -07:00
jtuki 57abc99315 feat(gateway): add per-group access control for Feishu
Add fine-grained authorization policies per Feishu group chat via
platforms.feishu.extra configuration.

- Add global bot-level admins that bypass all group restrictions
- Add per-group policies: open, allowlist, blacklist, admin_only, disabled
- Add default_group_policy fallback for chats without explicit rules
- Thread chat_id through group message gate for per-chat rule selection
- Match both open_id and user_id for backward compatibility
- Preserve existing FEISHU_ALLOWED_USERS / FEISHU_GROUP_POLICY behavior
- Add focused regression tests for all policy modes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:54:16 -07:00
jtuki 18727ca9aa refactor(gateway): simplify Feishu websocket config helpers
Consolidate coercion functions, extract loop readiness check, and deduplicate test mock setup to improve maintainability without changing behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:54:16 -07:00
jtuki 157d6184e3 fix(gateway): make Feishu websocket overrides effective at runtime
Reapply local reconnect and ping settings after the Feishu SDK refreshes its client config so user-provided websocket tuning actually takes effect.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:54:16 -07:00
jtuki ea31d9077c feat(gateway): add Feishu websocket ping timing overrides
Allow Feishu websocket keepalive timing to be configured via platform
extra config so disconnects can be detected faster in unstable networks.

New optional extra settings:
- ws_ping_interval
- ws_ping_timeout

These values are applied only when explicitly configured. Invalid values
fall back to the websocket library defaults by leaving the options unset.

This complements the reconnect timing settings added previously and helps
reduce total recovery time after network interruptions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:54:16 -07:00
jtuki 7d0bf15121 feat(gateway): add configurable Feishu websocket reconnect timing
Allow users to configure websocket reconnect behavior via platform extra
config to reduce reconnect latency in production environments.

The official Feishu SDK defaults to:
- First reconnect: random jitter 0-30 seconds
- Subsequent retries: 120 second intervals

This can cause 20-30 second delays before reconnection after network
interruptions. This commit makes these values configurable while keeping
the SDK defaults for backward compatibility.

Configuration via ~/.hermes/config.yaml:
```yaml
platforms:
  feishu:
    extra:
      ws_reconnect_nonce: 0        # Disable first-reconnect jitter (default: 30)
      ws_reconnect_interval: 3     # Retry every 3 seconds (default: 120)
```

Invalid values (negative numbers, non-integers) fall back to SDK defaults.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:54:16 -07:00
jtuki 7cf4bd06bf fix(gateway): fix Feishu reconnect message drops and shutdown hang
This commit fixes two critical bugs in the Feishu adapter that affect
message reliability and process lifecycle.

**Bug Fix 1: Intermittent Message Drops**

Root cause: Event handler was created once in __init__ and reused across
reconnects, causing callbacks to capture stale loop references. When the
adapter disconnected and reconnected, old callbacks continued firing with
invalid loop references, resulting in dropped messages with warnings:
"[Feishu] Dropping inbound message before adapter loop is ready"

Fix:
- Rebuild event handler on each connect (websocket/webhook)
- Clear handler on disconnect
- Ensure callbacks always capture current valid loop
- Add defensive loop.is_closed() checks with getattr for test compatibility
- Unify webhook dispatch path to use same loop checks as websocket mode

**Bug Fix 2: Process Hangs on Ctrl+C / SIGTERM**

Root cause: Feishu SDK's websocket client runs in a background thread with
an infinite _select() loop that never exits naturally. The thread was never
properly joined on disconnect, causing processes to hang indefinitely after
Ctrl+C or gateway stop commands.

Fix:
- Store reference to thread-local event loop (_ws_thread_loop)
- On disconnect, cancel all tasks in thread loop and stop it gracefully
  via call_soon_threadsafe()
- Await thread future with 10s timeout
- Clean up pending tasks in thread's finally block before closing loop
- Add detailed debug logging for disconnect flow

**Additional Improvements:**
- Add regression tests for disconnect cleanup and webhook dispatch
- Ensure all event callbacks check loop readiness before dispatching

Tested on Linux with websocket mode. All Feishu tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:54:16 -07:00
Ruzzgar abd24d381b Implement comprehensive browser path discovery for Windows 2026-04-06 16:54:16 -07:00
Tianxiao 8a29b49036 fix(cli): handle CJK wide chars in TUI input height 2026-04-06 16:54:16 -07:00
kshitijk4poor 05f9267938 fix(matrix): hard-fail E2EE when python-olm missing + stable MATRIX_DEVICE_ID
Two issues caused Matrix E2EE to silently not work in encrypted rooms:

1. When matrix-nio is installed without the [e2e] extra (no python-olm /
   libolm), nio.crypto.ENCRYPTION_ENABLED is False and client.olm is
   never initialized. The adapter logged warnings but returned True from
   connect(), so the bot appeared online but could never decrypt messages.
   Now: check_matrix_requirements() and connect() both hard-fail with a
   clear error message when MATRIX_ENCRYPTION=true but E2EE deps are
   missing.

2. Without a stable device_id, the bot gets a new device identity on each
   restart. Other clients see it as "unknown device" and refuse to share
   Megolm session keys. Now: MATRIX_DEVICE_ID env var lets users pin a
   stable device identity that persists across restarts and is passed to
   nio.AsyncClient constructor + restore_login().

Changes:
- gateway/platforms/matrix.py: add _check_e2ee_deps(), hard-fail in
  connect() and check_matrix_requirements(), MATRIX_DEVICE_ID support
  in constructor + restore_login
- gateway/config.py: plumb MATRIX_DEVICE_ID into platform extras
- hermes_cli/config.py: add MATRIX_DEVICE_ID to OPTIONAL_ENV_VARS

Closes #3521
2026-04-06 16:54:16 -07:00
tymrtn 40527ff5e3 fix(auth): actionable error message when Codex refresh token is reused
When the Codex CLI (or VS Code extension) consumes a refresh token before
Hermes can use it, Hermes previously surfaced a generic 401 error with no
actionable guidance.

- In `refresh_codex_oauth_pure`: detect `refresh_token_reused` from the
  OAuth endpoint and raise an AuthError explaining the cause and the exact
  steps to recover (run `codex` to refresh, then `hermes login`).
- In `run_agent.py`: when provider is `openai-codex` and HTTP 401 is
  received, show Codex-specific recovery steps instead of the generic
  "check your API key" message.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 16:50:10 -07:00
Zainan Victor Zhou 190471fdc0 docs: use HERMES_HOME in google-workspace skill examples
- avoid hard-coded ~/.hermes paths in the setup and API shorthands
- prefer HERMES_HOME with a sane default to /Users/peteradams/.hermes
- keep the examples aligned with profile-aware Hermes installs
2026-04-06 16:50:07 -07:00
Zainan Victor Zhou 83df001d01 fix: allow google-workspace skill scripts to run directly
- fall back to adding the repo root to sys.path when hermes_constants is not importable
- fixes direct execution of setup.py and google_api.py from the repo checkout
- keeps the upstream PR scoped to the google-workspace compatibility fix
2026-04-06 16:50:07 -07:00
WAXLYY 1c0183ec71 fix(gateway): sanitize media URLs in base platform logs 2026-04-06 16:50:05 -07:00
KangYu b26e85bf9d Fix compaction summary retries for temperature-restricted models 2026-04-06 16:49:57 -07:00
charliekerfoot e9b5864b3f fix: multiple platform adaptors concurrency 2026-04-06 16:49:54 -07:00
WAXLYY c1818b7e9e fix(tools): redact query secrets in send_message errors 2026-04-06 16:49:52 -07:00
Neri Cervin f3ae2491a3 fix: detect correct message type from file mime instead of blanket DOCUMENT
Images need PHOTO for vision, audio needs VOICE for STT,
and other files get DOCUMENT for text inlining.
2026-04-06 16:49:45 -07:00
Neri Cervin 3282b7066c fix(mattermost): set message type to DOCUMENT when post has file attachments
The Mattermost adapter downloads file attachments correctly but
never updates msg_type from TEXT to DOCUMENT. This means the
document enrichment block in gateway/run.py (which requires
MessageType.DOCUMENT) never executes — text files are not
inlined, and the agent is never notified about attached files.

The user sends a file, the adapter downloads it to the local
cache, but the agent sees an empty message and responds with
'I didn't receive any file'.

Set msg_type to DOCUMENT when file_ids is non-empty, matching
the behavior of the Telegram and Discord adapters.
2026-04-06 16:49:45 -07:00
ryanautomated 0f9aa57069 fix: silent memory flush failure on /new and /resume commands
The _async_flush_memories() helper accepts (session_id) but both the
/new and /resume handlers passed two arguments (session_id, session_key).
The TypeError was silently swallowed at DEBUG level, so memory extraction
never ran when users typed /new or /resume.

One call site (the session expiry watcher) was already fixed in 9c96f669,
but /new and /resume were missed.

- gateway/run.py:3247 — remove stray session_key from /new handler
- gateway/run.py:4989 — remove stray session_key from /resume handler
- tests/gateway/test_resume_command.py:222 — update test assertion
2026-04-06 16:49:42 -07:00
Myeongwon Choi ea16949422 fix(cron): suppress delivery when [SILENT] appears anywhere in response
Previously the scheduler checked startswith('[SILENT]'), so agents that
appended [SILENT] after an explanation (e.g. 'N items filtered.\n\n[SILENT]')
would still trigger delivery.

Change the check to 'in' so the marker is caught regardless of position.
Add test_silent_trailing_suppresses_delivery to cover this case.
2026-04-06 16:49:40 -07:00
charliekerfoot 3b4dfc8e22 fix(tools): portable base64 encoding for image reading on macOS 2026-04-06 16:49:32 -07:00
KangYu 77610961be Lower Telegram fallback activation log to info 2026-04-06 16:49:30 -07:00
Simon Brumfield e131f13662 fix(doctor): use recall_mode instead of memory_mode on HonchoClientConfig 2026-04-06 16:49:27 -07:00
dagbs e7698521e7 fix(openviking): add atexit safety net for session commit
Ensures pending sessions are committed on process exit even if
shutdown_memory_provider is never called (gateway crash, SIGKILL,
or exception in _async_flush_memories preventing shutdown).

Also reorders on_session_end to wait for the pending sync thread
before checking turn_count, so the last turn's messages are flushed.

Based on PR #4919 by dagbs.
2026-04-06 16:45:53 -07:00
Teknium f071b1832a docs: document rich requires_env format and install-time prompting
Updates the plugin build guide and features page to reflect the
interactive env var prompting added in PR #5470. Documents the rich
manifest format (name/description/url/secret) alongside the simple
string format.
2026-04-06 16:43:42 -07:00
Nick 4f03b9a419 feat(webhook): add {__raw__} template token and thread_id passthrough for forum topics
- {__raw__} in webhook prompt templates dumps the full JSON payload (truncated at 4000 chars)
- _deliver_cross_platform now passes thread_id/message_thread_id from deliver_extra as metadata, enabling Telegram forum topic delivery
- Tests for both features
2026-04-06 16:42:52 -07:00
Teknium 631d159864 fix: use display_hermes_home() for profile-aware paths in plugin env prompts
Follow-up to PR #5470. Replaces hardcoded ~/.hermes/.env references with
display_hermes_home() for correct behavior under profiles. Also updates
PluginManifest.requires_env type hint to List[Union[str, Dict[str, Any]]]
to document the rich format introduced in #5470.
2026-04-06 16:40:15 -07:00
kshitijk4poor 9201370c7e feat(plugins): prompt for required env vars during hermes plugins install
Read requires_env from plugin.yaml after install and interactively
prompt for any missing environment variables, saving them to
~/.hermes/.env.

Supports two manifest formats:

  Simple (backwards-compatible):
    requires_env:
      - MY_API_KEY

  Rich (with metadata):
    requires_env:
      - name: MY_API_KEY
        description: "API key for Acme"
        url: "https://acme.com/keys"
        secret: true

Already-set variables are skipped. Empty input skips gracefully.
Secret values use getpass (hidden input). Ctrl+C aborts remaining
prompts without error.
2026-04-06 16:37:53 -07:00
Teknium 539629923c docs(llm-wiki): add Obsidian Headless setup for servers (#5660)
Adds obsidian-headless (npm) setup guide to the Obsidian Integration
section — Node 22+, ob login, sync-create-remote, sync-setup, systemd
service for continuous background sync. Covers the full headless
workflow for agents running on servers syncing to Obsidian desktop on
other devices.
2026-04-06 16:37:14 -07:00
Siddharth Balyan e651e04100 fix(nix): read version, regen uv.lock, fix packages.nix to add hermes_logging (#5651)
* - read version from pyproject for nix
- regen uv.lock
- add hermes_logging to packages.nix

* fix secret regen w/ sops
2026-04-07 04:21:19 +05:30
Siddharth Balyan 7b129636f0 feat(tools): add Firecrawl cloud browser provider (#5628)
* feat(tools): add Firecrawl cloud browser provider

Adds Firecrawl (https://firecrawl.dev) as a cloud browser provider
alongside Browserbase and Browser Use. All browser tools route through
Firecrawl's cloud browser via CDP when selected.

- tools/browser_providers/firecrawl.py — FirecrawlProvider
- tools/browser_tool.py — register in _PROVIDER_REGISTRY
- hermes_cli/tools_config.py — add to onboarding provider picker
- hermes_cli/setup.py — add to setup summary
- hermes_cli/config.py — add FIRECRAWL_BROWSER_TTL config
- website/docs/ — browser docs and env var reference

Based on #4490 by @developersdigest.

Co-Authored-By: Developers Digest <124798203+developersdigest@users.noreply.github.com>

* refactor: simplify FirecrawlProvider.emergency_cleanup

Use self._headers() and self._api_url() instead of duplicating
env-var reads and header construction.

* fix: recognize Firecrawl in subscription browser detection

_resolve_browser_feature_state() now handles "firecrawl" as a direct
browser provider (same pattern as "browser-use"), so hermes setup
summary correctly shows "Browser Automation (Firecrawl)" instead of
misreporting as "Local browser".

Also fixes test_config_version_unchanged assertion (11 → 12).

---------

Co-authored-by: Developers Digest <124798203+developersdigest@users.noreply.github.com>
2026-04-07 02:35:26 +05:30
Teknium 150f70f821 feat(skills): add skill config interface + llm-wiki skill (#5635)
Skills can now declare config.yaml settings via metadata.hermes.config
in their SKILL.md frontmatter. Values are stored under skills.config.*
namespace, prompted during hermes config migrate, shown in hermes config
show, and injected into the skill context at load time.

Also adds the llm-wiki skill (Karpathy's LLM Wiki pattern) as the first
skill to use the new config interface, declaring wiki.path.

Skill config interface (new):
- agent/skill_utils.py: extract_skill_config_vars(), discover_all_skill_config_vars(),
  resolve_skill_config_values(), SKILL_CONFIG_PREFIX
- agent/skill_commands.py: _inject_skill_config() injects resolved values
  into skill messages as [Skill config: ...] block
- hermes_cli/config.py: get_missing_skill_config_vars(), skill config
  prompting in migrate_config(), Skill Settings in show_config()

LLM Wiki skill (skills/research/llm-wiki/SKILL.md):
- Three-layer architecture (raw sources, wiki pages, schema)
- Three operations (ingest, query, lint)
- Session orientation, page thresholds, tag taxonomy, update policy,
  scaling guidance, log rotation, archiving workflow

Docs: creating-skills.md, configuration.md, skills.md, skills-catalog.md

Closes #5100
2026-04-06 13:49:13 -07:00
Mikita Lisavets 29b5ec2555 fix: clear session-scoped model after session reset 2026-04-06 13:20:01 -07:00
Mikita Lisavets 9afb9a6cb2 fix: clear session-scoped model overrides during session reset 2026-04-06 13:20:01 -07:00
donrhmexe 2c814d7b5d fix: /model --global writes model.name instead of model.default
The canonical config key for model name is model.default (used by setup,
auth, runtime_provider, profile list, and CLI startup). But /model --global
wrote to model.name in both gateway and CLI paths.

This caused:
- hermes profile list showing the old model (reads model.default)
- Gateway restart reverting to the old model (_resolve_gateway_model reads model.default)
- CLI startup using the old model (main.py reads model.default)

The only reason it appeared to work in Telegram was the cached agent
staying alive with the in-place switch.

Fix: change all 3 write/read sites to use model.default.
2026-04-06 13:20:01 -07:00
BongSuCHOI ad567c9a8f fix: subagent toolset inheritance when parent enabled_toolsets is None
When parent_agent.enabled_toolsets is None (the default, meaning all tools
are enabled), subagents incorrectly fell back to DEFAULT_TOOLSETS
(['terminal', 'file', 'web']) instead of inheriting the parent's full
toolset.

Root cause:
- Line 188 used 'or' fallback: None or DEFAULT_TOOLSETS evaluates to
  DEFAULT_TOOLSETS
- Line 192 checked truthiness: None is falsy, falling through to else

Fix:
- Use 'is not None' checks instead of truthiness
- When enabled_toolsets is None, derive effective toolsets from
  parent_agent.valid_tool_names via the tool registry

Fixes the bug introduced in f75b1d21b and repeated in e5d14445e (PR #3269).
2026-04-06 13:20:01 -07:00
donrhmexe ff655de481 fix: model alias fallback uses authenticated providers instead of hardcoded openrouter/nous
When an alias like 'claude' can't be resolved on the current provider,
_resolve_alias_fallback() tries other providers. Previously it hardcoded
('openrouter', 'nous') — so '/model claude' on z.ai would resolve to
openrouter even if the user doesn't have openrouter credentials but does
have anthropic.

Now the fallback uses the user's actual authenticated providers (detected
via list_authenticated_providers which is backed by the models.dev
in-memory cache). If no authenticated providers are found, falls back to
the old ('openrouter', 'nous') for backwards compatibility.

New helper: get_authenticated_provider_slugs() returns just the slug
strings from list_authenticated_providers().
2026-04-06 13:20:01 -07:00
Ayman Kamal 96f85b03cd fix: handle launchctl kickstart exit code 113 in launchd_start()
launchctl kickstart returns exit code 113 ("Could not find service") when
the plist exists but the job hasn't been bootstrapped into the runtime domain.
The existing recovery path only caught exit code 3 ("unloaded"), causing an
unhandled CalledProcessError.

Exit code 113 means the same thing practically -- the service definition needs
bootstrapping before it can be kicked. Add it to the same recovery path that
already handles exit 3, matching the existing pattern in launchd_stop().

Follow-up: add a unit test covering the 113 recovery path.
2026-04-06 13:20:01 -07:00
Dusk1e 1a2f109d8e Ensure atomic writes for gateway channel directory cache to prevent truncation 2026-04-06 13:20:01 -07:00
Mariano A. Nicolini af9a9f773c fix(security): sanitize workdir parameter in terminal tool backends
Shell injection via unquoted workdir interpolation in docker, singularity,
and SSH backends.  When workdir contained shell metacharacters (e.g.
~/;id), arbitrary commands could execute.

Changes:
- Add shlex.quote() at each interpolation point in docker.py,
  singularity.py, and ssh.py with tilde-aware quoting (keep ~
  unquoted for shell expansion, quote only the subpath)
- Add _validate_workdir() allowlist in terminal_tool.py as
  defense-in-depth before workdir reaches any backend

Original work by Mariano A. Nicolini (PR #5620).  Salvaged with fixes
for tilde expansion (shlex.quote breaks cd ~/path) and replaced
incomplete deny-list with strict character allowlist.

Co-authored-by: Mariano A. Nicolini <entropidelic@users.noreply.github.com>
2026-04-06 13:19:22 -07:00
Teknium 537a2b8bb8 docs: add WSL2 networking guide for local model servers (#5616)
Windows users running Hermes in WSL2 with model servers on the Windows
host hit 'connection refused' because WSL2's NAT networking means
localhost points to the VM, not Windows.

Covers:
- Mirrored networking mode (Win 11 22H2+) — makes localhost work
- NAT mode fallback using the host IP via ip route
- Per-server bind address table (Ollama, LM Studio, llama-server,
  vLLM, SGLang)
- Detailed Ollama Windows service config for OLLAMA_HOST
- Windows Firewall rules for WSL2 connections
- Quick verification steps
- Cross-reference from Troubleshooting section
2026-04-06 13:01:18 -07:00
Teknium 261e2ee862 fix: restore Path import in env_passthrough.py (removed by #5526)
The ContextVar migration removed 'from pathlib import Path' but Path
is still used in _load_config_passthrough(). Without this import,
config-based env passthrough would raise NameError.
2026-04-06 12:42:16 -07:00
Awsh1 878b1d3d33 fix(cron): harden scheduler against path traversal and env leaks
Cherry-picked from PR #5503 by Awsh1.

- Validate ALL script paths (absolute, relative, tilde) against scripts_dir boundary
- Add API-boundary validation in cronjob_tools.py
- Move os.environ injections inside try block so finally cleanup always runs
- Comprehensive regression tests for path containment bypass
2026-04-06 12:42:16 -07:00
Dusk1e 7d0953d6ff security(gateway): isolate env/credential registries using ContextVars 2026-04-06 12:42:16 -07:00
Teknium da02a4e283 fix: auxiliary client payment fallback — retry with next provider on 402 (#5599)
When a user runs out of OpenRouter credits and switches to Codex (or any
other provider), auxiliary tasks (compression, vision, web_extract) would
still try OpenRouter first and fail with 402.  Two fixes:

1. Payment fallback in call_llm(): When a resolved provider returns HTTP 402
   or a credit-related error, automatically retry with the next available
   provider in the auto-detection chain.  Skips the depleted provider and
   tries Nous → Custom → Codex → API-key providers.

2. Remove hardcoded OpenRouter fallback: The old code fell back specifically
   to OpenRouter when auto/custom resolution returned no client.  Now falls
   back to the full auto-detection chain, which handles any available
   provider — not just OpenRouter.

Also extracts _get_provider_chain() as a shared function (replaces inline
tuple in _resolve_auto and the new fallback), built at call time so test
patches on _try_* functions remain visible.

Adds 16 tests covering _is_payment_error(), _get_provider_chain(),
_try_payment_fallback(), and call_llm() integration with 402 retry.
2026-04-06 12:41:40 -07:00
Teknium 8ffd44a6f9 feat(discord): register skills as native slash commands via shared gateway logic (#5603)
Centralize the skill → slash command registration that Telegram already had
in commands.py so Discord uses the exact same priority system, filtering,
and cap enforcement:

  1. Core/built-in commands (never trimmed)
  2. Plugin commands (never trimmed)
  3. Skill commands (fill remaining slots, alphabetical, only tier trimmed)

Changes:

hermes_cli/commands.py:
  - Rename _TG_NAME_LIMIT → _CMD_NAME_LIMIT (32 chars shared by both platforms)
  - Rename _clamp_telegram_names → _clamp_command_names (generic)
  - Extract _collect_gateway_skill_entries() — shared plugin + skill
    collection with platform filtering, name sanitization, description
    truncation, and cap enforcement
  - Refactor telegram_menu_commands() to use the shared helper
  - Add discord_skill_commands() that returns (name, desc, cmd_key) triples
  - Preserve _sanitize_telegram_name() for Telegram-specific name cleaning

gateway/platforms/discord.py:
  - Call discord_skill_commands() from _register_slash_commands()
  - Create app_commands.Command per skill entry with cmd_key callback
  - Respect 100-command global Discord limit
  - Log warning when skills are skipped due to cap

Backward-compat aliases preserved for _TG_NAME_LIMIT and
_clamp_telegram_names.

Tests: 9 new tests (7 Discord + 2 backward-compat), 98 total pass.

Inspired by PR #5498 (sprmn24). Closes #5480.
2026-04-06 12:09:36 -07:00
Julien Talbot 92c19924a9 feat: add xAI prompt caching via x-grok-conv-id header
When using xAI's API directly (base_url contains x.ai), send the
x-grok-conv-id header set to the Hermes session_id. This routes
consecutive requests to the same server, maximizing automatic
prompt cache hits.

Ref: https://docs.x.ai/developers/advanced-api-usage/prompt-caching
2026-04-06 12:06:33 -07:00
SHL0MS 0afa3a87d4 Merge pull request #5600 from SHL0MS/feat/p5js-skill
feat(skills): add p5js creative coding skill
2026-04-06 14:52:27 -04:00
Teknium 3d08a2fa1b fix: extract MEDIA: tags from cron delivery before sending (#5598)
The cron scheduler delivery path passed raw text including MEDIA: tags
to _send_to_platform(), so media attachments were delivered as literal
text instead of actual files. The send function already supports
media_files= but the cron path never used it.

Now calls BasePlatformAdapter.extract_media() to split media paths
from text before sending, matching the gateway's normal message flow.

Salvaged from PR #4877 by robert-hoffmann.
2026-04-06 11:42:44 -07:00
kshitijk4poor 5e88eb2ba0 fix(signal): implement send_image_file, send_voice, and send_video for MEDIA: tag delivery
The Signal adapter inherited base class defaults for send_image_file(),
send_voice(), and send_video() which only sent the file path as text
(e.g. '🖼️ Image: /tmp/chart.png') instead of actually delivering the file
as a Signal attachment.

When agent responses contain MEDIA:/path/to/file tags, the gateway
media pipeline extracts them and routes through these methods by file
type. Without proper overrides, image/audio/video files were never
actually delivered to Signal users.

Extract a shared _send_attachment() helper that handles all file
validation, size checking, group/DM routing, and RPC dispatch. The four
public methods (send_document, send_image_file, send_voice, send_video)
now delegate to this helper, following the same pattern used by WhatsApp
(_send_media_to_bridge) and Discord (_send_file_attachment).

The helper also uses a single stat() call with try/except FileNotFoundError
instead of the previous exists() + stat() two-syscall pattern, eliminating
a TOCTOU race. As a bonus, send_document() now gains the 100MB size check
that was previously missing (inconsistency with send_image).

Add 25 tests covering all methods plus MEDIA: tag extraction integration,
method-override guards, and send_document's new size check.

Fixes #5105
2026-04-06 11:41:34 -07:00
SHL0MS 17e2a27c51 feat(skills): add p5js creative coding skill
Production pipeline for interactive and generative visual art using p5.js.

Covers 7 modes: generative art, data visualization, interactive experiences,
animation/motion graphics, 3D scenes, image processing, and audio-reactive.

Includes:
- SKILL.md with creative standard, pipeline, and critical implementation notes
- 10 reference files covering core API, shapes, visual effects (noise, flow
  fields, particles, domain warp, attractors, L-systems, circle packing,
  bloom, reaction-diffusion), animation (easing, springs, state machines,
  scene transitions), typography, color systems, WebGL/3D/shaders,
  interaction, and comprehensive export pipeline
- Deterministic headless frame capture via Puppeteer (noLoop + redraw)
- ffmpeg render pipeline for MP4 video export
- Per-clip architecture for multi-scene video production
- Interactive viewer template with seed navigation and parameter controls
- Performance guidance: FES disable, Math.* hot loops, per-pixel budgets
- Addon library coverage: p5.brush, p5.grain, CCapture.js, p5.js-svg
- fxhash/Art Blocks generative platform conventions
- p5.js 2.0 migration guide (async setup, OKLCH, splineVertex, shader.modify)
- 13 documented common mistakes and troubleshooting patterns

17 files, ~5,900 lines.
2026-04-06 14:39:00 -04:00
kshitijk4poor 214e60c951 fix: sanitize Telegram command names to strip invalid characters
Telegram Bot API requires command names to contain only lowercase a-z,
digits 0-9, and underscores. Skill/plugin names containing characters
like +, /, @, or . caused set_my_commands to fail with
Bot_command_invalid.

Two-layer fix:
- scan_skill_commands(): strip non-alphanumeric/non-hyphen chars from
  cmd_key at source, collapse consecutive hyphens, trim edges, skip
  names that sanitize to empty string
- _sanitize_telegram_name(): centralized helper used by all 3 Telegram
  name generation sites (core commands, plugin commands, skill commands)
  with empty-name guard at each call site

Closes #5534
2026-04-06 11:27:28 -07:00
ClintonEmok f77be22c65 Fix #5211: Preserve dots in OpenCode Go model names
OpenCode Go model names with dots (minimax-m2.7, glm-4.5, kimi-k2.5)
were being mangled to hyphens (minimax-m2-7), causing HTTP 401 errors.

Two code paths were affected:
1. model_normalize.py: opencode-go was incorrectly in DOT_TO_HYPHEN_PROVIDERS
2. run_agent.py: _anthropic_preserve_dots() did not check for opencode-go

Fix:
- Remove opencode-go from _DOT_TO_HYPHEN_PROVIDERS (dots are correct for Go)
- Add opencode-go to _anthropic_preserve_dots() provider check
- Add opencode.ai/zen/go to base_url fallback check
- Add regression tests in tests/test_model_normalize.py

Co-authored-by: jacob3712 <jacob3712@users.noreply.github.com>
2026-04-06 11:25:06 -07:00
Teknium 582dbbbbf7 feat: add grok to TOOL_USE_ENFORCEMENT_MODELS for direct xAI usage (#5595)
Grok models (x-ai/grok-4.20-beta, grok-code-fast-1) now receive tool-use
enforcement guidance, steering them to actually call tools instead of
describing intended actions. Matches both OpenRouter (x-ai/grok-*) and
direct xAI API usage.
2026-04-06 11:22:07 -07:00
SHL0MS 0bac07ded3 Merge pull request #5588 from SHL0MS/feat/manim-skill-deep-expansion
docs(manim-video): add 5 new reference files — design thinking, updaters, paper explainer, decorations, production quality
2026-04-06 13:58:00 -04:00
SHL0MS a912cd4568 docs(manim-video): add 5 new reference files — design thinking, updaters, paper explainer, decorations, production quality
Five new reference files expanding the skill from rendering knowledge
into production methodology:

animation-design-thinking.md (161 lines):
  When to animate vs show static, concept decomposition into visual
  beats, pacing rules, narration sync, equation reveal strategies,
  architecture diagram patterns, common design mistakes.

updaters-and-trackers.md (260 lines):
  Deep ValueTracker mental model, lambda/time-based/always_redraw
  updaters, DecimalNumber and Variable live displays, animation-based
  updaters, 4 complete practical patterns (dot tracing, live area,
  connected diagram, parameter exploration).

paper-explainer.md (255 lines):
  Full workflow for turning research papers into animations. Audience
  selection, 5-minute template, pre-code gates (narration, scene list,
  style contract), equation reveal strategies, architecture diagram
  building, results animation, domain-specific patterns for ML/physics/
  biomedical papers.

decorations.md (202 lines):
  SurroundingRectangle, BackgroundRectangle, Brace, arrows (straight,
  curved, labeled), DashedLine, Angle/RightAngle, Cross, Underline,
  color highlighting workflows, annotation lifecycle pattern.

production-quality.md (190 lines):
  Pre-code, pre-render, post-render checklists. Text overlap prevention,
  spatial layout coordinate budget, max simultaneous elements, animation
  variety audit, tempo curve, color consistency, data viz minimums.

Total skill now: 14 reference files, 2614 lines.
2026-04-06 13:51:36 -04:00
Teknium cc7136b1ac fix: update Gemini model catalog + wire models.dev as live model source
Follow-up for salvaged PR #5494:
- Update model catalog to Gemini 3.x + Gemma 4 (drop deprecated 2.0)
- Add list_agentic_models() to models_dev.py with noise filter
- Wire models.dev into _model_flow_api_key_provider as primary source
  (static curated list serves as offline fallback)
- Add gemini -> google mapping in PROVIDER_TO_MODELS_DEV
- Fix Gemma 4 context lengths to 256K (models.dev values)
- Update auxiliary model to gemini-3-flash-preview
- Expand tests: 3.x catalog, context lengths, models.dev integration
2026-04-06 10:28:03 -07:00
Teknium 6dfab35501 feat(providers): add Google AI Studio (Gemini) as a first-class provider
Cherry-picked from PR #5494 by kshitijk4poor.
Adds native Gemini support via Google's OpenAI-compatible endpoint.
Zero new dependencies.
2026-04-06 10:28:03 -07:00
SHL0MS 85973e0082 fix(nous): don't use OAuth access_token as inference API key
When agent_key is missing from auth state (expired, not yet minted,
or mint failed silently), the fallback chain fell through to
access_token — an OAuth bearer token for the Nous portal API, not
an inference credential. The Nous inference API returns 404 because
the OAuth token is not a valid inference key.

Remove the access_token fallback so an empty agent_key correctly
triggers resolve_nous_runtime_credentials() to mint a fresh key.

Closes #5562
2026-04-06 10:04:02 -07:00
Austin Pickett eceb89b824 Merge pull request #4664 from NousResearch/fix/various-qa
fix: re-order providers, Quick Install
2026-04-06 08:35:34 -07:00
Austin Pickett 79aeaa97e6 fix: re-order providers,Quick Install, subscription polling 2026-04-06 11:16:07 -04:00
141 changed files with 16155 additions and 2658 deletions
+10
View File
@@ -14,6 +14,16 @@
# LLM_MODEL is no longer read from .env — this line is kept for reference only.
# LLM_MODEL=anthropic/claude-opus-4.6
# =============================================================================
# LLM PROVIDER (Google AI Studio / Gemini)
# =============================================================================
# Native Gemini API via Google's OpenAI-compatible endpoint.
# Get your key at: https://aistudio.google.com/app/apikey
# GOOGLE_API_KEY=your_google_ai_studio_key_here
# GEMINI_API_KEY=your_gemini_key_here # alias for GOOGLE_API_KEY
# Optional base URL override (default: Google's OpenAI-compatible endpoint)
# GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
# =============================================================================
# LLM PROVIDER (z.ai / GLM)
# =============================================================================
+123 -13
View File
@@ -34,6 +34,12 @@ than the provider's default.
Per-task direct endpoint overrides (e.g. AUXILIARY_VISION_BASE_URL,
AUXILIARY_VISION_API_KEY) let callers route a specific auxiliary task to a
custom OpenAI-compatible endpoint without touching the main model settings.
Payment / credit exhaustion fallback:
When a resolved provider returns HTTP 402 or a credit-related error,
call_llm() automatically retries with the next available provider in the
auto-detection chain. This handles the common case where a user depletes
their OpenRouter balance but has Codex OAuth or another provider available.
"""
import json
@@ -55,6 +61,7 @@ logger = logging.getLogger(__name__)
# Default auxiliary models for direct API-key providers (cheap/fast for side tasks)
_API_KEY_PROVIDER_AUX_MODELS: Dict[str, str] = {
"gemini": "gemini-3-flash-preview",
"zai": "glm-4.5-flash",
"kimi-coding": "kimi-k2-turbo-preview",
"minimax": "MiniMax-M2.7-highspeed",
@@ -842,7 +849,7 @@ def _resolve_forced_provider(forced: str) -> Tuple[Optional[OpenAI], Optional[st
if forced == "nous":
client, model = _try_nous()
if client is None:
logger.warning("auxiliary.provider=nous but Nous Portal not configured (run: hermes login)")
logger.warning("auxiliary.provider=nous but Nous Portal not configured (run: hermes auth)")
return client, model
if forced == "codex":
@@ -873,10 +880,90 @@ _AUTO_PROVIDER_LABELS = {
"_resolve_api_key_provider": "api-key",
}
_AGGREGATOR_PROVIDERS = frozenset({"openrouter", "nous"})
def _get_provider_chain() -> List[tuple]:
"""Return the ordered provider detection chain.
Built at call time (not module level) so that test patches
on the ``_try_*`` functions are picked up correctly.
"""
return [
("openrouter", _try_openrouter),
("nous", _try_nous),
("local/custom", _try_custom_endpoint),
("openai-codex", _try_codex),
("api-key", _resolve_api_key_provider),
]
def _is_payment_error(exc: Exception) -> bool:
"""Detect payment/credit/quota exhaustion errors.
Returns True for HTTP 402 (Payment Required) and for 429/other errors
whose message indicates billing exhaustion rather than rate limiting.
"""
status = getattr(exc, "status_code", None)
if status == 402:
return True
err_lower = str(exc).lower()
# OpenRouter and other providers include "credits" or "afford" in 402 bodies,
# but sometimes wrap them in 429 or other codes.
if status in (402, 429, None):
if any(kw in err_lower for kw in ("credits", "insufficient funds",
"can only afford", "billing",
"payment required")):
return True
return False
def _try_payment_fallback(
failed_provider: str,
task: str = None,
) -> Tuple[Optional[Any], Optional[str], str]:
"""Try alternative providers after a payment/credit error.
Iterates the standard auto-detection chain, skipping the provider that
returned a payment error.
Returns:
(client, model, provider_label) or (None, None, "") if no fallback.
"""
# Normalise the failed provider label for matching.
skip = failed_provider.lower().strip()
# Also skip Step-1 main-provider path if it maps to the same backend.
# (e.g. main_provider="openrouter" → skip "openrouter" in chain)
main_provider = _read_main_provider()
skip_labels = {skip}
if main_provider and main_provider.lower() in skip:
skip_labels.add(main_provider.lower())
# Map common resolved_provider values back to chain labels.
_alias_to_label = {"openrouter": "openrouter", "nous": "nous",
"openai-codex": "openai-codex", "codex": "openai-codex",
"custom": "local/custom", "local/custom": "local/custom"}
skip_chain_labels = {_alias_to_label.get(s, s) for s in skip_labels}
tried = []
for label, try_fn in _get_provider_chain():
if label in skip_chain_labels:
continue
client, model = try_fn()
if client is not None:
logger.info(
"Auxiliary %s: payment error on %s — falling back to %s (%s)",
task or "call", failed_provider, label, model or "default",
)
return client, model, label
tried.append(label)
logger.warning(
"Auxiliary %s: payment error on %s and no fallback available (tried: %s)",
task or "call", failed_provider, ", ".join(tried),
)
return None, None, ""
def _resolve_auto() -> Tuple[Optional[OpenAI], Optional[str]]:
"""Full auto-detection chain.
@@ -904,10 +991,7 @@ def _resolve_auto() -> Tuple[Optional[OpenAI], Optional[str]]:
# ── Step 2: aggregator / fallback chain ──────────────────────────────
tried = []
for try_fn in (_try_openrouter, _try_nous, _try_custom_endpoint,
_try_codex, _resolve_api_key_provider):
fn_name = getattr(try_fn, "__name__", "unknown")
label = _AUTO_PROVIDER_LABELS.get(fn_name, fn_name)
for label, try_fn in _get_provider_chain():
client, model = try_fn()
if client is not None:
if tried:
@@ -1035,7 +1119,7 @@ def resolve_provider_client(
client, default = _try_nous()
if client is None:
logger.warning("resolve_provider_client: nous requested "
"but Nous Portal not configured (run: hermes login)")
"but Nous Portal not configured (run: hermes auth)")
return None, None
final_model = model or default
return (_to_async_client(client, final_model) if async_mode
@@ -1785,12 +1869,15 @@ def call_llm(
f"was found. Set the {_explicit.upper()}_API_KEY environment "
f"variable, or switch to a different provider with `hermes model`."
)
# For auto/custom, fall back to OpenRouter
# For auto/custom with no credentials, try the full auto chain
# rather than hardcoding OpenRouter (which may be depleted).
# Pass model=None so each provider uses its own default —
# resolved_model may be an OpenRouter-format slug that doesn't
# work on other providers.
if not resolved_base_url:
logger.info("Auxiliary %s: provider %s unavailable, falling back to openrouter",
logger.info("Auxiliary %s: provider %s unavailable, trying auto-detection chain",
task or "call", resolved_provider)
client, final_model = _get_cached_client(
"openrouter", resolved_model or _OPENROUTER_MODEL)
client, final_model = _get_cached_client("auto")
if client is None:
raise RuntimeError(
f"No LLM provider configured for task={task} provider={resolved_provider}. "
@@ -1811,7 +1898,7 @@ def call_llm(
tools=tools, timeout=effective_timeout, extra_body=extra_body,
base_url=resolved_base_url)
# Handle max_tokens vs max_completion_tokens retry
# Handle max_tokens vs max_completion_tokens retry, then payment fallback.
try:
return client.chat.completions.create(**kwargs)
except Exception as first_err:
@@ -1819,7 +1906,30 @@ def call_llm(
if "max_tokens" in err_str or "unsupported_parameter" in err_str:
kwargs.pop("max_tokens", None)
kwargs["max_completion_tokens"] = max_tokens
return client.chat.completions.create(**kwargs)
try:
return client.chat.completions.create(**kwargs)
except Exception as retry_err:
# If the max_tokens retry also hits a payment error,
# fall through to the payment fallback below.
if not _is_payment_error(retry_err):
raise
first_err = retry_err
# ── Payment / credit exhaustion fallback ──────────────────────
# When the resolved provider returns 402 or a credit-related error,
# try alternative providers instead of giving up. This handles the
# common case where a user runs out of OpenRouter credits but has
# Codex OAuth or another provider available.
if _is_payment_error(first_err):
fb_client, fb_model, fb_label = _try_payment_fallback(
resolved_provider, task)
if fb_client is not None:
fb_kwargs = _build_call_kwargs(
fb_label, fb_model, messages,
temperature=temperature, max_tokens=max_tokens,
tools=tools, timeout=effective_timeout,
extra_body=extra_body)
return fb_client.chat.completions.create(**fb_kwargs)
raise
+24 -4
View File
@@ -14,6 +14,7 @@ Improvements over v1:
"""
import logging
import time
from typing import Any, Dict, List, Optional
from agent.auxiliary_client import call_llm
@@ -46,6 +47,7 @@ _PRUNED_TOOL_PLACEHOLDER = "[Old tool output cleared to save context space]"
# Chars per token rough estimate
_CHARS_PER_TOKEN = 4
_SUMMARY_FAILURE_COOLDOWN_SECONDS = 600
class ContextCompressor:
@@ -118,6 +120,7 @@ class ContextCompressor:
# Stores the previous compaction summary for iterative updates
self._previous_summary: Optional[str] = None
self._summary_failure_cooldown_until: float = 0.0
def update_from_response(self, usage: Dict[str, Any]):
"""Update tracked token usage from API response."""
@@ -258,6 +261,14 @@ class ContextCompressor:
the middle turns without a summary rather than inject a useless
placeholder.
"""
now = time.monotonic()
if now < self._summary_failure_cooldown_until:
logger.debug(
"Skipping context summary during cooldown (%.0fs remaining)",
self._summary_failure_cooldown_until - now,
)
return None
summary_budget = self._compute_summary_budget(turns_to_summarize)
content_to_summarize = self._serialize_for_summary(turns_to_summarize)
@@ -345,7 +356,6 @@ Write only the summary body. Do not include any preamble or prefix."""
call_kwargs = {
"task": "compression",
"messages": [{"role": "user", "content": prompt}],
"temperature": 0.3,
"max_tokens": summary_budget * 2,
# timeout resolved from auxiliary.compression.timeout config by call_llm
}
@@ -359,13 +369,23 @@ Write only the summary body. Do not include any preamble or prefix."""
summary = content.strip()
# Store for iterative updates on next compaction
self._previous_summary = summary
self._summary_failure_cooldown_until = 0.0
return self._with_summary_prefix(summary)
except RuntimeError:
self._summary_failure_cooldown_until = time.monotonic() + _SUMMARY_FAILURE_COOLDOWN_SECONDS
logging.warning("Context compression: no provider available for "
"summary. Middle turns will be dropped without summary.")
"summary. Middle turns will be dropped without summary "
"for %d seconds.",
_SUMMARY_FAILURE_COOLDOWN_SECONDS)
return None
except Exception as e:
logging.warning("Failed to generate context summary: %s", e)
self._summary_failure_cooldown_until = time.monotonic() + _SUMMARY_FAILURE_COOLDOWN_SECONDS
logging.warning(
"Failed to generate context summary: %s. "
"Further summary attempts paused for %d seconds.",
e,
_SUMMARY_FAILURE_COOLDOWN_SECONDS,
)
return None
@staticmethod
@@ -648,7 +668,7 @@ Write only the summary body. Do not include any preamble or prefix."""
compressed.append({"role": summary_role, "content": summary})
else:
if not self.quiet_mode:
logger.warning("No summary model available — middle turns dropped without summary")
logger.debug("No summary model available — middle turns dropped without summary")
for i in range(compress_end, n_messages):
msg = messages[i].copy()
+44
View File
@@ -23,6 +23,7 @@ from hermes_cli.auth import (
_agent_key_is_usable,
_codex_access_token_is_expiring,
_decode_jwt_claims,
_import_codex_cli_tokens,
_is_expiring,
_load_auth_store,
_load_provider_state,
@@ -440,6 +441,39 @@ class CredentialPool:
logger.debug("Failed to sync from credentials file: %s", exc)
return entry
def _sync_codex_entry_from_cli(self, entry: PooledCredential) -> PooledCredential:
"""Sync an openai-codex pool entry from ~/.codex/auth.json if tokens differ.
OpenAI OAuth refresh tokens are single-use and rotate on every refresh.
When the Codex CLI (or another Hermes profile) refreshes its token,
the pool entry's refresh_token becomes stale. This method detects that
by comparing against ~/.codex/auth.json and syncing the fresh pair.
"""
if self.provider != "openai-codex":
return entry
try:
cli_tokens = _import_codex_cli_tokens()
if not cli_tokens:
return entry
cli_refresh = cli_tokens.get("refresh_token", "")
cli_access = cli_tokens.get("access_token", "")
if cli_refresh and cli_refresh != entry.refresh_token:
logger.debug("Pool entry %s: syncing tokens from ~/.codex/auth.json (refresh token changed)", entry.id)
updated = replace(
entry,
access_token=cli_access,
refresh_token=cli_refresh,
last_status=None,
last_status_at=None,
last_error_code=None,
)
self._replace_entry(entry, updated)
self._persist()
return updated
except Exception as exc:
logger.debug("Failed to sync from ~/.codex/auth.json: %s", exc)
return entry
def _refresh_entry(self, entry: PooledCredential, *, force: bool) -> Optional[PooledCredential]:
if entry.auth_type != AUTH_TYPE_OAUTH or not entry.refresh_token:
if force:
@@ -629,6 +663,16 @@ class CredentialPool:
if synced is not entry:
entry = synced
cleared_any = True
# For openai-codex entries, sync from ~/.codex/auth.json before
# any status/refresh checks. This picks up tokens refreshed by
# the Codex CLI or another Hermes profile.
if (self.provider == "openai-codex"
and entry.last_status == STATUS_EXHAUSTED
and entry.refresh_token):
synced = self._sync_codex_entry_from_cli(entry)
if synced is not entry:
entry = synced
cleared_any = True
if entry.last_status == STATUS_EXHAUSTED:
exhausted_until = _exhausted_until(entry)
if exhausted_until is not None and now < exhausted_until:
+8 -2
View File
@@ -24,10 +24,11 @@ logger = logging.getLogger(__name__)
# are preserved so the full model name reaches cache lookups and server queries.
_PROVIDER_PREFIXES: frozenset[str] = frozenset({
"openrouter", "nous", "openai-codex", "copilot", "copilot-acp",
"zai", "kimi-coding", "minimax", "minimax-cn", "anthropic", "deepseek",
"gemini", "zai", "kimi-coding", "minimax", "minimax-cn", "anthropic", "deepseek",
"opencode-zen", "opencode-go", "ai-gateway", "kilocode", "alibaba",
"custom", "local",
# Common aliases
"google", "google-gemini", "google-ai-studio",
"glm", "z-ai", "z.ai", "zhipu", "github", "github-copilot",
"github-models", "kimi", "moonshot", "claude", "deep-seek",
"opencode", "zen", "go", "vercel", "kilo", "dashscope", "aliyun", "qwen",
@@ -101,6 +102,11 @@ DEFAULT_CONTEXT_LENGTHS = {
"gpt-4": 128000,
# Google
"gemini": 1048576,
# Gemma (open models served via AI Studio)
"gemma-4-31b": 256000,
"gemma-4-26b": 256000,
"gemma-3": 131072,
"gemma": 8192, # fallback for older gemma models
# DeepSeek
"deepseek": 128000,
# Meta
@@ -175,7 +181,7 @@ _URL_TO_PROVIDER: Dict[str, str] = {
"dashscope.aliyuncs.com": "alibaba",
"dashscope-intl.aliyuncs.com": "alibaba",
"openrouter.ai": "openrouter",
"generativelanguage.googleapis.com": "google",
"generativelanguage.googleapis.com": "gemini",
"inference-api.nousresearch.com": "nous",
"api.deepseek.com": "deepseek",
"api.githubcopilot.com": "copilot",
+34
View File
@@ -160,6 +160,7 @@ PROVIDER_TO_MODELS_DEV: Dict[str, str] = {
"kilocode": "kilo",
"fireworks": "fireworks-ai",
"huggingface": "huggingface",
"gemini": "google",
"google": "google",
"xai": "xai",
"nvidia": "nvidia",
@@ -422,6 +423,39 @@ def list_provider_models(provider: str) -> List[str]:
return list(models.keys())
# Patterns that indicate non-agentic or noise models (TTS, embedding,
# dated preview snapshots, live/streaming-only, image-only).
import re
_NOISE_PATTERNS: re.Pattern = re.compile(
r"-tts\b|embedding|live-|-(preview|exp)-\d{2,4}[-_]|"
r"-image\b|-image-preview\b|-customtools\b",
re.IGNORECASE,
)
def list_agentic_models(provider: str) -> List[str]:
"""Return model IDs suitable for agentic use from models.dev.
Filters for tool_call=True and excludes noise (TTS, embedding,
dated preview snapshots, live/streaming, image-only models).
Returns an empty list on any failure.
"""
models = _get_provider_models(provider)
if models is None:
return []
result = []
for mid, entry in models.items():
if not isinstance(entry, dict):
continue
if not entry.get("tool_call", False):
continue
if _NOISE_PATTERNS.search(mid):
continue
result.append(mid)
return result
def search_models_dev(
query: str, provider: str = None, limit: int = 5
) -> List[Dict[str, Any]]:
+1 -1
View File
@@ -187,7 +187,7 @@ TOOL_USE_ENFORCEMENT_GUIDANCE = (
# Model name substrings that trigger tool-use enforcement guidance.
# Add new patterns here when a model family needs explicit steering.
TOOL_USE_ENFORCEMENT_MODELS = ("gpt", "codex", "gemini", "gemma")
TOOL_USE_ENFORCEMENT_MODELS = ("gpt", "codex", "gemini", "gemma", "grok")
# OpenAI GPT/Codex-specific execution guidance. Addresses known failure modes
# where GPT models abandon work on partial results, skip prerequisite lookups,
+52
View File
@@ -16,6 +16,9 @@ logger = logging.getLogger(__name__)
_skill_commands: Dict[str, Dict[str, Any]] = {}
_PLAN_SLUG_RE = re.compile(r"[^a-z0-9]+")
# Patterns for sanitizing skill names into clean hyphen-separated slugs.
_SKILL_INVALID_CHARS = re.compile(r"[^a-z0-9-]")
_SKILL_MULTI_HYPHEN = re.compile(r"-{2,}")
def build_plan_path(
@@ -76,6 +79,45 @@ def _load_skill_payload(skill_identifier: str, task_id: str | None = None) -> tu
return loaded_skill, skill_dir, skill_name
def _inject_skill_config(loaded_skill: dict[str, Any], parts: list[str]) -> None:
"""Resolve and inject skill-declared config values into the message parts.
If the loaded skill's frontmatter declares ``metadata.hermes.config``
entries, their current values (from config.yaml or defaults) are appended
as a ``[Skill config: ...]`` block so the agent knows the configured values
without needing to read config.yaml itself.
"""
try:
from agent.skill_utils import (
extract_skill_config_vars,
parse_frontmatter,
resolve_skill_config_values,
)
# The loaded_skill dict contains the raw content which includes frontmatter
raw_content = str(loaded_skill.get("raw_content") or loaded_skill.get("content") or "")
if not raw_content:
return
frontmatter, _ = parse_frontmatter(raw_content)
config_vars = extract_skill_config_vars(frontmatter)
if not config_vars:
return
resolved = resolve_skill_config_values(config_vars)
if not resolved:
return
lines = ["", "[Skill config (from ~/.hermes/config.yaml):"]
for key, value in resolved.items():
display_val = str(value) if value else "(not set)"
lines.append(f" {key} = {display_val}")
lines.append("]")
parts.extend(lines)
except Exception:
pass # Non-critical — skill still loads without config injection
def _build_skill_message(
loaded_skill: dict[str, Any],
skill_dir: Path | None,
@@ -90,6 +132,9 @@ def _build_skill_message(
parts = [activation_note, "", content.strip()]
# ── Inject resolved skill config values ──
_inject_skill_config(loaded_skill, parts)
if loaded_skill.get("setup_skipped"):
parts.extend(
[
@@ -196,7 +241,14 @@ def scan_skill_commands() -> Dict[str, Dict[str, Any]]:
description = line[:80]
break
seen_names.add(name)
# Normalize to hyphen-separated slug, stripping
# non-alnum chars (e.g. +, /) to avoid invalid
# Telegram command names downstream.
cmd_name = name.lower().replace(' ', '-').replace('_', '-')
cmd_name = _SKILL_INVALID_CHARS.sub('', cmd_name)
cmd_name = _SKILL_MULTI_HYPHEN.sub('-', cmd_name).strip('-')
if not cmd_name:
continue
_skill_commands[f"/{cmd_name}"] = {
"name": name,
"description": description or f"Invoke the {name} skill",
+157
View File
@@ -254,6 +254,163 @@ def extract_skill_conditions(frontmatter: Dict[str, Any]) -> Dict[str, List]:
}
# ── Skill config extraction ───────────────────────────────────────────────
def extract_skill_config_vars(frontmatter: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Extract config variable declarations from parsed frontmatter.
Skills declare config.yaml settings they need via::
metadata:
hermes:
config:
- key: wiki.path
description: Path to the LLM Wiki knowledge base directory
default: "~/wiki"
prompt: Wiki directory path
Returns a list of dicts with keys: ``key``, ``description``, ``default``,
``prompt``. Invalid or incomplete entries are silently skipped.
"""
metadata = frontmatter.get("metadata")
if not isinstance(metadata, dict):
return []
hermes = metadata.get("hermes")
if not isinstance(hermes, dict):
return []
raw = hermes.get("config")
if not raw:
return []
if isinstance(raw, dict):
raw = [raw]
if not isinstance(raw, list):
return []
result: List[Dict[str, Any]] = []
seen: set = set()
for item in raw:
if not isinstance(item, dict):
continue
key = str(item.get("key", "")).strip()
if not key or key in seen:
continue
# Must have at least key and description
desc = str(item.get("description", "")).strip()
if not desc:
continue
entry: Dict[str, Any] = {
"key": key,
"description": desc,
}
default = item.get("default")
if default is not None:
entry["default"] = default
prompt_text = item.get("prompt")
if isinstance(prompt_text, str) and prompt_text.strip():
entry["prompt"] = prompt_text.strip()
else:
entry["prompt"] = desc
seen.add(key)
result.append(entry)
return result
def discover_all_skill_config_vars() -> List[Dict[str, Any]]:
"""Scan all enabled skills and collect their config variable declarations.
Walks every skills directory, parses each SKILL.md frontmatter, and returns
a deduplicated list of config var dicts. Each dict also includes a
``skill`` key with the skill name for attribution.
Disabled and platform-incompatible skills are excluded.
"""
all_vars: List[Dict[str, Any]] = []
seen_keys: set = set()
disabled = get_disabled_skill_names()
for skills_dir in get_all_skills_dirs():
if not skills_dir.is_dir():
continue
for skill_file in iter_skill_index_files(skills_dir, "SKILL.md"):
try:
raw = skill_file.read_text(encoding="utf-8")
frontmatter, _ = parse_frontmatter(raw)
except Exception:
continue
skill_name = frontmatter.get("name") or skill_file.parent.name
if str(skill_name) in disabled:
continue
if not skill_matches_platform(frontmatter):
continue
config_vars = extract_skill_config_vars(frontmatter)
for var in config_vars:
if var["key"] not in seen_keys:
var["skill"] = str(skill_name)
all_vars.append(var)
seen_keys.add(var["key"])
return all_vars
# Storage prefix: all skill config vars are stored under skills.config.*
# in config.yaml. Skill authors declare logical keys (e.g. "wiki.path");
# the system adds this prefix for storage and strips it for display.
SKILL_CONFIG_PREFIX = "skills.config"
def _resolve_dotpath(config: Dict[str, Any], dotted_key: str):
"""Walk a nested dict following a dotted key. Returns None if any part is missing."""
parts = dotted_key.split(".")
current = config
for part in parts:
if isinstance(current, dict) and part in current:
current = current[part]
else:
return None
return current
def resolve_skill_config_values(
config_vars: List[Dict[str, Any]],
) -> Dict[str, Any]:
"""Resolve current values for skill config vars from config.yaml.
Skill config is stored under ``skills.config.<key>`` in config.yaml.
Returns a dict mapping **logical** keys (as declared by skills) to their
current values (or the declared default if the key isn't set).
Path values are expanded via ``os.path.expanduser``.
"""
config_path = get_hermes_home() / "config.yaml"
config: Dict[str, Any] = {}
if config_path.exists():
try:
parsed = yaml_load(config_path.read_text(encoding="utf-8"))
if isinstance(parsed, dict):
config = parsed
except Exception:
pass
resolved: Dict[str, Any] = {}
for var in config_vars:
logical_key = var["key"]
storage_key = f"{SKILL_CONFIG_PREFIX}.{logical_key}"
value = _resolve_dotpath(config, storage_key)
if value is None or (isinstance(value, str) and not value.strip()):
value = var.get("default", "")
# Expand ~ in path-like values
if isinstance(value, str) and ("~" in value or "${" in value):
value = os.path.expanduser(os.path.expandvars(value))
resolved[logical_key] = value
return resolved
# ── Description extraction ────────────────────────────────────────────────
+4 -2
View File
@@ -18,7 +18,8 @@ model:
# "anthropic" - Direct Anthropic API (requires: ANTHROPIC_API_KEY)
# "openai-codex" - OpenAI Codex (requires: hermes login --provider openai-codex)
# "copilot" - GitHub Copilot / GitHub Models (requires: GITHUB_TOKEN)
# "zai" - z.ai / ZhipuAI GLM (requires: GLM_API_KEY)
# "gemini" - Use Google AI Studio direct (requires: GOOGLE_API_KEY or GEMINI_API_KEY)
# "zai" - Use z.ai / ZhipuAI GLM models (requires: GLM_API_KEY)
# "kimi-coding" - Kimi / Moonshot AI (requires: KIMI_API_KEY)
# "minimax" - MiniMax global (requires: MINIMAX_API_KEY)
# "minimax-cn" - MiniMax China (requires: MINIMAX_CN_API_KEY)
@@ -315,7 +316,8 @@ compression:
# "auto" - Best available: OpenRouter → Nous Portal → main endpoint (default)
# "openrouter" - Force OpenRouter (requires OPENROUTER_API_KEY)
# "nous" - Force Nous Portal (requires: hermes login)
# "codex" - Force Codex OAuth (requires: hermes model → Codex).
# "gemini" - Force Google AI Studio direct (requires: GOOGLE_API_KEY or GEMINI_API_KEY)
# "codex" - Force Codex OAuth (requires: hermes model → Codex).
# Uses gpt-5.3-codex which supports vision.
# "main" - Use your custom endpoint (OPENAI_BASE_URL + OPENAI_API_KEY).
# Works with OpenAI API, local models, or any OpenAI-compatible
+72 -25
View File
@@ -120,6 +120,63 @@ def _parse_reasoning_config(effort: str) -> dict | None:
return result
def _get_chrome_debug_candidates(system: str) -> list[str]:
"""Return likely browser executables for local CDP auto-launch."""
candidates: list[str] = []
seen: set[str] = set()
def _add_candidate(path: str | None) -> None:
if not path:
return
normalized = os.path.normcase(os.path.normpath(path))
if normalized in seen:
return
if os.path.isfile(path):
candidates.append(path)
seen.add(normalized)
def _add_from_path(*names: str) -> None:
for name in names:
_add_candidate(shutil.which(name))
if system == "Darwin":
for app in (
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser",
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",
):
_add_candidate(app)
elif system == "Windows":
_add_from_path(
"chrome.exe", "msedge.exe", "brave.exe", "chromium.exe",
"chrome", "msedge", "brave", "chromium",
)
for base in (
os.environ.get("ProgramFiles"),
os.environ.get("ProgramFiles(x86)"),
os.environ.get("LOCALAPPDATA"),
):
if not base:
continue
for parts in (
("Google", "Chrome", "Application", "chrome.exe"),
("Chromium", "Application", "chrome.exe"),
("Chromium", "Application", "chromium.exe"),
("BraveSoftware", "Brave-Browser", "Application", "brave.exe"),
("Microsoft", "Edge", "Application", "msedge.exe"),
):
_add_candidate(os.path.join(base, *parts))
else:
_add_from_path(
"google-chrome", "google-chrome-stable", "chromium-browser",
"chromium", "brave-browser", "microsoft-edge",
)
return candidates
def load_cli_config() -> Dict[str, Any]:
"""
Load CLI configuration from config files.
@@ -3721,7 +3778,7 @@ class HermesCLI:
# Persistence
if persist_global:
save_config_value("model.name", result.new_model)
save_config_value("model.default", result.new_model)
if result.provider_changed:
save_config_value("model.provider", result.target_provider)
_cprint(" Saved to config.yaml (--global)")
@@ -4838,27 +4895,9 @@ class HermesCLI:
Returns True if a launch command was executed (doesn't guarantee success).
"""
import shutil
import subprocess as _sp
candidates = []
if system == "Darwin":
# macOS: try common app bundle locations
for app in (
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser",
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",
):
if os.path.isfile(app):
candidates.append(app)
else:
# Linux: try common binary names
for name in ("google-chrome", "google-chrome-stable", "chromium-browser",
"chromium", "brave-browser", "microsoft-edge"):
path = shutil.which(name)
if path:
candidates.append(path)
candidates = _get_chrome_debug_candidates(system)
if not candidates:
return False
@@ -7469,18 +7508,26 @@ class HermesCLI:
# wrapping of long lines so the input area always fits its content.
def _input_height():
try:
from prompt_toolkit.application import get_app
from prompt_toolkit.utils import get_cwidth
doc = input_area.buffer.document
prompt_width = max(2, len(self._get_tui_prompt_text()))
available_width = shutil.get_terminal_size().columns - prompt_width
prompt_width = max(2, get_cwidth(self._get_tui_prompt_text()))
try:
available_width = get_app().output.get_size().columns - prompt_width
except Exception:
available_width = shutil.get_terminal_size((80, 24)).columns - prompt_width
if available_width < 10:
available_width = 40
visual_lines = 0
for line in doc.lines:
# Each logical line takes at least 1 visual row; long lines wrap
if len(line) == 0:
# Each logical line takes at least 1 visual row; long lines wrap.
# Use prompt_toolkit's cell width so CJK wide characters count as 2.
line_width = get_cwidth(line)
if line_width <= 0:
visual_lines += 1
else:
visual_lines += max(1, -(-len(line) // available_width)) # ceil division
visual_lines += max(1, -(-line_width // available_width)) # ceil division
return min(max(visual_lines, 1), 8)
except Exception:
return 1
+41 -21
View File
@@ -237,6 +237,10 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> None:
else:
delivery_content = content
# Extract MEDIA: tags so attachments are forwarded as files, not raw text
from gateway.platforms.base import BasePlatformAdapter
media_files, cleaned_delivery_content = BasePlatformAdapter.extract_media(delivery_content)
# Prefer the live adapter when the gateway is running — this supports E2EE
# rooms (e.g. Matrix) where the standalone HTTP path cannot encrypt.
runtime_adapter = (adapters or {}).get(platform)
@@ -264,7 +268,7 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> None:
)
# Standalone path: run the async send in a fresh event loop (safe from any thread)
coro = _send_to_platform(platform, pconfig, chat_id, delivery_content, thread_id=thread_id)
coro = _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files)
try:
result = asyncio.run(coro)
except RuntimeError:
@@ -275,7 +279,7 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> None:
coro.close()
import concurrent.futures
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
future = pool.submit(asyncio.run, _send_to_platform(platform, pconfig, chat_id, delivery_content, thread_id=thread_id))
future = pool.submit(asyncio.run, _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files))
result = future.result(timeout=30)
except Exception as e:
logger.error("Job '%s': delivery to %s:%s failed: %s", job["id"], platform_name, chat_id, e)
@@ -293,8 +297,15 @@ _SCRIPT_TIMEOUT = 120 # seconds
def _run_job_script(script_path: str) -> tuple[bool, str]:
"""Execute a cron job's data-collection script and capture its output.
Scripts must reside within HERMES_HOME/scripts/. Both relative and
absolute paths are resolved and validated against this directory to
prevent arbitrary script execution via path traversal or absolute
path injection.
Args:
script_path: Path to a Python script (resolved via HERMES_HOME/scripts/ or absolute).
script_path: Path to a Python script. Relative paths are resolved
against HERMES_HOME/scripts/. Absolute and ~-prefixed paths
are also validated to ensure they stay within the scripts dir.
Returns:
(success, output) — on failure *output* contains the error message so the
@@ -302,16 +313,25 @@ def _run_job_script(script_path: str) -> tuple[bool, str]:
"""
from hermes_constants import get_hermes_home
path = Path(script_path).expanduser()
if not path.is_absolute():
# Resolve relative paths against HERMES_HOME/scripts/
scripts_dir = get_hermes_home() / "scripts"
path = (scripts_dir / path).resolve()
# Guard against path traversal (e.g. "../../etc/passwd")
try:
path.relative_to(scripts_dir.resolve())
except ValueError:
return False, f"Script path escapes the scripts directory: {script_path!r}"
scripts_dir = get_hermes_home() / "scripts"
scripts_dir.mkdir(parents=True, exist_ok=True)
scripts_dir_resolved = scripts_dir.resolve()
raw = Path(script_path).expanduser()
if raw.is_absolute():
path = raw.resolve()
else:
path = (scripts_dir / raw).resolve()
# Guard against path traversal, absolute path injection, and symlink
# escape — scripts MUST reside within HERMES_HOME/scripts/.
try:
path.relative_to(scripts_dir_resolved)
except ValueError:
return False, (
f"Blocked: script path resolves outside the scripts directory "
f"({scripts_dir_resolved}): {script_path!r}"
)
if not path.exists():
return False, f"Script not found: {path}"
@@ -469,14 +489,14 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
logger.info("Running job '%s' (ID: %s)", job_name, job_id)
logger.info("Prompt: %s", prompt[:100])
# Inject origin context so the agent's send_message tool knows the chat
if origin:
os.environ["HERMES_SESSION_PLATFORM"] = origin["platform"]
os.environ["HERMES_SESSION_CHAT_ID"] = str(origin["chat_id"])
if origin.get("chat_name"):
os.environ["HERMES_SESSION_CHAT_NAME"] = origin["chat_name"]
try:
# Inject origin context so the agent's send_message tool knows the chat.
# Must be INSIDE the try block so the finally cleanup always runs.
if origin:
os.environ["HERMES_SESSION_PLATFORM"] = origin["platform"]
os.environ["HERMES_SESSION_CHAT_ID"] = str(origin["chat_id"])
if origin.get("chat_name"):
os.environ["HERMES_SESSION_CHAT_NAME"] = origin["chat_name"]
# Re-read .env and config.yaml fresh every run so provider/key
# changes take effect without a gateway restart.
from dotenv import load_dotenv
@@ -797,7 +817,7 @@ def tick(verbose: bool = True, adapters=None, loop=None) -> int:
# output is already saved above). Failed jobs always deliver.
deliver_content = final_response if success else f"⚠️ Cron job '{job.get('name', job['id'])}' failed:\n{error}"
should_deliver = bool(deliver_content)
if should_deliver and success and deliver_content.strip().upper().startswith(SILENT_MARKER):
if should_deliver and success and SILENT_MARKER in deliver_content.strip().upper():
logger.info("Job '%s': agent returned %s — skipping delivery", job["id"], SILENT_MARKER)
should_deliver = False
+27 -1
View File
@@ -164,6 +164,11 @@ class HermesAgentLoop:
self.max_tokens = max_tokens
self.extra_body = extra_body
# Per-result and per-turn output persistence (see tools/tool_result_storage.py)
from pathlib import Path
self._tool_result_storage_dir = Path(f"/tmp/hermes_tool_results/{self.task_id}")
self._tool_result_storage_dir.mkdir(parents=True, exist_ok=True)
async def run(self, messages: List[Dict[str, Any]]) -> AgentResult:
"""
Execute the full agent loop using standard OpenAI tool calling.
@@ -446,8 +451,18 @@ class HermesAgentLoop:
except (json.JSONDecodeError, TypeError):
pass
# Add tool response to conversation
# Persist oversized results to disk
tc_id = tc.get("id", "") if isinstance(tc, dict) else tc.id
try:
from tools.tool_result_storage import maybe_persist_tool_result
tool_result = maybe_persist_tool_result(
content=tool_result,
tool_name=tool_name,
tool_use_id=tc_id,
storage_dir=self._tool_result_storage_dir,
)
except Exception:
pass # Persistence is best-effort in eval path
messages.append(
{
"role": "tool",
@@ -456,6 +471,17 @@ class HermesAgentLoop:
}
)
# Per-turn aggregate budget enforcement
try:
from tools.tool_result_storage import enforce_turn_budget
num_tcs = len(assistant_msg.tool_calls)
if num_tcs > 0:
turn_msgs = [m for m in messages[-num_tcs * 2:]
if m.get("role") == "tool"]
enforce_turn_budget(turn_msgs, self._tool_result_storage_dir)
except Exception:
pass # Best-effort in eval path
turn_elapsed = _time.monotonic() - turn_start
logger.info(
"[%s] turn %d: api=%.1fs, %d tools, turn_total=%.1fs",
+2 -3
View File
@@ -12,6 +12,7 @@ from datetime import datetime
from typing import Any, Dict, List, Optional
from hermes_cli.config import get_hermes_home
from utils import atomic_json_write
logger = logging.getLogger(__name__)
@@ -86,9 +87,7 @@ def build_channel_directory(adapters: Dict[Any, Any]) -> Dict[str, Any]:
}
try:
DIRECTORY_PATH.parent.mkdir(parents=True, exist_ok=True)
with open(DIRECTORY_PATH, "w", encoding="utf-8") as f:
json.dump(directory, f, indent=2, ensure_ascii=False)
atomic_json_write(DIRECTORY_PATH, directory)
except Exception as e:
logger.warning("Channel directory: failed to write: %s", e)
+3
View File
@@ -779,6 +779,9 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
config.platforms[Platform.MATRIX].extra["password"] = matrix_password
matrix_e2ee = os.getenv("MATRIX_ENCRYPTION", "").lower() in ("true", "1", "yes")
config.platforms[Platform.MATRIX].extra["encryption"] = matrix_e2ee
matrix_device_id = os.getenv("MATRIX_DEVICE_ID", "")
if matrix_device_id:
config.platforms[Platform.MATRIX].extra["device_id"] = matrix_device_id
matrix_home = os.getenv("MATRIX_HOME_ROOM")
if matrix_home and Platform.MATRIX in config.platforms:
config.platforms[Platform.MATRIX].home_channel = HomeChannel(
+79 -54
View File
@@ -21,6 +21,8 @@ Storage: ~/.hermes/pairing/
import json
import os
import secrets
import tempfile
import threading
import time
from pathlib import Path
from typing import Optional
@@ -45,13 +47,29 @@ PAIRING_DIR = get_hermes_dir("platforms/pairing", "pairing")
def _secure_write(path: Path, data: str) -> None:
"""Write data to file with restrictive permissions (owner read/write only)."""
"""Write data to file with restrictive permissions (owner read/write only).
Uses a temp-file + atomic rename so readers always see either the old
complete file or the new one — never a partial write.
"""
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(data, encoding="utf-8")
fd, tmp_path = tempfile.mkstemp(dir=str(path.parent), suffix=".tmp")
try:
os.chmod(path, 0o600)
except OSError:
pass # Windows doesn't support chmod the same way
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(data)
f.flush()
os.fsync(f.fileno())
os.replace(tmp_path, str(path))
try:
os.chmod(path, 0o600)
except OSError:
pass # Windows doesn't support chmod the same way
except BaseException:
try:
os.unlink(tmp_path)
except OSError:
pass
raise
class PairingStore:
@@ -66,6 +84,9 @@ class PairingStore:
def __init__(self):
PAIRING_DIR.mkdir(parents=True, exist_ok=True)
# Protects all read-modify-write cycles. The gateway runs multiple
# platform adapters concurrently in threads sharing one PairingStore.
self._lock = threading.RLock()
def _pending_path(self, platform: str) -> Path:
return PAIRING_DIR / f"{platform}-pending.json"
@@ -105,7 +126,7 @@ class PairingStore:
return results
def _approve_user(self, platform: str, user_id: str, user_name: str = "") -> None:
"""Add a user to the approved list."""
"""Add a user to the approved list. Must be called under self._lock."""
approved = self._load_json(self._approved_path(platform))
approved[user_id] = {
"user_name": user_name,
@@ -116,11 +137,12 @@ class PairingStore:
def revoke(self, platform: str, user_id: str) -> bool:
"""Remove a user from the approved list. Returns True if found."""
path = self._approved_path(platform)
approved = self._load_json(path)
if user_id in approved:
del approved[user_id]
self._save_json(path, approved)
return True
with self._lock:
approved = self._load_json(path)
if user_id in approved:
del approved[user_id]
self._save_json(path, approved)
return True
return False
# ----- Pending codes -----
@@ -136,36 +158,37 @@ class PairingStore:
- Max pending codes reached for this platform
- User/platform is in lockout due to failed attempts
"""
self._cleanup_expired(platform)
with self._lock:
self._cleanup_expired(platform)
# Check lockout
if self._is_locked_out(platform):
return None
# Check lockout
if self._is_locked_out(platform):
return None
# Check rate limit for this specific user
if self._is_rate_limited(platform, user_id):
return None
# Check rate limit for this specific user
if self._is_rate_limited(platform, user_id):
return None
# Check max pending
pending = self._load_json(self._pending_path(platform))
if len(pending) >= MAX_PENDING_PER_PLATFORM:
return None
# Check max pending
pending = self._load_json(self._pending_path(platform))
if len(pending) >= MAX_PENDING_PER_PLATFORM:
return None
# Generate cryptographically random code
code = "".join(secrets.choice(ALPHABET) for _ in range(CODE_LENGTH))
# Generate cryptographically random code
code = "".join(secrets.choice(ALPHABET) for _ in range(CODE_LENGTH))
# Store pending request
pending[code] = {
"user_id": user_id,
"user_name": user_name,
"created_at": time.time(),
}
self._save_json(self._pending_path(platform), pending)
# Store pending request
pending[code] = {
"user_id": user_id,
"user_name": user_name,
"created_at": time.time(),
}
self._save_json(self._pending_path(platform), pending)
# Record rate limit
self._record_rate_limit(platform, user_id)
# Record rate limit
self._record_rate_limit(platform, user_id)
return code
return code
def approve_code(self, platform: str, code: str) -> Optional[dict]:
"""
@@ -173,24 +196,25 @@ class PairingStore:
Returns {user_id, user_name} on success, None if code is invalid/expired.
"""
self._cleanup_expired(platform)
code = code.upper().strip()
with self._lock:
self._cleanup_expired(platform)
code = code.upper().strip()
pending = self._load_json(self._pending_path(platform))
if code not in pending:
self._record_failed_attempt(platform)
return None
pending = self._load_json(self._pending_path(platform))
if code not in pending:
self._record_failed_attempt(platform)
return None
entry = pending.pop(code)
self._save_json(self._pending_path(platform), pending)
entry = pending.pop(code)
self._save_json(self._pending_path(platform), pending)
# Add to approved list
self._approve_user(platform, entry["user_id"], entry.get("user_name", ""))
# Add to approved list
self._approve_user(platform, entry["user_id"], entry.get("user_name", ""))
return {
"user_id": entry["user_id"],
"user_name": entry.get("user_name", ""),
}
return {
"user_id": entry["user_id"],
"user_name": entry.get("user_name", ""),
}
def list_pending(self, platform: str = None) -> list:
"""List pending pairing requests, optionally filtered by platform."""
@@ -212,12 +236,13 @@ class PairingStore:
def clear_pending(self, platform: str = None) -> int:
"""Clear all pending requests. Returns count removed."""
count = 0
platforms = [platform] if platform else self._all_platforms("pending")
for p in platforms:
pending = self._load_json(self._pending_path(p))
count += len(pending)
self._save_json(self._pending_path(p), {})
with self._lock:
count = 0
platforms = [platform] if platform else self._all_platforms("pending")
for p in platforms:
pending = self._load_json(self._pending_path(p))
count += len(pending)
self._save_json(self._pending_path(p), {})
return count
# ----- Rate limiting and lockout -----
+60 -5
View File
@@ -12,6 +12,7 @@ import random
import re
import uuid
from abc import ABC, abstractmethod
from urllib.parse import urlsplit
logger = logging.getLogger(__name__)
from dataclasses import dataclass, field
@@ -36,6 +37,43 @@ GATEWAY_SECRET_CAPTURE_UNSUPPORTED_MESSAGE = (
)
def _safe_url_for_log(url: str, max_len: int = 80) -> str:
"""Return a URL string safe for logs (no query/fragment/userinfo)."""
if max_len <= 0:
return ""
if url is None:
return ""
raw = str(url)
if not raw:
return ""
try:
parsed = urlsplit(raw)
except Exception:
return raw[:max_len]
if parsed.scheme and parsed.netloc:
# Strip potential embedded credentials (user:pass@host).
netloc = parsed.netloc.rsplit("@", 1)[-1]
base = f"{parsed.scheme}://{netloc}"
path = parsed.path or ""
if path and path != "/":
basename = path.rsplit("/", 1)[-1]
safe = f"{base}/.../{basename}" if basename else f"{base}/..."
else:
safe = base
else:
safe = raw
if len(safe) <= max_len:
return safe
if max_len <= 3:
return "." * max_len
return f"{safe[:max_len - 3]}..."
# ---------------------------------------------------------------------------
# Image cache utilities
#
@@ -112,8 +150,14 @@ async def cache_image_from_url(url: str, ext: str = ".jpg", retries: int = 2) ->
raise
if attempt < retries:
wait = 1.5 * (attempt + 1)
_log.debug("Media cache retry %d/%d for %s (%.1fs): %s",
attempt + 1, retries, url[:80], wait, exc)
_log.debug(
"Media cache retry %d/%d for %s (%.1fs): %s",
attempt + 1,
retries,
_safe_url_for_log(url),
wait,
exc,
)
await asyncio.sleep(wait)
continue
raise
@@ -214,8 +258,14 @@ async def cache_audio_from_url(url: str, ext: str = ".ogg", retries: int = 2) ->
raise
if attempt < retries:
wait = 1.5 * (attempt + 1)
_log.debug("Audio cache retry %d/%d for %s (%.1fs): %s",
attempt + 1, retries, url[:80], wait, exc)
_log.debug(
"Audio cache retry %d/%d for %s (%.1fs): %s",
attempt + 1,
retries,
_safe_url_for_log(url),
wait,
exc,
)
await asyncio.sleep(wait)
continue
raise
@@ -1266,7 +1316,12 @@ class BasePlatformAdapter(ABC):
if human_delay > 0:
await asyncio.sleep(human_delay)
try:
logger.info("[%s] Sending image: %s (alt=%s)", self.name, image_url[:80], alt_text[:30] if alt_text else "")
logger.info(
"[%s] Sending image: %s (alt=%s)",
self.name,
_safe_url_for_log(image_url),
alt_text[:30] if alt_text else "",
)
# Route animated GIFs through send_animation for proper playback
if self._is_animation_url(image_url):
img_result = await self.send_animation(
+41
View File
@@ -1695,6 +1695,47 @@ class DiscordAdapter(BasePlatformAdapter):
async def slash_btw(interaction: discord.Interaction, question: str):
await self._run_simple_slash(interaction, f"/btw {question}")
# Register installed skills as native slash commands (parity with
# Telegram, which uses telegram_menu_commands() in commands.py).
# Discord allows up to 100 application commands globally.
_DISCORD_CMD_LIMIT = 100
try:
from hermes_cli.commands import discord_skill_commands
existing_names = {cmd.name for cmd in tree.get_commands()}
remaining_slots = max(0, _DISCORD_CMD_LIMIT - len(existing_names))
skill_entries, skipped = discord_skill_commands(
max_slots=remaining_slots,
reserved_names=existing_names,
)
for discord_name, description, cmd_key in skill_entries:
# Closure factory to capture cmd_key per iteration
def _make_skill_handler(_key: str):
async def _skill_slash(interaction: discord.Interaction, args: str = ""):
await self._run_simple_slash(interaction, f"{_key} {args}".strip())
return _skill_slash
handler = _make_skill_handler(cmd_key)
handler.__name__ = f"skill_{discord_name.replace('-', '_')}"
cmd = discord.app_commands.Command(
name=discord_name,
description=description,
callback=handler,
)
discord.app_commands.describe(args="Optional arguments for the skill")(cmd)
tree.add_command(cmd)
if skipped:
logger.warning(
"[%s] Discord slash command limit reached (%d): %d skill(s) not registered",
self.name, _DISCORD_CMD_LIMIT, skipped,
)
except Exception as exc:
logger.warning("[%s] Failed to register skill slash commands: %s", self.name, exc)
def _build_slash_event(self, interaction: discord.Interaction, text: str) -> MessageEvent:
"""Build a MessageEvent from a Discord slash command interaction."""
is_dm = isinstance(interaction.channel, discord.DMChannel)
+211 -23
View File
@@ -270,6 +270,22 @@ class FeishuAdapterSettings:
webhook_host: str
webhook_port: int
webhook_path: str
ws_reconnect_nonce: int = 30
ws_reconnect_interval: int = 120
ws_ping_interval: Optional[int] = None
ws_ping_timeout: Optional[int] = None
admins: frozenset[str] = frozenset()
default_group_policy: str = ""
group_rules: Dict[str, FeishuGroupRule] = field(default_factory=dict)
@dataclass
class FeishuGroupRule:
"""Per-group policy rule for controlling which users may interact with the bot."""
policy: str # "open" | "allowlist" | "blacklist" | "admin_only" | "disabled"
allowlist: set[str] = field(default_factory=set)
blacklist: set[str] = field(default_factory=set)
@dataclass
@@ -358,6 +374,24 @@ def _strip_markdown_to_plain_text(text: str) -> str:
return plain.strip()
def _coerce_int(value: Any, default: Optional[int] = None, min_value: int = 0) -> Optional[int]:
"""Coerce value to int with optional default and minimum constraint."""
try:
parsed = int(value)
except (TypeError, ValueError):
return default
return parsed if parsed >= min_value else default
def _coerce_required_int(value: Any, default: int, min_value: int = 0) -> int:
parsed = _coerce_int(value, default=default, min_value=min_value)
return default if parsed is None else parsed
def _is_loop_ready(loop: Optional[asyncio.AbstractEventLoop]) -> bool:
return loop is not None and not bool(getattr(loop, "is_closed", lambda: False)())
# ---------------------------------------------------------------------------
# Post payload builders and parsers
# ---------------------------------------------------------------------------
@@ -913,14 +947,66 @@ def _unique_lines(lines: List[str]) -> List[str]:
return unique
def _run_official_feishu_ws_client(ws_client: Any) -> None:
def _run_official_feishu_ws_client(ws_client: Any, adapter: Any) -> None:
"""Run the official Lark WS client in its own thread-local event loop."""
import lark_oapi.ws.client as ws_client_module
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
ws_client_module.loop = loop
ws_client.start()
adapter._ws_thread_loop = loop
original_connect = ws_client_module.websockets.connect
original_configure = getattr(ws_client, "_configure", None)
def _apply_runtime_ws_overrides() -> None:
try:
setattr(ws_client, "_reconnect_nonce", adapter._ws_reconnect_nonce)
setattr(ws_client, "_reconnect_interval", adapter._ws_reconnect_interval)
if adapter._ws_ping_interval is not None:
setattr(ws_client, "_ping_interval", adapter._ws_ping_interval)
except Exception:
logger.debug("[Feishu] Failed to apply websocket runtime overrides", exc_info=True)
async def _connect_with_overrides(*args: Any, **kwargs: Any) -> Any:
if adapter._ws_ping_interval is not None and "ping_interval" not in kwargs:
kwargs["ping_interval"] = adapter._ws_ping_interval
if adapter._ws_ping_timeout is not None and "ping_timeout" not in kwargs:
kwargs["ping_timeout"] = adapter._ws_ping_timeout
return await original_connect(*args, **kwargs)
def _configure_with_overrides(conf: Any) -> Any:
assert original_configure is not None
result = original_configure(conf)
_apply_runtime_ws_overrides()
return result
ws_client_module.websockets.connect = _connect_with_overrides
if original_configure is not None:
setattr(ws_client, "_configure", _configure_with_overrides)
_apply_runtime_ws_overrides()
try:
ws_client.start()
except Exception:
pass
finally:
ws_client_module.websockets.connect = original_connect
if original_configure is not None:
setattr(ws_client, "_configure", original_configure)
pending = [t for t in asyncio.all_tasks(loop) if not t.done()]
for task in pending:
task.cancel()
if pending:
loop.run_until_complete(asyncio.gather(*pending, return_exceptions=True))
try:
loop.stop()
except Exception:
pass
try:
loop.close()
except Exception:
pass
adapter._ws_thread_loop = None
def check_feishu_requirements() -> bool:
@@ -945,10 +1031,11 @@ class FeishuAdapter(BasePlatformAdapter):
self._client: Optional[Any] = None
self._ws_client: Optional[Any] = None
self._ws_future: Optional[asyncio.Future] = None
self._ws_thread_loop: Optional[asyncio.AbstractEventLoop] = None
self._loop: Optional[asyncio.AbstractEventLoop] = None
self._webhook_runner: Optional[Any] = None
self._webhook_site: Optional[Any] = None
self._event_handler = self._build_event_handler()
self._event_handler: Optional[Any] = None
self._seen_message_ids: Dict[str, float] = {} # message_id → seen_at (time.time())
self._seen_message_order: List[str] = []
self._dedup_state_path = get_hermes_home() / "feishu_seen_message_ids.json"
@@ -974,6 +1061,26 @@ class FeishuAdapter(BasePlatformAdapter):
@staticmethod
def _load_settings(extra: Dict[str, Any]) -> FeishuAdapterSettings:
# Parse per-group rules from config
raw_group_rules = extra.get("group_rules", {})
group_rules: Dict[str, FeishuGroupRule] = {}
if isinstance(raw_group_rules, dict):
for chat_id, rule_cfg in raw_group_rules.items():
if not isinstance(rule_cfg, dict):
continue
group_rules[str(chat_id)] = FeishuGroupRule(
policy=str(rule_cfg.get("policy", "open")).strip().lower(),
allowlist=set(str(u).strip() for u in rule_cfg.get("allowlist", []) if str(u).strip()),
blacklist=set(str(u).strip() for u in rule_cfg.get("blacklist", []) if str(u).strip()),
)
# Bot-level admins
raw_admins = extra.get("admins", [])
admins = frozenset(str(u).strip() for u in raw_admins if str(u).strip())
# Default group policy (for groups not in group_rules)
default_group_policy = str(extra.get("default_group_policy", "")).strip().lower()
return FeishuAdapterSettings(
app_id=str(extra.get("app_id") or os.getenv("FEISHU_APP_ID", "")).strip(),
app_secret=str(extra.get("app_secret") or os.getenv("FEISHU_APP_SECRET", "")).strip(),
@@ -1020,6 +1127,13 @@ class FeishuAdapter(BasePlatformAdapter):
str(extra.get("webhook_path") or os.getenv("FEISHU_WEBHOOK_PATH", _DEFAULT_WEBHOOK_PATH)).strip()
or _DEFAULT_WEBHOOK_PATH
),
ws_reconnect_nonce=_coerce_required_int(extra.get("ws_reconnect_nonce"), default=30, min_value=0),
ws_reconnect_interval=_coerce_required_int(extra.get("ws_reconnect_interval"), default=120, min_value=1),
ws_ping_interval=_coerce_int(extra.get("ws_ping_interval"), default=None, min_value=1),
ws_ping_timeout=_coerce_int(extra.get("ws_ping_timeout"), default=None, min_value=1),
admins=admins,
default_group_policy=default_group_policy,
group_rules=group_rules,
)
def _apply_settings(self, settings: FeishuAdapterSettings) -> None:
@@ -1031,6 +1145,9 @@ class FeishuAdapter(BasePlatformAdapter):
self._verification_token = settings.verification_token
self._group_policy = settings.group_policy
self._allowed_group_users = set(settings.allowed_group_users)
self._admins = set(settings.admins)
self._default_group_policy = settings.default_group_policy or settings.group_policy
self._group_rules = settings.group_rules
self._bot_open_id = settings.bot_open_id
self._bot_user_id = settings.bot_user_id
self._bot_name = settings.bot_name
@@ -1042,6 +1159,10 @@ class FeishuAdapter(BasePlatformAdapter):
self._webhook_host = settings.webhook_host
self._webhook_port = settings.webhook_port
self._webhook_path = settings.webhook_path
self._ws_reconnect_nonce = settings.ws_reconnect_nonce
self._ws_reconnect_interval = settings.ws_reconnect_interval
self._ws_ping_interval = settings.ws_ping_interval
self._ws_ping_timeout = settings.ws_ping_timeout
def _build_event_handler(self) -> Any:
if EventDispatcherHandler is None:
@@ -1116,8 +1237,37 @@ class FeishuAdapter(BasePlatformAdapter):
self._reset_batch_buffers()
self._disable_websocket_auto_reconnect()
await self._stop_webhook_server()
ws_thread_loop = self._ws_thread_loop
if ws_thread_loop is not None and not ws_thread_loop.is_closed():
logger.debug("[Feishu] Cancelling websocket thread tasks and stopping loop")
def cancel_all_tasks() -> None:
tasks = [t for t in asyncio.all_tasks(ws_thread_loop) if not t.done()]
logger.debug("[Feishu] Found %d pending tasks in websocket thread", len(tasks))
for task in tasks:
task.cancel()
ws_thread_loop.call_later(0.1, ws_thread_loop.stop)
ws_thread_loop.call_soon_threadsafe(cancel_all_tasks)
ws_future = self._ws_future
if ws_future is not None:
try:
logger.debug("[Feishu] Waiting for websocket thread to exit (timeout=10s)")
await asyncio.wait_for(asyncio.shield(ws_future), timeout=10.0)
logger.debug("[Feishu] Websocket thread exited cleanly")
except asyncio.TimeoutError:
logger.warning("[Feishu] Websocket thread did not exit within 10s - may be stuck")
except asyncio.CancelledError:
logger.debug("[Feishu] Websocket thread cancelled during disconnect")
except Exception as exc:
logger.debug("[Feishu] Websocket thread exited with error: %s", exc, exc_info=True)
self._ws_future = None
self._ws_thread_loop = None
self._loop = None
self._event_handler = None
self._persist_seen_message_ids()
await self._release_app_lock()
@@ -1476,12 +1626,13 @@ class FeishuAdapter(BasePlatformAdapter):
def _on_message_event(self, data: Any) -> None:
"""Normalize Feishu inbound events into MessageEvent."""
if self._loop is None:
loop = self._loop
if loop is None or bool(getattr(loop, "is_closed", lambda: False)()):
logger.warning("[Feishu] Dropping inbound message before adapter loop is ready")
return
future = asyncio.run_coroutine_threadsafe(
self._handle_message_event_data(data),
self._loop,
loop,
)
future.add_done_callback(self._log_background_failure)
@@ -1504,7 +1655,8 @@ class FeishuAdapter(BasePlatformAdapter):
return
chat_type = getattr(message, "chat_type", "p2p")
if chat_type != "p2p" and not self._should_accept_group_message(message, sender_id):
chat_id = getattr(message, "chat_id", "") or ""
if chat_type != "p2p" and not self._should_accept_group_message(message, sender_id, chat_id):
logger.debug("[Feishu] Dropping group message that failed mention/policy gate: %s", message_id)
return
await self._process_inbound_message(
@@ -1553,27 +1705,30 @@ class FeishuAdapter(BasePlatformAdapter):
)
# Only process reactions from real users. Ignore app/bot-generated reactions
# and Hermes' own ACK emoji to avoid feedback loops.
loop = self._loop
if (
operator_type in {"bot", "app"}
or emoji_type == _FEISHU_ACK_EMOJI
or not message_id
or self._loop is None
or loop is None
or bool(getattr(loop, "is_closed", lambda: False)())
):
return
future = asyncio.run_coroutine_threadsafe(
self._handle_reaction_event(event_type, data),
self._loop,
loop,
)
future.add_done_callback(self._log_background_failure)
def _on_card_action_trigger(self, data: Any) -> Any:
"""Schedule Feishu card actions on the adapter loop and acknowledge immediately."""
if self._loop is None:
loop = self._loop
if loop is None or bool(getattr(loop, "is_closed", lambda: False)()):
logger.warning("[Feishu] Dropping card action before adapter loop is ready")
else:
future = asyncio.run_coroutine_threadsafe(
self._handle_card_action_event(data),
self._loop,
loop,
)
future.add_done_callback(self._log_background_failure)
if P2CardActionTriggerResponse is None:
@@ -2083,7 +2238,7 @@ class FeishuAdapter(BasePlatformAdapter):
event_type = str((payload.get("header") or {}).get("event_type") or "")
data = self._namespace_from_mapping(payload)
if event_type == "im.message.receive_v1":
await self._handle_message_event_data(data)
self._on_message_event(data)
elif event_type == "im.message.message_read_v1":
self._on_message_read_event(data)
elif event_type == "im.chat.member.bot.added_v1":
@@ -2093,7 +2248,7 @@ class FeishuAdapter(BasePlatformAdapter):
elif event_type in ("im.message.reaction.created_v1", "im.message.reaction.deleted_v1"):
self._on_reaction_event(event_type, data)
elif event_type == "card.action.trigger":
asyncio.ensure_future(self._handle_card_action_event(data))
self._on_card_action_trigger(data)
else:
logger.debug("[Feishu] Ignoring webhook event type: %s", event_type or "unknown")
return web.json_response({"code": 0, "msg": "ok"})
@@ -2657,18 +2812,41 @@ class FeishuAdapter(BasePlatformAdapter):
# Group policy and mention gating
# =========================================================================
def _allow_group_message(self, sender_id: Any) -> bool:
"""Current group policy gate for non-DM traffic."""
if self._group_policy == "disabled":
return False
sender_open_id = getattr(sender_id, "open_id", None) or getattr(sender_id, "user_id", None)
if self._group_policy == "open":
return True
return bool(sender_open_id and sender_open_id in self._allowed_group_users)
def _allow_group_message(self, sender_id: Any, chat_id: str = "") -> bool:
"""Per-group policy gate for non-DM traffic."""
sender_open_id = getattr(sender_id, "open_id", None)
sender_user_id = getattr(sender_id, "user_id", None)
sender_ids = {sender_open_id, sender_user_id} - {None}
def _should_accept_group_message(self, message: Any, sender_id: Any) -> bool:
if sender_ids and self._admins and (sender_ids & self._admins):
return True
rule = self._group_rules.get(chat_id) if chat_id else None
if rule:
policy = rule.policy
allowlist = rule.allowlist
blacklist = rule.blacklist
else:
policy = self._default_group_policy or self._group_policy
allowlist = self._allowed_group_users
blacklist = set()
if policy == "disabled":
return False
if policy == "open":
return True
if policy == "admin_only":
return False
if policy == "allowlist":
return bool(sender_ids and (sender_ids & allowlist))
if policy == "blacklist":
return bool(sender_ids and not (sender_ids & blacklist))
return bool(sender_ids and (sender_ids & self._allowed_group_users))
def _should_accept_group_message(self, message: Any, sender_id: Any, chat_id: str = "") -> bool:
"""Require an explicit @mention before group messages enter the agent."""
if not self._allow_group_message(sender_id):
if not self._allow_group_message(sender_id, chat_id):
return False
# @_all is Feishu's @everyone placeholder — always route to the bot.
raw_content = getattr(message, "content", "") or ""
@@ -2965,6 +3143,12 @@ class FeishuAdapter(BasePlatformAdapter):
raise RuntimeError("websockets not installed; websocket mode unavailable")
domain = FEISHU_DOMAIN if self._domain_name != "lark" else LARK_DOMAIN
self._client = self._build_lark_client(domain)
self._event_handler = self._build_event_handler()
if self._event_handler is None:
raise RuntimeError("failed to build Feishu event handler")
loop = self._loop
if loop is None or loop.is_closed():
raise RuntimeError("adapter loop is not ready")
await self._hydrate_bot_identity()
self._ws_client = FeishuWSClient(
app_id=self._app_id,
@@ -2973,10 +3157,11 @@ class FeishuAdapter(BasePlatformAdapter):
event_handler=self._event_handler,
domain=domain,
)
self._ws_future = self._loop.run_in_executor(
self._ws_future = loop.run_in_executor(
None,
_run_official_feishu_ws_client,
self._ws_client,
self,
)
async def _connect_webhook(self) -> None:
@@ -2984,6 +3169,9 @@ class FeishuAdapter(BasePlatformAdapter):
raise RuntimeError("aiohttp not installed; webhook mode unavailable")
domain = FEISHU_DOMAIN if self._domain_name != "lark" else LARK_DOMAIN
self._client = self._build_lark_client(domain)
self._event_handler = self._build_event_handler()
if self._event_handler is None:
raise RuntimeError("failed to build Feishu event handler")
await self._hydrate_bot_identity()
app = web.Application()
app.router.add_post(self._webhook_path, self._handle_webhook_request)
+81 -19
View File
@@ -10,6 +10,7 @@ Environment variables:
MATRIX_USER_ID Full user ID (@bot:server) required for password login
MATRIX_PASSWORD Password (alternative to access token)
MATRIX_ENCRYPTION Set "true" to enable E2EE
MATRIX_DEVICE_ID Stable device ID for E2EE persistence across restarts
MATRIX_ALLOWED_USERS Comma-separated Matrix user IDs (@user:server)
MATRIX_HOME_ROOM Room ID for cron/notification delivery
MATRIX_REACTIONS Set "false" to disable processing lifecycle reactions
@@ -65,6 +66,21 @@ _MAX_PENDING_EVENTS = 100
_PENDING_EVENT_TTL = 300 # seconds — stop retrying after 5 min
_E2EE_INSTALL_HINT = (
"Install with: pip install 'matrix-nio[e2e]' "
"(requires libolm C library)"
)
def _check_e2ee_deps() -> bool:
"""Return True if matrix-nio E2EE dependencies (python-olm) are available."""
try:
from nio.crypto import ENCRYPTION_ENABLED
return bool(ENCRYPTION_ENABLED)
except (ImportError, AttributeError):
return False
def check_matrix_requirements() -> bool:
"""Return True if the Matrix adapter can be used."""
token = os.getenv("MATRIX_ACCESS_TOKEN", "")
@@ -79,7 +95,6 @@ def check_matrix_requirements() -> bool:
return False
try:
import nio # noqa: F401
return True
except ImportError:
logger.warning(
"Matrix: matrix-nio not installed. "
@@ -87,6 +102,20 @@ def check_matrix_requirements() -> bool:
)
return False
# If encryption is requested, verify E2EE deps are available at startup
# rather than silently degrading to plaintext-only at connect time.
encryption_requested = os.getenv("MATRIX_ENCRYPTION", "").lower() in ("true", "1", "yes")
if encryption_requested and not _check_e2ee_deps():
logger.error(
"Matrix: MATRIX_ENCRYPTION=true but E2EE dependencies are missing. %s. "
"Without this, encrypted rooms will not work. "
"Set MATRIX_ENCRYPTION=false to disable E2EE.",
_E2EE_INSTALL_HINT,
)
return False
return True
class MatrixAdapter(BasePlatformAdapter):
"""Gateway adapter for Matrix (any homeserver)."""
@@ -111,6 +140,10 @@ class MatrixAdapter(BasePlatformAdapter):
"encryption",
os.getenv("MATRIX_ENCRYPTION", "").lower() in ("true", "1", "yes"),
)
self._device_id: str = (
config.extra.get("device_id", "")
or os.getenv("MATRIX_DEVICE_ID", "")
)
self._client: Any = None # nio.AsyncClient
self._sync_task: Optional[asyncio.Task] = None
@@ -169,24 +202,42 @@ class MatrixAdapter(BasePlatformAdapter):
_STORE_DIR.mkdir(parents=True, exist_ok=True)
# Create the client.
# When a stable device_id is configured, pass it to the constructor
# so matrix-nio binds to it from the start (important for E2EE
# crypto-store persistence across restarts).
ctor_device_id = self._device_id or None
if self._encryption:
if not _check_e2ee_deps():
logger.error(
"Matrix: MATRIX_ENCRYPTION=true but E2EE dependencies are missing. %s. "
"Refusing to connect — encrypted rooms would silently fail.",
_E2EE_INSTALL_HINT,
)
return False
try:
client = nio.AsyncClient(
self._homeserver,
self._user_id or "",
device_id=ctor_device_id,
store_path=store_path,
)
logger.info("Matrix: E2EE enabled (store: %s)", store_path)
except Exception as exc:
logger.warning(
"Matrix: failed to create E2EE client (%s), "
"falling back to plain client. Install: "
"pip install 'matrix-nio[e2e]'",
exc,
logger.info(
"Matrix: E2EE enabled (store: %s%s)",
store_path,
f", device_id={self._device_id}" if self._device_id else "",
)
client = nio.AsyncClient(self._homeserver, self._user_id or "")
except Exception as exc:
logger.error(
"Matrix: failed to create E2EE client: %s. %s",
exc, _E2EE_INSTALL_HINT,
)
return False
else:
client = nio.AsyncClient(self._homeserver, self._user_id or "")
client = nio.AsyncClient(
self._homeserver,
self._user_id or "",
device_id=ctor_device_id,
)
self._client = client
@@ -205,30 +256,36 @@ class MatrixAdapter(BasePlatformAdapter):
if resolved_user_id:
self._user_id = resolved_user_id
# Prefer the user-configured device_id (MATRIX_DEVICE_ID) so
# the bot reuses a stable identity across restarts. Fall back
# to whatever whoami returned.
effective_device_id = self._device_id or resolved_device_id
# restore_login() is the matrix-nio path that binds the access
# token to a specific device and loads the crypto store.
if resolved_device_id and hasattr(client, "restore_login"):
if effective_device_id and hasattr(client, "restore_login"):
client.restore_login(
self._user_id or resolved_user_id,
resolved_device_id,
effective_device_id,
self._access_token,
)
else:
if self._user_id:
client.user_id = self._user_id
if resolved_device_id:
client.device_id = resolved_device_id
if effective_device_id:
client.device_id = effective_device_id
client.access_token = self._access_token
if self._encryption:
logger.warning(
"Matrix: access-token login did not restore E2EE state; "
"encrypted rooms may fail until a device_id is available"
"encrypted rooms may fail until a device_id is available. "
"Set MATRIX_DEVICE_ID to a stable value."
)
logger.info(
"Matrix: using access token for %s%s",
self._user_id or "(unknown user)",
f" (device {resolved_device_id})" if resolved_device_id else "",
f" (device {effective_device_id})" if effective_device_id else "",
)
else:
logger.error(
@@ -271,10 +328,15 @@ class MatrixAdapter(BasePlatformAdapter):
except Exception as exc:
logger.debug("Matrix: could not import keys: %s", exc)
elif self._encryption:
logger.warning(
"Matrix: E2EE requested but crypto store is not loaded; "
"encrypted rooms may fail"
# E2EE was requested but the crypto store failed to load —
# this means encrypted rooms will silently not work. Hard-fail.
logger.error(
"Matrix: E2EE requested but crypto store is not loaded — "
"cannot decrypt or encrypt messages. %s",
_E2EE_INSTALL_HINT,
)
await client.close()
return False
# Register event callbacks.
client.add_event_callback(self._on_room_message, nio.RoomMessageText)
+9
View File
@@ -701,6 +701,15 @@ class MattermostAdapter(BasePlatformAdapter):
except Exception as exc:
logger.warning("Mattermost: error downloading file %s: %s", fid, exc)
# Set message type based on downloaded media types.
if media_types and msg_type == MessageType.TEXT:
if any(m.startswith("image/") for m in media_types):
msg_type = MessageType.PHOTO
elif any(m.startswith("audio/") for m in media_types):
msg_type = MessageType.VOICE
elif media_types:
msg_type = MessageType.DOCUMENT
source = self.build_source(
chat_id=channel_id,
chat_type=chat_type,
+67 -7
View File
@@ -717,19 +717,27 @@ class SignalAdapter(BasePlatformAdapter):
return SendResult(success=True)
return SendResult(success=False, error="RPC send with attachment failed")
async def send_document(
async def _send_attachment(
self,
chat_id: str,
file_path: str,
media_label: str,
caption: Optional[str] = None,
filename: Optional[str] = None,
**kwargs,
) -> SendResult:
"""Send a document/file attachment."""
"""Send any file as a Signal attachment via RPC.
Shared implementation for send_document, send_image_file, send_voice,
and send_video avoids duplicating the validation/routing/RPC logic.
"""
await self._stop_typing_indicator(chat_id)
if not Path(file_path).exists():
return SendResult(success=False, error="File not found")
try:
file_size = Path(file_path).stat().st_size
except FileNotFoundError:
return SendResult(success=False, error=f"{media_label} file not found: {file_path}")
if file_size > SIGNAL_MAX_ATTACHMENT_SIZE:
return SendResult(success=False, error=f"{media_label} too large ({file_size} bytes)")
params: Dict[str, Any] = {
"account": self.account,
@@ -746,7 +754,59 @@ class SignalAdapter(BasePlatformAdapter):
if result is not None:
self._track_sent_timestamp(result)
return SendResult(success=True)
return SendResult(success=False, error="RPC send document failed")
return SendResult(success=False, error=f"RPC send {media_label.lower()} failed")
async def send_document(
self,
chat_id: str,
file_path: str,
caption: Optional[str] = None,
filename: Optional[str] = None,
**kwargs,
) -> SendResult:
"""Send a document/file attachment."""
return await self._send_attachment(chat_id, file_path, "File", caption)
async def send_image_file(
self,
chat_id: str,
image_path: str,
caption: Optional[str] = None,
reply_to: Optional[str] = None,
**kwargs,
) -> SendResult:
"""Send a local image file as a native Signal attachment.
Called by the gateway media delivery flow when MEDIA: tags containing
image paths are extracted from agent responses.
"""
return await self._send_attachment(chat_id, image_path, "Image", caption)
async def send_voice(
self,
chat_id: str,
audio_path: str,
caption: Optional[str] = None,
reply_to: Optional[str] = None,
**kwargs,
) -> SendResult:
"""Send an audio file as a Signal attachment.
Signal does not distinguish voice messages from file attachments at
the API level, so this routes through the same RPC send path.
"""
return await self._send_attachment(chat_id, audio_path, "Audio", caption)
async def send_video(
self,
chat_id: str,
video_path: str,
caption: Optional[str] = None,
reply_to: Optional[str] = None,
**kwargs,
) -> SendResult:
"""Send a video file as a Signal attachment."""
return await self._send_attachment(chat_id, video_path, "Video", caption)
# ------------------------------------------------------------------
# Typing Indicators
+1 -1
View File
@@ -518,7 +518,7 @@ class TelegramAdapter(BasePlatformAdapter):
", ".join(fallback_ips),
)
if fallback_ips:
logger.warning(
logger.info(
"[%s] Telegram fallback IPs active: %s",
self.name,
", ".join(fallback_ips),
+14 -1
View File
@@ -484,6 +484,10 @@ class WebhookAdapter(BasePlatformAdapter):
Supports dot-notation access into nested dicts:
``{pull_request.title}`` ``payload["pull_request"]["title"]``
Special token ``{__raw__}`` dumps the entire payload as indented
JSON (truncated to 4000 chars). Useful for monitoring alerts or
any webhook where the agent needs to see the full payload.
"""
if not template:
truncated = json.dumps(payload, indent=2)[:4000]
@@ -494,6 +498,9 @@ class WebhookAdapter(BasePlatformAdapter):
def _resolve(match: re.Match) -> str:
key = match.group(1)
# Special token: dump the entire payload as JSON
if key == "__raw__":
return json.dumps(payload, indent=2)[:4000]
value: Any = payload
for part in key.split("."):
if isinstance(value, dict):
@@ -613,4 +620,10 @@ class WebhookAdapter(BasePlatformAdapter):
error=f"No chat_id or home channel for {platform_name}",
)
return await adapter.send(chat_id, content)
# Pass thread_id from deliver_extra so Telegram forum topics work
metadata = None
thread_id = extra.get("message_thread_id") or extra.get("thread_id")
if thread_id:
metadata = {"thread_id": thread_id}
return await adapter.send(chat_id, content, metadata=metadata)
+20 -4
View File
@@ -3244,7 +3244,7 @@ class GatewayRunner:
old_entry = self.session_store._entries.get(session_key)
if old_entry:
_flush_task = asyncio.create_task(
self._async_flush_memories(old_entry.session_id, session_key)
self._async_flush_memories(old_entry.session_id)
)
self._background_tasks.add(_flush_task)
_flush_task.add_done_callback(self._background_tasks.discard)
@@ -3252,9 +3252,25 @@ class GatewayRunner:
logger.debug("Gateway memory flush on reset failed: %s", e)
self._evict_cached_agent(session_key)
try:
from tools.env_passthrough import clear_env_passthrough
clear_env_passthrough()
except Exception:
pass
try:
from tools.credential_files import clear_credential_files
clear_credential_files()
except Exception:
pass
# Reset the session
new_entry = self.session_store.reset_session(session_key)
# Clear any session-scoped model override so the next agent picks up
# the configured default instead of the previously switched model.
self._session_model_overrides.pop(session_key, None)
# Emit session:end hook (session is ending)
await self.hooks.emit("session:end", {
"platform": source.platform.value if source.platform else "",
@@ -3481,7 +3497,7 @@ class GatewayRunner:
cfg = yaml.safe_load(f) or {}
model_cfg = cfg.get("model", {})
if isinstance(model_cfg, dict):
current_model = model_cfg.get("name", "")
current_model = model_cfg.get("default", "")
current_provider = model_cfg.get("provider", current_provider)
current_base_url = model_cfg.get("base_url", "")
user_provs = cfg.get("providers")
@@ -3591,7 +3607,7 @@ class GatewayRunner:
else:
cfg = {}
model_cfg = cfg.setdefault("model", {})
model_cfg["name"] = result.new_model
model_cfg["default"] = result.new_model
model_cfg["provider"] = result.target_provider
if result.base_url:
model_cfg["base_url"] = result.base_url
@@ -4974,7 +4990,7 @@ class GatewayRunner:
# Flush memories for current session before switching
try:
_flush_task = asyncio.create_task(
self._async_flush_memories(current_entry.session_id, session_key)
self._async_flush_memories(current_entry.session_id)
)
self._background_tasks.add(_flush_task)
_flush_task.add_done_callback(self._background_tasks.discard)
+113 -34
View File
@@ -69,6 +69,7 @@ DEVICE_AUTH_POLL_INTERVAL_CAP_SECONDS = 1 # poll at most every 1s
DEFAULT_CODEX_BASE_URL = "https://chatgpt.com/backend-api/codex"
DEFAULT_GITHUB_MODELS_BASE_URL = "https://api.githubcopilot.com"
DEFAULT_COPILOT_ACP_BASE_URL = "acp://copilot"
DEFAULT_GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta/openai"
CODEX_OAUTH_CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann"
CODEX_OAUTH_TOKEN_URL = "https://auth.openai.com/oauth/token"
CODEX_ACCESS_TOKEN_REFRESH_SKEW_SECONDS = 120
@@ -125,6 +126,14 @@ PROVIDER_REGISTRY: Dict[str, ProviderConfig] = {
inference_base_url=DEFAULT_COPILOT_ACP_BASE_URL,
base_url_env_var="COPILOT_ACP_BASE_URL",
),
"gemini": ProviderConfig(
id="gemini",
name="Google AI Studio",
auth_type="api_key",
inference_base_url="https://generativelanguage.googleapis.com/v1beta/openai",
api_key_env_vars=("GOOGLE_API_KEY", "GEMINI_API_KEY"),
base_url_env_var="GEMINI_BASE_URL",
),
"zai": ProviderConfig(
id="zai",
name="Z.AI / GLM",
@@ -758,6 +767,7 @@ def resolve_provider(
# Normalize provider aliases
_PROVIDER_ALIASES = {
"glm": "zai", "z-ai": "zai", "z.ai": "zai", "zhipu": "zai",
"google": "gemini", "google-gemini": "gemini", "google-ai-studio": "gemini",
"kimi": "kimi-coding", "moonshot": "kimi-coding",
"minimax-china": "minimax-cn", "minimax_cn": "minimax-cn",
"claude": "anthropic", "claude-code": "anthropic",
@@ -926,7 +936,7 @@ def _read_codex_tokens(*, _lock: bool = True) -> Dict[str, Any]:
state = _load_provider_state(auth_store, "openai-codex")
if not state:
raise AuthError(
"No Codex credentials stored. Run `hermes login` to authenticate.",
"No Codex credentials stored. Run `hermes auth` to authenticate.",
provider="openai-codex",
code="codex_auth_missing",
relogin_required=True,
@@ -934,7 +944,7 @@ def _read_codex_tokens(*, _lock: bool = True) -> Dict[str, Any]:
tokens = state.get("tokens")
if not isinstance(tokens, dict):
raise AuthError(
"Codex auth state is missing tokens. Run `hermes login` to re-authenticate.",
"Codex auth state is missing tokens. Run `hermes auth` to re-authenticate.",
provider="openai-codex",
code="codex_auth_invalid_shape",
relogin_required=True,
@@ -943,14 +953,14 @@ def _read_codex_tokens(*, _lock: bool = True) -> Dict[str, Any]:
refresh_token = tokens.get("refresh_token")
if not isinstance(access_token, str) or not access_token.strip():
raise AuthError(
"Codex auth is missing access_token. Run `hermes login` to re-authenticate.",
"Codex auth is missing access_token. Run `hermes auth` to re-authenticate.",
provider="openai-codex",
code="codex_auth_missing_access_token",
relogin_required=True,
)
if not isinstance(refresh_token, str) or not refresh_token.strip():
raise AuthError(
"Codex auth is missing refresh_token. Run `hermes login` to re-authenticate.",
"Codex auth is missing refresh_token. Run `hermes auth` to re-authenticate.",
provider="openai-codex",
code="codex_auth_missing_refresh_token",
relogin_required=True,
@@ -985,7 +995,7 @@ def refresh_codex_oauth_pure(
del access_token # Access token is only used by callers to decide whether to refresh.
if not isinstance(refresh_token, str) or not refresh_token.strip():
raise AuthError(
"Codex auth is missing refresh_token. Run `hermes login` to re-authenticate.",
"Codex auth is missing refresh_token. Run `hermes auth` to re-authenticate.",
provider="openai-codex",
code="codex_auth_missing_refresh_token",
relogin_required=True,
@@ -1020,6 +1030,14 @@ def refresh_codex_oauth_pure(
pass
if code in {"invalid_grant", "invalid_token", "invalid_request"}:
relogin_required = True
if code == "refresh_token_reused":
message = (
"Codex refresh token was already consumed by another client "
"(e.g. Codex CLI or VS Code extension). "
"Run `codex` in your terminal to generate fresh tokens, "
"then run `hermes auth` to re-authenticate."
)
relogin_required = True
raise AuthError(
message,
provider="openai-codex",
@@ -1081,7 +1099,8 @@ def _refresh_codex_auth_tokens(
def _import_codex_cli_tokens() -> Optional[Dict[str, str]]:
"""Try to read tokens from ~/.codex/auth.json (Codex CLI shared file).
Returns tokens dict if valid, None otherwise. Does NOT write to the shared file.
Returns tokens dict if valid and not expired, None otherwise.
Does NOT write to the shared file.
"""
codex_home = os.getenv("CODEX_HOME", "").strip()
if not codex_home:
@@ -1094,7 +1113,17 @@ def _import_codex_cli_tokens() -> Optional[Dict[str, str]]:
tokens = payload.get("tokens")
if not isinstance(tokens, dict):
return None
if not tokens.get("access_token") or not tokens.get("refresh_token"):
access_token = tokens.get("access_token")
refresh_token = tokens.get("refresh_token")
if not access_token or not refresh_token:
return None
# Reject expired tokens — importing stale tokens from ~/.codex/
# that can't be refreshed leaves the user stuck with "Login successful!"
# but no working credentials.
if _codex_access_token_is_expiring(access_token, 0):
logger.debug(
"Codex CLI tokens at %s are expired — skipping import.", auth_path,
)
return None
return dict(tokens)
except Exception:
@@ -1122,7 +1151,7 @@ def resolve_codex_runtime_credentials(
logger.info("Migrating Codex credentials from ~/.codex/ to Hermes auth store")
print("⚠️ Migrating Codex credentials to Hermes's own auth store.")
print(" This avoids conflicts with Codex CLI and VS Code.")
print(" Run `hermes login` to create a fully independent session.\n")
print(" Run `hermes auth` to create a fully independent session.\n")
_save_codex_tokens(cli_tokens)
data = _read_codex_tokens()
else:
@@ -1886,7 +1915,36 @@ def get_nous_auth_status() -> Dict[str, Any]:
def get_codex_auth_status() -> Dict[str, Any]:
"""Status snapshot for Codex auth."""
"""Status snapshot for Codex auth.
Checks the credential pool first (where `hermes auth` stores credentials),
then falls back to the legacy provider state.
"""
# Check credential pool first — this is where `hermes auth` and
# `hermes model` store device_code tokens.
try:
from agent.credential_pool import load_pool
pool = load_pool("openai-codex")
if pool and pool.has_credentials():
entry = pool.select()
if entry is not None:
api_key = (
getattr(entry, "runtime_api_key", None)
or getattr(entry, "access_token", "")
)
if api_key and not _codex_access_token_is_expiring(api_key, 0):
return {
"logged_in": True,
"auth_store": str(_auth_file_path()),
"last_refresh": getattr(entry, "last_refresh", None),
"auth_mode": "chatgpt",
"source": f"pool:{getattr(entry, 'label', 'unknown')}",
"api_key": api_key,
}
except Exception:
pass
# Fall back to legacy provider state
try:
creds = resolve_codex_runtime_credentials()
return {
@@ -1895,6 +1953,7 @@ def get_codex_auth_status() -> Dict[str, Any]:
"last_refresh": creds.get("last_refresh"),
"auth_mode": creds.get("auth_mode"),
"source": creds.get("source"),
"api_key": creds.get("api_key"),
}
except AuthError as exc:
return {
@@ -2078,7 +2137,7 @@ def detect_external_credentials() -> List[Dict[str, Any]]:
found.append({
"provider": "openai-codex",
"path": str(codex_path),
"label": f"Codex CLI credentials found ({codex_path}) — run `hermes login` to create a separate session",
"label": f"Codex CLI credentials found ({codex_path}) — run `hermes auth` to create a separate session",
})
return found
@@ -2327,8 +2386,8 @@ def _save_model_choice(model_id: str) -> None:
def login_command(args) -> None:
"""Deprecated: use 'hermes model' or 'hermes setup' instead."""
print("The 'hermes login' command has been removed.")
print("Use 'hermes model' to select a provider and model,")
print("or 'hermes setup' for full interactive setup.")
print("Use 'hermes auth' to manage credentials,")
print("'hermes model' to select a provider, or 'hermes setup' for full setup.")
raise SystemExit(0)
@@ -2338,17 +2397,25 @@ def _login_openai_codex(args, pconfig: ProviderConfig) -> None:
# Check for existing Hermes-owned credentials
try:
existing = resolve_codex_runtime_credentials()
print("Existing Codex credentials found in Hermes auth store.")
try:
reuse = input("Use existing credentials? [Y/n]: ").strip().lower()
except (EOFError, KeyboardInterrupt):
reuse = "y"
if reuse in ("", "y", "yes"):
config_path = _update_config_for_provider("openai-codex", existing.get("base_url", DEFAULT_CODEX_BASE_URL))
print()
print("Login successful!")
print(f" Config updated: {config_path} (model.provider=openai-codex)")
return
# Verify the resolved token is actually usable (not expired).
# resolve_codex_runtime_credentials attempts refresh, so if we get
# here the token should be valid — but double-check before telling
# the user "Login successful!".
_resolved_key = existing.get("api_key", "")
if isinstance(_resolved_key, str) and _resolved_key and not _codex_access_token_is_expiring(_resolved_key, 60):
print("Existing Codex credentials found in Hermes auth store.")
try:
reuse = input("Use existing credentials? [Y/n]: ").strip().lower()
except (EOFError, KeyboardInterrupt):
reuse = "y"
if reuse in ("", "y", "yes"):
config_path = _update_config_for_provider("openai-codex", existing.get("base_url", DEFAULT_CODEX_BASE_URL))
print()
print("Login successful!")
print(f" Config updated: {config_path} (model.provider=openai-codex)")
return
else:
print("Existing Codex credentials are expired. Starting fresh login...")
except AuthError:
pass
@@ -2643,13 +2710,26 @@ def _nous_device_code_login(
"agent_key_reused": None,
"agent_key_obtained_at": None,
}
return refresh_nous_oauth_from_state(
auth_state,
min_key_ttl_seconds=min_key_ttl_seconds,
timeout_seconds=timeout_seconds,
force_refresh=False,
force_mint=True,
)
try:
return refresh_nous_oauth_from_state(
auth_state,
min_key_ttl_seconds=min_key_ttl_seconds,
timeout_seconds=timeout_seconds,
force_refresh=False,
force_mint=True,
)
except AuthError as exc:
if exc.code == "subscription_required":
portal_url = auth_state.get(
"portal_base_url", DEFAULT_NOUS_PORTAL_URL
).rstrip("/")
print()
print("Your Nous Portal account does not have an active subscription.")
print(f" Subscribe here: {portal_url}/billing")
print()
print("After subscribing, run `hermes model` again to finish setup.")
raise SystemExit(1)
raise
def _login_nous(args, pconfig: ProviderConfig) -> None:
@@ -2666,14 +2746,15 @@ def _login_nous(args, pconfig: ProviderConfig) -> None:
auth_state = _nous_device_code_login(
portal_base_url=getattr(args, "portal_url", None),
inference_base_url=getattr(args, "inference_url", None),
client_id=getattr(args, "client_id", None),
scope=getattr(args, "scope", None),
client_id=getattr(args, "client_id", None) or pconfig.client_id,
scope=getattr(args, "scope", None) or pconfig.scope,
open_browser=not getattr(args, "no_browser", False),
timeout_seconds=timeout_seconds,
insecure=insecure,
ca_bundle=ca_bundle,
min_key_ttl_seconds=5 * 60,
)
inference_base_url = auth_state["inference_base_url"]
verify: bool | str = False if insecure else (ca_bundle if ca_bundle else True)
@@ -2697,8 +2778,6 @@ def _login_nous(args, pconfig: ProviderConfig) -> None:
code="invalid_token",
)
# Use curated model list (same as OpenRouter defaults) instead
# of the full /models dump which returns hundreds of models.
from hermes_cli.models import _PROVIDER_MODELS
model_ids = _PROVIDER_MODELS.get("nous", [])
+26
View File
@@ -305,6 +305,32 @@ def auth_remove_command(args) -> None:
if cleared:
print(f"Cleared {env_var} from .env")
# If this was a singleton-seeded credential (OAuth device_code, hermes_pkce),
# clear the underlying auth store / credential file so it doesn't get
# re-seeded on the next load_pool() call.
elif removed.source == "device_code" and provider in ("openai-codex", "nous"):
from hermes_cli.auth import (
_load_auth_store, _save_auth_store, _auth_store_lock,
)
with _auth_store_lock():
auth_store = _load_auth_store()
providers_dict = auth_store.get("providers")
if isinstance(providers_dict, dict) and provider in providers_dict:
del providers_dict[provider]
_save_auth_store(auth_store)
print(f"Cleared {provider} OAuth tokens from auth store")
elif removed.source == "hermes_pkce" and provider == "anthropic":
from hermes_constants import get_hermes_home
oauth_file = get_hermes_home() / ".anthropic_oauth.json"
if oauth_file.exists():
oauth_file.unlink()
print("Cleared Hermes Anthropic OAuth credentials")
elif removed.source == "claude_code" and provider == "anthropic":
print("Note: Claude Code credentials live in ~/.claude/.credentials.json")
print(" Remove them manually if you want to deauthorize Claude Code.")
def auth_reset_command(args) -> None:
provider = _normalize_provider(getattr(args, "provider", ""))
+196 -76
View File
@@ -366,21 +366,46 @@ def telegram_bot_commands() -> list[tuple[str, str]]:
for cmd in COMMAND_REGISTRY:
if not _is_gateway_available(cmd, overrides):
continue
tg_name = cmd.name.replace("-", "_")
result.append((tg_name, cmd.description))
tg_name = _sanitize_telegram_name(cmd.name)
if tg_name:
result.append((tg_name, cmd.description))
return result
_TG_NAME_LIMIT = 32
_CMD_NAME_LIMIT = 32
"""Max command name length shared by Telegram and Discord."""
# Backward-compat alias — tests and external code may reference the old name.
_TG_NAME_LIMIT = _CMD_NAME_LIMIT
# Telegram Bot API allows only lowercase a-z, 0-9, and underscores in
# command names. This regex strips everything else after initial conversion.
_TG_INVALID_CHARS = re.compile(r"[^a-z0-9_]")
_TG_MULTI_UNDERSCORE = re.compile(r"_{2,}")
def _clamp_telegram_names(
def _sanitize_telegram_name(raw: str) -> str:
"""Convert a command/skill/plugin name to a valid Telegram command name.
Telegram requires: 1-32 chars, lowercase a-z, digits 0-9, underscores only.
Steps: lowercase replace hyphens with underscores strip all other
invalid characters collapse consecutive underscores strip leading/
trailing underscores.
"""
name = raw.lower().replace("-", "_")
name = _TG_INVALID_CHARS.sub("", name)
name = _TG_MULTI_UNDERSCORE.sub("_", name)
return name.strip("_")
def _clamp_command_names(
entries: list[tuple[str, str]],
reserved: set[str],
) -> list[tuple[str, str]]:
"""Enforce Telegram's 32-char command name limit with collision avoidance.
"""Enforce 32-char command name limit with collision avoidance.
Names exceeding 32 chars are truncated. If truncation creates a duplicate
Both Telegram and Discord cap slash command names at 32 characters.
Names exceeding the limit are truncated. If truncation creates a duplicate
(against *reserved* names or earlier entries in the same batch), the name is
shortened to 31 chars and a digit ``0``-``9`` is appended to differentiate.
If all 10 digit slots are taken the entry is silently dropped.
@@ -388,10 +413,10 @@ def _clamp_telegram_names(
used: set[str] = set(reserved)
result: list[tuple[str, str]] = []
for name, desc in entries:
if len(name) > _TG_NAME_LIMIT:
candidate = name[:_TG_NAME_LIMIT]
if len(name) > _CMD_NAME_LIMIT:
candidate = name[:_CMD_NAME_LIMIT]
if candidate in used:
prefix = name[:_TG_NAME_LIMIT - 1]
prefix = name[:_CMD_NAME_LIMIT - 1]
for digit in range(10):
candidate = f"{prefix}{digit}"
if candidate not in used:
@@ -407,6 +432,129 @@ def _clamp_telegram_names(
return result
# Backward-compat alias.
_clamp_telegram_names = _clamp_command_names
# ---------------------------------------------------------------------------
# Shared skill/plugin collection for gateway platforms
# ---------------------------------------------------------------------------
def _collect_gateway_skill_entries(
platform: str,
max_slots: int,
reserved_names: set[str],
desc_limit: int = 100,
sanitize_name: "Callable[[str], str] | None" = None,
) -> tuple[list[tuple[str, str, str]], int]:
"""Collect plugin + skill entries for a gateway platform.
Priority order:
1. Plugin slash commands (take precedence over skills)
2. Built-in skill commands (fill remaining slots, alphabetical)
Only skills are trimmed when the cap is reached.
Hub-installed skills are excluded. Per-platform disabled skills are
excluded.
Args:
platform: Platform identifier for per-platform skill filtering
(``"telegram"``, ``"discord"``, etc.).
max_slots: Maximum number of entries to return (remaining slots after
built-in/core commands).
reserved_names: Names already taken by built-in commands. Mutated
in-place as new names are added.
desc_limit: Max description length (40 for Telegram, 100 for Discord).
sanitize_name: Optional name transform applied before clamping, e.g.
:func:`_sanitize_telegram_name` for Telegram. May return an
empty string to signal "skip this entry".
Returns:
``(entries, hidden_count)`` where *entries* is a list of
``(name, description, cmd_key)`` triples and *hidden_count* is the
number of skill entries dropped due to the cap. ``cmd_key`` is the
original ``/skill-name`` key from :func:`get_skill_commands`.
"""
all_entries: list[tuple[str, str, str]] = []
# --- Tier 1: Plugin slash commands (never trimmed) ---------------------
plugin_pairs: list[tuple[str, str]] = []
try:
from hermes_cli.plugins import get_plugin_manager
pm = get_plugin_manager()
plugin_cmds = getattr(pm, "_plugin_commands", {})
for cmd_name in sorted(plugin_cmds):
name = sanitize_name(cmd_name) if sanitize_name else cmd_name
if not name:
continue
desc = "Plugin command"
if len(desc) > desc_limit:
desc = desc[:desc_limit - 3] + "..."
plugin_pairs.append((name, desc))
except Exception:
pass
plugin_pairs = _clamp_command_names(plugin_pairs, reserved_names)
reserved_names.update(n for n, _ in plugin_pairs)
# Plugins have no cmd_key — use empty string as placeholder
for n, d in plugin_pairs:
all_entries.append((n, d, ""))
# --- Tier 2: Built-in skill commands (trimmed at cap) -----------------
_platform_disabled: set[str] = set()
try:
from agent.skill_utils import get_disabled_skill_names
_platform_disabled = get_disabled_skill_names(platform=platform)
except Exception:
pass
skill_triples: list[tuple[str, str, str]] = []
try:
from agent.skill_commands import get_skill_commands
from tools.skills_tool import SKILLS_DIR
_skills_dir = str(SKILLS_DIR.resolve())
_hub_dir = str((SKILLS_DIR / ".hub").resolve())
skill_cmds = get_skill_commands()
for cmd_key in sorted(skill_cmds):
info = skill_cmds[cmd_key]
skill_path = info.get("skill_md_path", "")
if not skill_path.startswith(_skills_dir):
continue
if skill_path.startswith(_hub_dir):
continue
skill_name = info.get("name", "")
if skill_name in _platform_disabled:
continue
raw_name = cmd_key.lstrip("/")
name = sanitize_name(raw_name) if sanitize_name else raw_name
if not name:
continue
desc = info.get("description", "")
if len(desc) > desc_limit:
desc = desc[:desc_limit - 3] + "..."
skill_triples.append((name, desc, cmd_key))
except Exception:
pass
# Clamp names; _clamp_command_names works on (name, desc) pairs so we
# need to zip/unzip.
skill_pairs = [(n, d) for n, d, _ in skill_triples]
key_by_pair = {(n, d): k for n, d, k in skill_triples}
skill_pairs = _clamp_command_names(skill_pairs, reserved_names)
# Skills fill remaining slots — only tier that gets trimmed
remaining = max(0, max_slots - len(all_entries))
hidden_count = max(0, len(skill_pairs) - remaining)
for n, d in skill_pairs[:remaining]:
all_entries.append((n, d, key_by_pair.get((n, d), "")))
return all_entries[:max_slots], hidden_count
# ---------------------------------------------------------------------------
# Platform-specific wrappers
# ---------------------------------------------------------------------------
def telegram_menu_commands(max_commands: int = 100) -> tuple[list[tuple[str, str]], int]:
"""Return Telegram menu commands capped to the Bot API limit.
@@ -425,80 +573,52 @@ def telegram_menu_commands(max_commands: int = 100) -> tuple[list[tuple[str, str
skill commands omitted due to the cap.
"""
core_commands = list(telegram_bot_commands())
# Reserve core names so plugin/skill truncation can't collide with them
reserved_names = {n for n, _ in core_commands}
all_commands = list(core_commands)
# Plugin slash commands get priority over skills
plugin_entries: list[tuple[str, str]] = []
try:
from hermes_cli.plugins import get_plugin_manager
pm = get_plugin_manager()
plugin_cmds = getattr(pm, "_plugin_commands", {})
for cmd_name in sorted(plugin_cmds):
tg_name = cmd_name.replace("-", "_")
desc = "Plugin command"
if len(desc) > 40:
desc = desc[:37] + "..."
plugin_entries.append((tg_name, desc))
except Exception:
pass
# Clamp plugin names to 32 chars with collision avoidance
plugin_entries = _clamp_telegram_names(plugin_entries, reserved_names)
reserved_names.update(n for n, _ in plugin_entries)
all_commands.extend(plugin_entries)
# Load per-platform disabled skills so they don't consume menu slots.
# get_skill_commands() already filters the *global* disabled list, but
# per-platform overrides (skills.platform_disabled.telegram) were never
# applied here — that's what this block fixes.
_platform_disabled: set[str] = set()
try:
from agent.skill_utils import get_disabled_skill_names
_platform_disabled = get_disabled_skill_names(platform="telegram")
except Exception:
pass
# Remaining slots go to built-in skill commands (not hub-installed).
skill_entries: list[tuple[str, str]] = []
try:
from agent.skill_commands import get_skill_commands
from tools.skills_tool import SKILLS_DIR
_skills_dir = str(SKILLS_DIR.resolve())
_hub_dir = str((SKILLS_DIR / ".hub").resolve())
skill_cmds = get_skill_commands()
for cmd_key in sorted(skill_cmds):
info = skill_cmds[cmd_key]
skill_path = info.get("skill_md_path", "")
if not skill_path.startswith(_skills_dir):
continue
if skill_path.startswith(_hub_dir):
continue
# Skip skills disabled for telegram
skill_name = info.get("name", "")
if skill_name in _platform_disabled:
continue
name = cmd_key.lstrip("/").replace("-", "_")
desc = info.get("description", "")
# Keep descriptions short — setMyCommands has an undocumented
# total payload limit. 40 chars fits 100 commands safely.
if len(desc) > 40:
desc = desc[:37] + "..."
skill_entries.append((name, desc))
except Exception:
pass
# Clamp skill names to 32 chars with collision avoidance
skill_entries = _clamp_telegram_names(skill_entries, reserved_names)
# Skills fill remaining slots — they're the only tier that gets trimmed
remaining_slots = max(0, max_commands - len(all_commands))
hidden_count = max(0, len(skill_entries) - remaining_slots)
all_commands.extend(skill_entries[:remaining_slots])
entries, hidden_count = _collect_gateway_skill_entries(
platform="telegram",
max_slots=remaining_slots,
reserved_names=reserved_names,
desc_limit=40,
sanitize_name=_sanitize_telegram_name,
)
# Drop the cmd_key — Telegram only needs (name, desc) pairs.
all_commands.extend((n, d) for n, d, _k in entries)
return all_commands[:max_commands], hidden_count
def discord_skill_commands(
max_slots: int,
reserved_names: set[str],
) -> tuple[list[tuple[str, str, str]], int]:
"""Return skill entries for Discord slash command registration.
Same priority and filtering logic as :func:`telegram_menu_commands`
(plugins > skills, hub excluded, per-platform disabled excluded), but
adapted for Discord's constraints:
- Hyphens are allowed in names (no ``-`` ``_`` sanitization)
- Descriptions capped at 100 chars (Discord's per-field max)
Args:
max_slots: Available command slots (100 minus existing built-in count).
reserved_names: Names of already-registered built-in commands.
Returns:
``(entries, hidden_count)`` where *entries* is a list of
``(discord_name, description, cmd_key)`` triples. ``cmd_key`` is
the original ``/skill-name`` key needed for the slash handler callback.
"""
return _collect_gateway_skill_entries(
platform="discord",
max_slots=max_slots,
reserved_names=set(reserved_names), # copy — don't mutate caller's set
desc_limit=100,
)
def slack_subcommand_map() -> dict[str, str]:
"""Return subcommand -> /command mapping for Slack /hermes handler.
+142 -6
View File
@@ -42,7 +42,7 @@ _EXTRA_ENV_KEYS = frozenset({
"TERMINAL_ENV", "TERMINAL_SSH_KEY", "TERMINAL_SSH_PORT",
"WHATSAPP_MODE", "WHATSAPP_ENABLED",
"MATTERMOST_HOME_CHANNEL", "MATTERMOST_REPLY_MODE",
"MATRIX_PASSWORD", "MATRIX_ENCRYPTION", "MATRIX_HOME_ROOM",
"MATRIX_PASSWORD", "MATRIX_ENCRYPTION", "MATRIX_DEVICE_ID", "MATRIX_HOME_ROOM",
"MATRIX_REQUIRE_MENTION", "MATRIX_FREE_RESPONSE_ROOMS", "MATRIX_AUTO_THREAD",
})
import yaml
@@ -590,6 +590,30 @@ OPTIONAL_ENV_VARS = {
"category": "provider",
"advanced": True,
},
"GOOGLE_API_KEY": {
"description": "Google AI Studio API key (also recognized as GEMINI_API_KEY)",
"prompt": "Google AI Studio API key",
"url": "https://aistudio.google.com/app/apikey",
"password": True,
"category": "provider",
"advanced": True,
},
"GEMINI_API_KEY": {
"description": "Google AI Studio API key (alias for GOOGLE_API_KEY)",
"prompt": "Gemini API key",
"url": "https://aistudio.google.com/app/apikey",
"password": True,
"category": "provider",
"advanced": True,
},
"GEMINI_BASE_URL": {
"description": "Google AI Studio base URL override",
"prompt": "Gemini base URL (leave empty for default)",
"url": None,
"password": False,
"category": "provider",
"advanced": True,
},
"GLM_API_KEY": {
"description": "Z.AI / GLM API key (also recognized as ZAI_API_KEY / Z_AI_API_KEY)",
"prompt": "Z.AI / GLM API key",
@@ -844,6 +868,13 @@ OPTIONAL_ENV_VARS = {
"password": True,
"category": "tool",
},
"FIRECRAWL_BROWSER_TTL": {
"description": "Firecrawl browser session TTL in seconds (optional, default 300)",
"prompt": "Browser session TTL (seconds)",
"tools": ["browser_navigate", "browser_click"],
"password": False,
"category": "tool",
},
"CAMOFOX_URL": {
"description": "Camofox browser server URL for local anti-detection browsing (e.g. http://localhost:9377)",
"prompt": "Camofox server URL",
@@ -1048,6 +1079,14 @@ OPTIONAL_ENV_VARS = {
"category": "messaging",
"advanced": True,
},
"MATRIX_DEVICE_ID": {
"description": "Stable Matrix device ID for E2EE persistence across restarts (e.g. HERMES_BOT)",
"prompt": "Matrix device ID (stable across restarts)",
"url": None,
"password": False,
"category": "messaging",
"advanced": True,
},
"GATEWAY_ALLOW_ALL_USERS": {
"description": "Allow all users to interact with messaging bots (true/false). Default: false.",
"prompt": "Allow all users (true/false)",
@@ -1240,6 +1279,43 @@ def get_missing_config_fields() -> List[Dict[str, Any]]:
return missing
def get_missing_skill_config_vars() -> List[Dict[str, Any]]:
"""Return skill-declared config vars that are missing or empty in config.yaml.
Scans all enabled skills for ``metadata.hermes.config`` entries, then checks
which ones are absent or empty under ``skills.config.<key>`` in the user's
config.yaml. Returns a list of dicts suitable for prompting.
"""
try:
from agent.skill_utils import discover_all_skill_config_vars, SKILL_CONFIG_PREFIX
except Exception:
return []
all_vars = discover_all_skill_config_vars()
if not all_vars:
return []
config = load_config()
missing: List[Dict[str, Any]] = []
for var in all_vars:
# Skill config is stored under skills.config.<logical_key>
storage_key = f"{SKILL_CONFIG_PREFIX}.{var['key']}"
parts = storage_key.split(".")
current = config
value = None
for part in parts:
if isinstance(current, dict) and part in current:
current = current[part]
value = current
else:
value = None
break
# Missing = key doesn't exist or is empty string
if value is None or (isinstance(value, str) and not value.strip()):
missing.append(var)
return missing
def check_config_version() -> Tuple[int, int]:
"""
Check config version.
@@ -1671,7 +1747,50 @@ def migrate_config(interactive: bool = True, quiet: bool = False) -> Dict[str, A
config = load_config()
config["_config_version"] = latest_ver
save_config(config)
# ── Skill-declared config vars ──────────────────────────────────────
# Skills can declare config.yaml settings they need via
# metadata.hermes.config in their SKILL.md frontmatter.
# Prompt for any that are missing/empty.
missing_skill_config = get_missing_skill_config_vars()
if missing_skill_config and interactive and not quiet:
print(f"\n {len(missing_skill_config)} skill setting(s) not configured:")
for var in missing_skill_config:
skill_name = var.get("skill", "unknown")
print(f"{var['key']}{var['description']} (from skill: {skill_name})")
print()
try:
answer = input(" Configure skill settings? [y/N]: ").strip().lower()
except (EOFError, KeyboardInterrupt):
answer = "n"
if answer in ("y", "yes"):
print()
config = load_config()
try:
from agent.skill_utils import SKILL_CONFIG_PREFIX
except Exception:
SKILL_CONFIG_PREFIX = "skills.config"
for var in missing_skill_config:
default = var.get("default", "")
default_hint = f" (default: {default})" if default else ""
value = input(f" {var['prompt']}{default_hint}: ").strip()
if not value and default:
value = str(default)
if value:
storage_key = f"{SKILL_CONFIG_PREFIX}.{var['key']}"
_set_nested(config, storage_key, value)
results["config_added"].append(var["key"])
print(f" ✓ Saved {var['key']} = {value}")
else:
results["warnings"].append(
f"Skipped {var['key']} — skill '{var.get('skill', '?')}' may ask for it later"
)
print()
save_config(config)
else:
print(" Set later with: hermes config set <key> <value>")
return results
@@ -1813,8 +1932,8 @@ _FALLBACK_COMMENT = """
#
# Supported providers:
# openrouter (OPENROUTER_API_KEY) — routes to any model
# openai-codex (OAuth — hermes login) — OpenAI Codex
# nous (OAuth — hermes login) — Nous Portal
# openai-codex (OAuth — hermes auth) — OpenAI Codex
# nous (OAuth — hermes auth) — Nous Portal
# zai (ZAI_API_KEY) — Z.AI / GLM
# kimi-coding (KIMI_API_KEY) — Kimi / Moonshot
# minimax (MINIMAX_API_KEY) — MiniMax
@@ -1856,8 +1975,8 @@ _COMMENTED_SECTIONS = """
#
# Supported providers:
# openrouter (OPENROUTER_API_KEY) — routes to any model
# openai-codex (OAuth — hermes login) — OpenAI Codex
# nous (OAuth — hermes login) — Nous Portal
# openai-codex (OAuth — hermes auth) — OpenAI Codex
# nous (OAuth — hermes auth) — Nous Portal
# zai (ZAI_API_KEY) — Z.AI / GLM
# kimi-coding (KIMI_API_KEY) — Kimi / Moonshot
# minimax (MINIMAX_API_KEY) — MiniMax
@@ -2325,6 +2444,23 @@ def show_config():
print(f" Telegram: {'configured' if telegram_token else color('not configured', Colors.DIM)}")
print(f" Discord: {'configured' if discord_token else color('not configured', Colors.DIM)}")
# Skill config
try:
from agent.skill_utils import discover_all_skill_config_vars, resolve_skill_config_values
skill_vars = discover_all_skill_config_vars()
if skill_vars:
resolved = resolve_skill_config_values(skill_vars)
print()
print(color("◆ Skill Settings", Colors.CYAN, Colors.BOLD))
for var in skill_vars:
key = var["key"]
value = resolved.get(key, "")
skill_name = var.get("skill", "")
display_val = str(value) if value else color("(not set)", Colors.DIM)
print(f" {key:<20s} {display_val} {color(f'[{skill_name}]', Colors.DIM)}")
except Exception:
pass
print()
print(color("" * 60, Colors.DIM))
print(color(" hermes config edit # Edit config file", Colors.DIM))
+1 -1
View File
@@ -836,7 +836,7 @@ def run_doctor(args):
get_honcho_client(hcfg)
check_ok(
"Honcho connected",
f"workspace={hcfg.workspace_id} mode={hcfg.memory_mode} freq={hcfg.write_frequency}",
f"workspace={hcfg.workspace_id} mode={hcfg.recall_mode} freq={hcfg.write_frequency}",
)
except Exception as _e:
check_fail("Honcho connection failed", str(_e))
+2 -2
View File
@@ -1121,7 +1121,7 @@ def launchd_start():
try:
subprocess.run(["launchctl", "kickstart", f"{_launchd_domain()}/{label}"], check=True, timeout=30)
except subprocess.CalledProcessError as e:
if e.returncode != 3:
if e.returncode not in (3, 113):
raise
print("↻ launchd job was unloaded; reloading service definition")
subprocess.run(["launchctl", "bootstrap", _launchd_domain(), str(plist_path)], check=True, timeout=30)
@@ -1183,7 +1183,7 @@ def launchd_restart():
subprocess.run(["launchctl", "kickstart", "-k", target], check=True, timeout=90)
print("✓ Service restarted")
except subprocess.CalledProcessError as e:
if e.returncode != 3:
if e.returncode not in (3, 113):
raise
# Job not loaded — bootstrap and start fresh
print("↻ launchd job was unloaded; reloading")
+123 -67
View File
@@ -908,7 +908,7 @@ def select_provider_and_model(args=None):
try:
active = resolve_provider("auto")
except AuthError:
active = "openrouter" # no provider yet; show full picker
active = None # no provider yet; default to first in list
# Detect custom endpoint
if active == "openrouter" and get_env_value("OPENAI_BASE_URL"):
@@ -921,6 +921,7 @@ def select_provider_and_model(args=None):
"copilot-acp": "GitHub Copilot ACP",
"copilot": "GitHub Copilot",
"anthropic": "Anthropic",
"gemini": "Google AI Studio",
"zai": "Z.AI / GLM",
"kimi-coding": "Kimi / Moonshot",
"minimax": "MiniMax",
@@ -933,21 +934,26 @@ def select_provider_and_model(args=None):
"huggingface": "Hugging Face",
"custom": "Custom endpoint",
}
active_label = provider_labels.get(active, active)
active_label = provider_labels.get(active, active) if active else "none"
print()
print(f" Current model: {current_model}")
print(f" Active provider: {active_label}")
print()
# Step 1: Provider selection — put active provider first with marker
providers = [
("openrouter", "OpenRouter (100+ models, pay-per-use)"),
# Step 1: Provider selection — top providers shown first, rest behind "More..."
top_providers = [
("nous", "Nous Portal (Nous Research subscription)"),
("openai-codex", "OpenAI Codex"),
("copilot-acp", "GitHub Copilot ACP (spawns `copilot --acp --stdio`)"),
("copilot", "GitHub Copilot (uses GITHUB_TOKEN or gh auth token)"),
("openrouter", "OpenRouter (100+ models, pay-per-use)"),
("anthropic", "Anthropic (Claude models — API key or Claude Code)"),
("openai-codex", "OpenAI Codex"),
("copilot", "GitHub Copilot (uses GITHUB_TOKEN or gh auth token)"),
("huggingface", "Hugging Face Inference Providers (20+ open models)"),
]
extended_providers = [
("copilot-acp", "GitHub Copilot ACP (spawns `copilot --acp --stdio`)"),
("gemini", "Google AI Studio (Gemini models — OpenAI-compatible endpoint)"),
("zai", "Z.AI / GLM (Zhipu AI direct API)"),
("kimi-coding", "Kimi / Moonshot (Moonshot AI direct API)"),
("minimax", "MiniMax (global direct API)"),
@@ -957,7 +963,6 @@ def select_provider_and_model(args=None):
("opencode-go", "OpenCode Go (open models, $10/month subscription)"),
("ai-gateway", "AI Gateway (Vercel — 200+ models, pay-per-use)"),
("alibaba", "Alibaba Cloud / DashScope Coding (Qwen + multi-provider)"),
("huggingface", "Hugging Face Inference Providers (20+ open models)"),
]
# Add user-defined custom providers from config.yaml
@@ -971,12 +976,11 @@ def select_provider_and_model(args=None):
base_url = (entry.get("base_url") or "").strip()
if not name or not base_url:
continue
# Generate a stable key from the name
key = "custom:" + name.lower().replace(" ", "-")
short_url = base_url.replace("https://", "").replace("http://", "").rstrip("/")
saved_model = entry.get("model", "")
model_hint = f"{saved_model}" if saved_model else ""
providers.append((key, f"{name} ({short_url}){model_hint}"))
top_providers.append((key, f"{name} ({short_url}){model_hint}"))
_custom_provider_map[key] = {
"name": name,
"base_url": base_url,
@@ -984,31 +988,54 @@ def select_provider_and_model(args=None):
"model": saved_model,
}
# Always add the manual custom endpoint option last
providers.append(("custom", "Custom endpoint (enter URL manually)"))
top_keys = {k for k, _ in top_providers}
extended_keys = {k for k, _ in extended_providers}
# Add removal option if there are saved custom providers
if _custom_provider_map:
providers.append(("remove-custom", "Remove a saved custom provider"))
# If the active provider is in the extended list, promote it into top
if active and active in extended_keys:
promoted = [(k, l) for k, l in extended_providers if k == active]
extended_providers = [(k, l) for k, l in extended_providers if k != active]
top_providers = promoted + top_providers
top_keys.add(active)
# Reorder so the active provider is at the top
known_keys = {k for k, _ in providers}
active_key = active if active in known_keys else "custom"
# Build the primary menu
ordered = []
for key, label in providers:
if key == active_key:
ordered.insert(0, (key, f"{label} ← currently active"))
default_idx = 0
for key, label in top_providers:
if active and key == active:
ordered.append((key, f"{label} ← currently active"))
default_idx = len(ordered) - 1
else:
ordered.append((key, label))
ordered.append(("more", "More providers..."))
ordered.append(("cancel", "Cancel"))
provider_idx = _prompt_provider_choice([label for _, label in ordered])
provider_idx = _prompt_provider_choice(
[label for _, label in ordered], default=default_idx,
)
if provider_idx is None or ordered[provider_idx][0] == "cancel":
print("No change.")
return
selected_provider = ordered[provider_idx][0]
# "More providers..." — show the extended list
if selected_provider == "more":
ext_ordered = list(extended_providers)
ext_ordered.append(("custom", "Custom endpoint (enter URL manually)"))
if _custom_provider_map:
ext_ordered.append(("remove-custom", "Remove a saved custom provider"))
ext_ordered.append(("cancel", "Cancel"))
ext_idx = _prompt_provider_choice(
[label for _, label in ext_ordered], default=0,
)
if ext_idx is None or ext_ordered[ext_idx][0] == "cancel":
print("No change.")
return
selected_provider = ext_ordered[ext_idx][0]
# Step 2: Provider-specific setup + model selection
if selected_provider == "openrouter":
_model_flow_openrouter(config, current_model)
@@ -1030,38 +1057,37 @@ def select_provider_and_model(args=None):
_model_flow_anthropic(config, current_model)
elif selected_provider == "kimi-coding":
_model_flow_kimi(config, current_model)
elif selected_provider in ("zai", "minimax", "minimax-cn", "kilocode", "opencode-zen", "opencode-go", "ai-gateway", "alibaba", "huggingface"):
elif selected_provider in ("gemini", "zai", "minimax", "minimax-cn", "kilocode", "opencode-zen", "opencode-go", "ai-gateway", "alibaba", "huggingface"):
_model_flow_api_key_provider(config, selected_provider, current_model)
def _prompt_provider_choice(choices):
"""Show provider selection menu. Returns index or None."""
def _prompt_provider_choice(choices, *, default=0):
"""Show provider selection menu with curses arrow-key navigation.
Falls back to a numbered list when curses is unavailable (e.g. piped
stdin, non-TTY environments). Returns the selected index, or None
if the user cancels.
"""
try:
from simple_term_menu import TerminalMenu
menu_items = [f" {c}" for c in choices]
menu = TerminalMenu(
menu_items, cursor_index=0,
menu_cursor="-> ", menu_cursor_style=("fg_green", "bold"),
menu_highlight_style=("fg_green",),
cycle_cursor=True, clear_screen=False,
title="Select provider:",
)
idx = menu.show()
print()
return idx
except (ImportError, NotImplementedError):
from hermes_cli.setup import _curses_prompt_choice
idx = _curses_prompt_choice("Select provider:", choices, default)
if idx >= 0:
print()
return idx
except Exception:
pass
# Fallback: numbered list
print("Select provider:")
for i, c in enumerate(choices, 1):
print(f" {i}. {c}")
marker = "" if i - 1 == default else " "
print(f" {marker} {i}. {c}")
print()
while True:
try:
val = input(f"Choice [1-{len(choices)}]: ").strip()
val = input(f"Choice [1-{len(choices)}] ({default + 1}): ").strip()
if not val:
return None
return default
idx = int(val) - 1
if 0 <= idx < len(choices):
return idx
@@ -1084,7 +1110,8 @@ def _model_flow_openrouter(config, current_model=""):
print("Get one at: https://openrouter.ai/keys")
print()
try:
key = input("OpenRouter API key (or Enter to cancel): ").strip()
import getpass
key = getpass.getpass("OpenRouter API key (or Enter to cancel): ").strip()
except (KeyboardInterrupt, EOFError):
print()
return
@@ -1267,12 +1294,21 @@ def _model_flow_openai_codex(config, current_model=""):
return
_codex_token = None
# Prefer credential pool (where `hermes auth` stores device_code tokens),
# fall back to legacy provider state.
try:
from hermes_cli.auth import resolve_codex_runtime_credentials
_codex_creds = resolve_codex_runtime_credentials()
_codex_token = _codex_creds.get("api_key")
_codex_status = get_codex_auth_status()
if _codex_status.get("logged_in"):
_codex_token = _codex_status.get("api_key")
except Exception:
pass
if not _codex_token:
try:
from hermes_cli.auth import resolve_codex_runtime_credentials
_codex_creds = resolve_codex_runtime_credentials()
_codex_token = _codex_creds.get("api_key")
except Exception:
pass
codex_models = get_codex_model_ids(access_token=_codex_token)
@@ -1307,7 +1343,8 @@ def _model_flow_custom(config):
try:
base_url = input(f"API base URL [{current_url or 'e.g. https://api.example.com/v1'}]: ").strip()
api_key = input(f"API key [{current_key[:8] + '...' if current_key else 'optional'}]: ").strip()
import getpass
api_key = getpass.getpass(f"API key [{current_key[:8] + '...' if current_key else 'optional'}]: ").strip()
except (KeyboardInterrupt, EOFError):
print("\nCancelled.")
return
@@ -1816,7 +1853,8 @@ def _model_flow_copilot(config, current_model=""):
return
elif choice == "2":
try:
new_key = input(" Token (COPILOT_GITHUB_TOKEN): ").strip()
import getpass
new_key = getpass.getpass(" Token (COPILOT_GITHUB_TOKEN): ").strip()
except (KeyboardInterrupt, EOFError):
print()
return
@@ -2057,7 +2095,8 @@ def _model_flow_kimi(config, current_model=""):
print(f"No {pconfig.name} API key configured.")
if key_env:
try:
new_key = input(f"{key_env} (or Enter to cancel): ").strip()
import getpass
new_key = getpass.getpass(f"{key_env} (or Enter to cancel): ").strip()
except (KeyboardInterrupt, EOFError):
print()
return
@@ -2151,7 +2190,8 @@ def _model_flow_api_key_provider(config, provider_id, current_model=""):
print(f"No {pconfig.name} API key configured.")
if key_env:
try:
new_key = input(f"{key_env} (or Enter to cancel): ").strip()
import getpass
new_key = getpass.getpass(f"{key_env} (or Enter to cancel): ").strip()
except (KeyboardInterrupt, EOFError):
print()
return
@@ -2180,24 +2220,37 @@ def _model_flow_api_key_provider(config, provider_id, current_model=""):
save_env_value(base_url_env, override)
effective_base = override
# Model selection — try live /models endpoint first, fall back to defaults.
# Providers with large live catalogs (100+ models) use a curated list instead
# so users see familiar model names rather than an overwhelming dump.
# Model selection — resolution order:
# 1. models.dev registry (cached, filtered for agentic/tool-capable models)
# 2. Curated static fallback list (offline insurance)
# 3. Live /models endpoint probe (small providers without models.dev data)
curated = _PROVIDER_MODELS.get(provider_id, [])
if curated and len(curated) >= 8:
# Try models.dev first — returns tool-capable models, filtered for noise
mdev_models: list = []
try:
from agent.models_dev import list_agentic_models
mdev_models = list_agentic_models(provider_id)
except Exception:
pass
if mdev_models:
model_list = mdev_models
print(f" Found {len(model_list)} model(s) from models.dev registry")
elif curated and len(curated) >= 8:
# Curated list is substantial — use it directly, skip live probe
live_models = None
model_list = curated
print(f" Showing {len(model_list)} curated models — use \"Enter custom model name\" for others.")
else:
api_key_for_probe = existing_key or (get_env_value(key_env) if key_env else "")
live_models = fetch_api_models(api_key_for_probe, effective_base)
if live_models and len(live_models) >= len(curated):
model_list = live_models
print(f" Found {len(model_list)} model(s) from {pconfig.name} API")
else:
model_list = curated
if model_list:
print(f" Showing {len(model_list)} curated models — use \"Enter custom model name\" for others.")
if live_models and len(live_models) >= len(curated):
model_list = live_models
print(f" Found {len(model_list)} model(s) from {pconfig.name} API")
else:
model_list = curated
if model_list:
print(f" Showing {len(model_list)} curated models — use \"Enter custom model name\" for others.")
# else: no defaults either, will fall through to raw input
if provider_id in {"opencode-zen", "opencode-go"}:
@@ -2285,7 +2338,8 @@ def _run_anthropic_oauth_flow(save_env_value):
print(" If the setup-token was displayed above, paste it here:")
print()
try:
manual_token = input(" Paste setup-token (or Enter to cancel): ").strip()
import getpass
manual_token = getpass.getpass(" Paste setup-token (or Enter to cancel): ").strip()
except (KeyboardInterrupt, EOFError):
print()
return False
@@ -2312,7 +2366,8 @@ def _run_anthropic_oauth_flow(save_env_value):
print(" Or paste an existing setup-token now (sk-ant-oat-...):")
print()
try:
token = input(" Setup-token (or Enter to cancel): ").strip()
import getpass
token = getpass.getpass(" Setup-token (or Enter to cancel): ").strip()
except (KeyboardInterrupt, EOFError):
print()
return False
@@ -2405,7 +2460,8 @@ def _model_flow_anthropic(config, current_model=""):
print(" Get an API key at: https://console.anthropic.com/settings/keys")
print()
try:
api_key = input(" API key (sk-ant-...): ").strip()
import getpass
api_key = getpass.getpass(" API key (sk-ant-...): ").strip()
except (KeyboardInterrupt, EOFError):
print()
return
@@ -4150,7 +4206,7 @@ For more help on a command:
)
chat_parser.add_argument(
"--provider",
choices=["auto", "openrouter", "nous", "openai-codex", "copilot-acp", "copilot", "anthropic", "huggingface", "zai", "kimi-coding", "minimax", "minimax-cn", "kilocode"],
choices=["auto", "openrouter", "nous", "openai-codex", "copilot-acp", "copilot", "anthropic", "gemini", "huggingface", "zai", "kimi-coding", "minimax", "minimax-cn", "kilocode"],
default=None,
help="Inference provider (default: auto)"
)
+4 -2
View File
@@ -8,8 +8,9 @@ Different LLM providers expect model identifiers in different formats:
hyphens: ``claude-sonnet-4-6``.
- **Copilot** expects bare names *with* dots preserved:
``claude-sonnet-4.6``.
- **OpenCode** (Zen & Go) follows the same dot-to-hyphen convention as
- **OpenCode Zen** follows the same dot-to-hyphen convention as
Anthropic: ``claude-sonnet-4-6``.
- **OpenCode Go** preserves dots in model names: ``minimax-m2.7``.
- **DeepSeek** only accepts two model identifiers:
``deepseek-chat`` and ``deepseek-reasoner``.
- **Custom** and remaining providers pass the name through as-is.
@@ -41,6 +42,7 @@ _VENDOR_PREFIXES: dict[str, str] = {
"o3": "openai",
"o4": "openai",
"gemini": "google",
"gemma": "google",
"deepseek": "deepseek",
"glm": "z-ai",
"kimi": "moonshotai",
@@ -66,7 +68,6 @@ _AGGREGATOR_PROVIDERS: frozenset[str] = frozenset({
_DOT_TO_HYPHEN_PROVIDERS: frozenset[str] = frozenset({
"anthropic",
"opencode-zen",
"opencode-go",
})
# Providers that want bare names with dots preserved.
@@ -77,6 +78,7 @@ _STRIP_VENDOR_ONLY_PROVIDERS: frozenset[str] = frozenset({
# Providers whose own naming is authoritative -- pass through unchanged.
_PASSTHROUGH_PROVIDERS: frozenset[str] = frozenset({
"gemini",
"zai",
"kimi-coding",
"minimax",
+33 -4
View File
@@ -339,12 +339,37 @@ def resolve_alias(
return None
def get_authenticated_provider_slugs(
current_provider: str = "",
user_providers: dict = None,
) -> list[str]:
"""Return slugs of providers that have credentials.
Uses ``list_authenticated_providers()`` which is backed by the models.dev
in-memory cache (1 hr TTL) no extra network cost.
"""
try:
providers = list_authenticated_providers(
current_provider=current_provider,
user_providers=user_providers,
max_models=0,
)
return [p["slug"] for p in providers]
except Exception:
return []
def _resolve_alias_fallback(
raw_input: str,
fallback_providers: tuple[str, ...] = ("openrouter", "nous"),
authenticated_providers: list[str] = (),
) -> Optional[tuple[str, str, str]]:
"""Try to resolve an alias on fallback providers."""
for provider in fallback_providers:
"""Try to resolve an alias on the user's authenticated providers.
Falls back to ``("openrouter", "nous")`` only when no authenticated
providers are supplied (backwards compat for non-interactive callers).
"""
providers = authenticated_providers or ("openrouter", "nous")
for provider in providers:
result = resolve_alias(raw_input, provider)
if result is not None:
return result
@@ -494,7 +519,11 @@ def switch_model(
# --- Step b: Alias exists but not on current provider -> fallback ---
key = raw_input.strip().lower()
if key in MODEL_ALIASES:
fallback_result = _resolve_alias_fallback(raw_input)
authed = get_authenticated_provider_slugs(
current_provider=current_provider,
user_providers=user_providers,
)
fallback_result = _resolve_alias_fallback(raw_input, authed)
if fallback_result is not None:
target_provider, new_model, resolved_alias = fallback_result
logger.debug(
+17 -1
View File
@@ -111,6 +111,17 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
"gemini-2.5-pro",
"grok-code-fast-1",
],
"gemini": [
"gemini-3.1-pro-preview",
"gemini-3-flash-preview",
"gemini-3.1-flash-lite-preview",
"gemini-2.5-pro",
"gemini-2.5-flash",
"gemini-2.5-flash-lite",
# Gemma open models (also served via AI Studio)
"gemma-4-31b-it",
"gemma-4-26b-it",
],
"zai": [
"glm-5",
"glm-5-turbo",
@@ -260,6 +271,7 @@ _PROVIDER_LABELS = {
"copilot-acp": "GitHub Copilot ACP",
"nous": "Nous Portal",
"copilot": "GitHub Copilot",
"gemini": "Google AI Studio",
"zai": "Z.AI / GLM",
"kimi-coding": "Kimi / Moonshot",
"minimax": "MiniMax",
@@ -286,6 +298,9 @@ _PROVIDER_ALIASES = {
"github-model": "copilot",
"github-copilot-acp": "copilot-acp",
"copilot-acp-agent": "copilot-acp",
"google": "gemini",
"google-gemini": "gemini",
"google-ai-studio": "gemini",
"kimi": "kimi-coding",
"moonshot": "kimi-coding",
"minimax-china": "minimax-cn",
@@ -550,7 +565,8 @@ def list_available_providers() -> list[dict[str, str]]:
# Canonical providers in display order
_PROVIDER_ORDER = [
"openrouter", "nous", "openai-codex", "copilot", "copilot-acp",
"huggingface", "zai", "kimi-coding", "minimax", "minimax-cn", "kilocode", "anthropic", "alibaba",
"gemini", "huggingface",
"zai", "kimi-coding", "minimax", "minimax-cn", "kilocode", "anthropic", "alibaba",
"opencode-zen", "opencode-go",
"ai-gateway", "deepseek", "custom",
]
+7
View File
@@ -131,6 +131,7 @@ def _browser_label(current_provider: str) -> str:
mapping = {
"browserbase": "Browserbase",
"browser-use": "Browser Use",
"firecrawl": "Firecrawl",
"camofox": "Camofox",
"local": "Local browser",
}
@@ -156,6 +157,7 @@ def _resolve_browser_feature_state(
direct_camofox: bool,
direct_browserbase: bool,
direct_browser_use: bool,
direct_firecrawl: bool,
managed_browser_available: bool,
) -> tuple[str, bool, bool, bool]:
"""Resolve browser availability using the same precedence as runtime."""
@@ -179,6 +181,10 @@ def _resolve_browser_feature_state(
available = bool(browser_local_available and direct_browser_use)
active = bool(browser_tool_enabled and available)
return current_provider, available, active, False
if current_provider == "firecrawl":
available = bool(browser_local_available and direct_firecrawl)
active = bool(browser_tool_enabled and available)
return current_provider, available, active, False
if current_provider == "camofox":
return current_provider, False, False, False
@@ -315,6 +321,7 @@ def get_nous_subscription_features(
direct_camofox=direct_camofox,
direct_browserbase=direct_browserbase,
direct_browser_use=direct_browser_use,
direct_firecrawl=direct_firecrawl,
managed_browser_available=managed_browser_available,
)
+2 -2
View File
@@ -36,7 +36,7 @@ import sys
import types
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Set
from typing import Any, Callable, Dict, List, Optional, Set, Union
from utils import env_var_enabled
@@ -95,7 +95,7 @@ class PluginManifest:
version: str = ""
description: str = ""
author: str = ""
requires_env: List[str] = field(default_factory=list)
requires_env: List[Union[str, Dict[str, Any]]] = field(default_factory=list)
provides_tools: List[str] = field(default_factory=list)
provides_hooks: List[str] = field(default_factory=list)
source: str = "" # "user", "project", or "entrypoint"
+82
View File
@@ -147,6 +147,82 @@ def _copy_example_files(plugin_dir: Path, console) -> None:
)
def _prompt_plugin_env_vars(manifest: dict, console) -> None:
"""Prompt for required environment variables declared in plugin.yaml.
``requires_env`` accepts two formats:
Simple list (backwards-compatible)::
requires_env:
- MY_API_KEY
Rich list with metadata::
requires_env:
- name: MY_API_KEY
description: "API key for Acme service"
url: "https://acme.com/keys"
secret: true
Already-set variables are skipped. Values are saved to the user's ``.env``.
"""
requires_env = manifest.get("requires_env") or []
if not requires_env:
return
from hermes_cli.config import get_env_value, save_env_value # noqa: F811
from hermes_constants import display_hermes_home
# Normalise to list-of-dicts
env_specs: list[dict] = []
for entry in requires_env:
if isinstance(entry, str):
env_specs.append({"name": entry})
elif isinstance(entry, dict) and entry.get("name"):
env_specs.append(entry)
# Filter to only vars that aren't already set
missing = [s for s in env_specs if not get_env_value(s["name"])]
if not missing:
return
plugin_name = manifest.get("name", "this plugin")
console.print(f"\n[bold]{plugin_name}[/bold] requires the following environment variables:\n")
for spec in missing:
name = spec["name"]
desc = spec.get("description", "")
url = spec.get("url", "")
secret = spec.get("secret", False)
label = f" {name}"
if desc:
label += f"{desc}"
console.print(label)
if url:
console.print(f" [dim]Get yours at: {url}[/dim]")
try:
if secret:
import getpass
value = getpass.getpass(f" {name}: ").strip()
else:
value = input(f" {name}: ").strip()
except (EOFError, KeyboardInterrupt):
console.print(f"\n[dim] Skipped (you can set these later in {display_hermes_home()}/.env)[/dim]")
return
if value:
save_env_value(name, value)
os.environ[name] = value
console.print(f" [green]✓[/green] Saved to {display_hermes_home()}/.env")
else:
console.print(f" [dim] Skipped (set {name} in {display_hermes_home()}/.env later)[/dim]")
console.print()
def _display_after_install(plugin_dir: Path, identifier: str) -> None:
"""Show after-install.md if it exists, otherwise a default message."""
from rich.console import Console
@@ -306,6 +382,12 @@ def cmd_install(identifier: str, force: bool = False) -> None:
# Copy .example files to their real names (e.g. config.yaml.example → config.yaml)
_copy_example_files(target, console)
# Re-read manifest from installed location (for env var prompting)
installed_manifest = _read_manifest(target)
# Prompt for required environment variables before showing after-install docs
_prompt_plugin_env_vars(installed_manifest, console)
_display_after_install(target, identifier)
console.print("[dim]Restart the gateway for the plugin to take effect:[/dim]")
+5 -1
View File
@@ -495,7 +495,11 @@ def _resolve_explicit_runtime(
explicit_base_url
or str(state.get("inference_base_url") or auth_mod.DEFAULT_NOUS_INFERENCE_URL).strip().rstrip("/")
)
api_key = explicit_api_key or str(state.get("agent_key") or state.get("access_token") or "").strip()
# Only use agent_key for inference — access_token is an OAuth token for the
# portal API (minting keys, refreshing tokens), not for the inference API.
# Falling back to access_token sends an OAuth bearer token to the inference
# endpoint, which returns 404 because it is not a valid inference credential.
api_key = explicit_api_key or str(state.get("agent_key") or "").strip()
expires_at = state.get("agent_key_expires_at") or state.get("expires_at")
if not api_key:
creds = resolve_nous_runtime_credentials(
+529 -520
View File
File diff suppressed because it is too large Load Diff
+9
View File
@@ -315,6 +315,15 @@ TOOL_CATEGORIES = {
"browser_provider": "browser-use",
"post_setup": "browserbase",
},
{
"name": "Firecrawl",
"tag": "Cloud browser with remote execution",
"env_vars": [
{"key": "FIRECRAWL_API_KEY", "prompt": "Firecrawl API key", "url": "https://firecrawl.dev"},
],
"browser_provider": "firecrawl",
"post_setup": "browserbase",
},
{
"name": "Camofox",
"tag": "Local anti-detection browser (Firefox/Camoufox)",
+1 -1
View File
@@ -561,7 +561,7 @@
# ── Activation: link config + auth + documents ────────────────────
{
system.activationScripts."hermes-agent-setup" = lib.stringAfter [ "users" ] ''
system.activationScripts."hermes-agent-setup" = lib.stringAfter [ "users" "setupSecrets" ] ''
# Ensure directories exist (activation runs before tmpfiles)
mkdir -p ${cfg.stateDir}/.hermes
mkdir -p ${cfg.stateDir}/home
+1 -1
View File
@@ -21,7 +21,7 @@
in {
packages.default = pkgs.stdenv.mkDerivation {
pname = "hermes-agent";
version = "0.1.0";
version = (builtins.fromTOML (builtins.readFile ../pyproject.toml)).project.version;
dontUnpack = true;
dontBuild = true;
+40 -2
View File
@@ -23,6 +23,7 @@ Capabilities:
from __future__ import annotations
import atexit
import json
import logging
import os
@@ -37,6 +38,30 @@ _DEFAULT_ENDPOINT = "http://127.0.0.1:1933"
_TIMEOUT = 30.0
# ---------------------------------------------------------------------------
# Process-level atexit safety net — ensures pending sessions are committed
# even if shutdown_memory_provider is never called (e.g. gateway crash,
# SIGKILL, or exception in _async_flush_memories preventing shutdown).
# ---------------------------------------------------------------------------
_last_active_provider: Optional["OpenVikingMemoryProvider"] = None
def _atexit_commit_sessions():
"""Fire on_session_end for the last active provider on process exit."""
global _last_active_provider
provider = _last_active_provider
if provider is None:
return
_last_active_provider = None
try:
provider.on_session_end([])
except Exception:
pass # best-effort at shutdown time
atexit.register(_atexit_commit_sessions)
# ---------------------------------------------------------------------------
# HTTP helper — uses httpx to avoid requiring the openviking SDK
# ---------------------------------------------------------------------------
@@ -277,6 +302,10 @@ class OpenVikingMemoryProvider(MemoryProvider):
logger.warning("httpx not installed — OpenViking plugin disabled")
self._client = None
# Register as the last active provider for atexit safety net
global _last_active_provider
_last_active_provider = self
def system_prompt_block(self) -> str:
if not self._client:
return ""
@@ -387,13 +416,18 @@ class OpenVikingMemoryProvider(MemoryProvider):
OpenViking automatically extracts 6 categories of memories:
profile, preferences, entities, events, cases, and patterns.
"""
if not self._client or self._turn_count == 0:
if not self._client:
return
# Wait for any pending sync to finish first
# Wait for any pending sync to finish first — do this before the
# turn_count check so the last turn's messages are flushed even if
# the count hasn't been incremented yet.
if self._sync_thread and self._sync_thread.is_alive():
self._sync_thread.join(timeout=10.0)
if self._turn_count == 0:
return
try:
self._client.post(f"/api/v1/sessions/{self._session_id}/commit")
logger.info("OpenViking session %s committed (%d turns)", self._session_id, self._turn_count)
@@ -449,6 +483,10 @@ class OpenVikingMemoryProvider(MemoryProvider):
for t in (self._sync_thread, self._prefetch_thread):
if t and t.is_alive():
t.join(timeout=5.0)
# Clear atexit reference so it doesn't double-commit
global _last_active_provider
if _last_active_provider is self:
_last_active_provider = None
# -- Tool implementations ------------------------------------------------
+1 -1
View File
@@ -102,7 +102,7 @@ hermes-agent = "run_agent:main"
hermes-acp = "acp_adapter.entry:main"
[tool.setuptools]
py-modules = ["run_agent", "model_tools", "toolsets", "batch_runner", "trajectory_compressor", "toolset_distributions", "cli", "hermes_constants", "hermes_state", "hermes_time", "rl_cli", "utils"]
py-modules = ["run_agent", "model_tools", "toolsets", "batch_runner", "trajectory_compressor", "toolset_distributions", "cli", "hermes_constants", "hermes_state", "hermes_time", "hermes_logging", "rl_cli", "utils"]
[tool.setuptools.packages.find]
include = ["agent", "tools", "tools.*", "hermes_cli", "gateway", "gateway.*", "cron", "acp_adapter", "plugins", "plugins.*"]
+123 -75
View File
@@ -407,68 +407,6 @@ def _strip_budget_warnings_from_history(messages: list) -> None:
msg["content"] = cleaned
# =========================================================================
# Large tool result handler — save oversized output to temp file
# =========================================================================
# Threshold at which tool results are saved to a file instead of kept inline.
# 100K chars ≈ 25K tokens — generous for any reasonable output but prevents
# catastrophic context explosions.
_LARGE_RESULT_CHARS = 100_000
# How many characters of the original result to include as an inline preview
# so the model has immediate context about what the tool returned.
_LARGE_RESULT_PREVIEW_CHARS = 1_500
def _save_oversized_tool_result(function_name: str, function_result: str) -> str:
"""Replace oversized tool results with a file reference + preview.
When a tool returns more than ``_LARGE_RESULT_CHARS`` characters, the full
content is written to a temporary file under ``HERMES_HOME/cache/tool_responses/``
and the result sent to the model is replaced with:
a brief head preview (first ``_LARGE_RESULT_PREVIEW_CHARS`` chars)
the file path so the model can use ``read_file`` / ``search_files``
Falls back to destructive truncation if the file write fails.
"""
original_len = len(function_result)
if original_len <= _LARGE_RESULT_CHARS:
return function_result
# Build the target directory
try:
response_dir = os.path.join(get_hermes_home(), "cache", "tool_responses")
os.makedirs(response_dir, exist_ok=True)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S_%f")
# Sanitize tool name for use in filename
safe_name = re.sub(r"[^\w\-]", "_", function_name)[:40]
filename = f"{safe_name}_{timestamp}.txt"
filepath = os.path.join(response_dir, filename)
with open(filepath, "w", encoding="utf-8") as f:
f.write(function_result)
preview = function_result[:_LARGE_RESULT_PREVIEW_CHARS]
return (
f"{preview}\n\n"
f"[Large tool response: {original_len:,} characters total — "
f"only the first {_LARGE_RESULT_PREVIEW_CHARS:,} shown above. "
f"Full output saved to: {filepath}\n"
f"Use read_file or search_files on that path to access the rest.]"
)
except Exception as exc:
# Fall back to destructive truncation if file write fails
logger.warning("Failed to save large tool result to file: %s", exc)
return (
function_result[:_LARGE_RESULT_CHARS]
+ f"\n\n[Truncated: tool response was {original_len:,} chars, "
f"exceeding the {_LARGE_RESULT_CHARS:,} char limit. "
f"File save failed: {exc}]"
)
class AIAgent:
"""
AI Agent with tool calling capabilities.
@@ -961,6 +899,10 @@ class AIAgent:
short_uuid = uuid.uuid4().hex[:6]
self.session_id = f"{timestamp_str}_{short_uuid}"
# Per-result and per-turn output persistence (see tools/tool_result_storage.py)
from tools.tool_result_storage import get_storage_dir as _get_tool_storage_dir
self._tool_result_storage_dir = _get_tool_storage_dir(self.session_id)
# Session logs go into ~/.hermes/sessions/ alongside gateway sessions
hermes_home = get_hermes_home()
self.logs_dir = hermes_home / "sessions"
@@ -3868,7 +3810,12 @@ class AIAgent:
has_tool_calls = False
first_delta_fired = False
self._reasoning_deltas_fired = False
# Accumulate streamed text so we can recover if get_final_response()
# returns empty output (e.g. chatgpt.com backend-api sends
# response.incomplete instead of response.completed).
self._codex_streamed_text_parts: list = []
for attempt in range(max_stream_retries + 1):
collected_output_items: list = []
try:
with active_client.responses.stream(**api_kwargs) as stream:
for event in stream:
@@ -3878,6 +3825,8 @@ class AIAgent:
# Fire callbacks on text content deltas (suppress during tool calls)
if "output_text.delta" in event_type or event_type == "response.output_text.delta":
delta_text = getattr(event, "delta", "")
if delta_text:
self._codex_streamed_text_parts.append(delta_text)
if delta_text and not has_tool_calls:
if not first_delta_fired:
first_delta_fired = True
@@ -3895,7 +3844,51 @@ class AIAgent:
reasoning_text = getattr(event, "delta", "")
if reasoning_text:
self._fire_reasoning_delta(reasoning_text)
return stream.get_final_response()
# Collect completed output items — some backends
# (chatgpt.com/backend-api/codex) stream valid items
# via response.output_item.done but the SDK's
# get_final_response() returns an empty output list.
elif event_type == "response.output_item.done":
done_item = getattr(event, "item", None)
if done_item is not None:
collected_output_items.append(done_item)
# Log non-completed terminal events for diagnostics
elif event_type in ("response.incomplete", "response.failed"):
resp_obj = getattr(event, "response", None)
status = getattr(resp_obj, "status", None) if resp_obj else None
incomplete_details = getattr(resp_obj, "incomplete_details", None) if resp_obj else None
logger.warning(
"Codex Responses stream received terminal event %s "
"(status=%s, incomplete_details=%s, streamed_chars=%d). %s",
event_type, status, incomplete_details,
sum(len(p) for p in self._codex_streamed_text_parts),
self._client_log_context(),
)
final_response = stream.get_final_response()
# PATCH: ChatGPT Codex backend streams valid output items
# but get_final_response() can return an empty output list.
# Backfill from collected items or synthesize from deltas.
_out = getattr(final_response, "output", None)
if isinstance(_out, list) and not _out:
if collected_output_items:
final_response.output = list(collected_output_items)
logger.debug(
"Codex stream: backfilled %d output items from stream events",
len(collected_output_items),
)
elif self._codex_streamed_text_parts and not has_tool_calls:
assembled = "".join(self._codex_streamed_text_parts)
final_response.output = [SimpleNamespace(
type="message",
role="assistant",
status="completed",
content=[SimpleNamespace(type="output_text", text=assembled)],
)]
logger.debug(
"Codex stream: synthesized output from %d text deltas (%d chars)",
len(self._codex_streamed_text_parts), len(assembled),
)
return final_response
except (_httpx.RemoteProtocolError, _httpx.ReadTimeout, _httpx.ConnectError, ConnectionError) as exc:
if attempt < max_stream_retries:
logger.debug(
@@ -5224,11 +5217,13 @@ class AIAgent:
return transformed
def _anthropic_preserve_dots(self) -> bool:
"""True when using Alibaba/DashScope anthropic-compatible endpoint (model names keep dots, e.g. qwen3.5-plus)."""
if (getattr(self, "provider", "") or "").lower() == "alibaba":
"""True when using an anthropic-compatible endpoint that preserves dots in model names.
Alibaba/DashScope keeps dots (e.g. qwen3.5-plus).
OpenCode Go keeps dots (e.g. minimax-m2.7)."""
if (getattr(self, "provider", "") or "").lower() in {"alibaba", "opencode-go"}:
return True
base = (getattr(self, "base_url", "") or "").lower()
return "dashscope" in base or "aliyuncs" in base
return "dashscope" in base or "aliyuncs" in base or "opencode.ai/zen/go" in base
def _build_api_kwargs(self, api_messages: list) -> dict:
"""Build the keyword arguments dict for the active API mode."""
@@ -5436,6 +5431,12 @@ class AIAgent:
if extra_body:
api_kwargs["extra_body"] = extra_body
# xAI prompt caching: send x-grok-conv-id header to route requests
# to the same server, maximizing automatic cache hits.
# https://docs.x.ai/developers/advanced-api-usage/prompt-caching
if "x.ai" in self._base_url_lower and hasattr(self, "session_id") and self.session_id:
api_kwargs["extra_headers"] = {"x-grok-conv-id": self.session_id}
return api_kwargs
def _supports_reasoning_extra_body(self) -> bool:
@@ -6194,8 +6195,15 @@ class AIAgent:
except Exception as cb_err:
logging.debug(f"Tool complete callback error: {cb_err}")
# Save oversized results to file instead of destructive truncation
function_result = _save_oversized_tool_result(name, function_result)
# Persist oversized results to disk (model can read_file to access full output)
from tools.tool_result_storage import maybe_persist_tool_result
function_result = maybe_persist_tool_result(
content=function_result,
tool_name=name,
tool_use_id=tc.id,
storage_dir=self._tool_result_storage_dir,
)
# Discover subdirectory context files from tool arguments
subdir_hints = self._subdirectory_hints.check_tool_call(name, args)
@@ -6210,6 +6218,13 @@ class AIAgent:
}
messages.append(tool_msg)
# ── Per-turn aggregate budget enforcement ─────────────────────────
from tools.tool_result_storage import enforce_turn_budget
num_tools = len(parsed_calls)
if num_tools > 0:
turn_tool_msgs = messages[-num_tools:]
enforce_turn_budget(turn_tool_msgs, self._tool_result_storage_dir)
# ── Budget pressure injection ────────────────────────────────────
budget_warning = self._get_budget_warning(api_call_count)
if budget_warning and messages and messages[-1].get("role") == "tool":
@@ -6494,8 +6509,15 @@ class AIAgent:
except Exception as cb_err:
logging.debug(f"Tool complete callback error: {cb_err}")
# Save oversized results to file instead of destructive truncation
function_result = _save_oversized_tool_result(function_name, function_result)
# Persist oversized results to disk (model can read_file to access full output)
from tools.tool_result_storage import maybe_persist_tool_result
function_result = maybe_persist_tool_result(
content=function_result,
tool_name=function_name,
tool_use_id=tool_call.id,
storage_dir=self._tool_result_storage_dir,
)
# Discover subdirectory context files from tool arguments
subdir_hints = self._subdirectory_hints.check_tool_call(function_name, function_args)
@@ -6533,6 +6555,14 @@ class AIAgent:
if self.tool_delay > 0 and i < len(assistant_message.tool_calls):
time.sleep(self.tool_delay)
# ── Per-turn aggregate budget enforcement ─────────────────────────
from tools.tool_result_storage import enforce_turn_budget as _enforce_budget
num_tools_seq = len(assistant_message.tool_calls)
if num_tools_seq > 0:
turn_tool_msgs_seq = [m for m in messages[-num_tools_seq * 2:]
if m.get("role") == "tool"]
_enforce_budget(turn_tool_msgs_seq, self._tool_result_storage_dir)
# ── Budget pressure injection ─────────────────────────────────
# After all tool calls in this turn are processed, check if we're
# approaching max_iterations. If so, inject a warning into the LAST
@@ -7358,6 +7388,18 @@ class AIAgent:
response_invalid = True
error_details.append("response.output is not a list")
elif len(output_items) == 0:
# If we reach here, _run_codex_stream's backfill
# from output_item.done events and text-delta
# synthesis both failed to populate output.
_resp_status = getattr(response, "status", None)
_resp_incomplete = getattr(response, "incomplete_details", None)
logging.warning(
"Codex response.output is empty after stream backfill "
"(status=%s, incomplete_details=%s, model=%s). %s",
_resp_status, _resp_incomplete,
getattr(response, "model", None),
f"api_mode={self.api_mode} provider={self.provider}",
)
response_invalid = True
error_details.append("response.output is empty")
elif self.api_mode == "anthropic_messages":
@@ -8179,11 +8221,17 @@ class AIAgent:
self._vprint(f"{self.log_prefix} 🌐 Endpoint: {_base}", force=True)
# Actionable guidance for common auth errors
if status_code in (401, 403) or "unauthorized" in error_msg or "forbidden" in error_msg or "permission" in error_msg:
self._vprint(f"{self.log_prefix} 💡 Your API key was rejected by the provider. Check:", force=True)
self._vprint(f"{self.log_prefix} • Is the key valid? Run: hermes setup", force=True)
self._vprint(f"{self.log_prefix} • Does your account have access to {_model}?", force=True)
if "openrouter" in str(_base).lower():
self._vprint(f"{self.log_prefix} • Check credits: https://openrouter.ai/settings/credits", force=True)
if _provider == "openai-codex" and status_code == 401:
self._vprint(f"{self.log_prefix} 💡 Codex OAuth token was rejected (HTTP 401). Your token may have been", force=True)
self._vprint(f"{self.log_prefix} refreshed by another client (Codex CLI, VS Code). To fix:", force=True)
self._vprint(f"{self.log_prefix} 1. Run `codex` in your terminal to generate fresh tokens.", force=True)
self._vprint(f"{self.log_prefix} 2. Then run `hermes auth` to re-authenticate.", force=True)
else:
self._vprint(f"{self.log_prefix} 💡 Your API key was rejected by the provider. Check:", force=True)
self._vprint(f"{self.log_prefix} • Is the key valid? Run: hermes setup", force=True)
self._vprint(f"{self.log_prefix} • Does your account have access to {_model}?", force=True)
if "openrouter" in str(_base).lower():
self._vprint(f"{self.log_prefix} • Check credits: https://openrouter.ai/settings/credits", force=True)
else:
self._vprint(f"{self.log_prefix} 💡 This type of error won't be fixed by retrying.", force=True)
logging.error(f"{self.log_prefix}Non-retryable client error: {api_error}")
+1 -1
View File
@@ -38,7 +38,7 @@ $NodeVersion = "22"
function Write-Banner {
Write-Host ""
Write-Host "┌─────────────────────────────────────────────────────────┐" -ForegroundColor Magenta
Write-Host "│ ⚕ Hermes Agent Installer │" -ForegroundColor Magenta
Write-Host "│ ⚕ Hermes Agent Installer " -ForegroundColor Magenta
Write-Host "├─────────────────────────────────────────────────────────┤" -ForegroundColor Magenta
Write-Host "│ An open source AI agent by Nous Research. │" -ForegroundColor Magenta
Write-Host "└─────────────────────────────────────────────────────────┘" -ForegroundColor Magenta
+5
View File
@@ -234,3 +234,8 @@ Always iterate at `-ql`. Only render `-qh` for final output.
| `references/scene-planning.md` | Narrative arcs, layout templates, scene transitions, planning template |
| `references/rendering.md` | CLI reference, quality presets, ffmpeg, voiceover workflow, GIF export |
| `references/troubleshooting.md` | LaTeX errors, animation errors, common mistakes, debugging |
| `references/animation-design-thinking.md` | When to animate vs show static, decomposition, pacing, narration sync |
| `references/updaters-and-trackers.md` | ValueTracker, add_updater, always_redraw, time-based updaters, patterns |
| `references/paper-explainer.md` | Turning research papers into animations — workflow, templates, domain patterns |
| `references/decorations.md` | SurroundingRectangle, Brace, arrows, DashedLine, Angle, annotation lifecycle |
| `references/production-quality.md` | Pre-code, pre-render, post-render checklists, spatial layout, color, tempo |
@@ -0,0 +1,161 @@
# Animation Design Thinking
How to decide WHAT to animate and HOW to structure it — before writing any code.
## Should I animate this?
Not everything benefits from animation. Motion adds cognitive load. Bad animation is worse than a good static diagram.
**Animate when:**
- A sequence unfolds over time (algorithm steps, derivation, pipeline stages)
- Spatial relationships change (transformation, deformation, rotation)
- Something is built from parts (construction, assembly, accumulation)
- You're comparing states (before/after, method A vs method B)
- Temporal evolution is the point (training curves, wave propagation, gradient descent)
**Show static when:**
- The concept is a single labeled diagram (circuit, anatomy, architecture overview)
- Motion would distract from spatial layout
- The viewer needs to study it carefully (dense table, reference chart)
- The concept is already intuitive from a well-labeled figure
**Rule of thumb:** If you'd explain it with "first X, then Y, then Z" — animate it. If you'd explain it by pointing at parts of one picture — show it static.
## Decomposing a concept into animation
### Step 1: Write the narration first
Before any code, write what the narrator would say. This determines:
- **Order** — what concept comes first
- **Duration** — how long each idea gets
- **Visuals** — what the viewer must SEE when they HEAR each sentence
A scene where the narration says "the gradient points uphill" must show a gradient arrow at that moment. If the visual doesn't match the audio, the viewer's brain splits attention and both tracks are lost.
### Step 2: Identify visual beats
A "beat" is a moment where something changes on screen. Mark each beat in your narration:
```
"Consider a function f of x." → [BEAT: axes + curve appear]
"At this point..." → [BEAT: dot appears on curve]
"...the slope is positive." → [BEAT: tangent line drawn]
"So the gradient tells us to go left." → [BEAT: arrow points left, dot moves]
```
Each beat is one `self.play()` call or a small group of simultaneous animations.
### Step 3: Choose the right tool per beat
| Visual need | Manim approach |
|-------------|----------------|
| Object appears for first time | `Create`, `Write`, `FadeIn`, `GrowFromCenter` |
| Object transforms into another | `Transform`, `ReplacementTransform`, `FadeTransform` |
| Attention drawn to existing object | `Indicate`, `Circumscribe`, `Flash`, `ShowPassingFlash` |
| Continuous relationship maintained | `add_updater`, `always_redraw`, `ValueTracker` |
| Object leaves the scene | `FadeOut`, `Uncreate`, `ShrinkToCenter` |
| Static context that stays visible | `self.add()` (no animation) |
## Pacing: the universal mistake is too fast
### Timing rules
| Content type | Minimum on-screen time |
|-------------|----------------------|
| New equation appearing | 2.0s animation + 2.0s pause |
| New concept label | 1.0s animation + 1.0s pause |
| Key insight ("aha moment") | 2.5s animation + 3.0s pause |
| Supporting annotation | 0.8s animation + 0.5s pause |
| Scene transition (FadeOut all) | 0.5s animation + 0.3s pause |
### Breathing room
After every reveal, add `self.wait()`. The viewer needs time to:
1. Read the new text
2. Connect it to what's already on screen
3. Form an expectation about what comes next
**No wait = the viewer is always behind you.** They're still reading the equation when you've already started transforming it.
### Tempo variation
Monotonous pacing feels like a lecture. Vary the tempo:
- **Slow build** for core concepts (long run_time, long pauses)
- **Quick succession** for supporting details (short run_time, minimal pauses)
- **Dramatic pause** before the key reveal (extra `self.wait(2.0)` before the "aha")
- **Rapid montage** for "and this applies to X, Y, Z..." sequences (`LaggedStart` with tight lag_ratio)
## Narration synchronization
### The "see then hear" principle
The visual should appear slightly BEFORE the narration describes it. When the viewer sees a circle appear and THEN hears "consider a circle," the visual primes their brain for the concept. The reverse — hearing first, seeing second — creates confusion because they're searching the screen for something that isn't there yet.
### Practical timing
```python
# Scene duration should match narration duration.
# If narration for this scene is 8 seconds:
# Total animation run_times + total self.wait() times = ~8 seconds.
# Use manim-voiceover for automatic sync:
with self.voiceover(text="The gradient points downhill") as tracker:
self.play(GrowArrow(gradient_arrow), run_time=tracker.duration)
```
## Equation decomposition strategy
### The "dim and reveal" pattern
When building a complex equation step by step:
1. Show the full equation dimmed at `opacity=0.2` (sets expectation for where you're going)
2. Highlight the first term at full opacity
3. Explain it
4. Highlight the next term, dim the first to `0.5` (it's now context)
5. Repeat until the full equation is bright
This is better than building left-to-right because the viewer always sees the destination.
### Term ordering
Animate terms in the order the viewer needs to understand them, not in the order they appear in the equation. For `E = mc²`:
- Show `E` (the thing we want to know)
- Then `m` (the input)
- Then `c²` (the constant that makes it work)
- Then the `=` (connecting them)
## Architecture and pipeline diagrams
### Box granularity
The most common mistake: too many boxes. Each box is a concept the viewer must track. Five boxes with clear labels beats twelve boxes with abbreviations.
**Rule:** If two consecutive boxes could be labeled "X" and "process X output," merge them into one box.
### Animation strategy
Build pipelines left-to-right (or top-to-bottom) with arrows connecting them:
1. First box appears alone → explain it
2. Arrow grows from first to second → "the output feeds into..."
3. Second box appears → explain it
4. Repeat
Then show data flowing through: `ShowPassingFlash` along the arrows, or a colored dot traversing the path.
### The zoom-and-return pattern
For complex systems:
1. Show the full overview (all boxes, small)
2. Zoom into one box (`MovingCameraScene.camera.frame.animate`)
3. Expand that box into its internal components
4. Zoom back out to the overview
5. Zoom into the next box
## Common design mistakes
1. **Animating everything at once.** The viewer can track 1-2 simultaneous animations. More than that and nothing registers.
2. **No visual hierarchy.** Everything at the same opacity/size/color means nothing stands out. Use opacity layering.
3. **Equations without context.** An equation appearing alone means nothing. Always show the geometric/visual interpretation first or simultaneously.
4. **Skipping the "why."** Showing HOW a transformation works without WHY it matters. Add a sentence/label explaining the purpose.
5. **Identical pacing throughout.** Every animation at run_time=1.5, every wait at 1.0. Vary it.
6. **Forgetting the audience.** A video for high schoolers needs different pacing and complexity than one for PhD students. Decide the audience in the planning phase.
@@ -0,0 +1,202 @@
# Decorations and Visual Polish
Decorations are mobjects that annotate, highlight, or frame other mobjects. They turn a technically correct animation into a visually polished one.
## SurroundingRectangle
Draws a rectangle around any mobject. The go-to for highlighting:
```python
highlight = SurroundingRectangle(
equation[2], # the term to highlight
color=YELLOW,
buff=0.15, # padding between content and border
corner_radius=0.1, # rounded corners
stroke_width=2
)
self.play(Create(highlight))
self.wait(1)
self.play(FadeOut(highlight))
```
### Around part of an equation
```python
eq = MathTex(r"E", r"=", r"m", r"c^2")
box = SurroundingRectangle(eq[2:], color=YELLOW, buff=0.1) # highlight "mc²"
label = Text("mass-energy", font_size=18, font="Menlo", color=YELLOW)
label.next_to(box, DOWN, buff=0.2)
self.play(Create(box), FadeIn(label))
```
## BackgroundRectangle
Semi-transparent background behind text for readability over complex scenes:
```python
bg = BackgroundRectangle(equation, fill_opacity=0.7, buff=0.2, color=BLACK)
self.play(FadeIn(bg), Write(equation))
# Or using set_stroke for a "backdrop" effect on the text itself:
label.set_stroke(BLACK, width=5, background=True)
```
The `set_stroke(background=True)` approach is cleaner for text labels over graphs/diagrams.
## Brace and BraceLabel
Curly braces that annotate sections of a diagram or equation:
```python
brace = Brace(equation[2:4], DOWN, color=YELLOW)
brace_label = brace.get_text("these terms", font_size=20)
self.play(GrowFromCenter(brace), FadeIn(brace_label))
# Between two specific points
brace = BraceBetweenPoints(point_a, point_b, direction=UP)
```
### Brace placement
```python
# Below a group
Brace(group, DOWN)
# Above a group
Brace(group, UP)
# Left of a group
Brace(group, LEFT)
# Right of a group
Brace(group, RIGHT)
```
## Arrows for Annotation
### Straight arrows pointing to mobjects
```python
arrow = Arrow(
start=label.get_bottom(),
end=target.get_top(),
color=YELLOW,
stroke_width=2,
buff=0.1, # gap between arrow tip and target
max_tip_length_to_length_ratio=0.15 # small arrowhead
)
self.play(GrowArrow(arrow), FadeIn(label))
```
### Curved arrows
```python
arrow = CurvedArrow(
start_point=source.get_right(),
end_point=target.get_left(),
angle=PI/4, # curve angle
color=PRIMARY
)
```
### Labeling with arrows
```python
# LabeledArrow: arrow with built-in text label
arr = LabeledArrow(
Text("gradient", font_size=16, font="Menlo"),
start=point_a, end=point_b, color=RED
)
```
## DashedLine and DashedVMobject
```python
# Dashed line (for asymptotes, construction lines, implied connections)
asymptote = DashedLine(
axes.c2p(2, -3), axes.c2p(2, 3),
color=YELLOW, dash_length=0.15
)
# Make any VMobject dashed
dashed_circle = DashedVMobject(Circle(radius=2, color=BLUE), num_dashes=30)
```
## Angle and RightAngle Markers
```python
line1 = Line(ORIGIN, RIGHT * 2)
line2 = Line(ORIGIN, UP * 2 + RIGHT)
# Angle arc between two lines
angle = Angle(line1, line2, radius=0.5, color=YELLOW)
angle_value = angle.get_value() # radians
# Right angle marker (the small square)
right_angle = RightAngle(line1, Line(ORIGIN, UP * 2), length=0.3, color=WHITE)
```
## Cross (strikethrough)
Mark something as wrong or deprecated:
```python
cross = Cross(old_equation, color=RED, stroke_width=4)
self.play(Create(cross))
# Then show the correct version
```
## Underline
```python
underline = Underline(important_text, color=ACCENT, stroke_width=3)
self.play(Create(underline))
```
## Color Highlighting Workflow
### Method 1: At creation with t2c
```python
text = Text("The gradient is negative here", t2c={"gradient": BLUE, "negative": RED})
```
### Method 2: set_color_by_tex after creation
```python
eq = MathTex(r"\nabla L = -\frac{\partial L}{\partial w}")
eq.set_color_by_tex(r"\nabla", BLUE)
eq.set_color_by_tex(r"\partial", RED)
```
### Method 3: Index into submobjects
```python
eq = MathTex(r"a", r"+", r"b", r"=", r"c")
eq[0].set_color(RED) # "a"
eq[2].set_color(BLUE) # "b"
eq[4].set_color(GREEN) # "c"
```
## Combining Annotations
Layer multiple annotations for emphasis:
```python
# Highlight a term, add a brace, and an arrow — in sequence
box = SurroundingRectangle(eq[2], color=YELLOW, buff=0.1)
brace = Brace(eq[2], DOWN, color=YELLOW)
label = brace.get_text("learning rate", font_size=18)
self.play(Create(box))
self.wait(0.5)
self.play(FadeOut(box), GrowFromCenter(brace), FadeIn(label))
self.wait(1.5)
self.play(FadeOut(brace), FadeOut(label))
```
### The annotation lifecycle
Annotations should follow a rhythm:
1. **Appear** — draw attention (Create, GrowFromCenter)
2. **Hold** — viewer reads and understands (self.wait)
3. **Disappear** — clear the stage for the next thing (FadeOut)
Never leave annotations on screen indefinitely — they become visual noise once their purpose is served.
@@ -0,0 +1,255 @@
# Paper Explainer Workflow
How to turn a research paper into an animated explainer video.
## Why animate a paper?
A research paper is optimized for precision and completeness. A video is optimized for understanding and retention. The translation is NOT "read the paper aloud with pictures" — it's "extract the core insight and make it feel obvious through visual storytelling."
The paper has one job: prove the claim is true. The video has a different job: make the viewer understand WHY the claim is true, and WHY it matters.
## Who is watching?
Before anything, decide the audience:
| Audience | Prerequisites | Pacing | Depth |
|----------|--------------|--------|-------|
| General public | None | Slow, many analogies | Intuition only, skip proofs |
| Undergrad students | Basic math/CS | Medium, some formalism | Key equations, skip derivations |
| Grad students / researchers | Domain knowledge | Faster, more notation | Full equations, sketch proofs |
This determines everything: vocabulary, pacing, which sections to animate, how much math to show.
## The 5-minute template
Most paper explainers fit this structure (scale times proportionally for longer videos):
| Section | Duration | Purpose |
|---------|----------|---------|
| **Hook** | 0:00-0:30 | Surprising result or provocative question |
| **Problem** | 0:30-1:30 | What was broken/missing before this paper |
| **Key insight** | 1:30-3:00 | The core idea, explained visually |
| **How it works** | 3:00-4:00 | Method/algorithm, simplified |
| **Evidence** | 4:00-4:30 | Key result that proves it works |
| **Implications** | 4:30-5:00 | Why it matters, what it enables |
### What to skip
- Related work survey → one sentence: "Previous approaches did X, which had problem Y"
- Implementation details → skip unless they're the contribution
- Ablation studies → show one chart at most
- Proofs → show the key step, not the full proof
- Hyperparameter tuning → skip entirely
### What to expand
- The core insight → this gets the most screen time
- Geometric/visual intuition → if the paper has math, show what it MEANS
- Before/after comparison → the most compelling evidence
## Pre-code workflow
### Gate 1: Narration script
Write the full narration before any code. Every sentence maps to a visual beat. If you can't write the narration, you don't understand the paper well enough to animate it.
```markdown
## Hook (30s)
"What if I told you that a model with 7 billion parameters can outperform
one with 70 billion — if you train it on the right data?"
## Problem (60s)
"The standard approach is to scale up. More parameters, more compute.
[VISUAL: bar chart showing model sizes growing exponentially]
But Chinchilla showed us that most models are undertrained..."
```
### Gate 2: Scene list
After the narration, break it into scenes. Each scene is one Manim class.
```markdown
Scene 1: Hook — surprising stat with animated counter
Scene 2: Problem — model size bar chart growing
Scene 3: Key insight — training data vs parameters, animated 2D plot
Scene 4: Method — pipeline diagram building left to right
Scene 5: Results — before/after comparison with animated bars
Scene 6: Closing — implications text
```
### Gate 3: Style constants
Before coding scenes, define the visual language:
```python
# style.py — import in every scene file
BG = "#0D1117"
PRIMARY = "#58C4DD"
SECONDARY = "#83C167"
ACCENT = "#FFFF00"
HIGHLIGHT = "#FF6B6B"
MONO = "Menlo"
# Color meanings for THIS paper
MODEL_COLOR = PRIMARY # "the model"
DATA_COLOR = SECONDARY # "training data"
BASELINE_COLOR = HIGHLIGHT # "previous approach"
RESULT_COLOR = ACCENT # "our result"
```
## First-principles equation explanation
When the paper has a key equation, don't just show it — build it from intuition:
### The "what would you do?" pattern
1. Pose the problem in plain language
2. Ask what the simplest solution would be
3. Show why it doesn't work (animate the failure)
4. Introduce the paper's solution as the fix
5. THEN show the equation — it now feels earned
```python
# Scene: Why we need attention (for a Transformer paper)
# Step 1: "How do we let each word look at every other word?"
# Step 2: Show naive approach (fully connected = O(n²) everything)
# Step 3: Show it breaks (information overload, no selectivity)
# Step 4: "What if each word could CHOOSE which words to attend to?"
# Step 5: Show attention equation — Q, K, V now mean something
```
### Equation reveal strategy
```python
# Show equation dimmed first (full destination)
eq = MathTex(r"Attention(Q,K,V) = softmax\left(\frac{QK^T}{\sqrt{d_k}}\right)V")
eq.set_opacity(0.15)
self.play(FadeIn(eq))
# Highlight Q, K, V one at a time with color + label
for part, color, label_text in [
(r"Q", PRIMARY, "Query: what am I looking for?"),
(r"K", SECONDARY, "Key: what do I contain?"),
(r"V", ACCENT, "Value: what do I output?"),
]:
eq.set_color_by_tex(part, color)
label = Text(label_text, font_size=18, color=color, font=MONO)
# position label, animate it, wait, then dim it
```
## Building architecture diagrams
### The progressive build pattern
Don't show the full architecture at once. Build it:
1. First component appears alone → explain
2. Arrow grows → "this feeds into..."
3. Second component appears → explain
4. Repeat until complete
```python
# Component factory
def make_box(label, color, width=2.0, height=0.8):
box = RoundedRectangle(corner_radius=0.1, width=width, height=height,
color=color, fill_opacity=0.1, stroke_width=1.5)
text = Text(label, font_size=18, font=MONO, color=color).move_to(box)
return Group(box, text)
encoder = make_box("Encoder", PRIMARY)
decoder = make_box("Decoder", SECONDARY).next_to(encoder, RIGHT, buff=1.5)
arrow = Arrow(encoder.get_right(), decoder.get_left(), color=DIM, stroke_width=1.5)
self.play(FadeIn(encoder))
self.wait(1) # explain encoder
self.play(GrowArrow(arrow))
self.play(FadeIn(decoder))
self.wait(1) # explain decoder
```
### Data flow animation
After building the diagram, show data moving through it:
```python
# Dot traveling along the pipeline
data_dot = Dot(color=ACCENT, radius=0.1).move_to(encoder)
self.play(FadeIn(data_dot))
self.play(MoveAlongPath(data_dot, arrow), run_time=1)
self.play(data_dot.animate.move_to(decoder), run_time=0.5)
self.play(Flash(data_dot.get_center(), color=ACCENT), run_time=0.3)
```
## Animating results
### Bar chart comparison (most common)
```python
# Before/after bars
before_data = [45, 52, 38, 61]
after_data = [78, 85, 72, 91]
labels = ["Task A", "Task B", "Task C", "Task D"]
before_chart = BarChart(before_data, bar_names=labels,
y_range=[0, 100, 20], bar_colors=[HIGHLIGHT]*4).scale(0.6).shift(LEFT*3)
after_chart = BarChart(after_data, bar_names=labels,
y_range=[0, 100, 20], bar_colors=[SECONDARY]*4).scale(0.6).shift(RIGHT*3)
before_label = Text("Baseline", font_size=20, color=HIGHLIGHT, font=MONO)
after_label = Text("Ours", font_size=20, color=SECONDARY, font=MONO)
# Reveal baseline first, then ours (dramatic comparison)
self.play(Create(before_chart), FadeIn(before_label))
self.wait(1.5)
self.play(Create(after_chart), FadeIn(after_label))
self.wait(0.5)
# Highlight the improvement
improvement = Text("+35% avg", font_size=24, color=ACCENT, font=MONO)
self.play(FadeIn(improvement))
```
### Training curve (for ML papers)
```python
tracker = ValueTracker(0)
curve = always_redraw(lambda: axes.plot(
lambda x: 1 - 0.8 * np.exp(-x / 3),
x_range=[0, tracker.get_value()], color=PRIMARY
))
epoch_label = always_redraw(lambda: Text(
f"Epoch {int(tracker.get_value())}", font_size=18, font=MONO
).to_corner(UR))
self.add(curve, epoch_label)
self.play(tracker.animate.set_value(10), run_time=5, rate_func=linear)
```
## Domain-specific patterns
### ML papers
- Show data flow through the model (animated pipeline)
- Training curves with `ValueTracker`
- Attention heatmaps as colored grids
- Embedding space as 2D scatter (PCA/t-SNE visualization)
- Loss landscape as 3D surface with gradient descent dot
### Physics/math papers
- Use `LinearTransformationScene` for linear algebra
- Vector fields with `ArrowVectorField` / `StreamLines`
- Phase spaces with `NumberPlane` + trajectories
- Wave equations with time-parameterized plots
### Systems/architecture papers
- Pipeline diagrams built progressively
- `ShowPassingFlash` for data flow along arrows
- `ZoomedScene` for zooming into components
- Before/after latency/throughput comparisons
## Common mistakes
1. **Trying to cover the whole paper.** A 5-minute video can explain ONE core insight well. Covering everything means explaining nothing.
2. **Reading the abstract as narration.** Academic writing is designed for readers, not listeners. Rewrite in conversational language.
3. **Showing notation without meaning.** Never show a symbol without first showing what it represents visually.
4. **Skipping the motivation.** Jumping straight to "here's our method" without showing why the problem matters. The Problem section is what makes the viewer care.
5. **Identical pacing throughout.** The hook and key insight need the most visual energy. The method section can be faster. Evidence should land with impact (pause after showing the big number).
@@ -0,0 +1,190 @@
# Production Quality Checklist
Standards and checks for ensuring animation output is publication-ready.
## Pre-Code Checklist
Before writing any Manim code:
- [ ] Narration script written with visual beats marked
- [ ] Scene list with purpose, duration, and layout for each
- [ ] Color palette defined with meaning assignments (`PRIMARY` = main concept, etc.)
- [ ] `MONO = "Menlo"` set as the font constant
- [ ] Target resolution and aspect ratio decided
## Text Quality
### Overlap prevention
```python
# RULE: buff >= 0.5 for edge text
label.to_edge(DOWN, buff=0.5) # GOOD
label.to_edge(DOWN, buff=0.3) # BAD — may clip
# RULE: FadeOut previous before adding new at same position
self.play(ReplacementTransform(note1, note2)) # GOOD
self.play(Write(note2)) # BAD — overlaps note1
# RULE: Reduce font size for dense scenes
# When > 4 text elements visible, use font_size=20 not 28
```
### Width enforcement
Long text strings overflow the frame:
```python
# RULE: Set max width for any text that might be long
text = Text("This is a potentially long description", font_size=22, font=MONO)
if text.width > config.frame_width - 1.0:
text.set_width(config.frame_width - 1.0)
```
### Font consistency
```python
# RULE: Define MONO once, use everywhere
MONO = "Menlo"
# WRONG: mixing fonts
Text("Title", font="Helvetica")
Text("Label", font="Arial")
Text("Code", font="Courier")
# RIGHT: one font
Text("Title", font=MONO, weight=BOLD, font_size=48)
Text("Label", font=MONO, font_size=20)
Text("Code", font=MONO, font_size=18)
```
## Spatial Layout
### The coordinate budget
The visible frame is approximately 14.2 wide × 8.0 tall (default 16:9). With mandatory margins:
```
Usable area: x ∈ [-6.5, 6.5], y ∈ [-3.5, 3.5]
Top title zone: y ∈ [2.5, 3.5]
Bottom note zone: y ∈ [-3.5, -2.5]
Main content: y ∈ [-2.5, 2.5], x ∈ [-6.0, 6.0]
```
### Fill the frame
Empty scenes look unfinished. If the main content is small, add context:
- A dimmed grid/axes behind the content
- A title/subtitle at the top
- A source citation at the bottom
- Decorative geometry at low opacity
### Maximum simultaneous elements
**Hard limit: 6 actively visible elements.** Beyond that, the viewer can't track everything. If you need more:
- Dim old elements to opacity 0.3
- Remove elements that have served their purpose
- Split into two scenes
## Animation Quality
### Variety audit
Check that no two consecutive scenes use the exact same:
- Animation type (if Scene 3 uses Write for everything, Scene 4 should use FadeIn or Create)
- Color emphasis (rotate through palette colors)
- Layout (center, left-right, grid — alternate)
- Pacing (if Scene 2 was slow and deliberate, Scene 3 can be faster)
### Tempo curve
A good video follows a tempo curve:
```
Slow ──→ Medium ──→ FAST (climax) ──→ Slow (conclusion)
Scene 1: Slow (introduction, setup)
Scene 2: Medium (building understanding)
Scene 3: Medium-Fast (core content, lots of animation)
Scene 4: FAST (montage of applications/results)
Scene 5: Slow (conclusion, key takeaway)
```
### Transition quality
Between scenes:
- **Clean exit**: `self.play(FadeOut(Group(*self.mobjects)), run_time=0.5)`
- **Brief pause**: `self.wait(0.3)` after fadeout, before next scene's first animation
- **Never hard-cut**: always animate the transition
## Color Quality
### Dimming on dark backgrounds
Colors that look vibrant on white look muddy on dark backgrounds (#0D1117, #1C1C1C). Test your palette:
```python
# Colors that work well on dark backgrounds:
# Bright and saturated: #58C4DD, #83C167, #FFFF00, #FF6B6B
# Colors that DON'T work: #666666 (invisible), #2244AA (too dark)
# RULE: Structural elements (axes, grids) at opacity 0.15
# Context elements at 0.3-0.4
# Primary elements at 1.0
```
### Color meaning consistency
Once a color is assigned a meaning, it keeps that meaning for the entire video:
```python
# If PRIMARY (#58C4DD) means "the model" in Scene 1,
# it means "the model" in every scene.
# Never reuse PRIMARY for a different concept later.
```
## Data Visualization Quality
### Minimum requirements for charts
- Axis labels on every axis
- Y-axis range starts at 0 (or has a clear break indicator)
- Bar/line colors match the legend
- Numbers on notable data points (at least the maximum and the comparison point)
### Animated counters
When showing a number changing:
```python
# GOOD: DecimalNumber with smooth animation
counter = DecimalNumber(0, font_size=48, num_decimal_places=0, font="Menlo")
self.play(counter.animate.set_value(1000), run_time=3, rate_func=rush_from)
# BAD: Text that jumps between values
```
## Pre-Render Checklist
Before running `manim -qh`:
- [ ] All scenes render without errors at `-ql`
- [ ] Preview stills at `-qm` for text-heavy scenes (check kerning)
- [ ] Background color set in every scene (`self.camera.background_color = BG`)
- [ ] `add_subcaption()` or `subcaption=` on every significant animation
- [ ] No text smaller than font_size=18
- [ ] No text using proportional fonts (use monospace)
- [ ] buff >= 0.5 on all `.to_edge()` calls
- [ ] Clean exit (FadeOut all) at end of every scene
- [ ] `self.wait()` after every reveal
- [ ] Color constants used (no hardcoded hex strings in scene code)
- [ ] All scenes use the same quality flag (don't mix `-ql` and `-qh`)
## Post-Render Checklist
After stitching the final video:
- [ ] Watch the complete video at 1x speed — does it feel rushed anywhere?
- [ ] Is there a moment where two things animate simultaneously and it's confusing?
- [ ] Does every text label have enough time to be read?
- [ ] Are transitions between scenes smooth (no black frames, no jarring cuts)?
- [ ] Is the audio in sync with the visuals (if using voiceover)?
- [ ] Is the Gibbs-like "first impression" good? The first 5 seconds determine if someone keeps watching
@@ -0,0 +1,260 @@
# Updaters and Value Trackers
## The problem updaters solve
Normal animations are discrete: `self.play()` goes from state A to state B. But what if you need continuous relationships — a label that always hovers above a moving dot, or a line that always connects two points?
Without updaters, you'd manually reposition every dependent object before every `self.play()`. Five animations that move a dot means five manual repositioning calls for the label. Miss one and it freezes in the wrong spot.
Updaters let you declare a relationship ONCE. Manim calls the updater function EVERY FRAME (15-60 fps depending on quality) to enforce that relationship, no matter what else is happening.
## ValueTracker: an invisible steering wheel
A ValueTracker is an invisible Mobject that holds a single float. It never appears on screen. It exists so you can ANIMATE it while other objects REACT to its value.
Think of it as a slider: drag the slider from 0 to 5, and every object wired to it responds in real time.
```python
tracker = ValueTracker(0) # invisible, stores 0.0
tracker.get_value() # read: 0.0
tracker.set_value(5) # write: jump to 5.0 instantly
tracker.animate.set_value(5) # animate: smoothly interpolate to 5.0
```
### The three-step pattern
Every ValueTracker usage follows this:
1. **Create the tracker** (the invisible slider)
2. **Create visible objects that READ the tracker** via updaters
3. **Animate the tracker** — all dependents update automatically
```python
# Step 1: Create tracker
x_tracker = ValueTracker(1)
# Step 2: Create dependent objects
dot = always_redraw(lambda: Dot(axes.c2p(x_tracker.get_value(), 0), color=YELLOW))
v_line = always_redraw(lambda: axes.get_vertical_line(
axes.c2p(x_tracker.get_value(), func(x_tracker.get_value())), color=BLUE
))
label = always_redraw(lambda: DecimalNumber(x_tracker.get_value(), font_size=24)
.next_to(dot, UP))
self.add(dot, v_line, label)
# Step 3: Animate the tracker — everything follows
self.play(x_tracker.animate.set_value(5), run_time=3)
```
## Types of updaters
### Lambda updater (most common)
Runs a function every frame, passing the mobject itself:
```python
# Label always stays above the dot
label.add_updater(lambda m: m.next_to(dot, UP, buff=0.2))
# Line always connects two points
line.add_updater(lambda m: m.put_start_and_end_on(
point_a.get_center(), point_b.get_center()
))
```
### Time-based updater (with dt)
The second argument `dt` is the time since the last frame (~0.017s at 60fps):
```python
# Continuous rotation
square.add_updater(lambda m, dt: m.rotate(0.5 * dt))
# Continuous rightward drift
dot.add_updater(lambda m, dt: m.shift(RIGHT * 0.3 * dt))
# Oscillation
dot.add_updater(lambda m, dt: m.move_to(
axes.c2p(m.get_center()[0], np.sin(self.time))
))
```
Use `dt` updaters for physics simulations, continuous motion, and time-dependent effects.
### always_redraw: full rebuild every frame
Creates a new mobject from scratch each frame. More expensive than `add_updater` but handles cases where the mobject's structure changes (not just position/color):
```python
# Brace that follows a resizing square
brace = always_redraw(Brace, square, UP)
# Area under curve that updates as function changes
area = always_redraw(lambda: axes.get_area(
graph, x_range=[0, x_tracker.get_value()], color=BLUE, opacity=0.3
))
# Label that reconstructs its text
counter = always_redraw(lambda: Text(
f"n = {int(x_tracker.get_value())}", font_size=24, font="Menlo"
).to_corner(UR))
```
**When to use which:**
- `add_updater` — position, color, opacity changes (cheap, preferred)
- `always_redraw` — when the shape/structure itself changes (expensive, use sparingly)
## DecimalNumber: showing live values
```python
# Counter that tracks a ValueTracker
tracker = ValueTracker(0)
number = DecimalNumber(0, font_size=48, num_decimal_places=1, color=PRIMARY)
number.add_updater(lambda m: m.set_value(tracker.get_value()))
number.add_updater(lambda m: m.next_to(dot, RIGHT, buff=0.3))
self.add(number)
self.play(tracker.animate.set_value(100), run_time=3)
```
### Variable: the labeled version
```python
var = Variable(0, Text("x", font_size=24, font="Menlo"), num_decimal_places=2)
self.add(var)
self.play(var.tracker.animate.set_value(PI), run_time=2)
# Displays: x = 3.14
```
## Removing updaters
```python
# Remove all updaters
mobject.clear_updaters()
# Suspend temporarily (during an animation that would fight the updater)
mobject.suspend_updating()
self.play(mobject.animate.shift(RIGHT))
mobject.resume_updating()
# Remove specific updater (if you stored a reference)
def my_updater(m):
m.next_to(dot, UP)
label.add_updater(my_updater)
# ... later ...
label.remove_updater(my_updater)
```
## Animation-based updaters
### UpdateFromFunc / UpdateFromAlphaFunc
These are ANIMATIONS (passed to `self.play`), not persistent updaters:
```python
# Call a function on each frame of the animation
self.play(UpdateFromFunc(mobject, lambda m: m.next_to(moving_target, UP)), run_time=3)
# With alpha (0 to 1) — useful for custom interpolation
self.play(UpdateFromAlphaFunc(circle, lambda m, a: m.set_fill(opacity=a)), run_time=2)
```
### turn_animation_into_updater
Convert a one-shot animation into a continuous updater:
```python
from manim import turn_animation_into_updater
# This would normally play once — now it loops forever
turn_animation_into_updater(Rotating(gear, rate=PI/4))
self.add(gear)
self.wait(5) # gear rotates for 5 seconds
```
## Practical patterns
### Pattern 1: Dot tracing a function
```python
tracker = ValueTracker(0)
graph = axes.plot(np.sin, x_range=[0, 2*PI], color=PRIMARY)
dot = always_redraw(lambda: Dot(
axes.c2p(tracker.get_value(), np.sin(tracker.get_value())),
color=YELLOW
))
tangent = always_redraw(lambda: axes.get_secant_slope_group(
x=tracker.get_value(), graph=graph, dx=0.01,
secant_line_color=HIGHLIGHT, secant_line_length=3
))
self.add(graph, dot, tangent)
self.play(tracker.animate.set_value(2*PI), run_time=6, rate_func=linear)
```
### Pattern 2: Live area under curve
```python
tracker = ValueTracker(0.5)
area = always_redraw(lambda: axes.get_area(
graph, x_range=[0, tracker.get_value()],
color=PRIMARY, opacity=0.3
))
area_label = always_redraw(lambda: DecimalNumber(
# Numerical integration
sum(func(x) * 0.01 for x in np.arange(0, tracker.get_value(), 0.01)),
font_size=24
).next_to(axes, RIGHT))
self.add(area, area_label)
self.play(tracker.animate.set_value(4), run_time=5)
```
### Pattern 3: Connected diagram
```python
# Nodes that can be moved, with edges that auto-follow
node_a = Dot(LEFT * 2, color=PRIMARY)
node_b = Dot(RIGHT * 2, color=SECONDARY)
edge = Line().add_updater(lambda m: m.put_start_and_end_on(
node_a.get_center(), node_b.get_center()
))
label = Text("edge", font_size=18, font="Menlo").add_updater(
lambda m: m.move_to(edge.get_center() + UP * 0.3)
)
self.add(node_a, node_b, edge, label)
self.play(node_a.animate.shift(UP * 2), run_time=2)
self.play(node_b.animate.shift(DOWN + RIGHT), run_time=2)
# Edge and label follow automatically
```
### Pattern 4: Parameter exploration
```python
# Explore how a parameter changes a curve
a_tracker = ValueTracker(1)
curve = always_redraw(lambda: axes.plot(
lambda x: a_tracker.get_value() * np.sin(x),
x_range=[0, 2*PI], color=PRIMARY
))
param_label = always_redraw(lambda: Text(
f"a = {a_tracker.get_value():.1f}", font_size=24, font="Menlo"
).to_corner(UR))
self.add(curve, param_label)
self.play(a_tracker.animate.set_value(3), run_time=3)
self.play(a_tracker.animate.set_value(0.5), run_time=2)
self.play(a_tracker.animate.set_value(1), run_time=1)
```
## Common mistakes
1. **Updater fights animation:** If a mobject has an updater that sets its position, and you try to animate it elsewhere, the updater wins every frame. Suspend updating first.
2. **always_redraw for simple moves:** If you only need to reposition, use `add_updater`. `always_redraw` reconstructs the entire mobject every frame — expensive and unnecessary for position tracking.
3. **Forgetting to add to scene:** Updaters only run on mobjects that are in the scene. `always_redraw` creates the mobject but you still need `self.add()`.
4. **Updater creates new mobjects without cleanup:** If your updater creates Text objects every frame, they accumulate. Use `always_redraw` (which handles cleanup) or update properties in-place.
+64
View File
@@ -0,0 +1,64 @@
# p5.js Skill
Production pipeline for interactive and generative visual art using [p5.js](https://p5js.org/).
## What it does
Creates browser-based visual art from text prompts. The agent handles the full pipeline: creative concept, code generation, preview, export, and iterative refinement. Output is a single self-contained HTML file that runs in any browser — no build step, no server, no dependencies beyond a CDN script tag.
The output is real interactive art. Not tutorial exercises. Generative systems, particle physics, noise fields, shader effects, kinetic typography — composed with intentional color palettes, layered composition, and visual hierarchy.
## Modes
| Mode | Input | Output |
|------|-------|--------|
| **Generative art** | Seed / parameters | Procedural visual composition |
| **Data visualization** | Dataset / API | Interactive charts, custom data displays |
| **Interactive experience** | None (user drives) | Mouse/keyboard/touch-driven sketch |
| **Animation / motion graphics** | Timeline / storyboard | Timed sequences, kinetic typography |
| **3D scene** | Concept description | WebGL geometry, lighting, shaders |
| **Image processing** | Image file(s) | Pixel manipulation, filters, pointillism |
| **Audio-reactive** | Audio file / mic | Sound-driven generative visuals |
## Export Formats
| Format | Method |
|--------|--------|
| **HTML** | Self-contained file, opens in any browser |
| **PNG** | `saveCanvas()` — press 's' to capture |
| **GIF** | `saveGif()` — press 'g' to capture |
| **MP4** | Frame sequence + ffmpeg via `scripts/render.sh` |
| **SVG** | p5.js-svg renderer for vector output |
## Prerequisites
A modern browser. That's it for basic use.
For headless export: Node.js, Puppeteer, ffmpeg.
```bash
bash skills/creative/p5js/scripts/setup.sh
```
## File Structure
```
├── SKILL.md # Modes, workflow, creative direction, critical notes
├── README.md # This file
├── references/
│ ├── core-api.md # Canvas, draw loop, transforms, offscreen buffers, math
│ ├── shapes-and-geometry.md # Primitives, vertices, curves, vectors, SDFs, clipping
│ ├── visual-effects.md # Noise, flow fields, particles, pixels, textures, feedback
│ ├── animation.md # Easing, springs, state machines, timelines, transitions
│ ├── typography.md # Fonts, textToPoints, kinetic text, text masks
│ ├── color-systems.md # HSB/RGB, palettes, gradients, blend modes, curated colors
│ ├── webgl-and-3d.md # 3D primitives, camera, lighting, shaders, framebuffers
│ ├── interaction.md # Mouse, keyboard, touch, DOM, audio, scroll
│ ├── export-pipeline.md # PNG, GIF, MP4, SVG, headless, tiling, batch export
│ └── troubleshooting.md # Performance, common mistakes, browser issues, debugging
└── scripts/
├── setup.sh # Dependency verification
├── serve.sh # Local dev server (for loading local assets)
├── render.sh # Headless render pipeline (HTML → frames → MP4)
└── export-frames.js # Puppeteer frame capture (Node.js)
```
+513
View File
@@ -0,0 +1,513 @@
---
name: p5js
description: "Production pipeline for interactive and generative visual art using p5.js. Creates browser-based sketches, generative art, data visualizations, interactive experiences, 3D scenes, audio-reactive visuals, and motion graphics — exported as HTML, PNG, GIF, MP4, or SVG. Covers: 2D/3D rendering, noise and particle systems, flow fields, shaders (GLSL), pixel manipulation, kinetic typography, WebGL scenes, audio analysis, mouse/keyboard interaction, and headless high-res export. Use when users request: p5.js sketches, creative coding, generative art, interactive visualizations, canvas animations, browser-based visual art, data viz, shader effects, or any p5.js project."
version: 1.0.0
metadata:
hermes:
tags: [creative-coding, generative-art, p5js, canvas, interactive, visualization, webgl, shaders, animation]
related_skills: [ascii-video, manim-video, excalidraw]
---
# p5.js Production Pipeline
## Creative Standard
This is visual art rendered in the browser. The canvas is the medium; the algorithm is the brush.
**Before writing a single line of code**, articulate the creative concept. What does this piece communicate? What makes the viewer stop scrolling? What separates this from a code tutorial example? The user's prompt is a starting point — interpret it with creative ambition.
**First-render excellence is non-negotiable.** The output must be visually striking on first load. If it looks like a p5.js tutorial exercise, a default configuration, or "AI-generated creative coding," it is wrong. Rethink before shipping.
**Go beyond the reference vocabulary.** The noise functions, particle systems, color palettes, and shader effects in the references are a starting vocabulary. For every project, combine, layer, and invent. The catalog is a palette of paints — you write the painting.
**Be proactively creative.** If the user asks for "a particle system," deliver a particle system with emergent flocking behavior, trailing ghost echoes, palette-shifted depth fog, and a background noise field that breathes. Include at least one visual detail the user didn't ask for but will appreciate.
**Dense, layered, considered.** Every frame should reward viewing. Never flat white backgrounds. Always compositional hierarchy. Always intentional color. Always micro-detail that only appears on close inspection.
**Cohesive aesthetic over feature count.** All elements must serve a unified visual language — shared color temperature, consistent stroke weight vocabulary, harmonious motion speeds. A sketch with ten unrelated effects is worse than one with three that belong together.
## Modes
| Mode | Input | Output | Reference |
|------|-------|--------|-----------|
| **Generative art** | Seed / parameters | Procedural visual composition (still or animated) | `references/visual-effects.md` |
| **Data visualization** | Dataset / API | Interactive charts, graphs, custom data displays | `references/interaction.md` |
| **Interactive experience** | None (user drives) | Mouse/keyboard/touch-driven sketch | `references/interaction.md` |
| **Animation / motion graphics** | Timeline / storyboard | Timed sequences, kinetic typography, transitions | `references/animation.md` |
| **3D scene** | Concept description | WebGL geometry, lighting, camera, materials | `references/webgl-and-3d.md` |
| **Image processing** | Image file(s) | Pixel manipulation, filters, mosaic, pointillism | `references/visual-effects.md` § Pixel Manipulation |
| **Audio-reactive** | Audio file / mic | Sound-driven generative visuals | `references/interaction.md` § Audio Input |
## Stack
Single self-contained HTML file per project. No build step required.
| Layer | Tool | Purpose |
|-------|------|---------|
| Core | p5.js 1.11.3 (CDN) | Canvas rendering, math, transforms, event handling |
| 3D | p5.js WebGL mode | 3D geometry, camera, lighting, GLSL shaders |
| Audio | p5.sound.js (CDN) | FFT analysis, amplitude, mic input, oscillators |
| Export | Built-in `saveCanvas()` / `saveGif()` / `saveFrames()` | PNG, GIF, frame sequence output |
| Capture | CCapture.js (optional) | Deterministic framerate video capture (WebM, GIF) |
| Headless | Puppeteer + Node.js (optional) | Automated high-res rendering, MP4 via ffmpeg |
| SVG | p5.js-svg 1.6.0 (optional) | Vector output for print — requires p5.js 1.x |
| Natural media | p5.brush (optional) | Watercolor, charcoal, pen — requires p5.js 2.x + WEBGL |
| Texture | p5.grain (optional) | Film grain, texture overlays |
| Fonts | Google Fonts / `loadFont()` | Custom typography via OTF/TTF/WOFF2 |
### Version Note
**p5.js 1.x** (1.11.3) is the default — stable, well-documented, broadest library compatibility. Use this unless a project requires 2.x features.
**p5.js 2.x** (2.2+) adds: `async setup()` replacing `preload()`, OKLCH/OKLAB color modes, `splineVertex()`, shader `.modify()` API, variable fonts, `textToContours()`, pointer events. Required for p5.brush. See `references/core-api.md` § p5.js 2.0.
## Pipeline
Every project follows the same 6-stage path:
```
CONCEPT → DESIGN → CODE → PREVIEW → EXPORT → VERIFY
```
1. **CONCEPT** — Articulate the creative vision: mood, color world, motion vocabulary, what makes this unique
2. **DESIGN** — Choose mode, canvas size, interaction model, color system, export format. Map concept to technical decisions
3. **CODE** — Write single HTML file with inline p5.js. Structure: globals → `preload()``setup()``draw()` → helpers → classes → event handlers
4. **PREVIEW** — Open in browser, verify visual quality. Test at target resolution. Check performance
5. **EXPORT** — Capture output: `saveCanvas()` for PNG, `saveGif()` for GIF, `saveFrames()` + ffmpeg for MP4, Puppeteer for headless batch
6. **VERIFY** — Does the output match the concept? Is it visually striking at the intended display size? Would you frame it?
## Creative Direction
### Aesthetic Dimensions
| Dimension | Options | Reference |
|-----------|---------|-----------|
| **Color system** | HSB/HSL, RGB, named palettes, procedural harmony, gradient interpolation | `references/color-systems.md` |
| **Noise vocabulary** | Perlin noise, simplex, fractal (octaved), domain warping, curl noise | `references/visual-effects.md` § Noise |
| **Particle systems** | Physics-based, flocking, trail-drawing, attractor-driven, flow-field following | `references/visual-effects.md` § Particles |
| **Shape language** | Geometric primitives, custom vertices, bezier curves, SVG paths | `references/shapes-and-geometry.md` |
| **Motion style** | Eased, spring-based, noise-driven, physics sim, lerped, stepped | `references/animation.md` |
| **Typography** | System fonts, loaded OTF, `textToPoints()` particle text, kinetic | `references/typography.md` |
| **Shader effects** | GLSL fragment/vertex, filter shaders, post-processing, feedback loops | `references/webgl-and-3d.md` § Shaders |
| **Composition** | Grid, radial, golden ratio, rule of thirds, organic scatter, tiled | `references/core-api.md` § Composition |
| **Interaction model** | Mouse follow, click spawn, drag, keyboard state, scroll-driven, mic input | `references/interaction.md` |
| **Blend modes** | `BLEND`, `ADD`, `MULTIPLY`, `SCREEN`, `DIFFERENCE`, `EXCLUSION`, `OVERLAY` | `references/color-systems.md` § Blend Modes |
| **Layering** | `createGraphics()` offscreen buffers, alpha compositing, masking | `references/core-api.md` § Offscreen Buffers |
| **Texture** | Perlin surface, stippling, hatching, halftone, pixel sorting | `references/visual-effects.md` § Texture Generation |
### Per-Project Variation Rules
Never use default configurations. For every project:
- **Custom color palette** — never raw `fill(255, 0, 0)`. Always a designed palette with 3-7 colors
- **Custom stroke weight vocabulary** — thin accents (0.5), medium structure (1-2), bold emphasis (3-5)
- **Background treatment** — never plain `background(0)` or `background(255)`. Always textured, gradient, or layered
- **Motion variety** — different speeds for different elements. Primary at 1x, secondary at 0.3x, ambient at 0.1x
- **At least one invented element** — a custom particle behavior, a novel noise application, a unique interaction response
### Project-Specific Invention
For every project, invent at least one of:
- A custom color palette matching the mood (not a preset)
- A novel noise field combination (e.g., curl noise + domain warp + feedback)
- A unique particle behavior (custom forces, custom trails, custom spawning)
- An interaction mechanic the user didn't request but that elevates the piece
- A compositional technique that creates visual hierarchy
### Parameter Design Philosophy
Parameters should emerge from the algorithm, not from a generic menu. Ask: "What properties of *this* system should be tunable?"
**Good parameters** expose the algorithm's character:
- **Quantities** — how many particles, branches, cells (controls density)
- **Scales** — noise frequency, element size, spacing (controls texture)
- **Rates** — speed, growth rate, decay (controls energy)
- **Thresholds** — when does behavior change? (controls drama)
- **Ratios** — proportions, balance between forces (controls harmony)
**Bad parameters** are generic controls unrelated to the algorithm:
- "color1", "color2", "size" — meaningless without context
- Toggle switches for unrelated effects
- Parameters that only change cosmetics, not behavior
Every parameter should change how the algorithm *thinks*, not just how it *looks*. A "turbulence" parameter that changes noise octaves is good. A "particle size" slider that only changes `ellipse()` radius is shallow.
## Workflow
### Step 1: Creative Vision
Before any code, articulate:
- **Mood / atmosphere**: What should the viewer feel? Contemplative? Energized? Unsettled? Playful?
- **Visual story**: What happens over time (or on interaction)? Build? Decay? Transform? Oscillate?
- **Color world**: Warm/cool? Monochrome? Complementary? What's the dominant hue? The accent?
- **Shape language**: Organic curves? Sharp geometry? Dots? Lines? Mixed?
- **Motion vocabulary**: Slow drift? Explosive burst? Breathing pulse? Mechanical precision?
- **What makes THIS different**: What is the one thing that makes this sketch unique?
Map the user's prompt to aesthetic choices. "Relaxing generative background" demands different everything from "glitch data visualization."
### Step 2: Technical Design
- **Mode** — which of the 7 modes from the table above
- **Canvas size** — landscape 1920x1080, portrait 1080x1920, square 1080x1080, or responsive `windowWidth/windowHeight`
- **Renderer**`P2D` (default) or `WEBGL` (for 3D, shaders, advanced blend modes)
- **Frame rate** — 60fps (interactive), 30fps (ambient animation), or `noLoop()` (static generative)
- **Export target** — browser display, PNG still, GIF loop, MP4 video, SVG vector
- **Interaction model** — passive (no input), mouse-driven, keyboard-driven, audio-reactive, scroll-driven
- **Viewer UI** — for interactive generative art, start from `templates/viewer.html` which provides seed navigation, parameter sliders, and download. For simple sketches or video export, use bare HTML
### Step 3: Code the Sketch
For **interactive generative art** (seed exploration, parameter tuning): start from `templates/viewer.html`. Read the template first, keep the fixed sections (seed nav, actions), replace the algorithm and parameter controls. This gives the user seed prev/next/random/jump, parameter sliders with live update, and PNG download — all wired up.
For **animations, video export, or simple sketches**: use bare HTML:
Single HTML file. Structure:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Project Name</title>
<script>p5.disableFriendlyErrors = true;</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.11.3/p5.min.js"></script>
<!-- <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.11.3/addons/p5.sound.min.js"></script> -->
<!-- <script src="https://unpkg.com/p5.js-svg@1.6.0"></script> --> <!-- SVG export -->
<!-- <script src="https://cdn.jsdelivr.net/npm/ccapture.js-npmfixed/build/CCapture.all.min.js"></script> --> <!-- video capture -->
<style>
html, body { margin: 0; padding: 0; overflow: hidden; }
canvas { display: block; }
</style>
</head>
<body>
<script>
// === Configuration ===
const CONFIG = {
seed: 42,
// ... project-specific params
};
// === Color Palette ===
const PALETTE = {
bg: '#0a0a0f',
primary: '#e8d5b7',
// ...
};
// === Global State ===
let particles = [];
// === Preload (fonts, images, data) ===
function preload() {
// font = loadFont('...');
}
// === Setup ===
function setup() {
createCanvas(1920, 1080);
randomSeed(CONFIG.seed);
noiseSeed(CONFIG.seed);
colorMode(HSB, 360, 100, 100, 100);
// Initialize state...
}
// === Draw Loop ===
function draw() {
// Render frame...
}
// === Helper Functions ===
// ...
// === Classes ===
class Particle {
// ...
}
// === Event Handlers ===
function mousePressed() { /* ... */ }
function keyPressed() { /* ... */ }
function windowResized() { resizeCanvas(windowWidth, windowHeight); }
</script>
</body>
</html>
```
Key implementation patterns:
- **Seeded randomness**: Always `randomSeed()` + `noiseSeed()` for reproducibility
- **Color mode**: Use `colorMode(HSB, 360, 100, 100, 100)` for intuitive color control
- **State separation**: CONFIG for parameters, PALETTE for colors, globals for mutable state
- **Class-based entities**: Particles, agents, shapes as classes with `update()` + `display()` methods
- **Offscreen buffers**: `createGraphics()` for layered composition, trails, masks
### Step 4: Preview & Iterate
- Open HTML file directly in browser — no server needed for basic sketches
- For `loadImage()`/`loadFont()` from local files: use `scripts/serve.sh` or `python3 -m http.server`
- Chrome DevTools Performance tab to verify 60fps
- Test at target export resolution, not just the window size
- Adjust parameters until the visual matches the concept from Step 1
### Step 5: Export
| Format | Method | Command |
|--------|--------|---------|
| **PNG** | `saveCanvas('output', 'png')` in `keyPressed()` | Press 's' to save |
| **High-res PNG** | Puppeteer headless capture | `node scripts/export-frames.js sketch.html --width 3840 --height 2160 --frames 1` |
| **GIF** | `saveGif('output', 5)` — captures N seconds | Press 'g' to save |
| **Frame sequence** | `saveFrames('frame', 'png', 10, 30)` — 10s at 30fps | Then `ffmpeg -i frame-%04d.png -c:v libx264 output.mp4` |
| **MP4** | Puppeteer frame capture + ffmpeg | `bash scripts/render.sh sketch.html output.mp4 --duration 30 --fps 30` |
| **SVG** | `createCanvas(w, h, SVG)` with p5.js-svg | `save('output.svg')` |
### Step 6: Quality Verification
- **Does it match the vision?** Compare output to the creative concept. If it looks generic, go back to Step 1
- **Resolution check**: Is it sharp at the target display size? No aliasing artifacts?
- **Performance check**: Does it hold 60fps in browser? (30fps minimum for animations)
- **Color check**: Do the colors work together? Test on both light and dark monitors
- **Edge cases**: What happens at canvas edges? On resize? After running for 10 minutes?
## Critical Implementation Notes
### Performance — Disable FES First
The Friendly Error System (FES) adds up to 10x overhead. Disable it in every production sketch:
```javascript
p5.disableFriendlyErrors = true; // BEFORE setup()
function setup() {
pixelDensity(1); // prevent 2x-4x overdraw on retina
createCanvas(1920, 1080);
}
```
In hot loops (particles, pixel ops), use `Math.*` instead of p5 wrappers — measurably faster:
```javascript
// In draw() or update() hot paths:
let a = Math.sin(t); // not sin(t)
let r = Math.sqrt(dx*dx+dy*dy); // not dist() — or better: skip sqrt, compare magSq
let v = Math.random(); // not random() — when seed not needed
let m = Math.min(a, b); // not min(a, b)
```
Never `console.log()` inside `draw()`. Never manipulate DOM in `draw()`. See `references/troubleshooting.md` § Performance.
### Seeded Randomness — Always
Every generative sketch must be reproducible. Same seed, same output.
```javascript
function setup() {
randomSeed(CONFIG.seed);
noiseSeed(CONFIG.seed);
// All random() and noise() calls now deterministic
}
```
Never use `Math.random()` for generative content — only for performance-critical non-visual code. Always `random()` for visual elements. If you need a random seed: `CONFIG.seed = floor(random(99999))`.
### Generative Art Platform Support (fxhash / Art Blocks)
For generative art platforms, replace p5's PRNG with the platform's deterministic random:
```javascript
// fxhash convention
const SEED = $fx.hash; // unique per mint
const rng = $fx.rand; // deterministic PRNG
$fx.features({ palette: 'warm', complexity: 'high' });
// In setup():
randomSeed(SEED); // for p5's noise()
noiseSeed(SEED);
// Replace random() with rng() for platform determinism
let x = rng() * width; // instead of random(width)
```
See `references/export-pipeline.md` § Platform Export.
### Color Mode — Use HSB
HSB (Hue, Saturation, Brightness) is dramatically easier to work with than RGB for generative art:
```javascript
colorMode(HSB, 360, 100, 100, 100);
// Now: fill(hue, sat, bri, alpha)
// Rotate hue: fill((baseHue + offset) % 360, 80, 90)
// Desaturate: fill(hue, sat * 0.3, bri)
// Darken: fill(hue, sat, bri * 0.5)
```
Never hardcode raw RGB values. Define a palette object, derive variations procedurally. See `references/color-systems.md`.
### Noise — Multi-Octave, Not Raw
Raw `noise(x, y)` looks like smooth blobs. Layer octaves for natural texture:
```javascript
function fbm(x, y, octaves = 4) {
let val = 0, amp = 1, freq = 1, sum = 0;
for (let i = 0; i < octaves; i++) {
val += noise(x * freq, y * freq) * amp;
sum += amp;
amp *= 0.5;
freq *= 2;
}
return val / sum;
}
```
For flowing organic forms, use **domain warping**: feed noise output back as noise input coordinates. See `references/visual-effects.md`.
### createGraphics() for Layers — Not Optional
Flat single-pass rendering looks flat. Use offscreen buffers for composition:
```javascript
let bgLayer, fgLayer, trailLayer;
function setup() {
createCanvas(1920, 1080);
bgLayer = createGraphics(width, height);
fgLayer = createGraphics(width, height);
trailLayer = createGraphics(width, height);
}
function draw() {
renderBackground(bgLayer);
renderTrails(trailLayer); // persistent, fading
renderForeground(fgLayer); // cleared each frame
image(bgLayer, 0, 0);
image(trailLayer, 0, 0);
image(fgLayer, 0, 0);
}
```
### Performance — Vectorize Where Possible
p5.js draw calls are expensive. For thousands of particles:
```javascript
// SLOW: individual shapes
for (let p of particles) {
ellipse(p.x, p.y, p.size);
}
// FAST: single shape with beginShape()
beginShape(POINTS);
for (let p of particles) {
vertex(p.x, p.y);
}
endShape();
// FASTEST: pixel buffer for massive counts
loadPixels();
for (let p of particles) {
let idx = 4 * (floor(p.y) * width + floor(p.x));
pixels[idx] = r; pixels[idx+1] = g; pixels[idx+2] = b; pixels[idx+3] = 255;
}
updatePixels();
```
See `references/troubleshooting.md` § Performance.
### Instance Mode for Multiple Sketches
Global mode pollutes `window`. For production, use instance mode:
```javascript
const sketch = (p) => {
p.setup = function() {
p.createCanvas(800, 800);
};
p.draw = function() {
p.background(0);
p.ellipse(p.mouseX, p.mouseY, 50);
};
};
new p5(sketch, 'canvas-container');
```
Required when embedding multiple sketches on one page or integrating with frameworks.
### WebGL Mode Gotchas
- `createCanvas(w, h, WEBGL)` — origin is center, not top-left
- Y-axis is inverted (positive Y goes up in WEBGL, down in P2D)
- `translate(-width/2, -height/2)` to get P2D-like coordinates
- `push()`/`pop()` around every transform — matrix stack overflows silently
- `texture()` before `rect()`/`plane()` — not after
- Custom shaders: `createShader(vert, frag)` — test on multiple browsers
### Export — Key Bindings Convention
Every sketch should include these in `keyPressed()`:
```javascript
function keyPressed() {
if (key === 's' || key === 'S') saveCanvas('output', 'png');
if (key === 'g' || key === 'G') saveGif('output', 5);
if (key === 'r' || key === 'R') { randomSeed(millis()); noiseSeed(millis()); }
if (key === ' ') CONFIG.paused = !CONFIG.paused;
}
```
### Headless Video Export — Use noLoop()
For headless rendering via Puppeteer, the sketch **must** use `noLoop()` in setup. Without it, p5's draw loop runs freely while screenshots are slow — the sketch races ahead and you get skipped/duplicate frames.
```javascript
function setup() {
createCanvas(1920, 1080);
pixelDensity(1);
noLoop(); // capture script controls frame advance
window._p5Ready = true; // signal readiness to capture script
}
```
The bundled `scripts/export-frames.js` detects `_p5Ready` and calls `redraw()` once per capture for exact 1:1 frame correspondence. See `references/export-pipeline.md` § Deterministic Capture.
For multi-scene videos, use the per-clip architecture: one HTML per scene, render independently, stitch with `ffmpeg -f concat`. See `references/export-pipeline.md` § Per-Clip Architecture.
### Agent Workflow
When building p5.js sketches:
1. **Write the HTML file** — single self-contained file, all code inline
2. **Open in browser**`open sketch.html` (macOS) or `xdg-open sketch.html` (Linux)
3. **Local assets** (fonts, images) require a server: `python3 -m http.server 8080` in the project directory, then open `http://localhost:8080/sketch.html`
4. **Export PNG/GIF** — add `keyPressed()` shortcuts as shown above, tell the user which key to press
5. **Headless export**`node scripts/export-frames.js sketch.html --frames 300` for automated frame capture (sketch must use `noLoop()` + `_p5Ready`)
6. **MP4 rendering**`bash scripts/render.sh sketch.html output.mp4 --duration 30`
7. **Iterative refinement** — edit the HTML file, user refreshes browser to see changes
8. **Load references on demand** — use `skill_view(name="p5js", file_path="references/...")` to load specific reference files as needed during implementation
## Performance Targets
| Metric | Target |
|--------|--------|
| Frame rate (interactive) | 60fps sustained |
| Frame rate (animated export) | 30fps minimum |
| Particle count (P2D shapes) | 5,000-10,000 at 60fps |
| Particle count (pixel buffer) | 50,000-100,000 at 60fps |
| Canvas resolution | Up to 3840x2160 (export), 1920x1080 (interactive) |
| File size (HTML) | < 100KB (excluding CDN libraries) |
| Load time | < 2s to first frame |
## References
| File | Contents |
|------|----------|
| `references/core-api.md` | Canvas setup, coordinate system, draw loop, `push()`/`pop()`, offscreen buffers, composition patterns, `pixelDensity()`, responsive design |
| `references/shapes-and-geometry.md` | 2D primitives, `beginShape()`/`endShape()`, Bezier/Catmull-Rom curves, `vertex()` systems, custom shapes, `p5.Vector`, signed distance fields, SVG path conversion |
| `references/visual-effects.md` | Noise (Perlin, fractal, domain warp, curl), flow fields, particle systems (physics, flocking, trails), pixel manipulation, texture generation (stipple, hatch, halftone), feedback loops, reaction-diffusion |
| `references/animation.md` | Frame-based animation, easing functions, `lerp()`/`map()`, spring physics, state machines, timeline sequencing, `millis()`-based timing, transition patterns |
| `references/typography.md` | `text()`, `loadFont()`, `textToPoints()`, kinetic typography, text masks, font metrics, responsive text sizing |
| `references/color-systems.md` | `colorMode()`, HSB/HSL/RGB, `lerpColor()`, `paletteLerp()`, procedural palettes, color harmony, `blendMode()`, gradient rendering, curated palette library |
| `references/webgl-and-3d.md` | WEBGL renderer, 3D primitives, camera, lighting, materials, custom geometry, GLSL shaders (`createShader()`, `createFilterShader()`), framebuffers, post-processing |
| `references/interaction.md` | Mouse events, keyboard state, touch input, DOM elements, `createSlider()`/`createButton()`, audio input (p5.sound FFT/amplitude), scroll-driven animation, responsive events |
| `references/export-pipeline.md` | `saveCanvas()`, `saveGif()`, `saveFrames()`, deterministic headless capture, ffmpeg frame-to-video, CCapture.js, SVG export, per-clip architecture, platform export (fxhash), video gotchas |
| `references/troubleshooting.md` | Performance profiling, per-pixel budgets, common mistakes, browser compatibility, WebGL debugging, font loading issues, pixel density traps, memory leaks, CORS |
| `templates/viewer.html` | Interactive viewer template: seed navigation (prev/next/random/jump), parameter sliders, download PNG, responsive canvas. Start from this for explorable generative art |
@@ -0,0 +1,439 @@
# Animation
## Frame-Based Animation
### The Draw Loop
```javascript
function draw() {
// Called ~60 times/sec by default
// frameCount — integer, starts at 1
// deltaTime — ms since last frame (use for framerate-independent motion)
// millis() — ms since sketch start
}
```
### Time-Based vs Frame-Based
```javascript
// Frame-based (speed varies with framerate)
x += speed;
// Time-based (consistent speed regardless of framerate)
x += speed * (deltaTime / 16.67); // normalized to 60fps
```
### Normalized Time
```javascript
// Progress from 0 to 1 over N seconds
let duration = 5000; // 5 seconds in ms
let t = constrain(millis() / duration, 0, 1);
// Looping progress (0 → 1 → 0 → 1...)
let period = 3000; // 3 second loop
let t = (millis() % period) / period;
// Ping-pong (0 → 1 → 0 → 1...)
let raw = (millis() % (period * 2)) / period;
let t = raw <= 1 ? raw : 2 - raw;
```
## Easing Functions
### Built-in Lerp
```javascript
// Linear interpolation — smooth but mechanical
let x = lerp(startX, endX, t);
// Map for non-0-1 ranges
let y = map(t, 0, 1, startY, endY);
```
### Common Easing Curves
```javascript
// Ease in (slow start)
function easeInQuad(t) { return t * t; }
function easeInCubic(t) { return t * t * t; }
function easeInExpo(t) { return t === 0 ? 0 : pow(2, 10 * (t - 1)); }
// Ease out (slow end)
function easeOutQuad(t) { return 1 - (1 - t) * (1 - t); }
function easeOutCubic(t) { return 1 - pow(1 - t, 3); }
function easeOutExpo(t) { return t === 1 ? 1 : 1 - pow(2, -10 * t); }
// Ease in-out (slow both ends)
function easeInOutCubic(t) {
return t < 0.5 ? 4 * t * t * t : 1 - pow(-2 * t + 2, 3) / 2;
}
function easeInOutQuint(t) {
return t < 0.5 ? 16 * t * t * t * t * t : 1 - pow(-2 * t + 2, 5) / 2;
}
// Elastic (spring overshoot)
function easeOutElastic(t) {
if (t === 0 || t === 1) return t;
return pow(2, -10 * t) * sin((t * 10 - 0.75) * (2 * PI / 3)) + 1;
}
// Bounce
function easeOutBounce(t) {
if (t < 1/2.75) return 7.5625 * t * t;
else if (t < 2/2.75) { t -= 1.5/2.75; return 7.5625 * t * t + 0.75; }
else if (t < 2.5/2.75) { t -= 2.25/2.75; return 7.5625 * t * t + 0.9375; }
else { t -= 2.625/2.75; return 7.5625 * t * t + 0.984375; }
}
// Smooth step (Hermite interpolation — great default)
function smoothstep(t) { return t * t * (3 - 2 * t); }
// Smoother step (Ken Perlin)
function smootherstep(t) { return t * t * t * (t * (t * 6 - 15) + 10); }
```
### Applying Easing
```javascript
// Animate from startVal to endVal over duration ms
function easedValue(startVal, endVal, startTime, duration, easeFn) {
let t = constrain((millis() - startTime) / duration, 0, 1);
return lerp(startVal, endVal, easeFn(t));
}
// Usage
let x = easedValue(100, 700, animStartTime, 2000, easeOutCubic);
```
## Spring Physics
More natural than easing — responds to force, overshoots, settles.
```javascript
class Spring {
constructor(value, target, stiffness = 0.1, damping = 0.7) {
this.value = value;
this.target = target;
this.velocity = 0;
this.stiffness = stiffness;
this.damping = damping;
}
update() {
let force = (this.target - this.value) * this.stiffness;
this.velocity += force;
this.velocity *= this.damping;
this.value += this.velocity;
return this.value;
}
setTarget(t) { this.target = t; }
isSettled(threshold = 0.01) {
return abs(this.velocity) < threshold && abs(this.value - this.target) < threshold;
}
}
// Usage
let springX = new Spring(0, 0, 0.08, 0.85);
function draw() {
springX.setTarget(mouseX);
let x = springX.update();
ellipse(x, height/2, 50);
}
```
### 2D Spring
```javascript
class Spring2D {
constructor(x, y) {
this.pos = createVector(x, y);
this.target = createVector(x, y);
this.vel = createVector(0, 0);
this.stiffness = 0.08;
this.damping = 0.85;
}
update() {
let force = p5.Vector.sub(this.target, this.pos).mult(this.stiffness);
this.vel.add(force).mult(this.damping);
this.pos.add(this.vel);
return this.pos;
}
}
```
## State Machines
For complex multi-phase animations.
```javascript
const STATES = { IDLE: 0, ENTER: 1, ACTIVE: 2, EXIT: 3 };
let state = STATES.IDLE;
let stateStart = 0;
function setState(newState) {
state = newState;
stateStart = millis();
}
function stateTime() {
return millis() - stateStart;
}
function draw() {
switch (state) {
case STATES.IDLE:
// waiting...
break;
case STATES.ENTER:
let t = constrain(stateTime() / 1000, 0, 1);
let alpha = easeOutCubic(t) * 255;
// fade in...
if (t >= 1) setState(STATES.ACTIVE);
break;
case STATES.ACTIVE:
// main animation...
break;
case STATES.EXIT:
let t2 = constrain(stateTime() / 500, 0, 1);
// fade out...
if (t2 >= 1) setState(STATES.IDLE);
break;
}
}
```
## Timeline Sequencing
For timed multi-scene animations (motion graphics, title sequences).
```javascript
class Timeline {
constructor() {
this.events = [];
}
at(timeMs, duration, fn) {
this.events.push({ start: timeMs, end: timeMs + duration, fn });
return this;
}
update() {
let now = millis();
for (let e of this.events) {
if (now >= e.start && now < e.end) {
let t = (now - e.start) / (e.end - e.start);
e.fn(t);
}
}
}
}
// Usage
let timeline = new Timeline();
timeline
.at(0, 2000, (t) => {
// Scene 1: title fade in (0-2s)
let alpha = easeOutCubic(t) * 255;
fill(255, alpha);
textSize(48);
text("Hello", width/2, height/2);
})
.at(2000, 1000, (t) => {
// Scene 2: title fade out (2-3s)
let alpha = (1 - easeInCubic(t)) * 255;
fill(255, alpha);
textSize(48);
text("Hello", width/2, height/2);
})
.at(3000, 5000, (t) => {
// Scene 3: main content (3-8s)
renderMainContent(t);
});
function draw() {
background(0);
timeline.update();
}
```
## Noise-Driven Motion
More organic than deterministic animation.
```javascript
// Smooth wandering position
let x = map(noise(frameCount * 0.005, 0), 0, 1, 0, width);
let y = map(noise(0, frameCount * 0.005), 0, 1, 0, height);
// Noise-driven rotation
let angle = noise(frameCount * 0.01) * TWO_PI;
// Noise-driven scale (breathing effect)
let s = map(noise(frameCount * 0.02), 0, 1, 0.8, 1.2);
// Noise-driven color shift
let hue = map(noise(frameCount * 0.003), 0, 1, 0, 360);
```
## Transition Patterns
### Fade In/Out
```javascript
function fadeIn(t) { return constrain(t, 0, 1); }
function fadeOut(t) { return constrain(1 - t, 0, 1); }
```
### Slide
```javascript
function slideIn(t, direction = 'left') {
let et = easeOutCubic(t);
switch (direction) {
case 'left': return lerp(-width, 0, et);
case 'right': return lerp(width, 0, et);
case 'up': return lerp(-height, 0, et);
case 'down': return lerp(height, 0, et);
}
}
```
### Scale Reveal
```javascript
function scaleReveal(t) {
let et = easeOutElastic(constrain(t, 0, 1));
push();
translate(width/2, height/2);
scale(et);
translate(-width/2, -height/2);
// draw content...
pop();
}
```
### Staggered Entry
```javascript
// N elements appear one after another
let staggerDelay = 100; // ms between each
for (let i = 0; i < elements.length; i++) {
let itemStart = baseTime + i * staggerDelay;
let t = constrain((millis() - itemStart) / 500, 0, 1);
let alpha = easeOutCubic(t) * 255;
let yOffset = lerp(30, 0, easeOutCubic(t));
// draw element with alpha and yOffset
}
```
## Recording Deterministic Animations
For frame-perfect export, use frame count instead of millis():
```javascript
const TOTAL_FRAMES = 300; // 10 seconds at 30fps
const FPS = 30;
function draw() {
let t = frameCount / TOTAL_FRAMES; // 0 to 1 over full duration
if (t > 1) { noLoop(); return; }
// Use t for all animation timing — deterministic
renderFrame(t);
// Export
if (CONFIG.recording) {
saveCanvas('frame-' + nf(frameCount, 4), 'png');
}
}
```
## Scene Fade Envelopes (Video)
Every scene in a multi-scene video needs fade-in and fade-out. Hard cuts between visually different generative scenes are jarring.
```javascript
const SCENE_FRAMES = 150; // 5 seconds at 30fps
const FADE = 15; // half-second fade
function draw() {
let lf = frameCount - 1; // 0-indexed local frame
let t = lf / SCENE_FRAMES; // 0..1 normalized progress
// Fade envelope: ramp up at start, ramp down at end
let fade = 1;
if (lf < FADE) fade = lf / FADE;
if (lf > SCENE_FRAMES - FADE) fade = (SCENE_FRAMES - lf) / FADE;
fade = fade * fade * (3 - 2 * fade); // smoothstep for organic feel
// Apply fade to all visual output
// Option 1: multiply alpha values by fade
fill(r, g, b, alpha * fade);
// Option 2: tint entire composited image
tint(255, fade * 255);
image(sceneBuffer, 0, 0);
noTint();
// Option 3: multiply pixel brightness (for pixel-level scenes)
pixels[i] = r * fade;
}
```
## Animating Static Algorithms
Some generative algorithms produce a single static result (attractors, circle packing, Voronoi). In video, static content reads as frozen/broken. Techniques to add motion:
### Progressive Reveal
Expand a mask from center outward to reveal the precomputed result:
```javascript
let revealRadius = easeOutCubic(min(t * 1.5, 1)) * (width * 0.8);
// In the render loop, skip pixels beyond revealRadius from center
let dx = x - width/2, dy = y - height/2;
if (sqrt(dx*dx + dy*dy) > revealRadius) continue;
// Soft edge:
let edgeFade = constrain((revealRadius - dist) / 40, 0, 1);
```
### Parameter Sweep
Slowly change a parameter to show the algorithm evolving:
```javascript
// Attractor with drifting parameters
let a = -1.7 + sin(t * 0.5) * 0.2; // oscillate around base value
let b = 1.3 + cos(t * 0.3) * 0.15;
```
### Slow Camera Motion
Apply subtle zoom or rotation to the final image:
```javascript
push();
translate(width/2, height/2);
scale(1 + t * 0.05); // slow 5% zoom over scene duration
rotate(t * 0.1); // gentle rotation
translate(-width/2, -height/2);
image(precomputedResult, 0, 0);
pop();
```
### Overlay Dynamic Elements
Add particles, grain, or subtle noise on top of static content:
```javascript
// Static background
image(staticResult, 0, 0);
// Dynamic overlay
for (let p of ambientParticles) {
p.update();
p.display(); // slow-moving specks add life
}
```
@@ -0,0 +1,352 @@
# Color Systems
## Color Modes
### HSB (Recommended for Generative Art)
```javascript
colorMode(HSB, 360, 100, 100, 100);
// Hue: 0-360 (color wheel position)
// Saturation: 0-100 (gray to vivid)
// Brightness: 0-100 (black to full)
// Alpha: 0-100
fill(200, 80, 90); // blue, vivid, bright
fill(200, 80, 90, 50); // 50% transparent
```
HSB advantages:
- Rotate hue: `(baseHue + offset) % 360`
- Desaturate: reduce S
- Darken: reduce B
- Monochrome variations: fix H, vary S and B
- Complementary: `(hue + 180) % 360`
- Analogous: `hue +/- 30`
### HSL
```javascript
colorMode(HSL, 360, 100, 100, 100);
// Lightness 50 = pure color, 0 = black, 100 = white
// More intuitive for tints (L > 50) and shades (L < 50)
```
### RGB
```javascript
colorMode(RGB, 255, 255, 255, 255); // default
// Direct channel control, less intuitive for procedural palettes
```
## Color Objects
```javascript
let c = color(200, 80, 90); // create color object
fill(c);
// Extract components
let h = hue(c);
let s = saturation(c);
let b = brightness(c);
let r = red(c);
let g = green(c);
let bl = blue(c);
let a = alpha(c);
// Hex colors work everywhere
fill('#e8d5b7');
fill('#e8d5b7cc'); // with alpha
// Modify via setters
c.setAlpha(128);
c.setRed(200);
```
## Color Interpolation
### lerpColor
```javascript
let c1 = color(0, 80, 100); // red
let c2 = color(200, 80, 100); // blue
let mixed = lerpColor(c1, c2, 0.5); // midpoint blend
// Works in current colorMode
```
### paletteLerp (p5.js 1.11+)
Interpolate through multiple colors at once.
```javascript
let colors = [
color('#2E0854'),
color('#850E35'),
color('#EE6C4D'),
color('#F5E663')
];
let c = paletteLerp(colors, t); // t = 0..1, interpolates through all
```
### Manual Multi-Stop Gradient
```javascript
function multiLerp(colors, t) {
t = constrain(t, 0, 1);
let segment = t * (colors.length - 1);
let idx = floor(segment);
let frac = segment - idx;
idx = min(idx, colors.length - 2);
return lerpColor(colors[idx], colors[idx + 1], frac);
}
```
## Gradient Rendering
### Linear Gradient
```javascript
function linearGradient(x1, y1, x2, y2, c1, c2) {
let steps = dist(x1, y1, x2, y2);
for (let i = 0; i <= steps; i++) {
let t = i / steps;
let c = lerpColor(c1, c2, t);
stroke(c);
let x = lerp(x1, x2, t);
let y = lerp(y1, y2, t);
// Draw perpendicular line at each point
let dx = -(y2 - y1) / steps * 1000;
let dy = (x2 - x1) / steps * 1000;
line(x - dx, y - dy, x + dx, y + dy);
}
}
```
### Radial Gradient
```javascript
function radialGradient(cx, cy, r, innerColor, outerColor) {
noStroke();
for (let i = r; i > 0; i--) {
let t = 1 - i / r;
fill(lerpColor(innerColor, outerColor, t));
ellipse(cx, cy, i * 2);
}
}
```
### Noise-Based Gradient
```javascript
function noiseGradient(colors, noiseScale, time) {
loadPixels();
for (let y = 0; y < height; y++) {
for (let x = 0; x < width; x++) {
let n = noise(x * noiseScale, y * noiseScale, time);
let c = multiLerp(colors, n);
let idx = 4 * (y * width + x);
pixels[idx] = red(c);
pixels[idx+1] = green(c);
pixels[idx+2] = blue(c);
pixels[idx+3] = 255;
}
}
updatePixels();
}
```
## Procedural Palette Generation
### Complementary
```javascript
function complementary(baseHue) {
return [baseHue, (baseHue + 180) % 360];
}
```
### Analogous
```javascript
function analogous(baseHue, spread = 30) {
return [
(baseHue - spread + 360) % 360,
baseHue,
(baseHue + spread) % 360
];
}
```
### Triadic
```javascript
function triadic(baseHue) {
return [baseHue, (baseHue + 120) % 360, (baseHue + 240) % 360];
}
```
### Split Complementary
```javascript
function splitComplementary(baseHue) {
return [baseHue, (baseHue + 150) % 360, (baseHue + 210) % 360];
}
```
### Tetradic (Rectangle)
```javascript
function tetradic(baseHue) {
return [baseHue, (baseHue + 60) % 360, (baseHue + 180) % 360, (baseHue + 240) % 360];
}
```
### Monochromatic Variations
```javascript
function monoVariations(hue, count = 5) {
let colors = [];
for (let i = 0; i < count; i++) {
let s = map(i, 0, count - 1, 20, 90);
let b = map(i, 0, count - 1, 95, 40);
colors.push(color(hue, s, b));
}
return colors;
}
```
## Curated Palette Library
### Warm Palettes
```javascript
const SUNSET = ['#2E0854', '#850E35', '#EE6C4D', '#F5E663'];
const EMBER = ['#1a0000', '#4a0000', '#8b2500', '#cd5c00', '#ffd700'];
const PEACH = ['#fff5eb', '#ffdab9', '#ff9a76', '#ff6b6b', '#c94c4c'];
const COPPER = ['#1c1108', '#3d2b1f', '#7b4b2a', '#b87333', '#daa06d'];
```
### Cool Palettes
```javascript
const OCEAN = ['#0a0e27', '#1a1b4b', '#2a4a7f', '#3d7cb8', '#87ceeb'];
const ARCTIC = ['#0d1b2a', '#1b263b', '#415a77', '#778da9', '#e0e1dd'];
const FOREST = ['#0b1a0b', '#1a3a1a', '#2d5a2d', '#4a8c4a', '#90c990'];
const DEEP_SEA = ['#000814', '#001d3d', '#003566', '#006d77', '#83c5be'];
```
### Neutral Palettes
```javascript
const GRAPHITE = ['#1a1a1a', '#333333', '#555555', '#888888', '#cccccc'];
const CREAM = ['#f4f0e8', '#e8dcc8', '#c9b99a', '#a89070', '#7a6450'];
const SLATE = ['#1e293b', '#334155', '#475569', '#64748b', '#94a3b8'];
```
### Vivid Palettes
```javascript
const NEON = ['#ff00ff', '#00ffff', '#ff0080', '#80ff00', '#0080ff'];
const RAINBOW = ['#ff0000', '#ff8000', '#ffff00', '#00ff00', '#0000ff', '#8000ff'];
const VAPOR = ['#ff71ce', '#01cdfe', '#05ffa1', '#b967ff', '#fffb96'];
const CYBER = ['#0f0f0f', '#00ff41', '#ff0090', '#00d4ff', '#ffd000'];
```
### Earth Tones
```javascript
const TERRA = ['#2c1810', '#5c3a2a', '#8b6b4a', '#c4a672', '#e8d5b7'];
const MOSS = ['#1a1f16', '#3d4a2e', '#6b7c4f', '#9aab7a', '#c8d4a9'];
const CLAY = ['#3b2f2f', '#6b4c4c', '#9e7676', '#c9a0a0', '#e8caca'];
```
## Blend Modes
```javascript
blendMode(BLEND); // default — alpha compositing
blendMode(ADD); // additive — bright glow effects
blendMode(MULTIPLY); // darkening — shadows, texture overlay
blendMode(SCREEN); // lightening — soft glow
blendMode(OVERLAY); // contrast boost — high/low emphasis
blendMode(DIFFERENCE); // color subtraction — psychedelic
blendMode(EXCLUSION); // softer difference
blendMode(REPLACE); // overwrite (no alpha blending)
blendMode(REMOVE); // subtract alpha
blendMode(LIGHTEST); // keep brighter pixel
blendMode(DARKEST); // keep darker pixel
blendMode(BURN); // darken + saturate
blendMode(DODGE); // lighten + saturate
blendMode(SOFT_LIGHT); // subtle overlay
blendMode(HARD_LIGHT); // strong overlay
// ALWAYS reset after use
blendMode(BLEND);
```
### Blend Mode Recipes
| Effect | Mode | Use case |
|--------|------|----------|
| Additive glow | `ADD` | Light beams, fire, particles |
| Shadow overlay | `MULTIPLY` | Texture, vignette |
| Soft light mix | `SCREEN` | Fog, mist, backlight |
| High contrast | `OVERLAY` | Dramatic compositing |
| Color negative | `DIFFERENCE` | Glitch, psychedelic |
| Layer compositing | `BLEND` | Standard alpha layering |
## Background Techniques
### Textured Background
```javascript
function texturedBackground(baseColor, noiseScale, noiseAmount) {
loadPixels();
let r = red(baseColor), g = green(baseColor), b = blue(baseColor);
for (let i = 0; i < pixels.length; i += 4) {
let x = (i / 4) % width;
let y = floor((i / 4) / width);
let n = (noise(x * noiseScale, y * noiseScale) - 0.5) * noiseAmount;
pixels[i] = constrain(r + n, 0, 255);
pixels[i+1] = constrain(g + n, 0, 255);
pixels[i+2] = constrain(b + n, 0, 255);
pixels[i+3] = 255;
}
updatePixels();
}
```
### Vignette
```javascript
function vignette(strength = 0.5, radius = 0.7) {
loadPixels();
let cx = width / 2, cy = height / 2;
let maxDist = dist(0, 0, cx, cy);
for (let i = 0; i < pixels.length; i += 4) {
let x = (i / 4) % width;
let y = floor((i / 4) / width);
let d = dist(x, y, cx, cy) / maxDist;
let factor = 1.0 - smoothstep(constrain((d - radius) / (1 - radius), 0, 1)) * strength;
pixels[i] *= factor;
pixels[i+1] *= factor;
pixels[i+2] *= factor;
}
updatePixels();
}
function smoothstep(t) { return t * t * (3 - 2 * t); }
```
### Film Grain
```javascript
function filmGrain(amount = 30) {
loadPixels();
for (let i = 0; i < pixels.length; i += 4) {
let grain = random(-amount, amount);
pixels[i] = constrain(pixels[i] + grain, 0, 255);
pixels[i+1] = constrain(pixels[i+1] + grain, 0, 255);
pixels[i+2] = constrain(pixels[i+2] + grain, 0, 255);
}
updatePixels();
}
```
+410
View File
@@ -0,0 +1,410 @@
# Core API Reference
## Canvas Setup
### createCanvas()
```javascript
// 2D (default renderer)
createCanvas(1920, 1080);
// WebGL (3D, shaders)
createCanvas(1920, 1080, WEBGL);
// Responsive
createCanvas(windowWidth, windowHeight);
```
### Pixel Density
High-DPI displays render at 2x by default. This doubles memory usage and halves performance.
```javascript
// Force 1x for consistent export and performance
pixelDensity(1);
// Match display (default) — sharp on retina but expensive
pixelDensity(displayDensity());
// ALWAYS call before createCanvas()
function setup() {
pixelDensity(1); // first
createCanvas(1920, 1080); // second
}
```
For export, always `pixelDensity(1)` and use the exact target resolution. Never rely on device scaling for final output.
### Responsive Resize
```javascript
function windowResized() {
resizeCanvas(windowWidth, windowHeight);
// Recreate offscreen buffers at new size
bgLayer = createGraphics(width, height);
// Reinitialize any size-dependent state
}
```
## Coordinate System
### P2D (Default)
- Origin: top-left (0, 0)
- X increases rightward
- Y increases downward
- Angles: radians by default, `angleMode(DEGREES)` to switch
### WEBGL
- Origin: center of canvas
- X increases rightward, Y increases **upward**, Z increases toward viewer
- To get P2D-like coordinates in WEBGL: `translate(-width/2, -height/2)`
## Draw Loop
```javascript
function preload() {
// Load assets before setup — fonts, images, JSON, CSV
// Blocks execution until all loads complete
font = loadFont('font.otf');
img = loadImage('texture.png');
data = loadJSON('data.json');
}
function setup() {
// Runs once. Create canvas, initialize state.
createCanvas(1920, 1080);
colorMode(HSB, 360, 100, 100, 100);
randomSeed(CONFIG.seed);
noiseSeed(CONFIG.seed);
}
function draw() {
// Runs every frame (default 60fps).
// Set frameRate(30) in setup() to change.
// Call noLoop() for static sketches (render once).
}
```
### Frame Control
```javascript
frameRate(30); // set target FPS
noLoop(); // stop draw loop (static pieces)
loop(); // restart draw loop
redraw(); // call draw() once (manual refresh)
frameCount // frames since start (integer)
deltaTime // milliseconds since last frame (float)
millis() // milliseconds since sketch started
```
## Transform Stack
Every transform is cumulative. Use `push()`/`pop()` to isolate.
```javascript
push();
translate(width / 2, height / 2);
rotate(angle);
scale(1.5);
// draw something at transformed position
ellipse(0, 0, 100, 100);
pop();
// back to original coordinate system
```
### Transform Functions
| Function | Effect |
|----------|--------|
| `translate(x, y)` | Move origin |
| `rotate(angle)` | Rotate around origin (radians) |
| `scale(s)` / `scale(sx, sy)` | Scale from origin |
| `shearX(angle)` | Skew X axis |
| `shearY(angle)` | Skew Y axis |
| `applyMatrix(a, b, c, d, e, f)` | Arbitrary 2D affine transform |
| `resetMatrix()` | Clear all transforms |
### Composition Pattern: Rotate Around Center
```javascript
push();
translate(cx, cy); // move origin to center
rotate(angle); // rotate around that center
translate(-cx, -cy); // move origin back
// draw at original coordinates, but rotated around (cx, cy)
rect(cx - 50, cy - 50, 100, 100);
pop();
```
## Offscreen Buffers (createGraphics)
Offscreen buffers are separate canvases you can draw to and composite. Essential for:
- **Layered composition** — background, midground, foreground
- **Persistent trails** — draw to buffer, fade with semi-transparent rect, never clear
- **Masking** — draw mask to buffer, apply with `image()` or pixel operations
- **Post-processing** — render scene to buffer, apply effects, draw to main canvas
```javascript
let layer;
function setup() {
createCanvas(1920, 1080);
layer = createGraphics(width, height);
}
function draw() {
// Draw to offscreen buffer
layer.background(0, 10); // semi-transparent clear = trails
layer.fill(255);
layer.ellipse(mouseX, mouseY, 20);
// Composite to main canvas
image(layer, 0, 0);
}
```
### Trail Effect Pattern
```javascript
let trailBuffer;
function setup() {
createCanvas(1920, 1080);
trailBuffer = createGraphics(width, height);
trailBuffer.background(0);
}
function draw() {
// Fade previous frame (lower alpha = longer trails)
trailBuffer.noStroke();
trailBuffer.fill(0, 0, 0, 15); // RGBA — 15/255 alpha
trailBuffer.rect(0, 0, width, height);
// Draw new content
trailBuffer.fill(255);
trailBuffer.ellipse(mouseX, mouseY, 10);
// Show
image(trailBuffer, 0, 0);
}
```
### Multi-Layer Composition
```javascript
let bgLayer, contentLayer, fxLayer;
function setup() {
createCanvas(1920, 1080);
bgLayer = createGraphics(width, height);
contentLayer = createGraphics(width, height);
fxLayer = createGraphics(width, height);
}
function draw() {
// Background — drawn once or slowly evolving
renderBackground(bgLayer);
// Content — main visual elements
contentLayer.clear();
renderContent(contentLayer);
// FX — overlays, vignettes, grain
fxLayer.clear();
renderEffects(fxLayer);
// Composite with blend modes
image(bgLayer, 0, 0);
blendMode(ADD);
image(contentLayer, 0, 0);
blendMode(MULTIPLY);
image(fxLayer, 0, 0);
blendMode(BLEND); // reset
}
```
## Composition Patterns
### Grid Layout
```javascript
let cols = 10, rows = 10;
let cellW = width / cols;
let cellH = height / rows;
for (let i = 0; i < cols; i++) {
for (let j = 0; j < rows; j++) {
let cx = cellW * (i + 0.5);
let cy = cellH * (j + 0.5);
// draw element at (cx, cy) within cell size (cellW, cellH)
}
}
```
### Radial Layout
```javascript
let n = 12;
for (let i = 0; i < n; i++) {
let angle = TWO_PI * i / n;
let r = 300;
let x = width/2 + cos(angle) * r;
let y = height/2 + sin(angle) * r;
// draw element at (x, y)
}
```
### Golden Ratio Spiral
```javascript
let phi = (1 + sqrt(5)) / 2;
let n = 500;
for (let i = 0; i < n; i++) {
let angle = i * TWO_PI / (phi * phi);
let r = sqrt(i) * 10;
let x = width/2 + cos(angle) * r;
let y = height/2 + sin(angle) * r;
let size = map(i, 0, n, 8, 2);
ellipse(x, y, size);
}
```
### Margin-Aware Composition
```javascript
const MARGIN = 80; // pixels from edge
const drawW = width - 2 * MARGIN;
const drawH = height - 2 * MARGIN;
// Map normalized [0,1] coordinates to drawable area
function mapX(t) { return MARGIN + t * drawW; }
function mapY(t) { return MARGIN + t * drawH; }
```
## Random and Noise
### Seeded Random
```javascript
randomSeed(42);
let x = random(100); // always same value for seed 42
let y = random(-1, 1); // range
let item = random(myArray); // random element
```
### Gaussian Random
```javascript
let x = randomGaussian(0, 1); // mean=0, stddev=1
// Useful for natural-looking distributions
```
### Perlin Noise
```javascript
noiseSeed(42);
noiseDetail(4, 0.5); // 4 octaves, 0.5 falloff
let v = noise(x * 0.01, y * 0.01); // returns 0.0 to 1.0
// Scale factor (0.01) controls feature size — smaller = smoother
```
## Math Utilities
| Function | Description |
|----------|-------------|
| `map(v, lo1, hi1, lo2, hi2)` | Remap value between ranges |
| `constrain(v, lo, hi)` | Clamp to range |
| `lerp(a, b, t)` | Linear interpolation |
| `norm(v, lo, hi)` | Normalize to 0-1 |
| `dist(x1, y1, x2, y2)` | Euclidean distance |
| `mag(x, y)` | Vector magnitude |
| `abs()`, `ceil()`, `floor()`, `round()` | Standard math |
| `sq(n)`, `sqrt(n)`, `pow(b, e)` | Powers |
| `sin()`, `cos()`, `tan()`, `atan2()` | Trig (radians) |
| `degrees(r)`, `radians(d)` | Angle conversion |
| `fract(n)` | Fractional part |
## p5.js 2.0 Changes
p5.js 2.0 (released Apr 2025, current: 2.2) introduces breaking changes. The p5.js editor defaults to 1.x until Aug 2026. Use 2.x only when you need its features.
### async setup() replaces preload()
```javascript
// p5.js 1.x
let img;
function preload() { img = loadImage('cat.jpg'); }
function setup() { createCanvas(800, 800); }
// p5.js 2.x
let img;
async function setup() {
createCanvas(800, 800);
img = await loadImage('cat.jpg');
}
```
### New Color Modes
```javascript
colorMode(OKLCH); // perceptually uniform — better gradients
// L: 0-1 (lightness), C: 0-0.4 (chroma), H: 0-360 (hue)
fill(0.7, 0.15, 200); // medium-bright saturated blue
colorMode(OKLAB); // perceptually uniform, no hue angle
colorMode(HWB); // Hue-Whiteness-Blackness
```
### splineVertex() replaces curveVertex()
No more doubling first/last control points:
```javascript
// p5.js 1.x — must repeat first and last
beginShape();
curveVertex(pts[0].x, pts[0].y); // doubled
for (let p of pts) curveVertex(p.x, p.y);
curveVertex(pts[pts.length-1].x, pts[pts.length-1].y); // doubled
endShape();
// p5.js 2.x — clean
beginShape();
for (let p of pts) splineVertex(p.x, p.y);
endShape();
```
### Shader .modify() API
Modify built-in shaders without writing full GLSL:
```javascript
let myShader = baseMaterialShader().modify({
vertexDeclarations: 'uniform float uTime;',
'vec4 getWorldPosition': `(vec4 pos) {
pos.y += sin(pos.x * 0.1 + uTime) * 20.0;
return pos;
}`
});
```
### Variable Fonts
```javascript
textWeight(700); // dynamic weight without loading multiple files
```
### textToContours() and textToModel()
```javascript
let contours = font.textToContours('HELLO', 0, 0, 200);
// Returns array of contour arrays (closed paths)
let geo = font.textToModel('HELLO', 0, 0, 200);
// Returns p5.Geometry for 3D extruded text
```
### CDN for p5.js 2.x
```html
<script src="https://cdn.jsdelivr.net/npm/p5@2/lib/p5.min.js"></script>
```
@@ -0,0 +1,566 @@
# Export Pipeline
## PNG Export
### In-Sketch (Keyboard Shortcut)
```javascript
function keyPressed() {
if (key === 's' || key === 'S') {
saveCanvas('output', 'png');
// Downloads output.png immediately
}
}
```
### Timed Export (Static Generative)
```javascript
function setup() {
createCanvas(3840, 2160);
pixelDensity(1);
randomSeed(CONFIG.seed);
noiseSeed(CONFIG.seed);
noLoop();
}
function draw() {
// ... render everything ...
saveCanvas('output-seed-' + CONFIG.seed, 'png');
}
```
### High-Resolution Export
For resolutions beyond screen size, use `pixelDensity()` or a large offscreen buffer:
```javascript
function exportHighRes(scale) {
let buffer = createGraphics(width * scale, height * scale);
buffer.scale(scale);
// Re-render everything to buffer at higher resolution
renderScene(buffer);
buffer.save('highres-output.png');
}
```
### Batch Seed Export
```javascript
function exportBatch(startSeed, count) {
for (let i = 0; i < count; i++) {
CONFIG.seed = startSeed + i;
randomSeed(CONFIG.seed);
noiseSeed(CONFIG.seed);
// Render
background(0);
renderScene();
saveCanvas('seed-' + nf(CONFIG.seed, 5), 'png');
}
}
```
## GIF Export
### saveGif()
```javascript
function keyPressed() {
if (key === 'g' || key === 'G') {
saveGif('output', 5);
// Captures 5 seconds of animation
// Options: saveGif(filename, duration, options)
}
}
// With options
saveGif('output', 5, {
delay: 0, // delay before starting capture (seconds)
units: 'seconds' // or 'frames'
});
```
Limitations:
- GIF is 256 colors max — dithering artifacts on gradients
- Large canvases produce huge files
- Use a smaller canvas (640x360) for GIF, higher for PNG/MP4
- Frame rate is approximate
### Optimal GIF Settings
```javascript
// For GIF output, use smaller canvas and lower framerate
function setup() {
createCanvas(640, 360);
frameRate(15); // GIF standard
pixelDensity(1);
}
```
## Frame Sequence Export
### saveFrames()
```javascript
function keyPressed() {
if (key === 'f') {
saveFrames('frame', 'png', 10, 30);
// 10 seconds, 30 fps → 300 PNG files
// Downloads as individual files (browser may block bulk downloads)
}
}
```
### Manual Frame Export (More Control)
```javascript
let recording = false;
let frameNum = 0;
const TOTAL_FRAMES = 300;
function keyPressed() {
if (key === 'r') recording = !recording;
}
function draw() {
// ... render frame ...
if (recording) {
saveCanvas('frame-' + nf(frameNum, 4), 'png');
frameNum++;
if (frameNum >= TOTAL_FRAMES) {
recording = false;
noLoop();
console.log('Recording complete: ' + frameNum + ' frames');
}
}
}
```
### Deterministic Capture (Critical for Video)
The `noLoop()` + `redraw()` pattern is **required** for frame-perfect headless capture. Without it, p5's draw loop runs freely in Chrome while Puppeteer screenshots are slow — the sketch runs ahead and you get duplicate/missing frames.
```javascript
function setup() {
createCanvas(1920, 1080);
pixelDensity(1);
noLoop(); // STOP the automatic draw loop
window._p5Ready = true; // Signal to capture script
}
function draw() {
// This only runs when redraw() is called by the capture script
// frameCount increments exactly once per redraw()
}
```
The bundled `scripts/export-frames.js` detects `window._p5Ready` and switches to deterministic mode automatically. Without it, falls back to timed capture (less precise).
### ffmpeg: Frames to MP4
```bash
# Basic encoding
ffmpeg -framerate 30 -i frame-%04d.png -c:v libx264 -pix_fmt yuv420p output.mp4
# High quality
ffmpeg -framerate 30 -i frame-%04d.png \
-c:v libx264 -preset slow -crf 18 -pix_fmt yuv420p \
output.mp4
# With audio
ffmpeg -framerate 30 -i frame-%04d.png -i audio.mp3 \
-c:v libx264 -c:a aac -shortest \
output.mp4
# Loop for social media (3 loops)
ffmpeg -stream_loop 2 -i output.mp4 -c copy output-looped.mp4
```
### Video Export Gotchas
**YUV420 clips dark values.** H.264 encodes in YUV420 color space, which rounds dark RGB values. Content below RGB(8,8,8) may become pure black. Subtle dark details (dim particle trails, faint noise textures) disappear in the encoded video even though they're visible in the PNG frames.
**Fix:** Ensure minimum brightness of ~10 for any visible content. Test by encoding a few frames and comparing the MP4 frame vs the source PNG.
```bash
# Extract a frame from MP4 for comparison
ffmpeg -i output.mp4 -vf "select=eq(n\,100)" -vframes 1 check.png
```
**Static frames look broken in video.** If an algorithm produces a single static image (like a pre-computed attractor heatmap), it reads as a freeze/glitch in video. Always add animation even to static content:
- Progressive reveal (expand from center, sweep across)
- Slow parameter drift (rotate color mapping, shift noise offset)
- Camera-like motion (slow zoom, slight pan)
- Overlay animated particles or grain
**Scene transitions are mandatory.** Hard cuts between visually different scenes are jarring. Use fade envelopes:
```javascript
const FADE_FRAMES = 15; // half-second at 30fps
let fade = 1;
if (localFrame < FADE_FRAMES) fade = localFrame / FADE_FRAMES;
if (localFrame > SCENE_FRAMES - FADE_FRAMES) fade = (SCENE_FRAMES - localFrame) / FADE_FRAMES;
fade = fade * fade * (3 - 2 * fade); // smoothstep
// Apply: multiply all alpha/brightness by fade
```
### Per-Clip Architecture (Multi-Scene Videos)
For videos with multiple scenes, render each as a separate HTML file + MP4 clip, then stitch with ffmpeg. This enables re-rendering individual scenes without touching the rest.
**Directory structure:**
```
project/
├── capture-scene.js # Shared: node capture-scene.js <html> <outdir> <frames>
├── render-all.sh # Renders all + stitches
├── scenes/
│ ├── 00-intro.html # Each scene is self-contained
│ ├── 01-particles.html
│ ├── 02-noise.html
│ └── 03-outro.html
└── clips/
├── 00-intro.mp4 # Each clip rendered independently
├── 01-particles.mp4
├── 02-noise.mp4
├── 03-outro.mp4
└── concat.txt
```
**Stitch clips with ffmpeg concat:**
```bash
# concat.txt (order determines final sequence)
file '00-intro.mp4'
file '01-particles.mp4'
file '02-noise.mp4'
file '03-outro.mp4'
# Lossless stitch (all clips must have same codec/resolution/fps)
ffmpeg -f concat -safe 0 -i concat.txt -c copy final.mp4
```
**Re-render a single scene:**
```bash
node capture-scene.js scenes/01-particles.html clips/01-particles 150
ffmpeg -y -framerate 30 -i clips/01-particles/frame-%04d.png \
-c:v libx264 -preset slow -crf 16 -pix_fmt yuv420p clips/01-particles.mp4
# Then re-stitch
ffmpeg -y -f concat -safe 0 -i clips/concat.txt -c copy final.mp4
```
**Re-order without re-rendering:** Just change the order in concat.txt and re-stitch. No frames need re-rendering.
**Each scene HTML must:**
- Call `noLoop()` in setup and set `window._p5Ready = true`
- Use `frameCount`-based timing (not `millis()`) for deterministic output
- Handle its own fade-in/fade-out envelope
- Be fully self-contained (no shared state between scenes)
### ffmpeg: Frames to GIF (Better Quality)
```bash
# Generate palette first for optimal colors
ffmpeg -i frame-%04d.png -vf "fps=15,palettegen=max_colors=256" palette.png
# Render GIF using palette
ffmpeg -i frame-%04d.png -i palette.png \
-lavfi "fps=15 [x]; [x][1:v] paletteuse=dither=bayer:bayer_scale=3" \
output.gif
```
## Headless Export (Puppeteer)
For automated, server-side, or CI rendering. Uses a headless Chrome browser to run the sketch.
### export-frames.js (Node.js Script)
See `scripts/export-frames.js` for the full implementation. Basic pattern:
```javascript
const puppeteer = require('puppeteer');
async function captureFrames(htmlPath, outputDir, options) {
const browser = await puppeteer.launch({
headless: true,
args: ['--no-sandbox', '--disable-setuid-sandbox']
});
const page = await browser.newPage();
await page.setViewport({
width: options.width || 1920,
height: options.height || 1080,
deviceScaleFactor: 1
});
await page.goto(`file://${path.resolve(htmlPath)}`, {
waitUntil: 'networkidle0'
});
// Wait for sketch to initialize
await page.waitForSelector('canvas');
await page.waitForTimeout(1000);
for (let i = 0; i < options.frames; i++) {
const canvas = await page.$('canvas');
await canvas.screenshot({
path: path.join(outputDir, `frame-${String(i).padStart(4, '0')}.png`)
});
// Advance one frame
await page.evaluate(() => { redraw(); });
await page.waitForTimeout(1000 / options.fps);
}
await browser.close();
}
```
### render.sh (Full Pipeline)
See `scripts/render.sh` for the complete render script. Pipeline:
```
1. Launch Puppeteer → open sketch HTML
2. Capture N frames as PNG sequence
3. Pipe to ffmpeg → encode H.264 MP4
4. Optional: add audio track
5. Clean up temp frames
```
## SVG Export
### Using p5.js-svg Library
```html
<script src="https://unpkg.com/p5.js-svg@1.5.1"></script>
```
```javascript
function setup() {
createCanvas(1920, 1080, SVG); // SVG renderer
noLoop();
}
function draw() {
// Only vector operations (no pixels, no blend modes)
stroke(0);
noFill();
for (let i = 0; i < 100; i++) {
let x = random(width);
let y = random(height);
ellipse(x, y, random(10, 50));
}
save('output.svg');
}
```
Limitations:
- No `loadPixels()`, `updatePixels()`, `filter()`, `blendMode()`
- No WebGL
- No pixel-level effects
- Great for: line art, geometric patterns, plots
### Hybrid: Raster Background + SVG Overlay
Render background effects to PNG, then SVG for crisp vector elements on top.
## Export Format Decision Guide
| Need | Format | Method |
|------|--------|--------|
| Single still image | PNG | `saveCanvas()` or `keyPressed()` |
| Print-quality still | PNG (high-res) | `pixelDensity(1)` + large canvas |
| Short animated loop | GIF | `saveGif()` |
| Long animation | MP4 | Frame sequence + ffmpeg |
| Social media video | MP4 | `scripts/render.sh` |
| Vector/print | SVG | p5.js-svg renderer |
| Batch variations | PNG sequence | Seed loop + `saveCanvas()` |
| Interactive deployment | HTML | Single self-contained file |
| Headless rendering | PNG/MP4 | Puppeteer + ffmpeg |
## Tiling for Ultra-High-Resolution
For resolutions too large for a single canvas (e.g., 10000x10000 for print):
```javascript
function renderTiled(totalW, totalH, tileSize) {
let cols = ceil(totalW / tileSize);
let rows = ceil(totalH / tileSize);
for (let ty = 0; ty < rows; ty++) {
for (let tx = 0; tx < cols; tx++) {
let buffer = createGraphics(tileSize, tileSize);
buffer.push();
buffer.translate(-tx * tileSize, -ty * tileSize);
renderScene(buffer, totalW, totalH);
buffer.pop();
buffer.save(`tile-${tx}-${ty}.png`);
buffer.remove(); // free memory
}
}
// Stitch with ImageMagick:
// montage tile-*.png -tile 4x4 -geometry +0+0 final.png
}
```
## CCapture.js — Deterministic Video Capture
The built-in `saveFrames()` has limitations: small frame counts, memory issues, browser download blocking. CCapture.js solves all of these by hooking into the browser's timing functions to simulate constant time steps regardless of actual render speed.
```html
<script src="https://cdn.jsdelivr.net/npm/ccapture.js-npmfixed/build/CCapture.all.min.js"></script>
```
### Basic Setup
```javascript
let capturer;
let recording = false;
function setup() {
createCanvas(1920, 1080);
pixelDensity(1);
capturer = new CCapture({
format: 'webm', // 'webm', 'gif', 'png', 'jpg'
framerate: 30,
quality: 99, // 0-100 for webm/jpg
// timeLimit: 10, // auto-stop after N seconds
// motionBlurFrames: 4 // supersampled motion blur
});
}
function draw() {
// ... render frame ...
if (recording) {
capturer.capture(document.querySelector('canvas'));
}
}
function keyPressed() {
if (key === 'c') {
if (!recording) {
capturer.start();
recording = true;
console.log('Recording started');
} else {
capturer.stop();
capturer.save(); // triggers download
recording = false;
console.log('Recording saved');
}
}
}
```
### Format Comparison
| Format | Quality | Size | Browser Support |
|--------|---------|------|-----------------|
| **WebM** | High | Medium | Chrome only |
| **GIF** | 256 colors | Large | All (via gif.js worker) |
| **PNG sequence** | Lossless | Very large (TAR) | All |
| **JPEG sequence** | Lossy | Large (TAR) | All |
### Important: Timing Hook
CCapture.js overrides `Date.now()`, `setTimeout`, `requestAnimationFrame`, and `performance.now()`. This means:
- `millis()` returns simulated time (perfect for recording)
- `deltaTime` is constant (1000/framerate)
- Complex sketches that take 500ms per frame still record at smooth 30fps
- **Caveat**: Audio sync breaks (audio plays in real-time, not simulated time)
## Programmatic Export (canvas API)
For custom export workflows beyond `saveCanvas()`:
```javascript
// Canvas to Blob (for upload, processing)
document.querySelector('canvas').toBlob((blob) => {
// Upload to server, process, etc.
let url = URL.createObjectURL(blob);
console.log('Blob URL:', url);
}, 'image/png');
// Canvas to Data URL (for inline embedding)
let dataUrl = document.querySelector('canvas').toDataURL('image/png');
// Use in <img src="..."> or send as base64
```
## SVG Export (p5.js-svg)
```html
<script src="https://unpkg.com/p5.js-svg@1.6.0"></script>
```
```javascript
function setup() {
createCanvas(1920, 1080, SVG); // SVG renderer
noLoop();
}
function draw() {
// Only vector operations work (no pixel ops, no blendMode)
stroke(0);
noFill();
for (let i = 0; i < 100; i++) {
ellipse(random(width), random(height), random(10, 50));
}
save('output.svg');
}
```
**Critical SVG caveats:**
- **Must call `clear()` in `draw()`** for animated sketches — SVG DOM accumulates child elements, causing memory bloat
- `blendMode()` is **not implemented** in SVG renderer
- `filter()`, `loadPixels()`, `updatePixels()` don't work
- Requires **p5.js 1.11.x** — not compatible with p5.js 2.x
- Perfect for: line art, geometric patterns, pen plotter output
## Platform Export
### fxhash Conventions
```javascript
// Replace p5's random with fxhash's deterministic PRNG
const rng = $fx.rand;
// Declare features for rarity/filtering
$fx.features({
'Palette': paletteName,
'Complexity': complexity > 0.7 ? 'High' : 'Low',
'Has Particles': particleCount > 0
});
// Declare on-chain parameters
$fx.params([
{ id: 'density', name: 'Density', type: 'number',
options: { min: 1, max: 100, step: 1 } },
{ id: 'palette', name: 'Palette', type: 'select',
options: { options: ['Warm', 'Cool', 'Mono'] } },
{ id: 'accent', name: 'Accent Color', type: 'color' }
]);
// Read params
let density = $fx.getParam('density');
// Build: npx fxhash build → upload.zip
// Dev: npx fxhash dev → localhost:3300
```
### Art Blocks / Generic Platform
```javascript
// Platform provides a hash string
const hash = tokenData.hash; // Art Blocks convention
// Build deterministic PRNG from hash
function prngFromHash(hash) {
let seed = parseInt(hash.slice(0, 16), 16);
// xoshiro128** or similar
return function() { /* ... */ };
}
const rng = prngFromHash(hash);
```
@@ -0,0 +1,398 @@
# Interaction
## Mouse Events
### Continuous State
```javascript
mouseX, mouseY // current position (relative to canvas)
pmouseX, pmouseY // previous frame position
mouseIsPressed // boolean
mouseButton // LEFT, RIGHT, CENTER (during press)
movedX, movedY // delta since last frame
winMouseX, winMouseY // relative to window (not canvas)
```
### Event Callbacks
```javascript
function mousePressed() {
// fires once on press
// mouseButton tells you which button
}
function mouseReleased() {
// fires once on release
}
function mouseClicked() {
// fires after press+release (same element)
}
function doubleClicked() {
// fires on double-click
}
function mouseMoved() {
// fires when mouse moves (no button pressed)
}
function mouseDragged() {
// fires when mouse moves WITH button pressed
}
function mouseWheel(event) {
// event.delta: positive = scroll down, negative = scroll up
zoom += event.delta * -0.01;
return false; // prevent page scroll
}
```
### Mouse Interaction Patterns
**Spawn on click:**
```javascript
function mousePressed() {
particles.push(new Particle(mouseX, mouseY));
}
```
**Mouse follow with spring:**
```javascript
let springX, springY;
function setup() {
springX = new Spring(width/2, width/2);
springY = new Spring(height/2, height/2);
}
function draw() {
springX.setTarget(mouseX);
springY.setTarget(mouseY);
let x = springX.update();
let y = springY.update();
ellipse(x, y, 50);
}
```
**Drag interaction:**
```javascript
let dragging = false;
let dragObj = null;
let offsetX, offsetY;
function mousePressed() {
for (let obj of objects) {
if (dist(mouseX, mouseY, obj.x, obj.y) < obj.radius) {
dragging = true;
dragObj = obj;
offsetX = mouseX - obj.x;
offsetY = mouseY - obj.y;
break;
}
}
}
function mouseDragged() {
if (dragging && dragObj) {
dragObj.x = mouseX - offsetX;
dragObj.y = mouseY - offsetY;
}
}
function mouseReleased() {
dragging = false;
dragObj = null;
}
```
**Mouse repulsion (particles flee cursor):**
```javascript
function draw() {
let mousePos = createVector(mouseX, mouseY);
for (let p of particles) {
let d = p.pos.dist(mousePos);
if (d < 150) {
let repel = p5.Vector.sub(p.pos, mousePos);
repel.normalize();
repel.mult(map(d, 0, 150, 5, 0));
p.applyForce(repel);
}
}
}
```
## Keyboard Events
### State
```javascript
keyIsPressed // boolean
key // last key as string ('a', 'A', ' ')
keyCode // numeric code (LEFT_ARROW, UP_ARROW, etc.)
```
### Event Callbacks
```javascript
function keyPressed() {
// fires once on press
if (keyCode === LEFT_ARROW) { /* ... */ }
if (key === 's') saveCanvas('output', 'png');
if (key === ' ') CONFIG.paused = !CONFIG.paused;
return false; // prevent default browser behavior
}
function keyReleased() {
// fires once on release
}
function keyTyped() {
// fires for printable characters only (not arrows, shift, etc.)
}
```
### Continuous Key State (Multiple Keys)
```javascript
let keys = {};
function keyPressed() { keys[keyCode] = true; }
function keyReleased() { keys[keyCode] = false; }
function draw() {
if (keys[LEFT_ARROW]) player.x -= 5;
if (keys[RIGHT_ARROW]) player.x += 5;
if (keys[UP_ARROW]) player.y -= 5;
if (keys[DOWN_ARROW]) player.y += 5;
}
```
### Key Constants
```
LEFT_ARROW, RIGHT_ARROW, UP_ARROW, DOWN_ARROW
BACKSPACE, DELETE, ENTER, RETURN, TAB, ESCAPE
SHIFT, CONTROL, OPTION, ALT
```
## Touch Events
```javascript
touches // array of { x, y, id } — all current touches
function touchStarted() {
// fires on first touch
return false; // prevent default (stops scroll on mobile)
}
function touchMoved() {
// fires on touch drag
return false;
}
function touchEnded() {
// fires on touch release
}
```
### Pinch Zoom
```javascript
let prevDist = 0;
let zoomLevel = 1;
function touchMoved() {
if (touches.length === 2) {
let d = dist(touches[0].x, touches[0].y, touches[1].x, touches[1].y);
if (prevDist > 0) {
zoomLevel *= d / prevDist;
}
prevDist = d;
}
return false;
}
function touchEnded() {
prevDist = 0;
}
```
## DOM Elements
### Creating Controls
```javascript
function setup() {
createCanvas(800, 800);
// Slider
let slider = createSlider(0, 255, 100, 1); // min, max, default, step
slider.position(10, height + 10);
slider.input(() => { CONFIG.value = slider.value(); });
// Button
let btn = createButton('Reset');
btn.position(10, height + 40);
btn.mousePressed(() => { resetSketch(); });
// Checkbox
let check = createCheckbox('Show grid', false);
check.position(10, height + 70);
check.changed(() => { CONFIG.showGrid = check.checked(); });
// Select / dropdown
let sel = createSelect();
sel.position(10, height + 100);
sel.option('Mode A');
sel.option('Mode B');
sel.changed(() => { CONFIG.mode = sel.value(); });
// Color picker
let picker = createColorPicker('#ff0000');
picker.position(10, height + 130);
picker.input(() => { CONFIG.color = picker.value(); });
// Text input
let inp = createInput('Hello');
inp.position(10, height + 160);
inp.input(() => { CONFIG.text = inp.value(); });
}
```
### Styling DOM Elements
```javascript
let slider = createSlider(0, 100, 50);
slider.position(10, 10);
slider.style('width', '200px');
slider.class('my-slider');
slider.parent('controls-div'); // attach to specific DOM element
```
## Audio Input (p5.sound)
Requires `p5.sound.min.js` addon.
```html
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.11.3/addons/p5.sound.min.js"></script>
```
### Microphone Input
```javascript
let mic, fft, amplitude;
function setup() {
createCanvas(800, 800);
userStartAudio(); // required — user gesture to enable audio
mic = new p5.AudioIn();
mic.start();
fft = new p5.FFT(0.8, 256); // smoothing, bins
fft.setInput(mic);
amplitude = new p5.Amplitude();
amplitude.setInput(mic);
}
function draw() {
let level = amplitude.getLevel(); // 0.0 to 1.0 (overall volume)
let spectrum = fft.analyze(); // array of 256 frequency values (0-255)
let waveform = fft.waveform(); // array of 256 time-domain samples (-1 to 1)
// Get energy in frequency bands
let bass = fft.getEnergy('bass'); // 20-140 Hz
let lowMid = fft.getEnergy('lowMid'); // 140-400 Hz
let mid = fft.getEnergy('mid'); // 400-2600 Hz
let highMid = fft.getEnergy('highMid'); // 2600-5200 Hz
let treble = fft.getEnergy('treble'); // 5200-14000 Hz
// Each returns 0-255
}
```
### Audio File Playback
```javascript
let song, fft;
function preload() {
song = loadSound('track.mp3');
}
function setup() {
createCanvas(800, 800);
fft = new p5.FFT(0.8, 512);
fft.setInput(song);
}
function mousePressed() {
if (song.isPlaying()) {
song.pause();
} else {
song.play();
}
}
```
### Beat Detection (Simple)
```javascript
let prevBass = 0;
let beatThreshold = 30;
let beatCooldown = 0;
function detectBeat() {
let bass = fft.getEnergy('bass');
let isBeat = bass - prevBass > beatThreshold && beatCooldown <= 0;
prevBass = bass;
if (isBeat) beatCooldown = 10; // frames
beatCooldown--;
return isBeat;
}
```
## Scroll-Driven Animation
```javascript
let scrollProgress = 0;
function setup() {
let canvas = createCanvas(windowWidth, windowHeight);
canvas.style('position', 'fixed');
// Make page scrollable
document.body.style.height = '500vh';
}
window.addEventListener('scroll', () => {
let maxScroll = document.body.scrollHeight - window.innerHeight;
scrollProgress = window.scrollY / maxScroll;
});
function draw() {
background(0);
// Use scrollProgress (0 to 1) to drive animation
let x = lerp(0, width, scrollProgress);
ellipse(x, height/2, 50);
}
```
## Responsive Events
```javascript
function windowResized() {
resizeCanvas(windowWidth, windowHeight);
// Recreate buffers
bgLayer = createGraphics(width, height);
// Recalculate layout
recalculateLayout();
}
// Visibility change (tab switching)
document.addEventListener('visibilitychange', () => {
if (document.hidden) {
noLoop(); // pause when tab not visible
} else {
loop();
}
});
```
@@ -0,0 +1,300 @@
# Shapes and Geometry
## 2D Primitives
```javascript
point(x, y);
line(x1, y1, x2, y2);
rect(x, y, w, h); // default: corner mode
rect(x, y, w, h, r); // rounded corners
rect(x, y, w, h, tl, tr, br, bl); // per-corner radius
square(x, y, size);
ellipse(x, y, w, h);
circle(x, y, d); // diameter, not radius
triangle(x1, y1, x2, y2, x3, y3);
quad(x1, y1, x2, y2, x3, y3, x4, y4);
arc(x, y, w, h, start, stop, mode); // mode: OPEN, CHORD, PIE
```
### Drawing Modes
```javascript
rectMode(CENTER); // x,y is center (default: CORNER)
rectMode(CORNERS); // x1,y1 to x2,y2
ellipseMode(CORNER); // x,y is top-left corner
ellipseMode(CENTER); // default — x,y is center
```
## Stroke and Fill
```javascript
fill(r, g, b, a); // or fill(gray), fill('#hex'), fill(h, s, b) in HSB mode
noFill();
stroke(r, g, b, a);
noStroke();
strokeWeight(2);
strokeCap(ROUND); // ROUND, SQUARE, PROJECT
strokeJoin(ROUND); // ROUND, MITER, BEVEL
```
## Custom Shapes with Vertices
### Basic vertex shape
```javascript
beginShape();
vertex(100, 100);
vertex(200, 50);
vertex(300, 100);
vertex(250, 200);
vertex(150, 200);
endShape(CLOSE); // CLOSE connects last vertex to first
```
### Shape modes
```javascript
beginShape(); // default: polygon connecting all vertices
beginShape(POINTS); // individual points
beginShape(LINES); // pairs of vertices as lines
beginShape(TRIANGLES); // triplets as triangles
beginShape(TRIANGLE_FAN);
beginShape(TRIANGLE_STRIP);
beginShape(QUADS); // groups of 4
beginShape(QUAD_STRIP);
```
### Contours (holes in shapes)
```javascript
beginShape();
// outer shape
vertex(100, 100);
vertex(300, 100);
vertex(300, 300);
vertex(100, 300);
// inner hole
beginContour();
vertex(150, 150);
vertex(150, 250);
vertex(250, 250);
vertex(250, 150);
endContour();
endShape(CLOSE);
```
## Bezier Curves
### Cubic Bezier
```javascript
bezier(x1, y1, cx1, cy1, cx2, cy2, x2, y2);
// x1,y1 = start point
// cx1,cy1 = first control point
// cx2,cy2 = second control point
// x2,y2 = end point
```
### Bezier in custom shapes
```javascript
beginShape();
vertex(100, 200);
bezierVertex(150, 50, 250, 50, 300, 200);
// control1, control2, endpoint
endShape();
```
### Quadratic Bezier
```javascript
beginShape();
vertex(100, 200);
quadraticVertex(200, 50, 300, 200);
// single control point + endpoint
endShape();
```
### Interpolation along Bezier
```javascript
let x = bezierPoint(x1, cx1, cx2, x2, t); // t = 0..1
let y = bezierPoint(y1, cy1, cy2, y2, t);
let tx = bezierTangent(x1, cx1, cx2, x2, t); // tangent
```
## Catmull-Rom Splines
```javascript
curve(cpx1, cpy1, x1, y1, x2, y2, cpx2, cpy2);
// cpx1,cpy1 = control point before start
// x1,y1 = start point (visible)
// x2,y2 = end point (visible)
// cpx2,cpy2 = control point after end
curveVertex(x, y); // in beginShape() — smooth curve through all points
curveTightness(0); // 0 = Catmull-Rom, 1 = straight lines, -1 = loose
```
### Smooth curve through points
```javascript
let points = [/* array of {x, y} */];
beginShape();
curveVertex(points[0].x, points[0].y); // repeat first for tangent
for (let p of points) {
curveVertex(p.x, p.y);
}
curveVertex(points[points.length-1].x, points[points.length-1].y); // repeat last
endShape();
```
## p5.Vector
Essential for physics, particle systems, and geometric computation.
```javascript
let v = createVector(x, y);
// Arithmetic (modifies in place)
v.add(other); // vector addition
v.sub(other); // subtraction
v.mult(scalar); // scale
v.div(scalar); // inverse scale
v.normalize(); // unit vector (length 1)
v.limit(max); // cap magnitude
v.setMag(len); // set exact magnitude
// Queries (non-destructive)
v.mag(); // magnitude (length)
v.magSq(); // squared magnitude (faster, no sqrt)
v.heading(); // angle in radians
v.dist(other); // distance to other vector
v.dot(other); // dot product
v.cross(other); // cross product (3D)
v.angleBetween(other); // angle between vectors
// Static methods (return new vector)
p5.Vector.add(a, b); // a + b → new vector
p5.Vector.sub(a, b); // a - b → new vector
p5.Vector.fromAngle(a); // unit vector at angle
p5.Vector.random2D(); // random unit vector
p5.Vector.lerp(a, b, t); // interpolate
// Copy
let copy = v.copy();
```
## Signed Distance Fields (2D)
SDFs return the distance from a point to the nearest edge of a shape. Negative inside, positive outside. Useful for smooth shapes, glow effects, boolean operations.
```javascript
// Circle SDF
function sdCircle(px, py, cx, cy, r) {
return dist(px, py, cx, cy) - r;
}
// Box SDF
function sdBox(px, py, cx, cy, hw, hh) {
let dx = abs(px - cx) - hw;
let dy = abs(py - cy) - hh;
return sqrt(max(dx, 0) ** 2 + max(dy, 0) ** 2) + min(max(dx, dy), 0);
}
// Line segment SDF
function sdSegment(px, py, ax, ay, bx, by) {
let pa = createVector(px - ax, py - ay);
let ba = createVector(bx - ax, by - ay);
let t = constrain(pa.dot(ba) / ba.dot(ba), 0, 1);
let closest = p5.Vector.add(createVector(ax, ay), p5.Vector.mult(ba, t));
return dist(px, py, closest.x, closest.y);
}
// Smooth boolean union
function opSmoothUnion(d1, d2, k) {
let h = constrain(0.5 + 0.5 * (d2 - d1) / k, 0, 1);
return lerp(d2, d1, h) - k * h * (1 - h);
}
// Rendering SDF as glow
let d = sdCircle(x, y, width/2, height/2, 200);
let glow = exp(-abs(d) * 0.02); // exponential falloff
fill(glow * 255);
```
## Useful Geometry Patterns
### Regular Polygon
```javascript
function regularPolygon(cx, cy, r, sides) {
beginShape();
for (let i = 0; i < sides; i++) {
let a = TWO_PI * i / sides - HALF_PI;
vertex(cx + cos(a) * r, cy + sin(a) * r);
}
endShape(CLOSE);
}
```
### Star Shape
```javascript
function star(cx, cy, r1, r2, npoints) {
beginShape();
let angle = TWO_PI / npoints;
let halfAngle = angle / 2;
for (let a = -HALF_PI; a < TWO_PI - HALF_PI; a += angle) {
vertex(cx + cos(a) * r2, cy + sin(a) * r2);
vertex(cx + cos(a + halfAngle) * r1, cy + sin(a + halfAngle) * r1);
}
endShape(CLOSE);
}
```
### Rounded Line (Capsule)
```javascript
function capsule(x1, y1, x2, y2, weight) {
strokeWeight(weight);
strokeCap(ROUND);
line(x1, y1, x2, y2);
}
```
### Soft Body / Blob
```javascript
function blob(cx, cy, baseR, noiseScale, noiseOffset, detail = 64) {
beginShape();
for (let i = 0; i < detail; i++) {
let a = TWO_PI * i / detail;
let r = baseR + noise(cos(a) * noiseScale + noiseOffset,
sin(a) * noiseScale + noiseOffset) * baseR * 0.4;
vertex(cx + cos(a) * r, cy + sin(a) * r);
}
endShape(CLOSE);
}
```
## Clipping and Masking
```javascript
// Clip shape — everything drawn after is masked by the clip shape
beginClip();
circle(width/2, height/2, 400);
endClip();
// Only content inside the circle is visible
image(myImage, 0, 0);
// Or functional form
clip(() => {
circle(width/2, height/2, 400);
});
// Erase mode — cut holes
erase();
circle(mouseX, mouseY, 100); // this area becomes transparent
noErase();
```
@@ -0,0 +1,532 @@
# Troubleshooting
## Performance
### Step Zero — Disable FES
The Friendly Error System (FES) adds massive overhead — up to 10x slowdown. Disable it in every production sketch:
```javascript
// BEFORE any p5 code
p5.disableFriendlyErrors = true;
// Or use p5.min.js instead of p5.js — FES is stripped from minified build
```
### Step One — pixelDensity(1)
Retina/HiDPI displays default to 2x or 3x density, multiplying pixel count by 4-9x:
```javascript
function setup() {
pixelDensity(1); // force 1:1 — always do this first
createCanvas(1920, 1080);
}
```
### Use Math.* in Hot Loops
p5's `sin()`, `cos()`, `random()`, `min()`, `max()`, `abs()` are wrapper functions with overhead. In hot loops (thousands of iterations per frame), use native `Math.*`:
```javascript
// SLOW — p5 wrappers
for (let p of particles) {
let a = sin(p.angle);
let d = dist(p.x, p.y, mx, my);
}
// FAST — native Math
for (let p of particles) {
let a = Math.sin(p.angle);
let dx = p.x - mx, dy = p.y - my;
let dSq = dx * dx + dy * dy; // skip sqrt entirely
}
```
Use `magSq()` instead of `mag()` for distance comparisons — avoids expensive `sqrt()`.
### Diagnosis
Open Chrome DevTools > Performance tab > Record while sketch runs.
Common bottlenecks:
1. **FES enabled** — 10x overhead on every p5 function call
2. **pixelDensity > 1** — 4x pixel count, 4x slower
3. **Too many draw calls** — thousands of `ellipse()`, `rect()` per frame
4. **Large canvas + pixel operations**`loadPixels()`/`updatePixels()` on 4K canvas
5. **Unoptimized particle systems** — checking all-vs-all distances (O(n^2))
6. **Memory leaks** — creating objects every frame without cleanup
7. **Shader compilation** — calling `createShader()` in `draw()` instead of `setup()`
8. **console.log() in draw()** — DOM write per frame, destroys performance
9. **DOM manipulation in draw()** — layout thrashing (400-500x slower than canvas ops)
### Solutions
**Reduce draw calls:**
```javascript
// BAD: 10000 individual circles
for (let p of particles) {
ellipse(p.x, p.y, p.size);
}
// GOOD: single shape with vertices
beginShape(POINTS);
for (let p of particles) {
vertex(p.x, p.y);
}
endShape();
// BEST: direct pixel manipulation
loadPixels();
for (let p of particles) {
let idx = 4 * (floor(p.y) * width + floor(p.x));
pixels[idx] = p.r;
pixels[idx+1] = p.g;
pixels[idx+2] = p.b;
pixels[idx+3] = 255;
}
updatePixels();
```
**Spatial hashing for neighbor queries:**
```javascript
class SpatialHash {
constructor(cellSize) {
this.cellSize = cellSize;
this.cells = new Map();
}
clear() { this.cells.clear(); }
_key(x, y) {
return `${floor(x / this.cellSize)},${floor(y / this.cellSize)}`;
}
insert(obj) {
let key = this._key(obj.pos.x, obj.pos.y);
if (!this.cells.has(key)) this.cells.set(key, []);
this.cells.get(key).push(obj);
}
query(x, y, radius) {
let results = [];
let minCX = floor((x - radius) / this.cellSize);
let maxCX = floor((x + radius) / this.cellSize);
let minCY = floor((y - radius) / this.cellSize);
let maxCY = floor((y + radius) / this.cellSize);
for (let cx = minCX; cx <= maxCX; cx++) {
for (let cy = minCY; cy <= maxCY; cy++) {
let key = `${cx},${cy}`;
let cell = this.cells.get(key);
if (cell) {
for (let obj of cell) {
if (dist(x, y, obj.pos.x, obj.pos.y) <= radius) {
results.push(obj);
}
}
}
}
}
return results;
}
}
```
**Object pooling:**
```javascript
class ParticlePool {
constructor(maxSize) {
this.pool = [];
this.active = [];
for (let i = 0; i < maxSize; i++) {
this.pool.push(new Particle(0, 0));
}
}
spawn(x, y) {
let p = this.pool.pop();
if (p) {
p.reset(x, y);
this.active.push(p);
}
}
update() {
for (let i = this.active.length - 1; i >= 0; i--) {
this.active[i].update();
if (this.active[i].isDead()) {
this.pool.push(this.active.splice(i, 1)[0]);
}
}
}
}
```
**Throttle heavy operations:**
```javascript
// Only update flow field every N frames
if (frameCount % 5 === 0) {
flowField.update(frameCount * 0.001);
}
```
### Frame Rate Targets
| Context | Target | Acceptable |
|---------|--------|------------|
| Interactive sketch | 60fps | 30fps |
| Ambient animation | 30fps | 20fps |
| Export/recording | 30fps render | Any (offline) |
| Mobile | 30fps | 20fps |
### Per-Pixel Rendering Budgets
Pixel-level operations (`loadPixels()` loops) are the most expensive common pattern. Budget depends on canvas size and computation per pixel.
| Canvas | Pixels | Simple noise (1 call) | fBM (4 octave) | Domain warp (3-layer fBM) |
|--------|--------|----------------------|----------------|--------------------------|
| 540x540 | 291K | ~5ms | ~20ms | ~80ms |
| 1080x1080 | 1.17M | ~20ms | ~80ms | ~300ms+ |
| 1920x1080 | 2.07M | ~35ms | ~140ms | ~500ms+ |
| 3840x2160 | 8.3M | ~140ms | ~560ms | WILL CRASH |
**Rules of thumb:**
- 1 `noise()` call per pixel at 1080x1080 = ~20ms/frame (OK at 30fps)
- 4-octave fBM per pixel at 1080x1080 = ~80ms/frame (borderline)
- Multi-layer domain warp at 1080x1080 = 300ms+ (too slow for real-time, fine for `noLoop()` export)
- **Headless Chrome is 2-5x slower** than desktop Chrome for pixel ops
**Solution: render at lower resolution, fill blocks:**
```javascript
let step = 3; // render 1/9 of pixels, fill 3x3 blocks
loadPixels();
for (let y = 0; y < H; y += step) {
for (let x = 0; x < W; x += step) {
let v = expensiveNoise(x, y);
for (let dy = 0; dy < step && y+dy < H; dy++)
for (let dx = 0; dx < step && x+dx < W; dx++) {
let i = 4 * ((y+dy) * W + (x+dx));
pixels[i] = v; pixels[i+1] = v; pixels[i+2] = v; pixels[i+3] = 255;
}
}
}
updatePixels();
```
Step=2 gives 4x speedup. Step=3 gives 9x. Visible at 1080p but acceptable for video (motion hides it).
## Common Mistakes
### 1. Forgetting to reset blend mode
```javascript
blendMode(ADD);
image(glowLayer, 0, 0);
// WRONG: everything after this is ADD blended
blendMode(BLEND); // ALWAYS reset
```
### 2. Creating objects in draw()
```javascript
// BAD: creates new font object every frame
function draw() {
let f = loadFont('font.otf'); // NEVER load in draw()
}
// GOOD: load in preload, use in draw
let f;
function preload() { f = loadFont('font.otf'); }
```
### 3. Not using push()/pop() with transforms
```javascript
// BAD: transforms accumulate
translate(100, 0);
rotate(0.1);
ellipse(0, 0, 50);
// Everything after this is also translated and rotated
// GOOD: isolated transforms
push();
translate(100, 0);
rotate(0.1);
ellipse(0, 0, 50);
pop();
```
### 4. Integer coordinates for crisp lines
```javascript
// BLURRY: sub-pixel rendering
line(10.5, 20.3, 100.7, 80.2);
// CRISP: integer + 0.5 for 1px lines
line(10.5, 20.5, 100.5, 80.5); // on pixel boundary
```
### 5. Pixel density confusion
```javascript
// WRONG: assuming pixel array matches canvas dimensions
loadPixels();
let idx = 4 * (y * width + x); // wrong if pixelDensity > 1
// RIGHT: account for pixel density
let d = pixelDensity();
loadPixels();
let idx = 4 * ((y * d) * (width * d) + (x * d));
// SIMPLEST: set pixelDensity(1) at the start
```
### 6. Color mode confusion
```javascript
// In HSB mode, fill(255) is NOT white
colorMode(HSB, 360, 100, 100);
fill(255); // This is hue=255, sat=100, bri=100 = vivid purple
// White in HSB:
fill(0, 0, 100); // any hue, 0 saturation, 100 brightness
// Black in HSB:
fill(0, 0, 0);
```
### 7. WebGL origin is center
```javascript
// In WEBGL mode, (0,0) is CENTER, not top-left
function draw() {
// This draws at the center, not the corner
rect(0, 0, 100, 100);
// For top-left behavior:
translate(-width/2, -height/2);
rect(0, 0, 100, 100); // now at top-left
}
```
### 8. createGraphics cleanup
```javascript
// BAD: memory leak — buffer never freed
function draw() {
let temp = createGraphics(width, height); // new buffer every frame!
// ...
}
// GOOD: create once, reuse
let temp;
function setup() {
temp = createGraphics(width, height);
}
function draw() {
temp.clear();
// ... reuse temp
}
// If you must create/destroy:
temp.remove(); // explicitly free
```
### 9. noise() returns 0-1, not -1 to 1
```javascript
let n = noise(x); // 0.0 to 1.0 (biased toward 0.5)
// For -1 to 1 range:
let n = noise(x) * 2 - 1;
// For a specific range:
let n = map(noise(x), 0, 1, -100, 100);
```
### 10. saveCanvas() in draw() saves every frame
```javascript
// BAD: saves a PNG every single frame
function draw() {
// ... render ...
saveCanvas('output', 'png'); // DON'T DO THIS
}
// GOOD: save once via keyboard
function keyPressed() {
if (key === 's') saveCanvas('output', 'png');
}
// GOOD: save once after rendering static piece
function draw() {
// ... render ...
saveCanvas('output', 'png');
noLoop(); // stop after saving
}
```
### 11. console.log() in draw()
```javascript
// BAD: writes to DOM console every frame — massive overhead
function draw() {
console.log(particles.length); // 60 DOM writes/second
}
// GOOD: log periodically or conditionally
function draw() {
if (frameCount % 60 === 0) console.log('FPS:', frameRate().toFixed(1));
}
```
### 12. DOM manipulation in draw()
```javascript
// BAD: layout thrashing — 400-500x slower than canvas ops
function draw() {
document.getElementById('counter').innerText = frameCount;
let el = document.querySelector('.info'); // DOM query per frame
}
// GOOD: cache DOM refs, update infrequently
let counterEl;
function setup() { counterEl = document.getElementById('counter'); }
function draw() {
if (frameCount % 30 === 0) counterEl.innerText = frameCount;
}
```
### 13. Not disabling FES in production
```javascript
// BAD: every p5 function call has error-checking overhead (up to 10x slower)
function setup() { createCanvas(800, 800); }
// GOOD: disable before any p5 code
p5.disableFriendlyErrors = true;
function setup() { createCanvas(800, 800); }
// ALSO GOOD: use p5.min.js (FES stripped from minified build)
```
## Browser Compatibility
### Safari Issues
- WebGL shader precision: always declare `precision mediump float;`
- `AudioContext` requires user gesture (`userStartAudio()`)
- Some `blendMode()` options behave differently
### Firefox Issues
- `textToPoints()` may return slightly different point counts
- WebGL extensions may differ from Chrome
- Color profile handling can shift colors
### Mobile Issues
- Touch events need `return false` to prevent scroll
- `devicePixelRatio` can be 2x or 3x — use `pixelDensity(1)` for performance
- Smaller canvas recommended (720p or less)
- Audio requires explicit user gesture to start
## CORS Issues
```javascript
// Loading images/fonts from external URLs requires CORS headers
// Local files need a server:
// python3 -m http.server 8080
// Or use a CORS proxy for external resources (not recommended for production)
```
## Memory Leaks
### Symptoms
- Framerate degrading over time
- Browser tab memory growing unbounded
- Page becomes unresponsive after minutes
### Common Causes
```javascript
// 1. Growing arrays
let history = [];
function draw() {
history.push(someData); // grows forever
}
// FIX: cap the array
if (history.length > 1000) history.shift();
// 2. Creating p5 objects in draw()
function draw() {
let v = createVector(0, 0); // allocation every frame
}
// FIX: reuse pre-allocated objects
// 3. Unreleased graphics buffers
let layers = [];
function reset() {
for (let l of layers) l.remove(); // free old buffers
layers = [];
}
// 4. Event listener accumulation
function setup() {
// BAD: adds new listener every time setup runs
window.addEventListener('resize', handler);
}
// FIX: use p5's built-in windowResized()
```
## Debugging Tips
### Console Logging
```javascript
// Log once (not every frame)
if (frameCount === 1) {
console.log('Canvas:', width, 'x', height);
console.log('Pixel density:', pixelDensity());
console.log('Renderer:', drawingContext.constructor.name);
}
// Log periodically
if (frameCount % 60 === 0) {
console.log('FPS:', frameRate().toFixed(1));
console.log('Particles:', particles.length);
}
```
### Visual Debugging
```javascript
// Show frame rate
function draw() {
// ... your sketch ...
if (CONFIG.debug) {
fill(255, 0, 0);
noStroke();
textSize(14);
textAlign(LEFT, TOP);
text('FPS: ' + frameRate().toFixed(1), 10, 10);
text('Particles: ' + particles.length, 10, 28);
text('Frame: ' + frameCount, 10, 46);
}
}
// Toggle debug with 'd' key
function keyPressed() {
if (key === 'd') CONFIG.debug = !CONFIG.debug;
}
```
### Isolating Issues
```javascript
// Comment out layers to find the slow one
function draw() {
renderBackground(); // comment out to test
// renderParticles(); // this might be slow
// renderPostEffects(); // or this
}
```
@@ -0,0 +1,302 @@
# Typography
## Loading Fonts
### System Fonts
```javascript
textFont('Helvetica');
textFont('Georgia');
textFont('monospace');
```
### Custom Fonts (OTF/TTF/WOFF2)
```javascript
let myFont;
function preload() {
myFont = loadFont('path/to/font.otf');
// Requires local server or CORS-enabled URL
}
function setup() {
textFont(myFont);
}
```
### Google Fonts via CSS
```html
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;700&display=swap" rel="stylesheet">
<script>
function setup() {
textFont('Inter');
}
</script>
```
Google Fonts work without `loadFont()` but only for `text()` — not for `textToPoints()`. For particle text, you need `loadFont()` with an OTF/TTF file.
## Text Rendering
### Basic Text
```javascript
textSize(32);
textAlign(CENTER, CENTER);
text('Hello World', width/2, height/2);
```
### Text Properties
```javascript
textSize(48); // pixel size
textAlign(LEFT, TOP); // horizontal: LEFT, CENTER, RIGHT
// vertical: TOP, CENTER, BOTTOM, BASELINE
textLeading(40); // line spacing (for multi-line text)
textStyle(BOLD); // NORMAL, BOLD, ITALIC, BOLDITALIC
textWrap(WORD); // WORD or CHAR (for text() with max width)
```
### Text Metrics
```javascript
let w = textWidth('Hello'); // pixel width of string
let a = textAscent(); // height above baseline
let d = textDescent(); // height below baseline
let totalH = a + d; // full line height
```
### Text Bounding Box
```javascript
let bounds = myFont.textBounds('Hello', x, y, size);
// bounds = { x, y, w, h }
// Useful for positioning, collision, background rectangles
```
### Multi-Line Text
```javascript
// With max width — auto wraps
textWrap(WORD);
text('Long text that wraps within the given width', x, y, maxWidth);
// With max width AND height — clips
text('Very long text', x, y, maxWidth, maxHeight);
```
## textToPoints() — Text as Particles
Convert text outline to array of points. Requires a loaded font (OTF/TTF via `loadFont()`).
```javascript
let font;
let points;
function preload() {
font = loadFont('font.otf'); // MUST be loadFont, not CSS
}
function setup() {
createCanvas(1200, 600);
points = font.textToPoints('HELLO', 100, 400, 200, {
sampleFactor: 0.1, // lower = more points (0.1-0.5 typical)
simplifyThreshold: 0
});
}
function draw() {
background(0);
for (let pt of points) {
let n = noise(pt.x * 0.01, pt.y * 0.01, frameCount * 0.01);
fill(255, n * 255);
noStroke();
ellipse(pt.x + random(-2, 2), pt.y + random(-2, 2), 3);
}
}
```
### Particle Text Class
```javascript
class TextParticle {
constructor(target) {
this.target = createVector(target.x, target.y);
this.pos = createVector(random(width), random(height));
this.vel = createVector(0, 0);
this.acc = createVector(0, 0);
this.maxSpeed = 10;
this.maxForce = 0.5;
}
arrive() {
let desired = p5.Vector.sub(this.target, this.pos);
let d = desired.mag();
let speed = d < 100 ? map(d, 0, 100, 0, this.maxSpeed) : this.maxSpeed;
desired.setMag(speed);
let steer = p5.Vector.sub(desired, this.vel);
steer.limit(this.maxForce);
this.acc.add(steer);
}
flee(target, radius) {
let d = this.pos.dist(target);
if (d < radius) {
let desired = p5.Vector.sub(this.pos, target);
desired.setMag(this.maxSpeed);
let steer = p5.Vector.sub(desired, this.vel);
steer.limit(this.maxForce * 2);
this.acc.add(steer);
}
}
update() {
this.vel.add(this.acc);
this.vel.limit(this.maxSpeed);
this.pos.add(this.vel);
this.acc.mult(0);
}
display() {
fill(255);
noStroke();
ellipse(this.pos.x, this.pos.y, 3);
}
}
// Usage: particles form text, scatter from mouse
let textParticles = [];
for (let pt of points) {
textParticles.push(new TextParticle(pt));
}
function draw() {
background(0);
for (let p of textParticles) {
p.arrive();
p.flee(createVector(mouseX, mouseY), 80);
p.update();
p.display();
}
}
```
## Kinetic Typography
### Wave Text
```javascript
function waveText(str, x, y, size, amplitude, frequency) {
textSize(size);
textAlign(LEFT, BASELINE);
let xOff = 0;
for (let i = 0; i < str.length; i++) {
let yOff = sin(frameCount * 0.05 + i * frequency) * amplitude;
text(str[i], x + xOff, y + yOff);
xOff += textWidth(str[i]);
}
}
```
### Typewriter Effect
```javascript
class Typewriter {
constructor(str, x, y, speed = 50) {
this.str = str;
this.x = x;
this.y = y;
this.speed = speed; // ms per character
this.startTime = millis();
this.cursor = true;
}
display() {
let elapsed = millis() - this.startTime;
let chars = min(floor(elapsed / this.speed), this.str.length);
let visible = this.str.substring(0, chars);
textAlign(LEFT, TOP);
text(visible, this.x, this.y);
// Blinking cursor
if (chars < this.str.length && floor(millis() / 500) % 2 === 0) {
let cursorX = this.x + textWidth(visible);
line(cursorX, this.y, cursorX, this.y + textAscent() + textDescent());
}
}
isDone() { return millis() - this.startTime >= this.str.length * this.speed; }
}
```
### Character-by-Character Animation
```javascript
function animatedText(str, x, y, size, delay = 50) {
textSize(size);
textAlign(LEFT, BASELINE);
let xOff = 0;
for (let i = 0; i < str.length; i++) {
let charStart = i * delay;
let t = constrain((millis() - charStart) / 500, 0, 1);
let et = easeOutElastic(t);
push();
translate(x + xOff, y);
scale(et);
let alpha = t * 255;
fill(255, alpha);
text(str[i], 0, 0);
pop();
xOff += textWidth(str[i]);
}
}
```
## Text as Mask
```javascript
let textBuffer;
function setup() {
createCanvas(800, 800);
textBuffer = createGraphics(width, height);
textBuffer.background(0);
textBuffer.fill(255);
textBuffer.textSize(200);
textBuffer.textAlign(CENTER, CENTER);
textBuffer.text('MASK', width/2, height/2);
}
function draw() {
// Draw content
background(0);
// ... render something colorful
// Apply text mask (show content only where text is white)
loadPixels();
textBuffer.loadPixels();
for (let i = 0; i < pixels.length; i += 4) {
let maskVal = textBuffer.pixels[i]; // white = show, black = hide
pixels[i + 3] = maskVal; // set alpha from mask
}
updatePixels();
}
```
## Responsive Text Sizing
```javascript
function responsiveTextSize(baseSize, baseWidth = 1920) {
return baseSize * (width / baseWidth);
}
// Usage
textSize(responsiveTextSize(48));
text('Scales with canvas', width/2, height/2);
```
@@ -0,0 +1,895 @@
# Visual Effects
## Noise
### Perlin Noise Basics
```javascript
noiseSeed(42);
noiseDetail(4, 0.5); // octaves, falloff
// 1D noise — smooth undulation
let y = noise(x * 0.01); // returns 0.0 to 1.0
// 2D noise — terrain/texture
let v = noise(x * 0.005, y * 0.005);
// 3D noise — animated 2D field (z = time)
let v = noise(x * 0.005, y * 0.005, frameCount * 0.005);
```
The scale factor (0.005 etc.) is critical:
- `0.001` — very smooth, large features
- `0.005` — smooth, medium features
- `0.01` — standard generative art scale
- `0.05` — detailed, small features
- `0.1` — near-random, grainy
### Fractal Brownian Motion (fBM)
Layered noise octaves for natural-looking texture. Each octave adds detail at smaller scale.
```javascript
function fbm(x, y, octaves = 6, lacunarity = 2.0, gain = 0.5) {
let value = 0;
let amplitude = 1.0;
let frequency = 1.0;
let maxValue = 0;
for (let i = 0; i < octaves; i++) {
value += noise(x * frequency, y * frequency) * amplitude;
maxValue += amplitude;
amplitude *= gain;
frequency *= lacunarity;
}
return value / maxValue;
}
```
### Domain Warping
Feed noise output back as input coordinates for flowing organic distortion.
```javascript
function domainWarp(x, y, scale, strength, time) {
// First warp pass
let qx = fbm(x + 0.0, y + 0.0);
let qy = fbm(x + 5.2, y + 1.3);
// Second warp pass (feed back)
let rx = fbm(x + strength * qx + 1.7, y + strength * qy + 9.2, 4, 2, 0.5);
let ry = fbm(x + strength * qx + 8.3, y + strength * qy + 2.8, 4, 2, 0.5);
return fbm(x + strength * rx + time, y + strength * ry + time);
}
```
### Curl Noise
Divergence-free noise field. Particles following curl noise never converge or diverge — they flow in smooth, swirling patterns.
```javascript
function curlNoise(x, y, scale, time) {
let eps = 0.001;
// Partial derivatives via finite differences
let dndx = (noise(x * scale + eps, y * scale, time) -
noise(x * scale - eps, y * scale, time)) / (2 * eps);
let dndy = (noise(x * scale, y * scale + eps, time) -
noise(x * scale, y * scale - eps, time)) / (2 * eps);
// Curl = perpendicular to gradient
return createVector(dndy, -dndx);
}
```
## Flow Fields
A grid of vectors that steer particles. The foundational generative art technique.
```javascript
class FlowField {
constructor(resolution, noiseScale) {
this.resolution = resolution;
this.cols = ceil(width / resolution);
this.rows = ceil(height / resolution);
this.field = new Array(this.cols * this.rows);
this.noiseScale = noiseScale;
}
update(time) {
for (let i = 0; i < this.cols; i++) {
for (let j = 0; j < this.rows; j++) {
let angle = noise(i * this.noiseScale, j * this.noiseScale, time) * TWO_PI * 2;
this.field[i + j * this.cols] = p5.Vector.fromAngle(angle);
}
}
}
lookup(x, y) {
let col = constrain(floor(x / this.resolution), 0, this.cols - 1);
let row = constrain(floor(y / this.resolution), 0, this.rows - 1);
return this.field[col + row * this.cols].copy();
}
}
```
### Flow Field Particle
```javascript
class FlowParticle {
constructor(x, y) {
this.pos = createVector(x, y);
this.vel = createVector(0, 0);
this.acc = createVector(0, 0);
this.prev = this.pos.copy();
this.maxSpeed = 2;
this.life = 1.0;
}
follow(field) {
let force = field.lookup(this.pos.x, this.pos.y);
force.mult(0.5); // force magnitude
this.acc.add(force);
}
update() {
this.prev = this.pos.copy();
this.vel.add(this.acc);
this.vel.limit(this.maxSpeed);
this.pos.add(this.vel);
this.acc.mult(0);
this.life -= 0.001;
}
edges() {
if (this.pos.x > width) this.pos.x = 0;
if (this.pos.x < 0) this.pos.x = width;
if (this.pos.y > height) this.pos.y = 0;
if (this.pos.y < 0) this.pos.y = height;
this.prev = this.pos.copy(); // prevent wrap line
}
display(buffer) {
buffer.stroke(255, this.life * 30);
buffer.strokeWeight(0.5);
buffer.line(this.prev.x, this.prev.y, this.pos.x, this.pos.y);
}
}
```
## Particle Systems
### Basic Physics Particle
```javascript
class Particle {
constructor(x, y) {
this.pos = createVector(x, y);
this.vel = p5.Vector.random2D().mult(random(1, 3));
this.acc = createVector(0, 0);
this.life = 255;
this.decay = random(1, 5);
this.size = random(3, 8);
}
applyForce(f) { this.acc.add(f); }
update() {
this.vel.add(this.acc);
this.pos.add(this.vel);
this.acc.mult(0);
this.life -= this.decay;
}
display() {
noStroke();
fill(255, this.life);
ellipse(this.pos.x, this.pos.y, this.size);
}
isDead() { return this.life <= 0; }
}
```
### Attractor-Driven Particles
```javascript
class Attractor {
constructor(x, y, strength) {
this.pos = createVector(x, y);
this.strength = strength;
}
attract(particle) {
let force = p5.Vector.sub(this.pos, particle.pos);
let d = constrain(force.mag(), 5, 200);
force.normalize();
force.mult(this.strength / (d * d));
particle.applyForce(force);
}
}
```
### Boid Flocking
```javascript
class Boid {
constructor(x, y) {
this.pos = createVector(x, y);
this.vel = p5.Vector.random2D().mult(random(2, 4));
this.acc = createVector(0, 0);
this.maxForce = 0.2;
this.maxSpeed = 4;
this.perceptionRadius = 50;
}
flock(boids) {
let alignment = createVector(0, 0);
let cohesion = createVector(0, 0);
let separation = createVector(0, 0);
let total = 0;
for (let other of boids) {
let d = this.pos.dist(other.pos);
if (other !== this && d < this.perceptionRadius) {
alignment.add(other.vel);
cohesion.add(other.pos);
let diff = p5.Vector.sub(this.pos, other.pos);
diff.div(d * d);
separation.add(diff);
total++;
}
}
if (total > 0) {
alignment.div(total).setMag(this.maxSpeed).sub(this.vel).limit(this.maxForce);
cohesion.div(total).sub(this.pos).setMag(this.maxSpeed).sub(this.vel).limit(this.maxForce);
separation.div(total).setMag(this.maxSpeed).sub(this.vel).limit(this.maxForce);
}
this.acc.add(alignment.mult(1.0));
this.acc.add(cohesion.mult(1.0));
this.acc.add(separation.mult(1.5));
}
update() {
this.vel.add(this.acc);
this.vel.limit(this.maxSpeed);
this.pos.add(this.vel);
this.acc.mult(0);
}
}
```
## Pixel Manipulation
### Reading and Writing Pixels
```javascript
loadPixels();
for (let y = 0; y < height; y++) {
for (let x = 0; x < width; x++) {
let idx = 4 * (y * width + x);
let r = pixels[idx];
let g = pixels[idx + 1];
let b = pixels[idx + 2];
let a = pixels[idx + 3];
// Modify
pixels[idx] = 255 - r; // invert red
pixels[idx + 1] = 255 - g; // invert green
pixels[idx + 2] = 255 - b; // invert blue
}
}
updatePixels();
```
### Pixel-Level Noise Texture
```javascript
loadPixels();
for (let i = 0; i < pixels.length; i += 4) {
let x = (i / 4) % width;
let y = floor((i / 4) / width);
let n = noise(x * 0.01, y * 0.01, frameCount * 0.02);
let c = n * 255;
pixels[i] = c;
pixels[i + 1] = c;
pixels[i + 2] = c;
pixels[i + 3] = 255;
}
updatePixels();
```
### Built-in Filters
```javascript
filter(BLUR, 3); // Gaussian blur (radius)
filter(THRESHOLD, 0.5); // Black/white threshold
filter(INVERT); // Color inversion
filter(POSTERIZE, 4); // Reduce color levels
filter(GRAY); // Desaturate
filter(ERODE); // Thin bright areas
filter(DILATE); // Expand bright areas
filter(OPAQUE); // Remove transparency
```
## Texture Generation
### Stippling / Pointillism
```javascript
function stipple(buffer, density, minSize, maxSize) {
buffer.loadPixels();
for (let i = 0; i < density; i++) {
let x = floor(random(width));
let y = floor(random(height));
let idx = 4 * (y * width + x);
let brightness = (buffer.pixels[idx] + buffer.pixels[idx+1] + buffer.pixels[idx+2]) / 3;
let size = map(brightness, 0, 255, maxSize, minSize);
if (random() < map(brightness, 0, 255, 0.8, 0.1)) {
noStroke();
fill(buffer.pixels[idx], buffer.pixels[idx+1], buffer.pixels[idx+2]);
ellipse(x, y, size);
}
}
}
```
### Halftone
```javascript
function halftone(sourceBuffer, dotSpacing, maxDotSize) {
sourceBuffer.loadPixels();
background(255);
fill(0);
noStroke();
for (let y = 0; y < height; y += dotSpacing) {
for (let x = 0; x < width; x += dotSpacing) {
let idx = 4 * (y * width + x);
let brightness = (sourceBuffer.pixels[idx] + sourceBuffer.pixels[idx+1] + sourceBuffer.pixels[idx+2]) / 3;
let dotSize = map(brightness, 0, 255, maxDotSize, 0);
ellipse(x + dotSpacing/2, y + dotSpacing/2, dotSize);
}
}
}
```
### Cross-Hatching
```javascript
function crossHatch(x, y, w, h, value, spacing) {
// value: 0 (dark) to 1 (light)
let numLayers = floor(map(value, 0, 1, 4, 0));
let angles = [PI/4, -PI/4, 0, PI/2];
for (let layer = 0; layer < numLayers; layer++) {
push();
translate(x + w/2, y + h/2);
rotate(angles[layer]);
let s = spacing + layer * 2;
for (let i = -max(w, h); i < max(w, h); i += s) {
line(i, -max(w, h), i, max(w, h));
}
pop();
}
}
```
## Feedback Loops
### Frame Feedback (Echo/Trail)
```javascript
let feedback;
function setup() {
createCanvas(800, 800);
feedback = createGraphics(width, height);
}
function draw() {
// Copy current feedback, slightly zoomed and rotated
let temp = feedback.get();
feedback.push();
feedback.translate(width/2, height/2);
feedback.scale(1.005); // slow zoom
feedback.rotate(0.002); // slow rotation
feedback.translate(-width/2, -height/2);
feedback.tint(255, 245); // slight fade
feedback.image(temp, 0, 0);
feedback.pop();
// Draw new content to feedback
feedback.noStroke();
feedback.fill(255);
feedback.ellipse(mouseX, mouseY, 20);
// Show
image(feedback, 0, 0);
}
```
### Bloom / Glow (Post-Processing)
Downsample the scene to a small buffer, blur it, overlay additively. Creates soft glow around bright areas. This is the standard generative art bloom technique.
```javascript
let scene, bloomBuf;
function setup() {
createCanvas(1080, 1080);
scene = createGraphics(width, height);
bloomBuf = createGraphics(width, height);
}
function draw() {
// 1. Render scene to offscreen buffer
scene.background(0);
scene.fill(255, 200, 100);
scene.noStroke();
// ... draw bright elements to scene ...
// 2. Build bloom: downsample → blur → upscale
bloomBuf.clear();
bloomBuf.image(scene, 0, 0, width / 4, height / 4); // 4x downsample
bloomBuf.filter(BLUR, 6); // blur the small version
// 3. Composite: scene + additive bloom
background(0);
image(scene, 0, 0); // base layer
blendMode(ADD); // additive = glow
tint(255, 80); // control bloom intensity (0-255)
image(bloomBuf, 0, 0, width, height); // upscale back to full size
noTint();
blendMode(BLEND); // ALWAYS reset blend mode
}
```
**Tuning:**
- Downsample ratio (1/4 is standard, 1/8 for softer, 1/2 for tighter)
- Blur radius (4-8 typical, higher = wider glow)
- Tint alpha (40-120, controls glow intensity)
- Update bloom every N frames to save perf: `if (frameCount % 2 === 0) { ... }`
**Common mistake:** Forgetting `blendMode(BLEND)` after the ADD pass — everything drawn after will be additive.
### Trail Buffer Brightness
Trail accumulation via `createGraphics()` + semi-transparent fade rect is the standard technique for particle trails, but **trails are always dimmer than you expect**. The fade rect's alpha compounds multiplicatively every frame.
```javascript
// The fade rect alpha controls trail length AND brightness:
trailBuf.fill(0, 0, 0, alpha);
trailBuf.rect(0, 0, width, height);
// alpha=5 → very long trails, very dim (content fades to 50% in ~35 frames)
// alpha=10 → long trails, dim
// alpha=20 → medium trails, visible
// alpha=40 → short trails, bright
// alpha=80 → very short trails, crisp
```
**The trap:** You set alpha=5 for long trails, but particle strokes at alpha=30 are invisible because they fade before accumulating enough density. Either:
- **Boost stroke alpha** to 80-150 (not the intuitive 20-40)
- **Reduce fade alpha** but accept shorter trails
- **Use additive blending** for the strokes: bright particles accumulate, dim ones stay dark
```javascript
// WRONG: low fade + low stroke = invisible
trailBuf.fill(0, 0, 0, 5); // long trails
trailBuf.rect(0, 0, W, H);
trailBuf.stroke(255, 30); // too dim to ever accumulate
trailBuf.line(px, py, x, y);
// RIGHT: low fade + high stroke = visible long trails
trailBuf.fill(0, 0, 0, 5);
trailBuf.rect(0, 0, W, H);
trailBuf.stroke(255, 100); // bright enough to persist through fade
trailBuf.line(px, py, x, y);
```
### Reaction-Diffusion (Gray-Scott)
```javascript
class ReactionDiffusion {
constructor(w, h) {
this.w = w;
this.h = h;
this.a = new Float32Array(w * h).fill(1);
this.b = new Float32Array(w * h).fill(0);
this.nextA = new Float32Array(w * h);
this.nextB = new Float32Array(w * h);
this.dA = 1.0;
this.dB = 0.5;
this.feed = 0.055;
this.kill = 0.062;
}
seed(cx, cy, r) {
for (let y = cy - r; y < cy + r; y++) {
for (let x = cx - r; x < cx + r; x++) {
if (dist(x, y, cx, cy) < r) {
let idx = y * this.w + x;
this.b[idx] = 1;
}
}
}
}
step() {
for (let y = 1; y < this.h - 1; y++) {
for (let x = 1; x < this.w - 1; x++) {
let idx = y * this.w + x;
let a = this.a[idx], b = this.b[idx];
let lapA = this.laplacian(this.a, x, y);
let lapB = this.laplacian(this.b, x, y);
let abb = a * b * b;
this.nextA[idx] = constrain(a + this.dA * lapA - abb + this.feed * (1 - a), 0, 1);
this.nextB[idx] = constrain(b + this.dB * lapB + abb - (this.kill + this.feed) * b, 0, 1);
}
}
[this.a, this.nextA] = [this.nextA, this.a];
[this.b, this.nextB] = [this.nextB, this.b];
}
laplacian(arr, x, y) {
let w = this.w;
return arr[(y-1)*w+x] + arr[(y+1)*w+x] + arr[y*w+(x-1)] + arr[y*w+(x+1)]
- 4 * arr[y*w+x];
}
}
```
## Pixel Sorting
```javascript
function pixelSort(buffer, threshold, direction = 'horizontal') {
buffer.loadPixels();
let px = buffer.pixels;
if (direction === 'horizontal') {
for (let y = 0; y < height; y++) {
let spans = findSpans(px, y, width, threshold, true);
for (let span of spans) {
sortSpan(px, span.start, span.end, y, true);
}
}
}
buffer.updatePixels();
}
function findSpans(px, row, w, threshold, horizontal) {
let spans = [];
let start = -1;
for (let i = 0; i < w; i++) {
let idx = horizontal ? 4 * (row * w + i) : 4 * (i * w + row);
let brightness = (px[idx] + px[idx+1] + px[idx+2]) / 3;
if (brightness > threshold && start === -1) {
start = i;
} else if (brightness <= threshold && start !== -1) {
spans.push({ start, end: i });
start = -1;
}
}
if (start !== -1) spans.push({ start, end: w });
return spans;
}
```
## Advanced Generative Techniques
### L-Systems (Lindenmayer Systems)
Grammar-based recursive growth for trees, plants, fractals.
```javascript
class LSystem {
constructor(axiom, rules) {
this.axiom = axiom;
this.rules = rules; // { 'F': 'F[+F]F[-F]F' }
this.sentence = axiom;
}
generate(iterations) {
for (let i = 0; i < iterations; i++) {
let next = '';
for (let ch of this.sentence) {
next += this.rules[ch] || ch;
}
this.sentence = next;
}
}
draw(len, angle) {
for (let ch of this.sentence) {
switch (ch) {
case 'F': line(0, 0, 0, -len); translate(0, -len); break;
case '+': rotate(angle); break;
case '-': rotate(-angle); break;
case '[': push(); break;
case ']': pop(); break;
}
}
}
}
// Usage: fractal plant
let lsys = new LSystem('X', {
'X': 'F+[[X]-X]-F[-FX]+X',
'F': 'FF'
});
lsys.generate(5);
translate(width/2, height);
lsys.draw(4, radians(25));
```
### Circle Packing
Fill a space with non-overlapping circles of varying size.
```javascript
class PackedCircle {
constructor(x, y, r) {
this.x = x; this.y = y; this.r = r;
this.growing = true;
}
grow() { if (this.growing) this.r += 0.5; }
overlaps(other) {
let d = dist(this.x, this.y, other.x, other.y);
return d < this.r + other.r + 2; // +2 gap
}
atEdge() {
return this.x - this.r < 0 || this.x + this.r > width ||
this.y - this.r < 0 || this.y + this.r > height;
}
}
let circles = [];
function packStep() {
// Try to place new circle
for (let attempts = 0; attempts < 100; attempts++) {
let x = random(width), y = random(height);
let valid = true;
for (let c of circles) {
if (dist(x, y, c.x, c.y) < c.r + 2) { valid = false; break; }
}
if (valid) { circles.push(new PackedCircle(x, y, 1)); break; }
}
// Grow existing circles
for (let c of circles) {
if (!c.growing) continue;
c.grow();
if (c.atEdge()) { c.growing = false; continue; }
for (let other of circles) {
if (c !== other && c.overlaps(other)) { c.growing = false; break; }
}
}
}
```
### Voronoi Diagram (Fortune's Algorithm Approximation)
```javascript
// Simple brute-force Voronoi (for small point counts)
function drawVoronoi(points, colors) {
loadPixels();
for (let y = 0; y < height; y++) {
for (let x = 0; x < width; x++) {
let minDist = Infinity;
let closest = 0;
for (let i = 0; i < points.length; i++) {
let d = (x - points[i].x) ** 2 + (y - points[i].y) ** 2; // magSq
if (d < minDist) { minDist = d; closest = i; }
}
let idx = 4 * (y * width + x);
let c = colors[closest % colors.length];
pixels[idx] = red(c);
pixels[idx+1] = green(c);
pixels[idx+2] = blue(c);
pixels[idx+3] = 255;
}
}
updatePixels();
}
```
### Fractal Trees
```javascript
function fractalTree(x, y, len, angle, depth, branchAngle) {
if (depth <= 0 || len < 2) return;
let x2 = x + Math.cos(angle) * len;
let y2 = y + Math.sin(angle) * len;
strokeWeight(map(depth, 0, 10, 0.5, 4));
line(x, y, x2, y2);
let shrink = 0.67 + noise(x * 0.01, y * 0.01) * 0.15;
fractalTree(x2, y2, len * shrink, angle - branchAngle, depth - 1, branchAngle);
fractalTree(x2, y2, len * shrink, angle + branchAngle, depth - 1, branchAngle);
}
// Usage
fractalTree(width/2, height, 120, -HALF_PI, 10, PI/6);
```
### Strange Attractors
```javascript
// Clifford Attractor
function cliffordAttractor(a, b, c, d, iterations) {
let x = 0, y = 0;
beginShape(POINTS);
for (let i = 0; i < iterations; i++) {
let nx = Math.sin(a * y) + c * Math.cos(a * x);
let ny = Math.sin(b * x) + d * Math.cos(b * y);
x = nx; y = ny;
let px = map(x, -3, 3, 0, width);
let py = map(y, -3, 3, 0, height);
vertex(px, py);
}
endShape();
}
// De Jong Attractor
function deJongAttractor(a, b, c, d, iterations) {
let x = 0, y = 0;
beginShape(POINTS);
for (let i = 0; i < iterations; i++) {
let nx = Math.sin(a * y) - Math.cos(b * x);
let ny = Math.sin(c * x) - Math.cos(d * y);
x = nx; y = ny;
let px = map(x, -2.5, 2.5, 0, width);
let py = map(y, -2.5, 2.5, 0, height);
vertex(px, py);
}
endShape();
}
```
### Poisson Disk Sampling
Even distribution that looks natural — better than pure random for placing elements.
```javascript
function poissonDiskSampling(r, k = 30) {
let cellSize = r / Math.sqrt(2);
let cols = Math.ceil(width / cellSize);
let rows = Math.ceil(height / cellSize);
let grid = new Array(cols * rows).fill(-1);
let points = [];
let active = [];
function gridIndex(x, y) {
return Math.floor(x / cellSize) + Math.floor(y / cellSize) * cols;
}
// Seed
let p0 = createVector(random(width), random(height));
points.push(p0);
active.push(p0);
grid[gridIndex(p0.x, p0.y)] = 0;
while (active.length > 0) {
let idx = Math.floor(Math.random() * active.length);
let pos = active[idx];
let found = false;
for (let n = 0; n < k; n++) {
let angle = Math.random() * TWO_PI;
let mag = r + Math.random() * r;
let sample = createVector(pos.x + Math.cos(angle) * mag, pos.y + Math.sin(angle) * mag);
if (sample.x < 0 || sample.x >= width || sample.y < 0 || sample.y >= height) continue;
let col = Math.floor(sample.x / cellSize);
let row = Math.floor(sample.y / cellSize);
let ok = true;
for (let dy = -2; dy <= 2; dy++) {
for (let dx = -2; dx <= 2; dx++) {
let nc = col + dx, nr = row + dy;
if (nc >= 0 && nc < cols && nr >= 0 && nr < rows) {
let gi = nc + nr * cols;
if (grid[gi] !== -1 && points[grid[gi]].dist(sample) < r) { ok = false; }
}
}
}
if (ok) {
points.push(sample);
active.push(sample);
grid[gridIndex(sample.x, sample.y)] = points.length - 1;
found = true;
break;
}
}
if (!found) active.splice(idx, 1);
}
return points;
}
```
## Addon Libraries
### p5.brush — Natural Media
Hand-drawn, organic aesthetics. Watercolor, charcoal, pen, marker. Requires **p5.js 2.x + WEBGL**.
```html
<script src="https://cdn.jsdelivr.net/npm/p5.brush@latest/dist/p5.brush.js"></script>
```
```javascript
function setup() {
createCanvas(1200, 1200, WEBGL);
brush.scaleBrushes(3); // essential for proper sizing
translate(-width/2, -height/2); // WEBGL origin is center
brush.pick('2B'); // pencil brush
brush.stroke(50, 50, 50);
brush.strokeWeight(2);
brush.line(100, 100, 500, 500);
brush.pick('watercolor');
brush.fill('#4a90d9', 150);
brush.circle(400, 400, 200);
}
```
Built-in brushes: `2B`, `HB`, `2H`, `cpencil`, `pen`, `rotring`, `spray`, `marker`, `charcoal`, `hatch_brush`.
Built-in vector fields: `hand`, `curved`, `zigzag`, `waves`, `seabed`, `spiral`, `columns`.
### p5.grain — Film Grain & Texture
```html
<script src="https://cdn.jsdelivr.net/npm/p5.grain@0.7.0/p5.grain.min.js"></script>
```
```javascript
function draw() {
// ... render scene ...
applyMonochromaticGrain(42); // uniform grain
// or: applyChromaticGrain(42); // per-channel randomization
}
```
### CCapture.js — Deterministic Video Capture
Records canvas at fixed framerate regardless of actual render speed. Essential for complex generative art.
```html
<script src="https://cdn.jsdelivr.net/npm/ccapture.js-npmfixed/build/CCapture.all.min.js"></script>
```
```javascript
let capturer;
function setup() {
createCanvas(1920, 1080);
capturer = new CCapture({
format: 'webm',
framerate: 60,
quality: 99,
// timeLimit: 10, // auto-stop after N seconds
// motionBlurFrames: 4 // supersampled motion blur
});
}
function startRecording() {
capturer.start();
}
function draw() {
// ... render frame ...
if (capturer) capturer.capture(document.querySelector('canvas'));
}
function stopRecording() {
capturer.stop();
capturer.save(); // triggers download
}
```
@@ -0,0 +1,423 @@
# WebGL and 3D
## WebGL Mode Setup
```javascript
function setup() {
createCanvas(1920, 1080, WEBGL);
// Origin is CENTER, not top-left
// Y-axis points UP (opposite of 2D mode)
// Z-axis points toward viewer
}
```
### Coordinate Conversion (WEBGL to P2D-like)
```javascript
function draw() {
translate(-width/2, -height/2); // shift origin to top-left
// Now coordinates work like P2D
}
```
## 3D Primitives
```javascript
box(w, h, d); // rectangular prism
sphere(radius, detailX, detailY);
cylinder(radius, height, detailX, detailY);
cone(radius, height, detailX, detailY);
torus(radius, tubeRadius, detailX, detailY);
plane(width, height); // flat rectangle
ellipsoid(rx, ry, rz); // stretched sphere
```
### 3D Transforms
```javascript
push();
translate(x, y, z);
rotateX(angleX);
rotateY(angleY);
rotateZ(angleZ);
scale(s);
box(100);
pop();
```
## Camera
### Default Camera
```javascript
camera(
eyeX, eyeY, eyeZ, // camera position
centerX, centerY, centerZ, // look-at target
upX, upY, upZ // up direction
);
// Default: camera(0, 0, (height/2)/tan(PI/6), 0, 0, 0, 0, 1, 0)
```
### Orbit Control
```javascript
function draw() {
orbitControl(); // mouse drag to rotate, scroll to zoom
box(200);
}
```
### createCamera
```javascript
let cam;
function setup() {
createCanvas(800, 800, WEBGL);
cam = createCamera();
cam.setPosition(300, -200, 500);
cam.lookAt(0, 0, 0);
}
// Camera methods
cam.setPosition(x, y, z);
cam.lookAt(x, y, z);
cam.move(dx, dy, dz); // relative to camera orientation
cam.pan(angle); // horizontal rotation
cam.tilt(angle); // vertical rotation
cam.roll(angle); // z-axis rotation
cam.slerp(otherCam, t); // smooth interpolation between cameras
```
### Perspective and Orthographic
```javascript
// Perspective (default)
perspective(fov, aspect, near, far);
// fov: field of view in radians (PI/3 default)
// aspect: width/height
// near/far: clipping planes
// Orthographic (no depth foreshortening)
ortho(-width/2, width/2, -height/2, height/2, 0, 2000);
```
## Lighting
```javascript
// Ambient (uniform, no direction)
ambientLight(50, 50, 50); // dim fill light
// Directional (parallel rays, like sun)
directionalLight(255, 255, 255, 0, -1, 0); // color + direction
// Point (radiates from position)
pointLight(255, 200, 150, 200, -300, 400); // color + position
// Spot (cone from position toward target)
spotLight(255, 255, 255, // color
0, -300, 300, // position
0, 1, -1, // direction
PI / 4, 5); // angle, concentration
// Image-based lighting
imageLight(myHDRI);
// No lights (flat shading)
noLights();
// Quick default lighting
lights();
```
### Three-Point Lighting Setup
```javascript
function setupLighting() {
ambientLight(30, 30, 40); // dim blue fill
// Key light (main, warm)
directionalLight(255, 240, 220, -1, -1, -1);
// Fill light (softer, cooler, opposite side)
directionalLight(80, 100, 140, 1, -0.5, -1);
// Rim light (behind subject, for edge definition)
pointLight(200, 200, 255, 0, -200, -400);
}
```
## Materials
```javascript
// Normal material (debug — colors from surface normals)
normalMaterial();
// Ambient (responds only to ambientLight)
ambientMaterial(200, 100, 100);
// Emissive (self-lit, no shadows)
emissiveMaterial(255, 0, 100);
// Specular (shiny reflections)
specularMaterial(255);
shininess(50); // 1-200 (higher = tighter highlight)
metalness(100); // 0-200 (metallic reflection)
// Fill works too (no lighting response)
fill(255, 0, 0);
```
### Texture
```javascript
let img;
function preload() { img = loadImage('texture.jpg'); }
function draw() {
texture(img);
textureMode(NORMAL); // UV coords 0-1
// textureMode(IMAGE); // UV coords in pixels
textureWrap(REPEAT); // or CLAMP, MIRROR
box(200);
}
```
## Custom Geometry
### buildGeometry
```javascript
let myShape;
function setup() {
createCanvas(800, 800, WEBGL);
myShape = buildGeometry(() => {
for (let i = 0; i < 50; i++) {
push();
translate(random(-200, 200), random(-200, 200), random(-200, 200));
sphere(10);
pop();
}
});
}
function draw() {
model(myShape); // renders once-built geometry efficiently
}
```
### beginGeometry / endGeometry
```javascript
beginGeometry();
// draw shapes here
box(50);
translate(100, 0, 0);
sphere(30);
let geo = endGeometry();
model(geo); // reuse
```
### Manual Geometry (p5.Geometry)
```javascript
let geo = new p5.Geometry(detailX, detailY, function() {
for (let i = 0; i <= detailX; i++) {
for (let j = 0; j <= detailY; j++) {
let u = i / detailX;
let v = j / detailY;
let x = cos(u * TWO_PI) * (100 + 30 * cos(v * TWO_PI));
let y = sin(u * TWO_PI) * (100 + 30 * cos(v * TWO_PI));
let z = 30 * sin(v * TWO_PI);
this.vertices.push(createVector(x, y, z));
this.uvs.push(u, v);
}
}
this.computeFaces();
this.computeNormals();
});
```
## GLSL Shaders
### createShader (Vertex + Fragment)
```javascript
let myShader;
function setup() {
createCanvas(800, 800, WEBGL);
let vert = `
precision mediump float;
attribute vec3 aPosition;
attribute vec2 aTexCoord;
varying vec2 vTexCoord;
uniform mat4 uModelViewMatrix;
uniform mat4 uProjectionMatrix;
void main() {
vTexCoord = aTexCoord;
vec4 pos = uProjectionMatrix * uModelViewMatrix * vec4(aPosition, 1.0);
gl_Position = pos;
}
`;
let frag = `
precision mediump float;
varying vec2 vTexCoord;
uniform float uTime;
uniform vec2 uResolution;
void main() {
vec2 uv = vTexCoord;
vec3 col = 0.5 + 0.5 * cos(uTime + uv.xyx + vec3(0, 2, 4));
gl_FragColor = vec4(col, 1.0);
}
`;
myShader = createShader(vert, frag);
}
function draw() {
shader(myShader);
myShader.setUniform('uTime', millis() / 1000.0);
myShader.setUniform('uResolution', [width, height]);
rect(0, 0, width, height);
resetShader();
}
```
### createFilterShader (Post-Processing)
Simpler — only needs a fragment shader. Automatically gets the canvas as a texture.
```javascript
let blurShader;
function setup() {
createCanvas(800, 800, WEBGL);
blurShader = createFilterShader(`
precision mediump float;
varying vec2 vTexCoord;
uniform sampler2D tex0;
uniform vec2 texelSize;
void main() {
vec4 sum = vec4(0.0);
for (int x = -2; x <= 2; x++) {
for (int y = -2; y <= 2; y++) {
sum += texture2D(tex0, vTexCoord + vec2(float(x), float(y)) * texelSize);
}
}
gl_FragColor = sum / 25.0;
}
`);
}
function draw() {
// Draw scene normally
background(0);
fill(255, 0, 0);
sphere(100);
// Apply post-processing filter
filter(blurShader);
}
```
### Common Shader Uniforms
```javascript
myShader.setUniform('uTime', millis() / 1000.0);
myShader.setUniform('uResolution', [width, height]);
myShader.setUniform('uMouse', [mouseX / width, mouseY / height]);
myShader.setUniform('uTexture', myGraphics); // pass p5.Graphics as texture
myShader.setUniform('uValue', 0.5); // float
myShader.setUniform('uColor', [1.0, 0.0, 0.5, 1.0]); // vec4
```
### Shader Recipes
**Chromatic Aberration:**
```glsl
vec4 r = texture2D(tex0, vTexCoord + vec2(0.005, 0.0));
vec4 g = texture2D(tex0, vTexCoord);
vec4 b = texture2D(tex0, vTexCoord - vec2(0.005, 0.0));
gl_FragColor = vec4(r.r, g.g, b.b, 1.0);
```
**Vignette:**
```glsl
float d = distance(vTexCoord, vec2(0.5));
float v = smoothstep(0.7, 0.4, d);
gl_FragColor = texture2D(tex0, vTexCoord) * v;
```
**Scanlines:**
```glsl
float scanline = sin(vTexCoord.y * uResolution.y * 3.14159) * 0.04;
vec4 col = texture2D(tex0, vTexCoord);
gl_FragColor = col - scanline;
```
## Framebuffers
```javascript
let fbo;
function setup() {
createCanvas(800, 800, WEBGL);
fbo = createFramebuffer();
}
function draw() {
// Render to framebuffer
fbo.begin();
clear();
rotateY(frameCount * 0.01);
box(200);
fbo.end();
// Use framebuffer as texture
texture(fbo.color);
plane(width, height);
}
```
### Multi-Pass Rendering
```javascript
let sceneBuffer, blurBuffer;
function setup() {
createCanvas(800, 800, WEBGL);
sceneBuffer = createFramebuffer();
blurBuffer = createFramebuffer();
}
function draw() {
// Pass 1: render scene
sceneBuffer.begin();
clear();
lights();
rotateY(frameCount * 0.01);
box(200);
sceneBuffer.end();
// Pass 2: blur
blurBuffer.begin();
shader(blurShader);
blurShader.setUniform('uTexture', sceneBuffer.color);
rect(0, 0, width, height);
resetShader();
blurBuffer.end();
// Final: composite
texture(blurBuffer.color);
plane(width, height);
}
```
+179
View File
@@ -0,0 +1,179 @@
#!/usr/bin/env node
/**
* p5.js Skill Headless Frame Export
*
* Captures frames from a p5.js sketch using Puppeteer (headless Chrome).
* Uses noLoop() + redraw() for DETERMINISTIC frame-by-frame control.
*
* IMPORTANT: Your sketch must call noLoop() in setup() and set
* window._p5Ready = true when initialized. This script calls redraw()
* for each frame capture, ensuring exact 1:1 correspondence between
* frameCount and captured frames.
*
* If the sketch does NOT set window._p5Ready, the script falls back to
* a timed capture mode (less precise, may drop/duplicate frames).
*
* Usage:
* node export-frames.js sketch.html [options]
*
* Options:
* --output <dir> Output directory (default: ./frames)
* --width <px> Canvas width (default: 1920)
* --height <px> Canvas height (default: 1080)
* --frames <n> Number of frames to capture (default: 1)
* --fps <n> Target FPS for timed fallback mode (default: 30)
* --wait <ms> Wait before first capture (default: 2000)
* --selector <sel> Canvas CSS selector (default: canvas)
*
* Examples:
* node export-frames.js sketch.html --frames 1 # single PNG
* node export-frames.js sketch.html --frames 300 --fps 30 # 10s at 30fps
* node export-frames.js sketch.html --width 3840 --height 2160 # 4K still
*
* Sketch template for deterministic capture:
* function setup() {
* createCanvas(1920, 1080);
* pixelDensity(1);
* noLoop(); // REQUIRED for deterministic capture
* window._p5Ready = true; // REQUIRED to signal readiness
* }
* function draw() { ... }
*/
const puppeteer = require('puppeteer');
const path = require('path');
const fs = require('fs');
// Parse CLI arguments
function parseArgs() {
const args = process.argv.slice(2);
const opts = {
input: null,
output: './frames',
width: 1920,
height: 1080,
frames: 1,
fps: 30,
wait: 2000,
selector: 'canvas',
};
for (let i = 0; i < args.length; i++) {
if (args[i].startsWith('--')) {
const key = args[i].slice(2);
const val = args[i + 1];
if (key in opts && val !== undefined) {
opts[key] = isNaN(Number(val)) ? val : Number(val);
i++;
}
} else if (!opts.input) {
opts.input = args[i];
}
}
if (!opts.input) {
console.error('Usage: node export-frames.js <sketch.html> [options]');
process.exit(1);
}
return opts;
}
async function main() {
const opts = parseArgs();
const inputPath = path.resolve(opts.input);
if (!fs.existsSync(inputPath)) {
console.error(`File not found: ${inputPath}`);
process.exit(1);
}
// Create output directory
fs.mkdirSync(opts.output, { recursive: true });
console.log(`Capturing ${opts.frames} frame(s) from ${opts.input}`);
console.log(`Resolution: ${opts.width}x${opts.height}`);
console.log(`Output: ${opts.output}/`);
const browser = await puppeteer.launch({
headless: 'new',
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--disable-gpu',
'--disable-dev-shm-usage',
'--disable-web-security',
'--allow-file-access-from-files',
],
});
const page = await browser.newPage();
await page.setViewport({
width: opts.width,
height: opts.height,
deviceScaleFactor: 1,
});
// Navigate to sketch
const fileUrl = `file://${inputPath}`;
await page.goto(fileUrl, { waitUntil: 'networkidle0', timeout: 30000 });
// Wait for canvas to appear
await page.waitForSelector(opts.selector, { timeout: 10000 });
// Detect capture mode: deterministic (noLoop+redraw) vs timed (fallback)
let deterministic = false;
try {
await page.waitForFunction('window._p5Ready === true', { timeout: 5000 });
deterministic = true;
console.log(`Mode: deterministic (noLoop + redraw)`);
} catch {
console.log(`Mode: timed fallback (sketch does not set window._p5Ready)`);
console.log(` For frame-perfect capture, add noLoop() and window._p5Ready=true to setup()`);
await new Promise(r => setTimeout(r, opts.wait));
}
const startTime = Date.now();
for (let i = 0; i < opts.frames; i++) {
if (deterministic) {
// Advance exactly one frame
await page.evaluate(() => { redraw(); });
// Brief settle time for render to complete
await new Promise(r => setTimeout(r, 20));
}
const frameName = `frame-${String(i).padStart(4, '0')}.png`;
const framePath = path.join(opts.output, frameName);
// Capture the canvas element
const canvas = await page.$(opts.selector);
if (!canvas) {
console.error('Canvas element not found');
break;
}
await canvas.screenshot({ path: framePath, type: 'png' });
// Progress
if (i % 30 === 0 || i === opts.frames - 1) {
const pct = ((i + 1) / opts.frames * 100).toFixed(1);
const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
process.stdout.write(`\r Frame ${i + 1}/${opts.frames} (${pct}%) — ${elapsed}s`);
}
// In timed mode, wait between frames
if (!deterministic && i < opts.frames - 1) {
await new Promise(r => setTimeout(r, 1000 / opts.fps));
}
}
console.log('\n Done.');
await browser.close();
}
main().catch(err => {
console.error('Error:', err.message);
process.exit(1);
});
+108
View File
@@ -0,0 +1,108 @@
#!/bin/bash
# p5.js Skill — Headless Render Pipeline
# Renders a p5.js sketch to MP4 video via Puppeteer + ffmpeg
#
# Usage:
# bash scripts/render.sh sketch.html output.mp4 [options]
#
# Options:
# --width Canvas width (default: 1920)
# --height Canvas height (default: 1080)
# --fps Frames per second (default: 30)
# --duration Duration in seconds (default: 10)
# --quality CRF value 0-51 (default: 18, lower = better)
# --frames-only Only export frames, skip MP4 encoding
#
# Examples:
# bash scripts/render.sh sketch.html output.mp4
# bash scripts/render.sh sketch.html output.mp4 --duration 30 --fps 60
# bash scripts/render.sh sketch.html output.mp4 --width 3840 --height 2160
set -euo pipefail
# Defaults
WIDTH=1920
HEIGHT=1080
FPS=30
DURATION=10
CRF=18
FRAMES_ONLY=false
# Parse arguments
INPUT="${1:?Usage: render.sh <input.html> <output.mp4> [options]}"
OUTPUT="${2:?Usage: render.sh <input.html> <output.mp4> [options]}"
shift 2
while [[ $# -gt 0 ]]; do
case $1 in
--width) WIDTH="$2"; shift 2 ;;
--height) HEIGHT="$2"; shift 2 ;;
--fps) FPS="$2"; shift 2 ;;
--duration) DURATION="$2"; shift 2 ;;
--quality) CRF="$2"; shift 2 ;;
--frames-only) FRAMES_ONLY=true; shift ;;
*) echo "Unknown option: $1"; exit 1 ;;
esac
done
TOTAL_FRAMES=$((FPS * DURATION))
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
FRAME_DIR=$(mktemp -d)
echo "=== p5.js Render Pipeline ==="
echo "Input: $INPUT"
echo "Output: $OUTPUT"
echo "Resolution: ${WIDTH}x${HEIGHT}"
echo "FPS: $FPS"
echo "Duration: ${DURATION}s (${TOTAL_FRAMES} frames)"
echo "Quality: CRF $CRF"
echo "Frame dir: $FRAME_DIR"
echo ""
# Check dependencies
command -v node >/dev/null 2>&1 || { echo "Error: Node.js required"; exit 1; }
if [ "$FRAMES_ONLY" = false ]; then
command -v ffmpeg >/dev/null 2>&1 || { echo "Error: ffmpeg required for MP4"; exit 1; }
fi
# Step 1: Capture frames via Puppeteer
echo "Step 1/2: Capturing ${TOTAL_FRAMES} frames..."
node "$SCRIPT_DIR/export-frames.js" \
"$INPUT" \
--output "$FRAME_DIR" \
--width "$WIDTH" \
--height "$HEIGHT" \
--frames "$TOTAL_FRAMES" \
--fps "$FPS"
echo "Frames captured to $FRAME_DIR"
if [ "$FRAMES_ONLY" = true ]; then
echo "Frames saved to: $FRAME_DIR"
echo "To encode manually:"
echo " ffmpeg -framerate $FPS -i $FRAME_DIR/frame-%04d.png -c:v libx264 -crf $CRF -pix_fmt yuv420p $OUTPUT"
exit 0
fi
# Step 2: Encode to MP4
echo "Step 2/2: Encoding MP4..."
ffmpeg -y \
-framerate "$FPS" \
-i "$FRAME_DIR/frame-%04d.png" \
-c:v libx264 \
-preset slow \
-crf "$CRF" \
-pix_fmt yuv420p \
-movflags +faststart \
"$OUTPUT" \
2>"$FRAME_DIR/ffmpeg.log"
# Cleanup
rm -rf "$FRAME_DIR"
# Report
FILE_SIZE=$(ls -lh "$OUTPUT" | awk '{print $5}')
echo ""
echo "=== Done ==="
echo "Output: $OUTPUT ($FILE_SIZE)"
echo "Duration: ${DURATION}s at ${FPS}fps, ${WIDTH}x${HEIGHT}"
+28
View File
@@ -0,0 +1,28 @@
#!/bin/bash
# p5.js Skill — Local Development Server
# Serves the current directory over HTTP for loading local assets (fonts, images)
#
# Usage:
# bash scripts/serve.sh [port] [directory]
#
# Examples:
# bash scripts/serve.sh # serve CWD on port 8080
# bash scripts/serve.sh 3000 # serve CWD on port 3000
# bash scripts/serve.sh 8080 ./my-project # serve specific directory
PORT="${1:-8080}"
DIR="${2:-.}"
echo "=== p5.js Dev Server ==="
echo "Serving: $(cd "$DIR" && pwd)"
echo "URL: http://localhost:$PORT"
echo "Press Ctrl+C to stop"
echo ""
cd "$DIR" && python3 -m http.server "$PORT" 2>/dev/null || {
echo "Python3 not found. Trying Node.js..."
npx serve -l "$PORT" "$DIR" 2>/dev/null || {
echo "Error: Need python3 or npx (Node.js) for local server"
exit 1
}
}
+87
View File
@@ -0,0 +1,87 @@
#!/bin/bash
# p5.js Skill — Dependency Verification
# Run: bash skills/creative/p5js/scripts/setup.sh
set -euo pipefail
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
ok() { echo -e "${GREEN}[OK]${NC} $1"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
fail() { echo -e "${RED}[FAIL]${NC} $1"; }
echo "=== p5.js Skill — Setup Check ==="
echo ""
# Required: Node.js (for Puppeteer headless export)
if command -v node &>/dev/null; then
NODE_VER=$(node -v)
ok "Node.js $NODE_VER"
else
warn "Node.js not found — optional, needed for headless export"
echo " Install: https://nodejs.org/ or 'brew install node'"
fi
# Required: npm (for Puppeteer install)
if command -v npm &>/dev/null; then
NPM_VER=$(npm -v)
ok "npm $NPM_VER"
else
warn "npm not found — optional, needed for headless export"
fi
# Optional: Puppeteer
if node -e "require('puppeteer')" 2>/dev/null; then
ok "Puppeteer installed"
else
warn "Puppeteer not installed — needed for headless export"
echo " Install: npm install puppeteer"
fi
# Optional: ffmpeg (for MP4 encoding from frame sequences)
if command -v ffmpeg &>/dev/null; then
FFMPEG_VER=$(ffmpeg -version 2>&1 | head -1 | awk '{print $3}')
ok "ffmpeg $FFMPEG_VER"
else
warn "ffmpeg not found — needed for MP4 export"
echo " Install: brew install ffmpeg (macOS) or apt install ffmpeg (Linux)"
fi
# Optional: Python3 (for local server)
if command -v python3 &>/dev/null; then
PY_VER=$(python3 --version 2>&1 | awk '{print $2}')
ok "Python $PY_VER (for local server: python3 -m http.server)"
else
warn "Python3 not found — needed for local file serving"
fi
# Browser check (macOS)
if [[ "$(uname)" == "Darwin" ]]; then
if open -Ra "Google Chrome" 2>/dev/null; then
ok "Google Chrome found"
elif open -Ra "Safari" 2>/dev/null; then
ok "Safari found"
else
warn "No browser detected"
fi
fi
echo ""
echo "=== Core Requirements ==="
echo " A modern browser (Chrome/Firefox/Safari/Edge)"
echo " p5.js loaded via CDN — no local install needed"
echo ""
echo "=== Optional (for export) ==="
echo " Node.js + Puppeteer — headless frame capture"
echo " ffmpeg — frame sequence to MP4"
echo " Python3 — local development server"
echo ""
echo "=== Quick Start ==="
echo " 1. Create an HTML file with inline p5.js sketch"
echo " 2. Open in browser: open sketch.html"
echo " 3. Press 's' to save PNG, 'g' to save GIF"
echo ""
echo "Setup check complete."
+395
View File
@@ -0,0 +1,395 @@
<!DOCTYPE html>
<!--
p5.js Interactive Viewer Template
=================================
USE THIS AS THE STARTING POINT for interactive generative art sketches.
FIXED (keep as-is):
✓ Layout structure (sidebar + canvas)
✓ Seed navigation (prev/next/random/jump)
✓ Action buttons (regenerate, reset, download PNG)
✓ Responsive canvas sizing
✓ Parameter update + regeneration wiring
VARIABLE (replace for each project):
✗ The p5.js algorithm (setup/draw/classes)
✗ The PARAMS object (define what your art needs)
✗ The parameter controls in the sidebar (sliders, pickers)
✗ The color palette
✗ The title and description
For headless export: add noLoop() and window._p5Ready=true in setup().
-->
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Generative Art Viewer</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.11.3/p5.min.js"></script>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: 'Segoe UI', system-ui, -apple-system, sans-serif;
background: #0a0a0f;
color: #c8c8d0;
display: flex;
min-height: 100vh;
overflow: hidden;
}
/* --- Sidebar --- */
.sidebar {
width: 280px;
flex-shrink: 0;
background: #12121a;
border-right: 1px solid #1e1e2a;
padding: 20px;
overflow-y: auto;
display: flex;
flex-direction: column;
gap: 20px;
}
.sidebar h1 {
font-size: 18px;
font-weight: 600;
color: #e8e8f0;
margin-bottom: 4px;
}
.sidebar .subtitle {
font-size: 12px;
color: #666;
margin-bottom: 8px;
}
.section-title {
font-size: 11px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 1px;
color: #555;
margin-bottom: 8px;
}
/* --- Seed Controls --- */
.seed-display {
font-family: 'SF Mono', 'Fira Code', monospace;
font-size: 24px;
font-weight: 700;
color: #e8e8f0;
text-align: center;
padding: 8px;
background: #1a1a25;
border-radius: 6px;
margin-bottom: 8px;
}
.seed-nav {
display: flex;
gap: 6px;
margin-bottom: 6px;
}
.seed-nav button {
flex: 1;
padding: 6px;
font-size: 12px;
}
.seed-jump {
display: flex;
gap: 6px;
}
.seed-jump input {
flex: 1;
padding: 6px 8px;
background: #1a1a25;
border: 1px solid #2a2a35;
border-radius: 4px;
color: #c8c8d0;
font-size: 12px;
font-family: monospace;
}
.seed-jump button { padding: 6px 12px; font-size: 12px; }
/* --- Parameter Controls --- */
.control-group {
margin-bottom: 12px;
}
.control-group label {
display: flex;
justify-content: space-between;
font-size: 12px;
color: #888;
margin-bottom: 4px;
}
.control-group .value {
color: #aaa;
font-family: monospace;
font-size: 11px;
}
.control-group input[type="range"] {
width: 100%;
height: 4px;
-webkit-appearance: none;
background: #2a2a35;
border-radius: 2px;
outline: none;
}
.control-group input[type="range"]::-webkit-slider-thumb {
-webkit-appearance: none;
width: 14px; height: 14px;
border-radius: 50%;
background: #6a9bcc;
cursor: pointer;
}
.control-group input[type="color"] {
width: 100%;
height: 28px;
border: 1px solid #2a2a35;
border-radius: 4px;
background: #1a1a25;
cursor: pointer;
}
/* --- Buttons --- */
button {
padding: 8px 12px;
background: #1e1e2a;
border: 1px solid #2a2a35;
border-radius: 4px;
color: #c8c8d0;
font-size: 12px;
cursor: pointer;
transition: background 0.15s;
}
button:hover { background: #2a2a3a; }
button.primary { background: #2a4a6a; border-color: #3a5a7a; }
button.primary:hover { background: #3a5a7a; }
.actions { display: flex; flex-direction: column; gap: 6px; }
.actions button { width: 100%; }
/* --- Canvas Area --- */
.canvas-area {
flex: 1;
display: flex;
align-items: center;
justify-content: center;
padding: 20px;
background: #08080c;
}
canvas { display: block; }
</style>
</head>
<body>
<!-- === SIDEBAR === -->
<div class="sidebar">
<!-- FIXED: Title (customize text, keep structure) -->
<div>
<h1 id="art-title">Generative Sketch</h1>
<div class="subtitle" id="art-subtitle">p5.js generative art</div>
</div>
<!-- FIXED: Seed Navigation -->
<div>
<div class="section-title">Seed</div>
<div class="seed-display" id="seed-display">42</div>
<div class="seed-nav">
<button onclick="changeSeed(-1)">&#9664; Prev</button>
<button onclick="changeSeed(1)">Next &#9654;</button>
<button onclick="randomizeSeed()">Random</button>
</div>
<div class="seed-jump">
<input type="number" id="seed-input" placeholder="Seed #" min="0">
<button onclick="jumpToSeed()">Go</button>
</div>
</div>
<!-- VARIABLE: Parameters (customize for each project) -->
<div id="params-section">
<div class="section-title">Parameters</div>
<!-- === REPLACE THESE WITH YOUR PARAMETERS === -->
<div class="control-group">
<label>Count <span class="value" id="count-val">500</span></label>
<input type="range" id="count" min="50" max="2000" step="50" value="500"
oninput="updateParam('count', +this.value)">
</div>
<div class="control-group">
<label>Scale <span class="value" id="scale-val">0.005</span></label>
<input type="range" id="scale" min="0.001" max="0.02" step="0.001" value="0.005"
oninput="updateParam('scale', +this.value)">
</div>
<div class="control-group">
<label>Speed <span class="value" id="speed-val">2.0</span></label>
<input type="range" id="speed" min="0.5" max="5" step="0.1" value="2.0"
oninput="updateParam('speed', +this.value)">
</div>
<!-- === END PARAMETER CONTROLS === -->
</div>
<!-- VARIABLE: Colors (optional — include if art needs adjustable palette) -->
<!--
<div>
<div class="section-title">Colors</div>
<div class="control-group">
<label>Background</label>
<input type="color" id="bg-color" value="#0a0a14"
oninput="updateParam('bgColor', this.value)">
</div>
<div class="control-group">
<label>Primary</label>
<input type="color" id="primary-color" value="#6a9bcc"
oninput="updateParam('primaryColor', this.value)">
</div>
</div>
-->
<!-- FIXED: Actions -->
<div class="actions">
<div class="section-title">Actions</div>
<button class="primary" onclick="regenerate()">Regenerate</button>
<button onclick="resetDefaults()">Reset Defaults</button>
<button onclick="downloadPNG()">Download PNG</button>
</div>
</div>
<!-- === CANVAS === -->
<div class="canvas-area" id="canvas-container"></div>
<script>
// ====================================================================
// CONFIGURATION — REPLACE FOR EACH PROJECT
// ====================================================================
const DEFAULTS = {
seed: 42,
count: 500,
scale: 0.005,
speed: 2.0,
// Add your parameters here
};
let PARAMS = { ...DEFAULTS };
// ====================================================================
// SEED NAVIGATION — FIXED (do not modify)
// ====================================================================
function changeSeed(delta) {
PARAMS.seed = Math.max(0, PARAMS.seed + delta);
document.getElementById('seed-display').textContent = PARAMS.seed;
regenerate();
}
function randomizeSeed() {
PARAMS.seed = Math.floor(Math.random() * 99999);
document.getElementById('seed-display').textContent = PARAMS.seed;
regenerate();
}
function jumpToSeed() {
let v = parseInt(document.getElementById('seed-input').value);
if (!isNaN(v) && v >= 0) {
PARAMS.seed = v;
document.getElementById('seed-display').textContent = PARAMS.seed;
document.getElementById('seed-input').value = '';
regenerate();
}
}
// ====================================================================
// PARAMETER UPDATES — CUSTOMIZE updateParam body as needed
// ====================================================================
function updateParam(name, value) {
PARAMS[name] = value;
let el = document.getElementById(name + '-val');
if (el) el.textContent = typeof value === 'number' && value < 1 ? value.toFixed(3) : value;
regenerate();
}
function resetDefaults() {
PARAMS = { ...DEFAULTS };
// Reset all sliders to default values
for (let [key, val] of Object.entries(DEFAULTS)) {
let el = document.getElementById(key);
if (el) el.value = val;
let valEl = document.getElementById(key + '-val');
if (valEl) valEl.textContent = typeof val === 'number' && val < 1 ? val.toFixed(3) : val;
}
document.getElementById('seed-display').textContent = PARAMS.seed;
regenerate();
}
function regenerate() {
randomSeed(PARAMS.seed);
noiseSeed(PARAMS.seed);
// Clear and redraw
clear();
initializeArt();
redraw();
}
function downloadPNG() {
saveCanvas('generative-art-seed-' + PARAMS.seed, 'png');
}
// ====================================================================
// P5.JS SKETCH — REPLACE ENTIRELY FOR EACH PROJECT
// ====================================================================
// Your state variables
let particles = [];
function initializeArt() {
// Initialize your generative system using PARAMS
// This is called on every regenerate()
particles = [];
for (let i = 0; i < PARAMS.count; i++) {
particles.push({
x: random(width),
y: random(height),
vx: 0, vy: 0
});
}
}
function setup() {
// Size canvas to fit container
let container = document.getElementById('canvas-container');
let size = Math.min(container.clientWidth - 40, container.clientHeight - 40, 1080);
let cnv = createCanvas(size, size);
cnv.parent('canvas-container');
pixelDensity(1);
colorMode(HSB, 360, 100, 100, 100);
randomSeed(PARAMS.seed);
noiseSeed(PARAMS.seed);
initializeArt();
// For interactive/animated sketches: remove noLoop()
// For static generation: keep noLoop()
noLoop();
}
function draw() {
background(0, 0, 5);
// === YOUR ALGORITHM HERE ===
// Use PARAMS.count, PARAMS.scale, PARAMS.speed, etc.
noStroke();
for (let p of particles) {
let n = noise(p.x * PARAMS.scale, p.y * PARAMS.scale);
let hue = (n * 200 + PARAMS.seed * 0.1) % 360;
fill(hue, 70, 80, 60);
circle(p.x, p.y, n * 10 + 2);
}
// === END ALGORITHM ===
}
function windowResized() {
let container = document.getElementById('canvas-container');
let size = Math.min(container.clientWidth - 40, container.clientHeight - 40, 1080);
resizeCanvas(size, size);
regenerate();
}
</script>
</body>
</html>
+14 -2
View File
@@ -37,7 +37,13 @@ on CLI, Telegram, Discord, or any platform.
Define a shorthand first:
```bash
GSETUP="python ~/.hermes/skills/productivity/google-workspace/scripts/setup.py"
HERMES_HOME="${HERMES_HOME:-$HOME/.hermes}"
GWORKSPACE_SKILL_DIR="$HERMES_HOME/skills/productivity/google-workspace"
PYTHON_BIN="${HERMES_PYTHON:-python3}"
if [ -x "$HERMES_HOME/hermes-agent/venv/bin/python" ]; then
PYTHON_BIN="$HERMES_HOME/hermes-agent/venv/bin/python"
fi
GSETUP="$PYTHON_BIN $GWORKSPACE_SKILL_DIR/scripts/setup.py"
```
### Step 0: Check if already set up
@@ -135,7 +141,13 @@ Should print `AUTHENTICATED`. Setup is complete — token refreshes automaticall
All commands go through the API script. Set `GAPI` as a shorthand:
```bash
GAPI="python ~/.hermes/skills/productivity/google-workspace/scripts/google_api.py"
HERMES_HOME="${HERMES_HOME:-$HOME/.hermes}"
GWORKSPACE_SKILL_DIR="$HERMES_HOME/skills/productivity/google-workspace"
PYTHON_BIN="${HERMES_PYTHON:-python3}"
if [ -x "$HERMES_HOME/hermes-agent/venv/bin/python" ]; then
PYTHON_BIN="$HERMES_HOME/hermes-agent/venv/bin/python"
fi
GAPI="$PYTHON_BIN $GWORKSPACE_SKILL_DIR/scripts/google_api.py"
```
### Gmail
@@ -27,7 +27,13 @@ from datetime import datetime, timedelta, timezone
from email.mime.text import MIMEText
from pathlib import Path
from hermes_constants import display_hermes_home, get_hermes_home
try:
from hermes_constants import display_hermes_home, get_hermes_home
except ModuleNotFoundError:
HERMES_AGENT_ROOT = Path(__file__).resolve().parents[4]
if HERMES_AGENT_ROOT.exists():
sys.path.insert(0, str(HERMES_AGENT_ROOT))
from hermes_constants import display_hermes_home, get_hermes_home
HERMES_HOME = get_hermes_home()
TOKEN_PATH = HERMES_HOME / "google_token.json"
@@ -27,7 +27,13 @@ import subprocess
import sys
from pathlib import Path
from hermes_constants import display_hermes_home, get_hermes_home
try:
from hermes_constants import display_hermes_home, get_hermes_home
except ModuleNotFoundError:
HERMES_AGENT_ROOT = Path(__file__).resolve().parents[4]
if HERMES_AGENT_ROOT.exists():
sys.path.insert(0, str(HERMES_AGENT_ROOT))
from hermes_constants import display_hermes_home, get_hermes_home
HERMES_HOME = get_hermes_home()
TOKEN_PATH = HERMES_HOME / "google_token.json"
+460
View File
@@ -0,0 +1,460 @@
---
name: llm-wiki
description: "Karpathy's LLM Wiki — build and maintain a persistent, interlinked markdown knowledge base. Ingest sources, query compiled knowledge, and lint for consistency."
version: 2.0.0
author: Hermes Agent
license: MIT
metadata:
hermes:
tags: [wiki, knowledge-base, research, notes, markdown, rag-alternative]
category: research
related_skills: [obsidian, arxiv, agentic-research-ideas]
config:
- key: wiki.path
description: Path to the LLM Wiki knowledge base directory
default: "~/wiki"
prompt: Wiki directory path
---
# Karpathy's LLM Wiki
Build and maintain a persistent, compounding knowledge base as interlinked markdown files.
Based on [Andrej Karpathy's LLM Wiki pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f).
Unlike traditional RAG (which rediscovers knowledge from scratch per query), the wiki
compiles knowledge once and keeps it current. Cross-references are already there.
Contradictions have already been flagged. Synthesis reflects everything ingested.
**Division of labor:** The human curates sources and directs analysis. The agent
summarizes, cross-references, files, and maintains consistency.
## When This Skill Activates
Use this skill when the user:
- Asks to create, build, or start a wiki or knowledge base
- Asks to ingest, add, or process a source into their wiki
- Asks a question and an existing wiki is present at the configured path
- Asks to lint, audit, or health-check their wiki
- References their wiki, knowledge base, or "notes" in a research context
## Wiki Location
Configured via `skills.config.wiki.path` in `~/.hermes/config.yaml` (prompted
during `hermes config migrate` or `hermes setup`):
```yaml
skills:
config:
wiki:
path: ~/wiki
```
Falls back to `~/wiki` default. The resolved path is injected when this
skill loads — check the `[Skill config: ...]` block above for the active value.
The wiki is just a directory of markdown files — open it in Obsidian, VS Code, or
any editor. No database, no special tooling required.
## Architecture: Three Layers
```
wiki/
├── SCHEMA.md # Conventions, structure rules, domain config
├── index.md # Sectioned content catalog with one-line summaries
├── log.md # Chronological action log (append-only, rotated yearly)
├── raw/ # Layer 1: Immutable source material
│ ├── articles/ # Web articles, clippings
│ ├── papers/ # PDFs, arxiv papers
│ ├── transcripts/ # Meeting notes, interviews
│ └── assets/ # Images, diagrams referenced by sources
├── entities/ # Layer 2: Entity pages (people, orgs, products, models)
├── concepts/ # Layer 2: Concept/topic pages
├── comparisons/ # Layer 2: Side-by-side analyses
└── queries/ # Layer 2: Filed query results worth keeping
```
**Layer 1 — Raw Sources:** Immutable. The agent reads but never modifies these.
**Layer 2 — The Wiki:** Agent-owned markdown files. Created, updated, and
cross-referenced by the agent.
**Layer 3 — The Schema:** `SCHEMA.md` defines structure, conventions, and tag taxonomy.
## Resuming an Existing Wiki (CRITICAL — do this every session)
When the user has an existing wiki, **always orient yourself before doing anything**:
**Read `SCHEMA.md`** — understand the domain, conventions, and tag taxonomy.
**Read `index.md`** — learn what pages exist and their summaries.
**Scan recent `log.md`** — read the last 20-30 entries to understand recent activity.
```bash
WIKI="${wiki_path:-$HOME/wiki}"
# Orientation reads at session start
read_file "$WIKI/SCHEMA.md"
read_file "$WIKI/index.md"
read_file "$WIKI/log.md" offset=<last 30 lines>
```
Only after orientation should you ingest, query, or lint. This prevents:
- Creating duplicate pages for entities that already exist
- Missing cross-references to existing content
- Contradicting the schema's conventions
- Repeating work already logged
For large wikis (100+ pages), also run a quick `search_files` for the topic
at hand before creating anything new.
## Initializing a New Wiki
When the user asks to create or start a wiki:
1. Determine the wiki path (from config, env var, or ask the user; default `~/wiki`)
2. Create the directory structure above
3. Ask the user what domain the wiki covers — be specific
4. Write `SCHEMA.md` customized to the domain (see template below)
5. Write initial `index.md` with sectioned header
6. Write initial `log.md` with creation entry
7. Confirm the wiki is ready and suggest first sources to ingest
### SCHEMA.md Template
Adapt to the user's domain. The schema constrains agent behavior and ensures consistency:
```markdown
# Wiki Schema
## Domain
[What this wiki covers — e.g., "AI/ML research", "personal health", "startup intelligence"]
## Conventions
- File names: lowercase, hyphens, no spaces (e.g., `transformer-architecture.md`)
- Every wiki page starts with YAML frontmatter (see below)
- Use `[[wikilinks]]` to link between pages (minimum 2 outbound links per page)
- When updating a page, always bump the `updated` date
- Every new page must be added to `index.md` under the correct section
- Every action must be appended to `log.md`
## Frontmatter
```yaml
---
title: Page Title
created: YYYY-MM-DD
updated: YYYY-MM-DD
type: entity | concept | comparison | query | summary
tags: [from taxonomy below]
sources: [raw/articles/source-name.md]
---
```
## Tag Taxonomy
[Define 10-20 top-level tags for the domain. Add new tags here BEFORE using them.]
Example for AI/ML:
- Models: model, architecture, benchmark, training
- People/Orgs: person, company, lab, open-source
- Techniques: optimization, fine-tuning, inference, alignment, data
- Meta: comparison, timeline, controversy, prediction
Rule: every tag on a page must appear in this taxonomy. If a new tag is needed,
add it here first, then use it. This prevents tag sprawl.
## Page Thresholds
- **Create a page** when an entity/concept appears in 2+ sources OR is central to one source
- **Add to existing page** when a source mentions something already covered
- **DON'T create a page** for passing mentions, minor details, or things outside the domain
- **Split a page** when it exceeds ~200 lines — break into sub-topics with cross-links
- **Archive a page** when its content is fully superseded — move to `_archive/`, remove from index
## Entity Pages
One page per notable entity. Include:
- Overview / what it is
- Key facts and dates
- Relationships to other entities ([[wikilinks]])
- Source references
## Concept Pages
One page per concept or topic. Include:
- Definition / explanation
- Current state of knowledge
- Open questions or debates
- Related concepts ([[wikilinks]])
## Comparison Pages
Side-by-side analyses. Include:
- What is being compared and why
- Dimensions of comparison (table format preferred)
- Verdict or synthesis
- Sources
## Update Policy
When new information conflicts with existing content:
1. Check the dates — newer sources generally supersede older ones
2. If genuinely contradictory, note both positions with dates and sources
3. Mark the contradiction in frontmatter: `contradictions: [page-name]`
4. Flag for user review in the lint report
```
### index.md Template
The index is sectioned by type. Each entry is one line: wikilink + summary.
```markdown
# Wiki Index
> Content catalog. Every wiki page listed under its type with a one-line summary.
> Read this first to find relevant pages for any query.
> Last updated: YYYY-MM-DD | Total pages: N
## Entities
<!-- Alphabetical within section -->
## Concepts
## Comparisons
## Queries
```
**Scaling rule:** When any section exceeds 50 entries, split it into sub-sections
by first letter or sub-domain. When the index exceeds 200 entries total, create
a `_meta/topic-map.md` that groups pages by theme for faster navigation.
### log.md Template
```markdown
# Wiki Log
> Chronological record of all wiki actions. Append-only.
> Format: `## [YYYY-MM-DD] action | subject`
> Actions: ingest, update, query, lint, create, archive, delete
> When this file exceeds 500 entries, rotate: rename to log-YYYY.md, start fresh.
## [YYYY-MM-DD] create | Wiki initialized
- Domain: [domain]
- Structure created with SCHEMA.md, index.md, log.md
```
## Core Operations
### 1. Ingest
When the user provides a source (URL, file, paste), integrate it into the wiki:
① **Capture the raw source:**
- URL → use `web_extract` to get markdown, save to `raw/articles/`
- PDF → use `web_extract` (handles PDFs), save to `raw/papers/`
- Pasted text → save to appropriate `raw/` subdirectory
- Name the file descriptively: `raw/articles/karpathy-llm-wiki-2026.md`
**Discuss takeaways** with the user — what's interesting, what matters for
the domain. (Skip this in automated/cron contexts — proceed directly.)
**Check what already exists** — search index.md and use `search_files` to find
existing pages for mentioned entities/concepts. This is the difference between
a growing wiki and a pile of duplicates.
④ **Write or update wiki pages:**
- **New entities/concepts:** Create pages only if they meet the Page Thresholds
in SCHEMA.md (2+ source mentions, or central to one source)
- **Existing pages:** Add new information, update facts, bump `updated` date.
When new info contradicts existing content, follow the Update Policy.
- **Cross-reference:** Every new or updated page must link to at least 2 other
pages via `[[wikilinks]]`. Check that existing pages link back.
- **Tags:** Only use tags from the taxonomy in SCHEMA.md
⑤ **Update navigation:**
- Add new pages to `index.md` under the correct section, alphabetically
- Update the "Total pages" count and "Last updated" date in index header
- Append to `log.md`: `## [YYYY-MM-DD] ingest | Source Title`
- List every file created or updated in the log entry
**Report what changed** — list every file created or updated to the user.
A single source can trigger updates across 5-15 wiki pages. This is normal
and desired — it's the compounding effect.
### 2. Query
When the user asks a question about the wiki's domain:
**Read `index.md`** to identify relevant pages.
**For wikis with 100+ pages**, also `search_files` across all `.md` files
for key terms — the index alone may miss relevant content.
**Read the relevant pages** using `read_file`.
**Synthesize an answer** from the compiled knowledge. Cite the wiki pages
you drew from: "Based on [[page-a]] and [[page-b]]..."
**File valuable answers back** — if the answer is a substantial comparison,
deep dive, or novel synthesis, create a page in `queries/` or `comparisons/`.
Don't file trivial lookups — only answers that would be painful to re-derive.
**Update log.md** with the query and whether it was filed.
### 3. Lint
When the user asks to lint, health-check, or audit the wiki:
**Orphan pages:** Find pages with no inbound `[[wikilinks]]` from other pages.
```python
# Use execute_code for this — programmatic scan across all wiki pages
import os, re
from collections import defaultdict
wiki = "<WIKI_PATH>"
# Scan all .md files in entities/, concepts/, comparisons/, queries/
# Extract all [[wikilinks]] — build inbound link map
# Pages with zero inbound links are orphans
```
**Broken wikilinks:** Find `[[links]]` that point to pages that don't exist.
**Index completeness:** Every wiki page should appear in `index.md`. Compare
the filesystem against index entries.
**Frontmatter validation:** Every wiki page must have all required fields
(title, created, updated, type, tags, sources). Tags must be in the taxonomy.
**Stale content:** Pages whose `updated` date is >90 days older than the most
recent source that mentions the same entities.
**Contradictions:** Pages on the same topic with conflicting claims. Look for
pages that share tags/entities but state different facts.
**Page size:** Flag pages over 200 lines — candidates for splitting.
**Tag audit:** List all tags in use, flag any not in the SCHEMA.md taxonomy.
**Log rotation:** If log.md exceeds 500 entries, rotate it.
**Report findings** with specific file paths and suggested actions, grouped by
severity (broken links > orphans > stale content > style issues).
**Append to log.md:** `## [YYYY-MM-DD] lint | N issues found`
## Working with the Wiki
### Searching
```bash
# Find pages by content
search_files "transformer" path="$WIKI" file_glob="*.md"
# Find pages by filename
search_files "*.md" target="files" path="$WIKI"
# Find pages by tag
search_files "tags:.*alignment" path="$WIKI" file_glob="*.md"
# Recent activity
read_file "$WIKI/log.md" offset=<last 20 lines>
```
### Bulk Ingest
When ingesting multiple sources at once, batch the updates:
1. Read all sources first
2. Identify all entities and concepts across all sources
3. Check existing pages for all of them (one search pass, not N)
4. Create/update pages in one pass (avoids redundant updates)
5. Update index.md once at the end
6. Write a single log entry covering the batch
### Archiving
When content is fully superseded or the domain scope changes:
1. Create `_archive/` directory if it doesn't exist
2. Move the page to `_archive/` with its original path (e.g., `_archive/entities/old-page.md`)
3. Remove from `index.md`
4. Update any pages that linked to it — replace wikilink with plain text + "(archived)"
5. Log the archive action
### Obsidian Integration
The wiki directory works as an Obsidian vault out of the box:
- `[[wikilinks]]` render as clickable links
- Graph View visualizes the knowledge network
- YAML frontmatter powers Dataview queries
- The `raw/assets/` folder holds images referenced via `![[image.png]]`
For best results:
- Set Obsidian's attachment folder to `raw/assets/`
- Enable "Wikilinks" in Obsidian settings (usually on by default)
- Install Dataview plugin for queries like `TABLE tags FROM "entities" WHERE contains(tags, "company")`
If using the Obsidian skill alongside this one, set `OBSIDIAN_VAULT_PATH` to the
same directory as the wiki path.
### Obsidian Headless (servers and headless machines)
On machines without a display, use `obsidian-headless` instead of the desktop app.
It syncs vaults via Obsidian Sync without a GUI — perfect for agents running on
servers that write to the wiki while Obsidian desktop reads it on another device.
**Setup:**
```bash
# Requires Node.js 22+
npm install -g obsidian-headless
# Login (requires Obsidian account with Sync subscription)
ob login --email <email> --password '<password>'
# Create a remote vault for the wiki
ob sync-create-remote --name "LLM Wiki"
# Connect the wiki directory to the vault
cd ~/wiki
ob sync-setup --vault "<vault-id>"
# Initial sync
ob sync
# Continuous sync (foreground — use systemd for background)
ob sync --continuous
```
**Continuous background sync via systemd:**
```ini
# ~/.config/systemd/user/obsidian-wiki-sync.service
[Unit]
Description=Obsidian LLM Wiki Sync
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/path/to/ob sync --continuous
WorkingDirectory=/home/user/wiki
Restart=on-failure
RestartSec=10
[Install]
WantedBy=default.target
```
```bash
systemctl --user daemon-reload
systemctl --user enable --now obsidian-wiki-sync
# Enable linger so sync survives logout:
sudo loginctl enable-linger $USER
```
This lets the agent write to `~/wiki` on a server while you browse the same
vault in Obsidian on your laptop/phone — changes appear within seconds.
## Pitfalls
- **Never modify files in `raw/`** — sources are immutable. Corrections go in wiki pages.
- **Always orient first** — read SCHEMA + index + recent log before any operation in a new session.
Skipping this causes duplicates and missed cross-references.
- **Always update index.md and log.md** — skipping this makes the wiki degrade. These are the
navigational backbone.
- **Don't create pages for passing mentions** — follow the Page Thresholds in SCHEMA.md. A name
appearing once in a footnote doesn't warrant an entity page.
- **Don't create pages without cross-references** — isolated pages are invisible. Every page must
link to at least 2 other pages.
- **Frontmatter is required** — it enables search, filtering, and staleness detection.
- **Tags must come from the taxonomy** — freeform tags decay into noise. Add new tags to SCHEMA.md
first, then use them.
- **Keep pages scannable** — a wiki page should be readable in 30 seconds. Split pages over
200 lines. Move detailed analysis to dedicated deep-dive pages.
- **Ask before mass-updating** — if an ingest would touch 10+ existing pages, confirm
the scope with the user first.
- **Rotate the log** — when log.md exceeds 500 entries, rename it `log-YYYY.md` and start fresh.
The agent should check log size during lint.
- **Handle contradictions explicitly** — don't silently overwrite. Note both claims with dates,
mark in frontmatter, flag for user review.
+184
View File
@@ -14,8 +14,12 @@ from agent.auxiliary_client import (
resolve_vision_provider_client,
resolve_provider_client,
auxiliary_max_tokens_param,
call_llm,
_read_codex_access_token,
_get_auxiliary_provider,
_get_provider_chain,
_is_payment_error,
_try_payment_fallback,
_resolve_forced_provider,
_resolve_auto,
)
@@ -1106,3 +1110,183 @@ class TestAuxiliaryMaxTokensParam:
patch("agent.auxiliary_client._read_codex_access_token", return_value=None):
result = auxiliary_max_tokens_param(1024)
assert result == {"max_tokens": 1024}
# ── Payment / credit exhaustion fallback ─────────────────────────────────
class TestIsPaymentError:
"""_is_payment_error detects 402 and credit-related errors."""
def test_402_status_code(self):
exc = Exception("Payment Required")
exc.status_code = 402
assert _is_payment_error(exc) is True
def test_402_with_credits_message(self):
exc = Exception("You requested up to 65535 tokens, but can only afford 8029")
exc.status_code = 402
assert _is_payment_error(exc) is True
def test_429_with_credits_message(self):
exc = Exception("insufficient credits remaining")
exc.status_code = 429
assert _is_payment_error(exc) is True
def test_429_without_credits_message_is_not_payment(self):
"""Normal rate limits should NOT be treated as payment errors."""
exc = Exception("Rate limit exceeded, try again in 2 seconds")
exc.status_code = 429
assert _is_payment_error(exc) is False
def test_generic_500_is_not_payment(self):
exc = Exception("Internal server error")
exc.status_code = 500
assert _is_payment_error(exc) is False
def test_no_status_code_with_billing_message(self):
exc = Exception("billing: payment required for this request")
assert _is_payment_error(exc) is True
def test_no_status_code_no_message(self):
exc = Exception("connection reset")
assert _is_payment_error(exc) is False
class TestGetProviderChain:
"""_get_provider_chain() resolves functions at call time (testable)."""
def test_returns_five_entries(self):
chain = _get_provider_chain()
assert len(chain) == 5
labels = [label for label, _ in chain]
assert labels == ["openrouter", "nous", "local/custom", "openai-codex", "api-key"]
def test_picks_up_patched_functions(self):
"""Patches on _try_* functions must be visible in the chain."""
sentinel = lambda: ("patched", "model")
with patch("agent.auxiliary_client._try_openrouter", sentinel):
chain = _get_provider_chain()
assert chain[0] == ("openrouter", sentinel)
class TestTryPaymentFallback:
"""_try_payment_fallback skips the failed provider and tries alternatives."""
def test_skips_failed_provider(self):
mock_client = MagicMock()
with patch("agent.auxiliary_client._try_openrouter", return_value=(None, None)), \
patch("agent.auxiliary_client._try_nous", return_value=(mock_client, "nous-model")), \
patch("agent.auxiliary_client._read_main_provider", return_value="openrouter"):
client, model, label = _try_payment_fallback("openrouter", task="compression")
assert client is mock_client
assert model == "nous-model"
assert label == "nous"
def test_returns_none_when_no_fallback(self):
with patch("agent.auxiliary_client._try_openrouter", return_value=(None, None)), \
patch("agent.auxiliary_client._try_nous", return_value=(None, None)), \
patch("agent.auxiliary_client._try_custom_endpoint", return_value=(None, None)), \
patch("agent.auxiliary_client._try_codex", return_value=(None, None)), \
patch("agent.auxiliary_client._resolve_api_key_provider", return_value=(None, None)), \
patch("agent.auxiliary_client._read_main_provider", return_value="openrouter"):
client, model, label = _try_payment_fallback("openrouter")
assert client is None
assert label == ""
def test_codex_alias_maps_to_chain_label(self):
"""'codex' should map to 'openai-codex' in the skip set."""
mock_client = MagicMock()
with patch("agent.auxiliary_client._try_openrouter", return_value=(mock_client, "or-model")), \
patch("agent.auxiliary_client._try_codex", return_value=(None, None)), \
patch("agent.auxiliary_client._read_main_provider", return_value="openai-codex"):
client, model, label = _try_payment_fallback("openai-codex", task="vision")
assert client is mock_client
assert label == "openrouter"
def test_skips_to_codex_when_or_and_nous_fail(self):
mock_codex = MagicMock()
with patch("agent.auxiliary_client._try_openrouter", return_value=(None, None)), \
patch("agent.auxiliary_client._try_nous", return_value=(None, None)), \
patch("agent.auxiliary_client._try_custom_endpoint", return_value=(None, None)), \
patch("agent.auxiliary_client._try_codex", return_value=(mock_codex, "gpt-5.2-codex")), \
patch("agent.auxiliary_client._read_main_provider", return_value="openrouter"):
client, model, label = _try_payment_fallback("openrouter")
assert client is mock_codex
assert model == "gpt-5.2-codex"
assert label == "openai-codex"
class TestCallLlmPaymentFallback:
"""call_llm() retries with a different provider on 402 / payment errors."""
def _make_402_error(self, msg="Payment Required: insufficient credits"):
exc = Exception(msg)
exc.status_code = 402
return exc
def test_402_triggers_fallback(self, monkeypatch):
"""When the primary provider returns 402, call_llm tries the next one."""
monkeypatch.setenv("OPENROUTER_API_KEY", "or-key")
primary_client = MagicMock()
primary_client.chat.completions.create.side_effect = self._make_402_error()
fallback_client = MagicMock()
fallback_response = MagicMock()
fallback_client.chat.completions.create.return_value = fallback_response
with patch("agent.auxiliary_client._get_cached_client",
return_value=(primary_client, "google/gemini-3-flash-preview")), \
patch("agent.auxiliary_client._resolve_task_provider_model",
return_value=("openrouter", "google/gemini-3-flash-preview", None, None)), \
patch("agent.auxiliary_client._try_payment_fallback",
return_value=(fallback_client, "gpt-5.2-codex", "openai-codex")) as mock_fb:
result = call_llm(
task="compression",
messages=[{"role": "user", "content": "hello"}],
)
assert result is fallback_response
mock_fb.assert_called_once_with("openrouter", "compression")
# Fallback call should use the fallback model
fb_kwargs = fallback_client.chat.completions.create.call_args.kwargs
assert fb_kwargs["model"] == "gpt-5.2-codex"
def test_non_payment_error_not_caught(self, monkeypatch):
"""Non-payment errors (500, connection, etc.) should NOT trigger fallback."""
monkeypatch.setenv("OPENROUTER_API_KEY", "or-key")
primary_client = MagicMock()
server_err = Exception("Internal Server Error")
server_err.status_code = 500
primary_client.chat.completions.create.side_effect = server_err
with patch("agent.auxiliary_client._get_cached_client",
return_value=(primary_client, "google/gemini-3-flash-preview")), \
patch("agent.auxiliary_client._resolve_task_provider_model",
return_value=("openrouter", "google/gemini-3-flash-preview", None, None)):
with pytest.raises(Exception, match="Internal Server Error"):
call_llm(
task="compression",
messages=[{"role": "user", "content": "hello"}],
)
def test_402_with_no_fallback_reraises(self, monkeypatch):
"""When 402 hits and no fallback is available, the original error propagates."""
monkeypatch.setenv("OPENROUTER_API_KEY", "or-key")
primary_client = MagicMock()
primary_client.chat.completions.create.side_effect = self._make_402_error()
with patch("agent.auxiliary_client._get_cached_client",
return_value=(primary_client, "google/gemini-3-flash-preview")), \
patch("agent.auxiliary_client._resolve_task_provider_model",
return_value=("openrouter", "google/gemini-3-flash-preview", None, None)), \
patch("agent.auxiliary_client._try_payment_fallback",
return_value=(None, None, "")):
with pytest.raises(Exception, match="insufficient credits"):
call_llm(
task="compression",
messages=[{"role": "user", "content": "hello"}],
)
+38
View File
@@ -197,6 +197,44 @@ class TestNonStringContent:
assert summary is not None
assert summary == SUMMARY_PREFIX
def test_summary_call_does_not_force_temperature(self):
mock_response = MagicMock()
mock_response.choices = [MagicMock()]
mock_response.choices[0].message.content = "ok"
with patch("agent.context_compressor.get_model_context_length", return_value=100000):
c = ContextCompressor(model="test", quiet_mode=True)
messages = [
{"role": "user", "content": "do something"},
{"role": "assistant", "content": "ok"},
]
with patch("agent.context_compressor.call_llm", return_value=mock_response) as mock_call:
c._generate_summary(messages)
kwargs = mock_call.call_args.kwargs
assert "temperature" not in kwargs
class TestSummaryFailureCooldown:
def test_summary_failure_enters_cooldown_and_skips_retry(self):
with patch("agent.context_compressor.get_model_context_length", return_value=100000):
c = ContextCompressor(model="test", quiet_mode=True)
messages = [
{"role": "user", "content": "do something"},
{"role": "assistant", "content": "ok"},
]
with patch("agent.context_compressor.call_llm", side_effect=Exception("boom")) as mock_call:
first = c._generate_summary(messages)
second = c._generate_summary(messages)
assert first is None
assert second is None
assert mock_call.call_count == 1
class TestSummaryPrefixNormalization:
def test_legacy_prefix_is_replaced(self):
+3
View File
@@ -1018,6 +1018,9 @@ class TestToolUseEnforcementGuidance:
def test_enforcement_models_includes_codex(self):
assert "codex" in TOOL_USE_ENFORCEMENT_MODELS
def test_enforcement_models_includes_grok(self):
assert "grok" in TOOL_USE_ENFORCEMENT_MODELS
def test_enforcement_models_is_tuple(self):
assert isinstance(TOOL_USE_ENFORCEMENT_MODELS, tuple)
+43
View File
@@ -102,6 +102,49 @@ class TestScanSkillCommands:
assert "/disabled-skill" not in result
def test_special_chars_stripped_from_cmd_key(self, tmp_path):
"""Skill names with +, /, or other special chars produce clean cmd keys."""
with patch("tools.skills_tool.SKILLS_DIR", tmp_path):
# Simulate a skill named "Jellyfin + Jellystat 24h Summary"
skill_dir = tmp_path / "jellyfin-plus"
skill_dir.mkdir()
(skill_dir / "SKILL.md").write_text(
"---\nname: Jellyfin + Jellystat 24h Summary\n"
"description: Test skill\n---\n\nBody.\n"
)
result = scan_skill_commands()
# The + should be stripped, not left as a literal character
assert "/jellyfin-jellystat-24h-summary" in result
# The old buggy key should NOT exist
assert "/jellyfin-+-jellystat-24h-summary" not in result
def test_allspecial_name_skipped(self, tmp_path):
"""Skill with name consisting only of special chars is silently skipped."""
with patch("tools.skills_tool.SKILLS_DIR", tmp_path):
skill_dir = tmp_path / "bad-name"
skill_dir.mkdir()
(skill_dir / "SKILL.md").write_text(
"---\nname: +++\ndescription: Bad skill\n---\n\nBody.\n"
)
result = scan_skill_commands()
# Should not create a "/" key or any entry
assert "/" not in result
assert result == {}
def test_slash_in_name_stripped_from_cmd_key(self, tmp_path):
"""Skill names with / chars produce clean cmd keys."""
with patch("tools.skills_tool.SKILLS_DIR", tmp_path):
skill_dir = tmp_path / "sonarr-api"
skill_dir.mkdir()
(skill_dir / "SKILL.md").write_text(
"---\nname: Sonarr v3/v4 API\n"
"description: Test skill\n---\n\nBody.\n"
)
result = scan_skill_commands()
assert "/sonarr-v3v4-api" in result
assert any("/" in k[1:] for k in result) is False # no unescaped /
class TestResolveSkillCommandKey:
"""Telegram bot-command names disallow hyphens, so the menu registers
skills with hyphens swapped for underscores. When Telegram autocomplete
+266 -9
View File
@@ -114,7 +114,7 @@ class TestRunJobScript:
def test_script_not_found(self, cron_env):
from cron.scheduler import _run_job_script
success, output = _run_job_script("/nonexistent/script.py")
success, output = _run_job_script("nonexistent_script.py")
assert success is False
assert "not found" in output.lower()
@@ -198,7 +198,7 @@ class TestBuildJobPromptWithScript:
job = {
"prompt": "Report status.",
"script": "/nonexistent/script.py",
"script": "nonexistent_monitor.py",
}
prompt = _build_job_prompt(job)
assert "## Script Error" in prompt
@@ -239,10 +239,10 @@ class TestCronjobToolScript:
action="create",
schedule="every 1h",
prompt="Monitor things",
script="/home/user/monitor.py",
script="monitor.py",
))
assert result["success"] is True
assert result["job"]["script"] == "/home/user/monitor.py"
assert result["job"]["script"] == "monitor.py"
def test_update_script(self, cron_env, monkeypatch):
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
@@ -258,10 +258,10 @@ class TestCronjobToolScript:
update_result = json.loads(cronjob(
action="update",
job_id=job_id,
script="/new/script.py",
script="new_script.py",
))
assert update_result["success"] is True
assert update_result["job"]["script"] == "/new/script.py"
assert update_result["job"]["script"] == "new_script.py"
def test_clear_script(self, cron_env, monkeypatch):
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
@@ -271,7 +271,7 @@ class TestCronjobToolScript:
action="create",
schedule="every 1h",
prompt="Monitor things",
script="/some/script.py",
script="some_script.py",
))
job_id = create_result["job_id"]
@@ -291,10 +291,267 @@ class TestCronjobToolScript:
action="create",
schedule="every 1h",
prompt="Monitor things",
script="/path/to/script.py",
script="data_collector.py",
)
list_result = json.loads(cronjob(action="list"))
assert list_result["success"] is True
assert len(list_result["jobs"]) == 1
assert list_result["jobs"][0]["script"] == "/path/to/script.py"
assert list_result["jobs"][0]["script"] == "data_collector.py"
class TestScriptPathContainment:
"""Regression tests for path containment bypass in _run_job_script().
Prior to the fix, absolute paths and ~-prefixed paths bypassed the
scripts_dir containment check entirely, allowing arbitrary script
execution through the cron system.
"""
def test_absolute_path_outside_scripts_dir_blocked(self, cron_env):
"""Absolute paths outside ~/.hermes/scripts/ must be rejected."""
from cron.scheduler import _run_job_script
# Create a script outside the scripts dir
outside_script = cron_env / "outside.py"
outside_script.write_text('print("should not run")\n')
success, output = _run_job_script(str(outside_script))
assert success is False
assert "blocked" in output.lower() or "outside" in output.lower()
def test_absolute_path_tmp_blocked(self, cron_env):
"""Absolute paths to /tmp must be rejected."""
from cron.scheduler import _run_job_script
success, output = _run_job_script("/tmp/evil.py")
assert success is False
assert "blocked" in output.lower() or "outside" in output.lower()
def test_tilde_path_blocked(self, cron_env):
"""~ prefixed paths must be rejected (expanduser bypasses check)."""
from cron.scheduler import _run_job_script
success, output = _run_job_script("~/evil.py")
assert success is False
assert "blocked" in output.lower() or "outside" in output.lower()
def test_tilde_traversal_blocked(self, cron_env):
"""~/../../../tmp/evil.py must be rejected."""
from cron.scheduler import _run_job_script
success, output = _run_job_script("~/../../../tmp/evil.py")
assert success is False
assert "blocked" in output.lower() or "outside" in output.lower()
def test_relative_traversal_still_blocked(self, cron_env):
"""../../etc/passwd style traversal must still be blocked."""
from cron.scheduler import _run_job_script
success, output = _run_job_script("../../etc/passwd")
assert success is False
assert "blocked" in output.lower() or "outside" in output.lower()
def test_relative_path_inside_scripts_dir_allowed(self, cron_env):
"""Relative paths within the scripts dir should still work."""
from cron.scheduler import _run_job_script
script = cron_env / "scripts" / "good.py"
script.write_text('print("ok")\n')
success, output = _run_job_script("good.py")
assert success is True
assert output == "ok"
def test_subdirectory_inside_scripts_dir_allowed(self, cron_env):
"""Relative paths to subdirectories within scripts/ should work."""
from cron.scheduler import _run_job_script
subdir = cron_env / "scripts" / "monitors"
subdir.mkdir()
script = subdir / "check.py"
script.write_text('print("sub ok")\n')
success, output = _run_job_script("monitors/check.py")
assert success is True
assert output == "sub ok"
def test_absolute_path_inside_scripts_dir_allowed(self, cron_env):
"""Absolute paths that resolve WITHIN scripts/ should work."""
from cron.scheduler import _run_job_script
script = cron_env / "scripts" / "abs_ok.py"
script.write_text('print("abs ok")\n')
success, output = _run_job_script(str(script))
assert success is True
assert output == "abs ok"
@pytest.mark.skipif(
sys.platform == "win32",
reason="Symlinks require elevated privileges on Windows",
)
def test_symlink_escape_blocked(self, cron_env, tmp_path):
"""Symlinks pointing outside scripts/ must be rejected."""
from cron.scheduler import _run_job_script
# Create a script outside the scripts dir
outside = tmp_path / "outside_evil.py"
outside.write_text('print("escaped")\n')
# Create a symlink inside scripts/ pointing outside
link = cron_env / "scripts" / "sneaky.py"
link.symlink_to(outside)
success, output = _run_job_script("sneaky.py")
assert success is False
assert "blocked" in output.lower() or "outside" in output.lower()
class TestCronjobToolScriptValidation:
"""Test API-boundary validation of cron script paths in cronjob_tools."""
def test_create_with_absolute_script_rejected(self, cron_env, monkeypatch):
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
from tools.cronjob_tools import cronjob
result = json.loads(cronjob(
action="create",
schedule="every 1h",
prompt="Monitor things",
script="/home/user/evil.py",
))
assert result["success"] is False
assert "relative" in result["error"].lower() or "absolute" in result["error"].lower()
def test_create_with_tilde_script_rejected(self, cron_env, monkeypatch):
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
from tools.cronjob_tools import cronjob
result = json.loads(cronjob(
action="create",
schedule="every 1h",
prompt="Monitor things",
script="~/monitor.py",
))
assert result["success"] is False
assert "relative" in result["error"].lower() or "absolute" in result["error"].lower()
def test_create_with_traversal_script_rejected(self, cron_env, monkeypatch):
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
from tools.cronjob_tools import cronjob
result = json.loads(cronjob(
action="create",
schedule="every 1h",
prompt="Monitor things",
script="../../etc/passwd",
))
assert result["success"] is False
assert "escapes" in result["error"].lower() or "traversal" in result["error"].lower()
def test_create_with_relative_script_allowed(self, cron_env, monkeypatch):
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
from tools.cronjob_tools import cronjob
result = json.loads(cronjob(
action="create",
schedule="every 1h",
prompt="Monitor things",
script="monitor.py",
))
assert result["success"] is True
assert result["job"]["script"] == "monitor.py"
def test_update_with_absolute_script_rejected(self, cron_env, monkeypatch):
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
from tools.cronjob_tools import cronjob
create_result = json.loads(cronjob(
action="create",
schedule="every 1h",
prompt="Monitor things",
))
job_id = create_result["job_id"]
update_result = json.loads(cronjob(
action="update",
job_id=job_id,
script="/tmp/evil.py",
))
assert update_result["success"] is False
assert "relative" in update_result["error"].lower() or "absolute" in update_result["error"].lower()
def test_update_clear_script_allowed(self, cron_env, monkeypatch):
"""Clearing a script (empty string) should always be permitted."""
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
from tools.cronjob_tools import cronjob
create_result = json.loads(cronjob(
action="create",
schedule="every 1h",
prompt="Monitor things",
script="monitor.py",
))
job_id = create_result["job_id"]
update_result = json.loads(cronjob(
action="update",
job_id=job_id,
script="",
))
assert update_result["success"] is True
assert "script" not in update_result["job"]
def test_windows_absolute_path_rejected(self, cron_env, monkeypatch):
monkeypatch.setenv("HERMES_INTERACTIVE", "1")
from tools.cronjob_tools import cronjob
result = json.loads(cronjob(
action="create",
schedule="every 1h",
prompt="Monitor things",
script="C:\\Users\\evil\\script.py",
))
assert result["success"] is False
class TestRunJobEnvVarCleanup:
"""Test that run_job() env vars are cleaned up even on early failure."""
def test_env_vars_cleaned_on_early_error(self, cron_env, monkeypatch):
"""Origin env vars must be cleaned up even if run_job fails early."""
# Ensure env vars are clean before test
for key in (
"HERMES_SESSION_PLATFORM",
"HERMES_SESSION_CHAT_ID",
"HERMES_SESSION_CHAT_NAME",
):
monkeypatch.delenv(key, raising=False)
# Build a job with origin info that will fail during execution
# (no valid model, no API key — will raise inside try block)
job = {
"id": "test-envleak",
"name": "env-leak-test",
"prompt": "test",
"schedule_display": "every 1h",
"origin": {
"platform": "telegram",
"chat_id": "12345",
"chat_name": "Test Chat",
},
}
from cron.scheduler import run_job
# Expect it to fail (no model/API key), but env vars must be cleaned
try:
run_job(job)
except Exception:
pass
# Verify env vars were cleaned up by the finally block
assert os.environ.get("HERMES_SESSION_PLATFORM") is None
assert os.environ.get("HERMES_SESSION_CHAT_ID") is None
assert os.environ.get("HERMES_SESSION_CHAT_NAME") is None
+39
View File
@@ -250,6 +250,33 @@ class TestDeliverResultWrapping:
assert "Cronjob Response" not in sent_content
assert "The agent cannot see" not in sent_content
def test_delivery_extracts_media_tags_before_send(self):
"""Cron delivery should pass MEDIA attachments separately to the send helper."""
from gateway.config import Platform
pconfig = MagicMock()
pconfig.enabled = True
mock_cfg = MagicMock()
mock_cfg.platforms = {Platform.TELEGRAM: pconfig}
with patch("gateway.config.load_gateway_config", return_value=mock_cfg), \
patch("tools.send_message_tool._send_to_platform", new=AsyncMock(return_value={"success": True})) as send_mock, \
patch("cron.scheduler.load_config", return_value={"cron": {"wrap_response": False}}):
job = {
"id": "voice-job",
"deliver": "origin",
"origin": {"platform": "telegram", "chat_id": "123"},
}
_deliver_result(job, "Title\nMEDIA:/tmp/test-voice.ogg")
send_mock.assert_called_once()
args, kwargs = send_mock.call_args
# Text content should have MEDIA: tag stripped
assert "MEDIA:" not in args[3]
assert "Title" in args[3]
# Media files should be forwarded separately
assert kwargs["media_files"] == [("/tmp/test-voice.ogg", False)]
def test_no_mirror_to_session_call(self):
"""Cron deliveries should NOT mirror into the gateway session."""
from gateway.config import Platform
@@ -682,6 +709,18 @@ class TestSilentDelivery:
tick(verbose=False)
deliver_mock.assert_not_called()
def test_silent_trailing_suppresses_delivery(self):
"""Agent appended [SILENT] after explanation text — must still suppress."""
response = "2 deals filtered out (like<10, reply<15).\n\n[SILENT]"
with patch("cron.scheduler.get_due_jobs", return_value=[self._make_job()]), \
patch("cron.scheduler.run_job", return_value=(True, "# output", response, None)), \
patch("cron.scheduler.save_job_output", return_value="/tmp/out.md"), \
patch("cron.scheduler._deliver_result") as deliver_mock, \
patch("cron.scheduler.mark_job_run"):
from cron.scheduler import tick
tick(verbose=False)
deliver_mock.assert_not_called()
def test_silent_is_case_insensitive(self):
with patch("cron.scheduler.get_due_jobs", return_value=[self._make_job()]), \
patch("cron.scheduler.run_job", return_value=(True, "# output", "[silent] nothing new", None)), \
+22
View File
@@ -6,6 +6,7 @@ from pathlib import Path
from unittest.mock import patch
from gateway.channel_directory import (
build_channel_directory,
resolve_channel_name,
format_directory_for_display,
load_directory,
@@ -45,6 +46,27 @@ class TestLoadDirectory:
assert result["updated_at"] is None
class TestBuildChannelDirectoryWrites:
def test_failed_write_preserves_previous_cache(self, tmp_path, monkeypatch):
cache_file = _write_directory(tmp_path, {
"telegram": [{"id": "123", "name": "Alice", "type": "dm"}]
})
previous = json.loads(cache_file.read_text())
def broken_dump(data, fp, *args, **kwargs):
fp.write('{"updated_at":')
fp.flush()
raise OSError("disk full")
monkeypatch.setattr(json, "dump", broken_dump)
with patch("gateway.channel_directory.DIRECTORY_PATH", cache_file):
build_channel_directory({})
result = load_directory()
assert result == previous
class TestResolveChannelName:
def _setup(self, tmp_path, platforms):
cache_file = _write_directory(tmp_path, platforms)
+385 -41
View File
@@ -8,7 +8,7 @@ import time
import unittest
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
from unittest.mock import AsyncMock, Mock, patch
try:
import lark_oapi
@@ -17,6 +17,18 @@ except ImportError:
_HAS_LARK_OAPI = False
def _mock_event_dispatcher_builder(mock_handler_class):
mock_builder = Mock()
mock_builder.register_p2_im_message_message_read_v1 = Mock(return_value=mock_builder)
mock_builder.register_p2_im_message_receive_v1 = Mock(return_value=mock_builder)
mock_builder.register_p2_im_message_reaction_created_v1 = Mock(return_value=mock_builder)
mock_builder.register_p2_im_message_reaction_deleted_v1 = Mock(return_value=mock_builder)
mock_builder.register_p2_card_action_trigger = Mock(return_value=mock_builder)
mock_builder.build = Mock(return_value=object())
mock_handler_class.builder = Mock(return_value=mock_builder)
return mock_builder
class TestPlatformEnum(unittest.TestCase):
def test_feishu_in_platform_enum(self):
from gateway.config import Platform
@@ -262,12 +274,14 @@ class TestFeishuAdapterMessaging(unittest.TestCase):
with (
patch("gateway.platforms.feishu.FEISHU_AVAILABLE", True),
patch("gateway.platforms.feishu.FEISHU_WEBHOOK_AVAILABLE", True),
patch("gateway.platforms.feishu.EventDispatcherHandler") as mock_handler_class,
patch("gateway.platforms.feishu.acquire_scoped_lock", return_value=(True, None)),
patch("gateway.platforms.feishu.release_scoped_lock"),
patch.object(adapter, "_hydrate_bot_identity", new=AsyncMock()),
patch.object(adapter, "_build_lark_client", return_value=SimpleNamespace()),
patch("gateway.platforms.feishu.web", web_module),
):
_mock_event_dispatcher_builder(mock_handler_class)
connected = asyncio.run(adapter.connect())
self.assertTrue(connected)
@@ -283,13 +297,13 @@ class TestFeishuAdapterMessaging(unittest.TestCase):
from gateway.platforms.feishu import FeishuAdapter
adapter = FeishuAdapter(PlatformConfig())
ws_client = object()
ws_client = SimpleNamespace()
with (
patch("gateway.platforms.feishu.FEISHU_AVAILABLE", True),
patch("gateway.platforms.feishu.FEISHU_WEBSOCKET_AVAILABLE", True),
patch("gateway.platforms.feishu.lark", SimpleNamespace(LogLevel=SimpleNamespace(INFO="INFO", WARNING="WARNING"))),
patch("gateway.platforms.feishu.EventDispatcherHandler", object()),
patch("gateway.platforms.feishu.EventDispatcherHandler") as mock_handler_class,
patch("gateway.platforms.feishu.FeishuWSClient", return_value=ws_client),
patch("gateway.platforms.feishu._run_official_feishu_ws_client"),
patch("gateway.platforms.feishu.acquire_scoped_lock", return_value=(True, None)) as acquire_lock,
@@ -297,6 +311,8 @@ class TestFeishuAdapterMessaging(unittest.TestCase):
patch.object(adapter, "_hydrate_bot_identity", new=AsyncMock()),
patch.object(adapter, "_build_lark_client", return_value=SimpleNamespace()),
):
_mock_event_dispatcher_builder(mock_handler_class)
loop = asyncio.new_event_loop()
future = loop.create_future()
future.set_result(None)
@@ -305,6 +321,9 @@ class TestFeishuAdapterMessaging(unittest.TestCase):
def run_in_executor(self, *_args, **_kwargs):
return future
def is_closed(self):
return False
try:
with patch("gateway.platforms.feishu.asyncio.get_running_loop", return_value=_Loop()):
connected = asyncio.run(adapter.connect())
@@ -313,6 +332,7 @@ class TestFeishuAdapterMessaging(unittest.TestCase):
loop.close()
self.assertTrue(connected)
self.assertIsNone(adapter._event_handler)
acquire_lock.assert_called_once_with(
"feishu-app-id",
"cli_app",
@@ -354,14 +374,14 @@ class TestFeishuAdapterMessaging(unittest.TestCase):
from gateway.platforms.feishu import FeishuAdapter
adapter = FeishuAdapter(PlatformConfig())
ws_client = object()
ws_client = SimpleNamespace()
sleeps = []
with (
patch("gateway.platforms.feishu.FEISHU_AVAILABLE", True),
patch("gateway.platforms.feishu.FEISHU_WEBSOCKET_AVAILABLE", True),
patch("gateway.platforms.feishu.lark", SimpleNamespace(LogLevel=SimpleNamespace(INFO="INFO", WARNING="WARNING"))),
patch("gateway.platforms.feishu.EventDispatcherHandler", object()),
patch("gateway.platforms.feishu.EventDispatcherHandler") as mock_handler_class,
patch("gateway.platforms.feishu.FeishuWSClient", return_value=ws_client),
patch("gateway.platforms.feishu.acquire_scoped_lock", return_value=(True, None)),
patch("gateway.platforms.feishu.release_scoped_lock"),
@@ -369,6 +389,8 @@ class TestFeishuAdapterMessaging(unittest.TestCase):
patch("gateway.platforms.feishu.asyncio.sleep", side_effect=lambda delay: sleeps.append(delay)),
patch.object(adapter, "_build_lark_client", return_value=SimpleNamespace()),
):
_mock_event_dispatcher_builder(mock_handler_class)
loop = asyncio.new_event_loop()
future = loop.create_future()
future.set_result(None)
@@ -383,6 +405,9 @@ class TestFeishuAdapterMessaging(unittest.TestCase):
raise OSError("temporary websocket failure")
return future
def is_closed(self):
return False
fake_loop = _Loop()
try:
with patch("gateway.platforms.feishu.asyncio.get_running_loop", return_value=fake_loop):
@@ -536,6 +561,113 @@ class TestAdapterModule(unittest.TestCase):
self.assertIn("register_p2_im_message_reaction_deleted_v1", source)
self.assertIn("register_p2_card_action_trigger", source)
def test_load_settings_uses_sdk_defaults_for_invalid_ws_reconnect_values(self):
from gateway.platforms.feishu import FeishuAdapter
settings = FeishuAdapter._load_settings(
{
"ws_reconnect_nonce": -1,
"ws_reconnect_interval": "bad",
}
)
self.assertEqual(settings.ws_reconnect_nonce, 30)
self.assertEqual(settings.ws_reconnect_interval, 120)
def test_load_settings_accepts_custom_ws_reconnect_values(self):
from gateway.platforms.feishu import FeishuAdapter
settings = FeishuAdapter._load_settings(
{
"ws_reconnect_nonce": 0,
"ws_reconnect_interval": 3,
}
)
self.assertEqual(settings.ws_reconnect_nonce, 0)
self.assertEqual(settings.ws_reconnect_interval, 3)
def test_load_settings_accepts_custom_ws_ping_values(self):
from gateway.platforms.feishu import FeishuAdapter
settings = FeishuAdapter._load_settings(
{
"ws_ping_interval": 10,
"ws_ping_timeout": 8,
}
)
self.assertEqual(settings.ws_ping_interval, 10)
self.assertEqual(settings.ws_ping_timeout, 8)
def test_load_settings_ignores_invalid_ws_ping_values(self):
from gateway.platforms.feishu import FeishuAdapter
settings = FeishuAdapter._load_settings(
{
"ws_ping_interval": 0,
"ws_ping_timeout": -1,
}
)
self.assertIsNone(settings.ws_ping_interval)
self.assertIsNone(settings.ws_ping_timeout)
def test_runtime_ws_overrides_reapply_after_sdk_configure(self):
import sys
from types import ModuleType
class _FakeWSClient:
def __init__(self):
self._reconnect_nonce = 30
self._reconnect_interval = 120
self._ping_interval = 120
self.configure_calls = []
def _configure(self, conf):
self.configure_calls.append(conf)
self._reconnect_nonce = conf.ReconnectNonce
self._reconnect_interval = conf.ReconnectInterval
self._ping_interval = conf.PingInterval
def start(self):
conf = SimpleNamespace(ReconnectNonce=99, ReconnectInterval=88, PingInterval=77)
self._configure(conf)
raise RuntimeError("stop test client")
fake_client = _FakeWSClient()
fake_adapter = SimpleNamespace(
_ws_thread_loop=None,
_ws_reconnect_nonce=2,
_ws_reconnect_interval=3,
_ws_ping_interval=4,
_ws_ping_timeout=5,
)
fake_client_module = ModuleType("lark_oapi.ws.client")
fake_client_module.loop = None
fake_client_module.websockets = SimpleNamespace(connect=AsyncMock())
fake_ws_module = ModuleType("lark_oapi.ws")
fake_ws_module.client = fake_client_module
fake_root_module = ModuleType("lark_oapi")
fake_root_module.ws = fake_ws_module
original_modules = sys.modules.copy()
sys.modules["lark_oapi"] = fake_root_module
sys.modules["lark_oapi.ws"] = fake_ws_module
sys.modules["lark_oapi.ws.client"] = fake_client_module
try:
from gateway.platforms.feishu import _run_official_feishu_ws_client
_run_official_feishu_ws_client(fake_client, fake_adapter)
finally:
sys.modules.clear()
sys.modules.update(original_modules)
self.assertEqual(len(fake_client.configure_calls), 1)
self.assertEqual(fake_client._reconnect_nonce, 2)
self.assertEqual(fake_client._reconnect_interval, 3)
self.assertEqual(fake_client._ping_interval, 4)
class TestAdapterBehavior(unittest.TestCase):
@patch.dict(os.environ, {}, clear=True)
@@ -690,10 +822,10 @@ class TestAdapterBehavior(unittest.TestCase):
adapter = FeishuAdapter(PlatformConfig())
message = SimpleNamespace(mentions=[])
sender_id = SimpleNamespace(open_id="ou_any", user_id=None)
self.assertFalse(adapter._should_accept_group_message(message, sender_id))
self.assertFalse(adapter._should_accept_group_message(message, sender_id, ""))
message_with_mention = SimpleNamespace(mentions=[SimpleNamespace(key="@_user_1")])
self.assertFalse(adapter._should_accept_group_message(message_with_mention, sender_id))
self.assertFalse(adapter._should_accept_group_message(message_with_mention, sender_id, ""))
@patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "open"}, clear=True)
def test_group_message_with_other_user_mention_is_rejected_when_bot_identity_unknown(self):
@@ -707,7 +839,7 @@ class TestAdapterBehavior(unittest.TestCase):
id=SimpleNamespace(open_id="ou_other", user_id="u_other"),
)
self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[other_mention]), sender_id))
self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[other_mention]), sender_id, ""))
@patch.dict(
os.environ,
@@ -736,28 +868,222 @@ class TestAdapterBehavior(unittest.TestCase):
adapter._should_accept_group_message(
mentioned,
SimpleNamespace(open_id="ou_allowed", user_id=None),
"",
)
)
self.assertFalse(
adapter._should_accept_group_message(
mentioned,
SimpleNamespace(open_id="ou_blocked", user_id=None),
"",
)
)
@patch.dict(
os.environ,
{
"FEISHU_GROUP_POLICY": "open",
"FEISHU_BOT_OPEN_ID": "ou_bot",
},
clear=True,
)
def test_per_group_allowlist_policy_gates_by_sender(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
config = PlatformConfig(
extra={
"group_rules": {
"oc_chat_a": {
"policy": "allowlist",
"allowlist": ["ou_alice", "ou_bob"],
}
}
}
)
adapter = FeishuAdapter(config)
adapter._bot_open_id = "ou_bot"
message = SimpleNamespace(
mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))]
)
self.assertTrue(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_alice", user_id=None),
"oc_chat_a",
)
)
self.assertFalse(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_charlie", user_id=None),
"oc_chat_a",
)
)
def test_per_group_blacklist_policy_blocks_specific_users(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
config = PlatformConfig(
extra={
"group_rules": {
"oc_chat_b": {
"policy": "blacklist",
"blacklist": ["ou_blocked"],
}
}
}
)
adapter = FeishuAdapter(config)
adapter._bot_open_id = "ou_bot"
message = SimpleNamespace(
mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))]
)
self.assertTrue(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_alice", user_id=None),
"oc_chat_b",
)
)
self.assertFalse(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_blocked", user_id=None),
"oc_chat_b",
)
)
def test_per_group_admin_only_policy_requires_admin(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
config = PlatformConfig(
extra={
"admins": ["ou_admin"],
"group_rules": {
"oc_chat_c": {
"policy": "admin_only",
}
},
}
)
adapter = FeishuAdapter(config)
adapter._bot_open_id = "ou_bot"
message = SimpleNamespace(
mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))]
)
self.assertTrue(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_admin", user_id=None),
"oc_chat_c",
)
)
self.assertFalse(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_regular", user_id=None),
"oc_chat_c",
)
)
def test_per_group_disabled_policy_blocks_all(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
config = PlatformConfig(
extra={
"admins": ["ou_admin"],
"group_rules": {
"oc_chat_d": {
"policy": "disabled",
}
},
}
)
adapter = FeishuAdapter(config)
adapter._bot_open_id = "ou_bot"
message = SimpleNamespace(
mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))]
)
self.assertTrue(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_admin", user_id=None),
"oc_chat_d",
)
)
self.assertFalse(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_regular", user_id=None),
"oc_chat_d",
)
)
def test_global_admins_bypass_all_group_rules(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
config = PlatformConfig(
extra={
"admins": ["ou_admin"],
"group_rules": {
"oc_chat_e": {
"policy": "allowlist",
"allowlist": ["ou_alice"],
}
},
}
)
adapter = FeishuAdapter(config)
adapter._bot_open_id = "ou_bot"
message = SimpleNamespace(
mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))]
)
self.assertTrue(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_admin", user_id=None),
"oc_chat_e",
)
)
def test_default_group_policy_fallback_for_chats_without_explicit_rule(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
config = PlatformConfig(
extra={
"default_group_policy": "open",
}
)
adapter = FeishuAdapter(config)
adapter._bot_open_id = "ou_bot"
message = SimpleNamespace(
mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))]
)
self.assertTrue(
adapter._should_accept_group_message(
message,
SimpleNamespace(open_id="ou_anyone", user_id=None),
"oc_chat_unknown",
)
)
@patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "open"}, clear=True)
def test_group_message_matches_bot_open_id_when_configured(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
adapter = FeishuAdapter(PlatformConfig())
adapter._bot_open_id = "ou_bot"
sender_id = SimpleNamespace(open_id="ou_any", user_id=None)
bot_mention = SimpleNamespace(
@@ -769,22 +1095,16 @@ class TestAdapterBehavior(unittest.TestCase):
id=SimpleNamespace(open_id="ou_other", user_id="u_other"),
)
self.assertTrue(adapter._should_accept_group_message(SimpleNamespace(mentions=[bot_mention]), sender_id))
self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[other_mention]), sender_id))
self.assertTrue(adapter._should_accept_group_message(SimpleNamespace(mentions=[bot_mention]), sender_id, ""))
self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[other_mention]), sender_id, ""))
@patch.dict(
os.environ,
{
"FEISHU_GROUP_POLICY": "open",
"FEISHU_BOT_NAME": "Hermes Bot",
},
clear=True,
)
@patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "open"}, clear=True)
def test_group_message_matches_bot_name_when_only_name_available(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
adapter = FeishuAdapter(PlatformConfig())
adapter._bot_name = "Hermes Bot"
sender_id = SimpleNamespace(open_id="ou_any", user_id=None)
named_mention = SimpleNamespace(
@@ -796,22 +1116,16 @@ class TestAdapterBehavior(unittest.TestCase):
id=SimpleNamespace(open_id="ou_other", user_id="u_other"),
)
self.assertTrue(adapter._should_accept_group_message(SimpleNamespace(mentions=[named_mention]), sender_id))
self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[different_mention]), sender_id))
self.assertTrue(adapter._should_accept_group_message(SimpleNamespace(mentions=[named_mention]), sender_id, ""))
self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[different_mention]), sender_id, ""))
@patch.dict(
os.environ,
{
"FEISHU_GROUP_POLICY": "open",
"FEISHU_BOT_OPEN_ID": "ou_bot",
},
clear=True,
)
@patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "open"}, clear=True)
def test_group_post_message_uses_parsed_mentions_when_sdk_mentions_missing(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
adapter = FeishuAdapter(PlatformConfig())
adapter._bot_open_id = "ou_bot"
sender_id = SimpleNamespace(open_id="ou_any", user_id=None)
message = SimpleNamespace(
message_type="post",
@@ -819,7 +1133,7 @@ class TestAdapterBehavior(unittest.TestCase):
content='{"en_us":{"content":[[{"tag":"at","user_name":"Hermes","open_id":"ou_bot"}]]}}',
)
self.assertTrue(adapter._should_accept_group_message(message, sender_id))
self.assertTrue(adapter._should_accept_group_message(message, sender_id, ""))
@patch.dict(os.environ, {}, clear=True)
def test_extract_post_message_as_text(self):
@@ -1196,7 +1510,12 @@ class TestAdapterBehavior(unittest.TestCase):
from gateway.platforms.feishu import FeishuAdapter
adapter = FeishuAdapter(PlatformConfig())
adapter._loop = object()
class _Loop:
def is_closed(self):
return False
adapter._loop = _Loop()
message = SimpleNamespace(
message_id="om_text",
@@ -1210,6 +1529,7 @@ class TestAdapterBehavior(unittest.TestCase):
data = SimpleNamespace(event=SimpleNamespace(message=message, sender=sender))
future = SimpleNamespace(add_done_callback=lambda *_args, **_kwargs: None)
def _submit(coro, _loop):
coro.close()
return future
@@ -1219,6 +1539,30 @@ class TestAdapterBehavior(unittest.TestCase):
self.assertTrue(submit.called)
@patch.dict(os.environ, {}, clear=True)
def test_webhook_request_uses_same_message_dispatch_path(self):
from gateway.config import PlatformConfig
from gateway.platforms.feishu import FeishuAdapter
adapter = FeishuAdapter(PlatformConfig())
adapter._on_message_event = Mock()
body = json.dumps({
"header": {"event_type": "im.message.receive_v1"},
"event": {"message": {"message_id": "om_test"}},
}).encode("utf-8")
request = SimpleNamespace(
remote="127.0.0.1",
content_length=None,
headers={},
read=AsyncMock(return_value=body),
)
response = asyncio.run(adapter._handle_webhook_request(request))
self.assertEqual(response.status, 200)
adapter._on_message_event.assert_called_once()
@patch.dict(os.environ, {}, clear=True)
def test_process_inbound_message_uses_event_sender_identity_only(self):
from gateway.config import PlatformConfig
@@ -2456,7 +2800,7 @@ class TestGroupMentionAtAll(unittest.TestCase):
mentions=[],
)
sender_id = SimpleNamespace(open_id="ou_any", user_id=None)
self.assertTrue(adapter._should_accept_group_message(message, sender_id))
self.assertTrue(adapter._should_accept_group_message(message, sender_id, ""))
@patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "allowlist", "FEISHU_ALLOWED_USERS": "ou_allowed"}, clear=True)
def test_at_all_still_requires_policy_gate(self):
@@ -2468,10 +2812,10 @@ class TestGroupMentionAtAll(unittest.TestCase):
message = SimpleNamespace(content='{"text":"@_all attention"}', mentions=[])
# Non-allowlisted user — should be blocked even with @_all.
blocked_sender = SimpleNamespace(open_id="ou_blocked", user_id=None)
self.assertFalse(adapter._should_accept_group_message(message, blocked_sender))
self.assertFalse(adapter._should_accept_group_message(message, blocked_sender, ""))
# Allowlisted user — should pass.
allowed_sender = SimpleNamespace(open_id="ou_allowed", user_id=None)
self.assertTrue(adapter._should_accept_group_message(message, allowed_sender))
self.assertTrue(adapter._should_accept_group_message(message, allowed_sender, ""))
@unittest.skipUnless(_HAS_LARK_OAPI, "lark-oapi not installed")
+372 -8
View File
@@ -428,6 +428,7 @@ class TestMatrixRequirements:
def test_check_requirements_with_token(self, monkeypatch):
monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_test")
monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org")
monkeypatch.delenv("MATRIX_ENCRYPTION", raising=False)
from gateway.platforms.matrix import check_matrix_requirements
try:
import nio # noqa: F401
@@ -448,6 +449,45 @@ class TestMatrixRequirements:
from gateway.platforms.matrix import check_matrix_requirements
assert check_matrix_requirements() is False
def test_check_requirements_encryption_true_no_e2ee_deps(self, monkeypatch):
"""MATRIX_ENCRYPTION=true should fail if python-olm is not installed."""
monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_test")
monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org")
monkeypatch.setenv("MATRIX_ENCRYPTION", "true")
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=False):
assert matrix_mod.check_matrix_requirements() is False
def test_check_requirements_encryption_false_no_e2ee_deps_ok(self, monkeypatch):
"""Without encryption, missing E2EE deps should not block startup."""
monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_test")
monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org")
monkeypatch.delenv("MATRIX_ENCRYPTION", raising=False)
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=False):
# Still needs nio itself to be importable
try:
import nio # noqa: F401
assert matrix_mod.check_matrix_requirements() is True
except ImportError:
assert matrix_mod.check_matrix_requirements() is False
def test_check_requirements_encryption_true_with_e2ee_deps(self, monkeypatch):
"""MATRIX_ENCRYPTION=true should pass if E2EE deps are available."""
monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_test")
monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org")
monkeypatch.setenv("MATRIX_ENCRYPTION", "true")
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True):
try:
import nio # noqa: F401
assert matrix_mod.check_matrix_requirements() is True
except ImportError:
assert matrix_mod.check_matrix_requirements() is False
# ---------------------------------------------------------------------------
# Access-token auth / E2EE bootstrap
@@ -516,10 +556,12 @@ class TestMatrixAccessTokenAuth:
fake_nio.InviteMemberEvent = type("InviteMemberEvent", (), {})
fake_nio.MegolmEvent = type("MegolmEvent", (), {})
with patch.dict("sys.modules", {"nio": fake_nio}):
with patch.object(adapter, "_refresh_dm_cache", AsyncMock()):
with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)):
assert await adapter.connect() is True
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True):
with patch.dict("sys.modules", {"nio": fake_nio}):
with patch.object(adapter, "_refresh_dm_cache", AsyncMock()):
with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)):
assert await adapter.connect() is True
fake_client.restore_login.assert_called_once_with(
"@bot:example.org", "DEV123", "syt_test_access_token"
@@ -532,6 +574,326 @@ class TestMatrixAccessTokenAuth:
await adapter.disconnect()
class TestMatrixE2EEHardFail:
"""connect() must refuse to start when E2EE is requested but deps are missing."""
@pytest.mark.asyncio
async def test_connect_fails_when_encryption_true_but_no_e2ee_deps(self):
from gateway.platforms.matrix import MatrixAdapter
config = PlatformConfig(
enabled=True,
token="syt_test_access_token",
extra={
"homeserver": "https://matrix.example.org",
"user_id": "@bot:example.org",
"encryption": True,
},
)
adapter = MatrixAdapter(config)
fake_nio = MagicMock()
fake_nio.AsyncClient = MagicMock()
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=False):
with patch.dict("sys.modules", {"nio": fake_nio}):
result = await adapter.connect()
assert result is False
@pytest.mark.asyncio
async def test_connect_fails_when_olm_not_loaded_after_login(self):
"""Even if _check_e2ee_deps passes, if olm is None after auth, hard-fail."""
from gateway.platforms.matrix import MatrixAdapter
config = PlatformConfig(
enabled=True,
token="syt_test_access_token",
extra={
"homeserver": "https://matrix.example.org",
"user_id": "@bot:example.org",
"encryption": True,
},
)
adapter = MatrixAdapter(config)
class FakeWhoamiResponse:
def __init__(self, user_id, device_id):
self.user_id = user_id
self.device_id = device_id
fake_client = MagicMock()
fake_client.whoami = AsyncMock(return_value=FakeWhoamiResponse("@bot:example.org", "DEV123"))
fake_client.close = AsyncMock()
# olm is None — crypto store not loaded
fake_client.olm = None
fake_client.should_upload_keys = False
def _restore_login(user_id, device_id, access_token):
fake_client.user_id = user_id
fake_client.device_id = device_id
fake_client.access_token = access_token
fake_client.restore_login = MagicMock(side_effect=_restore_login)
fake_nio = MagicMock()
fake_nio.AsyncClient = MagicMock(return_value=fake_client)
fake_nio.WhoamiResponse = FakeWhoamiResponse
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True):
with patch.dict("sys.modules", {"nio": fake_nio}):
result = await adapter.connect()
assert result is False
fake_client.close.assert_awaited_once()
class TestMatrixDeviceId:
"""MATRIX_DEVICE_ID should be used for stable device identity."""
def test_device_id_from_config_extra(self):
from gateway.platforms.matrix import MatrixAdapter
config = PlatformConfig(
enabled=True,
token="syt_test",
extra={
"homeserver": "https://matrix.example.org",
"device_id": "HERMES_BOT_STABLE",
},
)
adapter = MatrixAdapter(config)
assert adapter._device_id == "HERMES_BOT_STABLE"
def test_device_id_from_env(self, monkeypatch):
monkeypatch.setenv("MATRIX_DEVICE_ID", "FROM_ENV")
from gateway.platforms.matrix import MatrixAdapter
config = PlatformConfig(
enabled=True,
token="syt_test",
extra={
"homeserver": "https://matrix.example.org",
},
)
adapter = MatrixAdapter(config)
assert adapter._device_id == "FROM_ENV"
def test_device_id_config_takes_precedence_over_env(self, monkeypatch):
monkeypatch.setenv("MATRIX_DEVICE_ID", "FROM_ENV")
from gateway.platforms.matrix import MatrixAdapter
config = PlatformConfig(
enabled=True,
token="syt_test",
extra={
"homeserver": "https://matrix.example.org",
"device_id": "FROM_CONFIG",
},
)
adapter = MatrixAdapter(config)
assert adapter._device_id == "FROM_CONFIG"
@pytest.mark.asyncio
async def test_connect_uses_configured_device_id_over_whoami(self):
"""When MATRIX_DEVICE_ID is set, it should be used instead of whoami device_id."""
from gateway.platforms.matrix import MatrixAdapter
config = PlatformConfig(
enabled=True,
token="syt_test_access_token",
extra={
"homeserver": "https://matrix.example.org",
"user_id": "@bot:example.org",
"encryption": True,
"device_id": "MY_STABLE_DEVICE",
},
)
adapter = MatrixAdapter(config)
class FakeWhoamiResponse:
def __init__(self, user_id, device_id):
self.user_id = user_id
self.device_id = device_id
class FakeSyncResponse:
def __init__(self):
self.rooms = MagicMock(join={})
fake_client = MagicMock()
fake_client.whoami = AsyncMock(return_value=FakeWhoamiResponse("@bot:example.org", "WHOAMI_DEV"))
fake_client.sync = AsyncMock(return_value=FakeSyncResponse())
fake_client.keys_upload = AsyncMock()
fake_client.keys_query = AsyncMock()
fake_client.keys_claim = AsyncMock()
fake_client.send_to_device_messages = AsyncMock(return_value=[])
fake_client.get_users_for_key_claiming = MagicMock(return_value={})
fake_client.close = AsyncMock()
fake_client.add_event_callback = MagicMock()
fake_client.rooms = {}
fake_client.account_data = {}
fake_client.olm = object()
fake_client.should_upload_keys = False
fake_client.should_query_keys = False
fake_client.should_claim_keys = False
def _restore_login(user_id, device_id, access_token):
fake_client.user_id = user_id
fake_client.device_id = device_id
fake_client.access_token = access_token
fake_client.restore_login = MagicMock(side_effect=_restore_login)
fake_nio = MagicMock()
fake_nio.AsyncClient = MagicMock(return_value=fake_client)
fake_nio.WhoamiResponse = FakeWhoamiResponse
fake_nio.SyncResponse = FakeSyncResponse
fake_nio.LoginResponse = type("LoginResponse", (), {})
fake_nio.RoomMessageText = type("RoomMessageText", (), {})
fake_nio.RoomMessageImage = type("RoomMessageImage", (), {})
fake_nio.RoomMessageAudio = type("RoomMessageAudio", (), {})
fake_nio.RoomMessageVideo = type("RoomMessageVideo", (), {})
fake_nio.RoomMessageFile = type("RoomMessageFile", (), {})
fake_nio.InviteMemberEvent = type("InviteMemberEvent", (), {})
fake_nio.MegolmEvent = type("MegolmEvent", (), {})
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True):
with patch.dict("sys.modules", {"nio": fake_nio}):
with patch.object(adapter, "_refresh_dm_cache", AsyncMock()):
with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)):
assert await adapter.connect() is True
# The configured device_id should override the whoami device_id
fake_client.restore_login.assert_called_once_with(
"@bot:example.org", "MY_STABLE_DEVICE", "syt_test_access_token"
)
assert fake_client.device_id == "MY_STABLE_DEVICE"
# Verify device_id was passed to nio.AsyncClient constructor
ctor_call = fake_nio.AsyncClient.call_args
assert ctor_call.kwargs.get("device_id") == "MY_STABLE_DEVICE"
await adapter.disconnect()
class TestMatrixE2EEClientConstructorFailure:
"""connect() should hard-fail if nio.AsyncClient() raises when encryption is on."""
@pytest.mark.asyncio
async def test_connect_fails_when_e2ee_client_constructor_raises(self):
from gateway.platforms.matrix import MatrixAdapter
config = PlatformConfig(
enabled=True,
token="syt_test_access_token",
extra={
"homeserver": "https://matrix.example.org",
"user_id": "@bot:example.org",
"encryption": True,
},
)
adapter = MatrixAdapter(config)
fake_nio = MagicMock()
fake_nio.AsyncClient = MagicMock(side_effect=Exception("olm init failed"))
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True):
with patch.dict("sys.modules", {"nio": fake_nio}):
result = await adapter.connect()
assert result is False
class TestMatrixPasswordLoginDeviceId:
"""MATRIX_DEVICE_ID should be passed to nio.AsyncClient even with password login."""
@pytest.mark.asyncio
async def test_password_login_passes_device_id_to_constructor(self):
from gateway.platforms.matrix import MatrixAdapter
config = PlatformConfig(
enabled=True,
extra={
"homeserver": "https://matrix.example.org",
"user_id": "@bot:example.org",
"password": "secret",
"device_id": "STABLE_PW_DEVICE",
},
)
adapter = MatrixAdapter(config)
class FakeLoginResponse:
pass
class FakeSyncResponse:
def __init__(self):
self.rooms = MagicMock(join={})
fake_client = MagicMock()
fake_client.login = AsyncMock(return_value=FakeLoginResponse())
fake_client.sync = AsyncMock(return_value=FakeSyncResponse())
fake_client.close = AsyncMock()
fake_client.add_event_callback = MagicMock()
fake_client.rooms = {}
fake_client.account_data = {}
fake_nio = MagicMock()
fake_nio.AsyncClient = MagicMock(return_value=fake_client)
fake_nio.LoginResponse = FakeLoginResponse
fake_nio.SyncResponse = FakeSyncResponse
fake_nio.RoomMessageText = type("RoomMessageText", (), {})
fake_nio.RoomMessageImage = type("RoomMessageImage", (), {})
fake_nio.RoomMessageAudio = type("RoomMessageAudio", (), {})
fake_nio.RoomMessageVideo = type("RoomMessageVideo", (), {})
fake_nio.RoomMessageFile = type("RoomMessageFile", (), {})
fake_nio.InviteMemberEvent = type("InviteMemberEvent", (), {})
with patch.dict("sys.modules", {"nio": fake_nio}):
with patch.object(adapter, "_refresh_dm_cache", AsyncMock()):
with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)):
assert await adapter.connect() is True
# Verify device_id was passed to the nio.AsyncClient constructor
ctor_call = fake_nio.AsyncClient.call_args
assert ctor_call.kwargs.get("device_id") == "STABLE_PW_DEVICE"
await adapter.disconnect()
class TestMatrixDeviceIdConfig:
"""MATRIX_DEVICE_ID should be plumbed through gateway config."""
def test_device_id_in_config_extra(self, monkeypatch):
monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_abc123")
monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org")
monkeypatch.setenv("MATRIX_DEVICE_ID", "HERMES_BOT")
from gateway.config import GatewayConfig, _apply_env_overrides
config = GatewayConfig()
_apply_env_overrides(config)
mc = config.platforms[Platform.MATRIX]
assert mc.extra.get("device_id") == "HERMES_BOT"
def test_device_id_not_set_when_env_empty(self, monkeypatch):
monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_abc123")
monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org")
monkeypatch.delenv("MATRIX_DEVICE_ID", raising=False)
from gateway.config import GatewayConfig, _apply_env_overrides
config = GatewayConfig()
_apply_env_overrides(config)
mc = config.platforms[Platform.MATRIX]
assert "device_id" not in mc.extra
class TestMatrixE2EEMaintenance:
@pytest.mark.asyncio
async def test_sync_loop_runs_e2ee_maintenance_requests(self):
@@ -1071,10 +1433,12 @@ class TestMatrixEncryptedMedia:
fake_nio.InviteMemberEvent = FakeInviteMemberEvent
fake_nio.MegolmEvent = FakeMegolmEvent
with patch.dict("sys.modules", {"nio": fake_nio}):
with patch.object(adapter, "_refresh_dm_cache", AsyncMock()):
with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)):
assert await adapter.connect() is True
from gateway.platforms import matrix as matrix_mod
with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True):
with patch.dict("sys.modules", {"nio": fake_nio}):
with patch.object(adapter, "_refresh_dm_cache", AsyncMock()):
with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)):
assert await adapter.connect() is True
callback_classes = [call.args[1] for call in fake_client.add_event_callback.call_args_list]
assert FakeRoomEncryptedImage in callback_classes
+26
View File
@@ -8,6 +8,7 @@ from gateway.platforms.base import (
GATEWAY_SECRET_CAPTURE_UNSUPPORTED_MESSAGE,
MessageEvent,
MessageType,
_safe_url_for_log,
)
@@ -18,6 +19,31 @@ class TestSecretCaptureGuidance:
assert "~/.hermes/.env" in message
class TestSafeUrlForLog:
def test_strips_query_fragment_and_userinfo(self):
url = (
"https://user:pass@example.com/private/path/image.png"
"?X-Amz-Signature=supersecret&token=abc#frag"
)
result = _safe_url_for_log(url)
assert result == "https://example.com/.../image.png"
assert "supersecret" not in result
assert "token=abc" not in result
assert "user:pass@" not in result
def test_truncates_long_values(self):
long_url = "https://example.com/" + ("a" * 300)
result = _safe_url_for_log(long_url, max_len=40)
assert len(result) == 40
assert result.endswith("...")
def test_handles_small_and_non_positive_max_len(self):
url = "https://example.com/very/long/path/file.png?token=secret"
assert _safe_url_for_log(url, max_len=3) == "..."
assert _safe_url_for_log(url, max_len=2) == ".."
assert _safe_url_for_log(url, max_len=0) == ""
# ---------------------------------------------------------------------------
# MessageEvent — command parsing
# ---------------------------------------------------------------------------
+2 -3
View File
@@ -201,8 +201,8 @@ class TestHandleResumeCommand:
db.close()
@pytest.mark.asyncio
async def test_resume_flushes_memories_with_gateway_session_key(self, tmp_path):
"""Resume should preserve the gateway session key for Honcho flushes."""
async def test_resume_flushes_memories(self, tmp_path):
"""Resume should flush memories from the current session before switching."""
from hermes_state import SessionDB
db = SessionDB(db_path=tmp_path / "state.db")
@@ -221,6 +221,5 @@ class TestHandleResumeCommand:
runner._async_flush_memories.assert_called_once_with(
"current_session_001",
_session_key_for_event(event),
)
db.close()
+126
View File
@@ -0,0 +1,126 @@
"""Tests that /new (and its /reset alias) clears the session-scoped model override."""
from datetime import datetime
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock
import pytest
from gateway.config import GatewayConfig, Platform, PlatformConfig
from gateway.platforms.base import MessageEvent
from gateway.session import SessionEntry, SessionSource, build_session_key
def _make_source() -> SessionSource:
return SessionSource(
platform=Platform.TELEGRAM,
user_id="u1",
chat_id="c1",
user_name="tester",
chat_type="dm",
)
def _make_event(text: str) -> MessageEvent:
return MessageEvent(text=text, source=_make_source(), message_id="m1")
def _make_runner():
from gateway.run import GatewayRunner
runner = object.__new__(GatewayRunner)
runner.config = GatewayConfig(
platforms={Platform.TELEGRAM: PlatformConfig(enabled=True, token="***")}
)
adapter = MagicMock()
adapter.send = AsyncMock()
runner.adapters = {Platform.TELEGRAM: adapter}
runner._voice_mode = {}
runner.hooks = SimpleNamespace(emit=AsyncMock(), loaded_hooks=False)
runner._session_model_overrides = {}
runner._pending_model_notes = {}
runner._background_tasks = set()
session_key = build_session_key(_make_source())
session_entry = SessionEntry(
session_key=session_key,
session_id="sess-1",
created_at=datetime.now(),
updated_at=datetime.now(),
platform=Platform.TELEGRAM,
chat_type="dm",
)
runner.session_store = MagicMock()
runner.session_store.get_or_create_session.return_value = session_entry
runner.session_store.reset_session.return_value = session_entry
runner.session_store._entries = {session_key: session_entry}
runner.session_store._generate_session_key.return_value = session_key
runner._running_agents = {}
runner._pending_messages = {}
runner._pending_approvals = {}
runner._session_db = None
runner._agent_cache_lock = None # disables _evict_cached_agent lock path
runner._is_user_authorized = lambda _source: True
runner._format_session_info = lambda: ""
return runner
@pytest.mark.asyncio
async def test_new_command_clears_session_model_override():
"""/new must remove the session-scoped model override for that session."""
runner = _make_runner()
session_key = build_session_key(_make_source())
# Simulate a prior /model switch stored as a session override
runner._session_model_overrides[session_key] = {
"model": "gpt-4o",
"provider": "openai",
"api_key": "sk-test",
"base_url": "",
"api_mode": "openai",
}
await runner._handle_reset_command(_make_event("/new"))
assert session_key not in runner._session_model_overrides
@pytest.mark.asyncio
async def test_new_command_no_override_is_noop():
"""/new with no prior model override must not raise."""
runner = _make_runner()
session_key = build_session_key(_make_source())
assert session_key not in runner._session_model_overrides
await runner._handle_reset_command(_make_event("/new"))
assert session_key not in runner._session_model_overrides
@pytest.mark.asyncio
async def test_new_command_only_clears_own_session():
"""/new must only clear the override for the session that triggered it."""
runner = _make_runner()
session_key = build_session_key(_make_source())
other_key = "other_session_key"
runner._session_model_overrides[session_key] = {
"model": "gpt-4o",
"provider": "openai",
"api_key": "sk-test",
"base_url": "",
"api_mode": "openai",
}
runner._session_model_overrides[other_key] = {
"model": "claude-sonnet-4-6",
"provider": "anthropic",
"api_key": "sk-ant-test",
"base_url": "",
"api_mode": "anthropic",
}
await runner._handle_reset_command(_make_event("/new"))
assert session_key not in runner._session_model_overrides
assert other_key in runner._session_model_overrides
+339
View File
@@ -2,6 +2,7 @@
import base64
import json
import pytest
from pathlib import Path
from unittest.mock import MagicMock, patch, AsyncMock
from urllib.parse import quote
@@ -368,3 +369,341 @@ class TestSignalSendMessage:
# Just verify the import works and Signal is a valid platform
from gateway.config import Platform
assert Platform.SIGNAL.value == "signal"
# ---------------------------------------------------------------------------
# send_image_file method (#5105)
# ---------------------------------------------------------------------------
class TestSignalSendImageFile:
@pytest.mark.asyncio
async def test_send_image_file_sends_via_rpc(self, monkeypatch, tmp_path):
"""send_image_file should send image as attachment via signal-cli RPC."""
adapter = _make_signal_adapter(monkeypatch)
mock_rpc, captured = _stub_rpc({"timestamp": 1234567890})
adapter._rpc = mock_rpc
adapter._stop_typing_indicator = AsyncMock()
img_path = tmp_path / "chart.png"
img_path.write_bytes(b"\x89PNG" + b"\x00" * 100)
result = await adapter.send_image_file(chat_id="+155****4567", image_path=str(img_path))
assert result.success is True
assert len(captured) == 1
assert captured[0]["method"] == "send"
assert captured[0]["params"]["account"] == adapter.account
assert captured[0]["params"]["recipient"] == ["+155****4567"]
assert captured[0]["params"]["attachments"] == [str(img_path)]
assert captured[0]["params"]["message"] == "" # caption=None → ""
# Typing indicator must be stopped before sending
adapter._stop_typing_indicator.assert_awaited_once_with("+155****4567")
# Timestamp must be tracked for echo-back prevention
assert 1234567890 in adapter._recent_sent_timestamps
@pytest.mark.asyncio
async def test_send_image_file_to_group(self, monkeypatch, tmp_path):
"""send_image_file should route group chats via groupId."""
adapter = _make_signal_adapter(monkeypatch)
mock_rpc, captured = _stub_rpc({"timestamp": 1234567890})
adapter._rpc = mock_rpc
adapter._stop_typing_indicator = AsyncMock()
img_path = tmp_path / "photo.jpg"
img_path.write_bytes(b"\xff\xd8" + b"\x00" * 100)
result = await adapter.send_image_file(
chat_id="group:abc123==", image_path=str(img_path), caption="Here's the chart"
)
assert result.success is True
assert captured[0]["params"]["groupId"] == "abc123=="
assert captured[0]["params"]["message"] == "Here's the chart"
@pytest.mark.asyncio
async def test_send_image_file_missing(self, monkeypatch):
"""send_image_file should fail gracefully for nonexistent files."""
adapter = _make_signal_adapter(monkeypatch)
adapter._stop_typing_indicator = AsyncMock()
result = await adapter.send_image_file(chat_id="+155****4567", image_path="/nonexistent.png")
assert result.success is False
assert "not found" in result.error.lower()
@pytest.mark.asyncio
async def test_send_image_file_too_large(self, monkeypatch, tmp_path):
"""send_image_file should reject files over 100MB."""
adapter = _make_signal_adapter(monkeypatch)
adapter._stop_typing_indicator = AsyncMock()
img_path = tmp_path / "huge.png"
img_path.write_bytes(b"x")
def mock_stat(self, **kwargs):
class FakeStat:
st_size = 200 * 1024 * 1024 # 200 MB
return FakeStat()
with patch.object(Path, "stat", mock_stat):
result = await adapter.send_image_file(chat_id="+155****4567", image_path=str(img_path))
assert result.success is False
assert "too large" in result.error.lower()
@pytest.mark.asyncio
async def test_send_image_file_rpc_failure(self, monkeypatch, tmp_path):
"""send_image_file should return error when RPC returns None."""
adapter = _make_signal_adapter(monkeypatch)
mock_rpc, _ = _stub_rpc(None)
adapter._rpc = mock_rpc
adapter._stop_typing_indicator = AsyncMock()
img_path = tmp_path / "test.png"
img_path.write_bytes(b"\x89PNG" + b"\x00" * 100)
result = await adapter.send_image_file(chat_id="+155****4567", image_path=str(img_path))
assert result.success is False
assert "failed" in result.error.lower()
# ---------------------------------------------------------------------------
# send_voice method (#5105)
# ---------------------------------------------------------------------------
class TestSignalSendVoice:
@pytest.mark.asyncio
async def test_send_voice_sends_via_rpc(self, monkeypatch, tmp_path):
"""send_voice should send audio as attachment via signal-cli RPC."""
adapter = _make_signal_adapter(monkeypatch)
mock_rpc, captured = _stub_rpc({"timestamp": 1234567890})
adapter._rpc = mock_rpc
adapter._stop_typing_indicator = AsyncMock()
audio_path = tmp_path / "reply.ogg"
audio_path.write_bytes(b"OggS" + b"\x00" * 100)
result = await adapter.send_voice(chat_id="+155****4567", audio_path=str(audio_path))
assert result.success is True
assert captured[0]["method"] == "send"
assert captured[0]["params"]["attachments"] == [str(audio_path)]
assert captured[0]["params"]["message"] == "" # caption=None → ""
adapter._stop_typing_indicator.assert_awaited_once_with("+155****4567")
assert 1234567890 in adapter._recent_sent_timestamps
@pytest.mark.asyncio
async def test_send_voice_missing_file(self, monkeypatch):
"""send_voice should fail for nonexistent audio."""
adapter = _make_signal_adapter(monkeypatch)
adapter._stop_typing_indicator = AsyncMock()
result = await adapter.send_voice(chat_id="+155****4567", audio_path="/missing.ogg")
assert result.success is False
assert "not found" in result.error.lower()
@pytest.mark.asyncio
async def test_send_voice_to_group(self, monkeypatch, tmp_path):
"""send_voice should route group chats correctly."""
adapter = _make_signal_adapter(monkeypatch)
mock_rpc, captured = _stub_rpc({"timestamp": 9999})
adapter._rpc = mock_rpc
adapter._stop_typing_indicator = AsyncMock()
audio_path = tmp_path / "note.mp3"
audio_path.write_bytes(b"\xff\xe0" + b"\x00" * 100)
result = await adapter.send_voice(chat_id="group:grp1==", audio_path=str(audio_path))
assert result.success is True
assert captured[0]["params"]["groupId"] == "grp1=="
@pytest.mark.asyncio
async def test_send_voice_too_large(self, monkeypatch, tmp_path):
"""send_voice should reject files over 100MB."""
adapter = _make_signal_adapter(monkeypatch)
adapter._stop_typing_indicator = AsyncMock()
audio_path = tmp_path / "huge.ogg"
audio_path.write_bytes(b"x")
def mock_stat(self, **kwargs):
class FakeStat:
st_size = 200 * 1024 * 1024
return FakeStat()
with patch.object(Path, "stat", mock_stat):
result = await adapter.send_voice(chat_id="+155****4567", audio_path=str(audio_path))
assert result.success is False
assert "too large" in result.error.lower()
@pytest.mark.asyncio
async def test_send_voice_rpc_failure(self, monkeypatch, tmp_path):
"""send_voice should return error when RPC returns None."""
adapter = _make_signal_adapter(monkeypatch)
mock_rpc, _ = _stub_rpc(None)
adapter._rpc = mock_rpc
adapter._stop_typing_indicator = AsyncMock()
audio_path = tmp_path / "reply.ogg"
audio_path.write_bytes(b"OggS" + b"\x00" * 100)
result = await adapter.send_voice(chat_id="+155****4567", audio_path=str(audio_path))
assert result.success is False
assert "failed" in result.error.lower()
# ---------------------------------------------------------------------------
# send_video method (#5105)
# ---------------------------------------------------------------------------
class TestSignalSendVideo:
@pytest.mark.asyncio
async def test_send_video_sends_via_rpc(self, monkeypatch, tmp_path):
"""send_video should send video as attachment via signal-cli RPC."""
adapter = _make_signal_adapter(monkeypatch)
mock_rpc, captured = _stub_rpc({"timestamp": 1234567890})
adapter._rpc = mock_rpc
adapter._stop_typing_indicator = AsyncMock()
vid_path = tmp_path / "demo.mp4"
vid_path.write_bytes(b"\x00\x00\x00\x18ftyp" + b"\x00" * 100)
result = await adapter.send_video(chat_id="+155****4567", video_path=str(vid_path))
assert result.success is True
assert captured[0]["method"] == "send"
assert captured[0]["params"]["attachments"] == [str(vid_path)]
assert captured[0]["params"]["message"] == "" # caption=None → ""
adapter._stop_typing_indicator.assert_awaited_once_with("+155****4567")
assert 1234567890 in adapter._recent_sent_timestamps
@pytest.mark.asyncio
async def test_send_video_missing_file(self, monkeypatch):
"""send_video should fail for nonexistent video."""
adapter = _make_signal_adapter(monkeypatch)
adapter._stop_typing_indicator = AsyncMock()
result = await adapter.send_video(chat_id="+155****4567", video_path="/missing.mp4")
assert result.success is False
assert "not found" in result.error.lower()
@pytest.mark.asyncio
async def test_send_video_too_large(self, monkeypatch, tmp_path):
"""send_video should reject files over 100MB."""
adapter = _make_signal_adapter(monkeypatch)
adapter._stop_typing_indicator = AsyncMock()
vid_path = tmp_path / "huge.mp4"
vid_path.write_bytes(b"x")
def mock_stat(self, **kwargs):
class FakeStat:
st_size = 200 * 1024 * 1024
return FakeStat()
with patch.object(Path, "stat", mock_stat):
result = await adapter.send_video(chat_id="+155****4567", video_path=str(vid_path))
assert result.success is False
assert "too large" in result.error.lower()
@pytest.mark.asyncio
async def test_send_video_rpc_failure(self, monkeypatch, tmp_path):
"""send_video should return error when RPC returns None."""
adapter = _make_signal_adapter(monkeypatch)
mock_rpc, _ = _stub_rpc(None)
adapter._rpc = mock_rpc
adapter._stop_typing_indicator = AsyncMock()
vid_path = tmp_path / "demo.mp4"
vid_path.write_bytes(b"\x00\x00\x00\x18ftyp" + b"\x00" * 100)
result = await adapter.send_video(chat_id="+155****4567", video_path=str(vid_path))
assert result.success is False
assert "failed" in result.error.lower()
# ---------------------------------------------------------------------------
# MEDIA: tag extraction integration
# ---------------------------------------------------------------------------
class TestSignalMediaExtraction:
"""Verify the full pipeline: MEDIA: tag → extract → send_image_file/send_voice."""
def test_extract_media_finds_image_tag(self):
"""BasePlatformAdapter.extract_media should find MEDIA: image paths."""
from gateway.platforms.base import BasePlatformAdapter
media, cleaned = BasePlatformAdapter.extract_media(
"Here's the chart.\nMEDIA:/tmp/price_graph.png"
)
assert len(media) == 1
assert media[0][0] == "/tmp/price_graph.png"
assert "MEDIA:" not in cleaned
def test_extract_media_finds_audio_tag(self):
"""BasePlatformAdapter.extract_media should find MEDIA: audio paths."""
from gateway.platforms.base import BasePlatformAdapter
media, cleaned = BasePlatformAdapter.extract_media(
"[[audio_as_voice]]\nMEDIA:/tmp/reply.ogg"
)
assert len(media) == 1
assert media[0][0] == "/tmp/reply.ogg"
assert media[0][1] is True # is_voice flag
def test_signal_has_all_media_methods(self, monkeypatch):
"""SignalAdapter must override all media send methods used by gateway."""
adapter = _make_signal_adapter(monkeypatch)
from gateway.platforms.base import BasePlatformAdapter
# These methods must NOT be the base class defaults (which just send text)
assert type(adapter).send_image_file is not BasePlatformAdapter.send_image_file
assert type(adapter).send_voice is not BasePlatformAdapter.send_voice
assert type(adapter).send_video is not BasePlatformAdapter.send_video
assert type(adapter).send_document is not BasePlatformAdapter.send_document
assert type(adapter).send_image is not BasePlatformAdapter.send_image
# ---------------------------------------------------------------------------
# send_document now routes through _send_attachment (#5105 bonus)
# ---------------------------------------------------------------------------
class TestSignalSendDocumentViaHelper:
"""Verify send_document gained size check and path-in-error via _send_attachment."""
@pytest.mark.asyncio
async def test_send_document_too_large(self, monkeypatch, tmp_path):
"""send_document should now reject files over 100MB (was previously missing)."""
adapter = _make_signal_adapter(monkeypatch)
adapter._stop_typing_indicator = AsyncMock()
doc_path = tmp_path / "huge.pdf"
doc_path.write_bytes(b"x")
def mock_stat(self, **kwargs):
class FakeStat:
st_size = 200 * 1024 * 1024
return FakeStat()
with patch.object(Path, "stat", mock_stat):
result = await adapter.send_document(chat_id="+155****4567", file_path=str(doc_path))
assert result.success is False
assert "too large" in result.error.lower()
@pytest.mark.asyncio
async def test_send_document_error_includes_path(self, monkeypatch):
"""send_document error message should include the file path."""
adapter = _make_signal_adapter(monkeypatch)
adapter._stop_typing_indicator = AsyncMock()
result = await adapter.send_document(chat_id="+155****4567", file_path="/nonexistent.pdf")
assert result.success is False
assert "/nonexistent.pdf" in result.error
+104
View File
@@ -617,3 +617,107 @@ class TestCheckRequirements:
@patch("gateway.platforms.webhook.AIOHTTP_AVAILABLE", False)
def test_returns_false_without_aiohttp(self):
assert check_webhook_requirements() is False
# ===================================================================
# __raw__ template token
# ===================================================================
class TestRawTemplateToken:
"""Tests for the {__raw__} special token in _render_prompt."""
def test_raw_resolves_to_full_json_payload(self):
"""{__raw__} in a template dumps the entire payload as JSON."""
adapter = _make_adapter()
payload = {"action": "opened", "number": 42}
result = adapter._render_prompt(
"Payload: {__raw__}", payload, "push", "test"
)
expected_json = json.dumps(payload, indent=2)
assert result == f"Payload: {expected_json}"
def test_raw_truncated_at_4000_chars(self):
"""{__raw__} output is truncated at 4000 characters for large payloads."""
adapter = _make_adapter()
# Build a payload whose JSON repr exceeds 4000 chars
payload = {"data": "x" * 5000}
result = adapter._render_prompt("{__raw__}", payload, "push", "test")
assert len(result) <= 4000
def test_raw_mixed_with_other_variables(self):
"""{__raw__} can be mixed with regular template variables."""
adapter = _make_adapter()
payload = {"action": "closed", "number": 7}
result = adapter._render_prompt(
"Action={action} Raw={__raw__}", payload, "push", "test"
)
assert result.startswith("Action=closed Raw=")
assert '"action": "closed"' in result
assert '"number": 7' in result
# ===================================================================
# Cross-platform delivery thread_id passthrough
# ===================================================================
class TestDeliverCrossPlatformThreadId:
"""Tests for thread_id passthrough in _deliver_cross_platform."""
def _setup_adapter_with_mock_target(self):
"""Set up a webhook adapter with a mocked gateway_runner and target adapter."""
adapter = _make_adapter()
mock_target = AsyncMock()
mock_target.send = AsyncMock(return_value=SendResult(success=True))
mock_runner = MagicMock()
mock_runner.adapters = {Platform("telegram"): mock_target}
mock_runner.config.get_home_channel.return_value = None
adapter.gateway_runner = mock_runner
return adapter, mock_target
@pytest.mark.asyncio
async def test_thread_id_passed_as_metadata(self):
"""thread_id from deliver_extra is passed as metadata to adapter.send()."""
adapter, mock_target = self._setup_adapter_with_mock_target()
delivery = {
"deliver_extra": {
"chat_id": "12345",
"thread_id": "999",
}
}
await adapter._deliver_cross_platform("telegram", "hello", delivery)
mock_target.send.assert_awaited_once_with(
"12345", "hello", metadata={"thread_id": "999"}
)
@pytest.mark.asyncio
async def test_message_thread_id_passed_as_thread_id(self):
"""message_thread_id from deliver_extra is mapped to thread_id in metadata."""
adapter, mock_target = self._setup_adapter_with_mock_target()
delivery = {
"deliver_extra": {
"chat_id": "12345",
"message_thread_id": "888",
}
}
await adapter._deliver_cross_platform("telegram", "hello", delivery)
mock_target.send.assert_awaited_once_with(
"12345", "hello", metadata={"thread_id": "888"}
)
@pytest.mark.asyncio
async def test_no_thread_id_sends_no_metadata(self):
"""When no thread_id is present, metadata is None."""
adapter, mock_target = self._setup_adapter_with_mock_target()
delivery = {
"deliver_extra": {
"chat_id": "12345",
}
}
await adapter._deliver_cross_platform("telegram", "hello", delivery)
mock_target.send.assert_awaited_once_with(
"12345", "hello", metadata=None
)
+1 -1
View File
@@ -257,7 +257,7 @@ class TestCrossPlatformDelivery:
assert result.success is True
mock_tg_adapter.send.assert_awaited_once_with(
"12345", "I've acknowledged the alert."
"12345", "I've acknowledged the alert.", metadata=None
)
# Delivery info should be cleaned up
assert chat_id not in adapter._delivery_info
+361
View File
@@ -12,8 +12,12 @@ from hermes_cli.commands import (
SUBCOMMANDS,
SlashCommandAutoSuggest,
SlashCommandCompleter,
_CMD_NAME_LIMIT,
_TG_NAME_LIMIT,
_clamp_command_names,
_clamp_telegram_names,
_sanitize_telegram_name,
discord_skill_commands,
gateway_help_lines,
resolve_command,
slack_subcommand_map,
@@ -198,6 +202,13 @@ class TestTelegramBotCommands:
for name, _ in telegram_bot_commands():
assert "-" not in name, f"Telegram command '{name}' contains a hyphen"
def test_all_names_valid_telegram_chars(self):
"""Telegram requires: lowercase a-z, 0-9, underscores only."""
import re
tg_valid = re.compile(r"^[a-z0-9_]+$")
for name, _ in telegram_bot_commands():
assert tg_valid.match(name), f"Invalid Telegram command name: {name!r}"
def test_excludes_cli_only_without_config_gate(self):
names = {name for name, _ in telegram_bot_commands()}
for cmd in COMMAND_REGISTRY:
@@ -509,6 +520,53 @@ class TestGhostText:
assert _suggestion("hello") is None
# ---------------------------------------------------------------------------
# Telegram command name sanitization
# ---------------------------------------------------------------------------
class TestSanitizeTelegramName:
"""Tests for _sanitize_telegram_name() — Telegram requires [a-z0-9_] only."""
def test_hyphens_replaced_with_underscores(self):
assert _sanitize_telegram_name("my-skill-name") == "my_skill_name"
def test_plus_sign_stripped(self):
"""Regression: skill name 'Jellyfin + Jellystat 24h Summary'."""
assert _sanitize_telegram_name("jellyfin-+-jellystat-24h-summary") == "jellyfin_jellystat_24h_summary"
def test_slash_stripped(self):
"""Regression: skill name 'Sonarr v3/v4 API Integration'."""
assert _sanitize_telegram_name("sonarr-v3/v4-api-integration") == "sonarr_v3v4_api_integration"
def test_uppercase_lowercased(self):
assert _sanitize_telegram_name("MyCommand") == "mycommand"
def test_dots_and_special_chars_stripped(self):
assert _sanitize_telegram_name("skill.v2@beta!") == "skillv2beta"
def test_consecutive_underscores_collapsed(self):
assert _sanitize_telegram_name("a---b") == "a_b"
assert _sanitize_telegram_name("a-+-b") == "a_b"
def test_leading_trailing_underscores_stripped(self):
assert _sanitize_telegram_name("-leading") == "leading"
assert _sanitize_telegram_name("trailing-") == "trailing"
assert _sanitize_telegram_name("-both-") == "both"
def test_digits_preserved(self):
assert _sanitize_telegram_name("skill-24h") == "skill_24h"
def test_empty_after_sanitization(self):
assert _sanitize_telegram_name("+++") == ""
def test_spaces_only_becomes_empty(self):
assert _sanitize_telegram_name(" ") == ""
def test_already_valid(self):
assert _sanitize_telegram_name("valid_name_123") == "valid_name_123"
# ---------------------------------------------------------------------------
# Telegram command name clamping (32-char limit)
# ---------------------------------------------------------------------------
@@ -628,3 +686,306 @@ class TestTelegramMenuCommands:
menu_names = {n for n, _ in menu}
assert "my_enabled_skill" in menu_names
assert "my_disabled_skill" not in menu_names
def test_special_chars_in_skill_names_sanitized(self, tmp_path, monkeypatch):
"""Skills with +, /, or other special chars produce valid Telegram names."""
from unittest.mock import patch
import re
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
fake_skills_dir = str(tmp_path / "skills")
fake_cmds = {
"/jellyfin-+-jellystat-24h-summary": {
"name": "Jellyfin + Jellystat 24h Summary",
"description": "Test",
"skill_md_path": f"{fake_skills_dir}/jellyfin/SKILL.md",
"skill_dir": f"{fake_skills_dir}/jellyfin",
},
"/sonarr-v3/v4-api": {
"name": "Sonarr v3/v4 API",
"description": "Test",
"skill_md_path": f"{fake_skills_dir}/sonarr/SKILL.md",
"skill_dir": f"{fake_skills_dir}/sonarr",
},
}
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
(tmp_path / "skills").mkdir(exist_ok=True)
menu, _ = telegram_menu_commands(max_commands=100)
# Every name must match Telegram's [a-z0-9_] requirement
tg_valid = re.compile(r"^[a-z0-9_]+$")
for name, _ in menu:
assert tg_valid.match(name), f"Invalid Telegram command name: {name!r}"
def test_empty_sanitized_names_excluded(self, tmp_path, monkeypatch):
"""Skills whose names sanitize to empty string are silently dropped."""
from unittest.mock import patch
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
fake_skills_dir = str(tmp_path / "skills")
fake_cmds = {
"/+++": {
"name": "+++",
"description": "All special chars",
"skill_md_path": f"{fake_skills_dir}/bad/SKILL.md",
"skill_dir": f"{fake_skills_dir}/bad",
},
"/valid-skill": {
"name": "valid-skill",
"description": "Normal skill",
"skill_md_path": f"{fake_skills_dir}/valid/SKILL.md",
"skill_dir": f"{fake_skills_dir}/valid",
},
}
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
(tmp_path / "skills").mkdir(exist_ok=True)
menu, _ = telegram_menu_commands(max_commands=100)
menu_names = {n for n, _ in menu}
# The valid skill should be present, the empty one should not
assert "valid_skill" in menu_names
# No empty string in menu names
assert "" not in menu_names
# ---------------------------------------------------------------------------
# Backward-compat aliases
# ---------------------------------------------------------------------------
class TestBackwardCompatAliases:
"""The renamed constants/functions still exist under the old names."""
def test_tg_name_limit_alias(self):
assert _TG_NAME_LIMIT == _CMD_NAME_LIMIT == 32
def test_clamp_telegram_names_is_clamp_command_names(self):
assert _clamp_telegram_names is _clamp_command_names
# ---------------------------------------------------------------------------
# Discord skill command registration
# ---------------------------------------------------------------------------
class TestDiscordSkillCommands:
"""Tests for discord_skill_commands() — centralized skill registration."""
def test_returns_skill_entries(self, tmp_path, monkeypatch):
"""Skills under SKILLS_DIR (not .hub) should be returned."""
from unittest.mock import patch
fake_skills_dir = str(tmp_path / "skills")
fake_cmds = {
"/gif-search": {
"name": "gif-search",
"description": "Search for GIFs",
"skill_md_path": f"{fake_skills_dir}/gif-search/SKILL.md",
"skill_dir": f"{fake_skills_dir}/gif-search",
},
"/code-review": {
"name": "code-review",
"description": "Review code changes",
"skill_md_path": f"{fake_skills_dir}/code-review/SKILL.md",
"skill_dir": f"{fake_skills_dir}/code-review",
},
}
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
(tmp_path / "skills").mkdir(exist_ok=True)
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
entries, hidden = discord_skill_commands(
max_slots=50, reserved_names=set(),
)
names = {n for n, _d, _k in entries}
assert "gif-search" in names
assert "code-review" in names
assert hidden == 0
# Verify cmd_key is preserved for handler callbacks
keys = {k for _n, _d, k in entries}
assert "/gif-search" in keys
assert "/code-review" in keys
def test_names_allow_hyphens(self, tmp_path, monkeypatch):
"""Discord names should keep hyphens (unlike Telegram's _ sanitization)."""
from unittest.mock import patch
fake_skills_dir = str(tmp_path / "skills")
fake_cmds = {
"/my-cool-skill": {
"name": "my-cool-skill",
"description": "A cool skill",
"skill_md_path": f"{fake_skills_dir}/my-cool-skill/SKILL.md",
"skill_dir": f"{fake_skills_dir}/my-cool-skill",
},
}
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
(tmp_path / "skills").mkdir(exist_ok=True)
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
entries, _ = discord_skill_commands(
max_slots=50, reserved_names=set(),
)
assert entries[0][0] == "my-cool-skill" # hyphens preserved
def test_cap_enforcement(self, tmp_path, monkeypatch):
"""Entries beyond max_slots should be hidden."""
from unittest.mock import patch
fake_skills_dir = str(tmp_path / "skills")
fake_cmds = {
f"/skill-{i:03d}": {
"name": f"skill-{i:03d}",
"description": f"Skill {i}",
"skill_md_path": f"{fake_skills_dir}/skill-{i:03d}/SKILL.md",
"skill_dir": f"{fake_skills_dir}/skill-{i:03d}",
}
for i in range(20)
}
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
(tmp_path / "skills").mkdir(exist_ok=True)
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
entries, hidden = discord_skill_commands(
max_slots=5, reserved_names=set(),
)
assert len(entries) == 5
assert hidden == 15
def test_excludes_discord_disabled_skills(self, tmp_path, monkeypatch):
"""Skills disabled for discord should not appear."""
from unittest.mock import patch
config_file = tmp_path / "config.yaml"
config_file.write_text(
"skills:\n"
" platform_disabled:\n"
" discord:\n"
" - secret-skill\n"
)
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
fake_skills_dir = str(tmp_path / "skills")
fake_cmds = {
"/secret-skill": {
"name": "secret-skill",
"description": "Should not appear",
"skill_md_path": f"{fake_skills_dir}/secret-skill/SKILL.md",
"skill_dir": f"{fake_skills_dir}/secret-skill",
},
"/public-skill": {
"name": "public-skill",
"description": "Should appear",
"skill_md_path": f"{fake_skills_dir}/public-skill/SKILL.md",
"skill_dir": f"{fake_skills_dir}/public-skill",
},
}
(tmp_path / "skills").mkdir(exist_ok=True)
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
entries, _ = discord_skill_commands(
max_slots=50, reserved_names=set(),
)
names = {n for n, _d, _k in entries}
assert "secret-skill" not in names
assert "public-skill" in names
def test_reserved_names_not_overwritten(self, tmp_path, monkeypatch):
"""Skills whose names collide with built-in commands should be skipped."""
from unittest.mock import patch
fake_skills_dir = str(tmp_path / "skills")
fake_cmds = {
"/status": {
"name": "status",
"description": "Skill that collides with built-in",
"skill_md_path": f"{fake_skills_dir}/status/SKILL.md",
"skill_dir": f"{fake_skills_dir}/status",
},
}
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
(tmp_path / "skills").mkdir(exist_ok=True)
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
entries, _ = discord_skill_commands(
max_slots=50, reserved_names={"status"},
)
names = {n for n, _d, _k in entries}
assert "status" not in names
def test_description_truncated_at_100_chars(self, tmp_path, monkeypatch):
"""Descriptions exceeding 100 chars should be truncated."""
from unittest.mock import patch
fake_skills_dir = str(tmp_path / "skills")
long_desc = "x" * 150
fake_cmds = {
"/verbose-skill": {
"name": "verbose-skill",
"description": long_desc,
"skill_md_path": f"{fake_skills_dir}/verbose-skill/SKILL.md",
"skill_dir": f"{fake_skills_dir}/verbose-skill",
},
}
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
(tmp_path / "skills").mkdir(exist_ok=True)
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
entries, _ = discord_skill_commands(
max_slots=50, reserved_names=set(),
)
assert len(entries[0][1]) == 100
assert entries[0][1].endswith("...")
def test_all_names_within_32_chars(self, tmp_path, monkeypatch):
"""All returned names must respect the 32-char Discord limit."""
from unittest.mock import patch
fake_skills_dir = str(tmp_path / "skills")
long_name = "a" * 50
fake_cmds = {
f"/{long_name}": {
"name": long_name,
"description": "Long name skill",
"skill_md_path": f"{fake_skills_dir}/{long_name}/SKILL.md",
"skill_dir": f"{fake_skills_dir}/{long_name}",
},
}
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
(tmp_path / "skills").mkdir(exist_ok=True)
with (
patch("agent.skill_commands.get_skill_commands", return_value=fake_cmds),
patch("tools.skills_tool.SKILLS_DIR", tmp_path / "skills"),
):
entries, _ = discord_skill_commands(
max_slots=50, reserved_names=set(),
)
for name, _d, _k in entries:
assert len(name) <= _CMD_NAME_LIMIT, (
f"Name '{name}' is {len(name)} chars (limit {_CMD_NAME_LIMIT})"
)
+27
View File
@@ -205,6 +205,33 @@ class TestLaunchdServiceRecovery:
["launchctl", "kickstart", target],
]
def test_launchd_start_reloads_on_kickstart_exit_code_113(self, tmp_path, monkeypatch):
"""Exit code 113 (\"Could not find service\") should also trigger bootstrap recovery."""
plist_path = tmp_path / "ai.hermes.gateway.plist"
plist_path.write_text(gateway_cli.generate_launchd_plist(), encoding="utf-8")
label = gateway_cli.get_launchd_label()
calls = []
domain = gateway_cli._launchd_domain()
target = f"{domain}/{label}"
def fake_run(cmd, check=False, **kwargs):
calls.append(cmd)
if cmd == ["launchctl", "kickstart", target] and calls.count(cmd) == 1:
raise gateway_cli.subprocess.CalledProcessError(113, cmd, stderr="Could not find service")
return SimpleNamespace(returncode=0, stdout="", stderr="")
monkeypatch.setattr(gateway_cli, "get_launchd_plist_path", lambda: plist_path)
monkeypatch.setattr(gateway_cli.subprocess, "run", fake_run)
gateway_cli.launchd_start()
assert calls == [
["launchctl", "kickstart", target],
["launchctl", "bootstrap", domain, str(plist_path)],
["launchctl", "kickstart", target],
]
def test_launchd_status_reports_local_stale_plist_when_unloaded(self, tmp_path, monkeypatch, capsys):
plist_path = tmp_path / "ai.hermes.gateway.plist"
plist_path.write_text("<plist>old content</plist>", encoding="utf-8")
+46
View File
@@ -0,0 +1,46 @@
"""Tests for CLI browser CDP auto-launch helpers."""
import os
from unittest.mock import patch
from cli import HermesCLI
class TestChromeDebugLaunch:
def test_windows_launch_uses_browser_found_on_path(self):
captured = {}
def fake_popen(cmd, **kwargs):
captured["cmd"] = cmd
captured["kwargs"] = kwargs
return object()
with patch("cli.shutil.which", side_effect=lambda name: r"C:\Chrome\chrome.exe" if name == "chrome.exe" else None), \
patch("cli.os.path.isfile", side_effect=lambda path: path == r"C:\Chrome\chrome.exe"), \
patch("subprocess.Popen", side_effect=fake_popen):
assert HermesCLI._try_launch_chrome_debug(9333, "Windows") is True
assert captured["cmd"] == [r"C:\Chrome\chrome.exe", "--remote-debugging-port=9333"]
assert captured["kwargs"]["start_new_session"] is True
def test_windows_launch_falls_back_to_common_install_dirs(self, monkeypatch):
captured = {}
program_files = r"C:\Program Files"
# Use os.path.join so path separators match cross-platform
installed = os.path.join(program_files, "Google", "Chrome", "Application", "chrome.exe")
def fake_popen(cmd, **kwargs):
captured["cmd"] = cmd
captured["kwargs"] = kwargs
return object()
monkeypatch.setenv("ProgramFiles", program_files)
monkeypatch.delenv("ProgramFiles(x86)", raising=False)
monkeypatch.delenv("LOCALAPPDATA", raising=False)
with patch("cli.shutil.which", return_value=None), \
patch("cli.os.path.isfile", side_effect=lambda path: path == installed), \
patch("subprocess.Popen", side_effect=fake_popen):
assert HermesCLI._try_launch_chrome_debug(9222, "Windows") is True
assert captured["cmd"] == [installed, "--remote-debugging-port=9222"]
+87
View File
@@ -1,5 +1,6 @@
from datetime import datetime, timedelta
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
from cli import HermesCLI
@@ -78,6 +79,92 @@ class TestCLIStatusBar:
assert "$0.06" not in text # cost hidden by default
assert "15m" in text
def test_input_height_counts_wide_characters_using_cell_width(self):
cli_obj = _make_cli()
class _Doc:
lines = ["" * 10]
class _Buffer:
document = _Doc()
input_area = SimpleNamespace(buffer=_Buffer())
def _input_height():
try:
from prompt_toolkit.application import get_app
from prompt_toolkit.utils import get_cwidth
doc = input_area.buffer.document
prompt_width = max(2, get_cwidth(cli_obj._get_tui_prompt_text()))
try:
available_width = get_app().output.get_size().columns - prompt_width
except Exception:
import shutil
available_width = shutil.get_terminal_size((80, 24)).columns - prompt_width
if available_width < 10:
available_width = 40
visual_lines = 0
for line in doc.lines:
line_width = get_cwidth(line)
if line_width <= 0:
visual_lines += 1
else:
visual_lines += max(1, -(-line_width // available_width))
return min(max(visual_lines, 1), 8)
except Exception:
return 1
mock_app = MagicMock()
mock_app.output.get_size.return_value = MagicMock(columns=14)
with patch.object(HermesCLI, "_get_tui_prompt_text", return_value=" "), \
patch("prompt_toolkit.application.get_app", return_value=mock_app):
assert _input_height() == 2
def test_input_height_uses_prompt_toolkit_width_over_shutil(self):
cli_obj = _make_cli()
class _Doc:
lines = ["" * 10]
class _Buffer:
document = _Doc()
input_area = SimpleNamespace(buffer=_Buffer())
def _input_height():
try:
from prompt_toolkit.application import get_app
from prompt_toolkit.utils import get_cwidth
doc = input_area.buffer.document
prompt_width = max(2, get_cwidth(cli_obj._get_tui_prompt_text()))
try:
available_width = get_app().output.get_size().columns - prompt_width
except Exception:
import shutil
available_width = shutil.get_terminal_size((80, 24)).columns - prompt_width
if available_width < 10:
available_width = 40
visual_lines = 0
for line in doc.lines:
line_width = get_cwidth(line)
if line_width <= 0:
visual_lines += 1
else:
visual_lines += max(1, -(-line_width // available_width))
return min(max(visual_lines, 1), 8)
except Exception:
return 1
mock_app = MagicMock()
mock_app.output.get_size.return_value = MagicMock(columns=14)
with patch.object(HermesCLI, "_get_tui_prompt_text", return_value=" "), \
patch("prompt_toolkit.application.get_app", return_value=mock_app), \
patch("shutil.get_terminal_size") as mock_shutil:
assert _input_height() == 2
mock_shutil.assert_not_called()
def test_build_status_bar_text_no_cost_in_status_bar(self):
cli_obj = _attach_agent(
_make_cli(),
+432
View File
@@ -0,0 +1,432 @@
"""
Cross-environment backend compatibility tests.
Derived from analysis of 218 real Hermes agent sessions (590 terminal calls).
Tests the command execution patterns the agent actually uses, against any
environment backend (local, docker, ssh, modal, daytona, singularity).
Usage:
# Local (default)
uv run pytest tests/test_env_backend_compat.py -v
# Docker
TERMINAL_ENV=docker TERMINAL_DOCKER_IMAGE=ubuntu:24.04 \
uv run pytest tests/test_env_backend_compat.py -v
# SSH
TERMINAL_ENV=ssh TERMINAL_SSH_HOST=... TERMINAL_SSH_USER=... \
uv run pytest tests/test_env_backend_compat.py -v
# Modal
TERMINAL_ENV=modal uv run pytest tests/test_env_backend_compat.py -v
"""
import json
import os
import time
import pytest
ENV_TYPE = os.getenv("TERMINAL_ENV", "local")
def _get_env():
"""Create and return an environment backend based on TERMINAL_ENV."""
from tools.terminal_tool import _create_environment
env_type = ENV_TYPE
image = os.getenv("TERMINAL_DOCKER_IMAGE", "ubuntu:24.04")
cwd = os.getenv("TERMINAL_CWD", "/tmp")
timeout = int(os.getenv("TERMINAL_TIMEOUT", "30"))
ssh_config = None
if env_type == "ssh":
ssh_config = {
"host": os.environ["TERMINAL_SSH_HOST"],
"user": os.environ["TERMINAL_SSH_USER"],
"port": int(os.getenv("TERMINAL_SSH_PORT", "22")),
"key": os.getenv("TERMINAL_SSH_KEY", ""),
}
container_config = {
"container_cpu": int(os.getenv("TERMINAL_CPU", "1")),
"container_memory": int(os.getenv("TERMINAL_MEMORY", "2048")),
"container_disk": int(os.getenv("TERMINAL_DISK", "10240")),
"container_persistent": True,
}
return _create_environment(
env_type=env_type,
image=image,
cwd=cwd,
timeout=timeout,
ssh_config=ssh_config,
container_config=container_config,
task_id="test_env_compat",
)
@pytest.fixture(scope="module")
def env():
"""Module-scoped environment — created once, reused across tests."""
e = _get_env()
yield e
if hasattr(e, "cleanup"):
try:
e.cleanup()
except Exception:
pass
def _exec(env, command: str, timeout: int = 30) -> dict:
"""Execute a command and parse the result dict."""
result = env.execute(command, timeout=timeout)
assert isinstance(result, dict), f"Expected dict, got {type(result)}"
return result
def _output(result: dict) -> str:
return result.get("output", "")
def _rc(result: dict) -> int:
return result.get("returncode", result.get("exit_code", -999))
# ---------------------------------------------------------------------------
# Category 1: Basic execution
# From session data: simple single commands are the foundation
# ---------------------------------------------------------------------------
class TestBasicExecution:
def test_echo(self, env):
"""Most basic: can we run a command and get output?"""
r = _exec(env, "echo hello")
assert "hello" in _output(r)
assert _rc(r) == 0
def test_exit_code_success(self, env):
r = _exec(env, "true")
assert _rc(r) == 0
def test_exit_code_failure(self, env):
r = _exec(env, "false")
assert _rc(r) != 0
def test_stderr_captured(self, env):
"""Agent relies on 2>&1 patterns; stderr must be captured."""
r = _exec(env, "echo err >&2")
# stderr may be merged into output or separate — just verify no crash
assert isinstance(_output(r), str)
def test_multiline_output(self, env):
r = _exec(env, "printf 'line1\\nline2\\nline3'")
lines = _output(r).strip().split("\n")
assert len(lines) >= 3
# ---------------------------------------------------------------------------
# Category 2: cd && command chains
# 37% of all terminal commands use this pattern. CRITICAL.
# ---------------------------------------------------------------------------
class TestCdAndChain:
def test_cd_and_command(self, env):
"""cd /tmp && ls — the most common pattern in session data."""
r = _exec(env, "cd /tmp && echo 'in_tmp'")
assert "in_tmp" in _output(r)
def test_chained_and(self, env):
"""Multiple && chains: agent does cd X && source Y && cmd Z."""
r = _exec(env, "echo a && echo b && echo c")
out = _output(r)
assert "a" in out and "b" in out and "c" in out
def test_chained_semicolon(self, env):
"""Semicolon chains: agent uses '; echo "---"' as separators."""
r = _exec(env, "echo first; echo '---'; echo second")
out = _output(r)
assert "first" in out and "---" in out and "second" in out
def test_cd_nonexistent_and_fails(self, env):
"""cd to bad dir && cmd should fail (not run cmd)."""
r = _exec(env, "cd /nonexistent_dir_xyz && echo should_not_see")
assert "should_not_see" not in _output(r)
def test_cwd_persists_across_calls(self, env):
"""CWD now persists via cwdfile tracking (unified execution model).
Previously: 37% of commands needed 'cd X &&' prefix. Now automatic."""
_exec(env, "cd /tmp")
r = _exec(env, "pwd")
assert "/tmp" in _output(r)
# ---------------------------------------------------------------------------
# Category 3: Pipes
# 46% of commands use pipes. Essential.
# ---------------------------------------------------------------------------
class TestPipes:
def test_simple_pipe(self, env):
r = _exec(env, "echo 'hello world' | wc -w")
assert "2" in _output(r)
def test_multi_pipe(self, env):
"""Agent chains: find X | grep Y | head -N"""
r = _exec(env, "echo -e 'a\\nb\\nc\\nd\\ne' | grep -v c | wc -l")
assert "4" in _output(r)
def test_pipe_with_grep(self, env):
"""Common pattern: cmd 2>&1 | grep pattern"""
r = _exec(env, "echo -e 'foo\\nbar\\nbaz' | grep ba")
out = _output(r)
assert "bar" in out and "baz" in out
assert "foo" not in out
# ---------------------------------------------------------------------------
# Category 4: Environment variables and source
# 19% of commands use source. Agent does: source ~/.bashrc && cmd
# ---------------------------------------------------------------------------
class TestEnvAndSource:
def test_inline_env_var(self, env):
r = _exec(env, "MY_VAR=hello && echo $MY_VAR")
assert "hello" in _output(r)
def test_export_and_use(self, env):
r = _exec(env, "export FOO=bar && echo $FOO")
assert "bar" in _output(r)
def test_env_does_not_persist(self, env):
"""Env vars don't persist across execute() calls."""
_exec(env, "export HERMES_TEST_VAR=1234")
r = _exec(env, "echo ${HERMES_TEST_VAR:-unset}")
assert "unset" in _output(r)
def test_source_inline_script(self, env):
"""Agent pattern: write a file, source it, use its vars."""
r = _exec(env, (
"echo 'export TEST_SOURCED=yes' > /tmp/hermes_test_source.sh && "
"source /tmp/hermes_test_source.sh && "
"echo $TEST_SOURCED"
))
assert "yes" in _output(r)
# ---------------------------------------------------------------------------
# Category 5: File I/O via shell
# Agent uses cat, heredoc, find, ls extensively
# ---------------------------------------------------------------------------
class TestFileIO:
def test_write_and_read(self, env):
r = _exec(env, (
"echo 'test content' > /tmp/hermes_test_file.txt && "
"cat /tmp/hermes_test_file.txt"
))
assert "test content" in _output(r)
def test_heredoc_write(self, env):
"""0.5% of commands use heredoc — rare but important for config files."""
r = _exec(env, """cat > /tmp/hermes_heredoc_test.txt << 'EOF'
line one
line two
line three
EOF
cat /tmp/hermes_heredoc_test.txt""")
out = _output(r)
assert "line one" in out and "line three" in out
def test_mkdir_p(self, env):
r = _exec(env, "mkdir -p /tmp/hermes_test_deep/a/b/c && ls /tmp/hermes_test_deep/a/b/")
assert "c" in _output(r)
def test_find(self, env):
r = _exec(env, (
"mkdir -p /tmp/hermes_find_test && "
"touch /tmp/hermes_find_test/a.py /tmp/hermes_find_test/b.txt && "
"find /tmp/hermes_find_test -name '*.py'"
))
assert "a.py" in _output(r)
def test_file_persistence_within_session(self, env):
"""Files written and read in a single execute() call."""
r = _exec(env, (
"echo 'persistent' > /tmp/hermes_persist_test.txt && "
"cat /tmp/hermes_persist_test.txt"
))
assert "persistent" in _output(r)
# ---------------------------------------------------------------------------
# Category 6: Multiline commands
# 6% of commands are multiline. Agent sends literal newlines.
# ---------------------------------------------------------------------------
class TestMultiline:
def test_multiline_script(self, env):
r = _exec(env, """echo "step 1"
echo "step 2"
echo "step 3" """)
out = _output(r)
assert "step 1" in out and "step 3" in out
def test_multiline_with_variable(self, env):
r = _exec(env, """X=42
echo "value is $X" """)
assert "value is 42" in _output(r)
# ---------------------------------------------------------------------------
# Category 7: Timeouts
# 50% of terminal calls specify a timeout. Some go up to 1800s.
# ---------------------------------------------------------------------------
class TestTimeouts:
def test_fast_command_with_timeout(self, env):
r = _exec(env, "echo fast", timeout=5)
assert "fast" in _output(r)
def test_slow_command_timeout(self, env):
"""Command that exceeds timeout should be killed."""
start = time.time()
r = _exec(env, "sleep 60", timeout=3)
elapsed = time.time() - start
# Should return in roughly timeout seconds, not 60
assert elapsed < 15, f"Command took {elapsed}s, should have timed out at ~3s"
# ---------------------------------------------------------------------------
# Category 8: Output handling
# Verifying the contract: {output: str, exit_code/returncode: int, error: ...}
# ---------------------------------------------------------------------------
class TestOutputContract:
def test_result_has_output_key(self, env):
r = _exec(env, "echo test")
assert "output" in r
def test_result_has_returncode(self, env):
r = _exec(env, "echo test")
assert "returncode" in r or "exit_code" in r
def test_large_output_not_truncated_at_execute_level(self, env):
"""The env.execute() should return raw output.
Truncation happens in terminal_tool.py, not in the backend."""
r = _exec(env, "seq 1 5000")
lines = _output(r).strip().split("\n")
# Should get all 5000 lines from the backend itself
assert len(lines) >= 4900, f"Expected ~5000 lines, got {len(lines)}"
def test_binary_output_doesnt_crash(self, env):
"""Agent sometimes runs commands that produce partial binary output."""
r = _exec(env, "echo -e '\\x00\\x01\\x02hello\\x03'")
# Just verify it doesn't crash
assert isinstance(_output(r), str)
# ---------------------------------------------------------------------------
# Category 9: Package/tool availability
# Agent frequently checks for tools before using them
# ---------------------------------------------------------------------------
class TestToolAvailability:
def test_which_pattern(self, env):
"""Agent pattern: which X 2>/dev/null || echo 'not found'"""
r = _exec(env, "which bash 2>/dev/null || echo 'not found'")
out = _output(r)
assert "bash" in out or "not found" in out
def test_python_available(self, env):
"""Agent uses python3 extensively."""
r = _exec(env, "which python3 2>/dev/null && python3 --version || echo 'no python3'")
assert "Python" in _output(r) or "no python3" in _output(r)
def test_git_available(self, env):
"""52 git operations in session data."""
r = _exec(env, "which git 2>/dev/null && git --version || echo 'no git'")
assert "git" in _output(r).lower() or "no git" in _output(r)
# ---------------------------------------------------------------------------
# Category 10: Error handling edge cases
# ---------------------------------------------------------------------------
class TestEdgeCases:
def test_empty_command(self, env):
"""Empty or whitespace command shouldn't crash."""
try:
r = _exec(env, "")
# May succeed with empty output or fail gracefully
except Exception:
pass # Acceptable to raise
def test_command_not_found(self, env):
r = _exec(env, "nonexistent_command_xyz_123 2>&1")
assert _rc(r) != 0
def test_special_characters_in_output(self, env):
"""Agent processes JSON, YAML, code — special chars must survive."""
r = _exec(env, """echo '{"key": "value", "list": [1,2,3]}'""")
out = _output(r)
assert '"key"' in out
def test_long_command_string(self, env):
"""Agent sends commands up to ~500 chars. Verify no truncation on input."""
long_val = "A" * 500
r = _exec(env, f"echo {long_val} | wc -c")
count = int(_output(r).strip())
assert count >= 500
# ---------------------------------------------------------------------------
# Category 11: Unified execution model — new capabilities
# ---------------------------------------------------------------------------
class TestUnifiedExecution:
def test_cwd_tracking_updates_env(self, env):
"""env.cwd should update after cd command."""
_exec(env, "cd /tmp")
assert env.cwd == "/tmp"
def test_stdin_data(self, env):
"""stdin_data should be piped to the command."""
r = env.execute("cat", stdin_data="hello from stdin\n")
assert "hello from stdin" in _output(r)
def test_snapshot_fallback(self, env):
"""Commands work even when snapshot is missing/broken."""
old_snapshot = env._snapshot_ready
old_path = env._snapshot_path
env._snapshot_ready = False
env._snapshot_path = None
try:
r = _exec(env, "echo still_works")
assert "still_works" in _output(r)
finally:
env._snapshot_ready = old_snapshot
env._snapshot_path = old_path
def test_exit_code_preserved_through_wrapper(self, env):
"""Exit code from the user command should pass through the wrapper."""
r = _exec(env, "exit 42")
assert _rc(r) == 42
def test_single_quotes_in_command(self, env):
"""Commands with single quotes must survive the eval wrapper."""
r = _exec(env, "echo 'it'\\''s a test'")
assert "it's a test" in _output(r)
# ---------------------------------------------------------------------------
# Cleanup
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True, scope="module")
def cleanup_test_files(env):
"""Clean up test artifacts after all tests."""
yield
try:
env.execute("rm -rf /tmp/hermes_test_* /tmp/hermes_find_test /tmp/hermes_heredoc_test.txt /tmp/hermes_persist_test.txt /tmp/hermes_test_source.sh", timeout=5)
except Exception:
pass
+269
View File
@@ -0,0 +1,269 @@
"""Tests for Google AI Studio (Gemini) provider integration."""
import os
import pytest
from unittest.mock import patch, MagicMock
from hermes_cli.auth import PROVIDER_REGISTRY, resolve_provider, resolve_api_key_provider_credentials
from hermes_cli.models import _PROVIDER_MODELS, _PROVIDER_LABELS, _PROVIDER_ALIASES, normalize_provider
from hermes_cli.model_normalize import normalize_model_for_provider, detect_vendor
from agent.model_metadata import get_model_context_length
from agent.models_dev import PROVIDER_TO_MODELS_DEV, list_agentic_models, _NOISE_PATTERNS
# ── Provider Registry ──
class TestGeminiProviderRegistry:
def test_gemini_in_registry(self):
assert "gemini" in PROVIDER_REGISTRY
def test_gemini_config(self):
pconfig = PROVIDER_REGISTRY["gemini"]
assert pconfig.id == "gemini"
assert pconfig.name == "Google AI Studio"
assert pconfig.auth_type == "api_key"
assert pconfig.inference_base_url == "https://generativelanguage.googleapis.com/v1beta/openai"
def test_gemini_env_vars(self):
pconfig = PROVIDER_REGISTRY["gemini"]
assert pconfig.api_key_env_vars == ("GOOGLE_API_KEY", "GEMINI_API_KEY")
assert pconfig.base_url_env_var == "GEMINI_BASE_URL"
def test_gemini_base_url(self):
assert "generativelanguage.googleapis.com" in PROVIDER_REGISTRY["gemini"].inference_base_url
# ── Provider Aliases ──
PROVIDER_ENV_VARS = (
"OPENROUTER_API_KEY", "OPENAI_API_KEY", "ANTHROPIC_API_KEY",
"GOOGLE_API_KEY", "GEMINI_API_KEY", "GEMINI_BASE_URL",
"GLM_API_KEY", "ZAI_API_KEY", "KIMI_API_KEY",
"MINIMAX_API_KEY", "DEEPSEEK_API_KEY",
)
@pytest.fixture(autouse=True)
def _clean_provider_env(monkeypatch):
for var in PROVIDER_ENV_VARS:
monkeypatch.delenv(var, raising=False)
class TestGeminiAliases:
def test_explicit_gemini(self):
assert resolve_provider("gemini") == "gemini"
def test_alias_google(self):
assert resolve_provider("google") == "gemini"
def test_alias_google_gemini(self):
assert resolve_provider("google-gemini") == "gemini"
def test_alias_google_ai_studio(self):
assert resolve_provider("google-ai-studio") == "gemini"
def test_models_py_aliases(self):
assert _PROVIDER_ALIASES.get("google") == "gemini"
assert _PROVIDER_ALIASES.get("google-gemini") == "gemini"
assert _PROVIDER_ALIASES.get("google-ai-studio") == "gemini"
def test_normalize_provider(self):
assert normalize_provider("google") == "gemini"
assert normalize_provider("gemini") == "gemini"
assert normalize_provider("google-ai-studio") == "gemini"
# ── Auto-detection ──
class TestGeminiAutoDetection:
def test_auto_detects_google_api_key(self, monkeypatch):
monkeypatch.setenv("GOOGLE_API_KEY", "test-google-key")
assert resolve_provider("auto") == "gemini"
def test_auto_detects_gemini_api_key(self, monkeypatch):
monkeypatch.setenv("GEMINI_API_KEY", "test-gemini-key")
assert resolve_provider("auto") == "gemini"
def test_google_api_key_priority_over_gemini(self, monkeypatch):
monkeypatch.setenv("GOOGLE_API_KEY", "primary-key")
monkeypatch.setenv("GEMINI_API_KEY", "alias-key")
creds = resolve_api_key_provider_credentials("gemini")
assert creds["api_key"] == "primary-key"
assert creds["source"] == "GOOGLE_API_KEY"
# ── Credential Resolution ──
class TestGeminiCredentials:
def test_resolve_with_google_api_key(self, monkeypatch):
monkeypatch.setenv("GOOGLE_API_KEY", "google-secret")
creds = resolve_api_key_provider_credentials("gemini")
assert creds["provider"] == "gemini"
assert creds["api_key"] == "google-secret"
assert creds["base_url"] == "https://generativelanguage.googleapis.com/v1beta/openai"
def test_resolve_with_gemini_api_key(self, monkeypatch):
monkeypatch.setenv("GEMINI_API_KEY", "gemini-secret")
creds = resolve_api_key_provider_credentials("gemini")
assert creds["api_key"] == "gemini-secret"
def test_resolve_with_custom_base_url(self, monkeypatch):
monkeypatch.setenv("GOOGLE_API_KEY", "key")
monkeypatch.setenv("GEMINI_BASE_URL", "https://custom.endpoint/v1")
creds = resolve_api_key_provider_credentials("gemini")
assert creds["base_url"] == "https://custom.endpoint/v1"
def test_runtime_gemini(self, monkeypatch):
monkeypatch.setenv("GOOGLE_API_KEY", "google-key")
from hermes_cli.runtime_provider import resolve_runtime_provider
result = resolve_runtime_provider(requested="gemini")
assert result["provider"] == "gemini"
assert result["api_mode"] == "chat_completions"
assert result["api_key"] == "google-key"
assert result["base_url"] == "https://generativelanguage.googleapis.com/v1beta/openai"
# ── Model Catalog ──
class TestGeminiModelCatalog:
def test_provider_models_exist(self):
assert "gemini" in _PROVIDER_MODELS
models = _PROVIDER_MODELS["gemini"]
assert "gemini-2.5-pro" in models
assert "gemini-2.5-flash" in models
assert "gemma-4-31b-it" in models
def test_provider_models_has_3x(self):
models = _PROVIDER_MODELS["gemini"]
assert "gemini-3.1-pro-preview" in models
assert "gemini-3-flash-preview" in models
assert "gemini-3.1-flash-lite-preview" in models
def test_provider_label(self):
assert "gemini" in _PROVIDER_LABELS
assert _PROVIDER_LABELS["gemini"] == "Google AI Studio"
# ── Model Normalization ──
class TestGeminiModelNormalization:
def test_passthrough_bare_name(self):
assert normalize_model_for_provider("gemini-2.5-flash", "gemini") == "gemini-2.5-flash"
def test_strip_vendor_prefix(self):
assert normalize_model_for_provider("google/gemini-2.5-flash", "gemini") == "google/gemini-2.5-flash"
def test_gemma_vendor_detection(self):
assert detect_vendor("gemma-4-31b-it") == "google"
def test_gemini_vendor_detection(self):
assert detect_vendor("gemini-2.5-flash") == "google"
def test_aggregator_prepends_vendor(self):
result = normalize_model_for_provider("gemini-2.5-flash", "openrouter")
assert result == "google/gemini-2.5-flash"
def test_gemma_aggregator_prepends_vendor(self):
result = normalize_model_for_provider("gemma-4-31b-it", "openrouter")
assert result == "google/gemma-4-31b-it"
# ── Context Length ──
class TestGeminiContextLength:
def test_gemma_4_31b_context(self):
ctx = get_model_context_length("gemma-4-31b-it", provider="gemini")
assert ctx == 256000
def test_gemma_4_26b_context(self):
ctx = get_model_context_length("gemma-4-26b-it", provider="gemini")
assert ctx == 256000
def test_gemini_3_context(self):
ctx = get_model_context_length("gemini-3.1-pro-preview", provider="gemini")
assert ctx == 1048576
# ── Agent Init (no SyntaxError) ──
class TestGeminiAgentInit:
def test_agent_imports_without_error(self):
"""Verify run_agent.py has no SyntaxError (the critical bug)."""
import importlib
import run_agent
importlib.reload(run_agent)
def test_gemini_agent_uses_chat_completions(self, monkeypatch):
"""Gemini falls through to chat_completions — no special elif needed."""
monkeypatch.setenv("GOOGLE_API_KEY", "test-key")
with patch("run_agent.OpenAI") as mock_openai:
mock_openai.return_value = MagicMock()
from run_agent import AIAgent
agent = AIAgent(
model="gemini-2.5-flash",
provider="gemini",
api_key="test-key",
base_url="https://generativelanguage.googleapis.com/v1beta/openai",
)
assert agent.api_mode == "chat_completions"
assert agent.provider == "gemini"
# ── models.dev Integration ──
class TestGeminiModelsDev:
def test_gemini_mapped_to_google(self):
assert PROVIDER_TO_MODELS_DEV.get("gemini") == "google"
def test_noise_filter_excludes_tts(self):
assert _NOISE_PATTERNS.search("gemini-2.5-pro-preview-tts")
def test_noise_filter_excludes_dated_preview(self):
assert _NOISE_PATTERNS.search("gemini-2.5-flash-preview-04-17")
def test_noise_filter_excludes_embedding(self):
assert _NOISE_PATTERNS.search("gemini-embedding-001")
def test_noise_filter_excludes_live(self):
assert _NOISE_PATTERNS.search("gemini-live-2.5-flash")
def test_noise_filter_excludes_image(self):
assert _NOISE_PATTERNS.search("gemini-2.5-flash-image")
def test_noise_filter_excludes_customtools(self):
assert _NOISE_PATTERNS.search("gemini-3.1-pro-preview-customtools")
def test_noise_filter_passes_stable(self):
assert not _NOISE_PATTERNS.search("gemini-2.5-flash")
def test_noise_filter_passes_preview(self):
# Non-dated preview (e.g. gemini-3-flash-preview) should pass
assert not _NOISE_PATTERNS.search("gemini-3-flash-preview")
def test_noise_filter_passes_gemma(self):
assert not _NOISE_PATTERNS.search("gemma-4-31b-it")
def test_list_agentic_models_with_mock_data(self):
"""list_agentic_models filters correctly from mock models.dev data."""
mock_data = {
"google": {
"models": {
"gemini-3-flash-preview": {"tool_call": True},
"gemini-2.5-pro": {"tool_call": True},
"gemini-embedding-001": {"tool_call": False},
"gemini-2.5-flash-preview-tts": {"tool_call": False},
"gemini-live-2.5-flash": {"tool_call": True},
"gemini-2.5-flash-preview-04-17": {"tool_call": True},
"gemma-4-31b-it": {"tool_call": True},
}
}
}
with patch("agent.models_dev.fetch_models_dev", return_value=mock_data):
result = list_agentic_models("gemini")
assert "gemini-3-flash-preview" in result
assert "gemini-2.5-pro" in result
assert "gemma-4-31b-it" in result
# Filtered out:
assert "gemini-embedding-001" not in result # no tool_call
assert "gemini-2.5-flash-preview-tts" not in result # no tool_call
assert "gemini-live-2.5-flash" not in result # noise: live-
assert "gemini-2.5-flash-preview-04-17" not in result # noise: dated preview
-162
View File
@@ -1,162 +0,0 @@
"""Tests for _save_oversized_tool_result() — the large tool response handler.
When a tool returns more than _LARGE_RESULT_CHARS characters, the full content
is saved to a file and the model receives a preview + file path instead.
"""
import os
import re
import pytest
from run_agent import (
_save_oversized_tool_result,
_LARGE_RESULT_CHARS,
_LARGE_RESULT_PREVIEW_CHARS,
)
class TestSaveOversizedToolResult:
"""Unit tests for the large tool result handler."""
def test_small_result_returned_unchanged(self):
"""Results under the threshold pass through untouched."""
small = "x" * 1000
assert _save_oversized_tool_result("terminal", small) is small
def test_exactly_at_threshold_returned_unchanged(self):
"""Results exactly at the threshold pass through."""
exact = "y" * _LARGE_RESULT_CHARS
assert _save_oversized_tool_result("terminal", exact) is exact
def test_oversized_result_saved_to_file(self, tmp_path, monkeypatch):
"""Results over the threshold are written to a file."""
monkeypatch.setenv("HERMES_HOME", str(tmp_path / ".hermes"))
os.makedirs(tmp_path / ".hermes", exist_ok=True)
big = "A" * (_LARGE_RESULT_CHARS + 500)
result = _save_oversized_tool_result("terminal", big)
# Should contain the preview
assert result.startswith("A" * _LARGE_RESULT_PREVIEW_CHARS)
# Should mention the file path
assert "Full output saved to:" in result
# Should mention original size
assert f"{len(big):,}" in result
# Extract the file path and verify the file exists with full content
match = re.search(r"Full output saved to: (.+?)\n", result)
assert match, f"No file path found in result: {result[:300]}"
filepath = match.group(1)
assert os.path.isfile(filepath)
with open(filepath, "r", encoding="utf-8") as f:
saved = f.read()
assert saved == big
assert len(saved) == _LARGE_RESULT_CHARS + 500
def test_file_placed_in_cache_tool_responses(self, tmp_path, monkeypatch):
"""Saved file lives under HERMES_HOME/cache/tool_responses/."""
hermes_home = str(tmp_path / ".hermes")
monkeypatch.setenv("HERMES_HOME", hermes_home)
os.makedirs(hermes_home, exist_ok=True)
big = "B" * (_LARGE_RESULT_CHARS + 1)
result = _save_oversized_tool_result("web_search", big)
match = re.search(r"Full output saved to: (.+?)\n", result)
filepath = match.group(1)
expected_dir = os.path.join(hermes_home, "cache", "tool_responses")
assert filepath.startswith(expected_dir)
def test_filename_contains_tool_name(self, tmp_path, monkeypatch):
"""The saved filename includes a sanitized version of the tool name."""
monkeypatch.setenv("HERMES_HOME", str(tmp_path / ".hermes"))
os.makedirs(tmp_path / ".hermes", exist_ok=True)
big = "C" * (_LARGE_RESULT_CHARS + 1)
result = _save_oversized_tool_result("browser_navigate", big)
match = re.search(r"Full output saved to: (.+?)\n", result)
filename = os.path.basename(match.group(1))
assert filename.startswith("browser_navigate_")
assert filename.endswith(".txt")
def test_tool_name_sanitized(self, tmp_path, monkeypatch):
"""Special characters in tool names are replaced in the filename."""
monkeypatch.setenv("HERMES_HOME", str(tmp_path / ".hermes"))
os.makedirs(tmp_path / ".hermes", exist_ok=True)
big = "D" * (_LARGE_RESULT_CHARS + 1)
result = _save_oversized_tool_result("mcp:some/weird tool", big)
match = re.search(r"Full output saved to: (.+?)\n", result)
filename = os.path.basename(match.group(1))
# No slashes or colons in filename
assert "/" not in filename
assert ":" not in filename
def test_fallback_on_write_failure(self, tmp_path, monkeypatch):
"""When file write fails, falls back to destructive truncation."""
# Point HERMES_HOME to a path that will fail (file, not directory)
bad_path = str(tmp_path / "not_a_dir.txt")
with open(bad_path, "w") as f:
f.write("I'm a file, not a directory")
monkeypatch.setenv("HERMES_HOME", bad_path)
big = "E" * (_LARGE_RESULT_CHARS + 50_000)
result = _save_oversized_tool_result("terminal", big)
# Should still contain data (fallback truncation)
assert len(result) > 0
assert result.startswith("E" * 1000)
# Should mention the failure
assert "File save failed" in result
# Should be truncated to approximately _LARGE_RESULT_CHARS + error msg
assert len(result) < len(big)
def test_preview_length_capped(self, tmp_path, monkeypatch):
"""The inline preview is capped at _LARGE_RESULT_PREVIEW_CHARS."""
monkeypatch.setenv("HERMES_HOME", str(tmp_path / ".hermes"))
os.makedirs(tmp_path / ".hermes", exist_ok=True)
# Use distinct chars so we can measure the preview
big = "Z" * (_LARGE_RESULT_CHARS + 5000)
result = _save_oversized_tool_result("terminal", big)
# The preview section is the content before the "[Large tool response:" marker
marker_pos = result.index("[Large tool response:")
preview_section = result[:marker_pos].rstrip()
assert len(preview_section) == _LARGE_RESULT_PREVIEW_CHARS
def test_guidance_message_mentions_tools(self, tmp_path, monkeypatch):
"""The replacement message tells the model how to access the file."""
monkeypatch.setenv("HERMES_HOME", str(tmp_path / ".hermes"))
os.makedirs(tmp_path / ".hermes", exist_ok=True)
big = "F" * (_LARGE_RESULT_CHARS + 1)
result = _save_oversized_tool_result("terminal", big)
assert "read_file" in result
assert "search_files" in result
def test_empty_result_passes_through(self):
"""Empty strings are not oversized."""
assert _save_oversized_tool_result("terminal", "") == ""
def test_unicode_content_preserved(self, tmp_path, monkeypatch):
"""Unicode content is fully preserved in the saved file."""
monkeypatch.setenv("HERMES_HOME", str(tmp_path / ".hermes"))
os.makedirs(tmp_path / ".hermes", exist_ok=True)
# Mix of ASCII and multi-byte unicode to exceed threshold
unit = "Hello 世界! 🎉 " * 100 # ~1400 chars per repeat
big = unit * ((_LARGE_RESULT_CHARS // len(unit)) + 1)
assert len(big) > _LARGE_RESULT_CHARS
result = _save_oversized_tool_result("terminal", big)
match = re.search(r"Full output saved to: (.+?)\n", result)
filepath = match.group(1)
with open(filepath, "r", encoding="utf-8") as f:
saved = f.read()
assert saved == big
+116
View File
@@ -0,0 +1,116 @@
"""Tests for hermes_cli.model_normalize — provider-aware model name normalization.
Covers issue #5211: opencode-go model names with dots (e.g. minimax-m2.7)
must NOT be mangled to hyphens (minimax-m2-7).
"""
import pytest
from hermes_cli.model_normalize import (
normalize_model_for_provider,
_DOT_TO_HYPHEN_PROVIDERS,
_AGGREGATOR_PROVIDERS,
detect_vendor,
)
# ── Regression: issue #5211 ────────────────────────────────────────────
class TestIssue5211OpenCodeGoDotPreservation:
"""OpenCode Go model names with dots must pass through unchanged."""
@pytest.mark.parametrize("model,expected", [
("minimax-m2.7", "minimax-m2.7"),
("minimax-m2.5", "minimax-m2.5"),
("glm-4.5", "glm-4.5"),
("kimi-k2.5", "kimi-k2.5"),
("some-model-1.0.3", "some-model-1.0.3"),
])
def test_opencode_go_preserves_dots(self, model, expected):
result = normalize_model_for_provider(model, "opencode-go")
assert result == expected, f"Expected {expected!r}, got {result!r}"
def test_opencode_go_not_in_dot_to_hyphen_set(self):
"""opencode-go must NOT be in the dot-to-hyphen provider set."""
assert "opencode-go" not in _DOT_TO_HYPHEN_PROVIDERS
# ── Anthropic dot-to-hyphen conversion (regression) ────────────────────
class TestAnthropicDotToHyphen:
"""Anthropic API still needs dots→hyphens."""
@pytest.mark.parametrize("model,expected", [
("claude-sonnet-4.6", "claude-sonnet-4-6"),
("claude-opus-4.5", "claude-opus-4-5"),
])
def test_anthropic_converts_dots(self, model, expected):
result = normalize_model_for_provider(model, "anthropic")
assert result == expected
def test_anthropic_strips_vendor_prefix(self):
result = normalize_model_for_provider("anthropic/claude-sonnet-4.6", "anthropic")
assert result == "claude-sonnet-4-6"
# ── OpenCode Zen regression ────────────────────────────────────────────
class TestOpenCodeZenDotToHyphen:
"""OpenCode Zen follows Anthropic convention (dots→hyphens)."""
@pytest.mark.parametrize("model,expected", [
("claude-sonnet-4.6", "claude-sonnet-4-6"),
("glm-4.5", "glm-4-5"),
])
def test_zen_converts_dots(self, model, expected):
result = normalize_model_for_provider(model, "opencode-zen")
assert result == expected
def test_zen_strips_vendor_prefix(self):
result = normalize_model_for_provider("opencode-zen/claude-sonnet-4.6", "opencode-zen")
assert result == "claude-sonnet-4-6"
# ── Copilot dot preservation (regression) ──────────────────────────────
class TestCopilotDotPreservation:
"""Copilot preserves dots in model names."""
@pytest.mark.parametrize("model,expected", [
("claude-sonnet-4.6", "claude-sonnet-4.6"),
("gpt-5.4", "gpt-5.4"),
])
def test_copilot_preserves_dots(self, model, expected):
result = normalize_model_for_provider(model, "copilot")
assert result == expected
# ── Aggregator providers (regression) ──────────────────────────────────
class TestAggregatorProviders:
"""Aggregators need vendor/model slugs."""
def test_openrouter_prepends_vendor(self):
result = normalize_model_for_provider("claude-sonnet-4.6", "openrouter")
assert result == "anthropic/claude-sonnet-4.6"
def test_nous_prepends_vendor(self):
result = normalize_model_for_provider("gpt-5.4", "nous")
assert result == "openai/gpt-5.4"
def test_vendor_already_present(self):
result = normalize_model_for_provider("anthropic/claude-sonnet-4.6", "openrouter")
assert result == "anthropic/claude-sonnet-4.6"
# ── detect_vendor ──────────────────────────────────────────────────────
class TestDetectVendor:
@pytest.mark.parametrize("model,expected", [
("claude-sonnet-4.6", "anthropic"),
("gpt-5.4-mini", "openai"),
("minimax-m2.7", "minimax"),
("glm-4.5", "z-ai"),
("kimi-k2.5", "moonshotai"),
])
def test_detects_known_vendors(self, model, expected):
assert detect_vendor(model) == expected
+112
View File
@@ -443,3 +443,115 @@ class TestCopyExampleFiles:
# Should have printed a warning
assert any("Warning" in str(c) for c in console.print.call_args_list)
class TestPromptPluginEnvVars:
"""Tests for _prompt_plugin_env_vars."""
def test_skips_when_no_requires_env(self):
from hermes_cli.plugins_cmd import _prompt_plugin_env_vars
from unittest.mock import MagicMock
console = MagicMock()
_prompt_plugin_env_vars({}, console)
console.print.assert_not_called()
def test_skips_already_set_vars(self, monkeypatch):
from hermes_cli.plugins_cmd import _prompt_plugin_env_vars
from unittest.mock import MagicMock, patch
console = MagicMock()
with patch("hermes_cli.config.get_env_value", return_value="already-set"):
_prompt_plugin_env_vars({"requires_env": ["MY_KEY"]}, console)
# No prompt should appear — all vars are set
console.print.assert_not_called()
def test_prompts_for_missing_var_simple_format(self):
from hermes_cli.plugins_cmd import _prompt_plugin_env_vars
from unittest.mock import MagicMock, patch
console = MagicMock()
manifest = {
"name": "test_plugin",
"requires_env": ["MY_API_KEY"],
}
with patch("hermes_cli.config.get_env_value", return_value=None), \
patch("builtins.input", return_value="sk-test-123"), \
patch("hermes_cli.config.save_env_value") as mock_save:
_prompt_plugin_env_vars(manifest, console)
mock_save.assert_called_once_with("MY_API_KEY", "sk-test-123")
def test_prompts_for_missing_var_rich_format(self):
from hermes_cli.plugins_cmd import _prompt_plugin_env_vars
from unittest.mock import MagicMock, patch
console = MagicMock()
manifest = {
"name": "langfuse_tracing",
"requires_env": [
{
"name": "LANGFUSE_PUBLIC_KEY",
"description": "Public key",
"url": "https://langfuse.com",
"secret": False,
},
],
}
with patch("hermes_cli.config.get_env_value", return_value=None), \
patch("builtins.input", return_value="pk-lf-123"), \
patch("hermes_cli.config.save_env_value") as mock_save:
_prompt_plugin_env_vars(manifest, console)
mock_save.assert_called_once_with("LANGFUSE_PUBLIC_KEY", "pk-lf-123")
# Should show url hint
printed = " ".join(str(c) for c in console.print.call_args_list)
assert "langfuse.com" in printed
def test_secret_uses_getpass(self):
from hermes_cli.plugins_cmd import _prompt_plugin_env_vars
from unittest.mock import MagicMock, patch
console = MagicMock()
manifest = {
"name": "test",
"requires_env": [{"name": "SECRET_KEY", "secret": True}],
}
with patch("hermes_cli.config.get_env_value", return_value=None), \
patch("getpass.getpass", return_value="s3cret") as mock_gp, \
patch("hermes_cli.config.save_env_value"):
_prompt_plugin_env_vars(manifest, console)
mock_gp.assert_called_once()
def test_empty_input_skips(self):
from hermes_cli.plugins_cmd import _prompt_plugin_env_vars
from unittest.mock import MagicMock, patch
console = MagicMock()
manifest = {"name": "test", "requires_env": ["OPTIONAL_VAR"]}
with patch("hermes_cli.config.get_env_value", return_value=None), \
patch("builtins.input", return_value=""), \
patch("hermes_cli.config.save_env_value") as mock_save:
_prompt_plugin_env_vars(manifest, console)
mock_save.assert_not_called()
def test_keyboard_interrupt_skips_gracefully(self):
from hermes_cli.plugins_cmd import _prompt_plugin_env_vars
from unittest.mock import MagicMock, patch
console = MagicMock()
manifest = {"name": "test", "requires_env": ["KEY1", "KEY2"]}
with patch("hermes_cli.config.get_env_value", return_value=None), \
patch("builtins.input", side_effect=KeyboardInterrupt), \
patch("hermes_cli.config.save_env_value") as mock_save:
_prompt_plugin_env_vars(manifest, console)
# Should not crash, and not save anything
mock_save.assert_not_called()
+1 -1
View File
@@ -63,4 +63,4 @@ class TestCamofoxConfigDefaults:
from hermes_cli.config import DEFAULT_CONFIG
# managed_persistence is auto-merged by _deep_merge, no version bump needed
assert DEFAULT_CONFIG["_config_version"] == 11
assert DEFAULT_CONFIG["_config_version"] == 12
+107 -74
View File
@@ -123,14 +123,40 @@ class TestCwdResolution:
assert env.cwd == "/home/testuser"
def test_explicit_cwd_not_overridden(self, make_env):
env = make_env(cwd="/workspace", home_dir="/root")
"""Explicit cwd should be set before init_session.
After init_session(), the cwdfile may update cwd to whatever the
login shell reports. We make the mock return /workspace for the
cwdfile read so init_session doesn't override the explicit cwd.
"""
sb = _make_sandbox()
# Return /workspace for all exec calls including init_session's
# snapshot bootstrap and cwdfile reads
sb.process.exec.return_value = _make_exec_response(result="/workspace")
env = make_env(sandbox=sb, cwd="/workspace", home_dir="/workspace")
assert env.cwd == "/workspace"
def test_home_detection_failure_keeps_default_cwd(self, make_env):
"""When $HOME detection fails, cwd falls back to constructor default.
init_session() still runs but its cwdfile read returns empty,
so cwd is not overwritten.
"""
sb = _make_sandbox()
sb.process.exec.side_effect = RuntimeError("exec failed")
call_count = {"n": 0}
def _exec_side_effect(*args, **kwargs):
call_count["n"] += 1
if call_count["n"] == 1:
# $HOME detection fails
raise RuntimeError("exec failed")
# All subsequent calls (init_session, cwdfile reads) succeed
# but return empty so they don't override cwd
return _make_exec_response(result="", exit_code=0)
sb.process.exec.side_effect = _exec_side_effect
env = make_env(sandbox=sb)
assert env.cwd == "/home/daytona" # keeps constructor default
assert env.cwd == "/home/daytona"
def test_empty_home_keeps_default_cwd(self, make_env):
env = make_env(home_dir="")
@@ -221,104 +247,107 @@ class TestCleanup:
class TestExecute:
def test_basic_command(self, make_env):
sb = _make_sandbox()
# First call: $HOME detection; subsequent calls: actual commands
sb.process.exec.side_effect = [
_make_exec_response(result="/root"), # $HOME
_make_exec_response(result="hello", exit_code=0), # actual cmd
]
# Calls: $HOME detection, init_session bootstrap, init_session cat,
# _before_execute sandbox refresh, _run_bash command, _update_cwd cat
sb.process.exec.return_value = _make_exec_response(result="/root")
sb.state = "started"
env = make_env(sandbox=sb)
# Reset mock to control just the execute() calls
sb.process.exec.reset_mock()
sb.process.exec.return_value = _make_exec_response(result="hello", exit_code=0)
result = env.execute("echo hello")
assert result["output"] == "hello"
assert "hello" in result["output"]
assert result["returncode"] == 0
def test_command_wrapped_with_shell_timeout(self, make_env):
sb = _make_sandbox()
sb.process.exec.side_effect = [
_make_exec_response(result="/root"),
_make_exec_response(result="ok", exit_code=0),
]
sb.process.exec.return_value = _make_exec_response(result="/root")
sb.state = "started"
env = make_env(sandbox=sb, timeout=42)
sb.process.exec.reset_mock()
sb.process.exec.return_value = _make_exec_response(result="ok", exit_code=0)
env.execute("echo hello")
# The command sent to exec should be wrapped with `timeout N sh -c '...'`
# The command sent to _ThreadedProcessHandle should be wrapped with
# `timeout N bash -c '...'`
call_args = sb.process.exec.call_args_list[-1]
cmd = call_args[0][0]
assert cmd.startswith("timeout 42 sh -c ")
# SDK timeout param should NOT be passed
assert "timeout" not in call_args[1]
assert "timeout 42 bash -c " in cmd
def test_timeout_returns_exit_code_124(self, make_env):
"""Shell timeout utility returns exit code 124."""
sb = _make_sandbox()
sb.process.exec.side_effect = [
_make_exec_response(result="/root"),
_make_exec_response(result="", exit_code=124),
]
sb.process.exec.return_value = _make_exec_response(result="/root")
sb.state = "started"
env = make_env(sandbox=sb)
sb.process.exec.reset_mock()
sb.process.exec.return_value = _make_exec_response(result="", exit_code=124)
result = env.execute("sleep 300", timeout=5)
assert result["returncode"] == 124
def test_nonzero_exit_code(self, make_env):
sb = _make_sandbox()
sb.process.exec.side_effect = [
_make_exec_response(result="/root"),
_make_exec_response(result="not found", exit_code=127),
]
sb.process.exec.return_value = _make_exec_response(result="/root")
sb.state = "started"
env = make_env(sandbox=sb)
sb.process.exec.reset_mock()
sb.process.exec.return_value = _make_exec_response(result="not found", exit_code=127)
result = env.execute("bad_cmd")
assert result["returncode"] == 127
def test_stdin_data_wraps_heredoc(self, make_env):
sb = _make_sandbox()
sb.process.exec.side_effect = [
_make_exec_response(result="/root"),
_make_exec_response(result="ok", exit_code=0),
]
sb.process.exec.return_value = _make_exec_response(result="/root")
sb.state = "started"
env = make_env(sandbox=sb)
sb.process.exec.reset_mock()
sb.process.exec.return_value = _make_exec_response(result="ok", exit_code=0)
env.execute("python3", stdin_data="print('hi')")
# Check that the command passed to exec contains heredoc markers
# (single quotes get shell-escaped by shlex.quote, so check components)
call_args = sb.process.exec.call_args_list[-1]
cmd = call_args[0][0]
assert "HERMES_EOF_" in cmd
# Check that one of the exec calls contains heredoc markers.
# The last call may be the cwdfile read, so check all calls.
all_cmds = [
call_args[0][0]
for call_args in sb.process.exec.call_args_list
]
heredoc_cmd = [c for c in all_cmds if "HERMES_EOF_" in c]
assert heredoc_cmd, f"No heredoc found in exec calls: {all_cmds}"
cmd = heredoc_cmd[0]
assert "print" in cmd
assert "hi" in cmd
def test_custom_cwd_passed_through(self, make_env):
sb = _make_sandbox()
sb.process.exec.side_effect = [
_make_exec_response(result="/root"),
_make_exec_response(result="/tmp", exit_code=0),
]
sb.process.exec.return_value = _make_exec_response(result="/root")
sb.state = "started"
env = make_env(sandbox=sb)
sb.process.exec.reset_mock()
sb.process.exec.return_value = _make_exec_response(result="/tmp", exit_code=0)
env.execute("pwd", cwd="/tmp")
call_kwargs = sb.process.exec.call_args_list[-1][1]
assert call_kwargs["cwd"] == "/tmp"
# In the unified model, cwd is embedded in the _wrap_command output
# and the _ThreadedProcessHandle also passes cwd to the SDK
call_args = sb.process.exec.call_args_list[-1]
cmd = call_args[0][0]
# The wrapped command includes a cd to the cwd
assert "/tmp" in cmd
def test_daytona_error_triggers_retry(self, make_env, daytona_sdk):
def test_daytona_error_returns_error_result(self, make_env, daytona_sdk):
"""In the unified model, SDK errors are caught by _ThreadedProcessHandle
and returned as error results (no automatic retry)."""
sb = _make_sandbox()
sb.state = "started"
sb.process.exec.side_effect = [
_make_exec_response(result="/root"), # $HOME
daytona_sdk.DaytonaError("transient"), # first attempt fails
_make_exec_response(result="ok", exit_code=0), # retry succeeds
]
sb.process.exec.return_value = _make_exec_response(result="/root")
env = make_env(sandbox=sb)
sb.process.exec.reset_mock()
sb.process.exec.side_effect = daytona_sdk.DaytonaError("transient")
result = env.execute("echo retry")
assert result["output"] == "ok"
assert result["returncode"] == 0
assert result["returncode"] == 1
assert "transient" in result["output"]
# ---------------------------------------------------------------------------
@@ -349,51 +378,47 @@ class TestResourceConversion:
# ---------------------------------------------------------------------------
class TestInterrupt:
def test_interrupt_stops_sandbox_and_returns_130(self, make_env, monkeypatch):
def test_interrupt_returns_130(self, make_env, monkeypatch):
"""In the unified model, interrupt is handled by BaseEnvironment._wait_for_process."""
sb = _make_sandbox()
sb.state = "started"
event = threading.Event()
calls = {"n": 0}
sb.process.exec.return_value = _make_exec_response(result="/root")
env = make_env(sandbox=sb)
def exec_side_effect(*args, **kwargs):
calls["n"] += 1
if calls["n"] == 1:
return _make_exec_response(result="/root") # $HOME detection
event.wait(timeout=5) # simulate long-running command
# Make the SDK exec block long enough for the interrupt check to fire
import time as time_mod
def slow_exec(*args, **kwargs):
time_mod.sleep(5)
return _make_exec_response(result="done", exit_code=0)
sb.process.exec.side_effect = exec_side_effect
env = make_env(sandbox=sb)
sb.process.exec.reset_mock()
sb.process.exec.side_effect = slow_exec
# Patch is_interrupted in the base module where _wait_for_process uses it
monkeypatch.setattr(
"tools.environments.daytona.is_interrupted", lambda: True
"tools.environments.base.is_interrupted", lambda: True
)
try:
result = env.execute("sleep 10")
assert result["returncode"] == 130
sb.stop.assert_called()
finally:
event.set()
result = env.execute("sleep 10")
assert result["returncode"] == 130
# ---------------------------------------------------------------------------
# Retry exhaustion
# SDK error handling
# ---------------------------------------------------------------------------
class TestRetryExhausted:
def test_both_attempts_fail(self, make_env, daytona_sdk):
class TestSdkError:
def test_sdk_error_returns_error_result(self, make_env, daytona_sdk):
"""SDK errors in _ThreadedProcessHandle are caught and returned cleanly."""
sb = _make_sandbox()
sb.state = "started"
sb.process.exec.side_effect = [
_make_exec_response(result="/root"), # $HOME
daytona_sdk.DaytonaError("fail1"), # first attempt
daytona_sdk.DaytonaError("fail2"), # retry
]
sb.process.exec.return_value = _make_exec_response(result="/root")
env = make_env(sandbox=sb)
sb.process.exec.reset_mock()
sb.process.exec.side_effect = daytona_sdk.DaytonaError("fail")
result = env.execute("echo x")
assert result["returncode"] == 1
assert "Daytona execution error" in result["output"]
assert "fail" in result["output"]
# ---------------------------------------------------------------------------
@@ -403,12 +428,20 @@ class TestRetryExhausted:
class TestEnsureSandboxReady:
def test_restarts_stopped_sandbox(self, make_env):
env = make_env()
env._needs_refresh = True
env._sandbox.state = "stopped"
env._ensure_sandbox_ready()
env._sandbox.start.assert_called()
def test_no_restart_when_running(self, make_env):
env = make_env()
env._needs_refresh = True
env._sandbox.state = "started"
env._ensure_sandbox_ready()
env._sandbox.start.assert_not_called()
def test_skips_refresh_when_not_needed(self, make_env):
env = make_env()
env._needs_refresh = False
env._ensure_sandbox_ready()
env._sandbox.refresh_data.assert_not_called()

Some files were not shown because too many files have changed in this diff Show More