Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Teknium
|
08f3bcaa66 | ||
|
Teknium
|
aa1848d15d |
+1
-1
@@ -98,7 +98,7 @@ FAL_KEY=
|
||||
HONCHO_API_KEY=
|
||||
|
||||
# =============================================================================
|
||||
# TERMINAL TOOL CONFIGURATION
|
||||
# TERMINAL TOOL CONFIGURATION (mini-swe-agent backend)
|
||||
# =============================================================================
|
||||
# Backend type: "local", "singularity", "docker", "modal", or "ssh"
|
||||
# Terminal backend is configured in ~/.hermes/config.yaml (terminal.backend).
|
||||
|
||||
@@ -1,249 +0,0 @@
|
||||
# Hermes Agent v0.6.0 (v2026.3.30)
|
||||
|
||||
**Release Date:** March 30, 2026
|
||||
|
||||
> The multi-instance release — Profiles for running isolated agent instances, MCP server mode, Docker container, fallback provider chains, two new messaging platforms (Feishu/Lark and WeCom), Telegram webhook mode, Slack multi-workspace OAuth, 95 PRs and 16 resolved issues in 2 days.
|
||||
|
||||
---
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Profiles — Multi-Instance Hermes** — Run multiple isolated Hermes instances from the same installation. Each profile gets its own config, memory, sessions, skills, and gateway service. Create with `hermes profile create`, switch with `hermes -p <name>`, export/import for sharing. Full token-lock isolation prevents two profiles from using the same bot credential. ([#3681](https://github.com/NousResearch/hermes-agent/pull/3681))
|
||||
|
||||
- **MCP Server Mode** — Expose Hermes conversations and sessions to any MCP-compatible client (Claude Desktop, Cursor, VS Code, etc.) via `hermes mcp serve`. Browse conversations, read messages, search across sessions, and manage attachments — all through the Model Context Protocol. Supports both stdio and Streamable HTTP transports. ([#3795](https://github.com/NousResearch/hermes-agent/pull/3795))
|
||||
|
||||
- **Docker Container** — Official Dockerfile for running Hermes Agent in a container. Supports both CLI and gateway modes with volume-mounted config. ([#3668](https://github.com/NousResearch/hermes-agent/pull/3668), closes [#850](https://github.com/NousResearch/hermes-agent/issues/850))
|
||||
|
||||
- **Ordered Fallback Provider Chain** — Configure multiple inference providers with automatic failover. When your primary provider returns errors or is unreachable, Hermes automatically tries the next provider in the chain. Configure via `fallback_providers` in config.yaml. ([#3813](https://github.com/NousResearch/hermes-agent/pull/3813), closes [#1734](https://github.com/NousResearch/hermes-agent/issues/1734))
|
||||
|
||||
- **Feishu/Lark Platform Support** — Full gateway adapter for Feishu (飞书) and Lark with event subscriptions, message cards, group chat, image/file attachments, and interactive card callbacks. ([#3799](https://github.com/NousResearch/hermes-agent/pull/3799), [#3817](https://github.com/NousResearch/hermes-agent/pull/3817), closes [#1788](https://github.com/NousResearch/hermes-agent/issues/1788))
|
||||
|
||||
- **WeCom (Enterprise WeChat) Platform Support** — New gateway adapter for WeCom (企业微信) with text/image/voice messages, group chats, and callback verification. ([#3847](https://github.com/NousResearch/hermes-agent/pull/3847))
|
||||
|
||||
- **Slack Multi-Workspace OAuth** — Connect a single Hermes gateway to multiple Slack workspaces via OAuth token file. Each workspace gets its own bot token, resolved dynamically per incoming event. ([#3903](https://github.com/NousResearch/hermes-agent/pull/3903))
|
||||
|
||||
- **Telegram Webhook Mode & Group Controls** — Run the Telegram adapter in webhook mode as an alternative to polling — faster response times and better for production deployments behind a reverse proxy. New group mention gating controls when the bot responds: always, only when @mentioned, or via regex triggers. ([#3880](https://github.com/NousResearch/hermes-agent/pull/3880), [#3870](https://github.com/NousResearch/hermes-agent/pull/3870))
|
||||
|
||||
- **Exa Search Backend** — Add Exa as an alternative web search and content extraction backend alongside Firecrawl and DuckDuckGo. Set `EXA_API_KEY` and configure as preferred backend. ([#3648](https://github.com/NousResearch/hermes-agent/pull/3648))
|
||||
|
||||
- **Skills & Credentials on Remote Backends** — Mount skill directories and credential files into Modal and Docker containers, so remote terminal sessions have access to the same skills and secrets as local execution. ([#3890](https://github.com/NousResearch/hermes-agent/pull/3890), [#3671](https://github.com/NousResearch/hermes-agent/pull/3671), closes [#3665](https://github.com/NousResearch/hermes-agent/issues/3665), [#3433](https://github.com/NousResearch/hermes-agent/issues/3433))
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Core Agent & Architecture
|
||||
|
||||
### Provider & Model Support
|
||||
- **Ordered fallback provider chain** — automatic failover across multiple configured providers ([#3813](https://github.com/NousResearch/hermes-agent/pull/3813))
|
||||
- **Fix api_mode on provider switch** — switching providers via `hermes model` now correctly clears stale `api_mode` instead of hardcoding `chat_completions`, fixing 404s for providers with Anthropic-compatible endpoints ([#3726](https://github.com/NousResearch/hermes-agent/pull/3726), [#3857](https://github.com/NousResearch/hermes-agent/pull/3857), closes [#3685](https://github.com/NousResearch/hermes-agent/issues/3685))
|
||||
- **Stop silent OpenRouter fallback** — when no provider is configured, Hermes now raises a clear error instead of silently routing to OpenRouter ([#3807](https://github.com/NousResearch/hermes-agent/pull/3807), [#3862](https://github.com/NousResearch/hermes-agent/pull/3862))
|
||||
- **Gemini 3.1 preview models** — added to OpenRouter and Nous Portal catalogs ([#3803](https://github.com/NousResearch/hermes-agent/pull/3803), closes [#3753](https://github.com/NousResearch/hermes-agent/issues/3753))
|
||||
- **Gemini direct API context length** — full context length resolution for direct Google AI endpoints ([#3876](https://github.com/NousResearch/hermes-agent/pull/3876))
|
||||
- **gpt-5.4-mini** added to Codex fallback catalog ([#3855](https://github.com/NousResearch/hermes-agent/pull/3855))
|
||||
- **Curated model lists preferred** over live API probe when the probe returns fewer models ([#3856](https://github.com/NousResearch/hermes-agent/pull/3856), [#3867](https://github.com/NousResearch/hermes-agent/pull/3867))
|
||||
- **User-friendly 429 rate limit messages** with Retry-After countdown ([#3809](https://github.com/NousResearch/hermes-agent/pull/3809))
|
||||
- **Auxiliary client placeholder key** for local servers without auth requirements ([#3842](https://github.com/NousResearch/hermes-agent/pull/3842))
|
||||
- **INFO-level logging** for auxiliary provider resolution ([#3866](https://github.com/NousResearch/hermes-agent/pull/3866))
|
||||
|
||||
### Agent Loop & Conversation
|
||||
- **Subagent status reporting** — reports `completed` status when summary exists instead of generic failure ([#3829](https://github.com/NousResearch/hermes-agent/pull/3829))
|
||||
- **Session log file updated during compression** — prevents stale file references after context compression ([#3835](https://github.com/NousResearch/hermes-agent/pull/3835))
|
||||
- **Omit empty tools param** — sends no `tools` parameter when empty instead of `None`, fixing compatibility with strict providers ([#3820](https://github.com/NousResearch/hermes-agent/pull/3820))
|
||||
|
||||
### Profiles & Multi-Instance
|
||||
- **Profiles system** — `hermes profile create/list/switch/delete/export/import/rename`. Each profile gets isolated HERMES_HOME, gateway service, CLI wrapper. Token locks prevent credential collisions. Tab completion for profile names. ([#3681](https://github.com/NousResearch/hermes-agent/pull/3681))
|
||||
- **Profile-aware display paths** — all user-facing `~/.hermes` paths replaced with `display_hermes_home()` to show the correct profile directory ([#3623](https://github.com/NousResearch/hermes-agent/pull/3623))
|
||||
- **Lazy display_hermes_home imports** — prevents `ImportError` during `hermes update` when modules cache stale bytecode ([#3776](https://github.com/NousResearch/hermes-agent/pull/3776))
|
||||
- **HERMES_HOME for protected paths** — `.env` write-deny path now respects HERMES_HOME instead of hardcoded `~/.hermes` ([#3840](https://github.com/NousResearch/hermes-agent/pull/3840))
|
||||
|
||||
---
|
||||
|
||||
## 📱 Messaging Platforms (Gateway)
|
||||
|
||||
### New Platforms
|
||||
- **Feishu/Lark** — Full adapter with event subscriptions, message cards, group chat, image/file attachments, interactive card callbacks ([#3799](https://github.com/NousResearch/hermes-agent/pull/3799), [#3817](https://github.com/NousResearch/hermes-agent/pull/3817))
|
||||
- **WeCom (Enterprise WeChat)** — Text/image/voice messages, group chats, callback verification ([#3847](https://github.com/NousResearch/hermes-agent/pull/3847))
|
||||
|
||||
### Telegram
|
||||
- **Webhook mode** — run as webhook endpoint instead of polling for production deployments ([#3880](https://github.com/NousResearch/hermes-agent/pull/3880))
|
||||
- **Group mention gating & regex triggers** — configurable bot response behavior in groups: always, @mention-only, or regex-matched ([#3870](https://github.com/NousResearch/hermes-agent/pull/3870))
|
||||
- **Gracefully handle deleted reply targets** — no more crashes when the message being replied to was deleted ([#3858](https://github.com/NousResearch/hermes-agent/pull/3858), closes [#3229](https://github.com/NousResearch/hermes-agent/issues/3229))
|
||||
|
||||
### Discord
|
||||
- **Message processing reactions** — adds a reaction emoji while processing and removes it when done, giving visual feedback in channels ([#3871](https://github.com/NousResearch/hermes-agent/pull/3871))
|
||||
- **DISCORD_IGNORE_NO_MENTION** — skip messages that @mention other users/bots but not Hermes ([#3640](https://github.com/NousResearch/hermes-agent/pull/3640))
|
||||
- **Clean up deferred "thinking..."** — properly removes the "thinking..." indicator after slash commands complete ([#3674](https://github.com/NousResearch/hermes-agent/pull/3674), closes [#3595](https://github.com/NousResearch/hermes-agent/issues/3595))
|
||||
|
||||
### Slack
|
||||
- **Multi-workspace OAuth** — connect to multiple Slack workspaces from a single gateway via OAuth token file ([#3903](https://github.com/NousResearch/hermes-agent/pull/3903))
|
||||
|
||||
### WhatsApp
|
||||
- **Persistent aiohttp session** — reuse HTTP sessions across requests instead of creating new ones per message ([#3818](https://github.com/NousResearch/hermes-agent/pull/3818))
|
||||
- **LID↔phone alias resolution** — correctly match Linked ID and phone number formats in allowlists ([#3830](https://github.com/NousResearch/hermes-agent/pull/3830))
|
||||
- **Skip reply prefix in bot mode** — cleaner message formatting when running as a WhatsApp bot ([#3931](https://github.com/NousResearch/hermes-agent/pull/3931))
|
||||
|
||||
### Matrix
|
||||
- **Native voice messages via MSC3245** — send voice messages as proper Matrix voice events instead of file attachments ([#3877](https://github.com/NousResearch/hermes-agent/pull/3877))
|
||||
|
||||
### Mattermost
|
||||
- **Configurable mention behavior** — respond to messages without requiring @mention ([#3664](https://github.com/NousResearch/hermes-agent/pull/3664))
|
||||
|
||||
### Signal
|
||||
- **URL-encode phone numbers** and correct attachment RPC parameter — fixes delivery failures with certain phone number formats ([#3670](https://github.com/NousResearch/hermes-agent/pull/3670)) — @kshitijk4poor
|
||||
|
||||
### Email
|
||||
- **Close SMTP/IMAP connections on failure** — prevents connection leaks during error scenarios ([#3804](https://github.com/NousResearch/hermes-agent/pull/3804))
|
||||
|
||||
### Gateway Core
|
||||
- **Atomic config writes** — use atomic file writes for config.yaml to prevent data loss during crashes ([#3800](https://github.com/NousResearch/hermes-agent/pull/3800))
|
||||
- **Home channel env overrides** — apply environment variable overrides for home channels consistently ([#3796](https://github.com/NousResearch/hermes-agent/pull/3796), [#3808](https://github.com/NousResearch/hermes-agent/pull/3808))
|
||||
- **Replace print() with logger** — BasePlatformAdapter now uses proper logging instead of print statements ([#3669](https://github.com/NousResearch/hermes-agent/pull/3669))
|
||||
- **Cron delivery labels** — resolve human-friendly delivery labels via channel directory ([#3860](https://github.com/NousResearch/hermes-agent/pull/3860), closes [#1945](https://github.com/NousResearch/hermes-agent/issues/1945))
|
||||
- **Cron [SILENT] tightening** — prevent agents from prefixing reports with [SILENT] to suppress delivery ([#3901](https://github.com/NousResearch/hermes-agent/pull/3901))
|
||||
- **Background task media delivery** and vision download timeout fixes ([#3919](https://github.com/NousResearch/hermes-agent/pull/3919))
|
||||
- **Boot-md hook** — example built-in hook to run a BOOT.md file on gateway startup ([#3733](https://github.com/NousResearch/hermes-agent/pull/3733))
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ CLI & User Experience
|
||||
|
||||
### Interactive CLI
|
||||
- **Configurable tool preview length** — show full file paths by default instead of truncating at 40 chars ([#3841](https://github.com/NousResearch/hermes-agent/pull/3841))
|
||||
- **Tool token context display** — `hermes tools` checklist now shows estimated token cost per toolset ([#3805](https://github.com/NousResearch/hermes-agent/pull/3805))
|
||||
- **/bg spinner TUI fix** — route background task spinner through the TUI widget to prevent status bar collision ([#3643](https://github.com/NousResearch/hermes-agent/pull/3643))
|
||||
- **Prevent status bar wrapping** into duplicate rows ([#3883](https://github.com/NousResearch/hermes-agent/pull/3883)) — @kshitijk4poor
|
||||
- **Handle closed stdout ValueError** in safe print paths — fixes crashes when stdout is closed during gateway thread shutdown ([#3843](https://github.com/NousResearch/hermes-agent/pull/3843), closes [#3534](https://github.com/NousResearch/hermes-agent/issues/3534))
|
||||
- **Remove input() from /tools disable** — eliminates freeze in terminal when disabling tools ([#3918](https://github.com/NousResearch/hermes-agent/pull/3918))
|
||||
- **TTY guard for interactive CLI commands** — prevent CPU spin when launched without a terminal ([#3933](https://github.com/NousResearch/hermes-agent/pull/3933))
|
||||
- **Argparse entrypoint** — use argparse in the top-level launcher for cleaner error handling ([#3874](https://github.com/NousResearch/hermes-agent/pull/3874))
|
||||
- **Lazy-initialized tools show yellow** in banner instead of red, reducing false alarm about "missing" tools ([#3822](https://github.com/NousResearch/hermes-agent/pull/3822))
|
||||
- **Honcho tools shown in banner** when configured ([#3810](https://github.com/NousResearch/hermes-agent/pull/3810))
|
||||
|
||||
### Setup & Configuration
|
||||
- **Auto-install matrix-nio** during `hermes setup` when Matrix is selected ([#3802](https://github.com/NousResearch/hermes-agent/pull/3802), [#3873](https://github.com/NousResearch/hermes-agent/pull/3873))
|
||||
- **Session export stdout support** — export sessions to stdout with `-` for piping ([#3641](https://github.com/NousResearch/hermes-agent/pull/3641), closes [#3609](https://github.com/NousResearch/hermes-agent/issues/3609))
|
||||
- **Configurable approval timeouts** — set how long dangerous command approval prompts wait before auto-denying ([#3886](https://github.com/NousResearch/hermes-agent/pull/3886), closes [#3765](https://github.com/NousResearch/hermes-agent/issues/3765))
|
||||
- **Clear __pycache__ during update** — prevents stale bytecode ImportError after `hermes update` ([#3819](https://github.com/NousResearch/hermes-agent/pull/3819))
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Tool System
|
||||
|
||||
### MCP
|
||||
- **MCP Server Mode** — `hermes mcp serve` exposes conversations, sessions, and attachments to MCP clients via stdio or Streamable HTTP ([#3795](https://github.com/NousResearch/hermes-agent/pull/3795))
|
||||
- **Dynamic tool discovery** — respond to `notifications/tools/list_changed` events to pick up new tools from MCP servers without reconnecting ([#3812](https://github.com/NousResearch/hermes-agent/pull/3812))
|
||||
- **Non-deprecated HTTP transport** — switched from `sse_client` to `streamable_http_client` ([#3646](https://github.com/NousResearch/hermes-agent/pull/3646))
|
||||
|
||||
### Web Tools
|
||||
- **Exa search backend** — alternative to Firecrawl and DuckDuckGo for web search and extraction ([#3648](https://github.com/NousResearch/hermes-agent/pull/3648))
|
||||
|
||||
### Browser
|
||||
- **Guard against None LLM responses** in browser snapshot and vision tools ([#3642](https://github.com/NousResearch/hermes-agent/pull/3642))
|
||||
|
||||
### Terminal & Remote Backends
|
||||
- **Mount skill directories** into Modal and Docker containers ([#3890](https://github.com/NousResearch/hermes-agent/pull/3890))
|
||||
- **Mount credential files** into remote backends with mtime+size caching ([#3671](https://github.com/NousResearch/hermes-agent/pull/3671))
|
||||
- **Preserve partial output** when commands time out instead of losing everything ([#3868](https://github.com/NousResearch/hermes-agent/pull/3868))
|
||||
- **Stop marking persisted env vars as missing** on remote backends ([#3650](https://github.com/NousResearch/hermes-agent/pull/3650))
|
||||
|
||||
### Audio
|
||||
- **.aac format support** in transcription tool ([#3865](https://github.com/NousResearch/hermes-agent/pull/3865), closes [#1963](https://github.com/NousResearch/hermes-agent/issues/1963))
|
||||
- **Audio download retry** — retry logic for `cache_audio_from_url` matching the existing image download pattern ([#3401](https://github.com/NousResearch/hermes-agent/pull/3401)) — @binhnt92
|
||||
|
||||
### Vision
|
||||
- **Reject non-image files** and enforce website-only policy for vision analysis ([#3845](https://github.com/NousResearch/hermes-agent/pull/3845))
|
||||
|
||||
### Tool Schema
|
||||
- **Ensure name field** always present in tool definitions, fixing `KeyError: 'name'` crashes ([#3811](https://github.com/NousResearch/hermes-agent/pull/3811), closes [#3729](https://github.com/NousResearch/hermes-agent/issues/3729))
|
||||
|
||||
### ACP (Editor Integration)
|
||||
- **Complete session management surface** for VS Code/Zed/JetBrains clients — proper task lifecycle, cancel support, session persistence ([#3675](https://github.com/NousResearch/hermes-agent/pull/3675))
|
||||
|
||||
---
|
||||
|
||||
## 🧩 Skills & Plugins
|
||||
|
||||
### Skills System
|
||||
- **External skill directories** — configure additional skill directories via `skills.external_dirs` in config.yaml ([#3678](https://github.com/NousResearch/hermes-agent/pull/3678))
|
||||
- **Category path traversal blocked** — prevents `../` attacks in skill category names ([#3844](https://github.com/NousResearch/hermes-agent/pull/3844))
|
||||
- **parallel-cli moved to optional-skills** — reduces default skill footprint ([#3673](https://github.com/NousResearch/hermes-agent/pull/3673)) — @kshitijk4poor
|
||||
|
||||
### New Skills
|
||||
- **memento-flashcards** — spaced repetition flashcard system ([#3827](https://github.com/NousResearch/hermes-agent/pull/3827))
|
||||
- **songwriting-and-ai-music** — songwriting craft and AI music generation prompts ([#3834](https://github.com/NousResearch/hermes-agent/pull/3834))
|
||||
- **SiYuan Note** — integration with SiYuan note-taking app ([#3742](https://github.com/NousResearch/hermes-agent/pull/3742))
|
||||
- **Scrapling** — web scraping skill using Scrapling library ([#3742](https://github.com/NousResearch/hermes-agent/pull/3742))
|
||||
- **one-three-one-rule** — communication framework skill ([#3797](https://github.com/NousResearch/hermes-agent/pull/3797))
|
||||
|
||||
### Plugin System
|
||||
- **Plugin enable/disable commands** — `hermes plugins enable/disable <name>` for managing plugin state without removing them ([#3747](https://github.com/NousResearch/hermes-agent/pull/3747))
|
||||
- **Plugin message injection** — plugins can now inject messages into the conversation stream on behalf of the user via `ctx.inject_message()` ([#3778](https://github.com/NousResearch/hermes-agent/pull/3778)) — @winglian
|
||||
- **Honcho self-hosted support** — allow local Honcho instances without requiring an API key ([#3644](https://github.com/NousResearch/hermes-agent/pull/3644))
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security & Reliability
|
||||
|
||||
### Security Hardening
|
||||
- **Hardened dangerous command detection** — expanded pattern matching for risky shell commands and added file tool path guards for sensitive locations (`/etc/`, `/boot/`, docker.sock) ([#3872](https://github.com/NousResearch/hermes-agent/pull/3872))
|
||||
- **Sensitive path write checks** in approval system — catch writes to system config files through file tools, not just terminal ([#3859](https://github.com/NousResearch/hermes-agent/pull/3859))
|
||||
- **Secret redaction expansion** — now covers ElevenLabs, Tavily, and Exa API keys ([#3920](https://github.com/NousResearch/hermes-agent/pull/3920))
|
||||
- **Vision file rejection** — reject non-image files passed to vision analysis to prevent information disclosure ([#3845](https://github.com/NousResearch/hermes-agent/pull/3845))
|
||||
- **Category path traversal blocking** — prevent directory traversal in skill category names ([#3844](https://github.com/NousResearch/hermes-agent/pull/3844))
|
||||
|
||||
### Reliability
|
||||
- **Atomic config.yaml writes** — prevent data loss during gateway crashes ([#3800](https://github.com/NousResearch/hermes-agent/pull/3800))
|
||||
- **Clear __pycache__ on update** — prevent stale bytecode from causing ImportError after updates ([#3819](https://github.com/NousResearch/hermes-agent/pull/3819))
|
||||
- **Lazy imports for update safety** — prevent ImportError chains during `hermes update` when modules reference new functions ([#3776](https://github.com/NousResearch/hermes-agent/pull/3776))
|
||||
- **Restore terminalbench2 from patch corruption** — recovered file damaged by patch tool's secret redaction ([#3801](https://github.com/NousResearch/hermes-agent/pull/3801))
|
||||
- **Terminal timeout preserves partial output** — no more lost command output on timeout ([#3868](https://github.com/NousResearch/hermes-agent/pull/3868))
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Notable Bug Fixes
|
||||
|
||||
- **OpenClaw migration model config overwrite** — migration no longer overwrites model config dict with a string ([#3924](https://github.com/NousResearch/hermes-agent/pull/3924)) — @0xbyt4
|
||||
- **OpenClaw migration expanded** — covers full data footprint including sessions, cron, memory ([#3869](https://github.com/NousResearch/hermes-agent/pull/3869))
|
||||
- **Telegram deleted reply targets** — gracefully handle replies to deleted messages instead of crashing ([#3858](https://github.com/NousResearch/hermes-agent/pull/3858))
|
||||
- **Discord "thinking..." persistence** — properly cleans up deferred response indicators ([#3674](https://github.com/NousResearch/hermes-agent/pull/3674))
|
||||
- **WhatsApp LID↔phone aliases** — fixes allowlist matching failures with Linked ID format ([#3830](https://github.com/NousResearch/hermes-agent/pull/3830))
|
||||
- **Signal URL-encoded phone numbers** — fixes delivery failures with certain formats ([#3670](https://github.com/NousResearch/hermes-agent/pull/3670))
|
||||
- **Email connection leaks** — properly close SMTP/IMAP connections on error ([#3804](https://github.com/NousResearch/hermes-agent/pull/3804))
|
||||
- **_safe_print ValueError** — no more gateway thread crashes on closed stdout ([#3843](https://github.com/NousResearch/hermes-agent/pull/3843))
|
||||
- **Tool schema KeyError 'name'** — ensure name field always present in tool definitions ([#3811](https://github.com/NousResearch/hermes-agent/pull/3811))
|
||||
- **api_mode stale on provider switch** — correctly clear when switching providers via `hermes model` ([#3857](https://github.com/NousResearch/hermes-agent/pull/3857))
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
- Resolved 10+ CI failures across hooks, tiktoken, plugins, and skill tests ([#3848](https://github.com/NousResearch/hermes-agent/pull/3848), [#3721](https://github.com/NousResearch/hermes-agent/pull/3721), [#3936](https://github.com/NousResearch/hermes-agent/pull/3936))
|
||||
|
||||
---
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
- **Comprehensive OpenClaw migration guide** — step-by-step guide for migrating from OpenClaw/Claw3D to Hermes Agent ([#3864](https://github.com/NousResearch/hermes-agent/pull/3864), [#3900](https://github.com/NousResearch/hermes-agent/pull/3900))
|
||||
- **Credential file passthrough docs** — document how to forward credential files and env vars to remote backends ([#3677](https://github.com/NousResearch/hermes-agent/pull/3677))
|
||||
- **DuckDuckGo requirements clarified** — note runtime dependency on duckduckgo-search package ([#3680](https://github.com/NousResearch/hermes-agent/pull/3680))
|
||||
- **Skills catalog updated** — added red-teaming category and optional skills listing ([#3745](https://github.com/NousResearch/hermes-agent/pull/3745))
|
||||
- **Feishu docs MDX fix** — escape angle-bracket URLs that break Docusaurus build ([#3902](https://github.com/NousResearch/hermes-agent/pull/3902))
|
||||
|
||||
---
|
||||
|
||||
## 👥 Contributors
|
||||
|
||||
### Core
|
||||
- **@teknium1** — 90 PRs across all subsystems
|
||||
|
||||
### Community Contributors
|
||||
- **@kshitijk4poor** — 3 PRs: Signal phone number fix ([#3670](https://github.com/NousResearch/hermes-agent/pull/3670)), parallel-cli to optional-skills ([#3673](https://github.com/NousResearch/hermes-agent/pull/3673)), status bar wrapping fix ([#3883](https://github.com/NousResearch/hermes-agent/pull/3883))
|
||||
- **@winglian** — 1 PR: Plugin message injection interface ([#3778](https://github.com/NousResearch/hermes-agent/pull/3778))
|
||||
- **@binhnt92** — 1 PR: Audio download retry logic ([#3401](https://github.com/NousResearch/hermes-agent/pull/3401))
|
||||
- **@0xbyt4** — 1 PR: OpenClaw migration model config fix ([#3924](https://github.com/NousResearch/hermes-agent/pull/3924))
|
||||
|
||||
### Issues Resolved from Community
|
||||
@Material-Scientist ([#850](https://github.com/NousResearch/hermes-agent/issues/850)), @hanxu98121 ([#1734](https://github.com/NousResearch/hermes-agent/issues/1734)), @penwyp ([#1788](https://github.com/NousResearch/hermes-agent/issues/1788)), @dan-and ([#1945](https://github.com/NousResearch/hermes-agent/issues/1945)), @AdrianScott ([#1963](https://github.com/NousResearch/hermes-agent/issues/1963)), @clawdbot47 ([#3229](https://github.com/NousResearch/hermes-agent/issues/3229)), @alanfwilliams ([#3404](https://github.com/NousResearch/hermes-agent/issues/3404)), @kentimsit ([#3433](https://github.com/NousResearch/hermes-agent/issues/3433)), @hayka-pacha ([#3534](https://github.com/NousResearch/hermes-agent/issues/3534)), @primmer ([#3595](https://github.com/NousResearch/hermes-agent/issues/3595)), @dagelf ([#3609](https://github.com/NousResearch/hermes-agent/issues/3609)), @HenkDz ([#3685](https://github.com/NousResearch/hermes-agent/issues/3685)), @tmdgusya ([#3729](https://github.com/NousResearch/hermes-agent/issues/3729)), @TypQxQ ([#3753](https://github.com/NousResearch/hermes-agent/issues/3753)), @acsezen ([#3765](https://github.com/NousResearch/hermes-agent/issues/3765))
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: [v2026.3.28...v2026.3.30](https://github.com/NousResearch/hermes-agent/compare/v2026.3.28...v2026.3.30)
|
||||
@@ -627,6 +627,8 @@ def _resolve_custom_runtime() -> Tuple[Optional[str], Optional[str]]:
|
||||
custom_key = runtime.get("api_key")
|
||||
if not isinstance(custom_base, str) or not custom_base.strip():
|
||||
return None, None
|
||||
if not isinstance(custom_key, str) or not custom_key.strip():
|
||||
return None, None
|
||||
|
||||
custom_base = custom_base.strip().rstrip("/")
|
||||
if "openrouter.ai" in custom_base.lower():
|
||||
@@ -634,13 +636,6 @@ def _resolve_custom_runtime() -> Tuple[Optional[str], Optional[str]]:
|
||||
# configured. Treat that as "no custom endpoint" for auxiliary routing.
|
||||
return None, None
|
||||
|
||||
# Local servers (Ollama, llama.cpp, vLLM, LM Studio) don't require auth.
|
||||
# Use a placeholder key — the OpenAI SDK requires a non-empty string but
|
||||
# local servers ignore the Authorization header. Same fix as cli.py
|
||||
# _ensure_runtime_credentials() (PR #2556).
|
||||
if not isinstance(custom_key, str) or not custom_key.strip():
|
||||
custom_key = "no-key-required"
|
||||
|
||||
return custom_base, custom_key.strip()
|
||||
|
||||
|
||||
@@ -742,37 +737,16 @@ def _resolve_forced_provider(forced: str) -> Tuple[Optional[OpenAI], Optional[st
|
||||
return None, None
|
||||
|
||||
|
||||
_AUTO_PROVIDER_LABELS = {
|
||||
"_try_openrouter": "openrouter",
|
||||
"_try_nous": "nous",
|
||||
"_try_custom_endpoint": "local/custom",
|
||||
"_try_codex": "openai-codex",
|
||||
"_resolve_api_key_provider": "api-key",
|
||||
}
|
||||
|
||||
|
||||
def _resolve_auto() -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
"""Full auto-detection chain: OpenRouter → Nous → custom → Codex → API-key → None."""
|
||||
global auxiliary_is_nous
|
||||
auxiliary_is_nous = False # Reset — _try_nous() will set True if it wins
|
||||
tried = []
|
||||
for try_fn in (_try_openrouter, _try_nous, _try_custom_endpoint,
|
||||
_try_codex, _resolve_api_key_provider):
|
||||
fn_name = getattr(try_fn, "__name__", "unknown")
|
||||
label = _AUTO_PROVIDER_LABELS.get(fn_name, fn_name)
|
||||
client, model = try_fn()
|
||||
if client is not None:
|
||||
if tried:
|
||||
logger.info("Auxiliary auto-detect: using %s (%s) — skipped: %s",
|
||||
label, model or "default", ", ".join(tried))
|
||||
else:
|
||||
logger.info("Auxiliary auto-detect: using %s (%s)", label, model or "default")
|
||||
return client, model
|
||||
tried.append(label)
|
||||
logger.warning("Auxiliary auto-detect: no provider available (tried: %s). "
|
||||
"Compression, summarization, and memory flush will not work. "
|
||||
"Set OPENROUTER_API_KEY or configure a local model in config.yaml.",
|
||||
", ".join(tried))
|
||||
logger.debug("Auxiliary client: none available")
|
||||
return None, None
|
||||
|
||||
|
||||
@@ -923,12 +897,11 @@ def resolve_provider_client(
|
||||
custom_key = (
|
||||
(explicit_api_key or "").strip()
|
||||
or os.getenv("OPENAI_API_KEY", "").strip()
|
||||
or "no-key-required" # local servers don't need auth
|
||||
)
|
||||
if not custom_base:
|
||||
if not custom_base or not custom_key:
|
||||
logger.warning(
|
||||
"resolve_provider_client: explicit custom endpoint requested "
|
||||
"but base_url is empty"
|
||||
"but no API key was found (set explicit_api_key or OPENAI_API_KEY)"
|
||||
)
|
||||
return None, None
|
||||
final_model = model or _read_main_model() or "gpt-4o-mini"
|
||||
@@ -1639,8 +1612,8 @@ def call_llm(
|
||||
)
|
||||
# For auto/custom, fall back to OpenRouter
|
||||
if not resolved_base_url:
|
||||
logger.info("Auxiliary %s: provider %s unavailable, falling back to openrouter",
|
||||
task or "call", resolved_provider)
|
||||
logger.warning("Provider %s unavailable, falling back to openrouter",
|
||||
resolved_provider)
|
||||
client, final_model = _get_cached_client(
|
||||
"openrouter", resolved_model or _OPENROUTER_MODEL)
|
||||
if client is None:
|
||||
@@ -1650,13 +1623,6 @@ def call_llm(
|
||||
|
||||
effective_timeout = timeout if timeout is not None else _get_task_timeout(task)
|
||||
|
||||
# Log what we're about to do — makes auxiliary operations visible
|
||||
_base_info = str(getattr(client, "base_url", resolved_base_url) or "")
|
||||
if task:
|
||||
logger.info("Auxiliary %s: using %s (%s)%s",
|
||||
task, resolved_provider or "auto", final_model or "default",
|
||||
f" at {_base_info}" if _base_info and "openrouter" not in _base_info else "")
|
||||
|
||||
kwargs = _build_call_kwargs(
|
||||
resolved_provider, final_model, messages,
|
||||
temperature=temperature, max_tokens=max_tokens,
|
||||
|
||||
+3
-30
@@ -17,23 +17,6 @@ _RESET = "\033[0m"
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# =========================================================================
|
||||
# Configurable tool preview length (0 = no limit)
|
||||
# Set once at startup by CLI or gateway from display.tool_preview_length config.
|
||||
# =========================================================================
|
||||
_tool_preview_max_len: int = 0 # 0 = unlimited
|
||||
|
||||
|
||||
def set_tool_preview_max_len(n: int) -> None:
|
||||
"""Set the global max length for tool call previews. 0 = no limit."""
|
||||
global _tool_preview_max_len
|
||||
_tool_preview_max_len = max(int(n), 0) if n else 0
|
||||
|
||||
|
||||
def get_tool_preview_max_len() -> int:
|
||||
"""Return the configured max preview length (0 = unlimited)."""
|
||||
return _tool_preview_max_len
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# Skin-aware helpers (lazy import to avoid circular deps)
|
||||
@@ -111,14 +94,8 @@ def _oneline(text: str) -> str:
|
||||
return " ".join(text.split())
|
||||
|
||||
|
||||
def build_tool_preview(tool_name: str, args: dict, max_len: int | None = None) -> str | None:
|
||||
"""Build a short preview of a tool call's primary argument for display.
|
||||
|
||||
*max_len* controls truncation. ``None`` (default) defers to the global
|
||||
``_tool_preview_max_len`` set via config; ``0`` means unlimited.
|
||||
"""
|
||||
if max_len is None:
|
||||
max_len = _tool_preview_max_len
|
||||
def build_tool_preview(tool_name: str, args: dict, max_len: int = 40) -> str | None:
|
||||
"""Build a short preview of a tool call's primary argument for display."""
|
||||
if not args:
|
||||
return None
|
||||
primary_args = {
|
||||
@@ -213,7 +190,7 @@ def build_tool_preview(tool_name: str, args: dict, max_len: int | None = None) -
|
||||
preview = _oneline(str(value))
|
||||
if not preview:
|
||||
return None
|
||||
if max_len > 0 and len(preview) > max_len:
|
||||
if len(preview) > max_len:
|
||||
preview = preview[:max_len - 3] + "..."
|
||||
return preview
|
||||
|
||||
@@ -507,14 +484,10 @@ def get_cute_tool_message(
|
||||
|
||||
def _trunc(s, n=40):
|
||||
s = str(s)
|
||||
if _tool_preview_max_len == 0:
|
||||
return s # no limit
|
||||
return (s[:n-3] + "...") if len(s) > n else s
|
||||
|
||||
def _path(p, n=35):
|
||||
p = str(p)
|
||||
if _tool_preview_max_len == 0:
|
||||
return p # no limit
|
||||
return ("..." + p[-(n-3):]) if len(p) > n else p
|
||||
|
||||
def _wrap(line: str) -> str:
|
||||
|
||||
@@ -171,7 +171,6 @@ _URL_TO_PROVIDER: Dict[str, str] = {
|
||||
"dashscope.aliyuncs.com": "alibaba",
|
||||
"dashscope-intl.aliyuncs.com": "alibaba",
|
||||
"openrouter.ai": "openrouter",
|
||||
"generativelanguage.googleapis.com": "google",
|
||||
"inference-api.nousresearch.com": "nous",
|
||||
"api.deepseek.com": "deepseek",
|
||||
"api.githubcopilot.com": "copilot",
|
||||
|
||||
@@ -37,9 +37,6 @@ _PREFIX_PATTERNS = [
|
||||
r"dop_v1_[A-Za-z0-9]{10,}", # DigitalOcean PAT
|
||||
r"doo_v1_[A-Za-z0-9]{10,}", # DigitalOcean OAuth
|
||||
r"am_[A-Za-z0-9_-]{10,}", # AgentMail API key
|
||||
r"sk_[A-Za-z0-9_]{10,}", # ElevenLabs TTS key (sk_ underscore, not sk- dash)
|
||||
r"tvly-[A-Za-z0-9]{10,}", # Tavily search API key
|
||||
r"exa_[A-Za-z0-9]{10,}", # Exa search API key
|
||||
]
|
||||
|
||||
# ENV assignment patterns: KEY=value where KEY contains a secret-like name
|
||||
|
||||
+8
-26
@@ -11,29 +11,14 @@ model:
|
||||
default: "anthropic/claude-opus-4.6"
|
||||
|
||||
# Inference provider selection:
|
||||
# "auto" - Auto-detect from credentials (default)
|
||||
# "openrouter" - OpenRouter (requires: OPENROUTER_API_KEY or OPENAI_API_KEY)
|
||||
# "nous" - Nous Portal OAuth (requires: hermes login)
|
||||
# "nous-api" - Nous Portal API key (requires: NOUS_API_KEY)
|
||||
# "anthropic" - Direct Anthropic API (requires: ANTHROPIC_API_KEY)
|
||||
# "openai-codex" - OpenAI Codex (requires: hermes login --provider openai-codex)
|
||||
# "copilot" - GitHub Copilot / GitHub Models (requires: GITHUB_TOKEN)
|
||||
# "zai" - z.ai / ZhipuAI GLM (requires: GLM_API_KEY)
|
||||
# "kimi-coding" - Kimi / Moonshot AI (requires: KIMI_API_KEY)
|
||||
# "minimax" - MiniMax global (requires: MINIMAX_API_KEY)
|
||||
# "minimax-cn" - MiniMax China (requires: MINIMAX_CN_API_KEY)
|
||||
# "huggingface" - Hugging Face Inference (requires: HF_TOKEN)
|
||||
# "kilocode" - KiloCode gateway (requires: KILOCODE_API_KEY)
|
||||
# "ai-gateway" - Vercel AI Gateway (requires: AI_GATEWAY_API_KEY)
|
||||
#
|
||||
# Local servers (LM Studio, Ollama, vLLM, llama.cpp):
|
||||
# "custom" - Any OpenAI-compatible endpoint. Set base_url below.
|
||||
# Aliases: "lmstudio", "ollama", "vllm", "llamacpp" all map to "custom".
|
||||
# Example for LM Studio:
|
||||
# provider: "lmstudio"
|
||||
# base_url: "http://localhost:1234/v1"
|
||||
# No API key needed — local servers typically ignore auth.
|
||||
#
|
||||
# "auto" - Use Nous Portal if logged in, otherwise OpenRouter/env vars (default)
|
||||
# "nous-api" - Use Nous Portal via API key (requires: NOUS_API_KEY)
|
||||
# "openrouter" - Always use OpenRouter API key from OPENROUTER_API_KEY
|
||||
# "nous" - Always use Nous Portal (requires: hermes login)
|
||||
# "zai" - Use z.ai / ZhipuAI GLM models (requires: GLM_API_KEY)
|
||||
# "kimi-coding"- Use Kimi / Moonshot AI models (requires: KIMI_API_KEY)
|
||||
# "minimax" - Use MiniMax global endpoint (requires: MINIMAX_API_KEY)
|
||||
# "minimax-cn" - Use MiniMax China endpoint (requires: MINIMAX_CN_API_KEY)
|
||||
# Can also be overridden with --provider flag or HERMES_INFERENCE_PROVIDER env var.
|
||||
provider: "auto"
|
||||
|
||||
@@ -324,9 +309,6 @@ compression:
|
||||
# vision:
|
||||
# provider: "auto"
|
||||
# model: "" # e.g. "google/gemini-2.5-flash", "openai/gpt-4o"
|
||||
# timeout: 30 # LLM API call timeout (seconds)
|
||||
# download_timeout: 30 # Image HTTP download timeout (seconds)
|
||||
# # Increase for slow connections or self-hosted image servers
|
||||
#
|
||||
# # Web page scraping / summarization + browser page text extraction
|
||||
# web_extract:
|
||||
|
||||
@@ -449,14 +449,6 @@ try:
|
||||
except Exception:
|
||||
pass # Skin engine is optional — default skin used if unavailable
|
||||
|
||||
# Initialize tool preview length from config
|
||||
try:
|
||||
from agent.display import set_tool_preview_max_len
|
||||
_tpl = CLI_CONFIG.get("display", {}).get("tool_preview_length", 0)
|
||||
set_tool_preview_max_len(int(_tpl) if _tpl else 0)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Neuter AsyncHttpxClientWrapper.__del__ before any AsyncOpenAI clients are
|
||||
# created. The SDK's __del__ schedules aclose() on asyncio.get_running_loop()
|
||||
# which, during CLI idle time, finds prompt_toolkit's event loop and tries to
|
||||
@@ -1087,10 +1079,10 @@ class HermesCLI:
|
||||
# env vars would stomp each other.
|
||||
_model_config = CLI_CONFIG.get("model", {})
|
||||
_config_model = (_model_config.get("default") or _model_config.get("model") or "") if isinstance(_model_config, dict) else (_model_config or "")
|
||||
_DEFAULT_CONFIG_MODEL = "anthropic/claude-opus-4.6"
|
||||
self.model = model or _config_model or _DEFAULT_CONFIG_MODEL
|
||||
# Auto-detect model from local server if still on default
|
||||
if self.model == _DEFAULT_CONFIG_MODEL:
|
||||
_FALLBACK_MODEL = "anthropic/claude-opus-4.6"
|
||||
self.model = model or _config_model or _FALLBACK_MODEL
|
||||
# Auto-detect model from local server if still on fallback
|
||||
if self.model == _FALLBACK_MODEL:
|
||||
_base_url = (_model_config.get("base_url") or "") if isinstance(_model_config, dict) else ""
|
||||
if "localhost" in _base_url or "127.0.0.1" in _base_url:
|
||||
from hermes_cli.runtime_provider import _auto_detect_local_model
|
||||
@@ -1104,7 +1096,7 @@ class HermesCLI:
|
||||
# explicit choice — the user just never changed it. But a config model
|
||||
# like "gpt-5.3-codex" IS explicit and must be preserved.
|
||||
self._model_is_default = not model and (
|
||||
not _config_model or _config_model == _DEFAULT_CONFIG_MODEL
|
||||
not _config_model or _config_model == _FALLBACK_MODEL
|
||||
)
|
||||
|
||||
self._explicit_api_key = api_key
|
||||
@@ -1190,13 +1182,9 @@ class HermesCLI:
|
||||
self._provider_require_params = pr.get("require_parameters", False)
|
||||
self._provider_data_collection = pr.get("data_collection")
|
||||
|
||||
# Fallback provider chain — tried in order when primary fails after retries.
|
||||
# Supports new list format (fallback_providers) and legacy single-dict (fallback_model).
|
||||
fb = CLI_CONFIG.get("fallback_providers") or CLI_CONFIG.get("fallback_model") or []
|
||||
# Normalize legacy single-dict to a one-element list
|
||||
if isinstance(fb, dict):
|
||||
fb = [fb] if fb.get("provider") and fb.get("model") else []
|
||||
self._fallback_model = fb
|
||||
# Fallback model config — tried when primary provider fails after retries
|
||||
fb = CLI_CONFIG.get("fallback_model") or {}
|
||||
self._fallback_model = fb if fb.get("provider") and fb.get("model") else None
|
||||
|
||||
# Optional cheap-vs-strong routing for simple turns
|
||||
self._smart_model_routing = CLI_CONFIG.get("smart_model_routing", {}) or {}
|
||||
@@ -1355,49 +1343,6 @@ class HermesCLI:
|
||||
|
||||
return snapshot
|
||||
|
||||
@staticmethod
|
||||
def _status_bar_display_width(text: str) -> int:
|
||||
"""Return terminal cell width for status-bar text.
|
||||
|
||||
len() is not enough for prompt_toolkit layout decisions because some
|
||||
glyphs can render wider than one Python codepoint. Keeping the status
|
||||
bar within the real display width prevents it from wrapping onto a
|
||||
second line and leaving behind duplicate rows.
|
||||
"""
|
||||
try:
|
||||
from prompt_toolkit.utils import get_cwidth
|
||||
return get_cwidth(text or "")
|
||||
except Exception:
|
||||
return len(text or "")
|
||||
|
||||
@classmethod
|
||||
def _trim_status_bar_text(cls, text: str, max_width: int) -> str:
|
||||
"""Trim status-bar text to a single terminal row."""
|
||||
if max_width <= 0:
|
||||
return ""
|
||||
try:
|
||||
from prompt_toolkit.utils import get_cwidth
|
||||
except Exception:
|
||||
get_cwidth = None
|
||||
|
||||
if cls._status_bar_display_width(text) <= max_width:
|
||||
return text
|
||||
|
||||
ellipsis = "..."
|
||||
ellipsis_width = cls._status_bar_display_width(ellipsis)
|
||||
if max_width <= ellipsis_width:
|
||||
return ellipsis[:max_width]
|
||||
|
||||
out = []
|
||||
width = 0
|
||||
for ch in text:
|
||||
ch_width = get_cwidth(ch) if get_cwidth else len(ch)
|
||||
if width + ch_width + ellipsis_width > max_width:
|
||||
break
|
||||
out.append(ch)
|
||||
width += ch_width
|
||||
return "".join(out).rstrip() + ellipsis
|
||||
|
||||
def _build_status_bar_text(self, width: Optional[int] = None) -> str:
|
||||
try:
|
||||
snapshot = self._get_status_bar_snapshot()
|
||||
@@ -1412,12 +1357,11 @@ class HermesCLI:
|
||||
duration_label = snapshot["duration"]
|
||||
|
||||
if width < 52:
|
||||
text = f"⚕ {snapshot['model_short']} · {duration_label}"
|
||||
return self._trim_status_bar_text(text, width)
|
||||
return f"⚕ {snapshot['model_short']} · {duration_label}"
|
||||
if width < 76:
|
||||
parts = [f"⚕ {snapshot['model_short']}", percent_label]
|
||||
parts.append(duration_label)
|
||||
return self._trim_status_bar_text(" · ".join(parts), width)
|
||||
return " · ".join(parts)
|
||||
|
||||
if snapshot["context_length"]:
|
||||
ctx_total = _format_context_length(snapshot["context_length"])
|
||||
@@ -1428,7 +1372,7 @@ class HermesCLI:
|
||||
|
||||
parts = [f"⚕ {snapshot['model_short']}", context_label, percent_label]
|
||||
parts.append(duration_label)
|
||||
return self._trim_status_bar_text(" │ ".join(parts), width)
|
||||
return " │ ".join(parts)
|
||||
except Exception:
|
||||
return f"⚕ {self.model if getattr(self, 'model', None) else 'Hermes'}"
|
||||
|
||||
@@ -1450,54 +1394,53 @@ class HermesCLI:
|
||||
duration_label = snapshot["duration"]
|
||||
|
||||
if width < 52:
|
||||
frags = [
|
||||
return [
|
||||
("class:status-bar", " ⚕ "),
|
||||
("class:status-bar-strong", snapshot["model_short"]),
|
||||
("class:status-bar-dim", " · "),
|
||||
("class:status-bar-dim", duration_label),
|
||||
("class:status-bar", " "),
|
||||
]
|
||||
|
||||
percent = snapshot["context_percent"]
|
||||
percent_label = f"{percent}%" if percent is not None else "--"
|
||||
if width < 76:
|
||||
frags = [
|
||||
("class:status-bar", " ⚕ "),
|
||||
("class:status-bar-strong", snapshot["model_short"]),
|
||||
("class:status-bar-dim", " · "),
|
||||
(self._status_bar_context_style(percent), percent_label),
|
||||
]
|
||||
frags.extend([
|
||||
("class:status-bar-dim", " · "),
|
||||
("class:status-bar-dim", duration_label),
|
||||
("class:status-bar", " "),
|
||||
])
|
||||
return frags
|
||||
|
||||
if snapshot["context_length"]:
|
||||
ctx_total = _format_context_length(snapshot["context_length"])
|
||||
ctx_used = format_token_count_compact(snapshot["context_tokens"])
|
||||
context_label = f"{ctx_used}/{ctx_total}"
|
||||
else:
|
||||
percent = snapshot["context_percent"]
|
||||
percent_label = f"{percent}%" if percent is not None else "--"
|
||||
if width < 76:
|
||||
frags = [
|
||||
("class:status-bar", " ⚕ "),
|
||||
("class:status-bar-strong", snapshot["model_short"]),
|
||||
("class:status-bar-dim", " · "),
|
||||
(self._status_bar_context_style(percent), percent_label),
|
||||
("class:status-bar-dim", " · "),
|
||||
("class:status-bar-dim", duration_label),
|
||||
("class:status-bar", " "),
|
||||
]
|
||||
else:
|
||||
if snapshot["context_length"]:
|
||||
ctx_total = _format_context_length(snapshot["context_length"])
|
||||
ctx_used = format_token_count_compact(snapshot["context_tokens"])
|
||||
context_label = f"{ctx_used}/{ctx_total}"
|
||||
else:
|
||||
context_label = "ctx --"
|
||||
context_label = "ctx --"
|
||||
|
||||
bar_style = self._status_bar_context_style(percent)
|
||||
frags = [
|
||||
("class:status-bar", " ⚕ "),
|
||||
("class:status-bar-strong", snapshot["model_short"]),
|
||||
("class:status-bar-dim", " │ "),
|
||||
("class:status-bar-dim", context_label),
|
||||
("class:status-bar-dim", " │ "),
|
||||
(bar_style, self._build_context_bar(percent)),
|
||||
("class:status-bar-dim", " "),
|
||||
(bar_style, percent_label),
|
||||
("class:status-bar-dim", " │ "),
|
||||
("class:status-bar-dim", duration_label),
|
||||
("class:status-bar", " "),
|
||||
]
|
||||
|
||||
total_width = sum(self._status_bar_display_width(text) for _, text in frags)
|
||||
if total_width > width:
|
||||
plain_text = "".join(text for _, text in frags)
|
||||
trimmed = self._trim_status_bar_text(plain_text, width)
|
||||
return [("class:status-bar", trimmed)]
|
||||
bar_style = self._status_bar_context_style(percent)
|
||||
frags = [
|
||||
("class:status-bar", " ⚕ "),
|
||||
("class:status-bar-strong", snapshot["model_short"]),
|
||||
("class:status-bar-dim", " │ "),
|
||||
("class:status-bar-dim", context_label),
|
||||
("class:status-bar-dim", " │ "),
|
||||
(bar_style, self._build_context_bar(percent)),
|
||||
("class:status-bar-dim", " "),
|
||||
(bar_style, percent_label),
|
||||
]
|
||||
frags.extend([
|
||||
("class:status-bar-dim", " │ "),
|
||||
("class:status-bar-dim", duration_label),
|
||||
("class:status-bar", " "),
|
||||
])
|
||||
return frags
|
||||
except Exception:
|
||||
return [("class:status-bar", f" {self._build_status_bar_text()} ")]
|
||||
@@ -2789,12 +2732,22 @@ class HermesCLI:
|
||||
print(f" MCP tool: /tools {subcommand} github:create_issue")
|
||||
return
|
||||
|
||||
# Apply the change directly — the user typing the command is implicit
|
||||
# consent. Do NOT use input() here; it hangs inside prompt_toolkit's
|
||||
# TUI event loop (known pitfall).
|
||||
verb = "Disabling" if subcommand == "disable" else "Enabling"
|
||||
# Confirm session reset before applying
|
||||
verb = "Disable" if subcommand == "disable" else "Enable"
|
||||
label = ", ".join(names)
|
||||
_cprint(f"{_GOLD}{verb} {label}...{_RST}")
|
||||
_cprint(f"{_GOLD}{verb} {label}?{_RST}")
|
||||
_cprint(f"{_DIM}This will save to config and reset your session so the "
|
||||
f"change takes effect cleanly.{_RST}")
|
||||
try:
|
||||
answer = input(" Continue? [y/N] ").strip().lower()
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
print()
|
||||
_cprint(f"{_DIM}Cancelled.{_RST}")
|
||||
return
|
||||
|
||||
if answer not in ("y", "yes"):
|
||||
_cprint(f"{_DIM}Cancelled.{_RST}")
|
||||
return
|
||||
|
||||
tools_disable_enable_command(
|
||||
Namespace(tools_action=subcommand, names=names, platform="cli"))
|
||||
@@ -4825,10 +4778,8 @@ class HermesCLI:
|
||||
from agent.display import get_tool_emoji
|
||||
emoji = get_tool_emoji(function_name)
|
||||
label = preview or function_name
|
||||
from agent.display import get_tool_preview_max_len
|
||||
_pl = get_tool_preview_max_len()
|
||||
if _pl > 0 and len(label) > _pl:
|
||||
label = label[:_pl - 3] + "..."
|
||||
if len(label) > 50:
|
||||
label = label[:47] + "..."
|
||||
self._spinner_text = f"{emoji} {label}"
|
||||
self._invalidate()
|
||||
|
||||
@@ -6200,11 +6151,6 @@ class HermesCLI:
|
||||
self._interrupt_queue = queue.Queue() # For messages typed while agent is running
|
||||
self._should_exit = False
|
||||
self._last_ctrl_c_time = 0 # Track double Ctrl+C for force exit
|
||||
|
||||
# Give plugin manager a CLI reference so plugins can inject messages
|
||||
from hermes_cli.plugins import get_plugin_manager
|
||||
get_plugin_manager()._cli_ref = self
|
||||
|
||||
# Config file watcher — detect mcp_servers changes and auto-reload
|
||||
from hermes_cli.config import get_config_path as _get_config_path
|
||||
_cfg_path = _get_config_path()
|
||||
@@ -6554,24 +6500,6 @@ class HermesCLI:
|
||||
self._should_exit = True
|
||||
event.app.exit()
|
||||
|
||||
@kb.add('c-z')
|
||||
def handle_ctrl_z(event):
|
||||
"""Handle Ctrl+Z - suspend process to background (Unix only)."""
|
||||
import sys
|
||||
if sys.platform == 'win32':
|
||||
_cprint(f"\n{_DIM}Suspend (Ctrl+Z) is not supported on Windows.{_RST}")
|
||||
event.app.invalidate()
|
||||
return
|
||||
import os, signal as _sig
|
||||
from prompt_toolkit.application import run_in_terminal
|
||||
from hermes_cli.skin_engine import get_active_skin
|
||||
agent_name = get_active_skin().get_branding("agent_name", "Hermes Agent")
|
||||
msg = f"\n{agent_name} has been suspended. Run `fg` to bring {agent_name} back."
|
||||
def _suspend():
|
||||
os.write(1, msg.encode())
|
||||
os.kill(0, _sig.SIGTSTP)
|
||||
run_in_terminal(_suspend)
|
||||
|
||||
# Voice push-to-talk key: configurable via config.yaml (voice.record_key)
|
||||
# Default: Ctrl+B (avoids conflict with Ctrl+R readline reverse-search)
|
||||
# Config uses "ctrl+b" format; prompt_toolkit expects "c-b" format.
|
||||
|
||||
+17
-48
@@ -26,7 +26,6 @@ except ImportError:
|
||||
msvcrt = None
|
||||
from pathlib import Path
|
||||
from hermes_constants import get_hermes_home
|
||||
from hermes_cli.config import load_config
|
||||
from typing import Optional
|
||||
|
||||
from hermes_time import now as _hermes_now
|
||||
@@ -87,22 +86,6 @@ def _resolve_delivery_target(job: dict) -> Optional[dict]:
|
||||
chat_id, thread_id = rest.split(":", 1)
|
||||
else:
|
||||
chat_id, thread_id = rest, None
|
||||
|
||||
# Resolve human-friendly labels like "Alice (dm)" to real IDs.
|
||||
# send_message(action="list") shows labels with display suffixes
|
||||
# that aren't valid platform IDs (e.g. WhatsApp JIDs).
|
||||
try:
|
||||
from gateway.channel_directory import resolve_channel_name
|
||||
target = chat_id
|
||||
# Strip display suffix like " (dm)" or " (group)"
|
||||
if target.endswith(")") and " (" in target:
|
||||
target = target.rsplit(" (", 1)[0].strip()
|
||||
resolved = resolve_channel_name(platform_name.lower(), target)
|
||||
if resolved:
|
||||
chat_id = resolved
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return {
|
||||
"platform": platform_name,
|
||||
"chat_id": chat_id,
|
||||
@@ -162,8 +145,6 @@ def _deliver_result(job: dict, content: str) -> None:
|
||||
"mattermost": Platform.MATTERMOST,
|
||||
"homeassistant": Platform.HOMEASSISTANT,
|
||||
"dingtalk": Platform.DINGTALK,
|
||||
"feishu": Platform.FEISHU,
|
||||
"wecom": Platform.WECOM,
|
||||
"email": Platform.EMAIL,
|
||||
"sms": Platform.SMS,
|
||||
}
|
||||
@@ -183,29 +164,18 @@ def _deliver_result(job: dict, content: str) -> None:
|
||||
logger.warning("Job '%s': platform '%s' not configured/enabled", job["id"], platform_name)
|
||||
return
|
||||
|
||||
# Optionally wrap the content with a header/footer so the user knows this
|
||||
# is a cron delivery. Wrapping is on by default; set cron.wrap_response: false
|
||||
# in config.yaml for clean output.
|
||||
wrap_response = True
|
||||
try:
|
||||
user_cfg = load_config()
|
||||
wrap_response = user_cfg.get("cron", {}).get("wrap_response", True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if wrap_response:
|
||||
task_name = job.get("name", job["id"])
|
||||
delivery_content = (
|
||||
f"Cronjob Response: {task_name}\n"
|
||||
f"-------------\n\n"
|
||||
f"{content}\n\n"
|
||||
f"Note: The agent cannot see this message, and therefore cannot respond to it."
|
||||
)
|
||||
else:
|
||||
delivery_content = content
|
||||
# Wrap the content so the user knows this is a cron delivery and that
|
||||
# the interactive agent has no visibility into it.
|
||||
task_name = job.get("name", job["id"])
|
||||
wrapped = (
|
||||
f"Cronjob Response: {task_name}\n"
|
||||
f"-------------\n\n"
|
||||
f"{content}\n\n"
|
||||
f"Note: The agent cannot see this message, and therefore cannot respond to it."
|
||||
)
|
||||
|
||||
# Run the async send in a fresh event loop (safe from any thread)
|
||||
coro = _send_to_platform(platform, pconfig, chat_id, delivery_content, thread_id=thread_id)
|
||||
coro = _send_to_platform(platform, pconfig, chat_id, wrapped, thread_id=thread_id)
|
||||
try:
|
||||
result = asyncio.run(coro)
|
||||
except RuntimeError:
|
||||
@@ -216,7 +186,7 @@ def _deliver_result(job: dict, content: str) -> None:
|
||||
coro.close()
|
||||
import concurrent.futures
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
|
||||
future = pool.submit(asyncio.run, _send_to_platform(platform, pconfig, chat_id, delivery_content, thread_id=thread_id))
|
||||
future = pool.submit(asyncio.run, _send_to_platform(platform, pconfig, chat_id, wrapped, thread_id=thread_id))
|
||||
result = future.result(timeout=30)
|
||||
except Exception as e:
|
||||
logger.error("Job '%s': delivery to %s:%s failed: %s", job["id"], platform_name, chat_id, e)
|
||||
@@ -236,12 +206,11 @@ def _build_job_prompt(job: dict) -> str:
|
||||
# Always prepend [SILENT] guidance so the cron agent can suppress
|
||||
# delivery when it has nothing new or noteworthy to report.
|
||||
silent_hint = (
|
||||
"[SYSTEM: If you have a meaningful status report or findings, "
|
||||
"send them — that is the whole point of this job. Only respond "
|
||||
"with exactly \"[SILENT]\" (nothing else) when there is genuinely "
|
||||
"nothing new to report. [SILENT] suppresses delivery to the user. "
|
||||
"Never combine [SILENT] with content — either report your "
|
||||
"findings normally, or say [SILENT] and nothing more.]\n\n"
|
||||
"[SYSTEM: If you have nothing new or noteworthy to report, respond "
|
||||
"with exactly \"[SILENT]\" (optionally followed by a brief internal "
|
||||
"note). This suppresses delivery to the user while still saving "
|
||||
"output locally. Only use [SILENT] when there are genuinely no "
|
||||
"changes worth reporting.]\n\n"
|
||||
)
|
||||
prompt = silent_hint + prompt
|
||||
if skills is None:
|
||||
@@ -339,7 +308,7 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||
if delivery_target.get("thread_id") is not None:
|
||||
os.environ["HERMES_CRON_AUTO_DELIVER_THREAD_ID"] = str(delivery_target["thread_id"])
|
||||
|
||||
model = job.get("model") or os.getenv("HERMES_MODEL") or ""
|
||||
model = job.get("model") or os.getenv("HERMES_MODEL") or "anthropic/claude-opus-4.6"
|
||||
|
||||
# Load config.yaml for model, reasoning, prefill, toolsets, provider routing
|
||||
_cfg = {}
|
||||
|
||||
@@ -13,7 +13,6 @@ Core layers:
|
||||
Concrete environments:
|
||||
- terminal_test_env/: Simple file-creation tasks for testing the stack
|
||||
- hermes_swe_env/: SWE-bench style tasks with Modal sandboxes
|
||||
- endless_terminals/: Terminal tasks from HuggingFace dataset with Apptainer containers
|
||||
|
||||
Benchmarks (eval-only):
|
||||
- benchmarks/terminalbench_2/: Terminal-Bench 2.0 evaluation
|
||||
|
||||
@@ -209,7 +209,7 @@ class TerminalBench2EvalEnv(HermesAgentBaseEnv):
|
||||
|
||||
# Agent settings -- TB2 tasks are complex, need many turns
|
||||
max_agent_turns=60,
|
||||
max_token_length=16000,
|
||||
max_token_length=***
|
||||
agent_temperature=0.6,
|
||||
system_prompt=None,
|
||||
|
||||
@@ -233,7 +233,7 @@ class TerminalBench2EvalEnv(HermesAgentBaseEnv):
|
||||
steps_per_eval=1,
|
||||
total_steps=1,
|
||||
|
||||
tokenizer_name="NousResearch/Hermes-3-Llama-3.1-8B",
|
||||
tokenizer_name="NousRe...1-8B",
|
||||
use_wandb=True,
|
||||
wandb_name="terminal-bench-2",
|
||||
ensure_scores_are_not_same=False, # Binary rewards may all be 0 or 1
|
||||
@@ -245,7 +245,7 @@ class TerminalBench2EvalEnv(HermesAgentBaseEnv):
|
||||
base_url="https://openrouter.ai/api/v1",
|
||||
model_name="anthropic/claude-sonnet-4",
|
||||
server_type="openai",
|
||||
api_key=os.getenv("OPENROUTER_API_KEY", ""),
|
||||
api_key=os.get...EY", ""),
|
||||
health_check=False,
|
||||
)
|
||||
]
|
||||
@@ -513,446 +513,3 @@ class TerminalBench2EvalEnv(HermesAgentBaseEnv):
|
||||
reward = 0.0
|
||||
else:
|
||||
# Run tests in a thread so the blocking ctx.terminal() calls
|
||||
# don't freeze the entire event loop (which would stall all
|
||||
# other tasks, tqdm updates, and timeout timers).
|
||||
ctx = ToolContext(task_id)
|
||||
try:
|
||||
loop = asyncio.get_event_loop()
|
||||
reward = await loop.run_in_executor(
|
||||
None, # default thread pool
|
||||
self._run_tests, eval_item, ctx, task_name,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error("Task %s: test verification failed: %s", task_name, e)
|
||||
reward = 0.0
|
||||
finally:
|
||||
ctx.cleanup()
|
||||
|
||||
passed = reward == 1.0
|
||||
status = "PASS" if passed else "FAIL"
|
||||
elapsed = time.time() - task_start
|
||||
tqdm.write(f" [{status}] {task_name} (turns={result.turns_used}, {elapsed:.0f}s)")
|
||||
logger.info(
|
||||
"Task %s: reward=%.1f, turns=%d, finished=%s",
|
||||
task_name, reward, result.turns_used, result.finished_naturally,
|
||||
)
|
||||
|
||||
out = {
|
||||
"passed": passed,
|
||||
"reward": reward,
|
||||
"task_name": task_name,
|
||||
"category": category,
|
||||
"turns_used": result.turns_used,
|
||||
"finished_naturally": result.finished_naturally,
|
||||
"messages": result.messages,
|
||||
}
|
||||
self._save_result(out)
|
||||
return out
|
||||
|
||||
except Exception as e:
|
||||
elapsed = time.time() - task_start
|
||||
logger.error("Task %s: rollout failed: %s", task_name, e, exc_info=True)
|
||||
tqdm.write(f" [ERROR] {task_name}: {e} ({elapsed:.0f}s)")
|
||||
out = {
|
||||
"passed": False, "reward": 0.0,
|
||||
"task_name": task_name, "category": category,
|
||||
"error": str(e),
|
||||
}
|
||||
self._save_result(out)
|
||||
return out
|
||||
|
||||
finally:
|
||||
# --- Cleanup: clear overrides, sandbox, and temp files ---
|
||||
clear_task_env_overrides(task_id)
|
||||
try:
|
||||
cleanup_vm(task_id)
|
||||
except Exception as e:
|
||||
logger.debug("VM cleanup for %s: %s", task_id[:8], e)
|
||||
if task_dir and task_dir.exists():
|
||||
shutil.rmtree(task_dir, ignore_errors=True)
|
||||
|
||||
def _run_tests(
|
||||
self, item: Dict[str, Any], ctx: ToolContext, task_name: str
|
||||
) -> float:
|
||||
"""
|
||||
Upload and execute the test suite in the agent's sandbox, then
|
||||
download the verifier output locally to read the reward.
|
||||
|
||||
Follows Harbor's verification pattern:
|
||||
1. Upload tests/ directory into the sandbox
|
||||
2. Execute test.sh inside the sandbox
|
||||
3. Download /logs/verifier/ directory to a local temp dir
|
||||
4. Read reward.txt locally with native Python I/O
|
||||
|
||||
Downloading locally avoids issues with the file_read tool on
|
||||
the Modal VM and matches how Harbor handles verification.
|
||||
|
||||
TB2 test scripts (test.sh) typically:
|
||||
1. Install pytest via uv/pip
|
||||
2. Run pytest against the test files in /tests/
|
||||
3. Write results to /logs/verifier/reward.txt
|
||||
|
||||
Args:
|
||||
item: The TB2 task dict (contains tests_tar, test_sh)
|
||||
ctx: ToolContext scoped to this task's sandbox
|
||||
task_name: For logging
|
||||
|
||||
Returns:
|
||||
1.0 if tests pass, 0.0 otherwise
|
||||
"""
|
||||
tests_tar = item.get("tests_tar", "")
|
||||
test_sh = item.get("test_sh", "")
|
||||
|
||||
if not test_sh:
|
||||
logger.warning("Task %s: no test_sh content, reward=0", task_name)
|
||||
return 0.0
|
||||
|
||||
# Create required directories in the sandbox
|
||||
ctx.terminal("mkdir -p /tests /logs/verifier")
|
||||
|
||||
# Upload test files into the sandbox (binary-safe via base64)
|
||||
if tests_tar:
|
||||
tests_temp = Path(tempfile.mkdtemp(prefix=f"tb2-tests-{task_name}-"))
|
||||
try:
|
||||
_extract_base64_tar(tests_tar, tests_temp)
|
||||
ctx.upload_dir(str(tests_temp), "/tests")
|
||||
except Exception as e:
|
||||
logger.warning("Task %s: failed to upload test files: %s", task_name, e)
|
||||
finally:
|
||||
shutil.rmtree(tests_temp, ignore_errors=True)
|
||||
|
||||
# Write the test runner script (test.sh)
|
||||
ctx.write_file("/tests/test.sh", test_sh)
|
||||
ctx.terminal("chmod +x /tests/test.sh")
|
||||
|
||||
# Execute the test suite
|
||||
logger.info(
|
||||
"Task %s: running test suite (timeout=%ds)",
|
||||
task_name, self.config.test_timeout,
|
||||
)
|
||||
test_result = ctx.terminal(
|
||||
"bash /tests/test.sh",
|
||||
timeout=self.config.test_timeout,
|
||||
)
|
||||
|
||||
exit_code = test_result.get("exit_code", -1)
|
||||
output = test_result.get("output", "")
|
||||
|
||||
# Download the verifier output directory locally, then read reward.txt
|
||||
# with native Python I/O. This avoids issues with file_read on the
|
||||
# Modal VM and matches Harbor's verification pattern.
|
||||
reward = 0.0
|
||||
local_verifier_dir = Path(tempfile.mkdtemp(prefix=f"tb2-verifier-{task_name}-"))
|
||||
try:
|
||||
ctx.download_dir("/logs/verifier", str(local_verifier_dir))
|
||||
|
||||
reward_file = local_verifier_dir / "reward.txt"
|
||||
if reward_file.exists() and reward_file.stat().st_size > 0:
|
||||
content = reward_file.read_text().strip()
|
||||
if content == "1":
|
||||
reward = 1.0
|
||||
elif content == "0":
|
||||
reward = 0.0
|
||||
else:
|
||||
# Unexpected content -- try parsing as float
|
||||
try:
|
||||
reward = float(content)
|
||||
except (ValueError, TypeError):
|
||||
logger.warning(
|
||||
"Task %s: reward.txt content unexpected (%r), "
|
||||
"falling back to exit_code=%d",
|
||||
task_name, content, exit_code,
|
||||
)
|
||||
reward = 1.0 if exit_code == 0 else 0.0
|
||||
else:
|
||||
# reward.txt not written -- fall back to exit code
|
||||
logger.warning(
|
||||
"Task %s: reward.txt not found after download, "
|
||||
"falling back to exit_code=%d",
|
||||
task_name, exit_code,
|
||||
)
|
||||
reward = 1.0 if exit_code == 0 else 0.0
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Task %s: failed to download verifier dir: %s, "
|
||||
"falling back to exit_code=%d",
|
||||
task_name, e, exit_code,
|
||||
)
|
||||
reward = 1.0 if exit_code == 0 else 0.0
|
||||
finally:
|
||||
shutil.rmtree(local_verifier_dir, ignore_errors=True)
|
||||
|
||||
# Log test output for debugging failures
|
||||
if reward == 0.0:
|
||||
output_preview = output[-500:] if output else "(no output)"
|
||||
logger.info(
|
||||
"Task %s: FAIL (exit_code=%d)\n%s",
|
||||
task_name, exit_code, output_preview,
|
||||
)
|
||||
|
||||
return reward
|
||||
|
||||
# =========================================================================
|
||||
# Evaluate -- main entry point for the eval subcommand
|
||||
# =========================================================================
|
||||
|
||||
async def _eval_with_timeout(self, item: Dict[str, Any]) -> Dict:
|
||||
"""
|
||||
Wrap rollout_and_score_eval with a per-task wall-clock timeout.
|
||||
|
||||
If the task exceeds task_timeout seconds, it's automatically scored
|
||||
as FAIL. This prevents any single task from hanging indefinitely.
|
||||
"""
|
||||
task_name = item.get("task_name", "unknown")
|
||||
category = item.get("category", "unknown")
|
||||
try:
|
||||
return await asyncio.wait_for(
|
||||
self.rollout_and_score_eval(item),
|
||||
timeout=self.config.task_timeout,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
from tqdm import tqdm
|
||||
elapsed = self.config.task_timeout
|
||||
tqdm.write(f" [TIMEOUT] {task_name} (exceeded {elapsed}s wall-clock limit)")
|
||||
logger.error("Task %s: wall-clock timeout after %ds", task_name, elapsed)
|
||||
out = {
|
||||
"passed": False, "reward": 0.0,
|
||||
"task_name": task_name, "category": category,
|
||||
"error": f"timeout ({elapsed}s)",
|
||||
}
|
||||
self._save_result(out)
|
||||
return out
|
||||
|
||||
async def evaluate(self, *args, **kwargs) -> None:
|
||||
"""
|
||||
Run Terminal-Bench 2.0 evaluation over all tasks.
|
||||
|
||||
This is the main entry point when invoked via:
|
||||
python environments/terminalbench2_env.py evaluate
|
||||
|
||||
Runs all tasks through rollout_and_score_eval() via asyncio.gather()
|
||||
(same pattern as GPQA and other Atropos eval envs). Each task is
|
||||
wrapped with a wall-clock timeout so hung tasks auto-fail.
|
||||
|
||||
Suppresses noisy Modal/terminal output (HERMES_QUIET) so the tqdm
|
||||
bar stays visible.
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
||||
# Route all logging through tqdm.write() so the progress bar stays
|
||||
# pinned at the bottom while log lines scroll above it.
|
||||
from tqdm import tqdm
|
||||
|
||||
class _TqdmHandler(logging.Handler):
|
||||
def emit(self, record):
|
||||
try:
|
||||
tqdm.write(self.format(record))
|
||||
except Exception:
|
||||
self.handleError(record)
|
||||
|
||||
handler = _TqdmHandler()
|
||||
handler.setFormatter(logging.Formatter(
|
||||
"%(asctime)s [%(name)s] %(levelname)s: %(message)s",
|
||||
datefmt="%H:%M:%S",
|
||||
))
|
||||
root = logging.getLogger()
|
||||
root.handlers = [handler] # Replace any existing handlers
|
||||
root.setLevel(logging.INFO)
|
||||
|
||||
# Silence noisy third-party loggers that flood the output
|
||||
logging.getLogger("httpx").setLevel(logging.WARNING) # Every HTTP request
|
||||
logging.getLogger("openai").setLevel(logging.WARNING) # OpenAI client retries
|
||||
logging.getLogger("rex-deploy").setLevel(logging.WARNING) # Swerex deployment
|
||||
logging.getLogger("rex_image_builder").setLevel(logging.WARNING) # Image builds
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print("Starting Terminal-Bench 2.0 Evaluation")
|
||||
print(f"{'='*60}")
|
||||
print(f" Dataset: {self.config.dataset_name}")
|
||||
print(f" Total tasks: {len(self.all_eval_items)}")
|
||||
print(f" Max agent turns: {self.config.max_agent_turns}")
|
||||
print(f" Task timeout: {self.config.task_timeout}s")
|
||||
print(f" Terminal backend: {self.config.terminal_backend}")
|
||||
print(f" Tool thread pool: {self.config.tool_pool_size}")
|
||||
print(f" Terminal timeout: {self.config.terminal_timeout}s/cmd")
|
||||
print(f" Terminal lifetime: {self.config.terminal_lifetime}s (auto: task_timeout + 120)")
|
||||
print(f" Max concurrent tasks: {self.config.max_concurrent_tasks}")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
# Semaphore to limit concurrent Modal sandbox creations.
|
||||
# Without this, all 86 tasks fire simultaneously, each creating a Modal
|
||||
# sandbox via asyncio.run() inside a thread pool worker. Modal's blocking
|
||||
# calls (App.lookup, etc.) deadlock when too many are created at once.
|
||||
semaphore = asyncio.Semaphore(self.config.max_concurrent_tasks)
|
||||
|
||||
async def _eval_with_semaphore(item):
|
||||
async with semaphore:
|
||||
return await self._eval_with_timeout(item)
|
||||
|
||||
# Fire all tasks with wall-clock timeout, track live accuracy on the bar
|
||||
total_tasks = len(self.all_eval_items)
|
||||
eval_tasks = [
|
||||
asyncio.ensure_future(_eval_with_semaphore(item))
|
||||
for item in self.all_eval_items
|
||||
]
|
||||
|
||||
results = []
|
||||
passed_count = 0
|
||||
pbar = tqdm(total=total_tasks, desc="Evaluating TB2", dynamic_ncols=True)
|
||||
try:
|
||||
for coro in asyncio.as_completed(eval_tasks):
|
||||
result = await coro
|
||||
results.append(result)
|
||||
if result and result.get("passed"):
|
||||
passed_count += 1
|
||||
done = len(results)
|
||||
pct = (passed_count / done * 100) if done else 0
|
||||
pbar.set_postfix_str(f"pass={passed_count}/{done} ({pct:.1f}%)")
|
||||
pbar.update(1)
|
||||
except (KeyboardInterrupt, asyncio.CancelledError):
|
||||
pbar.close()
|
||||
print(f"\n\nInterrupted! Cleaning up {len(eval_tasks)} tasks...")
|
||||
# Cancel all pending tasks
|
||||
for task in eval_tasks:
|
||||
task.cancel()
|
||||
# Let cancellations propagate (finally blocks run cleanup_vm)
|
||||
await asyncio.gather(*eval_tasks, return_exceptions=True)
|
||||
# Belt-and-suspenders: clean up any remaining sandboxes
|
||||
from tools.terminal_tool import cleanup_all_environments
|
||||
cleanup_all_environments()
|
||||
print("All sandboxes cleaned up.")
|
||||
return
|
||||
finally:
|
||||
pbar.close()
|
||||
|
||||
end_time = time.time()
|
||||
|
||||
# Filter out None results (shouldn't happen, but be safe)
|
||||
valid_results = [r for r in results if r is not None]
|
||||
|
||||
if not valid_results:
|
||||
print("Warning: No valid evaluation results obtained")
|
||||
return
|
||||
|
||||
# ---- Compute metrics ----
|
||||
total = len(valid_results)
|
||||
passed = sum(1 for r in valid_results if r.get("passed"))
|
||||
overall_pass_rate = passed / total if total > 0 else 0.0
|
||||
|
||||
# Per-category breakdown
|
||||
cat_results: Dict[str, List[Dict]] = defaultdict(list)
|
||||
for r in valid_results:
|
||||
cat_results[r.get("category", "unknown")].append(r)
|
||||
|
||||
# Build metrics dict
|
||||
eval_metrics = {
|
||||
"eval/pass_rate": overall_pass_rate,
|
||||
"eval/total_tasks": total,
|
||||
"eval/passed_tasks": passed,
|
||||
"eval/evaluation_time_seconds": end_time - start_time,
|
||||
}
|
||||
|
||||
# Per-category metrics
|
||||
for category, cat_items in sorted(cat_results.items()):
|
||||
cat_passed = sum(1 for r in cat_items if r.get("passed"))
|
||||
cat_total = len(cat_items)
|
||||
cat_pass_rate = cat_passed / cat_total if cat_total > 0 else 0.0
|
||||
cat_key = category.replace(" ", "_").replace("-", "_").lower()
|
||||
eval_metrics[f"eval/pass_rate_{cat_key}"] = cat_pass_rate
|
||||
|
||||
# Store metrics for wandb_log
|
||||
self.eval_metrics = [(k, v) for k, v in eval_metrics.items()]
|
||||
|
||||
# ---- Print summary ----
|
||||
print(f"\n{'='*60}")
|
||||
print("Terminal-Bench 2.0 Evaluation Results")
|
||||
print(f"{'='*60}")
|
||||
print(f"Overall Pass Rate: {overall_pass_rate:.4f} ({passed}/{total})")
|
||||
print(f"Evaluation Time: {end_time - start_time:.1f} seconds")
|
||||
|
||||
print("\nCategory Breakdown:")
|
||||
for category, cat_items in sorted(cat_results.items()):
|
||||
cat_passed = sum(1 for r in cat_items if r.get("passed"))
|
||||
cat_total = len(cat_items)
|
||||
cat_rate = cat_passed / cat_total if cat_total > 0 else 0.0
|
||||
print(f" {category}: {cat_rate:.1%} ({cat_passed}/{cat_total})")
|
||||
|
||||
# Print individual task results
|
||||
print("\nTask Results:")
|
||||
for r in sorted(valid_results, key=lambda x: x.get("task_name", "")):
|
||||
status = "PASS" if r.get("passed") else "FAIL"
|
||||
turns = r.get("turns_used", "?")
|
||||
error = r.get("error", "")
|
||||
extra = f" (error: {error})" if error else ""
|
||||
print(f" [{status}] {r['task_name']} (turns={turns}){extra}")
|
||||
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
# Build sample records for evaluate_log (includes full conversations)
|
||||
samples = [
|
||||
{
|
||||
"task_name": r.get("task_name"),
|
||||
"category": r.get("category"),
|
||||
"passed": r.get("passed"),
|
||||
"reward": r.get("reward"),
|
||||
"turns_used": r.get("turns_used"),
|
||||
"error": r.get("error"),
|
||||
"messages": r.get("messages"),
|
||||
}
|
||||
for r in valid_results
|
||||
]
|
||||
|
||||
# Log evaluation results
|
||||
try:
|
||||
await self.evaluate_log(
|
||||
metrics=eval_metrics,
|
||||
samples=samples,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
generation_parameters={
|
||||
"temperature": self.config.agent_temperature,
|
||||
"max_tokens": self.config.max_token_length,
|
||||
"max_agent_turns": self.config.max_agent_turns,
|
||||
"terminal_backend": self.config.terminal_backend,
|
||||
},
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"Error logging evaluation results: {e}")
|
||||
|
||||
# Close streaming file
|
||||
if hasattr(self, "_streaming_file") and not self._streaming_file.closed:
|
||||
self._streaming_file.close()
|
||||
print(f" Live results saved to: {self._streaming_path}")
|
||||
|
||||
# Kill all remaining sandboxes. Timed-out tasks leave orphaned thread
|
||||
# pool workers still executing commands -- cleanup_all stops them.
|
||||
from tools.terminal_tool import cleanup_all_environments
|
||||
print("\nCleaning up all sandboxes...")
|
||||
cleanup_all_environments()
|
||||
|
||||
# Shut down the tool thread pool so orphaned workers from timed-out
|
||||
# tasks are killed immediately instead of retrying against dead
|
||||
# sandboxes and spamming the console with TimeoutError warnings.
|
||||
from environments.agent_loop import _tool_executor
|
||||
_tool_executor.shutdown(wait=False, cancel_futures=True)
|
||||
print("Done.")
|
||||
|
||||
# =========================================================================
|
||||
# Wandb logging
|
||||
# =========================================================================
|
||||
|
||||
async def wandb_log(self, wandb_metrics: Optional[Dict] = None):
|
||||
"""Log TB2-specific metrics to wandb."""
|
||||
if wandb_metrics is None:
|
||||
wandb_metrics = {}
|
||||
|
||||
# Add stored eval metrics
|
||||
for metric_name, metric_value in self.eval_metrics:
|
||||
wandb_metrics[metric_name] = metric_value
|
||||
self.eval_metrics = []
|
||||
|
||||
await super().wandb_log(wandb_metrics)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
TerminalBench2EvalEnv.cli()
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
"""Endless Terminals Environment - Terminal task training from HuggingFace dataset."""
|
||||
|
||||
from .endless_terminals_env import EndlessTerminalsEnv, EndlessTerminalsEnvConfig
|
||||
|
||||
__all__ = ["EndlessTerminalsEnv", "EndlessTerminalsEnvConfig"]
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,91 +0,0 @@
|
||||
# Endless Terminals - Qwen3-4B-Instruct-2507
|
||||
# Single config for both trainer (launch_training.py) and env (endless_terminals_env.py serve)
|
||||
#
|
||||
# Usage:
|
||||
# Terminal 1: run-api
|
||||
# Terminal 2: cd tinker-atropos && python launch_training.py --config ../environments/endless_terminals/tinker_qwen.yaml
|
||||
# Terminal 3: python environments/endless_terminals/endless_terminals_env.py serve --config environments/endless_terminals/tinker_qwen.yaml
|
||||
|
||||
env:
|
||||
# Toolsets
|
||||
enabled_toolsets: ["terminal", "file"]
|
||||
|
||||
# Model / tokenizer
|
||||
tokenizer_name: "Qwen/Qwen3-4B-Instruct-2507"
|
||||
|
||||
# Agent configuration
|
||||
max_agent_turns: 16
|
||||
max_token_length: 2048
|
||||
agent_temperature: 0.6
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
tool_call_parser: "hermes"
|
||||
|
||||
# Terminal backend
|
||||
terminal_backend: "docker"
|
||||
|
||||
# Dataset settings
|
||||
use_dataset: true
|
||||
dataset_name: "obiwan96/endless-terminals"
|
||||
dataset_split: "train"
|
||||
dataset_cache_dir: "~/.cache/huggingface/datasets"
|
||||
tasks_base_dir: "/Users/samherring/Desktop/Projects/Hermes-Agent/endless-terminals"
|
||||
|
||||
# Test execution
|
||||
test_timeout_s: 180
|
||||
default_docker_image: "ubuntu:22.04"
|
||||
max_concurrent_containers: 16
|
||||
|
||||
# Training configuration
|
||||
group_size: 16
|
||||
batch_size: 64 # 4 groups × 16 rollouts per step
|
||||
total_steps: 500
|
||||
steps_per_eval: 5
|
||||
min_items_sent_before_logging: 1
|
||||
ensure_scores_are_not_same: true
|
||||
max_num_workers: 2048
|
||||
worker_timeout: 3600
|
||||
inference_weight: 1.0
|
||||
eval_limit_ratio: 0.1
|
||||
rollout_server_url: "http://localhost:8000"
|
||||
|
||||
# Evaluation configuration
|
||||
num_eval_tasks: 20
|
||||
eval_split_ratio: 0.1
|
||||
|
||||
# Logging
|
||||
use_wandb: true
|
||||
wandb_name: "endless-terminals-qwen3-4b"
|
||||
|
||||
# System prompt
|
||||
system_prompt: >
|
||||
You are a skilled Linux system administrator and programmer.
|
||||
You have access to a terminal and file tools to complete system administration
|
||||
and programming tasks. Use the tools effectively to solve the given task,
|
||||
and verify your solution works correctly before finishing.
|
||||
Keep each command short and focused — break complex tasks into multiple steps
|
||||
rather than writing long one-liners.
|
||||
|
||||
tinker:
|
||||
lora_rank: 32
|
||||
learning_rate: 0.0000005
|
||||
max_token_trainer_length: 32768
|
||||
checkpoint_dir: "./temp/"
|
||||
save_checkpoint_interval: 50
|
||||
wandb_project: "endless-terminals"
|
||||
wandb_group: null
|
||||
wandb_run_name: "qwen3-4b"
|
||||
tool_call_parser: "hermes"
|
||||
|
||||
openai:
|
||||
- model_name: "Qwen/Qwen3-4B-Instruct-2507"
|
||||
base_url: "http://localhost:8001/v1"
|
||||
api_key: "x"
|
||||
weight: 1.0
|
||||
num_requests_for_eval: 64
|
||||
timeout: 600
|
||||
server_type: "sglang"
|
||||
|
||||
slurm: false
|
||||
testing: false
|
||||
@@ -298,6 +298,7 @@ class HermesAgentBaseEnv(BaseEnv):
|
||||
return False
|
||||
|
||||
server = self.server.servers[0]
|
||||
# If the server is an OpenAI server (not VLLM/SGLang), use direct mode
|
||||
from atroposlib.envs.server_handling.openai_server import OpenAIServer
|
||||
return not isinstance(server, OpenAIServer)
|
||||
|
||||
|
||||
@@ -48,13 +48,7 @@ class HermesToolCallParser(ToolCallParser):
|
||||
if not raw_json.strip():
|
||||
continue
|
||||
|
||||
try:
|
||||
tc_data = json.loads(raw_json)
|
||||
except json.JSONDecodeError:
|
||||
# Fix invalid backslash escapes from shell commands in JSON strings
|
||||
# e.g. \s \w \d \n (unescaped) → \\s \\w \\d \\n
|
||||
fixed = re.sub(r'\\([^"\\/bfnrtu0-9\n])', r'\\\\\1', raw_json)
|
||||
tc_data = json.loads(fixed)
|
||||
tc_data = json.loads(raw_json)
|
||||
tool_calls.append(
|
||||
ChatCompletionMessageToolCall(
|
||||
id=f"call_{uuid.uuid4().hex[:8]}",
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
"""Built-in gateway hooks that are always registered."""
|
||||
@@ -1,86 +0,0 @@
|
||||
"""Built-in boot-md hook — run ~/.hermes/BOOT.md on gateway startup.
|
||||
|
||||
This hook is always registered. It silently skips if no BOOT.md exists.
|
||||
To activate, create ``~/.hermes/BOOT.md`` with instructions for the
|
||||
agent to execute on every gateway restart.
|
||||
|
||||
Example BOOT.md::
|
||||
|
||||
# Startup Checklist
|
||||
|
||||
1. Check if any cron jobs failed overnight
|
||||
2. Send a status update to Discord #general
|
||||
3. If there are errors in /opt/app/deploy.log, summarize them
|
||||
|
||||
The agent runs in a background thread so it doesn't block gateway
|
||||
startup. If nothing needs attention, it replies with [SILENT] to
|
||||
suppress delivery.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger("hooks.boot-md")
|
||||
|
||||
HERMES_HOME = Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes"))
|
||||
BOOT_FILE = HERMES_HOME / "BOOT.md"
|
||||
|
||||
|
||||
def _build_boot_prompt(content: str) -> str:
|
||||
"""Wrap BOOT.md content in a system-level instruction."""
|
||||
return (
|
||||
"You are running a startup boot checklist. Follow the BOOT.md "
|
||||
"instructions below exactly.\n\n"
|
||||
"---\n"
|
||||
f"{content}\n"
|
||||
"---\n\n"
|
||||
"Execute each instruction. If you need to send a message to a "
|
||||
"platform, use the send_message tool.\n"
|
||||
"If nothing needs attention and there is nothing to report, "
|
||||
"reply with ONLY: [SILENT]"
|
||||
)
|
||||
|
||||
|
||||
def _run_boot_agent(content: str) -> None:
|
||||
"""Spawn a one-shot agent session to execute the boot instructions."""
|
||||
try:
|
||||
from run_agent import AIAgent
|
||||
|
||||
prompt = _build_boot_prompt(content)
|
||||
agent = AIAgent(
|
||||
quiet_mode=True,
|
||||
skip_context_files=True,
|
||||
skip_memory=True,
|
||||
max_iterations=20,
|
||||
)
|
||||
result = agent.run_conversation(prompt)
|
||||
response = result.get("final_response", "")
|
||||
if response and "[SILENT]" not in response:
|
||||
logger.info("boot-md completed: %s", response[:200])
|
||||
else:
|
||||
logger.info("boot-md completed (nothing to report)")
|
||||
except Exception as e:
|
||||
logger.error("boot-md agent failed: %s", e)
|
||||
|
||||
|
||||
async def handle(event_type: str, context: dict) -> None:
|
||||
"""Gateway startup handler — run BOOT.md if it exists."""
|
||||
if not BOOT_FILE.exists():
|
||||
return
|
||||
|
||||
content = BOOT_FILE.read_text(encoding="utf-8").strip()
|
||||
if not content:
|
||||
return
|
||||
|
||||
logger.info("Running BOOT.md (%d chars)", len(content))
|
||||
|
||||
# Run in a background thread so we don't block gateway startup.
|
||||
thread = threading.Thread(
|
||||
target=_run_boot_agent,
|
||||
args=(content,),
|
||||
name="boot-md",
|
||||
daemon=True,
|
||||
)
|
||||
thread.start()
|
||||
+43
-117
@@ -57,8 +57,6 @@ class Platform(Enum):
|
||||
DINGTALK = "dingtalk"
|
||||
API_SERVER = "api_server"
|
||||
WEBHOOK = "webhook"
|
||||
FEISHU = "feishu"
|
||||
WECOM = "wecom"
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -276,12 +274,6 @@ class GatewayConfig:
|
||||
# Webhook uses enabled flag only (secrets are per-route)
|
||||
elif platform == Platform.WEBHOOK:
|
||||
connected.append(platform)
|
||||
# Feishu uses extra dict for app credentials
|
||||
elif platform == Platform.FEISHU and config.extra.get("app_id"):
|
||||
connected.append(platform)
|
||||
# WeCom uses extra dict for bot credentials
|
||||
elif platform == Platform.WECOM and config.extra.get("bot_id"):
|
||||
connected.append(platform)
|
||||
return connected
|
||||
|
||||
def get_home_channel(self, platform: Platform) -> Optional[HomeChannel]:
|
||||
@@ -515,10 +507,6 @@ def load_gateway_config() -> GatewayConfig:
|
||||
)
|
||||
if "reply_prefix" in platform_cfg:
|
||||
bridged["reply_prefix"] = platform_cfg["reply_prefix"]
|
||||
if "require_mention" in platform_cfg:
|
||||
bridged["require_mention"] = platform_cfg["require_mention"]
|
||||
if "mention_patterns" in platform_cfg:
|
||||
bridged["mention_patterns"] = platform_cfg["mention_patterns"]
|
||||
if not bridged:
|
||||
continue
|
||||
plat_data = platforms_data.setdefault(plat.value, {})
|
||||
@@ -543,20 +531,6 @@ def load_gateway_config() -> GatewayConfig:
|
||||
os.environ["DISCORD_FREE_RESPONSE_CHANNELS"] = str(frc)
|
||||
if "auto_thread" in discord_cfg and not os.getenv("DISCORD_AUTO_THREAD"):
|
||||
os.environ["DISCORD_AUTO_THREAD"] = str(discord_cfg["auto_thread"]).lower()
|
||||
|
||||
# Telegram settings → env vars (env vars take precedence)
|
||||
telegram_cfg = yaml_cfg.get("telegram", {})
|
||||
if isinstance(telegram_cfg, dict):
|
||||
if "require_mention" in telegram_cfg and not os.getenv("TELEGRAM_REQUIRE_MENTION"):
|
||||
os.environ["TELEGRAM_REQUIRE_MENTION"] = str(telegram_cfg["require_mention"]).lower()
|
||||
if "mention_patterns" in telegram_cfg and not os.getenv("TELEGRAM_MENTION_PATTERNS"):
|
||||
import json as _json
|
||||
os.environ["TELEGRAM_MENTION_PATTERNS"] = _json.dumps(telegram_cfg["mention_patterns"])
|
||||
frc = telegram_cfg.get("free_response_chats")
|
||||
if frc is not None and not os.getenv("TELEGRAM_FREE_RESPONSE_CHATS"):
|
||||
if isinstance(frc, list):
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["TELEGRAM_FREE_RESPONSE_CHATS"] = str(frc)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Failed to process config.yaml — falling back to .env / gateway.json values. "
|
||||
@@ -673,13 +647,14 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
config.platforms[Platform.SLACK] = PlatformConfig()
|
||||
config.platforms[Platform.SLACK].enabled = True
|
||||
config.platforms[Platform.SLACK].token = slack_token
|
||||
slack_home = os.getenv("SLACK_HOME_CHANNEL")
|
||||
if slack_home and Platform.SLACK in config.platforms:
|
||||
config.platforms[Platform.SLACK].home_channel = HomeChannel(
|
||||
platform=Platform.SLACK,
|
||||
chat_id=slack_home,
|
||||
name=os.getenv("SLACK_HOME_CHANNEL_NAME", ""),
|
||||
)
|
||||
# Home channel
|
||||
slack_home = os.getenv("SLACK_HOME_CHANNEL")
|
||||
if slack_home:
|
||||
config.platforms[Platform.SLACK].home_channel = HomeChannel(
|
||||
platform=Platform.SLACK,
|
||||
chat_id=slack_home,
|
||||
name=os.getenv("SLACK_HOME_CHANNEL_NAME", ""),
|
||||
)
|
||||
|
||||
# Signal
|
||||
signal_url = os.getenv("SIGNAL_HTTP_URL")
|
||||
@@ -693,13 +668,13 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
"account": signal_account,
|
||||
"ignore_stories": os.getenv("SIGNAL_IGNORE_STORIES", "true").lower() in ("true", "1", "yes"),
|
||||
})
|
||||
signal_home = os.getenv("SIGNAL_HOME_CHANNEL")
|
||||
if signal_home and Platform.SIGNAL in config.platforms:
|
||||
config.platforms[Platform.SIGNAL].home_channel = HomeChannel(
|
||||
platform=Platform.SIGNAL,
|
||||
chat_id=signal_home,
|
||||
name=os.getenv("SIGNAL_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
signal_home = os.getenv("SIGNAL_HOME_CHANNEL")
|
||||
if signal_home:
|
||||
config.platforms[Platform.SIGNAL].home_channel = HomeChannel(
|
||||
platform=Platform.SIGNAL,
|
||||
chat_id=signal_home,
|
||||
name=os.getenv("SIGNAL_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# Mattermost
|
||||
mattermost_token = os.getenv("MATTERMOST_TOKEN")
|
||||
@@ -712,13 +687,13 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
config.platforms[Platform.MATTERMOST].enabled = True
|
||||
config.platforms[Platform.MATTERMOST].token = mattermost_token
|
||||
config.platforms[Platform.MATTERMOST].extra["url"] = mattermost_url
|
||||
mattermost_home = os.getenv("MATTERMOST_HOME_CHANNEL")
|
||||
if mattermost_home and Platform.MATTERMOST in config.platforms:
|
||||
config.platforms[Platform.MATTERMOST].home_channel = HomeChannel(
|
||||
platform=Platform.MATTERMOST,
|
||||
chat_id=mattermost_home,
|
||||
name=os.getenv("MATTERMOST_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
mattermost_home = os.getenv("MATTERMOST_HOME_CHANNEL")
|
||||
if mattermost_home:
|
||||
config.platforms[Platform.MATTERMOST].home_channel = HomeChannel(
|
||||
platform=Platform.MATTERMOST,
|
||||
chat_id=mattermost_home,
|
||||
name=os.getenv("MATTERMOST_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# Matrix
|
||||
matrix_token = os.getenv("MATRIX_ACCESS_TOKEN")
|
||||
@@ -740,13 +715,13 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
config.platforms[Platform.MATRIX].extra["password"] = matrix_password
|
||||
matrix_e2ee = os.getenv("MATRIX_ENCRYPTION", "").lower() in ("true", "1", "yes")
|
||||
config.platforms[Platform.MATRIX].extra["encryption"] = matrix_e2ee
|
||||
matrix_home = os.getenv("MATRIX_HOME_ROOM")
|
||||
if matrix_home and Platform.MATRIX in config.platforms:
|
||||
config.platforms[Platform.MATRIX].home_channel = HomeChannel(
|
||||
platform=Platform.MATRIX,
|
||||
chat_id=matrix_home,
|
||||
name=os.getenv("MATRIX_HOME_ROOM_NAME", "Home"),
|
||||
)
|
||||
matrix_home = os.getenv("MATRIX_HOME_ROOM")
|
||||
if matrix_home:
|
||||
config.platforms[Platform.MATRIX].home_channel = HomeChannel(
|
||||
platform=Platform.MATRIX,
|
||||
chat_id=matrix_home,
|
||||
name=os.getenv("MATRIX_HOME_ROOM_NAME", "Home"),
|
||||
)
|
||||
|
||||
# Home Assistant
|
||||
hass_token = os.getenv("HASS_TOKEN")
|
||||
@@ -773,13 +748,13 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
"imap_host": email_imap,
|
||||
"smtp_host": email_smtp,
|
||||
})
|
||||
email_home = os.getenv("EMAIL_HOME_ADDRESS")
|
||||
if email_home and Platform.EMAIL in config.platforms:
|
||||
config.platforms[Platform.EMAIL].home_channel = HomeChannel(
|
||||
platform=Platform.EMAIL,
|
||||
chat_id=email_home,
|
||||
name=os.getenv("EMAIL_HOME_ADDRESS_NAME", "Home"),
|
||||
)
|
||||
email_home = os.getenv("EMAIL_HOME_ADDRESS")
|
||||
if email_home:
|
||||
config.platforms[Platform.EMAIL].home_channel = HomeChannel(
|
||||
platform=Platform.EMAIL,
|
||||
chat_id=email_home,
|
||||
name=os.getenv("EMAIL_HOME_ADDRESS_NAME", "Home"),
|
||||
)
|
||||
|
||||
# SMS (Twilio)
|
||||
twilio_sid = os.getenv("TWILIO_ACCOUNT_SID")
|
||||
@@ -788,13 +763,13 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
config.platforms[Platform.SMS] = PlatformConfig()
|
||||
config.platforms[Platform.SMS].enabled = True
|
||||
config.platforms[Platform.SMS].api_key = os.getenv("TWILIO_AUTH_TOKEN", "")
|
||||
sms_home = os.getenv("SMS_HOME_CHANNEL")
|
||||
if sms_home and Platform.SMS in config.platforms:
|
||||
config.platforms[Platform.SMS].home_channel = HomeChannel(
|
||||
platform=Platform.SMS,
|
||||
chat_id=sms_home,
|
||||
name=os.getenv("SMS_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
sms_home = os.getenv("SMS_HOME_CHANNEL")
|
||||
if sms_home:
|
||||
config.platforms[Platform.SMS].home_channel = HomeChannel(
|
||||
platform=Platform.SMS,
|
||||
chat_id=sms_home,
|
||||
name=os.getenv("SMS_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# API Server
|
||||
api_server_enabled = os.getenv("API_SERVER_ENABLED", "").lower() in ("true", "1", "yes")
|
||||
@@ -836,55 +811,6 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
if webhook_secret:
|
||||
config.platforms[Platform.WEBHOOK].extra["secret"] = webhook_secret
|
||||
|
||||
# Feishu / Lark
|
||||
feishu_app_id = os.getenv("FEISHU_APP_ID")
|
||||
feishu_app_secret = os.getenv("FEISHU_APP_SECRET")
|
||||
if feishu_app_id and feishu_app_secret:
|
||||
if Platform.FEISHU not in config.platforms:
|
||||
config.platforms[Platform.FEISHU] = PlatformConfig()
|
||||
config.platforms[Platform.FEISHU].enabled = True
|
||||
config.platforms[Platform.FEISHU].extra.update({
|
||||
"app_id": feishu_app_id,
|
||||
"app_secret": feishu_app_secret,
|
||||
"domain": os.getenv("FEISHU_DOMAIN", "feishu"),
|
||||
"connection_mode": os.getenv("FEISHU_CONNECTION_MODE", "websocket"),
|
||||
})
|
||||
feishu_encrypt_key = os.getenv("FEISHU_ENCRYPT_KEY", "")
|
||||
if feishu_encrypt_key:
|
||||
config.platforms[Platform.FEISHU].extra["encrypt_key"] = feishu_encrypt_key
|
||||
feishu_verification_token = os.getenv("FEISHU_VERIFICATION_TOKEN", "")
|
||||
if feishu_verification_token:
|
||||
config.platforms[Platform.FEISHU].extra["verification_token"] = feishu_verification_token
|
||||
feishu_home = os.getenv("FEISHU_HOME_CHANNEL")
|
||||
if feishu_home:
|
||||
config.platforms[Platform.FEISHU].home_channel = HomeChannel(
|
||||
platform=Platform.FEISHU,
|
||||
chat_id=feishu_home,
|
||||
name=os.getenv("FEISHU_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# WeCom (Enterprise WeChat)
|
||||
wecom_bot_id = os.getenv("WECOM_BOT_ID")
|
||||
wecom_secret = os.getenv("WECOM_SECRET")
|
||||
if wecom_bot_id and wecom_secret:
|
||||
if Platform.WECOM not in config.platforms:
|
||||
config.platforms[Platform.WECOM] = PlatformConfig()
|
||||
config.platforms[Platform.WECOM].enabled = True
|
||||
config.platforms[Platform.WECOM].extra.update({
|
||||
"bot_id": wecom_bot_id,
|
||||
"secret": wecom_secret,
|
||||
})
|
||||
wecom_ws_url = os.getenv("WECOM_WEBSOCKET_URL", "")
|
||||
if wecom_ws_url:
|
||||
config.platforms[Platform.WECOM].extra["websocket_url"] = wecom_ws_url
|
||||
wecom_home = os.getenv("WECOM_HOME_CHANNEL")
|
||||
if wecom_home:
|
||||
config.platforms[Platform.WECOM].home_channel = HomeChannel(
|
||||
platform=Platform.WECOM,
|
||||
chat_id=wecom_home,
|
||||
name=os.getenv("WECOM_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# Session settings
|
||||
idle_minutes = os.getenv("SESSION_IDLE_MINUTES")
|
||||
if idle_minutes:
|
||||
|
||||
@@ -51,33 +51,14 @@ class HookRegistry:
|
||||
"""Return metadata about all loaded hooks."""
|
||||
return list(self._loaded_hooks)
|
||||
|
||||
def _register_builtin_hooks(self) -> None:
|
||||
"""Register built-in hooks that are always active."""
|
||||
try:
|
||||
from gateway.builtin_hooks.boot_md import handle as boot_md_handle
|
||||
|
||||
self._handlers.setdefault("gateway:startup", []).append(boot_md_handle)
|
||||
self._loaded_hooks.append({
|
||||
"name": "boot-md",
|
||||
"description": "Run ~/.hermes/BOOT.md on gateway startup",
|
||||
"events": ["gateway:startup"],
|
||||
"path": "(builtin)",
|
||||
})
|
||||
except Exception as e:
|
||||
print(f"[hooks] Could not load built-in boot-md hook: {e}", flush=True)
|
||||
|
||||
def discover_and_load(self) -> None:
|
||||
"""
|
||||
Scan the hooks directory for hook directories and load their handlers.
|
||||
|
||||
Also registers built-in hooks that are always active.
|
||||
|
||||
Each hook directory must contain:
|
||||
- HOOK.yaml with at least 'name' and 'events' keys
|
||||
- handler.py with a top-level 'handle' function (sync or async)
|
||||
"""
|
||||
self._register_builtin_hooks()
|
||||
|
||||
if not HOOKS_DIR.exists():
|
||||
return
|
||||
|
||||
|
||||
@@ -898,26 +898,6 @@ class BasePlatformAdapter(ABC):
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ── Processing lifecycle hooks ──────────────────────────────────────────
|
||||
# Subclasses override these to react to message processing events
|
||||
# (e.g. Discord adds 👀/✅/❌ reactions).
|
||||
|
||||
async def on_processing_start(self, event: MessageEvent) -> None:
|
||||
"""Hook called when background processing begins."""
|
||||
|
||||
async def on_processing_complete(self, event: MessageEvent, success: bool) -> None:
|
||||
"""Hook called when background processing completes."""
|
||||
|
||||
async def _run_processing_hook(self, hook_name: str, *args: Any, **kwargs: Any) -> None:
|
||||
"""Run a lifecycle hook without letting failures break message flow."""
|
||||
hook = getattr(self, hook_name, None)
|
||||
if not callable(hook):
|
||||
return
|
||||
try:
|
||||
await hook(*args, **kwargs)
|
||||
except Exception as e:
|
||||
logger.warning("[%s] %s hook failed: %s", self.name, hook_name, e)
|
||||
|
||||
@staticmethod
|
||||
def _is_retryable_error(error: Optional[str]) -> bool:
|
||||
"""Return True if the error string looks like a transient network failure."""
|
||||
@@ -1080,18 +1060,6 @@ class BasePlatformAdapter(ABC):
|
||||
|
||||
async def _process_message_background(self, event: MessageEvent, session_key: str) -> None:
|
||||
"""Background task that actually processes the message."""
|
||||
# Track delivery outcomes for the processing-complete hook
|
||||
delivery_attempted = False
|
||||
delivery_succeeded = False
|
||||
|
||||
def _record_delivery(result):
|
||||
nonlocal delivery_attempted, delivery_succeeded
|
||||
if result is None:
|
||||
return
|
||||
delivery_attempted = True
|
||||
if getattr(result, "success", False):
|
||||
delivery_succeeded = True
|
||||
|
||||
# Create interrupt event for this session
|
||||
interrupt_event = asyncio.Event()
|
||||
self._active_sessions[session_key] = interrupt_event
|
||||
@@ -1101,8 +1069,6 @@ class BasePlatformAdapter(ABC):
|
||||
typing_task = asyncio.create_task(self._keep_typing(event.source.chat_id, metadata=_thread_metadata))
|
||||
|
||||
try:
|
||||
await self._run_processing_hook("on_processing_start", event)
|
||||
|
||||
# Call the handler (this can take a while with tool calls)
|
||||
response = await self._message_handler(event)
|
||||
|
||||
@@ -1172,7 +1138,6 @@ class BasePlatformAdapter(ABC):
|
||||
reply_to=event.message_id,
|
||||
metadata=_thread_metadata,
|
||||
)
|
||||
_record_delivery(result)
|
||||
|
||||
# Human-like pacing delay between text and media
|
||||
human_delay = self._get_human_delay()
|
||||
@@ -1272,10 +1237,6 @@ class BasePlatformAdapter(ABC):
|
||||
except Exception as file_err:
|
||||
logger.error("[%s] Error sending local file %s: %s", self.name, file_path, file_err)
|
||||
|
||||
# Determine overall success for the processing hook
|
||||
processing_ok = delivery_succeeded if delivery_attempted else not bool(response)
|
||||
await self._run_processing_hook("on_processing_complete", event, processing_ok)
|
||||
|
||||
# Check if there's a pending message that was queued during our processing
|
||||
if session_key in self._pending_messages:
|
||||
pending_event = self._pending_messages.pop(session_key)
|
||||
@@ -1292,11 +1253,7 @@ class BasePlatformAdapter(ABC):
|
||||
await self._process_message_background(pending_event, session_key)
|
||||
return # Already cleaned up
|
||||
|
||||
except asyncio.CancelledError:
|
||||
await self._run_processing_hook("on_processing_complete", event, False)
|
||||
raise
|
||||
except Exception as e:
|
||||
await self._run_processing_hook("on_processing_complete", event, False)
|
||||
logger.error("[%s] Error handling message: %s", self.name, e, exc_info=True)
|
||||
# Send the error to the user so they aren't left with radio silence
|
||||
try:
|
||||
|
||||
@@ -488,8 +488,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
try:
|
||||
# Acquire scoped lock to prevent duplicate bot token usage
|
||||
from gateway.status import acquire_scoped_lock
|
||||
self._token_lock_identity = self.config.token
|
||||
acquired, existing = acquire_scoped_lock('discord-bot-token', self._token_lock_identity, metadata={'platform': 'discord'})
|
||||
acquired, existing = acquire_scoped_lock('discord-bot-token', self.config.token, metadata={'platform': 'discord'})
|
||||
if not acquired:
|
||||
owner_pid = existing.get('pid') if isinstance(existing, dict) else None
|
||||
message = f'Discord bot token already in use' + (f' (PID {owner_pid})' if owner_pid else '') + '. Stop the other gateway first.'
|
||||
@@ -653,48 +652,11 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
# Release the token lock
|
||||
try:
|
||||
from gateway.status import release_scoped_lock
|
||||
if getattr(self, '_token_lock_identity', None):
|
||||
release_scoped_lock('discord-bot-token', self._token_lock_identity)
|
||||
self._token_lock_identity = None
|
||||
release_scoped_lock('discord-bot-token', self.config.token)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
logger.info("[%s] Disconnected", self.name)
|
||||
|
||||
async def _add_reaction(self, message: Any, emoji: str) -> bool:
|
||||
"""Add an emoji reaction to a Discord message."""
|
||||
if not message or not hasattr(message, "add_reaction"):
|
||||
return False
|
||||
try:
|
||||
await message.add_reaction(emoji)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.debug("[%s] add_reaction failed (%s): %s", self.name, emoji, e)
|
||||
return False
|
||||
|
||||
async def _remove_reaction(self, message: Any, emoji: str) -> bool:
|
||||
"""Remove the bot's own emoji reaction from a Discord message."""
|
||||
if not message or not hasattr(message, "remove_reaction") or not self._client or not self._client.user:
|
||||
return False
|
||||
try:
|
||||
await message.remove_reaction(emoji, self._client.user)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.debug("[%s] remove_reaction failed (%s): %s", self.name, emoji, e)
|
||||
return False
|
||||
|
||||
async def on_processing_start(self, event: MessageEvent) -> None:
|
||||
"""Add an in-progress reaction for normal Discord message events."""
|
||||
message = event.raw_message
|
||||
if hasattr(message, "add_reaction"):
|
||||
await self._add_reaction(message, "👀")
|
||||
|
||||
async def on_processing_complete(self, event: MessageEvent, success: bool) -> None:
|
||||
"""Swap the in-progress reaction for a final success/failure reaction."""
|
||||
message = event.raw_message
|
||||
if hasattr(message, "add_reaction"):
|
||||
await self._remove_reaction(message, "👀")
|
||||
await self._add_reaction(message, "✅" if success else "❌")
|
||||
|
||||
async def send(
|
||||
self,
|
||||
|
||||
+55
-68
@@ -337,63 +337,60 @@ class EmailAdapter(BasePlatformAdapter):
|
||||
results = []
|
||||
try:
|
||||
imap = imaplib.IMAP4_SSL(self._imap_host, self._imap_port, timeout=30)
|
||||
try:
|
||||
imap.login(self._address, self._password)
|
||||
imap.select("INBOX")
|
||||
imap.login(self._address, self._password)
|
||||
imap.select("INBOX")
|
||||
|
||||
status, data = imap.uid("search", None, "UNSEEN")
|
||||
if status != "OK" or not data or not data[0]:
|
||||
return results
|
||||
status, data = imap.uid("search", None, "UNSEEN")
|
||||
if status != "OK" or not data or not data[0]:
|
||||
imap.logout()
|
||||
return results
|
||||
|
||||
for uid in data[0].split():
|
||||
if uid in self._seen_uids:
|
||||
continue
|
||||
self._seen_uids.add(uid)
|
||||
# Trim periodically to prevent unbounded memory growth
|
||||
if len(self._seen_uids) > self._seen_uids_max:
|
||||
self._trim_seen_uids()
|
||||
for uid in data[0].split():
|
||||
if uid in self._seen_uids:
|
||||
continue
|
||||
self._seen_uids.add(uid)
|
||||
# Trim periodically to prevent unbounded memory growth
|
||||
if len(self._seen_uids) > self._seen_uids_max:
|
||||
self._trim_seen_uids()
|
||||
|
||||
status, msg_data = imap.uid("fetch", uid, "(RFC822)")
|
||||
if status != "OK":
|
||||
continue
|
||||
status, msg_data = imap.uid("fetch", uid, "(RFC822)")
|
||||
if status != "OK":
|
||||
continue
|
||||
|
||||
raw_email = msg_data[0][1]
|
||||
msg = email_lib.message_from_bytes(raw_email)
|
||||
raw_email = msg_data[0][1]
|
||||
msg = email_lib.message_from_bytes(raw_email)
|
||||
|
||||
sender_raw = msg.get("From", "")
|
||||
sender_addr = _extract_email_address(sender_raw)
|
||||
sender_name = _decode_header_value(sender_raw)
|
||||
# Remove email from name if present
|
||||
if "<" in sender_name:
|
||||
sender_name = sender_name.split("<")[0].strip().strip('"')
|
||||
sender_raw = msg.get("From", "")
|
||||
sender_addr = _extract_email_address(sender_raw)
|
||||
sender_name = _decode_header_value(sender_raw)
|
||||
# Remove email from name if present
|
||||
if "<" in sender_name:
|
||||
sender_name = sender_name.split("<")[0].strip().strip('"')
|
||||
|
||||
subject = _decode_header_value(msg.get("Subject", "(no subject)"))
|
||||
message_id = msg.get("Message-ID", "")
|
||||
in_reply_to = msg.get("In-Reply-To", "")
|
||||
# Skip automated/noreply senders before any processing
|
||||
msg_headers = dict(msg.items())
|
||||
if _is_automated_sender(sender_addr, msg_headers):
|
||||
logger.debug("[Email] Skipping automated sender: %s", sender_addr)
|
||||
continue
|
||||
body = _extract_text_body(msg)
|
||||
attachments = _extract_attachments(msg, skip_attachments=self._skip_attachments)
|
||||
subject = _decode_header_value(msg.get("Subject", "(no subject)"))
|
||||
message_id = msg.get("Message-ID", "")
|
||||
in_reply_to = msg.get("In-Reply-To", "")
|
||||
# Skip automated/noreply senders before any processing
|
||||
msg_headers = dict(msg.items())
|
||||
if _is_automated_sender(sender_addr, msg_headers):
|
||||
logger.debug("[Email] Skipping automated sender: %s", sender_addr)
|
||||
continue
|
||||
body = _extract_text_body(msg)
|
||||
attachments = _extract_attachments(msg, skip_attachments=self._skip_attachments)
|
||||
|
||||
results.append({
|
||||
"uid": uid,
|
||||
"sender_addr": sender_addr,
|
||||
"sender_name": sender_name,
|
||||
"subject": subject,
|
||||
"message_id": message_id,
|
||||
"in_reply_to": in_reply_to,
|
||||
"body": body,
|
||||
"attachments": attachments,
|
||||
"date": msg.get("Date", ""),
|
||||
})
|
||||
finally:
|
||||
try:
|
||||
imap.logout()
|
||||
except Exception:
|
||||
pass
|
||||
results.append({
|
||||
"uid": uid,
|
||||
"sender_addr": sender_addr,
|
||||
"sender_name": sender_name,
|
||||
"subject": subject,
|
||||
"message_id": message_id,
|
||||
"in_reply_to": in_reply_to,
|
||||
"body": body,
|
||||
"attachments": attachments,
|
||||
"date": msg.get("Date", ""),
|
||||
})
|
||||
|
||||
imap.logout()
|
||||
except Exception as e:
|
||||
logger.error("[Email] IMAP fetch error: %s", e)
|
||||
return results
|
||||
@@ -506,15 +503,10 @@ class EmailAdapter(BasePlatformAdapter):
|
||||
msg.attach(MIMEText(body, "plain", "utf-8"))
|
||||
|
||||
smtp = smtplib.SMTP(self._smtp_host, self._smtp_port, timeout=30)
|
||||
try:
|
||||
smtp.starttls(context=ssl.create_default_context())
|
||||
smtp.login(self._address, self._password)
|
||||
smtp.send_message(msg)
|
||||
finally:
|
||||
try:
|
||||
smtp.quit()
|
||||
except Exception:
|
||||
smtp.close()
|
||||
smtp.starttls(context=ssl.create_default_context())
|
||||
smtp.login(self._address, self._password)
|
||||
smtp.send_message(msg)
|
||||
smtp.quit()
|
||||
|
||||
logger.info("[Email] Sent reply to %s (subject: %s)", to_addr, subject)
|
||||
return msg_id
|
||||
@@ -598,15 +590,10 @@ class EmailAdapter(BasePlatformAdapter):
|
||||
msg.attach(part)
|
||||
|
||||
smtp = smtplib.SMTP(self._smtp_host, self._smtp_port, timeout=30)
|
||||
try:
|
||||
smtp.starttls(context=ssl.create_default_context())
|
||||
smtp.login(self._address, self._password)
|
||||
smtp.send_message(msg)
|
||||
finally:
|
||||
try:
|
||||
smtp.quit()
|
||||
except Exception:
|
||||
smtp.close()
|
||||
smtp.starttls(context=ssl.create_default_context())
|
||||
smtp.login(self._address, self._password)
|
||||
smtp.send_message(msg)
|
||||
smtp.quit()
|
||||
|
||||
return msg_id
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -17,8 +17,6 @@ Environment variables:
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import mimetypes
|
||||
import os
|
||||
@@ -514,11 +512,8 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
"""Upload an audio file as a voice message (MSC3245 native voice)."""
|
||||
return await self._send_local_file(
|
||||
chat_id, audio_path, "m.audio", caption, reply_to,
|
||||
metadata=metadata, is_voice=True
|
||||
)
|
||||
"""Upload an audio file as a voice message."""
|
||||
return await self._send_local_file(chat_id, audio_path, "m.audio", caption, reply_to, metadata=metadata)
|
||||
|
||||
async def send_video(
|
||||
self,
|
||||
@@ -551,16 +546,13 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
is_voice: bool = False,
|
||||
) -> SendResult:
|
||||
"""Upload bytes to Matrix and send as a media message."""
|
||||
import nio
|
||||
|
||||
# Upload to homeserver.
|
||||
# nio expects a DataProvider (callable) or file-like object, not raw bytes.
|
||||
# nio.upload() returns a tuple (UploadResponse|UploadError, Optional[Dict])
|
||||
resp, maybe_encryption_info = await self._client.upload(
|
||||
io.BytesIO(data),
|
||||
resp = await self._client.upload(
|
||||
data,
|
||||
content_type=content_type,
|
||||
filename=filename,
|
||||
)
|
||||
@@ -582,10 +574,6 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
},
|
||||
}
|
||||
|
||||
# Add MSC3245 voice flag for native voice messages.
|
||||
if is_voice:
|
||||
msg_content["org.matrix.msc3245.voice"] = {}
|
||||
|
||||
if reply_to:
|
||||
msg_content["m.relates_to"] = {
|
||||
"m.in_reply_to": {"event_id": reply_to}
|
||||
@@ -613,7 +601,6 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
reply_to: Optional[str] = None,
|
||||
file_name: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
is_voice: bool = False,
|
||||
) -> SendResult:
|
||||
"""Read a local file and upload it."""
|
||||
p = Path(file_path)
|
||||
@@ -626,7 +613,7 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
ct = mimetypes.guess_type(fname)[0] or "application/octet-stream"
|
||||
data = p.read_bytes()
|
||||
|
||||
return await self._upload_and_send(room_id, data, fname, ct, msgtype, caption, reply_to, metadata, is_voice)
|
||||
return await self._upload_and_send(room_id, data, fname, ct, msgtype, caption, reply_to, metadata)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Sync loop
|
||||
@@ -821,19 +808,11 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
event_mimetype = (content_info.get("info") or {}).get("mimetype", "")
|
||||
media_type = "application/octet-stream"
|
||||
msg_type = MessageType.DOCUMENT
|
||||
is_voice_message = False
|
||||
|
||||
if isinstance(event, nio.RoomMessageImage):
|
||||
msg_type = MessageType.PHOTO
|
||||
media_type = event_mimetype or "image/png"
|
||||
elif isinstance(event, nio.RoomMessageAudio):
|
||||
# Check for MSC3245 voice flag: org.matrix.msc3245.voice: {}
|
||||
source_content = getattr(event, "source", {}).get("content", {})
|
||||
if source_content.get("org.matrix.msc3245.voice") is not None:
|
||||
is_voice_message = True
|
||||
msg_type = MessageType.VOICE
|
||||
else:
|
||||
msg_type = MessageType.AUDIO
|
||||
msg_type = MessageType.AUDIO
|
||||
media_type = event_mimetype or "audio/ogg"
|
||||
elif isinstance(event, nio.RoomMessageVideo):
|
||||
msg_type = MessageType.VIDEO
|
||||
@@ -871,31 +850,6 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
if relates_to.get("rel_type") == "m.thread":
|
||||
thread_id = relates_to.get("event_id")
|
||||
|
||||
# For voice messages, cache audio locally for transcription tools.
|
||||
# Use the authenticated nio client to download (Matrix requires auth for media).
|
||||
media_urls = [http_url] if http_url else None
|
||||
media_types = [media_type] if http_url else None
|
||||
|
||||
if is_voice_message and url and url.startswith("mxc://"):
|
||||
try:
|
||||
import nio
|
||||
from gateway.platforms.base import cache_audio_from_bytes
|
||||
|
||||
resp = await self._client.download(mxc=url)
|
||||
if isinstance(resp, nio.MemoryDownloadResponse):
|
||||
# Extract extension from mimetype or default to .ogg
|
||||
ext = ".ogg"
|
||||
if media_type and "/" in media_type:
|
||||
subtype = media_type.split("/")[1]
|
||||
ext = f".{subtype}" if subtype else ".ogg"
|
||||
local_path = cache_audio_from_bytes(resp.body, ext)
|
||||
media_urls = [local_path]
|
||||
logger.debug("Matrix: cached voice message to %s", local_path)
|
||||
else:
|
||||
logger.warning("Matrix: failed to download voice: %s", getattr(resp, "message", resp))
|
||||
except Exception as e:
|
||||
logger.warning("Matrix: failed to cache voice message, using HTTP URL: %s", e)
|
||||
|
||||
source = self.build_source(
|
||||
chat_id=room.room_id,
|
||||
chat_type=chat_type,
|
||||
@@ -904,9 +858,8 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
thread_id=thread_id,
|
||||
)
|
||||
|
||||
# Use cached local path for images (voice messages already handled above).
|
||||
if cached_path:
|
||||
media_urls = [cached_path]
|
||||
# Use cached local path for images, HTTP URL for other media types
|
||||
media_urls = [cached_path] if cached_path else ([http_url] if http_url else None)
|
||||
media_types = [media_type] if media_urls else None
|
||||
|
||||
msg_event = MessageEvent(
|
||||
|
||||
+35
-99
@@ -9,7 +9,6 @@ Uses slack-bolt (Python) with Socket Mode for:
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
@@ -74,10 +73,6 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
self._bot_user_id: Optional[str] = None
|
||||
self._user_name_cache: Dict[str, str] = {} # user_id → display name
|
||||
self._socket_mode_task: Optional[asyncio.Task] = None
|
||||
# Multi-workspace support
|
||||
self._team_clients: Dict[str, AsyncWebClient] = {} # team_id → WebClient
|
||||
self._team_bot_user_ids: Dict[str, str] = {} # team_id → bot_user_id
|
||||
self._channel_team: Dict[str, str] = {} # channel_id → team_id
|
||||
|
||||
async def connect(self) -> bool:
|
||||
"""Connect to Slack via Socket Mode."""
|
||||
@@ -87,38 +82,19 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
)
|
||||
return False
|
||||
|
||||
raw_token = self.config.token
|
||||
bot_token = self.config.token
|
||||
app_token = os.getenv("SLACK_APP_TOKEN")
|
||||
|
||||
if not raw_token:
|
||||
if not bot_token:
|
||||
logger.error("[Slack] SLACK_BOT_TOKEN not set")
|
||||
return False
|
||||
if not app_token:
|
||||
logger.error("[Slack] SLACK_APP_TOKEN not set")
|
||||
return False
|
||||
|
||||
# Support comma-separated bot tokens for multi-workspace
|
||||
bot_tokens = [t.strip() for t in raw_token.split(",") if t.strip()]
|
||||
|
||||
# Also load tokens from OAuth token file
|
||||
from hermes_constants import get_hermes_home
|
||||
tokens_file = get_hermes_home() / "slack_tokens.json"
|
||||
if tokens_file.exists():
|
||||
try:
|
||||
saved = json.loads(tokens_file.read_text(encoding="utf-8"))
|
||||
for team_id, entry in saved.items():
|
||||
tok = entry.get("token", "") if isinstance(entry, dict) else ""
|
||||
if tok and tok not in bot_tokens:
|
||||
bot_tokens.append(tok)
|
||||
team_label = entry.get("team_name", team_id) if isinstance(entry, dict) else team_id
|
||||
logger.info("[Slack] Loaded saved token for workspace %s", team_label)
|
||||
except Exception as e:
|
||||
logger.warning("[Slack] Failed to read %s: %s", tokens_file, e)
|
||||
|
||||
try:
|
||||
# Acquire scoped lock to prevent duplicate app token usage
|
||||
from gateway.status import acquire_scoped_lock
|
||||
self._token_lock_identity = app_token
|
||||
acquired, existing = acquire_scoped_lock('slack-app-token', app_token, metadata={'platform': 'slack'})
|
||||
if not acquired:
|
||||
owner_pid = existing.get('pid') if isinstance(existing, dict) else None
|
||||
@@ -127,30 +103,12 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
self._set_fatal_error('slack_token_lock', message, retryable=False)
|
||||
return False
|
||||
|
||||
# First token is the primary — used for AsyncApp / Socket Mode
|
||||
primary_token = bot_tokens[0]
|
||||
self._app = AsyncApp(token=primary_token)
|
||||
self._app = AsyncApp(token=bot_token)
|
||||
|
||||
# Register each bot token and map team_id → client
|
||||
for token in bot_tokens:
|
||||
client = AsyncWebClient(token=token)
|
||||
auth_response = await client.auth_test()
|
||||
team_id = auth_response.get("team_id", "")
|
||||
bot_user_id = auth_response.get("user_id", "")
|
||||
bot_name = auth_response.get("user", "unknown")
|
||||
team_name = auth_response.get("team", "unknown")
|
||||
|
||||
self._team_clients[team_id] = client
|
||||
self._team_bot_user_ids[team_id] = bot_user_id
|
||||
|
||||
# First token sets the primary bot_user_id (backward compat)
|
||||
if self._bot_user_id is None:
|
||||
self._bot_user_id = bot_user_id
|
||||
|
||||
logger.info(
|
||||
"[Slack] Authenticated as @%s in workspace %s (team: %s)",
|
||||
bot_name, team_name, team_id,
|
||||
)
|
||||
# Get our own bot user ID for mention detection
|
||||
auth_response = await self._app.client.auth_test()
|
||||
self._bot_user_id = auth_response.get("user_id")
|
||||
bot_name = auth_response.get("user", "unknown")
|
||||
|
||||
# Register message event handler
|
||||
@self._app.event("message")
|
||||
@@ -175,10 +133,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
self._socket_mode_task = asyncio.create_task(self._handler.start_async())
|
||||
|
||||
self._running = True
|
||||
logger.info(
|
||||
"[Slack] Socket Mode connected (%d workspace(s))",
|
||||
len(self._team_clients),
|
||||
)
|
||||
logger.info("[Slack] Connected as @%s (Socket Mode)", bot_name)
|
||||
return True
|
||||
|
||||
except Exception as e: # pragma: no cover - defensive logging
|
||||
@@ -194,24 +149,17 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
logger.warning("[Slack] Error while closing Socket Mode handler: %s", e, exc_info=True)
|
||||
self._running = False
|
||||
|
||||
# Release the token lock (use stored identity, not re-read env)
|
||||
# Release the token lock
|
||||
try:
|
||||
from gateway.status import release_scoped_lock
|
||||
if getattr(self, '_token_lock_identity', None):
|
||||
release_scoped_lock('slack-app-token', self._token_lock_identity)
|
||||
self._token_lock_identity = None
|
||||
app_token = os.getenv("SLACK_APP_TOKEN")
|
||||
if app_token:
|
||||
release_scoped_lock('slack-app-token', app_token)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
logger.info("[Slack] Disconnected")
|
||||
|
||||
def _get_client(self, chat_id: str) -> AsyncWebClient:
|
||||
"""Return the workspace-specific WebClient for a channel."""
|
||||
team_id = self._channel_team.get(chat_id)
|
||||
if team_id and team_id in self._team_clients:
|
||||
return self._team_clients[team_id]
|
||||
return self._app.client # fallback to primary
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
@@ -248,7 +196,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if broadcast and i == 0:
|
||||
kwargs["reply_broadcast"] = True
|
||||
|
||||
last_result = await self._get_client(chat_id).chat_postMessage(**kwargs)
|
||||
last_result = await self._app.client.chat_postMessage(**kwargs)
|
||||
|
||||
return SendResult(
|
||||
success=True,
|
||||
@@ -270,7 +218,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if not self._app:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
try:
|
||||
await self._get_client(chat_id).chat_update(
|
||||
await self._app.client.chat_update(
|
||||
channel=chat_id,
|
||||
ts=message_id,
|
||||
text=content,
|
||||
@@ -304,7 +252,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
return # Can only set status in a thread context
|
||||
|
||||
try:
|
||||
await self._get_client(chat_id).assistant_threads_setStatus(
|
||||
await self._app.client.assistant_threads_setStatus(
|
||||
channel_id=chat_id,
|
||||
thread_ts=thread_ts,
|
||||
status="is thinking...",
|
||||
@@ -346,7 +294,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if not os.path.exists(file_path):
|
||||
raise FileNotFoundError(f"File not found: {file_path}")
|
||||
|
||||
result = await self._get_client(chat_id).files_upload_v2(
|
||||
result = await self._app.client.files_upload_v2(
|
||||
channel=chat_id,
|
||||
file=file_path,
|
||||
filename=os.path.basename(file_path),
|
||||
@@ -448,7 +396,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if not self._app:
|
||||
return False
|
||||
try:
|
||||
await self._get_client(channel).reactions_add(
|
||||
await self._app.client.reactions_add(
|
||||
channel=channel, timestamp=timestamp, name=emoji
|
||||
)
|
||||
return True
|
||||
@@ -464,7 +412,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if not self._app:
|
||||
return False
|
||||
try:
|
||||
await self._get_client(channel).reactions_remove(
|
||||
await self._app.client.reactions_remove(
|
||||
channel=channel, timestamp=timestamp, name=emoji
|
||||
)
|
||||
return True
|
||||
@@ -474,7 +422,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
|
||||
# ----- User identity resolution -----
|
||||
|
||||
async def _resolve_user_name(self, user_id: str, chat_id: str = "") -> str:
|
||||
async def _resolve_user_name(self, user_id: str) -> str:
|
||||
"""Resolve a Slack user ID to a display name, with caching."""
|
||||
if not user_id:
|
||||
return ""
|
||||
@@ -485,8 +433,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
return user_id
|
||||
|
||||
try:
|
||||
client = self._get_client(chat_id) if chat_id else self._app.client
|
||||
result = await client.users_info(user=user_id)
|
||||
result = await self._app.client.users_info(user=user_id)
|
||||
user = result.get("user", {})
|
||||
# Prefer display_name → real_name → user_id
|
||||
profile = user.get("profile", {})
|
||||
@@ -550,7 +497,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
response = await client.get(image_url)
|
||||
response.raise_for_status()
|
||||
|
||||
result = await self._get_client(chat_id).files_upload_v2(
|
||||
result = await self._app.client.files_upload_v2(
|
||||
channel=chat_id,
|
||||
content=response.content,
|
||||
filename="image.png",
|
||||
@@ -610,7 +557,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error=f"Video file not found: {video_path}")
|
||||
|
||||
try:
|
||||
result = await self._get_client(chat_id).files_upload_v2(
|
||||
result = await self._app.client.files_upload_v2(
|
||||
channel=chat_id,
|
||||
file=video_path,
|
||||
filename=os.path.basename(video_path),
|
||||
@@ -651,7 +598,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
display_name = file_name or os.path.basename(file_path)
|
||||
|
||||
try:
|
||||
result = await self._get_client(chat_id).files_upload_v2(
|
||||
result = await self._app.client.files_upload_v2(
|
||||
channel=chat_id,
|
||||
file=file_path,
|
||||
filename=display_name,
|
||||
@@ -679,7 +626,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
return {"name": chat_id, "type": "unknown"}
|
||||
|
||||
try:
|
||||
result = await self._get_client(chat_id).conversations_info(channel=chat_id)
|
||||
result = await self._app.client.conversations_info(channel=chat_id)
|
||||
channel = result.get("channel", {})
|
||||
is_dm = channel.get("is_im", False)
|
||||
return {
|
||||
@@ -712,11 +659,6 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
user_id = event.get("user", "")
|
||||
channel_id = event.get("channel", "")
|
||||
ts = event.get("ts", "")
|
||||
team_id = event.get("team", "")
|
||||
|
||||
# Track which workspace owns this channel
|
||||
if team_id and channel_id:
|
||||
self._channel_team[channel_id] = team_id
|
||||
|
||||
# Determine if this is a DM or channel message
|
||||
channel_type = event.get("channel_type", "")
|
||||
@@ -733,12 +675,11 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
thread_ts = event.get("thread_ts") or ts # ts fallback for channels
|
||||
|
||||
# In channels, only respond if bot is mentioned
|
||||
bot_uid = self._team_bot_user_ids.get(team_id, self._bot_user_id)
|
||||
if not is_dm and bot_uid:
|
||||
if f"<@{bot_uid}>" not in text:
|
||||
if not is_dm and self._bot_user_id:
|
||||
if f"<@{self._bot_user_id}>" not in text:
|
||||
return
|
||||
# Strip the bot mention from the text
|
||||
text = text.replace(f"<@{bot_uid}>", "").strip()
|
||||
text = text.replace(f"<@{self._bot_user_id}>", "").strip()
|
||||
|
||||
# Determine message type
|
||||
msg_type = MessageType.TEXT
|
||||
@@ -758,7 +699,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if ext not in (".jpg", ".jpeg", ".png", ".gif", ".webp"):
|
||||
ext = ".jpg"
|
||||
# Slack private URLs require the bot token as auth header
|
||||
cached = await self._download_slack_file(url, ext, team_id=team_id)
|
||||
cached = await self._download_slack_file(url, ext)
|
||||
media_urls.append(cached)
|
||||
media_types.append(mimetype)
|
||||
msg_type = MessageType.PHOTO
|
||||
@@ -769,7 +710,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
ext = "." + mimetype.split("/")[-1].split(";")[0]
|
||||
if ext not in (".ogg", ".mp3", ".wav", ".webm", ".m4a"):
|
||||
ext = ".ogg"
|
||||
cached = await self._download_slack_file(url, ext, audio=True, team_id=team_id)
|
||||
cached = await self._download_slack_file(url, ext, audio=True)
|
||||
media_urls.append(cached)
|
||||
media_types.append(mimetype)
|
||||
msg_type = MessageType.VOICE
|
||||
@@ -800,7 +741,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
continue
|
||||
|
||||
# Download and cache
|
||||
raw_bytes = await self._download_slack_file_bytes(url, team_id=team_id)
|
||||
raw_bytes = await self._download_slack_file_bytes(url)
|
||||
cached_path = cache_document_from_bytes(
|
||||
raw_bytes, original_filename or f"document{ext}"
|
||||
)
|
||||
@@ -829,7 +770,7 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
logger.warning("[Slack] Failed to cache document from %s: %s", url, e, exc_info=True)
|
||||
|
||||
# Resolve user display name (cached after first lookup)
|
||||
user_name = await self._resolve_user_name(user_id, chat_id=channel_id)
|
||||
user_name = await self._resolve_user_name(user_id)
|
||||
|
||||
# Build source
|
||||
source = self.build_source(
|
||||
@@ -866,11 +807,6 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
text = command.get("text", "").strip()
|
||||
user_id = command.get("user_id", "")
|
||||
channel_id = command.get("channel_id", "")
|
||||
team_id = command.get("team_id", "")
|
||||
|
||||
# Track which workspace owns this channel
|
||||
if team_id and channel_id:
|
||||
self._channel_team[channel_id] = team_id
|
||||
|
||||
# Map subcommands to gateway commands — derived from central registry.
|
||||
# Also keep "compact" as a Slack-specific alias for /compress.
|
||||
@@ -902,12 +838,12 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
|
||||
await self.handle_message(event)
|
||||
|
||||
async def _download_slack_file(self, url: str, ext: str, audio: bool = False, team_id: str = "") -> str:
|
||||
async def _download_slack_file(self, url: str, ext: str, audio: bool = False) -> str:
|
||||
"""Download a Slack file using the bot token for auth, with retry."""
|
||||
import asyncio
|
||||
import httpx
|
||||
|
||||
bot_token = self._team_clients[team_id].token if team_id and team_id in self._team_clients else self.config.token
|
||||
bot_token = self.config.token
|
||||
last_exc = None
|
||||
|
||||
async with httpx.AsyncClient(timeout=30.0, follow_redirects=True) as client:
|
||||
@@ -937,12 +873,12 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
raise
|
||||
raise last_exc
|
||||
|
||||
async def _download_slack_file_bytes(self, url: str, team_id: str = "") -> bytes:
|
||||
async def _download_slack_file_bytes(self, url: str) -> bytes:
|
||||
"""Download a Slack file and return raw bytes, with retry."""
|
||||
import asyncio
|
||||
import httpx
|
||||
|
||||
bot_token = self._team_clients[team_id].token if team_id and team_id in self._team_clients else self.config.token
|
||||
bot_token = self.config.token
|
||||
last_exc = None
|
||||
|
||||
async with httpx.AsyncClient(timeout=30.0, follow_redirects=True) as client:
|
||||
|
||||
+22
-229
@@ -8,7 +8,6 @@ Uses python-telegram-bot library for:
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
@@ -123,8 +122,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
super().__init__(config, Platform.TELEGRAM)
|
||||
self._app: Optional[Application] = None
|
||||
self._bot: Optional[Bot] = None
|
||||
self._webhook_mode: bool = False
|
||||
self._mention_patterns = self._compile_mention_patterns()
|
||||
self._reply_to_mode: str = getattr(config, 'reply_to_mode', 'first') or 'first'
|
||||
# Buffer rapid/album photo updates so Telegram image bursts are handled
|
||||
# as a single MessageEvent instead of self-interrupting multiple turns.
|
||||
@@ -459,19 +456,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
self._persist_dm_topic_thread_id(int(chat_id), topic_name, thread_id)
|
||||
|
||||
async def connect(self) -> bool:
|
||||
"""Connect to Telegram via polling or webhook.
|
||||
|
||||
By default, uses long polling (outbound connection to Telegram).
|
||||
If ``TELEGRAM_WEBHOOK_URL`` is set, starts an HTTP webhook server
|
||||
instead. Webhook mode is useful for cloud deployments (Fly.io,
|
||||
Railway) where inbound HTTP can wake a suspended machine.
|
||||
|
||||
Env vars for webhook mode::
|
||||
|
||||
TELEGRAM_WEBHOOK_URL Public HTTPS URL (e.g. https://app.fly.dev/telegram)
|
||||
TELEGRAM_WEBHOOK_PORT Local listen port (default 8443)
|
||||
TELEGRAM_WEBHOOK_SECRET Secret token for update verification
|
||||
"""
|
||||
"""Connect to Telegram and start polling for updates."""
|
||||
if not TELEGRAM_AVAILABLE:
|
||||
logger.error(
|
||||
"[%s] python-telegram-bot not installed. Run: pip install python-telegram-bot",
|
||||
@@ -565,57 +550,27 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
else:
|
||||
raise
|
||||
await self._app.start()
|
||||
loop = asyncio.get_running_loop()
|
||||
|
||||
# Decide between webhook and polling mode
|
||||
webhook_url = os.getenv("TELEGRAM_WEBHOOK_URL", "").strip()
|
||||
def _polling_error_callback(error: Exception) -> None:
|
||||
if self._polling_error_task and not self._polling_error_task.done():
|
||||
return
|
||||
if self._looks_like_polling_conflict(error):
|
||||
self._polling_error_task = loop.create_task(self._handle_polling_conflict(error))
|
||||
elif self._looks_like_network_error(error):
|
||||
logger.warning("[%s] Telegram network error, scheduling reconnect: %s", self.name, error)
|
||||
self._polling_error_task = loop.create_task(self._handle_polling_network_error(error))
|
||||
else:
|
||||
logger.error("[%s] Telegram polling error: %s", self.name, error, exc_info=True)
|
||||
|
||||
if webhook_url:
|
||||
# ── Webhook mode ─────────────────────────────────────
|
||||
# Telegram pushes updates to our HTTP endpoint. This
|
||||
# enables cloud platforms (Fly.io, Railway) to auto-wake
|
||||
# suspended machines on inbound HTTP traffic.
|
||||
webhook_port = int(os.getenv("TELEGRAM_WEBHOOK_PORT", "8443"))
|
||||
webhook_secret = os.getenv("TELEGRAM_WEBHOOK_SECRET", "").strip() or None
|
||||
from urllib.parse import urlparse
|
||||
webhook_path = urlparse(webhook_url).path or "/telegram"
|
||||
# Store reference for retry use in _handle_polling_conflict
|
||||
self._polling_error_callback_ref = _polling_error_callback
|
||||
|
||||
await self._app.updater.start_webhook(
|
||||
listen="0.0.0.0",
|
||||
port=webhook_port,
|
||||
url_path=webhook_path,
|
||||
webhook_url=webhook_url,
|
||||
secret_token=webhook_secret,
|
||||
allowed_updates=Update.ALL_TYPES,
|
||||
drop_pending_updates=True,
|
||||
)
|
||||
self._webhook_mode = True
|
||||
logger.info(
|
||||
"[%s] Webhook server listening on 0.0.0.0:%d%s",
|
||||
self.name, webhook_port, webhook_path,
|
||||
)
|
||||
else:
|
||||
# ── Polling mode (default) ───────────────────────────
|
||||
loop = asyncio.get_running_loop()
|
||||
|
||||
def _polling_error_callback(error: Exception) -> None:
|
||||
if self._polling_error_task and not self._polling_error_task.done():
|
||||
return
|
||||
if self._looks_like_polling_conflict(error):
|
||||
self._polling_error_task = loop.create_task(self._handle_polling_conflict(error))
|
||||
elif self._looks_like_network_error(error):
|
||||
logger.warning("[%s] Telegram network error, scheduling reconnect: %s", self.name, error)
|
||||
self._polling_error_task = loop.create_task(self._handle_polling_network_error(error))
|
||||
else:
|
||||
logger.error("[%s] Telegram polling error: %s", self.name, error, exc_info=True)
|
||||
|
||||
# Store reference for retry use in _handle_polling_conflict
|
||||
self._polling_error_callback_ref = _polling_error_callback
|
||||
|
||||
await self._app.updater.start_polling(
|
||||
allowed_updates=Update.ALL_TYPES,
|
||||
drop_pending_updates=True,
|
||||
error_callback=_polling_error_callback,
|
||||
)
|
||||
await self._app.updater.start_polling(
|
||||
allowed_updates=Update.ALL_TYPES,
|
||||
drop_pending_updates=True,
|
||||
error_callback=_polling_error_callback,
|
||||
)
|
||||
|
||||
# Register bot commands so Telegram shows a hint menu when users type /
|
||||
# List is derived from the central COMMAND_REGISTRY — adding a new
|
||||
@@ -635,8 +590,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
)
|
||||
|
||||
self._mark_connected()
|
||||
mode = "webhook" if self._webhook_mode else "polling"
|
||||
logger.info("[%s] Connected to Telegram (%s mode)", self.name, mode)
|
||||
logger.info("[%s] Connected and polling for Telegram updates", self.name)
|
||||
|
||||
# Set up DM topics (Bot API 9.4 — Private Chat Topics)
|
||||
# Runs after connection is established so the bot can call createForumTopic.
|
||||
@@ -664,7 +618,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
return False
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
"""Stop polling/webhook, cancel pending album flushes, and disconnect."""
|
||||
"""Stop polling, cancel pending album flushes, and disconnect."""
|
||||
pending_media_group_tasks = list(self._media_group_tasks.values())
|
||||
for task in pending_media_group_tasks:
|
||||
task.cancel()
|
||||
@@ -808,16 +762,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
)
|
||||
effective_thread_id = None
|
||||
continue
|
||||
if "message to be replied not found" in err_lower and reply_to_id is not None:
|
||||
# Original message was deleted before we
|
||||
# could reply — clear reply target and retry
|
||||
# so the response is still delivered.
|
||||
logger.warning(
|
||||
"[%s] Reply target deleted, retrying without reply_to: %s",
|
||||
self.name, send_err,
|
||||
)
|
||||
reply_to_id = None
|
||||
continue
|
||||
# Other BadRequest errors are permanent — don't retry
|
||||
raise
|
||||
if _send_attempt < 2:
|
||||
@@ -1371,148 +1315,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
|
||||
return text
|
||||
|
||||
# ── Group mention gating ──────────────────────────────────────────────
|
||||
|
||||
def _telegram_require_mention(self) -> bool:
|
||||
"""Return whether group chats should require an explicit bot trigger."""
|
||||
configured = self.config.extra.get("require_mention")
|
||||
if configured is not None:
|
||||
if isinstance(configured, str):
|
||||
return configured.lower() in ("true", "1", "yes", "on")
|
||||
return bool(configured)
|
||||
return os.getenv("TELEGRAM_REQUIRE_MENTION", "false").lower() in ("true", "1", "yes", "on")
|
||||
|
||||
def _telegram_free_response_chats(self) -> set[str]:
|
||||
raw = self.config.extra.get("free_response_chats")
|
||||
if raw is None:
|
||||
raw = os.getenv("TELEGRAM_FREE_RESPONSE_CHATS", "")
|
||||
if isinstance(raw, list):
|
||||
return {str(part).strip() for part in raw if str(part).strip()}
|
||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
||||
|
||||
def _compile_mention_patterns(self) -> List[re.Pattern]:
|
||||
"""Compile optional regex wake-word patterns for group triggers."""
|
||||
patterns = self.config.extra.get("mention_patterns")
|
||||
if patterns is None:
|
||||
raw = os.getenv("TELEGRAM_MENTION_PATTERNS", "").strip()
|
||||
if raw:
|
||||
try:
|
||||
loaded = json.loads(raw)
|
||||
except Exception:
|
||||
loaded = [part.strip() for part in raw.splitlines() if part.strip()]
|
||||
if not loaded:
|
||||
loaded = [part.strip() for part in raw.split(",") if part.strip()]
|
||||
patterns = loaded
|
||||
|
||||
if patterns is None:
|
||||
return []
|
||||
if isinstance(patterns, str):
|
||||
patterns = [patterns]
|
||||
if not isinstance(patterns, list):
|
||||
logger.warning(
|
||||
"[%s] telegram mention_patterns must be a list or string; got %s",
|
||||
self.name,
|
||||
type(patterns).__name__,
|
||||
)
|
||||
return []
|
||||
|
||||
compiled: List[re.Pattern] = []
|
||||
for pattern in patterns:
|
||||
if not isinstance(pattern, str) or not pattern.strip():
|
||||
continue
|
||||
try:
|
||||
compiled.append(re.compile(pattern, re.IGNORECASE))
|
||||
except re.error as exc:
|
||||
logger.warning("[%s] Invalid Telegram mention pattern %r: %s", self.name, pattern, exc)
|
||||
if compiled:
|
||||
logger.info("[%s] Loaded %d Telegram mention pattern(s)", self.name, len(compiled))
|
||||
return compiled
|
||||
|
||||
def _is_group_chat(self, message: Message) -> bool:
|
||||
chat = getattr(message, "chat", None)
|
||||
if not chat:
|
||||
return False
|
||||
chat_type = str(getattr(chat, "type", "")).split(".")[-1].lower()
|
||||
return chat_type in ("group", "supergroup")
|
||||
|
||||
def _is_reply_to_bot(self, message: Message) -> bool:
|
||||
if not self._bot or not getattr(message, "reply_to_message", None):
|
||||
return False
|
||||
reply_user = getattr(message.reply_to_message, "from_user", None)
|
||||
return bool(reply_user and getattr(reply_user, "id", None) == getattr(self._bot, "id", None))
|
||||
|
||||
def _message_mentions_bot(self, message: Message) -> bool:
|
||||
if not self._bot:
|
||||
return False
|
||||
|
||||
bot_username = (getattr(self._bot, "username", None) or "").lstrip("@").lower()
|
||||
bot_id = getattr(self._bot, "id", None)
|
||||
|
||||
def _iter_sources():
|
||||
yield getattr(message, "text", None) or "", getattr(message, "entities", None) or []
|
||||
yield getattr(message, "caption", None) or "", getattr(message, "caption_entities", None) or []
|
||||
|
||||
for source_text, entities in _iter_sources():
|
||||
if bot_username and f"@{bot_username}" in source_text.lower():
|
||||
return True
|
||||
for entity in entities:
|
||||
entity_type = str(getattr(entity, "type", "")).split(".")[-1].lower()
|
||||
if entity_type == "mention" and bot_username:
|
||||
offset = int(getattr(entity, "offset", -1))
|
||||
length = int(getattr(entity, "length", 0))
|
||||
if offset < 0 or length <= 0:
|
||||
continue
|
||||
if source_text[offset:offset + length].strip().lower() == f"@{bot_username}":
|
||||
return True
|
||||
elif entity_type == "text_mention":
|
||||
user = getattr(entity, "user", None)
|
||||
if user and getattr(user, "id", None) == bot_id:
|
||||
return True
|
||||
return False
|
||||
|
||||
def _message_matches_mention_patterns(self, message: Message) -> bool:
|
||||
if not self._mention_patterns:
|
||||
return False
|
||||
for candidate in (getattr(message, "text", None), getattr(message, "caption", None)):
|
||||
if not candidate:
|
||||
continue
|
||||
for pattern in self._mention_patterns:
|
||||
if pattern.search(candidate):
|
||||
return True
|
||||
return False
|
||||
|
||||
def _clean_bot_trigger_text(self, text: Optional[str]) -> Optional[str]:
|
||||
if not text or not self._bot or not getattr(self._bot, "username", None):
|
||||
return text
|
||||
username = re.escape(self._bot.username)
|
||||
cleaned = re.sub(rf"(?i)@{username}\b[,:\-]*\s*", "", text).strip()
|
||||
return cleaned or text
|
||||
|
||||
def _should_process_message(self, message: Message, *, is_command: bool = False) -> bool:
|
||||
"""Apply Telegram group trigger rules.
|
||||
|
||||
DMs remain unrestricted. Group/supergroup messages are accepted when:
|
||||
- the chat is explicitly allowlisted in ``free_response_chats``
|
||||
- ``require_mention`` is disabled
|
||||
- the message is a command
|
||||
- the message replies to the bot
|
||||
- the bot is @mentioned
|
||||
- the text/caption matches a configured regex wake-word pattern
|
||||
"""
|
||||
if not self._is_group_chat(message):
|
||||
return True
|
||||
if str(getattr(getattr(message, "chat", None), "id", "")) in self._telegram_free_response_chats():
|
||||
return True
|
||||
if not self._telegram_require_mention():
|
||||
return True
|
||||
if is_command:
|
||||
return True
|
||||
if self._is_reply_to_bot(message):
|
||||
return True
|
||||
if self._message_mentions_bot(message):
|
||||
return True
|
||||
return self._message_matches_mention_patterns(message)
|
||||
|
||||
async def _handle_text_message(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
||||
"""Handle incoming text messages.
|
||||
|
||||
@@ -1522,19 +1324,14 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
"""
|
||||
if not update.message or not update.message.text:
|
||||
return
|
||||
if not self._should_process_message(update.message):
|
||||
return
|
||||
|
||||
event = self._build_message_event(update.message, MessageType.TEXT)
|
||||
event.text = self._clean_bot_trigger_text(event.text)
|
||||
self._enqueue_text_event(event)
|
||||
|
||||
async def _handle_command(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
||||
"""Handle incoming command messages."""
|
||||
if not update.message or not update.message.text:
|
||||
return
|
||||
if not self._should_process_message(update.message, is_command=True):
|
||||
return
|
||||
|
||||
event = self._build_message_event(update.message, MessageType.COMMAND)
|
||||
await self.handle_message(event)
|
||||
@@ -1543,8 +1340,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
"""Handle incoming location/venue pin messages."""
|
||||
if not update.message:
|
||||
return
|
||||
if not self._should_process_message(update.message):
|
||||
return
|
||||
|
||||
msg = update.message
|
||||
venue = getattr(msg, "venue", None)
|
||||
@@ -1688,8 +1483,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
"""Handle incoming media messages, downloading images to local cache."""
|
||||
if not update.message:
|
||||
return
|
||||
if not self._should_process_message(update.message):
|
||||
return
|
||||
|
||||
msg = update.message
|
||||
|
||||
@@ -1713,7 +1506,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
|
||||
# Add caption as text
|
||||
if msg.caption:
|
||||
event.text = self._clean_bot_trigger_text(msg.caption)
|
||||
event.text = msg.caption
|
||||
|
||||
# Handle stickers: describe via vision tool with caching
|
||||
if msg.sticker:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+103
-108
@@ -142,7 +142,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
self._bridge_log_fh = None
|
||||
self._bridge_log: Optional[Path] = None
|
||||
self._poll_task: Optional[asyncio.Task] = None
|
||||
self._http_session: Optional["aiohttp.ClientSession"] = None
|
||||
self._session_lock_identity: Optional[str] = None
|
||||
|
||||
async def connect(self) -> bool:
|
||||
@@ -225,7 +224,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
print(f"[{self.name}] Using existing bridge (status: {bridge_status})")
|
||||
self._mark_connected()
|
||||
self._bridge_process = None # Not managed by us
|
||||
self._http_session = aiohttp.ClientSession()
|
||||
self._poll_task = asyncio.create_task(self._poll_messages())
|
||||
return True
|
||||
else:
|
||||
@@ -331,9 +329,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
print(f"[{self.name}] Bridge log: {self._bridge_log}")
|
||||
print(f"[{self.name}] If session expired, re-pair: hermes whatsapp")
|
||||
|
||||
# Create a persistent HTTP session for all bridge communication
|
||||
self._http_session = aiohttp.ClientSession()
|
||||
|
||||
# Start message polling task
|
||||
self._poll_task = asyncio.create_task(self._poll_messages())
|
||||
|
||||
@@ -405,21 +400,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
else:
|
||||
# Bridge was not started by us, don't kill it
|
||||
print(f"[{self.name}] Disconnecting (external bridge left running)")
|
||||
|
||||
# Cancel the poll task explicitly
|
||||
if self._poll_task and not self._poll_task.done():
|
||||
self._poll_task.cancel()
|
||||
try:
|
||||
await self._poll_task
|
||||
except (asyncio.CancelledError, Exception):
|
||||
pass
|
||||
self._poll_task = None
|
||||
|
||||
# Close the persistent HTTP session
|
||||
if self._http_session and not self._http_session.closed:
|
||||
await self._http_session.close()
|
||||
self._http_session = None
|
||||
|
||||
|
||||
if self._session_lock_identity:
|
||||
try:
|
||||
from gateway.status import release_scoped_lock
|
||||
@@ -441,7 +422,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
metadata: Optional[Dict[str, Any]] = None
|
||||
) -> SendResult:
|
||||
"""Send a message via the WhatsApp bridge."""
|
||||
if not self._running or not self._http_session:
|
||||
if not self._running:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
bridge_exit = await self._check_managed_bridge_exit()
|
||||
if bridge_exit:
|
||||
@@ -449,29 +430,36 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
|
||||
try:
|
||||
import aiohttp
|
||||
|
||||
payload = {
|
||||
"chatId": chat_id,
|
||||
"message": content,
|
||||
}
|
||||
if reply_to:
|
||||
payload["replyTo"] = reply_to
|
||||
|
||||
async with self._http_session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/send",
|
||||
json=payload,
|
||||
timeout=aiohttp.ClientTimeout(total=30)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
data = await resp.json()
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=data.get("messageId"),
|
||||
raw_response=data
|
||||
)
|
||||
else:
|
||||
error = await resp.text()
|
||||
return SendResult(success=False, error=error)
|
||||
async with aiohttp.ClientSession() as session:
|
||||
payload = {
|
||||
"chatId": chat_id,
|
||||
"message": content,
|
||||
}
|
||||
if reply_to:
|
||||
payload["replyTo"] = reply_to
|
||||
|
||||
async with session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/send",
|
||||
json=payload,
|
||||
timeout=aiohttp.ClientTimeout(total=30)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
data = await resp.json()
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=data.get("messageId"),
|
||||
raw_response=data
|
||||
)
|
||||
else:
|
||||
error = await resp.text()
|
||||
return SendResult(success=False, error=error)
|
||||
|
||||
except ImportError:
|
||||
return SendResult(
|
||||
success=False,
|
||||
error="aiohttp not installed. Run: pip install aiohttp"
|
||||
)
|
||||
except Exception as e:
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
@@ -482,27 +470,28 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
content: str,
|
||||
) -> SendResult:
|
||||
"""Edit a previously sent message via the WhatsApp bridge."""
|
||||
if not self._running or not self._http_session:
|
||||
if not self._running:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
bridge_exit = await self._check_managed_bridge_exit()
|
||||
if bridge_exit:
|
||||
return SendResult(success=False, error=bridge_exit)
|
||||
try:
|
||||
import aiohttp
|
||||
async with self._http_session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/edit",
|
||||
json={
|
||||
"chatId": chat_id,
|
||||
"messageId": message_id,
|
||||
"message": content,
|
||||
},
|
||||
timeout=aiohttp.ClientTimeout(total=15)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
else:
|
||||
error = await resp.text()
|
||||
return SendResult(success=False, error=error)
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/edit",
|
||||
json={
|
||||
"chatId": chat_id,
|
||||
"messageId": message_id,
|
||||
"message": content,
|
||||
},
|
||||
timeout=aiohttp.ClientTimeout(total=15)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
else:
|
||||
error = await resp.text()
|
||||
return SendResult(success=False, error=error)
|
||||
except Exception as e:
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
@@ -515,7 +504,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
file_name: Optional[str] = None,
|
||||
) -> SendResult:
|
||||
"""Send any media file via bridge /send-media endpoint."""
|
||||
if not self._running or not self._http_session:
|
||||
if not self._running:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
bridge_exit = await self._check_managed_bridge_exit()
|
||||
if bridge_exit:
|
||||
@@ -536,21 +525,22 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
if file_name:
|
||||
payload["fileName"] = file_name
|
||||
|
||||
async with self._http_session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/send-media",
|
||||
json=payload,
|
||||
timeout=aiohttp.ClientTimeout(total=120),
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
data = await resp.json()
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=data.get("messageId"),
|
||||
raw_response=data,
|
||||
)
|
||||
else:
|
||||
error = await resp.text()
|
||||
return SendResult(success=False, error=error)
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/send-media",
|
||||
json=payload,
|
||||
timeout=aiohttp.ClientTimeout(total=120),
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
data = await resp.json()
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=data.get("messageId"),
|
||||
raw_response=data,
|
||||
)
|
||||
else:
|
||||
error = await resp.text()
|
||||
return SendResult(success=False, error=error)
|
||||
|
||||
except Exception as e:
|
||||
return SendResult(success=False, error=str(e))
|
||||
@@ -608,43 +598,45 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
|
||||
async def send_typing(self, chat_id: str, metadata=None) -> None:
|
||||
"""Send typing indicator via bridge."""
|
||||
if not self._running or not self._http_session:
|
||||
if not self._running:
|
||||
return
|
||||
if await self._check_managed_bridge_exit():
|
||||
return
|
||||
|
||||
try:
|
||||
import aiohttp
|
||||
|
||||
await self._http_session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/typing",
|
||||
json={"chatId": chat_id},
|
||||
timeout=aiohttp.ClientTimeout(total=5)
|
||||
)
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
await session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/typing",
|
||||
json={"chatId": chat_id},
|
||||
timeout=aiohttp.ClientTimeout(total=5)
|
||||
)
|
||||
except Exception:
|
||||
pass # Ignore typing indicator failures
|
||||
|
||||
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||
"""Get information about a WhatsApp chat."""
|
||||
if not self._running or not self._http_session:
|
||||
if not self._running:
|
||||
return {"name": "Unknown", "type": "dm"}
|
||||
if await self._check_managed_bridge_exit():
|
||||
return {"name": chat_id, "type": "dm"}
|
||||
|
||||
try:
|
||||
import aiohttp
|
||||
|
||||
async with self._http_session.get(
|
||||
f"http://127.0.0.1:{self._bridge_port}/chat/{chat_id}",
|
||||
timeout=aiohttp.ClientTimeout(total=10)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
data = await resp.json()
|
||||
return {
|
||||
"name": data.get("name", chat_id),
|
||||
"type": "group" if data.get("isGroup") else "dm",
|
||||
"participants": data.get("participants", []),
|
||||
}
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.get(
|
||||
f"http://127.0.0.1:{self._bridge_port}/chat/{chat_id}",
|
||||
timeout=aiohttp.ClientTimeout(total=10)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
data = await resp.json()
|
||||
return {
|
||||
"name": data.get("name", chat_id),
|
||||
"type": "group" if data.get("isGroup") else "dm",
|
||||
"participants": data.get("participants", []),
|
||||
}
|
||||
except Exception as e:
|
||||
logger.debug("Could not get WhatsApp chat info for %s: %s", chat_id, e)
|
||||
|
||||
@@ -652,26 +644,29 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
|
||||
async def _poll_messages(self) -> None:
|
||||
"""Poll the bridge for incoming messages."""
|
||||
import aiohttp
|
||||
|
||||
try:
|
||||
import aiohttp
|
||||
except ImportError:
|
||||
print(f"[{self.name}] aiohttp not installed, message polling disabled")
|
||||
return
|
||||
|
||||
while self._running:
|
||||
if not self._http_session:
|
||||
break
|
||||
bridge_exit = await self._check_managed_bridge_exit()
|
||||
if bridge_exit:
|
||||
print(f"[{self.name}] {bridge_exit}")
|
||||
break
|
||||
try:
|
||||
async with self._http_session.get(
|
||||
f"http://127.0.0.1:{self._bridge_port}/messages",
|
||||
timeout=aiohttp.ClientTimeout(total=30)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
messages = await resp.json()
|
||||
for msg_data in messages:
|
||||
event = await self._build_message_event(msg_data)
|
||||
if event:
|
||||
await self.handle_message(event)
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.get(
|
||||
f"http://127.0.0.1:{self._bridge_port}/messages",
|
||||
timeout=aiohttp.ClientTimeout(total=30)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
messages = await resp.json()
|
||||
for msg_data in messages:
|
||||
event = await self._build_message_event(msg_data)
|
||||
if event:
|
||||
await self.handle_message(event)
|
||||
except asyncio.CancelledError:
|
||||
break
|
||||
except Exception as e:
|
||||
|
||||
+25
-110
@@ -77,7 +77,6 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
# Resolve Hermes home directory (respects HERMES_HOME override)
|
||||
from hermes_constants import get_hermes_home
|
||||
from utils import atomic_yaml_write
|
||||
_hermes_home = get_hermes_home()
|
||||
|
||||
# Load environment variables from ~/.hermes/.env first.
|
||||
@@ -225,49 +224,6 @@ from gateway.session import (
|
||||
from gateway.delivery import DeliveryRouter
|
||||
from gateway.platforms.base import BasePlatformAdapter, MessageEvent, MessageType
|
||||
|
||||
|
||||
def _normalize_whatsapp_identifier(value: str) -> str:
|
||||
"""Strip WhatsApp JID/LID syntax down to its stable numeric identifier."""
|
||||
return (
|
||||
str(value or "")
|
||||
.strip()
|
||||
.replace("+", "", 1)
|
||||
.split(":", 1)[0]
|
||||
.split("@", 1)[0]
|
||||
)
|
||||
|
||||
|
||||
def _expand_whatsapp_auth_aliases(identifier: str) -> set:
|
||||
"""Resolve WhatsApp phone/LID aliases using bridge session mapping files."""
|
||||
normalized = _normalize_whatsapp_identifier(identifier)
|
||||
if not normalized:
|
||||
return set()
|
||||
|
||||
session_dir = _hermes_home / "whatsapp" / "session"
|
||||
resolved = set()
|
||||
queue = [normalized]
|
||||
|
||||
while queue:
|
||||
current = queue.pop(0)
|
||||
if not current or current in resolved:
|
||||
continue
|
||||
|
||||
resolved.add(current)
|
||||
for suffix in ("", "_reverse"):
|
||||
mapping_path = session_dir / f"lid-mapping-{current}{suffix}.json"
|
||||
if not mapping_path.exists():
|
||||
continue
|
||||
try:
|
||||
mapped = _normalize_whatsapp_identifier(
|
||||
json.loads(mapping_path.read_text(encoding="utf-8"))
|
||||
)
|
||||
except Exception:
|
||||
continue
|
||||
if mapped and mapped not in resolved:
|
||||
queue.append(mapped)
|
||||
|
||||
return resolved
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Sentinel placed into _running_agents immediately when a session starts
|
||||
@@ -323,10 +279,10 @@ def _resolve_gateway_model(config: dict | None = None) -> str:
|
||||
"""Read model from env/config — mirrors the resolution in _run_agent_sync.
|
||||
|
||||
Without this, temporary AIAgent instances (memory flush, /compress) fall
|
||||
back to the hardcoded default which fails when the active provider is
|
||||
openai-codex.
|
||||
back to the hardcoded default ("anthropic/claude-opus-4.6") which fails
|
||||
when the active provider is openai-codex.
|
||||
"""
|
||||
model = os.getenv("HERMES_MODEL") or os.getenv("LLM_MODEL") or ""
|
||||
model = os.getenv("HERMES_MODEL") or os.getenv("LLM_MODEL") or "anthropic/claude-opus-4.6"
|
||||
cfg = config if config is not None else _load_gateway_config()
|
||||
model_cfg = cfg.get("model", {})
|
||||
if isinstance(model_cfg, str):
|
||||
@@ -962,12 +918,11 @@ class GatewayRunner:
|
||||
return {}
|
||||
|
||||
@staticmethod
|
||||
def _load_fallback_model() -> list | dict | None:
|
||||
"""Load fallback provider chain from config.yaml.
|
||||
def _load_fallback_model() -> dict | None:
|
||||
"""Load fallback model config from config.yaml.
|
||||
|
||||
Returns a list of provider dicts (``fallback_providers``), a single
|
||||
dict (legacy ``fallback_model``), or None if not configured.
|
||||
AIAgent.__init__ normalizes both formats into a chain.
|
||||
Returns a dict with 'provider' and 'model' keys, or None if
|
||||
not configured / both fields empty.
|
||||
"""
|
||||
try:
|
||||
import yaml as _y
|
||||
@@ -975,8 +930,8 @@ class GatewayRunner:
|
||||
if cfg_path.exists():
|
||||
with open(cfg_path, encoding="utf-8") as _f:
|
||||
cfg = _y.safe_load(_f) or {}
|
||||
fb = cfg.get("fallback_providers") or cfg.get("fallback_model") or None
|
||||
if fb:
|
||||
fb = cfg.get("fallback_model", {}) or {}
|
||||
if fb.get("provider") and fb.get("model"):
|
||||
return fb
|
||||
except Exception:
|
||||
pass
|
||||
@@ -1026,8 +981,6 @@ class GatewayRunner:
|
||||
"EMAIL_ALLOWED_USERS",
|
||||
"SMS_ALLOWED_USERS", "MATTERMOST_ALLOWED_USERS",
|
||||
"MATRIX_ALLOWED_USERS", "DINGTALK_ALLOWED_USERS",
|
||||
"FEISHU_ALLOWED_USERS",
|
||||
"WECOM_ALLOWED_USERS",
|
||||
"GATEWAY_ALLOWED_USERS")
|
||||
)
|
||||
_allow_all = os.getenv("GATEWAY_ALLOW_ALL_USERS", "").lower() in ("true", "1", "yes") or any(
|
||||
@@ -1036,9 +989,7 @@ class GatewayRunner:
|
||||
"WHATSAPP_ALLOW_ALL_USERS", "SLACK_ALLOW_ALL_USERS",
|
||||
"SIGNAL_ALLOW_ALL_USERS", "EMAIL_ALLOW_ALL_USERS",
|
||||
"SMS_ALLOW_ALL_USERS", "MATTERMOST_ALLOW_ALL_USERS",
|
||||
"MATRIX_ALLOW_ALL_USERS", "DINGTALK_ALLOW_ALL_USERS",
|
||||
"FEISHU_ALLOW_ALL_USERS",
|
||||
"WECOM_ALLOW_ALL_USERS")
|
||||
"MATRIX_ALLOW_ALL_USERS", "DINGTALK_ALLOW_ALL_USERS")
|
||||
)
|
||||
if not _any_allowlist and not _allow_all:
|
||||
logger.warning(
|
||||
@@ -1481,20 +1432,6 @@ class GatewayRunner:
|
||||
return None
|
||||
return DingTalkAdapter(config)
|
||||
|
||||
elif platform == Platform.FEISHU:
|
||||
from gateway.platforms.feishu import FeishuAdapter, check_feishu_requirements
|
||||
if not check_feishu_requirements():
|
||||
logger.warning("Feishu: lark-oapi not installed or FEISHU_APP_ID/SECRET not set")
|
||||
return None
|
||||
return FeishuAdapter(config)
|
||||
|
||||
elif platform == Platform.WECOM:
|
||||
from gateway.platforms.wecom import WeComAdapter, check_wecom_requirements
|
||||
if not check_wecom_requirements():
|
||||
logger.warning("WeCom: aiohttp not installed or WECOM_BOT_ID/SECRET not set")
|
||||
return None
|
||||
return WeComAdapter(config)
|
||||
|
||||
elif platform == Platform.MATTERMOST:
|
||||
from gateway.platforms.mattermost import MattermostAdapter, check_mattermost_requirements
|
||||
if not check_mattermost_requirements():
|
||||
@@ -1561,8 +1498,6 @@ class GatewayRunner:
|
||||
Platform.MATTERMOST: "MATTERMOST_ALLOWED_USERS",
|
||||
Platform.MATRIX: "MATRIX_ALLOWED_USERS",
|
||||
Platform.DINGTALK: "DINGTALK_ALLOWED_USERS",
|
||||
Platform.FEISHU: "FEISHU_ALLOWED_USERS",
|
||||
Platform.WECOM: "WECOM_ALLOWED_USERS",
|
||||
}
|
||||
platform_allow_all_map = {
|
||||
Platform.TELEGRAM: "TELEGRAM_ALLOW_ALL_USERS",
|
||||
@@ -1575,8 +1510,6 @@ class GatewayRunner:
|
||||
Platform.MATTERMOST: "MATTERMOST_ALLOW_ALL_USERS",
|
||||
Platform.MATRIX: "MATRIX_ALLOW_ALL_USERS",
|
||||
Platform.DINGTALK: "DINGTALK_ALLOW_ALL_USERS",
|
||||
Platform.FEISHU: "FEISHU_ALLOW_ALL_USERS",
|
||||
Platform.WECOM: "WECOM_ALLOW_ALL_USERS",
|
||||
}
|
||||
|
||||
# Per-platform allow-all flag (e.g., DISCORD_ALLOW_ALL_USERS=true)
|
||||
@@ -1604,23 +1537,10 @@ class GatewayRunner:
|
||||
if global_allowlist:
|
||||
allowed_ids.update(uid.strip() for uid in global_allowlist.split(",") if uid.strip())
|
||||
|
||||
# WhatsApp JIDs have @s.whatsapp.net suffix — strip it for comparison
|
||||
check_ids = {user_id}
|
||||
if "@" in user_id:
|
||||
check_ids.add(user_id.split("@")[0])
|
||||
|
||||
# WhatsApp: resolve phone↔LID aliases from bridge session mapping files
|
||||
if source.platform == Platform.WHATSAPP:
|
||||
normalized_allowed_ids = set()
|
||||
for allowed_id in allowed_ids:
|
||||
normalized_allowed_ids.update(_expand_whatsapp_auth_aliases(allowed_id))
|
||||
if normalized_allowed_ids:
|
||||
allowed_ids = normalized_allowed_ids
|
||||
|
||||
check_ids.update(_expand_whatsapp_auth_aliases(user_id))
|
||||
normalized_user_id = _normalize_whatsapp_identifier(user_id)
|
||||
if normalized_user_id:
|
||||
check_ids.add(normalized_user_id)
|
||||
|
||||
return bool(check_ids & allowed_ids)
|
||||
|
||||
def _get_unauthorized_dm_behavior(self, platform: Optional[Platform]) -> str:
|
||||
@@ -3175,7 +3095,8 @@ class GatewayRunner:
|
||||
if "agent" not in config or not isinstance(config.get("agent"), dict):
|
||||
config["agent"] = {}
|
||||
config["agent"]["system_prompt"] = ""
|
||||
atomic_yaml_write(config_path, config)
|
||||
with open(config_path, "w") as f:
|
||||
yaml.dump(config, f, default_flow_style=False, sort_keys=False)
|
||||
except Exception as e:
|
||||
return f"⚠️ Failed to save personality change: {e}"
|
||||
self._ephemeral_system_prompt = ""
|
||||
@@ -3188,7 +3109,8 @@ class GatewayRunner:
|
||||
if "agent" not in config or not isinstance(config.get("agent"), dict):
|
||||
config["agent"] = {}
|
||||
config["agent"]["system_prompt"] = new_prompt
|
||||
atomic_yaml_write(config_path, config)
|
||||
with open(config_path, 'w', encoding="utf-8") as f:
|
||||
yaml.dump(config, f, default_flow_style=False, sort_keys=False)
|
||||
except Exception as e:
|
||||
return f"⚠️ Failed to save personality change: {e}"
|
||||
|
||||
@@ -3278,7 +3200,8 @@ class GatewayRunner:
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
user_config = yaml.safe_load(f) or {}
|
||||
user_config[env_key] = chat_id
|
||||
atomic_yaml_write(config_path, user_config)
|
||||
with open(config_path, 'w', encoding="utf-8") as f:
|
||||
yaml.dump(user_config, f, default_flow_style=False)
|
||||
# Also set in the current environment so it takes effect immediately
|
||||
os.environ[env_key] = str(chat_id)
|
||||
except Exception as e:
|
||||
@@ -3891,7 +3814,7 @@ class GatewayRunner:
|
||||
# Send media files
|
||||
for media_path in (media_files or []):
|
||||
try:
|
||||
await adapter.send_document(
|
||||
await adapter.send_file(
|
||||
chat_id=source.chat_id,
|
||||
file_path=media_path,
|
||||
)
|
||||
@@ -3946,7 +3869,8 @@ class GatewayRunner:
|
||||
current[k] = {}
|
||||
current = current[k]
|
||||
current[keys[-1]] = value
|
||||
atomic_yaml_write(config_path, user_config)
|
||||
with open(config_path, "w", encoding="utf-8") as f:
|
||||
yaml.dump(user_config, f, default_flow_style=False, sort_keys=False)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error("Failed to save config key %s: %s", key_path, e)
|
||||
@@ -4054,7 +3978,8 @@ class GatewayRunner:
|
||||
if "display" not in user_config or not isinstance(user_config.get("display"), dict):
|
||||
user_config["display"] = {}
|
||||
user_config["display"]["tool_progress"] = new_mode
|
||||
atomic_yaml_write(config_path, user_config)
|
||||
with open(config_path, "w", encoding="utf-8") as f:
|
||||
yaml.dump(user_config, f, default_flow_style=False, sort_keys=False)
|
||||
return f"{descriptions[new_mode]}\n_(saved to config — takes effect on next message)_"
|
||||
except Exception as e:
|
||||
logger.warning("Failed to save tool_progress mode: %s", e)
|
||||
@@ -5019,14 +4944,6 @@ class GatewayRunner:
|
||||
from hermes_cli.tools_config import _get_platform_tools
|
||||
enabled_toolsets = sorted(_get_platform_tools(user_config, platform_key))
|
||||
|
||||
# Apply tool preview length config (0 = no limit)
|
||||
try:
|
||||
from agent.display import set_tool_preview_max_len
|
||||
_tpl = user_config.get("display", {}).get("tool_preview_length", 0)
|
||||
set_tool_preview_max_len(int(_tpl) if _tpl else 0)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Tool progress mode from config.yaml: "all", "new", "verbose", "off"
|
||||
# Falls back to env vars for backward compatibility.
|
||||
# YAML 1.1 parses bare `off` as boolean False — normalise before
|
||||
@@ -5072,11 +4989,9 @@ class GatewayRunner:
|
||||
return
|
||||
|
||||
if preview:
|
||||
# Truncate preview unless config says unlimited
|
||||
from agent.display import get_tool_preview_max_len
|
||||
_pl = get_tool_preview_max_len()
|
||||
if _pl > 0 and len(preview) > _pl:
|
||||
preview = preview[:_pl - 3] + "..."
|
||||
# Truncate preview to keep messages clean
|
||||
if len(preview) > 80:
|
||||
preview = preview[:77] + "..."
|
||||
msg = f"{emoji} {tool_name}: \"{preview}\""
|
||||
else:
|
||||
msg = f"{emoji} {tool_name}..."
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Hermes Agent CLI launcher.
|
||||
Hermes Agent CLI Launcher
|
||||
|
||||
This wrapper should behave like the installed `hermes` command, including
|
||||
subcommands such as `gateway`, `cron`, and `doctor`.
|
||||
This is a convenience wrapper to launch the Hermes CLI.
|
||||
Usage: ./hermes [options]
|
||||
"""
|
||||
|
||||
if __name__ == "__main__":
|
||||
from hermes_cli.main import main
|
||||
main()
|
||||
from cli import main
|
||||
import fire
|
||||
fire.Fire(main)
|
||||
|
||||
@@ -11,5 +11,5 @@ Provides subcommands for:
|
||||
- hermes cron - Manage cron jobs
|
||||
"""
|
||||
|
||||
__version__ = "0.6.0"
|
||||
__release_date__ = "2026.3.30"
|
||||
__version__ = "0.5.0"
|
||||
__release_date__ = "2026.3.28"
|
||||
|
||||
+10
-19
@@ -38,7 +38,7 @@ import httpx
|
||||
import yaml
|
||||
|
||||
from hermes_cli.config import get_hermes_home, get_config_path
|
||||
from hermes_constants import OPENROUTER_BASE_URL
|
||||
from hermes_constants import OPENROUTER_BASE_URL, display_hermes_home
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -696,10 +696,6 @@ def resolve_provider(
|
||||
"hf": "huggingface", "hugging-face": "huggingface", "huggingface-hub": "huggingface",
|
||||
"go": "opencode-go", "opencode-go-sub": "opencode-go",
|
||||
"kilo": "kilocode", "kilo-code": "kilocode", "kilo-gateway": "kilocode",
|
||||
# Local server aliases — route through the generic custom provider
|
||||
"lmstudio": "custom", "lm-studio": "custom", "lm_studio": "custom",
|
||||
"ollama": "custom", "vllm": "custom", "llamacpp": "custom",
|
||||
"llama.cpp": "custom", "llama-cpp": "custom",
|
||||
}
|
||||
normalized = _PROVIDER_ALIASES.get(normalized, normalized)
|
||||
|
||||
@@ -746,12 +742,7 @@ def resolve_provider(
|
||||
if has_usable_secret(os.getenv(env_var, "")):
|
||||
return pid
|
||||
|
||||
raise AuthError(
|
||||
"No inference provider configured. Run 'hermes model' to choose a "
|
||||
"provider and model, or set an API key (OPENROUTER_API_KEY, "
|
||||
"OPENAI_API_KEY, etc.) in ~/.hermes/.env.",
|
||||
code="no_provider_configured",
|
||||
)
|
||||
return "openrouter"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
@@ -2030,8 +2021,7 @@ def _login_openai_codex(args, pconfig: ProviderConfig) -> None:
|
||||
config_path = _update_config_for_provider("openai-codex", creds.get("base_url", DEFAULT_CODEX_BASE_URL))
|
||||
print()
|
||||
print("Login successful!")
|
||||
from hermes_constants import display_hermes_home as _dhh
|
||||
print(f" Auth state: {_dhh()}/auth.json")
|
||||
print(f" Auth state: {display_hermes_home()}/auth.json")
|
||||
print(f" Config updated: {config_path} (model.provider=openai-codex)")
|
||||
|
||||
|
||||
@@ -2310,20 +2300,21 @@ def _login_nous(args, pconfig: ProviderConfig) -> None:
|
||||
raise AuthError("No runtime API key available to fetch models",
|
||||
provider="nous", code="invalid_token")
|
||||
|
||||
# Use curated model list (same as OpenRouter defaults) instead
|
||||
# of the full /models dump which returns hundreds of models.
|
||||
from hermes_cli.models import _PROVIDER_MODELS
|
||||
model_ids = _PROVIDER_MODELS.get("nous", [])
|
||||
model_ids = fetch_nous_models(
|
||||
inference_base_url=runtime_base_url,
|
||||
api_key=runtime_key,
|
||||
timeout_seconds=timeout_seconds,
|
||||
verify=verify,
|
||||
)
|
||||
|
||||
print()
|
||||
if model_ids:
|
||||
print(f"Showing {len(model_ids)} curated models — use \"Enter custom model name\" for others.")
|
||||
selected_model = _prompt_model_selection(model_ids)
|
||||
if selected_model:
|
||||
_save_model_choice(selected_model)
|
||||
print(f"Default model set to: {selected_model}")
|
||||
else:
|
||||
print("No curated models available for Nous Portal.")
|
||||
print("No models were returned by the inference API.")
|
||||
except Exception as exc:
|
||||
message = format_auth_error(exc) if isinstance(exc, AuthError) else str(exc)
|
||||
print()
|
||||
|
||||
+2
-16
@@ -258,7 +258,7 @@ def build_welcome_banner(console: Console, model: str, cwd: str,
|
||||
get_toolset_for_tool: Callable to map tool name -> toolset name.
|
||||
context_length: Model's context window size in tokens.
|
||||
"""
|
||||
from model_tools import check_tool_availability, TOOLSET_REQUIREMENTS
|
||||
from model_tools import check_tool_availability
|
||||
if get_toolset_for_tool is None:
|
||||
from model_tools import get_toolset_for_tool
|
||||
|
||||
@@ -267,18 +267,8 @@ def build_welcome_banner(console: Console, model: str, cwd: str,
|
||||
|
||||
_, unavailable_toolsets = check_tool_availability(quiet=True)
|
||||
disabled_tools = set()
|
||||
# Tools whose toolset has a check_fn are lazy-initialized (e.g. honcho,
|
||||
# homeassistant) — they show as unavailable at banner time because the
|
||||
# check hasn't run yet, but they aren't misconfigured.
|
||||
lazy_tools = set()
|
||||
for item in unavailable_toolsets:
|
||||
toolset_name = item.get("name", "")
|
||||
ts_req = TOOLSET_REQUIREMENTS.get(toolset_name, {})
|
||||
tools_in_ts = item.get("tools", [])
|
||||
if ts_req.get("check_fn"):
|
||||
lazy_tools.update(tools_in_ts)
|
||||
else:
|
||||
disabled_tools.update(tools_in_ts)
|
||||
disabled_tools.update(item.get("tools", []))
|
||||
|
||||
layout_table = Table.grid(padding=(0, 2))
|
||||
layout_table.add_column("left", justify="center")
|
||||
@@ -338,8 +328,6 @@ def build_welcome_banner(console: Console, model: str, cwd: str,
|
||||
for name in sorted(tool_names):
|
||||
if name in disabled_tools:
|
||||
colored_names.append(f"[red]{name}[/]")
|
||||
elif name in lazy_tools:
|
||||
colored_names.append(f"[yellow]{name}[/]")
|
||||
else:
|
||||
colored_names.append(f"[{text}]{name}[/]")
|
||||
|
||||
@@ -359,8 +347,6 @@ def build_welcome_banner(console: Console, model: str, cwd: str,
|
||||
colored_names.append("[dim]...[/]")
|
||||
elif name in disabled_tools:
|
||||
colored_names.append(f"[red]{name}[/]")
|
||||
elif name in lazy_tools:
|
||||
colored_names.append(f"[yellow]{name}[/]")
|
||||
else:
|
||||
colored_names.append(f"[{text}]{name}[/]")
|
||||
tools_str = ", ".join(colored_names)
|
||||
|
||||
@@ -241,8 +241,7 @@ def approval_callback(cli, command: str, description: str) -> str:
|
||||
lock = cli._approval_lock
|
||||
|
||||
with lock:
|
||||
from cli import CLI_CONFIG
|
||||
timeout = CLI_CONFIG.get("approvals", {}).get("timeout", 60)
|
||||
timeout = 60
|
||||
response_queue = queue.Queue()
|
||||
choices = ["once", "session", "always", "deny"]
|
||||
if len(command) > 70:
|
||||
|
||||
@@ -5,7 +5,6 @@ toggleable list of items. Falls back to a numbered text UI when
|
||||
curses is unavailable (Windows without curses, piped stdin, etc.).
|
||||
"""
|
||||
|
||||
import sys
|
||||
from typing import List, Set
|
||||
|
||||
from hermes_cli.colors import Colors, color
|
||||
@@ -27,10 +26,6 @@ def curses_checklist(
|
||||
The indices the user confirmed as checked. On cancel (ESC/q),
|
||||
returns ``pre_selected`` unchanged.
|
||||
"""
|
||||
# Safety: return defaults when stdin is not a terminal.
|
||||
if not sys.stdin.isatty():
|
||||
return set(pre_selected)
|
||||
|
||||
try:
|
||||
import curses
|
||||
selected = set(pre_selected)
|
||||
|
||||
+1
-13
@@ -88,19 +88,7 @@ def claw_command(args):
|
||||
|
||||
def _cmd_migrate(args):
|
||||
"""Run the OpenClaw → Hermes migration."""
|
||||
# Check current and legacy OpenClaw directories
|
||||
explicit_source = getattr(args, "source", None)
|
||||
if explicit_source:
|
||||
source_dir = Path(explicit_source)
|
||||
else:
|
||||
source_dir = Path.home() / ".openclaw"
|
||||
if not source_dir.is_dir():
|
||||
# Try legacy directory names
|
||||
for legacy in (".clawdbot", ".moldbot"):
|
||||
candidate = Path.home() / legacy
|
||||
if candidate.is_dir():
|
||||
source_dir = candidate
|
||||
break
|
||||
source_dir = Path(getattr(args, "source", None) or Path.home() / ".openclaw")
|
||||
dry_run = getattr(args, "dry_run", False)
|
||||
preset = getattr(args, "preset", "full")
|
||||
overwrite = getattr(args, "overwrite", False)
|
||||
|
||||
@@ -12,8 +12,6 @@ import os
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_CODEX_MODELS: List[str] = [
|
||||
"gpt-5.4-mini",
|
||||
"gpt-5.4",
|
||||
"gpt-5.3-codex",
|
||||
"gpt-5.2-codex",
|
||||
"gpt-5.1-codex-max",
|
||||
@@ -21,9 +19,8 @@ DEFAULT_CODEX_MODELS: List[str] = [
|
||||
]
|
||||
|
||||
_FORWARD_COMPAT_TEMPLATE_MODELS: List[tuple[str, tuple[str, ...]]] = [
|
||||
("gpt-5.4-mini", ("gpt-5.3-codex", "gpt-5.2-codex")),
|
||||
("gpt-5.4", ("gpt-5.3-codex", "gpt-5.2-codex")),
|
||||
("gpt-5.3-codex", ("gpt-5.2-codex",)),
|
||||
("gpt-5.4", ("gpt-5.3-codex", "gpt-5.2-codex")),
|
||||
("gpt-5.3-codex-spark", ("gpt-5.3-codex", "gpt-5.2-codex")),
|
||||
]
|
||||
|
||||
|
||||
+1
-13
@@ -34,8 +34,6 @@ _EXTRA_ENV_KEYS = frozenset({
|
||||
"SIGNAL_ACCOUNT", "SIGNAL_HTTP_URL",
|
||||
"SIGNAL_ALLOWED_USERS", "SIGNAL_GROUP_ALLOWED_USERS",
|
||||
"DINGTALK_CLIENT_ID", "DINGTALK_CLIENT_SECRET",
|
||||
"FEISHU_APP_ID", "FEISHU_APP_SECRET", "FEISHU_ENCRYPT_KEY", "FEISHU_VERIFICATION_TOKEN",
|
||||
"WECOM_BOT_ID", "WECOM_SECRET",
|
||||
"TERMINAL_ENV", "TERMINAL_SSH_KEY", "TERMINAL_SSH_PORT",
|
||||
"WHATSAPP_MODE", "WHATSAPP_ENABLED",
|
||||
"MATTERMOST_HOME_CHANNEL", "MATTERMOST_REPLY_MODE",
|
||||
@@ -137,7 +135,6 @@ def ensure_hermes_home():
|
||||
|
||||
DEFAULT_CONFIG = {
|
||||
"model": "anthropic/claude-opus-4.6",
|
||||
"fallback_providers": [],
|
||||
"toolsets": ["hermes-cli"],
|
||||
"agent": {
|
||||
"max_turns": 90,
|
||||
@@ -223,8 +220,7 @@ DEFAULT_CONFIG = {
|
||||
"model": "", # e.g. "google/gemini-2.5-flash", "gpt-4o"
|
||||
"base_url": "", # direct OpenAI-compatible endpoint (takes precedence over provider)
|
||||
"api_key": "", # API key for base_url (falls back to OPENAI_API_KEY)
|
||||
"timeout": 30, # seconds — LLM API call timeout; increase for slow local vision models
|
||||
"download_timeout": 30, # seconds — image HTTP download timeout; increase for slow connections
|
||||
"timeout": 30, # seconds — increase for slow local vision models
|
||||
},
|
||||
"web_extract": {
|
||||
"provider": "auto",
|
||||
@@ -288,7 +284,6 @@ DEFAULT_CONFIG = {
|
||||
"show_cost": False, # Show $ cost in the status bar (off by default)
|
||||
"skin": "default",
|
||||
"tool_progress_command": False, # Enable /verbose command in messaging gateway
|
||||
"tool_preview_length": 0, # Max chars for tool call previews (0 = no limit, show full paths/commands)
|
||||
},
|
||||
|
||||
# Privacy settings
|
||||
@@ -408,7 +403,6 @@ DEFAULT_CONFIG = {
|
||||
# off — skip all approval prompts (equivalent to --yolo)
|
||||
"approvals": {
|
||||
"mode": "manual",
|
||||
"timeout": 60,
|
||||
},
|
||||
|
||||
# Permanently allowed dangerous command patterns (added via "always" approval)
|
||||
@@ -434,12 +428,6 @@ DEFAULT_CONFIG = {
|
||||
},
|
||||
},
|
||||
|
||||
"cron": {
|
||||
# Wrap delivered cron responses with a header (task name) and footer
|
||||
# ("The agent cannot see this message"). Set to false for clean output.
|
||||
"wrap_response": True,
|
||||
},
|
||||
|
||||
# Config schema version - bump this when adding new required fields
|
||||
"_config_version": 10,
|
||||
}
|
||||
|
||||
+4
-36
@@ -4,8 +4,7 @@ Used by `hermes tools` and `hermes skills` for interactive checklists.
|
||||
Provides a curses multi-select with keyboard navigation, plus a
|
||||
text-based numbered fallback for terminals without curses support.
|
||||
"""
|
||||
import sys
|
||||
from typing import Callable, List, Optional, Set
|
||||
from typing import List, Set
|
||||
|
||||
from hermes_cli.colors import Colors, color
|
||||
|
||||
@@ -16,7 +15,6 @@ def curses_checklist(
|
||||
selected: Set[int],
|
||||
*,
|
||||
cancel_returns: Set[int] | None = None,
|
||||
status_fn: Optional[Callable[[Set[int]], str]] = None,
|
||||
) -> Set[int]:
|
||||
"""Curses multi-select checklist. Returns set of selected indices.
|
||||
|
||||
@@ -25,18 +23,10 @@ def curses_checklist(
|
||||
items: Display labels for each row.
|
||||
selected: Indices that start checked (pre-selected).
|
||||
cancel_returns: Returned on ESC/q. Defaults to the original *selected*.
|
||||
status_fn: Optional callback ``f(chosen_indices) -> str`` whose return
|
||||
value is rendered on the bottom row of the terminal. Use this for
|
||||
live aggregate info (e.g. estimated token counts).
|
||||
"""
|
||||
if cancel_returns is None:
|
||||
cancel_returns = set(selected)
|
||||
|
||||
# Safety: curses and input() both hang or spin when stdin is not a
|
||||
# terminal (e.g. subprocess pipe). Return defaults immediately.
|
||||
if not sys.stdin.isatty():
|
||||
return cancel_returns
|
||||
|
||||
try:
|
||||
import curses
|
||||
chosen = set(selected)
|
||||
@@ -57,9 +47,6 @@ def curses_checklist(
|
||||
stdscr.clear()
|
||||
max_y, max_x = stdscr.getmaxyx()
|
||||
|
||||
# Reserve bottom row for status bar when status_fn provided
|
||||
footer_rows = 1 if status_fn else 0
|
||||
|
||||
# Header
|
||||
try:
|
||||
hattr = curses.A_BOLD
|
||||
@@ -75,7 +62,7 @@ def curses_checklist(
|
||||
pass
|
||||
|
||||
# Scrollable item list
|
||||
visible_rows = max_y - 3 - footer_rows
|
||||
visible_rows = max_y - 3
|
||||
if cursor < scroll_offset:
|
||||
scroll_offset = cursor
|
||||
elif cursor >= scroll_offset + visible_rows:
|
||||
@@ -85,7 +72,7 @@ def curses_checklist(
|
||||
range(scroll_offset, min(len(items), scroll_offset + visible_rows))
|
||||
):
|
||||
y = draw_i + 3
|
||||
if y >= max_y - 1 - footer_rows:
|
||||
if y >= max_y - 1:
|
||||
break
|
||||
check = "✓" if i in chosen else " "
|
||||
arrow = "→" if i == cursor else " "
|
||||
@@ -100,20 +87,6 @@ def curses_checklist(
|
||||
except curses.error:
|
||||
pass
|
||||
|
||||
# Status bar (bottom row, right-aligned)
|
||||
if status_fn:
|
||||
try:
|
||||
status_text = status_fn(chosen)
|
||||
if status_text:
|
||||
# Right-align on the bottom row
|
||||
sx = max(0, max_x - len(status_text) - 1)
|
||||
sattr = curses.A_DIM
|
||||
if curses.has_colors():
|
||||
sattr |= curses.color_pair(3)
|
||||
stdscr.addnstr(max_y - 1, sx, status_text, max_x - sx - 1, sattr)
|
||||
except curses.error:
|
||||
pass
|
||||
|
||||
stdscr.refresh()
|
||||
key = stdscr.getch()
|
||||
|
||||
@@ -134,7 +107,7 @@ def curses_checklist(
|
||||
return result_holder[0] if result_holder[0] is not None else cancel_returns
|
||||
|
||||
except Exception:
|
||||
return _numbered_fallback(title, items, selected, cancel_returns, status_fn)
|
||||
return _numbered_fallback(title, items, selected, cancel_returns)
|
||||
|
||||
|
||||
def _numbered_fallback(
|
||||
@@ -142,7 +115,6 @@ def _numbered_fallback(
|
||||
items: List[str],
|
||||
selected: Set[int],
|
||||
cancel_returns: Set[int],
|
||||
status_fn: Optional[Callable[[Set[int]], str]] = None,
|
||||
) -> Set[int]:
|
||||
"""Text-based toggle fallback for terminals without curses."""
|
||||
chosen = set(selected)
|
||||
@@ -153,10 +125,6 @@ def _numbered_fallback(
|
||||
for i, label in enumerate(items):
|
||||
marker = color("[✓]", Colors.GREEN) if i in chosen else "[ ]"
|
||||
print(f" {marker} {i + 1:>2}. {label}")
|
||||
if status_fn:
|
||||
status_text = status_fn(chosen)
|
||||
if status_text:
|
||||
print(color(f"\n {status_text}", Colors.DIM))
|
||||
print()
|
||||
try:
|
||||
val = input(color(" Toggle # (or Enter to confirm): ", Colors.DIM)).strip()
|
||||
|
||||
+2
-57
@@ -15,8 +15,7 @@ from pathlib import Path
|
||||
PROJECT_ROOT = Path(__file__).parent.parent.resolve()
|
||||
|
||||
from hermes_cli.config import get_env_value, get_hermes_home, save_env_value, is_managed, managed_error
|
||||
# display_hermes_home is imported lazily at call sites to avoid ImportError
|
||||
# when hermes_constants is cached from a pre-update version during `hermes update`.
|
||||
from hermes_constants import display_hermes_home
|
||||
from hermes_cli.setup import (
|
||||
print_header, print_info, print_success, print_warning, print_error,
|
||||
prompt, prompt_choice, prompt_yes_no,
|
||||
@@ -937,8 +936,7 @@ def launchd_install(force: bool = False):
|
||||
print()
|
||||
print("Next steps:")
|
||||
print(" hermes gateway status # Check status")
|
||||
from hermes_constants import display_hermes_home as _dhh
|
||||
print(f" tail -f {_dhh()}/logs/gateway.log # View logs")
|
||||
print(f" tail -f {display_hermes_home()}/logs/gateway.log # View logs")
|
||||
|
||||
def launchd_uninstall():
|
||||
plist_path = get_launchd_plist_path()
|
||||
@@ -1322,59 +1320,6 @@ _PLATFORMS = [
|
||||
"help": "The AppSecret from your DingTalk application credentials."},
|
||||
],
|
||||
},
|
||||
{
|
||||
"key": "feishu",
|
||||
"label": "Feishu / Lark",
|
||||
"emoji": "🪽",
|
||||
"token_var": "FEISHU_APP_ID",
|
||||
"setup_instructions": [
|
||||
"1. Go to https://open.feishu.cn/ (or https://open.larksuite.com/ for Lark)",
|
||||
"2. Create an app and copy the App ID and App Secret",
|
||||
"3. Enable the Bot capability for the app",
|
||||
"4. Choose WebSocket (recommended) or Webhook connection mode",
|
||||
"5. Add the bot to a group chat or message it directly",
|
||||
"6. Restrict access with FEISHU_ALLOWED_USERS for production use",
|
||||
],
|
||||
"vars": [
|
||||
{"name": "FEISHU_APP_ID", "prompt": "App ID", "password": False,
|
||||
"help": "The App ID from your Feishu/Lark application."},
|
||||
{"name": "FEISHU_APP_SECRET", "prompt": "App Secret", "password": True,
|
||||
"help": "The App Secret from your Feishu/Lark application."},
|
||||
{"name": "FEISHU_DOMAIN", "prompt": "Domain — feishu or lark (default: feishu)", "password": False,
|
||||
"help": "Use 'feishu' for Feishu China, or 'lark' for Lark international."},
|
||||
{"name": "FEISHU_CONNECTION_MODE", "prompt": "Connection mode — websocket or webhook (default: websocket)", "password": False,
|
||||
"help": "websocket is recommended unless you specifically need webhook mode."},
|
||||
{"name": "FEISHU_ALLOWED_USERS", "prompt": "Allowed user IDs (comma-separated, or empty)", "password": False,
|
||||
"is_allowlist": True,
|
||||
"help": "Restrict which Feishu/Lark users can interact with the bot."},
|
||||
{"name": "FEISHU_HOME_CHANNEL", "prompt": "Home chat ID (optional, for cron/notifications)", "password": False,
|
||||
"help": "Chat ID for scheduled results and notifications."},
|
||||
],
|
||||
},
|
||||
{
|
||||
"key": "wecom",
|
||||
"label": "WeCom (Enterprise WeChat)",
|
||||
"emoji": "💬",
|
||||
"token_var": "WECOM_BOT_ID",
|
||||
"setup_instructions": [
|
||||
"1. Go to WeCom Admin Console → Applications → Create AI Bot",
|
||||
"2. Copy the Bot ID and Secret from the bot's credentials page",
|
||||
"3. The bot connects via WebSocket — no public endpoint needed",
|
||||
"4. Add the bot to a group chat or message it directly in WeCom",
|
||||
"5. Restrict access with WECOM_ALLOWED_USERS for production use",
|
||||
],
|
||||
"vars": [
|
||||
{"name": "WECOM_BOT_ID", "prompt": "Bot ID", "password": False,
|
||||
"help": "The Bot ID from your WeCom AI Bot."},
|
||||
{"name": "WECOM_SECRET", "prompt": "Secret", "password": True,
|
||||
"help": "The secret from your WeCom AI Bot."},
|
||||
{"name": "WECOM_ALLOWED_USERS", "prompt": "Allowed user IDs (comma-separated, or empty)", "password": False,
|
||||
"is_allowlist": True,
|
||||
"help": "Restrict which WeCom users can interact with the bot."},
|
||||
{"name": "WECOM_HOME_CHANNEL", "prompt": "Home chat ID (optional, for cron/notifications)", "password": False,
|
||||
"help": "Chat ID for scheduled results and notifications."},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
|
||||
+20
-115
@@ -50,23 +50,6 @@ import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
def _require_tty(command_name: str) -> None:
|
||||
"""Exit with a clear error if stdin is not a terminal.
|
||||
|
||||
Interactive TUI commands (hermes tools, hermes setup, hermes model) use
|
||||
curses or input() prompts that spin at 100% CPU when stdin is a pipe.
|
||||
This guard prevents accidental non-interactive invocation.
|
||||
"""
|
||||
if not sys.stdin.isatty():
|
||||
print(
|
||||
f"Error: 'hermes {command_name}' requires an interactive terminal.\n"
|
||||
f"It cannot be run through a pipe or non-interactive subprocess.\n"
|
||||
f"Run it directly in your terminal instead.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# Add project root to path
|
||||
PROJECT_ROOT = Path(__file__).parent.parent.resolve()
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
@@ -634,7 +617,6 @@ def cmd_gateway(args):
|
||||
|
||||
def cmd_whatsapp(args):
|
||||
"""Set up WhatsApp: choose mode, configure, install bridge, pair via QR."""
|
||||
_require_tty("whatsapp")
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from hermes_cli.config import get_env_value, save_env_value
|
||||
@@ -821,14 +803,12 @@ def cmd_whatsapp(args):
|
||||
|
||||
def cmd_setup(args):
|
||||
"""Interactive setup wizard."""
|
||||
_require_tty("setup")
|
||||
from hermes_cli.setup import run_setup_wizard
|
||||
run_setup_wizard(args)
|
||||
|
||||
|
||||
def cmd_model(args):
|
||||
"""Select default model — starts with provider selection, then model picker."""
|
||||
_require_tty("model")
|
||||
from hermes_cli.auth import (
|
||||
resolve_provider, AuthError, format_auth_error,
|
||||
)
|
||||
@@ -1104,20 +1084,14 @@ def _model_flow_nous(config, current_model=""):
|
||||
# login_nous already handles model selection + config update
|
||||
return
|
||||
|
||||
# Already logged in — use curated model list (same as OpenRouter defaults).
|
||||
# The live /models endpoint returns hundreds of models; the curated list
|
||||
# shows only agentic models users recognize from OpenRouter.
|
||||
from hermes_cli.models import _PROVIDER_MODELS
|
||||
model_ids = _PROVIDER_MODELS.get("nous", [])
|
||||
if not model_ids:
|
||||
print("No curated models available for Nous Portal.")
|
||||
return
|
||||
|
||||
print(f"Showing {len(model_ids)} curated models — use \"Enter custom model name\" for others.")
|
||||
|
||||
# Verify credentials are still valid (catches expired sessions early)
|
||||
# Already logged in — fetch models and select
|
||||
print("Fetching models from Nous Portal...")
|
||||
try:
|
||||
creds = resolve_nous_runtime_credentials(min_key_ttl_seconds=5 * 60)
|
||||
model_ids = fetch_nous_models(
|
||||
inference_base_url=creds.get("base_url", ""),
|
||||
api_key=creds.get("api_key", ""),
|
||||
)
|
||||
except Exception as exc:
|
||||
relogin = isinstance(exc, AuthError) and exc.relogin_required
|
||||
msg = format_auth_error(exc) if isinstance(exc, AuthError) else str(exc)
|
||||
@@ -1134,7 +1108,11 @@ def _model_flow_nous(config, current_model=""):
|
||||
except Exception as login_exc:
|
||||
print(f"Re-login failed: {login_exc}")
|
||||
return
|
||||
print(f"Could not verify credentials: {msg}")
|
||||
print(f"Could not fetch models: {msg}")
|
||||
return
|
||||
|
||||
if not model_ids:
|
||||
print("No models returned by the inference API.")
|
||||
return
|
||||
|
||||
selected = _prompt_model_selection(model_ids, current_model=current_model)
|
||||
@@ -1291,7 +1269,7 @@ def _model_flow_custom(config):
|
||||
cfg["model"] = model
|
||||
model["provider"] = "custom"
|
||||
model["base_url"] = effective_url
|
||||
model.pop("api_mode", None) # let runtime auto-detect from URL
|
||||
model["api_mode"] = "chat_completions"
|
||||
save_config(cfg)
|
||||
deactivate_provider()
|
||||
|
||||
@@ -2073,7 +2051,7 @@ def _model_flow_kimi(config, current_model=""):
|
||||
cfg["model"] = model
|
||||
model["provider"] = provider_id
|
||||
model["base_url"] = effective_base
|
||||
model.pop("api_mode", None) # let runtime auto-detect from URL
|
||||
model["api_mode"] = "chat_completions"
|
||||
save_config(cfg)
|
||||
deactivate_provider()
|
||||
|
||||
@@ -2147,7 +2125,7 @@ def _model_flow_api_key_provider(config, provider_id, current_model=""):
|
||||
api_key_for_probe = existing_key or (get_env_value(key_env) if key_env else "")
|
||||
live_models = fetch_api_models(api_key_for_probe, effective_base)
|
||||
|
||||
if live_models and len(live_models) >= len(curated):
|
||||
if live_models:
|
||||
model_list = live_models
|
||||
print(f" Found {len(model_list)} model(s) from {pconfig.name} API")
|
||||
else:
|
||||
@@ -2180,7 +2158,7 @@ def _model_flow_api_key_provider(config, provider_id, current_model=""):
|
||||
cfg["model"] = model
|
||||
model["provider"] = provider_id
|
||||
model["base_url"] = effective_base
|
||||
model.pop("api_mode", None) # let runtime auto-detect from URL
|
||||
model["api_mode"] = "chat_completions"
|
||||
save_config(cfg)
|
||||
deactivate_provider()
|
||||
|
||||
@@ -2479,39 +2457,10 @@ def cmd_version(args):
|
||||
|
||||
def cmd_uninstall(args):
|
||||
"""Uninstall Hermes Agent."""
|
||||
_require_tty("uninstall")
|
||||
from hermes_cli.uninstall import run_uninstall
|
||||
run_uninstall(args)
|
||||
|
||||
|
||||
def _clear_bytecode_cache(root: Path) -> int:
|
||||
"""Remove all __pycache__ directories under *root*.
|
||||
|
||||
Stale .pyc files can cause ImportError after code updates when Python
|
||||
loads a cached bytecode file that references names that no longer exist
|
||||
(or don't yet exist) in the updated source. Clearing them forces Python
|
||||
to recompile from the .py source on next import.
|
||||
|
||||
Returns the number of directories removed.
|
||||
"""
|
||||
removed = 0
|
||||
for dirpath, dirnames, _ in os.walk(root):
|
||||
# Skip venv / node_modules / .git entirely
|
||||
dirnames[:] = [
|
||||
d for d in dirnames
|
||||
if d not in ("venv", ".venv", "node_modules", ".git", ".worktrees")
|
||||
]
|
||||
if os.path.basename(dirpath) == "__pycache__":
|
||||
try:
|
||||
import shutil as _shutil
|
||||
_shutil.rmtree(dirpath)
|
||||
removed += 1
|
||||
except OSError:
|
||||
pass
|
||||
dirnames.clear() # nothing left to recurse into
|
||||
return removed
|
||||
|
||||
|
||||
def _update_via_zip(args):
|
||||
"""Update Hermes Agent by downloading a ZIP archive.
|
||||
|
||||
@@ -2553,7 +2502,7 @@ def _update_via_zip(args):
|
||||
break
|
||||
|
||||
# Copy updated files over existing installation, preserving venv/node_modules/.git
|
||||
preserve = {'venv', 'node_modules', '.git', '.env'}
|
||||
preserve = {'venv', 'node_modules', '.git', '__pycache__', '.env'}
|
||||
update_count = 0
|
||||
for item in os.listdir(extracted):
|
||||
if item in preserve:
|
||||
@@ -2576,11 +2525,6 @@ def _update_via_zip(args):
|
||||
except Exception as e:
|
||||
print(f"✗ ZIP update failed: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# Clear stale bytecode after ZIP extraction
|
||||
removed = _clear_bytecode_cache(PROJECT_ROOT)
|
||||
if removed:
|
||||
print(f" ✓ Cleared {removed} stale __pycache__ director{'y' if removed == 1 else 'ies'}")
|
||||
|
||||
# Reinstall Python dependencies (try .[all] first for optional extras,
|
||||
# fall back to . if extras fail — mirrors the install script behavior)
|
||||
@@ -2979,13 +2923,6 @@ def cmd_update(args):
|
||||
)
|
||||
|
||||
_invalidate_update_cache()
|
||||
|
||||
# Clear stale .pyc bytecode cache — prevents ImportError on gateway
|
||||
# restart when updated source references names that didn't exist in
|
||||
# the old bytecode (e.g. get_hermes_home added to hermes_constants).
|
||||
removed = _clear_bytecode_cache(PROJECT_ROOT)
|
||||
if removed:
|
||||
print(f" ✓ Cleared {removed} stale __pycache__ director{'y' if removed == 1 else 'ies'}")
|
||||
|
||||
# Reinstall Python dependencies (try .[all] first for optional extras,
|
||||
# fall back to . if extras fail — mirrors the install script behavior)
|
||||
@@ -3034,17 +2971,6 @@ def cmd_update(args):
|
||||
print()
|
||||
print("✓ Code updated!")
|
||||
|
||||
# After git pull, source files on disk are newer than cached Python
|
||||
# modules in this process. Reload hermes_constants so that any lazy
|
||||
# import executed below (skills sync, gateway restart) sees new
|
||||
# attributes like display_hermes_home() added since the last release.
|
||||
try:
|
||||
import importlib
|
||||
import hermes_constants as _hc
|
||||
importlib.reload(_hc)
|
||||
except Exception:
|
||||
pass # non-fatal — worst case a lazy import fails gracefully
|
||||
|
||||
# Sync bundled skills (copies new, updates changed, respects user deletions)
|
||||
try:
|
||||
from tools.skills_sync import sync_skills
|
||||
@@ -4152,7 +4078,6 @@ For more help on a command:
|
||||
def cmd_skills(args):
|
||||
# Route 'config' action to skills_config module
|
||||
if getattr(args, 'skills_action', None) == 'config':
|
||||
_require_tty("skills config")
|
||||
from hermes_cli.skills_config import skills_command as skills_config_command
|
||||
skills_config_command(args)
|
||||
else:
|
||||
@@ -4195,16 +4120,6 @@ For more help on a command:
|
||||
|
||||
plugins_subparsers.add_parser("list", aliases=["ls"], help="List installed plugins")
|
||||
|
||||
plugins_enable = plugins_subparsers.add_parser(
|
||||
"enable", help="Enable a disabled plugin"
|
||||
)
|
||||
plugins_enable.add_argument("name", help="Plugin name to enable")
|
||||
|
||||
plugins_disable = plugins_subparsers.add_parser(
|
||||
"disable", help="Disable a plugin without removing it"
|
||||
)
|
||||
plugins_disable.add_argument("name", help="Plugin name to disable")
|
||||
|
||||
def cmd_plugins(args):
|
||||
from hermes_cli.plugins_cmd import plugins_command
|
||||
plugins_command(args)
|
||||
@@ -4363,7 +4278,6 @@ For more help on a command:
|
||||
from hermes_cli.tools_config import tools_disable_enable_command
|
||||
tools_disable_enable_command(args)
|
||||
else:
|
||||
_require_tty("tools")
|
||||
from hermes_cli.tools_config import tools_command
|
||||
tools_command(args)
|
||||
|
||||
@@ -4373,25 +4287,16 @@ For more help on a command:
|
||||
# =========================================================================
|
||||
mcp_parser = subparsers.add_parser(
|
||||
"mcp",
|
||||
help="Manage MCP servers and run Hermes as an MCP server",
|
||||
help="Manage MCP server connections",
|
||||
description=(
|
||||
"Manage MCP server connections and run Hermes as an MCP server.\n\n"
|
||||
"Add, remove, list, test, and configure MCP server connections.\n\n"
|
||||
"MCP servers provide additional tools via the Model Context Protocol.\n"
|
||||
"Use 'hermes mcp add' to connect to a new server, or\n"
|
||||
"'hermes mcp serve' to expose Hermes conversations over MCP."
|
||||
"Use 'hermes mcp add' to connect to a new server with interactive\n"
|
||||
"tool discovery. Run 'hermes mcp' with no subcommand to list servers."
|
||||
),
|
||||
)
|
||||
mcp_sub = mcp_parser.add_subparsers(dest="mcp_action")
|
||||
|
||||
mcp_serve_p = mcp_sub.add_parser(
|
||||
"serve",
|
||||
help="Run Hermes as an MCP server (expose conversations to other agents)",
|
||||
)
|
||||
mcp_serve_p.add_argument(
|
||||
"-v", "--verbose", action="store_true",
|
||||
help="Enable verbose logging on stderr",
|
||||
)
|
||||
|
||||
mcp_add_p = mcp_sub.add_parser("add", help="Add an MCP server (discovery-first install)")
|
||||
mcp_add_p.add_argument("name", help="Server name (used as config key)")
|
||||
mcp_add_p.add_argument("--url", help="HTTP/SSE endpoint URL")
|
||||
|
||||
@@ -511,10 +511,6 @@ def _interpolate_value(value: str) -> str:
|
||||
|
||||
def cmd_mcp_configure(args):
|
||||
"""Reconfigure which tools are enabled for an existing MCP server."""
|
||||
import sys as _sys
|
||||
if not _sys.stdin.isatty():
|
||||
print("Error: 'hermes mcp configure' requires an interactive terminal.", file=_sys.stderr)
|
||||
_sys.exit(1)
|
||||
name = args.name
|
||||
servers = _get_mcp_servers()
|
||||
|
||||
@@ -612,11 +608,6 @@ def mcp_command(args):
|
||||
"""Main dispatcher for ``hermes mcp`` subcommands."""
|
||||
action = getattr(args, "mcp_action", None)
|
||||
|
||||
if action == "serve":
|
||||
from mcp_serve import run_mcp_server
|
||||
run_mcp_server(verbose=getattr(args, "verbose", False))
|
||||
return
|
||||
|
||||
handlers = {
|
||||
"add": cmd_mcp_add,
|
||||
"remove": cmd_mcp_remove,
|
||||
@@ -635,7 +626,6 @@ def mcp_command(args):
|
||||
# No subcommand — show list
|
||||
cmd_mcp_list()
|
||||
print(color(" Commands:", Colors.CYAN))
|
||||
_info("hermes mcp serve Run as MCP server")
|
||||
_info("hermes mcp add <name> --url <endpoint> Add an MCP server")
|
||||
_info("hermes mcp add <name> --command <cmd> Add a stdio server")
|
||||
_info("hermes mcp remove <name> Remove a server")
|
||||
|
||||
@@ -35,8 +35,6 @@ OPENROUTER_MODELS: list[tuple[str, str]] = [
|
||||
("openai/gpt-5.3-codex", ""),
|
||||
("google/gemini-3-pro-preview", ""),
|
||||
("google/gemini-3-flash-preview", ""),
|
||||
("google/gemini-3.1-pro-preview", ""),
|
||||
("google/gemini-3.1-flash-lite-preview", ""),
|
||||
("qwen/qwen3.5-plus-02-15", ""),
|
||||
("qwen/qwen3.5-35b-a3b", ""),
|
||||
("stepfun/step-3.5-flash", ""),
|
||||
@@ -64,8 +62,6 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
"openai/gpt-5.3-codex",
|
||||
"google/gemini-3-pro-preview",
|
||||
"google/gemini-3-flash-preview",
|
||||
"google/gemini-3.1-pro-preview",
|
||||
"google/gemini-3.1-flash-lite-preview",
|
||||
"qwen/qwen3.5-plus-02-15",
|
||||
"qwen/qwen3.5-35b-a3b",
|
||||
"stepfun/step-3.5-flash",
|
||||
|
||||
+1
-48
@@ -68,17 +68,6 @@ def _env_enabled(name: str) -> bool:
|
||||
return os.getenv(name, "").strip().lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def _get_disabled_plugins() -> set:
|
||||
"""Read the disabled plugins list from config.yaml."""
|
||||
try:
|
||||
from hermes_cli.config import load_config
|
||||
config = load_config()
|
||||
disabled = config.get("plugins", {}).get("disabled", [])
|
||||
return set(disabled) if isinstance(disabled, list) else set()
|
||||
except Exception:
|
||||
return set()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Data classes
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -152,34 +141,6 @@ class PluginContext:
|
||||
self._manager._plugin_tool_names.add(name)
|
||||
logger.debug("Plugin %s registered tool: %s", self.manifest.name, name)
|
||||
|
||||
# -- message injection --------------------------------------------------
|
||||
|
||||
def inject_message(self, content: str, role: str = "user") -> bool:
|
||||
"""Inject a message into the active conversation.
|
||||
|
||||
If the agent is idle (waiting for user input), this starts a new turn.
|
||||
If the agent is running, this interrupts and injects the message.
|
||||
|
||||
This enables plugins (e.g. remote control viewers, messaging bridges)
|
||||
to send messages into the conversation from external sources.
|
||||
|
||||
Returns True if the message was queued successfully.
|
||||
"""
|
||||
cli = self._manager._cli_ref
|
||||
if cli is None:
|
||||
logger.warning("inject_message: no CLI reference (not available in gateway mode)")
|
||||
return False
|
||||
|
||||
msg = content if role == "user" else f"[{role}] {content}"
|
||||
|
||||
if getattr(cli, "_agent_running", False):
|
||||
# Agent is mid-turn — interrupt with the message
|
||||
cli._interrupt_queue.put(msg)
|
||||
else:
|
||||
# Agent is idle — queue as next input
|
||||
cli._pending_input.put(msg)
|
||||
return True
|
||||
|
||||
# -- hook registration --------------------------------------------------
|
||||
|
||||
def register_hook(self, hook_name: str, callback: Callable) -> None:
|
||||
@@ -212,7 +173,6 @@ class PluginManager:
|
||||
self._hooks: Dict[str, List[Callable]] = {}
|
||||
self._plugin_tool_names: Set[str] = set()
|
||||
self._discovered: bool = False
|
||||
self._cli_ref = None # Set by CLI after plugin discovery
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# Public
|
||||
@@ -239,15 +199,8 @@ class PluginManager:
|
||||
# 3. Pip / entry-point plugins
|
||||
manifests.extend(self._scan_entry_points())
|
||||
|
||||
# Load each manifest (skip user-disabled plugins)
|
||||
disabled = _get_disabled_plugins()
|
||||
# Load each manifest
|
||||
for manifest in manifests:
|
||||
if manifest.name in disabled:
|
||||
loaded = LoadedPlugin(manifest=manifest, enabled=False)
|
||||
loaded.error = "disabled via config"
|
||||
self._plugins[manifest.name] = loaded
|
||||
logger.debug("Skipping disabled plugin '%s'", manifest.name)
|
||||
continue
|
||||
self._load_plugin(manifest)
|
||||
|
||||
if manifests:
|
||||
|
||||
+2
-153
@@ -374,73 +374,6 @@ def cmd_remove(name: str) -> None:
|
||||
_display_removed(name, plugins_dir)
|
||||
|
||||
|
||||
def _get_disabled_set() -> set:
|
||||
"""Read the disabled plugins set from config.yaml."""
|
||||
try:
|
||||
from hermes_cli.config import load_config
|
||||
config = load_config()
|
||||
disabled = config.get("plugins", {}).get("disabled", [])
|
||||
return set(disabled) if isinstance(disabled, list) else set()
|
||||
except Exception:
|
||||
return set()
|
||||
|
||||
|
||||
def _save_disabled_set(disabled: set) -> None:
|
||||
"""Write the disabled plugins list to config.yaml."""
|
||||
from hermes_cli.config import load_config, save_config
|
||||
config = load_config()
|
||||
if "plugins" not in config:
|
||||
config["plugins"] = {}
|
||||
config["plugins"]["disabled"] = sorted(disabled)
|
||||
save_config(config)
|
||||
|
||||
|
||||
def cmd_enable(name: str) -> None:
|
||||
"""Enable a previously disabled plugin."""
|
||||
from rich.console import Console
|
||||
|
||||
console = Console()
|
||||
plugins_dir = _plugins_dir()
|
||||
|
||||
# Verify the plugin exists
|
||||
target = plugins_dir / name
|
||||
if not target.is_dir():
|
||||
console.print(f"[red]Plugin '{name}' is not installed.[/red]")
|
||||
sys.exit(1)
|
||||
|
||||
disabled = _get_disabled_set()
|
||||
if name not in disabled:
|
||||
console.print(f"[dim]Plugin '{name}' is already enabled.[/dim]")
|
||||
return
|
||||
|
||||
disabled.discard(name)
|
||||
_save_disabled_set(disabled)
|
||||
console.print(f"[green]✓[/green] Plugin [bold]{name}[/bold] enabled. Takes effect on next session.")
|
||||
|
||||
|
||||
def cmd_disable(name: str) -> None:
|
||||
"""Disable a plugin without removing it."""
|
||||
from rich.console import Console
|
||||
|
||||
console = Console()
|
||||
plugins_dir = _plugins_dir()
|
||||
|
||||
# Verify the plugin exists
|
||||
target = plugins_dir / name
|
||||
if not target.is_dir():
|
||||
console.print(f"[red]Plugin '{name}' is not installed.[/red]")
|
||||
sys.exit(1)
|
||||
|
||||
disabled = _get_disabled_set()
|
||||
if name in disabled:
|
||||
console.print(f"[dim]Plugin '{name}' is already disabled.[/dim]")
|
||||
return
|
||||
|
||||
disabled.add(name)
|
||||
_save_disabled_set(disabled)
|
||||
console.print(f"[yellow]⊘[/yellow] Plugin [bold]{name}[/bold] disabled. Takes effect on next session.")
|
||||
|
||||
|
||||
def cmd_list() -> None:
|
||||
"""List installed plugins."""
|
||||
from rich.console import Console
|
||||
@@ -460,11 +393,8 @@ def cmd_list() -> None:
|
||||
console.print("[dim]Install with:[/dim] hermes plugins install owner/repo")
|
||||
return
|
||||
|
||||
disabled = _get_disabled_set()
|
||||
|
||||
table = Table(title="Installed Plugins", show_lines=False)
|
||||
table.add_column("Name", style="bold")
|
||||
table.add_column("Status")
|
||||
table.add_column("Version", style="dim")
|
||||
table.add_column("Description")
|
||||
table.add_column("Source", style="dim")
|
||||
@@ -490,86 +420,11 @@ def cmd_list() -> None:
|
||||
if (d / ".git").exists():
|
||||
source = "git"
|
||||
|
||||
is_disabled = name in disabled or d.name in disabled
|
||||
status = "[red]disabled[/red]" if is_disabled else "[green]enabled[/green]"
|
||||
table.add_row(name, status, str(version), description, source)
|
||||
table.add_row(name, str(version), description, source)
|
||||
|
||||
console.print()
|
||||
console.print(table)
|
||||
console.print()
|
||||
console.print("[dim]Interactive toggle:[/dim] hermes plugins")
|
||||
console.print("[dim]Enable/disable:[/dim] hermes plugins enable/disable <name>")
|
||||
|
||||
|
||||
def cmd_toggle() -> None:
|
||||
"""Interactive curses checklist to enable/disable installed plugins."""
|
||||
from rich.console import Console
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
yaml = None
|
||||
|
||||
console = Console()
|
||||
plugins_dir = _plugins_dir()
|
||||
|
||||
dirs = sorted(d for d in plugins_dir.iterdir() if d.is_dir())
|
||||
if not dirs:
|
||||
console.print("[dim]No plugins installed.[/dim]")
|
||||
console.print("[dim]Install with:[/dim] hermes plugins install owner/repo")
|
||||
return
|
||||
|
||||
disabled = _get_disabled_set()
|
||||
|
||||
# Build items list: "name — description" for display
|
||||
names = []
|
||||
labels = []
|
||||
selected = set()
|
||||
|
||||
for i, d in enumerate(dirs):
|
||||
manifest_file = d / "plugin.yaml"
|
||||
name = d.name
|
||||
description = ""
|
||||
|
||||
if manifest_file.exists() and yaml:
|
||||
try:
|
||||
with open(manifest_file) as f:
|
||||
manifest = yaml.safe_load(f) or {}
|
||||
name = manifest.get("name", d.name)
|
||||
description = manifest.get("description", "")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
names.append(name)
|
||||
label = f"{name} — {description}" if description else name
|
||||
labels.append(label)
|
||||
|
||||
if name not in disabled and d.name not in disabled:
|
||||
selected.add(i)
|
||||
|
||||
from hermes_cli.curses_ui import curses_checklist
|
||||
|
||||
result = curses_checklist(
|
||||
title="Plugins — toggle enabled/disabled",
|
||||
items=labels,
|
||||
selected=selected,
|
||||
)
|
||||
|
||||
# Compute new disabled set from deselected items
|
||||
new_disabled = set()
|
||||
for i, name in enumerate(names):
|
||||
if i not in result:
|
||||
new_disabled.add(name)
|
||||
|
||||
if new_disabled != disabled:
|
||||
_save_disabled_set(new_disabled)
|
||||
enabled_count = len(names) - len(new_disabled)
|
||||
console.print(
|
||||
f"\n[green]✓[/green] {enabled_count} enabled, {len(new_disabled)} disabled. "
|
||||
f"Takes effect on next session."
|
||||
)
|
||||
else:
|
||||
console.print("\n[dim]No changes.[/dim]")
|
||||
|
||||
|
||||
def plugins_command(args) -> None:
|
||||
@@ -582,14 +437,8 @@ def plugins_command(args) -> None:
|
||||
cmd_update(args.name)
|
||||
elif action in ("remove", "rm", "uninstall"):
|
||||
cmd_remove(args.name)
|
||||
elif action == "enable":
|
||||
cmd_enable(args.name)
|
||||
elif action == "disable":
|
||||
cmd_disable(args.name)
|
||||
elif action in ("list", "ls"):
|
||||
elif action in ("list", "ls") or action is None:
|
||||
cmd_list()
|
||||
elif action is None:
|
||||
cmd_toggle()
|
||||
else:
|
||||
from rich.console import Console
|
||||
|
||||
|
||||
+10
-40
@@ -289,7 +289,7 @@ from hermes_cli.config import (
|
||||
get_env_value,
|
||||
ensure_hermes_home,
|
||||
)
|
||||
# display_hermes_home imported lazily at call sites (stale-module safety during hermes update)
|
||||
from hermes_constants import display_hermes_home
|
||||
|
||||
from hermes_cli.colors import Colors, color
|
||||
|
||||
@@ -684,8 +684,7 @@ def _print_setup_summary(config: dict, hermes_home):
|
||||
print_warning(
|
||||
"Some tools are disabled. Run 'hermes setup tools' to configure them,"
|
||||
)
|
||||
from hermes_constants import display_hermes_home as _dhh
|
||||
print_warning(f"or edit {_dhh()}/.env directly to add the missing API keys.")
|
||||
print_warning(f"or edit {display_hermes_home()}/.env directly to add the missing API keys.")
|
||||
print()
|
||||
|
||||
# Done banner
|
||||
@@ -708,8 +707,7 @@ def _print_setup_summary(config: dict, hermes_home):
|
||||
print()
|
||||
|
||||
# Show file locations prominently
|
||||
from hermes_constants import display_hermes_home as _dhh
|
||||
print(color(f"📁 All your files are in {_dhh()}/:", Colors.CYAN, Colors.BOLD))
|
||||
print(color(f"📁 All your files are in {display_hermes_home()}/:", Colors.CYAN, Colors.BOLD))
|
||||
print()
|
||||
print(f" {color('Settings:', Colors.YELLOW)} {get_config_path()}")
|
||||
print(f" {color('API Keys:', Colors.YELLOW)} {get_env_path()}")
|
||||
@@ -1002,9 +1000,10 @@ def setup_model_provider(config: dict):
|
||||
min_key_ttl_seconds=5 * 60,
|
||||
timeout_seconds=15.0,
|
||||
)
|
||||
# Use curated model list instead of full /models dump
|
||||
from hermes_cli.models import _PROVIDER_MODELS
|
||||
nous_models = _PROVIDER_MODELS.get("nous", [])
|
||||
nous_models = fetch_nous_models(
|
||||
inference_base_url=creds.get("base_url", ""),
|
||||
api_key=creds.get("api_key", ""),
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug("Could not fetch Nous models after login: %s", e)
|
||||
|
||||
@@ -2709,38 +2708,10 @@ def setup_gateway(config: dict):
|
||||
if token or get_env_value("MATRIX_PASSWORD"):
|
||||
# E2EE
|
||||
print()
|
||||
want_e2ee = prompt_yes_no("Enable end-to-end encryption (E2EE)?", False)
|
||||
if want_e2ee:
|
||||
if prompt_yes_no("Enable end-to-end encryption (E2EE)?", False):
|
||||
save_env_value("MATRIX_ENCRYPTION", "true")
|
||||
print_success("E2EE enabled")
|
||||
|
||||
# Auto-install matrix-nio
|
||||
matrix_pkg = "matrix-nio[e2e]" if want_e2ee else "matrix-nio"
|
||||
try:
|
||||
__import__("nio")
|
||||
except ImportError:
|
||||
print_info(f"Installing {matrix_pkg}...")
|
||||
import subprocess
|
||||
|
||||
uv_bin = shutil.which("uv")
|
||||
if uv_bin:
|
||||
result = subprocess.run(
|
||||
[uv_bin, "pip", "install", "--python", sys.executable, matrix_pkg],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
else:
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "pip", "install", matrix_pkg],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
print_success(f"{matrix_pkg} installed")
|
||||
else:
|
||||
print_warning(f"Install failed — run manually: pip install '{matrix_pkg}'")
|
||||
if result.stderr:
|
||||
print_info(f" Error: {result.stderr.strip().splitlines()[-1]}")
|
||||
print_info(" Requires: pip install 'matrix-nio[e2e]'")
|
||||
|
||||
# Allowed users
|
||||
print()
|
||||
@@ -2867,8 +2838,7 @@ def setup_gateway(config: dict):
|
||||
save_env_value("WEBHOOK_ENABLED", "true")
|
||||
print()
|
||||
print_success("Webhooks enabled! Next steps:")
|
||||
from hermes_constants import display_hermes_home as _dhh
|
||||
print_info(f" 1. Define webhook routes in {_dhh()}/config.yaml")
|
||||
print_info(f" 1. Define webhook routes in {display_hermes_home()}/config.yaml")
|
||||
print_info(" 2. Point your service (GitHub, GitLab, etc.) at:")
|
||||
print_info(" http://your-server:8644/webhooks/<route-name>")
|
||||
print()
|
||||
|
||||
@@ -28,8 +28,6 @@ PLATFORMS = {
|
||||
"mattermost": "💬 Mattermost",
|
||||
"matrix": "💬 Matrix",
|
||||
"dingtalk": "💬 DingTalk",
|
||||
"feishu": "🪽 Feishu",
|
||||
"wecom": "💬 WeCom",
|
||||
}
|
||||
|
||||
# ─── Config Helpers ───────────────────────────────────────────────────────────
|
||||
|
||||
@@ -354,14 +354,7 @@ def do_install(identifier: str, category: str = "", force: bool = False,
|
||||
extra_metadata.update(getattr(bundle, "metadata", {}) or {})
|
||||
|
||||
# Quarantine the bundle
|
||||
try:
|
||||
q_path = quarantine_bundle(bundle)
|
||||
except ValueError as exc:
|
||||
c.print(f"[bold red]Installation blocked:[/] {exc}\n")
|
||||
from tools.skills_hub import append_audit_log
|
||||
append_audit_log("BLOCKED", bundle.name, bundle.source,
|
||||
bundle.trust_level, "invalid_path", str(exc))
|
||||
return
|
||||
q_path = quarantine_bundle(bundle)
|
||||
c.print(f"[dim]Quarantined to {q_path.relative_to(q_path.parent.parent.parent)}[/]")
|
||||
|
||||
# Scan
|
||||
@@ -421,15 +414,7 @@ def do_install(identifier: str, category: str = "", force: bool = False,
|
||||
return
|
||||
|
||||
# Install
|
||||
try:
|
||||
install_dir = install_from_quarantine(q_path, bundle.name, category, bundle, result)
|
||||
except ValueError as exc:
|
||||
c.print(f"[bold red]Installation blocked:[/] {exc}\n")
|
||||
shutil.rmtree(q_path, ignore_errors=True)
|
||||
from tools.skills_hub import append_audit_log
|
||||
append_audit_log("BLOCKED", bundle.name, bundle.source,
|
||||
bundle.trust_level, "invalid_path", str(exc))
|
||||
return
|
||||
install_dir = install_from_quarantine(q_path, bundle.name, category, bundle, result)
|
||||
from tools.skills_hub import SKILLS_DIR
|
||||
c.print(f"[bold green]Installed:[/] {install_dir.relative_to(SKILLS_DIR)}")
|
||||
c.print(f"[dim]Files: {', '.join(bundle.files.keys())}[/]\n")
|
||||
|
||||
@@ -254,9 +254,6 @@ def show_status(args):
|
||||
"Slack": ("SLACK_BOT_TOKEN", None),
|
||||
"Email": ("EMAIL_ADDRESS", "EMAIL_HOME_ADDRESS"),
|
||||
"SMS": ("TWILIO_ACCOUNT_SID", "SMS_HOME_CHANNEL"),
|
||||
"DingTalk": ("DINGTALK_CLIENT_ID", None),
|
||||
"Feishu": ("FEISHU_APP_ID", "FEISHU_HOME_CHANNEL"),
|
||||
"WeCom": ("WECOM_BOT_ID", "WECOM_HOME_CHANNEL"),
|
||||
}
|
||||
|
||||
for name, (token_var, home_var) in platforms.items():
|
||||
|
||||
@@ -9,8 +9,6 @@ Saves per-platform tool configuration to ~/.hermes/config.yaml under
|
||||
the `platform_toolsets` key.
|
||||
"""
|
||||
|
||||
import json as _json
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Set
|
||||
@@ -21,8 +19,6 @@ from hermes_cli.config import (
|
||||
)
|
||||
from hermes_cli.colors import Colors, color
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
PROJECT_ROOT = Path(__file__).parent.parent.resolve()
|
||||
|
||||
|
||||
@@ -140,9 +136,7 @@ PLATFORMS = {
|
||||
"homeassistant": {"label": "🏠 Home Assistant", "default_toolset": "hermes-homeassistant"},
|
||||
"email": {"label": "📧 Email", "default_toolset": "hermes-email"},
|
||||
"matrix": {"label": "💬 Matrix", "default_toolset": "hermes-matrix"},
|
||||
"dingtalk": {"label": "💬 DingTalk", "default_toolset": "hermes-dingtalk"},
|
||||
"feishu": {"label": "🪽 Feishu", "default_toolset": "hermes-feishu"},
|
||||
"wecom": {"label": "💬 WeCom", "default_toolset": "hermes-wecom"},
|
||||
"dingtalk": {"label": "💬 DingTalk", "default_toolset": "hermes-dingtalk"},
|
||||
"api_server": {"label": "🌐 API Server", "default_toolset": "hermes-api-server"},
|
||||
"mattermost": {"label": "💬 Mattermost", "default_toolset": "hermes-mattermost"},
|
||||
}
|
||||
@@ -659,61 +653,9 @@ def _prompt_choice(question: str, choices: list, default: int = 0) -> int:
|
||||
return default
|
||||
|
||||
|
||||
# ─── Token Estimation ────────────────────────────────────────────────────────
|
||||
|
||||
# Module-level cache so discovery + tokenization runs at most once per process.
|
||||
_tool_token_cache: Optional[Dict[str, int]] = None
|
||||
|
||||
|
||||
def _estimate_tool_tokens() -> Dict[str, int]:
|
||||
"""Return estimated token counts per individual tool name.
|
||||
|
||||
Uses tiktoken (cl100k_base) to count tokens in the JSON-serialised
|
||||
OpenAI-format tool schema. Triggers tool discovery on first call,
|
||||
then caches the result for the rest of the process.
|
||||
|
||||
Returns an empty dict when tiktoken or the registry is unavailable.
|
||||
"""
|
||||
global _tool_token_cache
|
||||
if _tool_token_cache is not None:
|
||||
return _tool_token_cache
|
||||
|
||||
try:
|
||||
import tiktoken
|
||||
enc = tiktoken.get_encoding("cl100k_base")
|
||||
except Exception:
|
||||
logger.debug("tiktoken unavailable; skipping tool token estimation")
|
||||
_tool_token_cache = {}
|
||||
return _tool_token_cache
|
||||
|
||||
try:
|
||||
# Trigger full tool discovery (imports all tool modules).
|
||||
import model_tools # noqa: F401
|
||||
from tools.registry import registry
|
||||
except Exception:
|
||||
logger.debug("Tool registry unavailable; skipping token estimation")
|
||||
_tool_token_cache = {}
|
||||
return _tool_token_cache
|
||||
|
||||
counts: Dict[str, int] = {}
|
||||
for name in registry.get_all_tool_names():
|
||||
schema = registry.get_schema(name)
|
||||
if schema:
|
||||
# Mirror what gets sent to the API:
|
||||
# {"type": "function", "function": <schema>}
|
||||
text = _json.dumps({"type": "function", "function": schema})
|
||||
counts[name] = len(enc.encode(text))
|
||||
_tool_token_cache = counts
|
||||
return _tool_token_cache
|
||||
|
||||
|
||||
def _prompt_toolset_checklist(platform_label: str, enabled: Set[str]) -> Set[str]:
|
||||
"""Multi-select checklist of toolsets. Returns set of selected toolset keys."""
|
||||
from hermes_cli.curses_ui import curses_checklist
|
||||
from toolsets import resolve_toolset
|
||||
|
||||
# Pre-compute per-tool token counts (cached after first call).
|
||||
tool_tokens = _estimate_tool_tokens()
|
||||
|
||||
effective = _get_effective_configurable_toolsets()
|
||||
|
||||
@@ -729,27 +671,11 @@ def _prompt_toolset_checklist(platform_label: str, enabled: Set[str]) -> Set[str
|
||||
if ts_key in enabled
|
||||
}
|
||||
|
||||
# Build a live status function that shows deduplicated total token cost.
|
||||
status_fn = None
|
||||
if tool_tokens:
|
||||
ts_keys = [ts_key for ts_key, _, _ in effective]
|
||||
|
||||
def status_fn(chosen: set) -> str:
|
||||
# Collect unique tool names across all selected toolsets
|
||||
all_tools: set = set()
|
||||
for idx in chosen:
|
||||
all_tools.update(resolve_toolset(ts_keys[idx]))
|
||||
total = sum(tool_tokens.get(name, 0) for name in all_tools)
|
||||
if total >= 1000:
|
||||
return f"Est. tool context: ~{total / 1000:.1f}k tokens"
|
||||
return f"Est. tool context: ~{total} tokens"
|
||||
|
||||
chosen = curses_checklist(
|
||||
f"Tools for {platform_label}",
|
||||
labels,
|
||||
pre_selected,
|
||||
cancel_returns=pre_selected,
|
||||
status_fn=status_fn,
|
||||
)
|
||||
return {effective[i][0] for i in chosen}
|
||||
|
||||
|
||||
-868
@@ -1,868 +0,0 @@
|
||||
"""
|
||||
Hermes MCP Server — expose messaging conversations as MCP tools.
|
||||
|
||||
Starts a stdio MCP server that lets any MCP client (Claude Code, Cursor, Codex,
|
||||
etc.) list conversations, read message history, send messages, poll for live
|
||||
events, and manage approval requests across all connected platforms.
|
||||
|
||||
Matches OpenClaw's 9-tool MCP channel bridge surface:
|
||||
conversations_list, conversation_get, messages_read, attachments_fetch,
|
||||
events_poll, events_wait, messages_send, permissions_list_open,
|
||||
permissions_respond
|
||||
|
||||
Plus: channels_list (Hermes-specific extra)
|
||||
|
||||
Usage:
|
||||
hermes mcp serve
|
||||
hermes mcp serve --verbose
|
||||
|
||||
MCP client config (e.g. claude_desktop_config.json):
|
||||
{
|
||||
"mcpServers": {
|
||||
"hermes": {
|
||||
"command": "hermes",
|
||||
"args": ["mcp", "serve"]
|
||||
}
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
logger = logging.getLogger("hermes.mcp_serve")
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Lazy MCP SDK import
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_MCP_SERVER_AVAILABLE = False
|
||||
try:
|
||||
from mcp.server.fastmcp import FastMCP
|
||||
|
||||
_MCP_SERVER_AVAILABLE = True
|
||||
except ImportError:
|
||||
FastMCP = None # type: ignore[assignment,misc]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _get_sessions_dir() -> Path:
|
||||
"""Return the sessions directory using HERMES_HOME."""
|
||||
try:
|
||||
from hermes_constants import get_hermes_home
|
||||
return get_hermes_home() / "sessions"
|
||||
except ImportError:
|
||||
return Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes")) / "sessions"
|
||||
|
||||
|
||||
def _get_session_db():
|
||||
"""Get a SessionDB instance for reading message transcripts."""
|
||||
try:
|
||||
from hermes_state import SessionDB
|
||||
return SessionDB()
|
||||
except Exception as e:
|
||||
logger.debug("SessionDB unavailable: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
def _load_sessions_index() -> dict:
|
||||
"""Load the gateway sessions.json index directly.
|
||||
|
||||
Returns a dict of session_key -> entry_dict with platform routing info.
|
||||
This avoids importing the full SessionStore which needs GatewayConfig.
|
||||
"""
|
||||
sessions_file = _get_sessions_dir() / "sessions.json"
|
||||
if not sessions_file.exists():
|
||||
return {}
|
||||
try:
|
||||
with open(sessions_file, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
except Exception as e:
|
||||
logger.debug("Failed to load sessions.json: %s", e)
|
||||
return {}
|
||||
|
||||
|
||||
def _load_channel_directory() -> dict:
|
||||
"""Load the cached channel directory for available targets."""
|
||||
try:
|
||||
from hermes_constants import get_hermes_home
|
||||
directory_file = get_hermes_home() / "channel_directory.json"
|
||||
except ImportError:
|
||||
directory_file = Path(
|
||||
os.environ.get("HERMES_HOME", Path.home() / ".hermes")
|
||||
) / "channel_directory.json"
|
||||
|
||||
if not directory_file.exists():
|
||||
return {}
|
||||
try:
|
||||
with open(directory_file, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
except Exception as e:
|
||||
logger.debug("Failed to load channel_directory.json: %s", e)
|
||||
return {}
|
||||
|
||||
|
||||
def _extract_message_content(msg: dict) -> str:
|
||||
"""Extract text content from a message, handling multi-part content."""
|
||||
content = msg.get("content", "")
|
||||
if isinstance(content, list):
|
||||
text_parts = [
|
||||
p.get("text", "") for p in content
|
||||
if isinstance(p, dict) and p.get("type") == "text"
|
||||
]
|
||||
return "\n".join(text_parts)
|
||||
return str(content) if content else ""
|
||||
|
||||
|
||||
def _extract_attachments(msg: dict) -> List[dict]:
|
||||
"""Extract non-text attachments from a message.
|
||||
|
||||
Finds: multi-part image/file content blocks, MEDIA: tags in text,
|
||||
image URLs, and file references.
|
||||
"""
|
||||
attachments = []
|
||||
content = msg.get("content", "")
|
||||
|
||||
# Multi-part content blocks (image_url, file, etc.)
|
||||
if isinstance(content, list):
|
||||
for part in content:
|
||||
if not isinstance(part, dict):
|
||||
continue
|
||||
ptype = part.get("type", "")
|
||||
if ptype == "image_url":
|
||||
url = part.get("image_url", {}).get("url", "") if isinstance(part.get("image_url"), dict) else ""
|
||||
if url:
|
||||
attachments.append({"type": "image", "url": url})
|
||||
elif ptype == "image":
|
||||
url = part.get("url", part.get("source", {}).get("url", ""))
|
||||
if url:
|
||||
attachments.append({"type": "image", "url": url})
|
||||
elif ptype not in ("text",):
|
||||
# Unknown non-text content type
|
||||
attachments.append({"type": ptype, "data": part})
|
||||
|
||||
# MEDIA: tags in text content
|
||||
text = _extract_message_content(msg)
|
||||
if text:
|
||||
media_pattern = re.compile(r'MEDIA:\s*(\S+)')
|
||||
for match in media_pattern.finditer(text):
|
||||
path = match.group(1)
|
||||
attachments.append({"type": "media", "path": path})
|
||||
|
||||
return attachments
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Event Bridge — polls SessionDB for new messages, maintains event queue
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
QUEUE_LIMIT = 1000
|
||||
POLL_INTERVAL = 0.2 # seconds between DB polls (200ms)
|
||||
|
||||
|
||||
@dataclass
|
||||
class QueueEvent:
|
||||
"""An event in the bridge's in-memory queue."""
|
||||
cursor: int
|
||||
type: str # "message", "approval_requested", "approval_resolved"
|
||||
session_key: str = ""
|
||||
data: dict = field(default_factory=dict)
|
||||
|
||||
|
||||
class EventBridge:
|
||||
"""Background poller that watches SessionDB for new messages and
|
||||
maintains an in-memory event queue with waiter support.
|
||||
|
||||
This is the Hermes equivalent of OpenClaw's WebSocket gateway bridge.
|
||||
Instead of WebSocket events, we poll the SQLite database for changes.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self._queue: List[QueueEvent] = []
|
||||
self._cursor = 0
|
||||
self._lock = threading.Lock()
|
||||
self._new_event = threading.Event()
|
||||
self._running = False
|
||||
self._thread: Optional[threading.Thread] = None
|
||||
self._last_poll_timestamps: Dict[str, float] = {} # session_key -> unix timestamp
|
||||
# In-memory approval tracking (populated from events)
|
||||
self._pending_approvals: Dict[str, dict] = {}
|
||||
# mtime cache — skip expensive work when files haven't changed
|
||||
self._sessions_json_mtime: float = 0.0
|
||||
self._state_db_mtime: float = 0.0
|
||||
self._cached_sessions_index: dict = {}
|
||||
|
||||
def start(self):
|
||||
"""Start the background polling thread."""
|
||||
if self._running:
|
||||
return
|
||||
self._running = True
|
||||
self._thread = threading.Thread(target=self._poll_loop, daemon=True)
|
||||
self._thread.start()
|
||||
logger.debug("EventBridge started")
|
||||
|
||||
def stop(self):
|
||||
"""Stop the background polling thread."""
|
||||
self._running = False
|
||||
self._new_event.set() # Wake any waiters
|
||||
if self._thread:
|
||||
self._thread.join(timeout=5)
|
||||
logger.debug("EventBridge stopped")
|
||||
|
||||
def poll_events(
|
||||
self,
|
||||
after_cursor: int = 0,
|
||||
session_key: Optional[str] = None,
|
||||
limit: int = 20,
|
||||
) -> dict:
|
||||
"""Return events since after_cursor, optionally filtered by session_key."""
|
||||
with self._lock:
|
||||
events = [
|
||||
e for e in self._queue
|
||||
if e.cursor > after_cursor
|
||||
and (not session_key or e.session_key == session_key)
|
||||
][:limit]
|
||||
|
||||
next_cursor = events[-1].cursor if events else after_cursor
|
||||
return {
|
||||
"events": [
|
||||
{"cursor": e.cursor, "type": e.type,
|
||||
"session_key": e.session_key, **e.data}
|
||||
for e in events
|
||||
],
|
||||
"next_cursor": next_cursor,
|
||||
}
|
||||
|
||||
def wait_for_event(
|
||||
self,
|
||||
after_cursor: int = 0,
|
||||
session_key: Optional[str] = None,
|
||||
timeout_ms: int = 30000,
|
||||
) -> Optional[dict]:
|
||||
"""Block until a matching event arrives or timeout expires."""
|
||||
deadline = time.monotonic() + (timeout_ms / 1000.0)
|
||||
|
||||
while time.monotonic() < deadline:
|
||||
with self._lock:
|
||||
for e in self._queue:
|
||||
if e.cursor > after_cursor and (
|
||||
not session_key or e.session_key == session_key
|
||||
):
|
||||
return {
|
||||
"cursor": e.cursor, "type": e.type,
|
||||
"session_key": e.session_key, **e.data,
|
||||
}
|
||||
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
break
|
||||
self._new_event.clear()
|
||||
self._new_event.wait(timeout=min(remaining, POLL_INTERVAL))
|
||||
|
||||
return None
|
||||
|
||||
def list_pending_approvals(self) -> List[dict]:
|
||||
"""List approval requests observed during this bridge session."""
|
||||
with self._lock:
|
||||
return sorted(
|
||||
self._pending_approvals.values(),
|
||||
key=lambda a: a.get("created_at", ""),
|
||||
)
|
||||
|
||||
def respond_to_approval(self, approval_id: str, decision: str) -> dict:
|
||||
"""Resolve a pending approval (best-effort without gateway IPC)."""
|
||||
with self._lock:
|
||||
approval = self._pending_approvals.pop(approval_id, None)
|
||||
|
||||
if not approval:
|
||||
return {"error": f"Approval not found: {approval_id}"}
|
||||
|
||||
self._enqueue(QueueEvent(
|
||||
cursor=0, # Will be set by _enqueue
|
||||
type="approval_resolved",
|
||||
session_key=approval.get("session_key", ""),
|
||||
data={"approval_id": approval_id, "decision": decision},
|
||||
))
|
||||
|
||||
return {"resolved": True, "approval_id": approval_id, "decision": decision}
|
||||
|
||||
def _enqueue(self, event: QueueEvent) -> None:
|
||||
"""Add an event to the queue and wake any waiters."""
|
||||
with self._lock:
|
||||
self._cursor += 1
|
||||
event.cursor = self._cursor
|
||||
self._queue.append(event)
|
||||
# Trim queue to limit
|
||||
while len(self._queue) > QUEUE_LIMIT:
|
||||
self._queue.pop(0)
|
||||
self._new_event.set()
|
||||
|
||||
def _poll_loop(self):
|
||||
"""Background loop: poll SessionDB for new messages."""
|
||||
db = _get_session_db()
|
||||
if not db:
|
||||
logger.warning("EventBridge: SessionDB unavailable, event polling disabled")
|
||||
return
|
||||
|
||||
while self._running:
|
||||
try:
|
||||
self._poll_once(db)
|
||||
except Exception as e:
|
||||
logger.debug("EventBridge poll error: %s", e)
|
||||
time.sleep(POLL_INTERVAL)
|
||||
|
||||
def _poll_once(self, db):
|
||||
"""Check for new messages across all sessions.
|
||||
|
||||
Uses mtime checks on sessions.json and state.db to skip work
|
||||
when nothing has changed — makes 200ms polling essentially free.
|
||||
"""
|
||||
# Check if sessions.json has changed (mtime check is ~1μs)
|
||||
sessions_file = _get_sessions_dir() / "sessions.json"
|
||||
try:
|
||||
sj_mtime = sessions_file.stat().st_mtime if sessions_file.exists() else 0.0
|
||||
except OSError:
|
||||
sj_mtime = 0.0
|
||||
|
||||
if sj_mtime != self._sessions_json_mtime:
|
||||
self._sessions_json_mtime = sj_mtime
|
||||
self._cached_sessions_index = _load_sessions_index()
|
||||
|
||||
# Check if state.db has changed
|
||||
try:
|
||||
from hermes_constants import get_hermes_home
|
||||
db_file = get_hermes_home() / "state.db"
|
||||
except ImportError:
|
||||
db_file = Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes")) / "state.db"
|
||||
|
||||
try:
|
||||
db_mtime = db_file.stat().st_mtime if db_file.exists() else 0.0
|
||||
except OSError:
|
||||
db_mtime = 0.0
|
||||
|
||||
if db_mtime == self._state_db_mtime and sj_mtime == self._sessions_json_mtime:
|
||||
return # Nothing changed since last poll — skip entirely
|
||||
|
||||
self._state_db_mtime = db_mtime
|
||||
entries = self._cached_sessions_index
|
||||
|
||||
for session_key, entry in entries.items():
|
||||
session_id = entry.get("session_id", "")
|
||||
if not session_id:
|
||||
continue
|
||||
|
||||
last_seen = self._last_poll_timestamps.get(session_key, 0.0)
|
||||
|
||||
try:
|
||||
messages = db.get_messages(session_id)
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
if not messages:
|
||||
continue
|
||||
|
||||
# Normalize timestamps to float for comparison
|
||||
def _ts_float(ts) -> float:
|
||||
if isinstance(ts, (int, float)):
|
||||
return float(ts)
|
||||
if isinstance(ts, str) and ts:
|
||||
try:
|
||||
return float(ts)
|
||||
except ValueError:
|
||||
# ISO string — parse to epoch
|
||||
try:
|
||||
from datetime import datetime
|
||||
return datetime.fromisoformat(ts).timestamp()
|
||||
except Exception:
|
||||
return 0.0
|
||||
return 0.0
|
||||
|
||||
# Find messages newer than our last seen timestamp
|
||||
new_messages = []
|
||||
for msg in messages:
|
||||
ts = _ts_float(msg.get("timestamp", 0))
|
||||
role = msg.get("role", "")
|
||||
if role not in ("user", "assistant"):
|
||||
continue
|
||||
if ts > last_seen:
|
||||
new_messages.append(msg)
|
||||
|
||||
for msg in new_messages:
|
||||
content = _extract_message_content(msg)
|
||||
if not content:
|
||||
continue
|
||||
self._enqueue(QueueEvent(
|
||||
cursor=0,
|
||||
type="message",
|
||||
session_key=session_key,
|
||||
data={
|
||||
"role": msg.get("role", ""),
|
||||
"content": content[:500],
|
||||
"timestamp": str(msg.get("timestamp", "")),
|
||||
"message_id": str(msg.get("id", "")),
|
||||
},
|
||||
))
|
||||
|
||||
# Update last seen to the most recent message timestamp
|
||||
all_ts = [_ts_float(m.get("timestamp", 0)) for m in messages]
|
||||
if all_ts:
|
||||
latest = max(all_ts)
|
||||
if latest > last_seen:
|
||||
self._last_poll_timestamps[session_key] = latest
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# MCP Server
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
||||
"""Create and return the Hermes MCP server with all tools registered."""
|
||||
if not _MCP_SERVER_AVAILABLE:
|
||||
raise ImportError(
|
||||
"MCP server requires the 'mcp' package. "
|
||||
"Install with: pip install 'hermes-agent[mcp]'"
|
||||
)
|
||||
|
||||
mcp = FastMCP(
|
||||
"hermes",
|
||||
instructions=(
|
||||
"Hermes Agent messaging bridge. Use these tools to interact with "
|
||||
"conversations across Telegram, Discord, Slack, WhatsApp, Signal, "
|
||||
"Matrix, and other connected platforms."
|
||||
),
|
||||
)
|
||||
|
||||
bridge = event_bridge or EventBridge()
|
||||
|
||||
# -- conversations_list ------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def conversations_list(
|
||||
platform: Optional[str] = None,
|
||||
limit: int = 50,
|
||||
search: Optional[str] = None,
|
||||
) -> str:
|
||||
"""List active messaging conversations across connected platforms.
|
||||
|
||||
Returns conversations with their session keys (needed for messages_read),
|
||||
platform, chat type, display name, and last activity time.
|
||||
|
||||
Args:
|
||||
platform: Filter by platform name (telegram, discord, slack, etc.)
|
||||
limit: Maximum number of conversations to return (default 50)
|
||||
search: Optional text to filter conversations by name
|
||||
"""
|
||||
entries = _load_sessions_index()
|
||||
conversations = []
|
||||
|
||||
for key, entry in entries.items():
|
||||
origin = entry.get("origin", {})
|
||||
entry_platform = entry.get("platform") or origin.get("platform", "")
|
||||
|
||||
if platform and entry_platform.lower() != platform.lower():
|
||||
continue
|
||||
|
||||
display_name = entry.get("display_name", "")
|
||||
chat_name = origin.get("chat_name", "")
|
||||
if search:
|
||||
search_lower = search.lower()
|
||||
if (search_lower not in display_name.lower()
|
||||
and search_lower not in chat_name.lower()
|
||||
and search_lower not in key.lower()):
|
||||
continue
|
||||
|
||||
conversations.append({
|
||||
"session_key": key,
|
||||
"session_id": entry.get("session_id", ""),
|
||||
"platform": entry_platform,
|
||||
"chat_type": entry.get("chat_type", origin.get("chat_type", "")),
|
||||
"display_name": display_name,
|
||||
"chat_name": chat_name,
|
||||
"user_name": origin.get("user_name", ""),
|
||||
"updated_at": entry.get("updated_at", ""),
|
||||
})
|
||||
|
||||
conversations.sort(key=lambda c: c.get("updated_at", ""), reverse=True)
|
||||
conversations = conversations[:limit]
|
||||
|
||||
return json.dumps({
|
||||
"count": len(conversations),
|
||||
"conversations": conversations,
|
||||
}, indent=2)
|
||||
|
||||
# -- conversation_get --------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def conversation_get(session_key: str) -> str:
|
||||
"""Get detailed info about one conversation by its session key.
|
||||
|
||||
Args:
|
||||
session_key: The session key from conversations_list
|
||||
"""
|
||||
entries = _load_sessions_index()
|
||||
entry = entries.get(session_key)
|
||||
|
||||
if not entry:
|
||||
return json.dumps({"error": f"Conversation not found: {session_key}"})
|
||||
|
||||
origin = entry.get("origin", {})
|
||||
return json.dumps({
|
||||
"session_key": session_key,
|
||||
"session_id": entry.get("session_id", ""),
|
||||
"platform": entry.get("platform") or origin.get("platform", ""),
|
||||
"chat_type": entry.get("chat_type", origin.get("chat_type", "")),
|
||||
"display_name": entry.get("display_name", ""),
|
||||
"user_name": origin.get("user_name", ""),
|
||||
"chat_name": origin.get("chat_name", ""),
|
||||
"chat_id": origin.get("chat_id", ""),
|
||||
"thread_id": origin.get("thread_id"),
|
||||
"updated_at": entry.get("updated_at", ""),
|
||||
"created_at": entry.get("created_at", ""),
|
||||
"input_tokens": entry.get("input_tokens", 0),
|
||||
"output_tokens": entry.get("output_tokens", 0),
|
||||
"total_tokens": entry.get("total_tokens", 0),
|
||||
}, indent=2)
|
||||
|
||||
# -- messages_read -----------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def messages_read(
|
||||
session_key: str,
|
||||
limit: int = 50,
|
||||
) -> str:
|
||||
"""Read recent messages from a conversation.
|
||||
|
||||
Returns the message history in chronological order with role, content,
|
||||
and timestamp for each message.
|
||||
|
||||
Args:
|
||||
session_key: The session key from conversations_list
|
||||
limit: Maximum number of messages to return (default 50, most recent)
|
||||
"""
|
||||
entries = _load_sessions_index()
|
||||
entry = entries.get(session_key)
|
||||
if not entry:
|
||||
return json.dumps({"error": f"Conversation not found: {session_key}"})
|
||||
|
||||
session_id = entry.get("session_id", "")
|
||||
if not session_id:
|
||||
return json.dumps({"error": "No session ID for this conversation"})
|
||||
|
||||
db = _get_session_db()
|
||||
if not db:
|
||||
return json.dumps({"error": "Session database unavailable"})
|
||||
|
||||
try:
|
||||
all_messages = db.get_messages(session_id)
|
||||
except Exception as e:
|
||||
return json.dumps({"error": f"Failed to read messages: {e}"})
|
||||
|
||||
filtered = []
|
||||
for msg in all_messages:
|
||||
role = msg.get("role", "")
|
||||
if role in ("user", "assistant"):
|
||||
content = _extract_message_content(msg)
|
||||
if content:
|
||||
filtered.append({
|
||||
"id": str(msg.get("id", "")),
|
||||
"role": role,
|
||||
"content": content[:2000],
|
||||
"timestamp": msg.get("timestamp", ""),
|
||||
})
|
||||
|
||||
messages = filtered[-limit:]
|
||||
|
||||
return json.dumps({
|
||||
"session_key": session_key,
|
||||
"count": len(messages),
|
||||
"total_in_session": len(filtered),
|
||||
"messages": messages,
|
||||
}, indent=2)
|
||||
|
||||
# -- attachments_fetch -------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def attachments_fetch(
|
||||
session_key: str,
|
||||
message_id: str,
|
||||
) -> str:
|
||||
"""List non-text attachments for a message in a conversation.
|
||||
|
||||
Extracts images, media files, and other non-text content blocks
|
||||
from the specified message.
|
||||
|
||||
Args:
|
||||
session_key: The session key from conversations_list
|
||||
message_id: The message ID from messages_read
|
||||
"""
|
||||
entries = _load_sessions_index()
|
||||
entry = entries.get(session_key)
|
||||
if not entry:
|
||||
return json.dumps({"error": f"Conversation not found: {session_key}"})
|
||||
|
||||
session_id = entry.get("session_id", "")
|
||||
if not session_id:
|
||||
return json.dumps({"error": "No session ID for this conversation"})
|
||||
|
||||
db = _get_session_db()
|
||||
if not db:
|
||||
return json.dumps({"error": "Session database unavailable"})
|
||||
|
||||
try:
|
||||
all_messages = db.get_messages(session_id)
|
||||
except Exception as e:
|
||||
return json.dumps({"error": f"Failed to read messages: {e}"})
|
||||
|
||||
# Find the target message
|
||||
target_msg = None
|
||||
for msg in all_messages:
|
||||
if str(msg.get("id", "")) == message_id:
|
||||
target_msg = msg
|
||||
break
|
||||
|
||||
if not target_msg:
|
||||
return json.dumps({"error": f"Message not found: {message_id}"})
|
||||
|
||||
attachments = _extract_attachments(target_msg)
|
||||
|
||||
return json.dumps({
|
||||
"message_id": message_id,
|
||||
"count": len(attachments),
|
||||
"attachments": attachments,
|
||||
}, indent=2)
|
||||
|
||||
# -- events_poll -------------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def events_poll(
|
||||
after_cursor: int = 0,
|
||||
session_key: Optional[str] = None,
|
||||
limit: int = 20,
|
||||
) -> str:
|
||||
"""Poll for new conversation events since a cursor position.
|
||||
|
||||
Returns events that have occurred since the given cursor. Use the
|
||||
returned next_cursor value for subsequent polls.
|
||||
|
||||
Event types: message, approval_requested, approval_resolved
|
||||
|
||||
Args:
|
||||
after_cursor: Return events after this cursor (0 for all)
|
||||
session_key: Optional filter to one conversation
|
||||
limit: Maximum events to return (default 20)
|
||||
"""
|
||||
result = bridge.poll_events(
|
||||
after_cursor=after_cursor,
|
||||
session_key=session_key,
|
||||
limit=limit,
|
||||
)
|
||||
return json.dumps(result, indent=2)
|
||||
|
||||
# -- events_wait -------------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def events_wait(
|
||||
after_cursor: int = 0,
|
||||
session_key: Optional[str] = None,
|
||||
timeout_ms: int = 30000,
|
||||
) -> str:
|
||||
"""Wait for the next conversation event (long-poll).
|
||||
|
||||
Blocks until a matching event arrives or the timeout expires.
|
||||
Use this for near-real-time event delivery without polling.
|
||||
|
||||
Args:
|
||||
after_cursor: Wait for events after this cursor
|
||||
session_key: Optional filter to one conversation
|
||||
timeout_ms: Maximum wait time in milliseconds (default 30000)
|
||||
"""
|
||||
event = bridge.wait_for_event(
|
||||
after_cursor=after_cursor,
|
||||
session_key=session_key,
|
||||
timeout_ms=min(timeout_ms, 300000), # Cap at 5 minutes
|
||||
)
|
||||
if event:
|
||||
return json.dumps({"event": event}, indent=2)
|
||||
return json.dumps({"event": None, "reason": "timeout"}, indent=2)
|
||||
|
||||
# -- messages_send -----------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def messages_send(
|
||||
target: str,
|
||||
message: str,
|
||||
) -> str:
|
||||
"""Send a message to a platform conversation.
|
||||
|
||||
The target format is "platform:chat_id" — same format used by the
|
||||
channels_list tool. You can also use human-friendly channel names
|
||||
that will be resolved automatically.
|
||||
|
||||
Examples:
|
||||
target="telegram:6308981865"
|
||||
target="discord:#general"
|
||||
target="slack:#engineering"
|
||||
|
||||
Args:
|
||||
target: Platform target in "platform:identifier" format
|
||||
message: The message text to send
|
||||
"""
|
||||
if not target or not message:
|
||||
return json.dumps({"error": "Both target and message are required"})
|
||||
|
||||
try:
|
||||
from tools.send_message_tool import send_message_tool
|
||||
result_str = send_message_tool(
|
||||
{"action": "send", "target": target, "message": message}
|
||||
)
|
||||
return result_str
|
||||
except ImportError:
|
||||
return json.dumps({"error": "Send message tool not available"})
|
||||
except Exception as e:
|
||||
return json.dumps({"error": f"Send failed: {e}"})
|
||||
|
||||
# -- channels_list -----------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def channels_list(platform: Optional[str] = None) -> str:
|
||||
"""List available messaging channels and targets across platforms.
|
||||
|
||||
Returns channels that you can send messages to. The target strings
|
||||
returned here can be used directly with the messages_send tool.
|
||||
|
||||
Args:
|
||||
platform: Filter by platform name (telegram, discord, slack, etc.)
|
||||
"""
|
||||
directory = _load_channel_directory()
|
||||
if not directory:
|
||||
entries = _load_sessions_index()
|
||||
targets = []
|
||||
seen = set()
|
||||
for key, entry in entries.items():
|
||||
origin = entry.get("origin", {})
|
||||
p = entry.get("platform") or origin.get("platform", "")
|
||||
chat_id = origin.get("chat_id", "")
|
||||
if not p or not chat_id:
|
||||
continue
|
||||
if platform and p.lower() != platform.lower():
|
||||
continue
|
||||
target_str = f"{p}:{chat_id}"
|
||||
if target_str in seen:
|
||||
continue
|
||||
seen.add(target_str)
|
||||
targets.append({
|
||||
"target": target_str,
|
||||
"platform": p,
|
||||
"name": entry.get("display_name") or origin.get("chat_name", ""),
|
||||
"chat_type": entry.get("chat_type", origin.get("chat_type", "")),
|
||||
})
|
||||
return json.dumps({"count": len(targets), "channels": targets}, indent=2)
|
||||
|
||||
channels = []
|
||||
for plat, entries_list in directory.items():
|
||||
if platform and plat.lower() != platform.lower():
|
||||
continue
|
||||
if isinstance(entries_list, list):
|
||||
for ch in entries_list:
|
||||
if isinstance(ch, dict):
|
||||
chat_id = ch.get("id", ch.get("chat_id", ""))
|
||||
channels.append({
|
||||
"target": f"{plat}:{chat_id}" if chat_id else plat,
|
||||
"platform": plat,
|
||||
"name": ch.get("name", ch.get("display_name", "")),
|
||||
"chat_type": ch.get("type", ""),
|
||||
})
|
||||
|
||||
return json.dumps({"count": len(channels), "channels": channels}, indent=2)
|
||||
|
||||
# -- permissions_list_open ---------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def permissions_list_open() -> str:
|
||||
"""List pending approval requests observed during this bridge session.
|
||||
|
||||
Returns exec and plugin approval requests that the bridge has seen
|
||||
since it started. Approvals are live-session only — older approvals
|
||||
from before the bridge connected are not included.
|
||||
"""
|
||||
approvals = bridge.list_pending_approvals()
|
||||
return json.dumps({
|
||||
"count": len(approvals),
|
||||
"approvals": approvals,
|
||||
}, indent=2)
|
||||
|
||||
# -- permissions_respond -----------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def permissions_respond(
|
||||
id: str,
|
||||
decision: str,
|
||||
) -> str:
|
||||
"""Respond to a pending approval request.
|
||||
|
||||
Args:
|
||||
id: The approval ID from permissions_list_open
|
||||
decision: One of "allow-once", "allow-always", or "deny"
|
||||
"""
|
||||
if decision not in ("allow-once", "allow-always", "deny"):
|
||||
return json.dumps({
|
||||
"error": f"Invalid decision: {decision}. "
|
||||
f"Must be allow-once, allow-always, or deny"
|
||||
})
|
||||
|
||||
result = bridge.respond_to_approval(id, decision)
|
||||
return json.dumps(result, indent=2)
|
||||
|
||||
return mcp
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Entry point
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def run_mcp_server(verbose: bool = False) -> None:
|
||||
"""Start the Hermes MCP server on stdio."""
|
||||
if not _MCP_SERVER_AVAILABLE:
|
||||
print(
|
||||
"Error: MCP server requires the 'mcp' package.\n"
|
||||
"Install with: pip install 'hermes-agent[mcp]'",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
if verbose:
|
||||
logging.basicConfig(level=logging.DEBUG, stream=sys.stderr)
|
||||
else:
|
||||
logging.basicConfig(level=logging.WARNING, stream=sys.stderr)
|
||||
|
||||
bridge = EventBridge()
|
||||
bridge.start()
|
||||
|
||||
server = create_mcp_server(event_bridge=bridge)
|
||||
|
||||
import asyncio
|
||||
|
||||
async def _run():
|
||||
try:
|
||||
await server.run_stdio_async()
|
||||
finally:
|
||||
bridge.stop()
|
||||
|
||||
try:
|
||||
asyncio.run(_run())
|
||||
except KeyboardInterrupt:
|
||||
bridge.stop()
|
||||
@@ -1 +0,0 @@
|
||||
Communication and decision-making frameworks — structured response formats for proposals, trade-off analysis, and stakeholder-ready recommendations.
|
||||
@@ -1,103 +0,0 @@
|
||||
---
|
||||
name: one-three-one-rule
|
||||
description: >
|
||||
Structured decision-making framework for technical proposals and trade-off analysis.
|
||||
When the user faces a choice between multiple approaches (architecture decisions,
|
||||
tool selection, refactoring strategies, migration paths), this skill produces a
|
||||
1-3-1 format: one clear problem statement, three distinct options with pros/cons,
|
||||
and one concrete recommendation with definition of done and implementation plan.
|
||||
Use when the user asks for a "1-3-1", says "give me options", or needs help
|
||||
choosing between competing approaches.
|
||||
version: 1.0.0
|
||||
author: Willard Moore
|
||||
license: MIT
|
||||
category: communication
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [communication, decision-making, proposals, trade-offs]
|
||||
---
|
||||
|
||||
# 1-3-1 Communication Rule
|
||||
|
||||
Structured decision-making format for when a task has multiple viable approaches and the user needs a clear recommendation. Produces a concise problem framing, three options with trade-offs, and an actionable plan for the recommended path.
|
||||
|
||||
## When to Use
|
||||
|
||||
- The user explicitly asks for a "1-3-1" response.
|
||||
- The user says "give me options" or "what are my choices" for a technical decision.
|
||||
- A task has multiple viable approaches with meaningful trade-offs (architecture, tooling, migration strategy).
|
||||
- The user needs a proposal they can forward to a team or stakeholder.
|
||||
|
||||
Do NOT use for simple questions with one obvious answer, debugging sessions, or tasks where the user has already decided on an approach.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Problem** (one sentence)
|
||||
- State the core decision or desired outcome in a single concise sentence.
|
||||
- Focus on the *what*, not the *how* — no implementation details, tool names, or specific technologies.
|
||||
- Keep it tight. If you need "and", you're describing two problems.
|
||||
|
||||
2. **Options** (exactly three)
|
||||
- Present three distinct, viable approaches labeled A, B, C.
|
||||
- Each option gets a brief description, pros, and cons.
|
||||
- Options should represent genuinely different strategies, not minor variations of the same approach.
|
||||
|
||||
3. **Recommendation** (one option)
|
||||
- State which option you recommend and why, based on the user's context and priorities.
|
||||
- Be direct — this is your professional judgment, not a hedge.
|
||||
|
||||
4. **Definition of Done**
|
||||
- List the specific success criteria for the recommended option.
|
||||
- These are concrete, verifiable outcomes — not vague aspirations.
|
||||
- If the user picks a different option, revise this section to match.
|
||||
|
||||
5. **Implementation Plan**
|
||||
- Concrete steps to execute the recommended option.
|
||||
- Include specific commands, tools, or actions where applicable.
|
||||
- If the user picks a different option, revise this section to match.
|
||||
|
||||
## Verification
|
||||
|
||||
- Response has exactly one Problem sentence.
|
||||
- Response has exactly three Options (A, B, C) with pros and cons for each.
|
||||
- Response has a single Recommendation that picks one option with reasoning.
|
||||
- Definition of Done and Implementation Plan align with the recommended option.
|
||||
- If the user selects a different option, Recommendation, DoD, and Implementation Plan update accordingly.
|
||||
|
||||
## Example
|
||||
|
||||
User: "Give me a 1-3-1 for adding retry logic to our API client."
|
||||
|
||||
Agent:
|
||||
|
||||
**Problem:** API calls to external services fail intermittently under load and need graceful retry handling.
|
||||
|
||||
**Options:**
|
||||
|
||||
- **Option A: Exponential backoff with jitter in a shared utility.**
|
||||
Pros: Simple, well-understood pattern; single implementation shared across all callers.
|
||||
Cons: No circuit-breaking; callers still block during retries.
|
||||
|
||||
- **Option B: Retry library (tenacity) as a decorator on each endpoint call.**
|
||||
Pros: Configurable per-endpoint; built-in support for circuit breakers and rate limits.
|
||||
Cons: Adds a dependency; decorator pattern can obscure control flow.
|
||||
|
||||
- **Option C: Queue-based async retry with dead letter handling.**
|
||||
Pros: Non-blocking; failed calls are tracked and replayable; scales well.
|
||||
Cons: Significant complexity increase; requires a queue backend; overkill for low-volume services.
|
||||
|
||||
**Recommendation:** Option A. The call volume doesn't justify queue infrastructure, and a shared utility with exponential backoff + jitter covers the failure mode cleanly without new dependencies.
|
||||
|
||||
**Definition of Done:**
|
||||
- `api_retry()` utility handles transient HTTP errors (429, 502, 503, 504).
|
||||
- Retries up to 3 times with exponential backoff and jitter.
|
||||
- All external API calls route through the retry utility.
|
||||
- Logs each retry attempt with endpoint and attempt number.
|
||||
- Tests cover retry exhaustion, immediate success, and transient-then-success scenarios.
|
||||
|
||||
**Implementation Plan:**
|
||||
1. Create `utils/api_retry.py` with configurable max retries, base delay, and retryable status codes.
|
||||
2. Add jitter using `random.uniform(0, base_delay)` to prevent thundering herd.
|
||||
3. Wrap existing API calls in `api_client.py` with the retry utility.
|
||||
4. Add unit tests mocking HTTP responses for each retry scenario.
|
||||
5. Verify under load with a simple stress test against a flaky endpoint mock.
|
||||
@@ -304,29 +304,6 @@ def ensure_parent(path: Path) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def resolve_secret_input(value: Any, env: Optional[Dict[str, str]] = None) -> Optional[str]:
|
||||
"""Resolve an OpenClaw SecretInput value to a plain string.
|
||||
|
||||
SecretInput can be:
|
||||
- A plain string: "sk-..."
|
||||
- An env template: "${OPENROUTER_API_KEY}"
|
||||
- A SecretRef object: {"source": "env", "id": "OPENROUTER_API_KEY"}
|
||||
"""
|
||||
if isinstance(value, str):
|
||||
# Check for env template: "${VAR_NAME}"
|
||||
m = re.match(r"^\$\{(\w+)\}$", value.strip())
|
||||
if m and env:
|
||||
return env.get(m.group(1), "").strip() or None
|
||||
return value.strip() or None
|
||||
if isinstance(value, dict):
|
||||
source = value.get("source", "")
|
||||
ref_id = value.get("id", "")
|
||||
if source == "env" and ref_id and env:
|
||||
return env.get(ref_id, "").strip() or None
|
||||
# File/exec sources can't be resolved here — return None
|
||||
return None
|
||||
|
||||
|
||||
def load_yaml_file(path: Path) -> Dict[str, Any]:
|
||||
if yaml is None or not path.exists():
|
||||
return {}
|
||||
@@ -913,20 +890,14 @@ class Migrator:
|
||||
self.record("command-allowlist", source, destination, "migrated", "Would merge patterns", added_patterns=added)
|
||||
|
||||
def load_openclaw_config(self) -> Dict[str, Any]:
|
||||
# Check current name and legacy config filenames
|
||||
for name in ("openclaw.json", "clawdbot.json", "moldbot.json"):
|
||||
config_path = self.source_root / name
|
||||
if config_path.exists():
|
||||
try:
|
||||
data = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
return data if isinstance(data, dict) else {}
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
return {}
|
||||
|
||||
def load_openclaw_env(self) -> Dict[str, str]:
|
||||
"""Load the OpenClaw .env file for secrets that live there instead of config."""
|
||||
return parse_env_file(self.source_root / ".env")
|
||||
config_path = self.source_root / "openclaw.json"
|
||||
if not config_path.exists():
|
||||
return {}
|
||||
try:
|
||||
data = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
return data if isinstance(data, dict) else {}
|
||||
except json.JSONDecodeError:
|
||||
return {}
|
||||
|
||||
def merge_env_values(self, additions: Dict[str, str], kind: str, source: Path) -> None:
|
||||
destination = self.target_root / ".env"
|
||||
@@ -1053,10 +1024,6 @@ class Migrator:
|
||||
supported_targets=sorted(SUPPORTED_SECRET_TARGETS),
|
||||
)
|
||||
|
||||
def _resolve_channel_secret(self, value: Any) -> Optional[str]:
|
||||
"""Resolve a channel config value that may be a SecretRef."""
|
||||
return resolve_secret_input(value, self.load_openclaw_env())
|
||||
|
||||
def migrate_discord_settings(self, config: Optional[Dict[str, Any]] = None) -> None:
|
||||
config = config or self.load_openclaw_config()
|
||||
additions: Dict[str, str] = {}
|
||||
@@ -1151,17 +1118,15 @@ class Migrator:
|
||||
secret_additions: Dict[str, str] = {}
|
||||
|
||||
# Extract provider API keys from models.providers
|
||||
# Note: apiKey values can be strings, env templates, or SecretRef objects
|
||||
openclaw_env = self.load_openclaw_env()
|
||||
providers = config.get("models", {}).get("providers", {})
|
||||
if isinstance(providers, dict):
|
||||
for provider_name, provider_cfg in providers.items():
|
||||
if not isinstance(provider_cfg, dict):
|
||||
continue
|
||||
raw_key = provider_cfg.get("apiKey")
|
||||
api_key = resolve_secret_input(raw_key, openclaw_env)
|
||||
if not api_key:
|
||||
api_key = provider_cfg.get("apiKey")
|
||||
if not isinstance(api_key, str) or not api_key.strip():
|
||||
continue
|
||||
api_key = api_key.strip()
|
||||
|
||||
base_url = provider_cfg.get("baseUrl", "")
|
||||
api_type = provider_cfg.get("api", "")
|
||||
@@ -1205,50 +1170,6 @@ class Migrator:
|
||||
if isinstance(oai_key, str) and oai_key.strip():
|
||||
secret_additions["VOICE_TOOLS_OPENAI_KEY"] = oai_key.strip()
|
||||
|
||||
# Also check the OpenClaw .env file — many users store keys there
|
||||
# instead of inline in openclaw.json
|
||||
openclaw_env = self.load_openclaw_env()
|
||||
env_key_mapping = {
|
||||
"OPENROUTER_API_KEY": "OPENROUTER_API_KEY",
|
||||
"OPENAI_API_KEY": "OPENAI_API_KEY",
|
||||
"ANTHROPIC_API_KEY": "ANTHROPIC_API_KEY",
|
||||
"ELEVENLABS_API_KEY": "ELEVENLABS_API_KEY",
|
||||
"TELEGRAM_BOT_TOKEN": "TELEGRAM_BOT_TOKEN",
|
||||
"DEEPSEEK_API_KEY": "DEEPSEEK_API_KEY",
|
||||
"GEMINI_API_KEY": "GEMINI_API_KEY",
|
||||
"ZAI_API_KEY": "ZAI_API_KEY",
|
||||
"MINIMAX_API_KEY": "MINIMAX_API_KEY",
|
||||
}
|
||||
for oc_key, hermes_key in env_key_mapping.items():
|
||||
val = openclaw_env.get(oc_key, "").strip()
|
||||
if val and hermes_key not in secret_additions:
|
||||
secret_additions[hermes_key] = val
|
||||
|
||||
# Check per-agent auth-profiles.json for additional credentials
|
||||
auth_profiles_path = self.source_root / "agents" / "main" / "agent" / "auth-profiles.json"
|
||||
if auth_profiles_path.exists():
|
||||
try:
|
||||
profiles = json.loads(auth_profiles_path.read_text(encoding="utf-8"))
|
||||
if isinstance(profiles, dict):
|
||||
# auth-profiles.json wraps profiles in a "profiles" key
|
||||
profile_entries = profiles.get("profiles", profiles) if isinstance(profiles.get("profiles"), dict) else profiles
|
||||
for profile_name, profile_data in profile_entries.items():
|
||||
if not isinstance(profile_data, dict):
|
||||
continue
|
||||
# Canonical field is "key", "apiKey" is accepted as alias
|
||||
api_key = profile_data.get("key", "") or profile_data.get("apiKey", "")
|
||||
if not isinstance(api_key, str) or not api_key.strip():
|
||||
continue
|
||||
name_lower = profile_name.lower()
|
||||
if "openrouter" in name_lower and "OPENROUTER_API_KEY" not in secret_additions:
|
||||
secret_additions["OPENROUTER_API_KEY"] = api_key.strip()
|
||||
elif "openai" in name_lower and "OPENAI_API_KEY" not in secret_additions:
|
||||
secret_additions["OPENAI_API_KEY"] = api_key.strip()
|
||||
elif "anthropic" in name_lower and "ANTHROPIC_API_KEY" not in secret_additions:
|
||||
secret_additions["ANTHROPIC_API_KEY"] = api_key.strip()
|
||||
except (json.JSONDecodeError, OSError):
|
||||
pass
|
||||
|
||||
if secret_additions:
|
||||
self.merge_env_values(secret_additions, "provider-keys", self.source_root / "openclaw.json")
|
||||
else:
|
||||
@@ -1297,11 +1218,7 @@ class Migrator:
|
||||
|
||||
if self.execute:
|
||||
backup_path = self.maybe_backup(destination)
|
||||
existing_model = hermes_config.get("model")
|
||||
if isinstance(existing_model, dict):
|
||||
existing_model["default"] = model_str
|
||||
else:
|
||||
hermes_config["model"] = {"default": model_str}
|
||||
hermes_config["model"] = model_str
|
||||
dump_yaml_file(destination, hermes_config)
|
||||
self.record("model-config", source_path, destination, "migrated", backup=str(backup_path) if backup_path else "", model=model_str)
|
||||
else:
|
||||
@@ -1327,44 +1244,22 @@ class Migrator:
|
||||
if isinstance(provider, str) and provider in ("elevenlabs", "openai", "edge"):
|
||||
tts_data["provider"] = provider
|
||||
|
||||
# TTS provider settings live under messages.tts.providers.{provider}
|
||||
# in OpenClaw (not messages.tts.elevenlabs directly)
|
||||
providers = tts.get("providers") or {}
|
||||
|
||||
# Also check the top-level "talk" config which has provider settings too
|
||||
talk_cfg = (config or self.load_openclaw_config()).get("talk") or {}
|
||||
talk_providers = talk_cfg.get("providers") or {}
|
||||
|
||||
# Merge: messages.tts.providers takes priority, then talk.providers,
|
||||
# then legacy flat keys (messages.tts.elevenlabs, etc.)
|
||||
elevenlabs = (
|
||||
(providers.get("elevenlabs") or {})
|
||||
if isinstance(providers.get("elevenlabs"), dict) else
|
||||
(talk_providers.get("elevenlabs") or {})
|
||||
if isinstance(talk_providers.get("elevenlabs"), dict) else
|
||||
(tts.get("elevenlabs") or {})
|
||||
)
|
||||
elevenlabs = tts.get("elevenlabs", {})
|
||||
if isinstance(elevenlabs, dict):
|
||||
el_settings: Dict[str, str] = {}
|
||||
voice_id = elevenlabs.get("voiceId") or talk_cfg.get("voiceId")
|
||||
voice_id = elevenlabs.get("voiceId")
|
||||
if isinstance(voice_id, str) and voice_id.strip():
|
||||
el_settings["voice_id"] = voice_id.strip()
|
||||
model_id = elevenlabs.get("modelId") or talk_cfg.get("modelId")
|
||||
model_id = elevenlabs.get("modelId")
|
||||
if isinstance(model_id, str) and model_id.strip():
|
||||
el_settings["model_id"] = model_id.strip()
|
||||
if el_settings:
|
||||
tts_data["elevenlabs"] = el_settings
|
||||
|
||||
openai_tts = (
|
||||
(providers.get("openai") or {})
|
||||
if isinstance(providers.get("openai"), dict) else
|
||||
(talk_providers.get("openai") or {})
|
||||
if isinstance(talk_providers.get("openai"), dict) else
|
||||
(tts.get("openai") or {})
|
||||
)
|
||||
openai_tts = tts.get("openai", {})
|
||||
if isinstance(openai_tts, dict):
|
||||
oai_settings: Dict[str, str] = {}
|
||||
oai_model = openai_tts.get("model") or openai_tts.get("modelId")
|
||||
oai_model = openai_tts.get("model")
|
||||
if isinstance(oai_model, str) and oai_model.strip():
|
||||
oai_settings["model"] = oai_model.strip()
|
||||
oai_voice = openai_tts.get("voice")
|
||||
@@ -1373,11 +1268,7 @@ class Migrator:
|
||||
if oai_settings:
|
||||
tts_data["openai"] = oai_settings
|
||||
|
||||
edge_tts = (
|
||||
(providers.get("edge") or {})
|
||||
if isinstance(providers.get("edge"), dict) else
|
||||
(tts.get("edge") or {})
|
||||
)
|
||||
edge_tts = tts.get("edge", {})
|
||||
if isinstance(edge_tts, dict):
|
||||
edge_voice = edge_tts.get("voice")
|
||||
if isinstance(edge_voice, str) and edge_voice.strip():
|
||||
@@ -1407,29 +1298,15 @@ class Migrator:
|
||||
self.record("tts-config", source_path, destination, "migrated", "Would set TTS config", settings=list(tts_data.keys()))
|
||||
|
||||
def migrate_shared_skills(self) -> None:
|
||||
# Check all OpenClaw skill sources: managed, personal, project-level
|
||||
skill_sources = [
|
||||
(self.source_root / "skills", "shared-skills", "managed skills"),
|
||||
(Path.home() / ".agents" / "skills", "personal-skills", "personal cross-project skills"),
|
||||
(self.source_root / "workspace" / ".agents" / "skills", "project-skills", "project-level shared skills"),
|
||||
(self.source_root / "workspace.default" / ".agents" / "skills", "project-skills", "project-level shared skills"),
|
||||
]
|
||||
found_any = False
|
||||
for source_root, kind_label, desc in skill_sources:
|
||||
if source_root.exists():
|
||||
found_any = True
|
||||
self._import_skill_directory(source_root, kind_label, desc)
|
||||
if not found_any:
|
||||
destination_root = self.target_root / "skills" / SKILL_CATEGORY_DIRNAME
|
||||
self.record("shared-skills", None, destination_root, "skipped", "No shared OpenClaw skills directories found")
|
||||
|
||||
def _import_skill_directory(self, source_root: Path, kind_label: str, desc: str) -> None:
|
||||
"""Import skills from a single source directory into openclaw-imports."""
|
||||
source_root = self.source_root / "skills"
|
||||
destination_root = self.target_root / "skills" / SKILL_CATEGORY_DIRNAME
|
||||
if not source_root.exists():
|
||||
self.record("shared-skills", None, destination_root, "skipped", "No shared OpenClaw skills directory found")
|
||||
return
|
||||
|
||||
skill_dirs = [p for p in sorted(source_root.iterdir()) if p.is_dir() and (p / "SKILL.md").exists()]
|
||||
if not skill_dirs:
|
||||
self.record(kind_label, source_root, destination_root, "skipped", f"No skills with SKILL.md found in {desc}")
|
||||
self.record("shared-skills", source_root, destination_root, "skipped", "No shared skills with SKILL.md found")
|
||||
return
|
||||
|
||||
for skill_dir in skill_dirs:
|
||||
@@ -1437,7 +1314,7 @@ class Migrator:
|
||||
final_destination = destination
|
||||
if destination.exists():
|
||||
if self.skill_conflict_mode == "skip":
|
||||
self.record(kind_label, skill_dir, destination, "conflict", "Destination skill already exists")
|
||||
self.record("shared-skill", skill_dir, destination, "conflict", "Destination skill already exists")
|
||||
continue
|
||||
if self.skill_conflict_mode == "rename":
|
||||
final_destination = self.resolve_skill_destination(destination)
|
||||
@@ -1452,19 +1329,19 @@ class Migrator:
|
||||
details: Dict[str, Any] = {"backup": str(backup_path) if backup_path else ""}
|
||||
if final_destination != destination:
|
||||
details["renamed_from"] = str(destination)
|
||||
self.record(kind_label, skill_dir, final_destination, "migrated", **details)
|
||||
self.record("shared-skill", skill_dir, final_destination, "migrated", **details)
|
||||
else:
|
||||
if final_destination != destination:
|
||||
self.record(
|
||||
kind_label,
|
||||
"shared-skill",
|
||||
skill_dir,
|
||||
final_destination,
|
||||
"migrated",
|
||||
f"Would copy {desc} directory under a renamed folder",
|
||||
"Would copy shared skill directory under a renamed folder",
|
||||
renamed_from=str(destination),
|
||||
)
|
||||
else:
|
||||
self.record(kind_label, skill_dir, final_destination, "migrated", f"Would copy {desc} directory")
|
||||
self.record("shared-skill", skill_dir, final_destination, "migrated", "Would copy shared skill directory")
|
||||
|
||||
desc_path = destination_root / "DESCRIPTION.md"
|
||||
if self.execute:
|
||||
@@ -1641,7 +1518,6 @@ class Migrator:
|
||||
self.source_candidate("workspace/IDENTITY.md", "workspace.default/IDENTITY.md"),
|
||||
self.source_candidate("workspace/TOOLS.md", "workspace.default/TOOLS.md"),
|
||||
self.source_candidate("workspace/HEARTBEAT.md", "workspace.default/HEARTBEAT.md"),
|
||||
self.source_candidate("workspace/BOOTSTRAP.md", "workspace.default/BOOTSTRAP.md"),
|
||||
]
|
||||
for candidate in candidates:
|
||||
if candidate:
|
||||
@@ -1913,9 +1789,8 @@ class Migrator:
|
||||
human_delay = defaults.get("humanDelay") or {}
|
||||
if human_delay:
|
||||
hd = hermes_cfg.get("human_delay") or {}
|
||||
hd_mode = human_delay.get("mode") or ("natural" if human_delay.get("enabled") else None)
|
||||
if hd_mode and hd_mode != "off":
|
||||
hd["mode"] = hd_mode
|
||||
if human_delay.get("enabled"):
|
||||
hd["mode"] = "natural"
|
||||
if human_delay.get("minMs"):
|
||||
hd["min_ms"] = human_delay["minMs"]
|
||||
if human_delay.get("maxMs"):
|
||||
@@ -1929,11 +1804,11 @@ class Migrator:
|
||||
changes = True
|
||||
|
||||
# Map terminal/exec settings
|
||||
exec_cfg = (config.get("tools") or {}).get("exec") or {}
|
||||
exec_cfg = defaults.get("exec") or (config.get("tools") or {}).get("exec") or {}
|
||||
if exec_cfg:
|
||||
terminal_cfg = hermes_cfg.get("terminal") or {}
|
||||
if exec_cfg.get("timeoutSec") or exec_cfg.get("timeout"):
|
||||
terminal_cfg["timeout"] = exec_cfg.get("timeoutSec") or exec_cfg.get("timeout")
|
||||
if exec_cfg.get("timeout"):
|
||||
terminal_cfg["timeout"] = exec_cfg["timeout"]
|
||||
changes = True
|
||||
hermes_cfg["terminal"] = terminal_cfg
|
||||
|
||||
@@ -2008,34 +1883,24 @@ class Migrator:
|
||||
sr = hermes_cfg.get("session_reset") or {}
|
||||
changes = False
|
||||
|
||||
# OpenClaw uses session.reset (structured) and session.resetTriggers (string array)
|
||||
reset = session.get("reset") or {}
|
||||
reset_triggers = session.get("resetTriggers") or session.get("reset_triggers") or []
|
||||
reset_triggers = session.get("resetTriggers") or session.get("reset_triggers") or {}
|
||||
if reset_triggers:
|
||||
daily = reset_triggers.get("daily") or {}
|
||||
idle = reset_triggers.get("idle") or {}
|
||||
|
||||
if reset:
|
||||
# Structured reset config: has mode, atHour, idleMinutes
|
||||
mode = reset.get("mode", "")
|
||||
if mode == "daily":
|
||||
if daily.get("enabled") and idle.get("enabled"):
|
||||
sr["mode"] = "both"
|
||||
elif daily.get("enabled"):
|
||||
sr["mode"] = "daily"
|
||||
elif mode == "idle":
|
||||
elif idle.get("enabled"):
|
||||
sr["mode"] = "idle"
|
||||
else:
|
||||
sr["mode"] = mode or "none"
|
||||
if reset.get("atHour") is not None:
|
||||
sr["at_hour"] = reset["atHour"]
|
||||
if reset.get("idleMinutes"):
|
||||
sr["idle_minutes"] = reset["idleMinutes"]
|
||||
changes = True
|
||||
elif isinstance(reset_triggers, list) and reset_triggers:
|
||||
# Simple string triggers: ["daily", "idle"]
|
||||
has_daily = "daily" in reset_triggers
|
||||
has_idle = "idle" in reset_triggers
|
||||
if has_daily and has_idle:
|
||||
sr["mode"] = "both"
|
||||
elif has_daily:
|
||||
sr["mode"] = "daily"
|
||||
elif has_idle:
|
||||
sr["mode"] = "idle"
|
||||
sr["mode"] = "none"
|
||||
|
||||
if daily.get("hour") is not None:
|
||||
sr["at_hour"] = daily["hour"]
|
||||
if idle.get("minutes") or idle.get("timeoutMinutes"):
|
||||
sr["idle_minutes"] = idle.get("minutes") or idle.get("timeoutMinutes")
|
||||
changes = True
|
||||
|
||||
if changes:
|
||||
@@ -2227,12 +2092,11 @@ class Migrator:
|
||||
browser_hermes = hermes_cfg.get("browser") or {}
|
||||
changed = False
|
||||
|
||||
# Map fields that have Hermes equivalents
|
||||
if browser.get("cdpUrl"):
|
||||
browser_hermes["cdp_url"] = browser["cdpUrl"]
|
||||
if browser.get("inactivityTimeoutMs"):
|
||||
browser_hermes["inactivity_timeout"] = browser["inactivityTimeoutMs"] // 1000
|
||||
changed = True
|
||||
if browser.get("headless") is not None:
|
||||
browser_hermes["headless"] = browser["headless"]
|
||||
if browser.get("commandTimeoutMs"):
|
||||
browser_hermes["command_timeout"] = browser["commandTimeoutMs"] // 1000
|
||||
changed = True
|
||||
|
||||
if changed:
|
||||
@@ -2243,9 +2107,9 @@ class Migrator:
|
||||
self.record("browser-config", "openclaw.json browser.*", "config.yaml browser",
|
||||
"migrated")
|
||||
|
||||
# Archive remaining browser settings
|
||||
# Archive advanced browser settings
|
||||
advanced = {k: v for k, v in browser.items()
|
||||
if k not in ("cdpUrl", "headless") and v}
|
||||
if k not in ("inactivityTimeoutMs", "commandTimeoutMs") and v}
|
||||
if advanced and self.archive_dir:
|
||||
if self.execute:
|
||||
self.archive_dir.mkdir(parents=True, exist_ok=True)
|
||||
@@ -2266,22 +2130,18 @@ class Migrator:
|
||||
hermes_cfg = load_yaml_file(hermes_cfg_path)
|
||||
changed = False
|
||||
|
||||
# Map exec timeout -> terminal timeout (field is timeoutSec in OpenClaw)
|
||||
# Map exec timeout -> terminal timeout
|
||||
exec_cfg = tools.get("exec") or {}
|
||||
timeout_val = exec_cfg.get("timeoutSec") or exec_cfg.get("timeout")
|
||||
if timeout_val:
|
||||
if exec_cfg.get("timeout"):
|
||||
terminal_cfg = hermes_cfg.get("terminal") or {}
|
||||
terminal_cfg["timeout"] = timeout_val
|
||||
terminal_cfg["timeout"] = exec_cfg["timeout"]
|
||||
hermes_cfg["terminal"] = terminal_cfg
|
||||
changed = True
|
||||
|
||||
# Map web search API key (path: tools.web.search.brave.apiKey in OpenClaw)
|
||||
web_cfg = tools.get("web") or tools.get("webSearch") or {}
|
||||
search_cfg = web_cfg.get("search") or web_cfg if not web_cfg.get("search") else web_cfg["search"]
|
||||
brave_cfg = search_cfg.get("brave") or {}
|
||||
brave_key = brave_cfg.get("apiKey") or search_cfg.get("braveApiKey") or web_cfg.get("braveApiKey")
|
||||
if brave_key and isinstance(brave_key, str) and self.migrate_secrets:
|
||||
self._set_env_var("BRAVE_API_KEY", brave_key, "tools.web.search.brave.apiKey")
|
||||
# Map web search API key
|
||||
web_cfg = tools.get("webSearch") or tools.get("web") or {}
|
||||
if web_cfg.get("braveApiKey") and self.migrate_secrets:
|
||||
self._set_env_var("BRAVE_API_KEY", web_cfg["braveApiKey"], "tools.webSearch.braveApiKey")
|
||||
|
||||
if changed and self.execute:
|
||||
self.maybe_backup(hermes_cfg_path)
|
||||
@@ -2309,9 +2169,8 @@ class Migrator:
|
||||
hermes_cfg_path = self.target_root / "config.yaml"
|
||||
hermes_cfg = load_yaml_file(hermes_cfg_path)
|
||||
|
||||
# Map approval mode (nested under approvals.exec.mode in OpenClaw)
|
||||
exec_approvals = approvals.get("exec") or {}
|
||||
mode = (exec_approvals.get("mode") if isinstance(exec_approvals, dict) else None) or approvals.get("mode") or approvals.get("defaultMode")
|
||||
# Map approval mode
|
||||
mode = approvals.get("mode") or approvals.get("defaultMode")
|
||||
if mode:
|
||||
mode_map = {"auto": "off", "always": "manual", "smart": "smart", "manual": "manual"}
|
||||
hermes_mode = mode_map.get(mode, "manual")
|
||||
|
||||
@@ -1,97 +0,0 @@
|
||||
---
|
||||
name: canvas
|
||||
description: Canvas LMS integration — fetch enrolled courses and assignments using API token authentication.
|
||||
version: 1.0.0
|
||||
author: community
|
||||
license: MIT
|
||||
prerequisites:
|
||||
env_vars: [CANVAS_API_TOKEN, CANVAS_BASE_URL]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Canvas, LMS, Education, Courses, Assignments]
|
||||
---
|
||||
|
||||
# Canvas LMS — Course & Assignment Access
|
||||
|
||||
Read-only access to Canvas LMS for listing courses and assignments.
|
||||
|
||||
## Scripts
|
||||
|
||||
- `scripts/canvas_api.py` — Python CLI for Canvas API calls
|
||||
|
||||
## Setup
|
||||
|
||||
1. Log in to your Canvas instance in a browser
|
||||
2. Go to **Account → Settings** (click your profile icon, then Settings)
|
||||
3. Scroll to **Approved Integrations** and click **+ New Access Token**
|
||||
4. Name the token (e.g., "Hermes Agent"), set an optional expiry, and click **Generate Token**
|
||||
5. Copy the token and add to `~/.hermes/.env`:
|
||||
|
||||
```
|
||||
CANVAS_API_TOKEN=your_token_here
|
||||
CANVAS_BASE_URL=https://yourschool.instructure.com
|
||||
```
|
||||
|
||||
The base URL is whatever appears in your browser when you're logged into Canvas (no trailing slash).
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
CANVAS="python $HERMES_HOME/skills/productivity/canvas/scripts/canvas_api.py"
|
||||
|
||||
# List all active courses
|
||||
$CANVAS list_courses --enrollment-state active
|
||||
|
||||
# List all courses (any state)
|
||||
$CANVAS list_courses
|
||||
|
||||
# List assignments for a specific course
|
||||
$CANVAS list_assignments 12345
|
||||
|
||||
# List assignments ordered by due date
|
||||
$CANVAS list_assignments 12345 --order-by due_at
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
**list_courses** returns:
|
||||
```json
|
||||
[{"id": 12345, "name": "Intro to CS", "course_code": "CS101", "workflow_state": "available", "start_at": "...", "end_at": "..."}]
|
||||
```
|
||||
|
||||
**list_assignments** returns:
|
||||
```json
|
||||
[{"id": 67890, "name": "Homework 1", "due_at": "2025-02-15T23:59:00Z", "points_possible": 100, "submission_types": ["online_upload"], "html_url": "...", "description": "...", "course_id": 12345}]
|
||||
```
|
||||
|
||||
Note: Assignment descriptions are truncated to 500 characters. The `html_url` field links to the full assignment page in Canvas.
|
||||
|
||||
## API Reference (curl)
|
||||
|
||||
```bash
|
||||
# List courses
|
||||
curl -s -H "Authorization: Bearer $CANVAS_API_TOKEN" \
|
||||
"$CANVAS_BASE_URL/api/v1/courses?enrollment_state=active&per_page=10"
|
||||
|
||||
# List assignments for a course
|
||||
curl -s -H "Authorization: Bearer $CANVAS_API_TOKEN" \
|
||||
"$CANVAS_BASE_URL/api/v1/courses/COURSE_ID/assignments?per_page=10&order_by=due_at"
|
||||
```
|
||||
|
||||
Canvas uses `Link` headers for pagination. The Python script handles pagination automatically.
|
||||
|
||||
## Rules
|
||||
|
||||
- This skill is **read-only** — it only fetches data, never modifies courses or assignments
|
||||
- On first use, verify auth by running `$CANVAS list_courses` — if it fails with 401, guide the user through setup
|
||||
- Canvas rate-limits to ~700 requests per 10 minutes; check `X-Rate-Limit-Remaining` header if hitting limits
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Fix |
|
||||
|---------|-----|
|
||||
| 401 Unauthorized | Token invalid or expired — regenerate in Canvas Settings |
|
||||
| 403 Forbidden | Token lacks permission for this course |
|
||||
| Empty course list | Try `--enrollment-state active` or omit the flag to see all states |
|
||||
| Wrong institution | Verify `CANVAS_BASE_URL` matches the URL in your browser |
|
||||
| Timeout errors | Check network connectivity to your Canvas instance |
|
||||
@@ -1,157 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Canvas LMS API CLI for Hermes Agent.
|
||||
|
||||
A thin CLI wrapper around the Canvas REST API.
|
||||
Authenticates using a personal access token from environment variables.
|
||||
|
||||
Usage:
|
||||
python canvas_api.py list_courses [--per-page N] [--enrollment-state STATE]
|
||||
python canvas_api.py list_assignments COURSE_ID [--per-page N] [--order-by FIELD]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
import requests
|
||||
|
||||
CANVAS_API_TOKEN = os.environ.get("CANVAS_API_TOKEN", "")
|
||||
CANVAS_BASE_URL = os.environ.get("CANVAS_BASE_URL", "").rstrip("/")
|
||||
|
||||
|
||||
def _check_config():
|
||||
"""Validate required environment variables are set."""
|
||||
missing = []
|
||||
if not CANVAS_API_TOKEN:
|
||||
missing.append("CANVAS_API_TOKEN")
|
||||
if not CANVAS_BASE_URL:
|
||||
missing.append("CANVAS_BASE_URL")
|
||||
if missing:
|
||||
print(
|
||||
f"Missing required environment variables: {', '.join(missing)}\n"
|
||||
"Set them in ~/.hermes/.env or export them in your shell.\n"
|
||||
"See the canvas skill SKILL.md for setup instructions.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def _headers():
|
||||
return {"Authorization": f"Bearer {CANVAS_API_TOKEN}"}
|
||||
|
||||
|
||||
def _paginated_get(url, params=None, max_items=200):
|
||||
"""Fetch all pages up to max_items, following Canvas Link headers."""
|
||||
results = []
|
||||
while url and len(results) < max_items:
|
||||
resp = requests.get(url, headers=_headers(), params=params, timeout=30)
|
||||
resp.raise_for_status()
|
||||
results.extend(resp.json())
|
||||
params = None # params are included in the Link URL for subsequent pages
|
||||
url = None
|
||||
link = resp.headers.get("Link", "")
|
||||
for part in link.split(","):
|
||||
if 'rel="next"' in part:
|
||||
url = part.split(";")[0].strip().strip("<>")
|
||||
return results[:max_items]
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# Commands
|
||||
# =========================================================================
|
||||
|
||||
|
||||
def list_courses(args):
|
||||
"""List enrolled courses."""
|
||||
_check_config()
|
||||
url = f"{CANVAS_BASE_URL}/api/v1/courses"
|
||||
params = {"per_page": args.per_page}
|
||||
if args.enrollment_state:
|
||||
params["enrollment_state"] = args.enrollment_state
|
||||
try:
|
||||
courses = _paginated_get(url, params)
|
||||
except requests.HTTPError as e:
|
||||
print(f"API error: {e.response.status_code} {e.response.text}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
output = [
|
||||
{
|
||||
"id": c["id"],
|
||||
"name": c.get("name", ""),
|
||||
"course_code": c.get("course_code", ""),
|
||||
"enrollment_term_id": c.get("enrollment_term_id"),
|
||||
"start_at": c.get("start_at"),
|
||||
"end_at": c.get("end_at"),
|
||||
"workflow_state": c.get("workflow_state", ""),
|
||||
}
|
||||
for c in courses
|
||||
]
|
||||
print(json.dumps(output, indent=2))
|
||||
|
||||
|
||||
def list_assignments(args):
|
||||
"""List assignments for a course."""
|
||||
_check_config()
|
||||
url = f"{CANVAS_BASE_URL}/api/v1/courses/{args.course_id}/assignments"
|
||||
params = {"per_page": args.per_page}
|
||||
if args.order_by:
|
||||
params["order_by"] = args.order_by
|
||||
try:
|
||||
assignments = _paginated_get(url, params)
|
||||
except requests.HTTPError as e:
|
||||
print(f"API error: {e.response.status_code} {e.response.text}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
output = [
|
||||
{
|
||||
"id": a["id"],
|
||||
"name": a.get("name", ""),
|
||||
"description": (a.get("description") or "")[:500],
|
||||
"due_at": a.get("due_at"),
|
||||
"points_possible": a.get("points_possible"),
|
||||
"submission_types": a.get("submission_types", []),
|
||||
"html_url": a.get("html_url", ""),
|
||||
"course_id": a.get("course_id"),
|
||||
}
|
||||
for a in assignments
|
||||
]
|
||||
print(json.dumps(output, indent=2))
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# CLI parser
|
||||
# =========================================================================
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Canvas LMS API CLI for Hermes Agent"
|
||||
)
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
# --- list_courses ---
|
||||
p = sub.add_parser("list_courses", help="List enrolled courses")
|
||||
p.add_argument("--per-page", type=int, default=50, help="Results per page (default 50)")
|
||||
p.add_argument(
|
||||
"--enrollment-state",
|
||||
default="",
|
||||
help="Filter by enrollment state (active, invited_or_pending, completed)",
|
||||
)
|
||||
p.set_defaults(func=list_courses)
|
||||
|
||||
# --- list_assignments ---
|
||||
p = sub.add_parser("list_assignments", help="List assignments for a course")
|
||||
p.add_argument("course_id", help="Canvas course ID")
|
||||
p.add_argument("--per-page", type=int, default=50, help="Results per page (default 50)")
|
||||
p.add_argument(
|
||||
"--order-by",
|
||||
default="",
|
||||
help="Order by field (due_at, name, position)",
|
||||
)
|
||||
p.set_defaults(func=list_assignments)
|
||||
|
||||
args = parser.parse_args()
|
||||
args.func(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,324 +0,0 @@
|
||||
---
|
||||
name: memento-flashcards
|
||||
description: >-
|
||||
Spaced-repetition flashcard system. Create cards from facts or text,
|
||||
chat with flashcards using free-text answers graded by the agent,
|
||||
generate quizzes from YouTube transcripts, review due cards with
|
||||
adaptive scheduling, and export/import decks as CSV.
|
||||
version: 1.0.0
|
||||
author: Memento AI
|
||||
license: MIT
|
||||
platforms: [macos, linux]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Education, Flashcards, Spaced Repetition, Learning, Quiz, YouTube]
|
||||
requires_toolsets: [terminal]
|
||||
category: productivity
|
||||
---
|
||||
|
||||
# Memento Flashcards — Spaced-Repetition Flashcard Skill
|
||||
|
||||
## Overview
|
||||
|
||||
Memento gives you a local, file-based flashcard system with spaced-repetition scheduling.
|
||||
Users can chat with their flashcards by answering in free text and having the agent grade the response before scheduling the next review.
|
||||
Use it whenever the user wants to:
|
||||
|
||||
- **Remember a fact** — turn any statement into a Q/A flashcard
|
||||
- **Study with spaced repetition** — review due cards with adaptive intervals and agent-graded free-text answers
|
||||
- **Quiz from a YouTube video** — fetch a transcript and generate a 5-question quiz
|
||||
- **Manage decks** — organise cards into collections, export/import CSV
|
||||
|
||||
All card data lives in a single JSON file. No external API keys are required — you (the agent) generate flashcard content and quiz questions directly.
|
||||
|
||||
User-facing response style for Memento Flashcards:
|
||||
- Use plain text only. Do not use Markdown formatting in replies to the user.
|
||||
- Keep review and quiz feedback brief and neutral. Avoid extra praise, pep, or long explanations.
|
||||
|
||||
## When to Use
|
||||
|
||||
Use this skill when the user wants to:
|
||||
- Save facts as flashcards for later review
|
||||
- Review due cards with spaced repetition
|
||||
- Generate a quiz from a YouTube video transcript
|
||||
- Import, export, inspect, or delete flashcard data
|
||||
|
||||
Do not use this skill for general Q&A, coding help, or non-memory tasks.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| User intent | Action |
|
||||
|---|---|
|
||||
| "Remember that X" / "save this as a flashcard" | Generate a Q/A card, call `memento_cards.py add` |
|
||||
| Sends a fact without mentioning flashcards | Ask "Want me to save this as a Memento flashcard?" — only create if confirmed |
|
||||
| "Create a flashcard" | Ask for Q, A, collection; call `memento_cards.py add` |
|
||||
| "Review my cards" | Call `memento_cards.py due`, present cards one-by-one |
|
||||
| "Quiz me on [YouTube URL]" | Call `youtube_quiz.py fetch VIDEO_ID`, generate 5 questions, call `memento_cards.py add-quiz` |
|
||||
| "Export my cards" | Call `memento_cards.py export --output PATH` |
|
||||
| "Import cards from CSV" | Call `memento_cards.py import --file PATH --collection NAME` |
|
||||
| "Show my stats" | Call `memento_cards.py stats` |
|
||||
| "Delete a card" | Call `memento_cards.py delete --id ID` |
|
||||
| "Delete a collection" | Call `memento_cards.py delete-collection --collection NAME` |
|
||||
|
||||
## Card Storage
|
||||
|
||||
Cards are stored in a JSON file at:
|
||||
|
||||
```
|
||||
~/.hermes/skills/productivity/memento-flashcards/data/cards.json
|
||||
```
|
||||
|
||||
**Never edit this file directly.** Always use `memento_cards.py` subcommands. The script handles atomic writes (write to temp file, then rename) to prevent corruption.
|
||||
|
||||
The file is created automatically on first use.
|
||||
|
||||
## Procedure
|
||||
|
||||
### Creating Cards from Facts
|
||||
|
||||
### Activation Rules
|
||||
|
||||
Not every factual statement should become a flashcard. Use this three-tier check:
|
||||
|
||||
1. **Explicit intent** — the user mentions "memento", "flashcard", "remember this", "save this card", "add a card", or similar phrasing that clearly requests a flashcard → **create the card directly**, no confirmation needed.
|
||||
2. **Implicit intent** — the user sends a factual statement without mentioning flashcards (e.g. "The speed of light is 299,792 km/s") → **ask first**: "Want me to save this as a Memento flashcard?" Only create the card if the user confirms.
|
||||
3. **No intent** — the message is a coding task, a question, instructions, normal conversation, or anything that is clearly not a fact to memorize → **do NOT activate this skill at all**. Let other skills or default behavior handle it.
|
||||
|
||||
When activation is confirmed (tier 1 directly, tier 2 after confirmation), generate a flashcard:
|
||||
|
||||
**Step 1:** Turn the statement into a Q/A pair. Use this format internally:
|
||||
|
||||
```
|
||||
Turn the factual statement into a front-back pair.
|
||||
Return exactly two lines:
|
||||
Q: <question text>
|
||||
A: <answer text>
|
||||
|
||||
Statement: "{statement}"
|
||||
```
|
||||
|
||||
Rules:
|
||||
- The question should test recall of the key fact
|
||||
- The answer should be concise and direct
|
||||
|
||||
**Step 2:** Call the script to store the card:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py add \
|
||||
--question "What year did World War 2 end?" \
|
||||
--answer "1945" \
|
||||
--collection "History"
|
||||
```
|
||||
|
||||
If the user doesn't specify a collection, use `"General"` as the default.
|
||||
|
||||
The script outputs JSON confirming the created card.
|
||||
|
||||
### Manual Card Creation
|
||||
|
||||
When the user explicitly asks to create a flashcard, ask them for:
|
||||
1. The question (front of card)
|
||||
2. The answer (back of card)
|
||||
3. The collection name (optional — default to `"General"`)
|
||||
|
||||
Then call `memento_cards.py add` as above.
|
||||
|
||||
### Reviewing Due Cards
|
||||
|
||||
When the user wants to review, fetch all due cards:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py due
|
||||
```
|
||||
|
||||
This returns a JSON array of cards where `next_review_at <= now`. If a collection filter is needed:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py due --collection "History"
|
||||
```
|
||||
|
||||
**Review flow (free-text grading):**
|
||||
|
||||
Here is an example of the EXACT interaction pattern you must follow. The user answers, you grade them, tell them the correct answer, then rate the card.
|
||||
|
||||
**Example interaction:**
|
||||
|
||||
> **Agent:** What year did the Berlin Wall fall?
|
||||
>
|
||||
> **User:** 1991
|
||||
>
|
||||
> **Agent:** Not quite. The Berlin Wall fell in 1989. Next review is tomorrow.
|
||||
> *(agent calls: memento_cards.py rate --id ABC --rating hard --user-answer "1991")*
|
||||
>
|
||||
> Next question: Who was the first person to walk on the moon?
|
||||
|
||||
**The rules:**
|
||||
|
||||
1. Show only the question. Wait for the user to answer.
|
||||
2. After receiving their answer, compare it to the expected answer and grade it:
|
||||
- **correct** → user got the key fact right (even if worded differently)
|
||||
- **partial** → right track but missing the core detail
|
||||
- **incorrect** → wrong or off-topic
|
||||
3. **You MUST tell the user the correct answer and how they did.** Keep it short and plain-text. Use this format:
|
||||
- correct: "Correct. Answer: {answer}. Next review in 7 days."
|
||||
- partial: "Close. Answer: {answer}. {what they missed}. Next review in 3 days."
|
||||
- incorrect: "Not quite. Answer: {answer}. Next review tomorrow."
|
||||
4. Then call the rate command: correct→easy, partial→good, incorrect→hard.
|
||||
5. Then show the next question.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py rate \
|
||||
--id CARD_ID --rating easy --user-answer "what the user said"
|
||||
```
|
||||
|
||||
**Never skip step 3.** The user must always see the correct answer and feedback before you move on.
|
||||
|
||||
If no cards are due, tell the user: "No cards due for review right now. Check back later!"
|
||||
|
||||
**Retire override:** At any point the user can say "retire this card" to permanently remove it from reviews. Use `--rating retire` for this.
|
||||
|
||||
### Spaced Repetition Algorithm
|
||||
|
||||
The rating determines the next review interval:
|
||||
|
||||
| Rating | Interval | ease_streak | Status change |
|
||||
|---|---|---|---|
|
||||
| **hard** | +1 day | reset to 0 | stays learning |
|
||||
| **good** | +3 days | reset to 0 | stays learning |
|
||||
| **easy** | +7 days | +1 | if ease_streak >= 3 → retired |
|
||||
| **retire** | permanent | reset to 0 | → retired |
|
||||
|
||||
- **learning**: card is actively in rotation
|
||||
- **retired**: card won't appear in reviews (user has mastered it or manually retired it)
|
||||
- Three consecutive "easy" ratings automatically retire a card
|
||||
|
||||
### YouTube Quiz Generation
|
||||
|
||||
When the user sends a YouTube URL and wants a quiz:
|
||||
|
||||
**Step 1:** Extract the video ID from the URL (e.g. `dQw4w9WgXcQ` from `https://www.youtube.com/watch?v=dQw4w9WgXcQ`).
|
||||
|
||||
**Step 2:** Fetch the transcript:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/youtube_quiz.py fetch VIDEO_ID
|
||||
```
|
||||
|
||||
This returns `{"title": "...", "transcript": "..."}` or an error.
|
||||
|
||||
If the script reports `missing_dependency`, tell the user to install it:
|
||||
```bash
|
||||
pip install youtube-transcript-api
|
||||
```
|
||||
|
||||
**Step 3:** Generate 5 quiz questions from the transcript. Use these rules:
|
||||
|
||||
```
|
||||
You are creating a 5-question quiz for a podcast episode.
|
||||
Return ONLY a JSON array with exactly 5 objects.
|
||||
Each object must contain keys 'question' and 'answer'.
|
||||
|
||||
Selection criteria:
|
||||
- Prioritize important, surprising, or foundational facts.
|
||||
- Skip filler, obvious details, and facts that require heavy context.
|
||||
- Never return true/false questions.
|
||||
- Never ask only for a date.
|
||||
|
||||
Question rules:
|
||||
- Each question must test exactly one discrete fact.
|
||||
- Use clear, unambiguous wording.
|
||||
- Prefer What, Who, How many, Which.
|
||||
- Avoid open-ended Describe or Explain prompts.
|
||||
|
||||
Answer rules:
|
||||
- Each answer must be under 240 characters.
|
||||
- Lead with the answer itself, not preamble.
|
||||
- Add only minimal clarifying detail if needed.
|
||||
```
|
||||
|
||||
Use the first 15,000 characters of the transcript as context. Generate the questions yourself (you are the LLM).
|
||||
|
||||
**Step 4:** Validate the output is valid JSON with exactly 5 items, each having non-empty `question` and `answer` strings. If validation fails, retry once.
|
||||
|
||||
**Step 5:** Store quiz cards:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py add-quiz \
|
||||
--video-id "VIDEO_ID" \
|
||||
--questions '[{"question":"...","answer":"..."},...]' \
|
||||
--collection "Quiz - Episode Title"
|
||||
```
|
||||
|
||||
The script deduplicates by `video_id` — if cards for that video already exist, it skips creation and reports the existing cards.
|
||||
|
||||
**Step 6:** Present questions one-by-one using the same free-text grading flow:
|
||||
1. Show "Question 1/5: ..." and wait for the user's answer. Never include the answer or any hint about revealing it.
|
||||
2. Wait for the user to answer in their own words
|
||||
3. Grade their answer using the grading prompt (see "Reviewing Due Cards" section)
|
||||
4. **IMPORTANT: You MUST reply to the user with feedback before doing anything else.** Show the grade, the correct answer, and when the card is next due. Do NOT silently skip to the next question. Keep it short and plain-text. Example: "Not quite. Answer: {answer}. Next review tomorrow."
|
||||
5. **After showing feedback**, call the rate command and then show the next question in the same message:
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py rate \
|
||||
--id CARD_ID --rating easy --user-answer "what the user said"
|
||||
```
|
||||
6. Repeat. Every answer MUST receive visible feedback before the next question.
|
||||
|
||||
### Export/Import CSV
|
||||
|
||||
**Export:**
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py export \
|
||||
--output ~/flashcards.csv
|
||||
```
|
||||
|
||||
Produces a 3-column CSV: `question,answer,collection` (no header row).
|
||||
|
||||
**Import:**
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py import \
|
||||
--file ~/flashcards.csv \
|
||||
--collection "Imported"
|
||||
```
|
||||
|
||||
Reads a CSV with columns: question, answer, and optionally collection (column 3). If the collection column is missing, uses the `--collection` argument.
|
||||
|
||||
### Statistics
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py stats
|
||||
```
|
||||
|
||||
Returns JSON with:
|
||||
- `total`: total card count
|
||||
- `learning`: cards in active rotation
|
||||
- `retired`: mastered cards
|
||||
- `due_now`: cards due for review right now
|
||||
- `collections`: breakdown by collection name
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Never edit `cards.json` directly** — always use the script subcommands to avoid corruption
|
||||
- **Transcript failures** — some YouTube videos have no English transcript or have transcripts disabled; inform the user and suggest another video
|
||||
- **Optional dependency** — `youtube_quiz.py` needs `youtube-transcript-api`; if missing, tell the user to run `pip install youtube-transcript-api`
|
||||
- **Large imports** — CSV imports with thousands of rows work fine but the JSON output may be verbose; summarize the result for the user
|
||||
- **Video ID extraction** — support both `youtube.com/watch?v=ID` and `youtu.be/ID` URL formats
|
||||
|
||||
## Verification
|
||||
|
||||
Verify the helper scripts directly:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py stats
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py add --question "Capital of France?" --answer "Paris" --collection "General"
|
||||
python3 ~/.hermes/skills/productivity/memento-flashcards/scripts/memento_cards.py due
|
||||
```
|
||||
|
||||
If you are testing from the repo checkout, run:
|
||||
|
||||
```bash
|
||||
pytest tests/skills/test_memento_cards.py tests/skills/test_youtube_quiz.py -q
|
||||
```
|
||||
|
||||
Agent-level verification:
|
||||
- Start a review and confirm feedback is plain text, brief, and always includes the correct answer before the next card
|
||||
- Run a YouTube quiz flow and confirm each answer receives visible feedback before the next question
|
||||
@@ -1,353 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Memento card storage, spaced-repetition engine, and CSV I/O.
|
||||
|
||||
Stdlib-only. All output is JSON for agent parsing.
|
||||
Data file: $HERMES_HOME/skills/productivity/memento-flashcards/data/cards.json
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
_HERMES_HOME = Path(os.environ.get("HERMES_HOME", Path.home() / ".hermes"))
|
||||
DATA_DIR = _HERMES_HOME / "skills" / "productivity" / "memento-flashcards" / "data"
|
||||
CARDS_FILE = DATA_DIR / "cards.json"
|
||||
|
||||
RETIRED_SENTINEL = "9999-12-31T23:59:59+00:00"
|
||||
|
||||
|
||||
def _now() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def _iso(dt: datetime) -> str:
|
||||
return dt.isoformat()
|
||||
|
||||
|
||||
def _parse_iso(s: str) -> datetime:
|
||||
return datetime.fromisoformat(s)
|
||||
|
||||
|
||||
def _empty_store() -> dict:
|
||||
return {"cards": [], "version": 1}
|
||||
|
||||
|
||||
def _load() -> dict:
|
||||
if not CARDS_FILE.exists():
|
||||
return _empty_store()
|
||||
try:
|
||||
with open(CARDS_FILE, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, dict) or "cards" not in data:
|
||||
return _empty_store()
|
||||
return data
|
||||
except (json.JSONDecodeError, OSError):
|
||||
return _empty_store()
|
||||
|
||||
|
||||
def _save(data: dict) -> None:
|
||||
DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
fd, tmp = tempfile.mkstemp(dir=DATA_DIR, suffix=".tmp")
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
f.write("\n")
|
||||
os.replace(tmp, CARDS_FILE)
|
||||
except BaseException:
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
def _out(obj: object) -> None:
|
||||
json.dump(obj, sys.stdout, indent=2, ensure_ascii=False)
|
||||
sys.stdout.write("\n")
|
||||
|
||||
|
||||
# ── Subcommands ──────────────────────────────────────────────────────────────
|
||||
|
||||
def cmd_add(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
now = _now()
|
||||
card = {
|
||||
"id": str(uuid.uuid4()),
|
||||
"question": args.question,
|
||||
"answer": args.answer,
|
||||
"collection": args.collection or "General",
|
||||
"status": "learning",
|
||||
"ease_streak": 0,
|
||||
"next_review_at": _iso(now),
|
||||
"created_at": _iso(now),
|
||||
"video_id": None,
|
||||
"last_user_answer": None,
|
||||
}
|
||||
data["cards"].append(card)
|
||||
_save(data)
|
||||
_out({"ok": True, "card": card})
|
||||
|
||||
|
||||
def cmd_add_quiz(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
now = _now()
|
||||
|
||||
try:
|
||||
questions = json.loads(args.questions)
|
||||
except json.JSONDecodeError as exc:
|
||||
_out({"ok": False, "error": f"Invalid JSON for --questions: {exc}"})
|
||||
sys.exit(1)
|
||||
|
||||
# Dedup: skip if cards with this video_id already exist
|
||||
existing_ids = {c["video_id"] for c in data["cards"] if c.get("video_id")}
|
||||
if args.video_id in existing_ids:
|
||||
existing = [c for c in data["cards"] if c.get("video_id") == args.video_id]
|
||||
_out({"ok": True, "skipped": True, "reason": "duplicate_video_id", "existing_count": len(existing), "cards": existing})
|
||||
return
|
||||
|
||||
created = []
|
||||
for qa in questions:
|
||||
card = {
|
||||
"id": str(uuid.uuid4()),
|
||||
"question": qa["question"],
|
||||
"answer": qa["answer"],
|
||||
"collection": args.collection or "Quiz",
|
||||
"status": "learning",
|
||||
"ease_streak": 0,
|
||||
"next_review_at": _iso(now),
|
||||
"created_at": _iso(now),
|
||||
"video_id": args.video_id,
|
||||
"last_user_answer": None,
|
||||
}
|
||||
data["cards"].append(card)
|
||||
created.append(card)
|
||||
|
||||
_save(data)
|
||||
_out({"ok": True, "created_count": len(created), "cards": created})
|
||||
|
||||
|
||||
def cmd_due(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
now = _now()
|
||||
due = []
|
||||
for card in data["cards"]:
|
||||
if card["status"] == "retired":
|
||||
continue
|
||||
review_at = _parse_iso(card["next_review_at"])
|
||||
if review_at <= now:
|
||||
if args.collection and card["collection"] != args.collection:
|
||||
continue
|
||||
due.append(card)
|
||||
_out({"ok": True, "count": len(due), "cards": due})
|
||||
|
||||
|
||||
def cmd_rate(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
now = _now()
|
||||
card = None
|
||||
for c in data["cards"]:
|
||||
if c["id"] == args.id:
|
||||
card = c
|
||||
break
|
||||
if not card:
|
||||
_out({"ok": False, "error": f"Card not found: {args.id}"})
|
||||
sys.exit(1)
|
||||
|
||||
rating = args.rating
|
||||
user_answer = getattr(args, "user_answer", None)
|
||||
if user_answer is not None:
|
||||
card["last_user_answer"] = user_answer
|
||||
|
||||
if rating == "retire":
|
||||
card["status"] = "retired"
|
||||
card["next_review_at"] = RETIRED_SENTINEL
|
||||
card["ease_streak"] = 0
|
||||
elif rating == "hard":
|
||||
card["next_review_at"] = _iso(now + timedelta(days=1))
|
||||
card["ease_streak"] = 0
|
||||
elif rating == "good":
|
||||
card["next_review_at"] = _iso(now + timedelta(days=3))
|
||||
card["ease_streak"] = 0
|
||||
elif rating == "easy":
|
||||
card["next_review_at"] = _iso(now + timedelta(days=7))
|
||||
card["ease_streak"] = card.get("ease_streak", 0) + 1
|
||||
if card["ease_streak"] >= 3:
|
||||
card["status"] = "retired"
|
||||
|
||||
_save(data)
|
||||
_out({"ok": True, "card": card})
|
||||
|
||||
|
||||
def cmd_list(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
cards = data["cards"]
|
||||
if args.collection:
|
||||
cards = [c for c in cards if c["collection"] == args.collection]
|
||||
if args.status:
|
||||
cards = [c for c in cards if c["status"] == args.status]
|
||||
_out({"ok": True, "count": len(cards), "cards": cards})
|
||||
|
||||
|
||||
def cmd_stats(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
now = _now()
|
||||
total = len(data["cards"])
|
||||
learning = sum(1 for c in data["cards"] if c["status"] == "learning")
|
||||
retired = sum(1 for c in data["cards"] if c["status"] == "retired")
|
||||
due_now = 0
|
||||
for c in data["cards"]:
|
||||
if c["status"] != "retired" and _parse_iso(c["next_review_at"]) <= now:
|
||||
due_now += 1
|
||||
|
||||
collections: dict[str, int] = {}
|
||||
for c in data["cards"]:
|
||||
name = c["collection"]
|
||||
collections[name] = collections.get(name, 0) + 1
|
||||
|
||||
_out({
|
||||
"ok": True,
|
||||
"total": total,
|
||||
"learning": learning,
|
||||
"retired": retired,
|
||||
"due_now": due_now,
|
||||
"collections": collections,
|
||||
})
|
||||
|
||||
|
||||
def cmd_export(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
output_path = Path(args.output).expanduser()
|
||||
with open(output_path, "w", newline="", encoding="utf-8") as f:
|
||||
writer = csv.writer(f, lineterminator="\n")
|
||||
for card in data["cards"]:
|
||||
writer.writerow([card["question"], card["answer"], card["collection"]])
|
||||
_out({"ok": True, "exported": len(data["cards"]), "path": str(output_path)})
|
||||
|
||||
|
||||
def cmd_import(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
now = _now()
|
||||
file_path = Path(args.file).expanduser()
|
||||
|
||||
if not file_path.exists():
|
||||
_out({"ok": False, "error": f"File not found: {file_path}"})
|
||||
sys.exit(1)
|
||||
|
||||
created = 0
|
||||
with open(file_path, "r", encoding="utf-8") as f:
|
||||
reader = csv.reader(f)
|
||||
for row in reader:
|
||||
if len(row) < 2:
|
||||
continue
|
||||
question = row[0].strip()
|
||||
answer = row[1].strip()
|
||||
collection = row[2].strip() if len(row) >= 3 and row[2].strip() else (args.collection or "Imported")
|
||||
if not question or not answer:
|
||||
continue
|
||||
card = {
|
||||
"id": str(uuid.uuid4()),
|
||||
"question": question,
|
||||
"answer": answer,
|
||||
"collection": collection,
|
||||
"status": "learning",
|
||||
"ease_streak": 0,
|
||||
"next_review_at": _iso(now),
|
||||
"created_at": _iso(now),
|
||||
"video_id": None,
|
||||
"last_user_answer": None,
|
||||
}
|
||||
data["cards"].append(card)
|
||||
created += 1
|
||||
|
||||
_save(data)
|
||||
_out({"ok": True, "imported": created})
|
||||
|
||||
|
||||
def cmd_delete(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
original = len(data["cards"])
|
||||
data["cards"] = [c for c in data["cards"] if c["id"] != args.id]
|
||||
removed = original - len(data["cards"])
|
||||
if removed == 0:
|
||||
_out({"ok": False, "error": f"Card not found: {args.id}"})
|
||||
sys.exit(1)
|
||||
_save(data)
|
||||
_out({"ok": True, "deleted": args.id})
|
||||
|
||||
|
||||
def cmd_delete_collection(args: argparse.Namespace) -> None:
|
||||
data = _load()
|
||||
original = len(data["cards"])
|
||||
data["cards"] = [c for c in data["cards"] if c["collection"] != args.collection]
|
||||
removed = original - len(data["cards"])
|
||||
_save(data)
|
||||
_out({"ok": True, "deleted_count": removed, "collection": args.collection})
|
||||
|
||||
|
||||
# ── CLI ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Memento flashcard manager")
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
p_add = sub.add_parser("add", help="Create one card")
|
||||
p_add.add_argument("--question", required=True)
|
||||
p_add.add_argument("--answer", required=True)
|
||||
p_add.add_argument("--collection", default="General")
|
||||
|
||||
p_quiz = sub.add_parser("add-quiz", help="Batch-add quiz cards")
|
||||
p_quiz.add_argument("--video-id", required=True)
|
||||
p_quiz.add_argument("--questions", required=True, help="JSON array of {question, answer}")
|
||||
p_quiz.add_argument("--collection", default="Quiz")
|
||||
|
||||
p_due = sub.add_parser("due", help="List due cards")
|
||||
p_due.add_argument("--collection", default=None)
|
||||
|
||||
p_rate = sub.add_parser("rate", help="Rate a card")
|
||||
p_rate.add_argument("--id", required=True)
|
||||
p_rate.add_argument("--rating", required=True, choices=["easy", "good", "hard", "retire"])
|
||||
p_rate.add_argument("--user-answer", default=None)
|
||||
|
||||
p_list = sub.add_parser("list", help="List cards")
|
||||
p_list.add_argument("--collection", default=None)
|
||||
p_list.add_argument("--status", default=None, choices=["learning", "retired"])
|
||||
|
||||
sub.add_parser("stats", help="Show statistics")
|
||||
|
||||
p_export = sub.add_parser("export", help="Export cards to CSV")
|
||||
p_export.add_argument("--output", required=True)
|
||||
|
||||
p_import = sub.add_parser("import", help="Import cards from CSV")
|
||||
p_import.add_argument("--file", required=True)
|
||||
p_import.add_argument("--collection", default="Imported")
|
||||
|
||||
p_del = sub.add_parser("delete", help="Delete one card")
|
||||
p_del.add_argument("--id", required=True)
|
||||
|
||||
p_delcol = sub.add_parser("delete-collection", help="Delete all cards in a collection")
|
||||
p_delcol.add_argument("--collection", required=True)
|
||||
|
||||
args = parser.parse_args()
|
||||
cmd_map = {
|
||||
"add": cmd_add,
|
||||
"add-quiz": cmd_add_quiz,
|
||||
"due": cmd_due,
|
||||
"rate": cmd_rate,
|
||||
"list": cmd_list,
|
||||
"stats": cmd_stats,
|
||||
"export": cmd_export,
|
||||
"import": cmd_import,
|
||||
"delete": cmd_delete,
|
||||
"delete-collection": cmd_delete_collection,
|
||||
}
|
||||
cmd_map[args.command](args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,88 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fetch YouTube transcripts for Memento quiz generation.
|
||||
|
||||
Requires: pip install youtube-transcript-api
|
||||
The quiz question *generation* is done by the agent's LLM — this script only fetches transcripts.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
|
||||
|
||||
def _out(obj: object) -> None:
|
||||
json.dump(obj, sys.stdout, indent=2, ensure_ascii=False)
|
||||
sys.stdout.write("\n")
|
||||
|
||||
|
||||
def _normalize_segments(segments: list) -> str:
|
||||
parts = []
|
||||
for seg in segments:
|
||||
text = str(seg.get("text", "")).strip()
|
||||
if text:
|
||||
parts.append(text)
|
||||
return re.sub(r"\s+", " ", " ".join(parts)).strip()
|
||||
|
||||
|
||||
def cmd_fetch(args: argparse.Namespace) -> None:
|
||||
try:
|
||||
import youtube_transcript_api # noqa: F811
|
||||
except ImportError:
|
||||
_out({
|
||||
"ok": False,
|
||||
"error": "missing_dependency",
|
||||
"message": "Run: pip install youtube-transcript-api",
|
||||
})
|
||||
sys.exit(1)
|
||||
|
||||
video_id = args.video_id
|
||||
languages = ["en", "en-US", "en-GB", "en-CA", "en-AU"]
|
||||
|
||||
api = youtube_transcript_api.YouTubeTranscriptApi()
|
||||
try:
|
||||
raw = api.fetch(video_id, languages=languages)
|
||||
except Exception as exc:
|
||||
error_type = type(exc).__name__
|
||||
_out({
|
||||
"ok": False,
|
||||
"error": "transcript_unavailable",
|
||||
"error_type": error_type,
|
||||
"message": f"Could not fetch transcript for {video_id}: {exc}",
|
||||
})
|
||||
sys.exit(1)
|
||||
|
||||
segments = raw
|
||||
if hasattr(raw, "to_raw_data"):
|
||||
segments = raw.to_raw_data()
|
||||
|
||||
text = _normalize_segments(segments)
|
||||
if not text:
|
||||
_out({
|
||||
"ok": False,
|
||||
"error": "empty_transcript",
|
||||
"message": f"Transcript for {video_id} contained no usable text.",
|
||||
})
|
||||
sys.exit(1)
|
||||
|
||||
_out({
|
||||
"ok": True,
|
||||
"video_id": video_id,
|
||||
"transcript": text,
|
||||
})
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Memento YouTube transcript fetcher")
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
p_fetch = sub.add_parser("fetch", help="Fetch transcript for a video")
|
||||
p_fetch.add_argument("video_id", help="YouTube video ID")
|
||||
|
||||
args = parser.parse_args()
|
||||
if args.command == "fetch":
|
||||
cmd_fetch(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,297 +0,0 @@
|
||||
---
|
||||
name: siyuan
|
||||
description: SiYuan Note API for searching, reading, creating, and managing blocks and documents in a self-hosted knowledge base via curl.
|
||||
version: 1.0.0
|
||||
author: FEUAZUR
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [SiYuan, Notes, Knowledge Base, PKM, API]
|
||||
related_skills: [obsidian, notion]
|
||||
homepage: https://github.com/siyuan-note/siyuan
|
||||
prerequisites:
|
||||
env_vars: [SIYUAN_TOKEN]
|
||||
commands: [curl, jq]
|
||||
required_environment_variables:
|
||||
- name: SIYUAN_TOKEN
|
||||
prompt: SiYuan API token
|
||||
help: "Settings > About in SiYuan desktop app"
|
||||
- name: SIYUAN_URL
|
||||
prompt: SiYuan instance URL (default http://127.0.0.1:6806)
|
||||
required_for: remote instances
|
||||
---
|
||||
|
||||
# SiYuan Note API
|
||||
|
||||
Use the [SiYuan](https://github.com/siyuan-note/siyuan) kernel API via curl to search, read, create, update, and delete blocks and documents in a self-hosted knowledge base. No extra tools needed -- just curl and an API token.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Install and run SiYuan (desktop or Docker)
|
||||
2. Get your API token: **Settings > About > API token**
|
||||
3. Store it in `~/.hermes/.env`:
|
||||
```
|
||||
SIYUAN_TOKEN=your_token_here
|
||||
SIYUAN_URL=http://127.0.0.1:6806
|
||||
```
|
||||
`SIYUAN_URL` defaults to `http://127.0.0.1:6806` if not set.
|
||||
|
||||
## API Basics
|
||||
|
||||
All SiYuan API calls are **POST with JSON body**. Every request follows this pattern:
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/..." \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"param": "value"}'
|
||||
```
|
||||
|
||||
Responses are JSON with this structure:
|
||||
```json
|
||||
{"code": 0, "msg": "", "data": { ... }}
|
||||
```
|
||||
`code: 0` means success. Any other value is an error -- check `msg` for details.
|
||||
|
||||
**ID format:** SiYuan IDs look like `20210808180117-6v0mkxr` (14-digit timestamp + 7 alphanumeric chars).
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Operation | Endpoint |
|
||||
|-----------|----------|
|
||||
| Full-text search | `/api/search/fullTextSearchBlock` |
|
||||
| SQL query | `/api/query/sql` |
|
||||
| Read block | `/api/block/getBlockKramdown` |
|
||||
| Read children | `/api/block/getChildBlocks` |
|
||||
| Get path | `/api/filetree/getHPathByID` |
|
||||
| Get attributes | `/api/attr/getBlockAttrs` |
|
||||
| List notebooks | `/api/notebook/lsNotebooks` |
|
||||
| List documents | `/api/filetree/listDocsByPath` |
|
||||
| Create notebook | `/api/notebook/createNotebook` |
|
||||
| Create document | `/api/filetree/createDocWithMd` |
|
||||
| Append block | `/api/block/appendBlock` |
|
||||
| Update block | `/api/block/updateBlock` |
|
||||
| Rename document | `/api/filetree/renameDocByID` |
|
||||
| Set attributes | `/api/attr/setBlockAttrs` |
|
||||
| Delete block | `/api/block/deleteBlock` |
|
||||
| Delete document | `/api/filetree/removeDocByID` |
|
||||
| Export as Markdown | `/api/export/exportMdContent` |
|
||||
|
||||
## Common Operations
|
||||
|
||||
### Search (Full-Text)
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/search/fullTextSearchBlock" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"query": "meeting notes", "page": 0}' | jq '.data.blocks[:5]'
|
||||
```
|
||||
|
||||
### Search (SQL)
|
||||
|
||||
Query the blocks database directly. Only SELECT statements are safe.
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/query/sql" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"stmt": "SELECT id, content, type, box FROM blocks WHERE content LIKE '\''%keyword%'\'' AND type='\''p'\'' LIMIT 20"}' | jq '.data'
|
||||
```
|
||||
|
||||
Useful columns: `id`, `parent_id`, `root_id`, `box` (notebook ID), `path`, `content`, `type`, `subtype`, `created`, `updated`.
|
||||
|
||||
### Read Block Content
|
||||
|
||||
Returns block content in Kramdown (Markdown-like) format.
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/block/getBlockKramdown" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"id": "20210808180117-6v0mkxr"}' | jq '.data.kramdown'
|
||||
```
|
||||
|
||||
### Read Child Blocks
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/block/getChildBlocks" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"id": "20210808180117-6v0mkxr"}' | jq '.data'
|
||||
```
|
||||
|
||||
### Get Human-Readable Path
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/filetree/getHPathByID" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"id": "20210808180117-6v0mkxr"}' | jq '.data'
|
||||
```
|
||||
|
||||
### Get Block Attributes
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/attr/getBlockAttrs" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"id": "20210808180117-6v0mkxr"}' | jq '.data'
|
||||
```
|
||||
|
||||
### List Notebooks
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/notebook/lsNotebooks" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{}' | jq '.data.notebooks[] | {id, name, closed}'
|
||||
```
|
||||
|
||||
### List Documents in a Notebook
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/filetree/listDocsByPath" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"notebook": "NOTEBOOK_ID", "path": "/"}' | jq '.data.files[] | {id, name}'
|
||||
```
|
||||
|
||||
### Create a Document
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/filetree/createDocWithMd" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"notebook": "NOTEBOOK_ID",
|
||||
"path": "/Meeting Notes/2026-03-22",
|
||||
"markdown": "# Meeting Notes\n\n- Discussed project timeline\n- Assigned tasks"
|
||||
}' | jq '.data'
|
||||
```
|
||||
|
||||
### Create a Notebook
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/notebook/createNotebook" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name": "My New Notebook"}' | jq '.data.notebook.id'
|
||||
```
|
||||
|
||||
### Append Block to Document
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/block/appendBlock" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"parentID": "DOCUMENT_OR_BLOCK_ID",
|
||||
"data": "New paragraph added at the end.",
|
||||
"dataType": "markdown"
|
||||
}' | jq '.data'
|
||||
```
|
||||
|
||||
Also available: `/api/block/prependBlock` (same params, inserts at the beginning) and `/api/block/insertBlock` (uses `previousID` instead of `parentID` to insert after a specific block).
|
||||
|
||||
### Update Block Content
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/block/updateBlock" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"id": "BLOCK_ID",
|
||||
"data": "Updated content here.",
|
||||
"dataType": "markdown"
|
||||
}' | jq '.data'
|
||||
```
|
||||
|
||||
### Rename a Document
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/filetree/renameDocByID" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"id": "DOCUMENT_ID", "title": "New Title"}'
|
||||
```
|
||||
|
||||
### Set Block Attributes
|
||||
|
||||
Custom attributes must be prefixed with `custom-`:
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/attr/setBlockAttrs" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"id": "BLOCK_ID",
|
||||
"attrs": {
|
||||
"custom-status": "reviewed",
|
||||
"custom-priority": "high"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### Delete a Block
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/block/deleteBlock" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"id": "BLOCK_ID"}'
|
||||
```
|
||||
|
||||
To delete a whole document: use `/api/filetree/removeDocByID` with `{"id": "DOC_ID"}`.
|
||||
To delete a notebook: use `/api/notebook/removeNotebook` with `{"notebook": "NOTEBOOK_ID"}`.
|
||||
|
||||
### Export Document as Markdown
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${SIYUAN_URL:-http://127.0.0.1:6806}/api/export/exportMdContent" \
|
||||
-H "Authorization: Token $SIYUAN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"id": "DOCUMENT_ID"}' | jq -r '.data.content'
|
||||
```
|
||||
|
||||
## Block Types
|
||||
|
||||
Common `type` values in SQL queries:
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `d` | Document (root block) |
|
||||
| `p` | Paragraph |
|
||||
| `h` | Heading |
|
||||
| `l` | List |
|
||||
| `i` | List item |
|
||||
| `c` | Code block |
|
||||
| `m` | Math block |
|
||||
| `t` | Table |
|
||||
| `b` | Blockquote |
|
||||
| `s` | Super block |
|
||||
| `html` | HTML block |
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **All endpoints are POST** -- even read-only operations. Do not use GET.
|
||||
- **SQL safety**: only use SELECT queries. INSERT/UPDATE/DELETE/DROP are dangerous and should never be sent.
|
||||
- **ID validation**: IDs match the pattern `YYYYMMDDHHmmss-xxxxxxx`. Reject anything else.
|
||||
- **Error responses**: always check `code != 0` in responses before processing `data`.
|
||||
- **Large documents**: block content and export results can be very large. Use `LIMIT` in SQL and pipe through `jq` to extract only what you need.
|
||||
- **Notebook IDs**: when working with a specific notebook, get its ID first via `lsNotebooks`.
|
||||
|
||||
## Alternative: MCP Server
|
||||
|
||||
If you prefer a native integration instead of curl, install the SiYuan MCP server:
|
||||
|
||||
```yaml
|
||||
# In ~/.hermes/config.yaml under mcp_servers:
|
||||
mcp_servers:
|
||||
siyuan:
|
||||
command: npx
|
||||
args: ["-y", "@porkll/siyuan-mcp"]
|
||||
env:
|
||||
SIYUAN_TOKEN: "your_token"
|
||||
SIYUAN_URL: "http://127.0.0.1:6806"
|
||||
```
|
||||
@@ -1,335 +0,0 @@
|
||||
---
|
||||
name: scrapling
|
||||
description: Web scraping with Scrapling - HTTP fetching, stealth browser automation, Cloudflare bypass, and spider crawling via CLI and Python.
|
||||
version: 1.0.0
|
||||
author: FEUAZUR
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Web Scraping, Browser, Cloudflare, Stealth, Crawling, Spider]
|
||||
related_skills: [duckduckgo-search, domain-intel]
|
||||
homepage: https://github.com/D4Vinci/Scrapling
|
||||
prerequisites:
|
||||
commands: [scrapling, python]
|
||||
---
|
||||
|
||||
# Scrapling
|
||||
|
||||
[Scrapling](https://github.com/D4Vinci/Scrapling) is a web scraping framework with anti-bot bypass, stealth browser automation, and a spider framework. It provides three fetching strategies (HTTP, dynamic JS, stealth/Cloudflare) and a full CLI.
|
||||
|
||||
**This skill is for educational and research purposes only.** Users must comply with local/international data scraping laws and respect website Terms of Service.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Scraping static HTML pages (faster than browser tools)
|
||||
- Scraping JS-rendered pages that need a real browser
|
||||
- Bypassing Cloudflare Turnstile or bot detection
|
||||
- Crawling multiple pages with a spider
|
||||
- When the built-in `web_extract` tool does not return the data you need
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install "scrapling[all]"
|
||||
scrapling install
|
||||
```
|
||||
|
||||
Minimal install (HTTP only, no browser):
|
||||
```bash
|
||||
pip install scrapling
|
||||
```
|
||||
|
||||
With browser automation only:
|
||||
```bash
|
||||
pip install "scrapling[fetchers]"
|
||||
scrapling install
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Approach | Class | Use When |
|
||||
|----------|-------|----------|
|
||||
| HTTP | `Fetcher` / `FetcherSession` | Static pages, APIs, fast bulk requests |
|
||||
| Dynamic | `DynamicFetcher` / `DynamicSession` | JS-rendered content, SPAs |
|
||||
| Stealth | `StealthyFetcher` / `StealthySession` | Cloudflare, anti-bot protected sites |
|
||||
| Spider | `Spider` | Multi-page crawling with link following |
|
||||
|
||||
## CLI Usage
|
||||
|
||||
### Extract Static Page
|
||||
|
||||
```bash
|
||||
scrapling extract get 'https://example.com' output.md
|
||||
```
|
||||
|
||||
With CSS selector and browser impersonation:
|
||||
|
||||
```bash
|
||||
scrapling extract get 'https://example.com' output.md \
|
||||
--css-selector '.content' \
|
||||
--impersonate 'chrome'
|
||||
```
|
||||
|
||||
### Extract JS-Rendered Page
|
||||
|
||||
```bash
|
||||
scrapling extract fetch 'https://example.com' output.md \
|
||||
--css-selector '.dynamic-content' \
|
||||
--disable-resources \
|
||||
--network-idle
|
||||
```
|
||||
|
||||
### Extract Cloudflare-Protected Page
|
||||
|
||||
```bash
|
||||
scrapling extract stealthy-fetch 'https://protected-site.com' output.html \
|
||||
--solve-cloudflare \
|
||||
--block-webrtc \
|
||||
--hide-canvas
|
||||
```
|
||||
|
||||
### POST Request
|
||||
|
||||
```bash
|
||||
scrapling extract post 'https://example.com/api' output.json \
|
||||
--json '{"query": "search term"}'
|
||||
```
|
||||
|
||||
### Output Formats
|
||||
|
||||
The output format is determined by the file extension:
|
||||
- `.html` -- raw HTML
|
||||
- `.md` -- converted to Markdown
|
||||
- `.txt` -- plain text
|
||||
- `.json` / `.jsonl` -- JSON
|
||||
|
||||
## Python: HTTP Scraping
|
||||
|
||||
### Single Request
|
||||
|
||||
```python
|
||||
from scrapling.fetchers import Fetcher
|
||||
|
||||
page = Fetcher.get('https://quotes.toscrape.com/')
|
||||
quotes = page.css('.quote .text::text').getall()
|
||||
for q in quotes:
|
||||
print(q)
|
||||
```
|
||||
|
||||
### Session (Persistent Cookies)
|
||||
|
||||
```python
|
||||
from scrapling.fetchers import FetcherSession
|
||||
|
||||
with FetcherSession(impersonate='chrome') as session:
|
||||
page = session.get('https://example.com/', stealthy_headers=True)
|
||||
links = page.css('a::attr(href)').getall()
|
||||
for link in links[:5]:
|
||||
sub = session.get(link)
|
||||
print(sub.css('h1::text').get())
|
||||
```
|
||||
|
||||
### POST / PUT / DELETE
|
||||
|
||||
```python
|
||||
page = Fetcher.post('https://api.example.com/data', json={"key": "value"})
|
||||
page = Fetcher.put('https://api.example.com/item/1', data={"name": "updated"})
|
||||
page = Fetcher.delete('https://api.example.com/item/1')
|
||||
```
|
||||
|
||||
### With Proxy
|
||||
|
||||
```python
|
||||
page = Fetcher.get('https://example.com', proxy='http://user:pass@proxy:8080')
|
||||
```
|
||||
|
||||
## Python: Dynamic Pages (JS-Rendered)
|
||||
|
||||
For pages that require JavaScript execution (SPAs, lazy-loaded content):
|
||||
|
||||
```python
|
||||
from scrapling.fetchers import DynamicFetcher
|
||||
|
||||
page = DynamicFetcher.fetch('https://example.com', headless=True)
|
||||
data = page.css('.js-loaded-content::text').getall()
|
||||
```
|
||||
|
||||
### Wait for Specific Element
|
||||
|
||||
```python
|
||||
page = DynamicFetcher.fetch(
|
||||
'https://example.com',
|
||||
wait_selector=('.results', 'visible'),
|
||||
network_idle=True,
|
||||
)
|
||||
```
|
||||
|
||||
### Disable Resources for Speed
|
||||
|
||||
Blocks fonts, images, media, stylesheets (~25% faster):
|
||||
|
||||
```python
|
||||
from scrapling.fetchers import DynamicSession
|
||||
|
||||
with DynamicSession(headless=True, disable_resources=True, network_idle=True) as session:
|
||||
page = session.fetch('https://example.com')
|
||||
items = page.css('.item::text').getall()
|
||||
```
|
||||
|
||||
### Custom Page Automation
|
||||
|
||||
```python
|
||||
from playwright.sync_api import Page
|
||||
from scrapling.fetchers import DynamicFetcher
|
||||
|
||||
def scroll_and_click(page: Page):
|
||||
page.mouse.wheel(0, 3000)
|
||||
page.wait_for_timeout(1000)
|
||||
page.click('button.load-more')
|
||||
page.wait_for_selector('.extra-results')
|
||||
|
||||
page = DynamicFetcher.fetch('https://example.com', page_action=scroll_and_click)
|
||||
results = page.css('.extra-results .item::text').getall()
|
||||
```
|
||||
|
||||
## Python: Stealth Mode (Anti-Bot Bypass)
|
||||
|
||||
For Cloudflare-protected or heavily fingerprinted sites:
|
||||
|
||||
```python
|
||||
from scrapling.fetchers import StealthyFetcher
|
||||
|
||||
page = StealthyFetcher.fetch(
|
||||
'https://protected-site.com',
|
||||
headless=True,
|
||||
solve_cloudflare=True,
|
||||
block_webrtc=True,
|
||||
hide_canvas=True,
|
||||
)
|
||||
content = page.css('.protected-content::text').getall()
|
||||
```
|
||||
|
||||
### Stealth Session
|
||||
|
||||
```python
|
||||
from scrapling.fetchers import StealthySession
|
||||
|
||||
with StealthySession(headless=True, solve_cloudflare=True) as session:
|
||||
page1 = session.fetch('https://protected-site.com/page1')
|
||||
page2 = session.fetch('https://protected-site.com/page2')
|
||||
```
|
||||
|
||||
## Element Selection
|
||||
|
||||
All fetchers return a `Selector` object with these methods:
|
||||
|
||||
### CSS Selectors
|
||||
|
||||
```python
|
||||
page.css('h1::text').get() # First h1 text
|
||||
page.css('a::attr(href)').getall() # All link hrefs
|
||||
page.css('.quote .text::text').getall() # Nested selection
|
||||
```
|
||||
|
||||
### XPath
|
||||
|
||||
```python
|
||||
page.xpath('//div[@class="content"]/text()').getall()
|
||||
page.xpath('//a/@href').getall()
|
||||
```
|
||||
|
||||
### Find Methods
|
||||
|
||||
```python
|
||||
page.find_all('div', class_='quote') # By tag + attribute
|
||||
page.find_by_text('Read more', tag='a') # By text content
|
||||
page.find_by_regex(r'\$\d+\.\d{2}') # By regex pattern
|
||||
```
|
||||
|
||||
### Similar Elements
|
||||
|
||||
Find elements with similar structure (useful for product listings, etc.):
|
||||
|
||||
```python
|
||||
first_product = page.css('.product')[0]
|
||||
all_similar = first_product.find_similar()
|
||||
```
|
||||
|
||||
### Navigation
|
||||
|
||||
```python
|
||||
el = page.css('.target')[0]
|
||||
el.parent # Parent element
|
||||
el.children # Child elements
|
||||
el.next_sibling # Next sibling
|
||||
el.prev_sibling # Previous sibling
|
||||
```
|
||||
|
||||
## Python: Spider Framework
|
||||
|
||||
For multi-page crawling with link following:
|
||||
|
||||
```python
|
||||
from scrapling.spiders import Spider, Request, Response
|
||||
|
||||
class QuotesSpider(Spider):
|
||||
name = "quotes"
|
||||
start_urls = ["https://quotes.toscrape.com/"]
|
||||
concurrent_requests = 10
|
||||
download_delay = 1
|
||||
|
||||
async def parse(self, response: Response):
|
||||
for quote in response.css('.quote'):
|
||||
yield {
|
||||
"text": quote.css('.text::text').get(),
|
||||
"author": quote.css('.author::text').get(),
|
||||
"tags": quote.css('.tag::text').getall(),
|
||||
}
|
||||
|
||||
next_page = response.css('.next a::attr(href)').get()
|
||||
if next_page:
|
||||
yield response.follow(next_page)
|
||||
|
||||
result = QuotesSpider().start()
|
||||
print(f"Scraped {len(result.items)} quotes")
|
||||
result.items.to_json("quotes.json")
|
||||
```
|
||||
|
||||
### Multi-Session Spider
|
||||
|
||||
Route requests to different fetcher types:
|
||||
|
||||
```python
|
||||
from scrapling.fetchers import FetcherSession, AsyncStealthySession
|
||||
|
||||
class SmartSpider(Spider):
|
||||
name = "smart"
|
||||
start_urls = ["https://example.com/"]
|
||||
|
||||
def configure_sessions(self, manager):
|
||||
manager.add("fast", FetcherSession(impersonate="chrome"))
|
||||
manager.add("stealth", AsyncStealthySession(headless=True), lazy=True)
|
||||
|
||||
async def parse(self, response: Response):
|
||||
for link in response.css('a::attr(href)').getall():
|
||||
if "protected" in link:
|
||||
yield Request(link, sid="stealth")
|
||||
else:
|
||||
yield Request(link, sid="fast", callback=self.parse)
|
||||
```
|
||||
|
||||
### Pause/Resume Crawling
|
||||
|
||||
```python
|
||||
spider = QuotesSpider(crawldir="./crawl_checkpoint")
|
||||
spider.start() # Ctrl+C to pause, re-run to resume from checkpoint
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Browser install required**: run `scrapling install` after pip install -- without it, `DynamicFetcher` and `StealthyFetcher` will fail
|
||||
- **Timeouts**: DynamicFetcher/StealthyFetcher timeout is in **milliseconds** (default 30000), Fetcher timeout is in **seconds**
|
||||
- **Cloudflare bypass**: `solve_cloudflare=True` adds 5-15 seconds to fetch time -- only enable when needed
|
||||
- **Resource usage**: StealthyFetcher runs a real browser -- limit concurrent usage
|
||||
- **Legal**: always check robots.txt and website ToS before scraping. This library is for educational and research purposes
|
||||
- **Python version**: requires Python 3.10+
|
||||
+1
-4
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "hermes-agent"
|
||||
version = "0.6.0"
|
||||
version = "0.5.0"
|
||||
description = "The self-improving AI agent — creates skills from experience, improves them during use, and runs anywhere"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
@@ -58,7 +58,6 @@ homeassistant = ["aiohttp>=3.9.0,<4"]
|
||||
sms = ["aiohttp>=3.9.0,<4"]
|
||||
acp = ["agent-client-protocol>=0.8.1,<0.9"]
|
||||
dingtalk = ["dingtalk-stream>=0.1.0,<1"]
|
||||
feishu = ["lark-oapi>=1.5.3,<2"]
|
||||
rl = [
|
||||
"atroposlib @ git+https://github.com/NousResearch/atropos.git",
|
||||
"tinker @ git+https://github.com/thinking-machines-lab/tinker.git",
|
||||
@@ -71,7 +70,6 @@ all = [
|
||||
"hermes-agent[modal]",
|
||||
"hermes-agent[daytona]",
|
||||
"hermes-agent[messaging]",
|
||||
"hermes-agent[matrix]",
|
||||
"hermes-agent[cron]",
|
||||
"hermes-agent[cli]",
|
||||
"hermes-agent[dev]",
|
||||
@@ -85,7 +83,6 @@ all = [
|
||||
"hermes-agent[acp]",
|
||||
"hermes-agent[voice]",
|
||||
"hermes-agent[dingtalk]",
|
||||
"hermes-agent[feishu]",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
|
||||
+36
-87
@@ -45,7 +45,7 @@ import fire
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from hermes_constants import get_hermes_home
|
||||
from hermes_constants import get_hermes_home, display_hermes_home
|
||||
|
||||
# Load .env from ~/.hermes/.env first, then project root as dev fallback.
|
||||
# User-managed env files should override stale shell exports on restart.
|
||||
@@ -896,30 +896,16 @@ class AIAgent:
|
||||
except Exception as e:
|
||||
raise RuntimeError(f"Failed to initialize OpenAI client: {e}")
|
||||
|
||||
# Provider fallback chain — ordered list of backup providers tried
|
||||
# when the primary is exhausted (rate-limit, overload, connection
|
||||
# failure). Supports both legacy single-dict ``fallback_model`` and
|
||||
# new list ``fallback_providers`` format.
|
||||
if isinstance(fallback_model, list):
|
||||
self._fallback_chain = [
|
||||
f for f in fallback_model
|
||||
if isinstance(f, dict) and f.get("provider") and f.get("model")
|
||||
]
|
||||
elif isinstance(fallback_model, dict) and fallback_model.get("provider") and fallback_model.get("model"):
|
||||
self._fallback_chain = [fallback_model]
|
||||
else:
|
||||
self._fallback_chain = []
|
||||
self._fallback_index = 0
|
||||
# Provider fallback — a single backup model/provider tried when the
|
||||
# primary is exhausted (rate-limit, overload, connection failure).
|
||||
# Config shape: {"provider": "openrouter", "model": "anthropic/claude-sonnet-4"}
|
||||
self._fallback_model = fallback_model if isinstance(fallback_model, dict) else None
|
||||
self._fallback_activated = False
|
||||
# Legacy attribute kept for backward compat (tests, external callers)
|
||||
self._fallback_model = self._fallback_chain[0] if self._fallback_chain else None
|
||||
if self._fallback_chain and not self.quiet_mode:
|
||||
if len(self._fallback_chain) == 1:
|
||||
fb = self._fallback_chain[0]
|
||||
print(f"🔄 Fallback model: {fb['model']} ({fb['provider']})")
|
||||
else:
|
||||
print(f"🔄 Fallback chain ({len(self._fallback_chain)} providers): " +
|
||||
" → ".join(f"{f['model']} ({f['provider']})" for f in self._fallback_chain))
|
||||
if self._fallback_model:
|
||||
fb_p = self._fallback_model.get("provider", "")
|
||||
fb_m = self._fallback_model.get("model", "")
|
||||
if fb_p and fb_m and not self.quiet_mode:
|
||||
print(f"🔄 Fallback model: {fb_m} ({fb_p})")
|
||||
|
||||
# Get available tools with filtering
|
||||
self.tools = get_tool_definitions(
|
||||
@@ -1285,7 +1271,7 @@ class AIAgent:
|
||||
try:
|
||||
fn = self._print_fn or print
|
||||
fn(*args, **kwargs)
|
||||
except (OSError, ValueError):
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _vprint(self, *args, force: bool = False, **kwargs):
|
||||
@@ -2907,19 +2893,6 @@ class AIAgent:
|
||||
})
|
||||
return converted or None
|
||||
|
||||
@staticmethod
|
||||
def _deterministic_call_id(fn_name: str, arguments: str, index: int = 0) -> str:
|
||||
"""Generate a deterministic call_id from tool call content.
|
||||
|
||||
Used as a fallback when the API doesn't provide a call_id.
|
||||
Deterministic IDs prevent cache invalidation — random UUIDs would
|
||||
make every API call's prefix unique, breaking OpenAI's prompt cache.
|
||||
"""
|
||||
import hashlib
|
||||
seed = f"{fn_name}:{arguments}:{index}"
|
||||
digest = hashlib.sha256(seed.encode("utf-8", errors="replace")).hexdigest()[:12]
|
||||
return f"call_{digest}"
|
||||
|
||||
@staticmethod
|
||||
def _split_responses_tool_id(raw_id: Any) -> tuple[Optional[str], Optional[str]]:
|
||||
"""Split a stored tool id into (call_id, response_item_id)."""
|
||||
@@ -3026,8 +2999,7 @@ class AIAgent:
|
||||
):
|
||||
call_id = f"call_{embedded_response_item_id[len('fc_'):]}"
|
||||
else:
|
||||
_raw_args = str(fn.get("arguments", "{}"))
|
||||
call_id = self._deterministic_call_id(fn_name, _raw_args, len(items))
|
||||
call_id = f"call_{uuid.uuid4().hex[:12]}"
|
||||
call_id = call_id.strip()
|
||||
|
||||
arguments = fn.get("arguments", "{}")
|
||||
@@ -3391,7 +3363,7 @@ class AIAgent:
|
||||
embedded_call_id, _ = self._split_responses_tool_id(raw_item_id)
|
||||
call_id = raw_call_id if isinstance(raw_call_id, str) and raw_call_id.strip() else embedded_call_id
|
||||
if not isinstance(call_id, str) or not call_id.strip():
|
||||
call_id = self._deterministic_call_id(fn_name, arguments, len(tool_calls))
|
||||
call_id = f"call_{uuid.uuid4().hex[:12]}"
|
||||
call_id = call_id.strip()
|
||||
response_item_id = raw_item_id if isinstance(raw_item_id, str) else None
|
||||
response_item_id = self._derive_responses_function_call_id(call_id, response_item_id)
|
||||
@@ -3412,7 +3384,7 @@ class AIAgent:
|
||||
embedded_call_id, _ = self._split_responses_tool_id(raw_item_id)
|
||||
call_id = raw_call_id if isinstance(raw_call_id, str) and raw_call_id.strip() else embedded_call_id
|
||||
if not isinstance(call_id, str) or not call_id.strip():
|
||||
call_id = self._deterministic_call_id(fn_name, arguments, len(tool_calls))
|
||||
call_id = f"call_{uuid.uuid4().hex[:12]}"
|
||||
call_id = call_id.strip()
|
||||
response_item_id = raw_item_id if isinstance(raw_item_id, str) else None
|
||||
response_item_id = self._derive_responses_function_call_id(call_id, response_item_id)
|
||||
@@ -4346,26 +4318,25 @@ class AIAgent:
|
||||
# ── Provider fallback ──────────────────────────────────────────────────
|
||||
|
||||
def _try_activate_fallback(self) -> bool:
|
||||
"""Switch to the next fallback model/provider in the chain.
|
||||
"""Switch to the configured fallback model/provider.
|
||||
|
||||
Called when the current model is failing after retries. Swaps the
|
||||
Called when the primary model is failing after retries. Swaps the
|
||||
OpenAI client, model slug, and provider in-place so the retry loop
|
||||
can continue with the new backend. Advances through the chain on
|
||||
each call; returns False when exhausted.
|
||||
can continue with the new backend. One-shot: returns False if
|
||||
already activated or not configured.
|
||||
|
||||
Uses the centralized provider router (resolve_provider_client) for
|
||||
auth resolution and client construction — no duplicated provider→key
|
||||
mappings.
|
||||
"""
|
||||
if self._fallback_index >= len(self._fallback_chain):
|
||||
if self._fallback_activated or not self._fallback_model:
|
||||
return False
|
||||
|
||||
fb = self._fallback_chain[self._fallback_index]
|
||||
self._fallback_index += 1
|
||||
fb = self._fallback_model
|
||||
fb_provider = (fb.get("provider") or "").strip().lower()
|
||||
fb_model = (fb.get("model") or "").strip()
|
||||
if not fb_provider or not fb_model:
|
||||
return self._try_activate_fallback() # skip invalid, try next
|
||||
return False
|
||||
|
||||
# Use centralized router for client construction.
|
||||
# raw_codex=True because the main agent needs direct responses.stream()
|
||||
@@ -4378,7 +4349,7 @@ class AIAgent:
|
||||
logging.warning(
|
||||
"Fallback to %s failed: provider not configured",
|
||||
fb_provider)
|
||||
return self._try_activate_fallback() # try next in chain
|
||||
return False
|
||||
|
||||
# Determine api_mode from provider / base URL
|
||||
fb_api_mode = "chat_completions"
|
||||
@@ -4453,8 +4424,8 @@ class AIAgent:
|
||||
)
|
||||
return True
|
||||
except Exception as e:
|
||||
logging.error("Failed to activate fallback %s: %s", fb_model, e)
|
||||
return self._try_activate_fallback() # try next in chain
|
||||
logging.error("Failed to activate fallback model: %s", e)
|
||||
return False
|
||||
|
||||
# ── End provider fallback ──────────────────────────────────────────────
|
||||
|
||||
@@ -4735,10 +4706,9 @@ class AIAgent:
|
||||
api_kwargs = {
|
||||
"model": self.model,
|
||||
"messages": sanitized_messages,
|
||||
"tools": self.tools if self.tools else None,
|
||||
"timeout": float(os.getenv("HERMES_API_TIMEOUT", 1800.0)),
|
||||
}
|
||||
if self.tools:
|
||||
api_kwargs["tools"] = self.tools
|
||||
|
||||
if self.max_tokens is not None:
|
||||
api_kwargs.update(self._max_tokens_param(self.max_tokens))
|
||||
@@ -4947,10 +4917,7 @@ class AIAgent:
|
||||
if isinstance(raw_id, str) and raw_id.strip():
|
||||
call_id = raw_id.strip()
|
||||
else:
|
||||
_fn = getattr(tool_call, "function", None)
|
||||
_fn_name = getattr(_fn, "name", "") if _fn else ""
|
||||
_fn_args = getattr(_fn, "arguments", "{}") if _fn else "{}"
|
||||
call_id = self._deterministic_call_id(_fn_name, _fn_args, len(tool_calls))
|
||||
call_id = f"call_{uuid.uuid4().hex[:12]}"
|
||||
call_id = call_id.strip()
|
||||
|
||||
response_item_id = getattr(tool_call, "response_item_id", None)
|
||||
@@ -5200,8 +5167,6 @@ class AIAgent:
|
||||
self._session_db.end_session(self.session_id, "compression")
|
||||
old_session_id = self.session_id
|
||||
self.session_id = f"{datetime.now().strftime('%Y%m%d_%H%M%S')}_{uuid.uuid4().hex[:6]}"
|
||||
# Update session_log_file to point to the new session's JSON file
|
||||
self.session_log_file = self.logs_dir / f"session_{self.session_id}.json"
|
||||
self._session_db.create_session(
|
||||
session_id=self.session_id,
|
||||
source=self.platform or os.environ.get("HERMES_SESSION_SOURCE", "cli"),
|
||||
@@ -5679,6 +5644,8 @@ class AIAgent:
|
||||
face = random.choice(KawaiiSpinner.KAWAII_WAITING)
|
||||
emoji = _get_tool_emoji(function_name)
|
||||
preview = _build_tool_preview(function_name, function_args) or function_name
|
||||
if len(preview) > 30:
|
||||
preview = preview[:27] + "..."
|
||||
spinner = KawaiiSpinner(f"{face} {emoji} {preview}", spinner_type='dots', print_fn=self._print_fn)
|
||||
spinner.start()
|
||||
_spinner_result = None
|
||||
@@ -6561,9 +6528,9 @@ class AIAgent:
|
||||
# Eager fallback: empty/malformed responses are a common
|
||||
# rate-limit symptom. Switch to fallback immediately
|
||||
# rather than retrying with extended backoff.
|
||||
if self._fallback_index < len(self._fallback_chain):
|
||||
if not self._fallback_activated:
|
||||
self._emit_status("⚠️ Empty/malformed response — switching to fallback...")
|
||||
if self._try_activate_fallback():
|
||||
if not self._fallback_activated and self._try_activate_fallback():
|
||||
retry_count = 0
|
||||
continue
|
||||
|
||||
@@ -6957,8 +6924,7 @@ class AIAgent:
|
||||
print(f"{self.log_prefix} Auth method: {auth_method}")
|
||||
print(f"{self.log_prefix} Token prefix: {key[:12]}..." if key and len(key) > 12 else f"{self.log_prefix} Token: (empty or short)")
|
||||
print(f"{self.log_prefix} Troubleshooting:")
|
||||
from hermes_constants import display_hermes_home as _dhh_fn
|
||||
_dhh = _dhh_fn()
|
||||
_dhh = display_hermes_home()
|
||||
print(f"{self.log_prefix} • Check ANTHROPIC_TOKEN in {_dhh}/.env for Hermes-managed OAuth/setup tokens")
|
||||
print(f"{self.log_prefix} • Check ANTHROPIC_API_KEY in {_dhh}/.env for API keys or legacy token values")
|
||||
print(f"{self.log_prefix} • For API keys: verify at https://console.anthropic.com/settings/keys")
|
||||
@@ -7026,7 +6992,7 @@ class AIAgent:
|
||||
or "usage limit" in error_msg
|
||||
or "quota" in error_msg
|
||||
)
|
||||
if is_rate_limited and self._fallback_index < len(self._fallback_chain):
|
||||
if is_rate_limited and not self._fallback_activated:
|
||||
self._emit_status("⚠️ Rate limited — switching to fallback provider...")
|
||||
if self._try_activate_fallback():
|
||||
retry_count = 0
|
||||
@@ -7262,10 +7228,7 @@ class AIAgent:
|
||||
retry_count = 0
|
||||
continue
|
||||
_final_summary = self._summarize_api_error(api_error)
|
||||
if is_rate_limited:
|
||||
self._vprint(f"{self.log_prefix}❌ Rate limit persisted after {max_retries} retries. Please try again later.", force=True)
|
||||
else:
|
||||
self._vprint(f"{self.log_prefix}❌ Max retries ({max_retries}) exceeded. Giving up.", force=True)
|
||||
self._vprint(f"{self.log_prefix}❌ Max retries ({max_retries}) exceeded. Giving up.", force=True)
|
||||
self._vprint(f"{self.log_prefix} 💀 Final error: {_final_summary}", force=True)
|
||||
|
||||
# Detect SSE stream-drop pattern (e.g. "Network
|
||||
@@ -7325,22 +7288,8 @@ class AIAgent:
|
||||
"error": _final_summary,
|
||||
}
|
||||
|
||||
# For rate limits, respect the Retry-After header if present
|
||||
_retry_after = None
|
||||
if is_rate_limited:
|
||||
_resp_headers = getattr(getattr(api_error, "response", None), "headers", None)
|
||||
if _resp_headers and hasattr(_resp_headers, "get"):
|
||||
_ra_raw = _resp_headers.get("retry-after") or _resp_headers.get("Retry-After")
|
||||
if _ra_raw:
|
||||
try:
|
||||
_retry_after = min(int(_ra_raw), 120) # Cap at 2 minutes
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
wait_time = _retry_after if _retry_after else min(2 ** retry_count, 60)
|
||||
if is_rate_limited:
|
||||
self._emit_status(f"⏱️ Rate limit reached. Waiting {wait_time}s before retry (attempt {retry_count + 1}/{max_retries})...")
|
||||
else:
|
||||
self._emit_status(f"⏳ Retrying in {wait_time}s (attempt {retry_count}/{max_retries})...")
|
||||
wait_time = min(2 ** retry_count, 60) # Exponential backoff: 2s, 4s, 8s, 16s, 32s, 60s, 60s
|
||||
self._emit_status(f"⏳ Retrying in {wait_time}s (attempt {retry_count}/{max_retries})...")
|
||||
logger.warning(
|
||||
"Retrying API call in %ss (attempt %s/%s) %s error=%s",
|
||||
wait_time,
|
||||
@@ -7926,7 +7875,7 @@ class AIAgent:
|
||||
error_msg = f"Error during OpenAI-compatible API call #{api_call_count}: {str(e)}"
|
||||
try:
|
||||
print(f"❌ {error_msg}")
|
||||
except (OSError, ValueError):
|
||||
except OSError:
|
||||
logger.error(error_msg)
|
||||
|
||||
if self.verbose_logging:
|
||||
|
||||
@@ -1,79 +0,0 @@
|
||||
import path from 'path';
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
|
||||
export function normalizeWhatsAppIdentifier(value) {
|
||||
return String(value || '')
|
||||
.trim()
|
||||
.replace(/:.*@/, '@')
|
||||
.replace(/@.*/, '')
|
||||
.replace(/^\+/, '');
|
||||
}
|
||||
|
||||
export function parseAllowedUsers(rawValue) {
|
||||
return new Set(
|
||||
String(rawValue || '')
|
||||
.split(',')
|
||||
.map((value) => normalizeWhatsAppIdentifier(value))
|
||||
.filter(Boolean)
|
||||
);
|
||||
}
|
||||
|
||||
function readMappingFile(sessionDir, identifier, suffix = '') {
|
||||
const filePath = path.join(sessionDir, `lid-mapping-${identifier}${suffix}.json`);
|
||||
if (!existsSync(filePath)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(readFileSync(filePath, 'utf8'));
|
||||
const normalized = normalizeWhatsAppIdentifier(parsed);
|
||||
return normalized || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function expandWhatsAppIdentifiers(identifier, sessionDir) {
|
||||
const normalized = normalizeWhatsAppIdentifier(identifier);
|
||||
if (!normalized) {
|
||||
return new Set();
|
||||
}
|
||||
|
||||
// Walk both phone->LID and LID->phone mapping files so allowlists can use
|
||||
// either form transparently in bot mode.
|
||||
const resolved = new Set();
|
||||
const queue = [normalized];
|
||||
|
||||
while (queue.length > 0) {
|
||||
const current = queue.shift();
|
||||
if (!current || resolved.has(current)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
resolved.add(current);
|
||||
|
||||
for (const suffix of ['', '_reverse']) {
|
||||
const mapped = readMappingFile(sessionDir, current, suffix);
|
||||
if (mapped && !resolved.has(mapped)) {
|
||||
queue.push(mapped);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return resolved;
|
||||
}
|
||||
|
||||
export function matchesAllowedUser(senderId, allowedUsers, sessionDir) {
|
||||
if (!allowedUsers || allowedUsers.size === 0) {
|
||||
return true;
|
||||
}
|
||||
|
||||
const aliases = expandWhatsAppIdentifiers(senderId, sessionDir);
|
||||
for (const alias of aliases) {
|
||||
if (allowedUsers.has(alias)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
@@ -1,47 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
|
||||
import {
|
||||
expandWhatsAppIdentifiers,
|
||||
matchesAllowedUser,
|
||||
normalizeWhatsAppIdentifier,
|
||||
parseAllowedUsers,
|
||||
} from './allowlist.js';
|
||||
|
||||
test('normalizeWhatsAppIdentifier strips jid syntax and plus prefix', () => {
|
||||
assert.equal(normalizeWhatsAppIdentifier('+19175395595@s.whatsapp.net'), '19175395595');
|
||||
assert.equal(normalizeWhatsAppIdentifier('267383306489914@lid'), '267383306489914');
|
||||
assert.equal(normalizeWhatsAppIdentifier('19175395595:12@s.whatsapp.net'), '19175395595');
|
||||
});
|
||||
|
||||
test('expandWhatsAppIdentifiers resolves phone and lid aliases from session files', () => {
|
||||
const sessionDir = mkdtempSync(path.join(os.tmpdir(), 'hermes-wa-allowlist-'));
|
||||
|
||||
try {
|
||||
writeFileSync(path.join(sessionDir, 'lid-mapping-19175395595.json'), JSON.stringify('267383306489914'));
|
||||
writeFileSync(path.join(sessionDir, 'lid-mapping-267383306489914_reverse.json'), JSON.stringify('19175395595'));
|
||||
|
||||
const aliases = expandWhatsAppIdentifiers('267383306489914@lid', sessionDir);
|
||||
assert.deepEqual([...aliases].sort(), ['19175395595', '267383306489914']);
|
||||
} finally {
|
||||
rmSync(sessionDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('matchesAllowedUser accepts mapped lid sender when allowlist only contains phone number', () => {
|
||||
const sessionDir = mkdtempSync(path.join(os.tmpdir(), 'hermes-wa-allowlist-'));
|
||||
|
||||
try {
|
||||
writeFileSync(path.join(sessionDir, 'lid-mapping-19175395595.json'), JSON.stringify('267383306489914'));
|
||||
writeFileSync(path.join(sessionDir, 'lid-mapping-267383306489914_reverse.json'), JSON.stringify('19175395595'));
|
||||
|
||||
const allowedUsers = parseAllowedUsers('+19175395595');
|
||||
assert.equal(matchesAllowedUser('267383306489914@lid', allowedUsers, sessionDir), true);
|
||||
assert.equal(matchesAllowedUser('188012763865257@lid', allowedUsers, sessionDir), false);
|
||||
} finally {
|
||||
rmSync(sessionDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
@@ -26,7 +26,6 @@ import path from 'path';
|
||||
import { mkdirSync, readFileSync, writeFileSync, existsSync, readdirSync } from 'fs';
|
||||
import { randomBytes } from 'crypto';
|
||||
import qrcode from 'qrcode-terminal';
|
||||
import { matchesAllowedUser, parseAllowedUsers } from './allowlist.js';
|
||||
|
||||
// Parse CLI args
|
||||
const args = process.argv.slice(2);
|
||||
@@ -48,17 +47,13 @@ const DOCUMENT_CACHE_DIR = path.join(process.env.HOME || '~', '.hermes', 'docume
|
||||
const AUDIO_CACHE_DIR = path.join(process.env.HOME || '~', '.hermes', 'audio_cache');
|
||||
const PAIR_ONLY = args.includes('--pair-only');
|
||||
const WHATSAPP_MODE = getArg('mode', process.env.WHATSAPP_MODE || 'self-chat'); // "bot" or "self-chat"
|
||||
const ALLOWED_USERS = parseAllowedUsers(process.env.WHATSAPP_ALLOWED_USERS || '');
|
||||
const ALLOWED_USERS = (process.env.WHATSAPP_ALLOWED_USERS || '').split(',').map(s => s.trim()).filter(Boolean);
|
||||
const DEFAULT_REPLY_PREFIX = '⚕ *Hermes Agent*\n────────────\n';
|
||||
const REPLY_PREFIX = process.env.WHATSAPP_REPLY_PREFIX === undefined
|
||||
? DEFAULT_REPLY_PREFIX
|
||||
: process.env.WHATSAPP_REPLY_PREFIX.replace(/\\n/g, '\n');
|
||||
|
||||
function formatOutgoingMessage(message) {
|
||||
// In bot mode, messages come from a different number so the prefix is
|
||||
// redundant — the sender identity is already clear. Only prepend in
|
||||
// self-chat mode where bot and user share the same number.
|
||||
if (WHATSAPP_MODE !== 'self-chat') return message;
|
||||
return REPLY_PREFIX ? `${REPLY_PREFIX}${message}` : message;
|
||||
}
|
||||
|
||||
@@ -195,9 +190,10 @@ async function startSocket() {
|
||||
if (!isSelfChat) continue;
|
||||
}
|
||||
|
||||
// Check allowlist for messages from others (resolve LID ↔ phone aliases)
|
||||
if (!msg.key.fromMe && !matchesAllowedUser(senderId, ALLOWED_USERS, SESSION_DIR)) {
|
||||
continue;
|
||||
// Check allowlist for messages from others (resolve LID → phone if needed)
|
||||
if (!msg.key.fromMe && ALLOWED_USERS.length > 0) {
|
||||
const resolvedNumber = lidToPhone[senderNumber] || senderNumber;
|
||||
if (!ALLOWED_USERS.includes(resolvedNumber)) continue;
|
||||
}
|
||||
|
||||
// Extract message body
|
||||
@@ -519,8 +515,8 @@ if (PAIR_ONLY) {
|
||||
app.listen(PORT, '127.0.0.1', () => {
|
||||
console.log(`🌉 WhatsApp bridge listening on port ${PORT} (mode: ${WHATSAPP_MODE})`);
|
||||
console.log(`📁 Session stored in: ${SESSION_DIR}`);
|
||||
if (ALLOWED_USERS.size > 0) {
|
||||
console.log(`🔒 Allowed users: ${Array.from(ALLOWED_USERS).join(', ')}`);
|
||||
if (ALLOWED_USERS.length > 0) {
|
||||
console.log(`🔒 Allowed users: ${ALLOWED_USERS.join(', ')}`);
|
||||
} else {
|
||||
console.log(`⚠️ No WHATSAPP_ALLOWED_USERS set — all messages will be processed`);
|
||||
}
|
||||
|
||||
@@ -1,289 +0,0 @@
|
||||
---
|
||||
name: songwriting-and-ai-music
|
||||
description: >
|
||||
Songwriting craft, AI music generation prompts (Suno focus), parody/adaptation
|
||||
techniques, phonetic tricks, and lessons learned. These are tools and ideas,
|
||||
not rules. Break any of them when the art calls for it.
|
||||
tags: [songwriting, music, suno, parody, lyrics, creative]
|
||||
triggers:
|
||||
- writing a song
|
||||
- song lyrics
|
||||
- music prompt
|
||||
- suno prompt
|
||||
- parody song
|
||||
- adapting a song
|
||||
- AI music generation
|
||||
---
|
||||
|
||||
# Songwriting & AI Music Generation
|
||||
|
||||
Everything here is a GUIDELINE, not a rule. Art breaks rules on purpose.
|
||||
Use what serves the song. Ignore what doesn't.
|
||||
|
||||
---
|
||||
|
||||
## 1. Song Structure (Pick One or Invent Your Own)
|
||||
|
||||
Common skeletons — mix, modify, or throw out as needed:
|
||||
|
||||
```
|
||||
ABABCB Verse/Chorus/Verse/Chorus/Bridge/Chorus (most pop/rock)
|
||||
AABA Verse/Verse/Bridge/Verse (refrain-based) (jazz standards, ballads)
|
||||
ABAB Verse/Chorus alternating (simple, direct)
|
||||
AAA Verse/Verse/Verse (strophic, no chorus) (folk, storytelling)
|
||||
```
|
||||
|
||||
The six building blocks:
|
||||
- Intro — set the mood, pull the listener in
|
||||
- Verse — the story, the details, the world-building
|
||||
- Pre-Chorus — optional tension ramp before the payoff
|
||||
- Chorus — the emotional core, the part people remember
|
||||
- Bridge — a detour, a shift in perspective or key
|
||||
- Outro — the farewell, can echo or subvert the rest
|
||||
|
||||
You don't need all of these. Some great songs are just one section
|
||||
that evolves. Structure serves the emotion, not the other way around.
|
||||
|
||||
---
|
||||
|
||||
## 2. Rhyme, Meter, and Sound
|
||||
|
||||
RHYME TYPES (from tight to loose):
|
||||
- Perfect: lean/mean
|
||||
- Family: crate/braid
|
||||
- Assonance: had/glass (same vowels, different endings)
|
||||
- Consonance: scene/when (different vowels, similar endings)
|
||||
- Near/slant: enough to suggest connection without locking it down
|
||||
|
||||
Mix them. All perfect rhymes can sound like a nursery rhyme.
|
||||
All slant rhymes can sound lazy. The blend is where it lives.
|
||||
|
||||
INTERNAL RHYME: Rhyming within a line, not just at the ends.
|
||||
"We pruned the lies from bleeding trees / Distilled the storm
|
||||
from entropy" — "lies/flies," "trees/entropy" create internal echoes.
|
||||
|
||||
METER: The rhythm of stressed vs unstressed syllables.
|
||||
- Matching syllable counts between parallel lines helps singability
|
||||
- The STRESSED syllables matter more than total count
|
||||
- Say it out loud. If you stumble, the meter needs work.
|
||||
- Intentionally breaking meter can create emphasis or surprise
|
||||
|
||||
---
|
||||
|
||||
## 3. Emotional Arc and Dynamics
|
||||
|
||||
Think of a song as a journey, not a flat road.
|
||||
|
||||
ENERGY MAPPING (rough idea, not prescription):
|
||||
Intro: 2-3 | Verse: 5-6 | Pre-Chorus: 7
|
||||
Chorus: 8-9 | Bridge: varies | Final Chorus: 9-10
|
||||
|
||||
The most powerful dynamic trick: CONTRAST.
|
||||
- Whisper before a scream hits harder than just screaming
|
||||
- Sparse before dense. Slow before fast. Low before high.
|
||||
- The drop only works because of the buildup
|
||||
- Silence is an instrument
|
||||
|
||||
"Whisper to roar to whisper" — start intimate, build to full power,
|
||||
strip back to vulnerability. Works for ballads, epics, anthems.
|
||||
|
||||
---
|
||||
|
||||
## 4. Writing Lyrics That Work
|
||||
|
||||
SHOW, DON'T TELL (usually):
|
||||
- "I was sad" = flat
|
||||
- "Your hoodie's still on the hook by the door" = alive
|
||||
- But sometimes "I give my life" said plainly IS the power
|
||||
|
||||
THE HOOK:
|
||||
- The line people remember, hum, repeat
|
||||
- Usually the title or core phrase
|
||||
- Works best when melody + lyric + emotion all align
|
||||
- Place it where it lands hardest (often first/last line of chorus)
|
||||
|
||||
PROSODY — lyrics and music supporting each other:
|
||||
- Stable feelings (resolution, peace) pair with settled melodies,
|
||||
perfect rhymes, resolved chords
|
||||
- Unstable feelings (longing, doubt) pair with wandering melodies,
|
||||
near-rhymes, unresolved chords
|
||||
- Verse melody typically sits lower, chorus goes higher
|
||||
- But flip this if it serves the song
|
||||
|
||||
AVOID (unless you're doing it on purpose):
|
||||
- Cliches on autopilot ("heart of gold" without earning it)
|
||||
- Forcing word order to hit a rhyme ("Yoda-speak")
|
||||
- Same energy in every section (flat dynamics)
|
||||
- Treating your first draft as sacred — revision is creation
|
||||
|
||||
---
|
||||
|
||||
## 5. Parody and Adaptation
|
||||
|
||||
When rewriting an existing song with new lyrics:
|
||||
|
||||
THE SKELETON: Map the original's structure first.
|
||||
- Count syllables per line
|
||||
- Mark the rhyme scheme (ABAB, AABB, etc.)
|
||||
- Identify which syllables are STRESSED
|
||||
- Note where held/sustained notes fall
|
||||
|
||||
FITTING NEW WORDS:
|
||||
- Match stressed syllables to the same beats as the original
|
||||
- Total syllable count can flex by 1-2 unstressed syllables
|
||||
- On long held notes, try to match the VOWEL SOUND of the original
|
||||
(if original holds "LOOOVE" with an "oo" vowel, "FOOOD" fits
|
||||
better than "LIFE")
|
||||
- Monosyllabic swaps in key spots keep rhythm intact
|
||||
(Crime -> Code, Snake -> Noose)
|
||||
- Sing your new words over the original — if you stumble, revise
|
||||
|
||||
CONCEPT:
|
||||
- Pick a concept strong enough to sustain the whole song
|
||||
- Start from the title/hook and build outward
|
||||
- Generate lots of raw material (puns, phrases, images) FIRST,
|
||||
then fit the best ones into the structure
|
||||
- If you need a specific line somewhere, reverse-engineer the
|
||||
rhyme scheme backward to set it up
|
||||
|
||||
KEEP SOME ORIGINALS: Leaving a few original lines or structures
|
||||
intact adds recognizability and lets the audience feel the connection.
|
||||
|
||||
---
|
||||
|
||||
## 6. Suno AI Prompt Engineering
|
||||
|
||||
### Style/Genre Description Field
|
||||
|
||||
FORMULA (adapt as needed):
|
||||
Genre + Mood + Era + Instruments + Vocal Style + Production + Dynamics
|
||||
|
||||
```
|
||||
BAD: "sad rock song"
|
||||
GOOD: "Cinematic orchestral spy thriller, 1960s Cold War era, smoky
|
||||
sultry female vocalist, big band jazz, brass section with
|
||||
trumpets and french horns, sweeping strings, minor key,
|
||||
vintage analog warmth"
|
||||
```
|
||||
|
||||
DESCRIBE THE JOURNEY, not just the genre:
|
||||
```
|
||||
"Begins as a haunting whisper over sparse piano. Gradually layers
|
||||
in muted brass. Builds through the chorus with full orchestra.
|
||||
Second verse erupts with raw belting intensity. Outro strips back
|
||||
to a lone piano and a fragile whisper fading to silence."
|
||||
```
|
||||
|
||||
TIPS:
|
||||
- V4.5+ supports up to 1,000 chars in Style field — use them
|
||||
- NO artist names or trademarks. Describe the sound instead.
|
||||
"1960s Cold War spy thriller brass" not "James Bond style"
|
||||
"90s grunge" not "Nirvana-style"
|
||||
- Specify BPM and key when you have a preference
|
||||
- Use Exclude Styles field for what you DON'T want
|
||||
- Unexpected genre combos can be gold: "bossa nova trap",
|
||||
"Appalachian gothic", "chiptune jazz"
|
||||
- Build a vocal PERSONA, not just a gender:
|
||||
"A weathered torch singer with a smoky alto, slight rasp,
|
||||
who starts vulnerable and builds to devastating power"
|
||||
|
||||
### Metatags (place in [brackets] inside lyrics field)
|
||||
|
||||
STRUCTURE:
|
||||
[Intro] [Verse] [Verse 1] [Pre-Chorus] [Chorus]
|
||||
[Post-Chorus] [Hook] [Bridge] [Interlude]
|
||||
[Instrumental] [Instrumental Break] [Guitar Solo]
|
||||
[Breakdown] [Build-up] [Outro] [Silence] [End]
|
||||
|
||||
VOCAL PERFORMANCE:
|
||||
[Whispered] [Spoken Word] [Belted] [Falsetto] [Powerful]
|
||||
[Soulful] [Raspy] [Breathy] [Smooth] [Gritty]
|
||||
[Staccato] [Legato] [Vibrato] [Melismatic]
|
||||
[Harmonies] [Choir] [Harmonized Chorus]
|
||||
|
||||
DYNAMICS:
|
||||
[High Energy] [Low Energy] [Building Energy] [Explosive]
|
||||
[Emotional Climax] [Gradual swell] [Orchestral swell]
|
||||
[Quiet arrangement] [Falling tension] [Slow Down]
|
||||
|
||||
GENDER:
|
||||
[Female Vocals] [Male Vocals]
|
||||
|
||||
ATMOSPHERE:
|
||||
[Melancholic] [Euphoric] [Nostalgic] [Aggressive]
|
||||
[Dreamy] [Intimate] [Dark Atmosphere]
|
||||
|
||||
SFX:
|
||||
[Vinyl Crackle] [Rain] [Applause] [Static] [Thunder]
|
||||
|
||||
Put tags in BOTH style field AND lyrics for reinforcement.
|
||||
Keep to 5-8 tags per section max — too many confuses the AI.
|
||||
Don't contradict yourself ([Calm] + [Aggressive] in same section).
|
||||
|
||||
### Custom Mode
|
||||
- Always use Custom Mode for serious work (separate Style + Lyrics)
|
||||
- Lyrics field limit: ~3,000 chars (~40-60 lines)
|
||||
- Always add structural tags — without them Suno defaults to
|
||||
flat verse/chorus/verse with no emotional arc
|
||||
|
||||
---
|
||||
|
||||
## 7. Phonetic Tricks for AI Singers
|
||||
|
||||
AI vocalists don't read — they pronounce. Help them:
|
||||
|
||||
PHONETIC RESPELLING:
|
||||
- Spell words as they SOUND: "through" -> "thru"
|
||||
- Proper nouns are highest failure rate — test early
|
||||
- "Nous" -> "Noose" (forces correct pronunciation)
|
||||
- Hyphenate to guide syllables: "Re-search", "bio-engineering"
|
||||
|
||||
DELIVERY CONTROL:
|
||||
- ALL CAPS = louder, more intense
|
||||
- Vowel extension: "lo-o-o-ove" = sustained/melisma
|
||||
- Ellipses: "I... need... you" = dramatic pauses
|
||||
- Hyphenated stretch: "ne-e-ed" = emotional stretch
|
||||
|
||||
ALWAYS:
|
||||
- Spell out numbers: "24/7" -> "twenty four seven"
|
||||
- Space acronyms: "AI" -> "A I" or "A-I"
|
||||
- Test proper nouns/unusual words in a short 30-second clip first
|
||||
- Once generated, pronunciation is baked in — fix in lyrics BEFORE
|
||||
|
||||
---
|
||||
|
||||
## 8. Workflow
|
||||
|
||||
1. Write the concept/hook first — what's the emotional core?
|
||||
2. If adapting, map the original structure (syllables, rhyme, stress)
|
||||
3. Generate raw material — brainstorm freely before structuring
|
||||
4. Draft lyrics into the structure
|
||||
5. Read/sing aloud — catch stumbles, fix meter
|
||||
6. Build the Suno style description — paint the dynamic journey
|
||||
7. Add metatags to lyrics for performance direction
|
||||
8. Generate 3-5 variations minimum — treat them like recording takes
|
||||
9. Pick the best, use Extend/Continue to build on promising sections
|
||||
10. If something great happens by accident, keep it
|
||||
|
||||
EXPECT: ~3-5 generations per 1 good result. Revision is normal.
|
||||
Style can drift in extensions — restate genre/mood when extending.
|
||||
|
||||
---
|
||||
|
||||
## 9. Lessons Learned
|
||||
|
||||
- Describing the dynamic ARC in the style field matters way more
|
||||
than just listing genres. "Whisper to roar to whisper" gives
|
||||
Suno a performance map.
|
||||
- Keeping some original lines intact in a parody adds recognizability
|
||||
and emotional weight — the audience feels the ghost of the original.
|
||||
- The bridge slot in a song is where you can transform imagery.
|
||||
Swap the original's specific references for your theme's metaphors
|
||||
while keeping the emotional function (reflection, shift, revelation).
|
||||
- Monosyllabic word swaps in hooks/tags are the cleanest way to
|
||||
maintain rhythm while changing meaning.
|
||||
- A strong vocal persona description in the style field makes a
|
||||
bigger difference than any single metatag.
|
||||
- Don't be precious about rules. If a line breaks meter but hits
|
||||
harder, keep it. The feeling is what matters. Craft serves art,
|
||||
not the other way around.
|
||||
@@ -0,0 +1,3 @@
|
||||
---
|
||||
description: Skills for generating, editing, and processing music and audio using AI models and audio tools.
|
||||
---
|
||||
@@ -491,17 +491,15 @@ class TestGetTextAuxiliaryClient:
|
||||
assert mock_openai.call_args.kwargs["base_url"] == "http://localhost:2345/v1"
|
||||
assert mock_openai.call_args.kwargs["api_key"] == "task-key"
|
||||
|
||||
def test_task_direct_endpoint_without_openai_key_uses_placeholder(self, monkeypatch):
|
||||
"""Local endpoints without an API key should use 'no-key-required' placeholder."""
|
||||
def test_task_direct_endpoint_without_openai_key_does_not_fall_back(self, monkeypatch):
|
||||
monkeypatch.setenv("OPENROUTER_API_KEY", "or-key")
|
||||
monkeypatch.setenv("AUXILIARY_WEB_EXTRACT_BASE_URL", "http://localhost:2345/v1")
|
||||
monkeypatch.setenv("AUXILIARY_WEB_EXTRACT_MODEL", "task-model")
|
||||
with patch("agent.auxiliary_client.OpenAI") as mock_openai:
|
||||
client, model = get_text_auxiliary_client("web_extract")
|
||||
assert client is not None
|
||||
assert model == "task-model"
|
||||
assert mock_openai.call_args.kwargs["api_key"] == "no-key-required"
|
||||
assert mock_openai.call_args.kwargs["base_url"] == "http://localhost:2345/v1"
|
||||
assert client is None
|
||||
assert model is None
|
||||
mock_openai.assert_not_called()
|
||||
|
||||
def test_custom_endpoint_uses_config_saved_base_url(self, monkeypatch):
|
||||
config = {
|
||||
@@ -698,16 +696,15 @@ class TestVisionClientFallback:
|
||||
assert mock_openai.call_args.kwargs["base_url"] == "http://localhost:4567/v1"
|
||||
assert mock_openai.call_args.kwargs["api_key"] == "vision-key"
|
||||
|
||||
def test_vision_direct_endpoint_without_key_uses_placeholder(self, monkeypatch):
|
||||
"""Vision endpoint without API key should use 'no-key-required' placeholder."""
|
||||
def test_vision_direct_endpoint_requires_openai_api_key(self, monkeypatch):
|
||||
monkeypatch.setenv("OPENROUTER_API_KEY", "or-key")
|
||||
monkeypatch.setenv("AUXILIARY_VISION_BASE_URL", "http://localhost:4567/v1")
|
||||
monkeypatch.setenv("AUXILIARY_VISION_MODEL", "vision-model")
|
||||
with patch("agent.auxiliary_client.OpenAI") as mock_openai:
|
||||
client, model = get_vision_auxiliary_client()
|
||||
assert client is not None
|
||||
assert model == "vision-model"
|
||||
assert mock_openai.call_args.kwargs["api_key"] == "no-key-required"
|
||||
assert client is None
|
||||
assert model is None
|
||||
mock_openai.assert_not_called()
|
||||
|
||||
def test_vision_uses_openrouter_when_available(self, monkeypatch):
|
||||
monkeypatch.setenv("OPENROUTER_API_KEY", "or-key")
|
||||
|
||||
@@ -201,52 +201,3 @@ class TestSecretCapturePayloadRedaction:
|
||||
text = '{"raw_secret": "ghp_abc123def456ghi789jkl"}'
|
||||
result = redact_sensitive_text(text)
|
||||
assert "abc123def456" not in result
|
||||
|
||||
|
||||
class TestElevenLabsTavilyExaKeys:
|
||||
"""Regression tests for ElevenLabs (sk_), Tavily (tvly-), and Exa (exa_) keys."""
|
||||
|
||||
def test_elevenlabs_key_redacted(self):
|
||||
text = "ELEVENLABS_API_KEY=sk_abc123def456ghi789jklmnopqrstu"
|
||||
result = redact_sensitive_text(text)
|
||||
assert "abc123def456ghi" not in result
|
||||
|
||||
def test_elevenlabs_key_in_log_line(self):
|
||||
text = "Connecting to ElevenLabs with key sk_abc123def456ghi789jklmnopqrstu"
|
||||
result = redact_sensitive_text(text)
|
||||
assert "abc123def456ghi" not in result
|
||||
|
||||
def test_tavily_key_redacted(self):
|
||||
text = "TAVILY_API_KEY=tvly-ABCdef123456789GHIJKL0000"
|
||||
result = redact_sensitive_text(text)
|
||||
assert "ABCdef123456789" not in result
|
||||
|
||||
def test_tavily_key_in_log_line(self):
|
||||
text = "Initialising Tavily client with tvly-ABCdef123456789GHIJKL0000"
|
||||
result = redact_sensitive_text(text)
|
||||
assert "ABCdef123456789" not in result
|
||||
|
||||
def test_exa_key_redacted(self):
|
||||
text = "EXA_API_KEY=exa_XYZ789abcdef000000000000000"
|
||||
result = redact_sensitive_text(text)
|
||||
assert "XYZ789abcdef" not in result
|
||||
|
||||
def test_exa_key_in_log_line(self):
|
||||
text = "Using Exa client with key exa_XYZ789abcdef000000000000000"
|
||||
result = redact_sensitive_text(text)
|
||||
assert "XYZ789abcdef" not in result
|
||||
|
||||
def test_all_three_in_env_dump(self):
|
||||
env_dump = (
|
||||
"HOME=/home/user\n"
|
||||
"ELEVENLABS_API_KEY=sk_abc123def456ghi789jklmnopqrstu\n"
|
||||
"TAVILY_API_KEY=tvly-ABCdef123456789GHIJKL0000\n"
|
||||
"EXA_API_KEY=exa_XYZ789abcdef000000000000000\n"
|
||||
"SHELL=/bin/bash\n"
|
||||
)
|
||||
result = redact_sensitive_text(env_dump)
|
||||
assert "abc123def456ghi" not in result
|
||||
assert "ABCdef123456789" not in result
|
||||
assert "XYZ789abcdef" not in result
|
||||
assert "HOME=/home/user" in result
|
||||
assert "SHELL=/bin/bash" in result
|
||||
|
||||
@@ -84,48 +84,6 @@ class TestResolveDeliveryTarget:
|
||||
"thread_id": None,
|
||||
}
|
||||
|
||||
def test_human_friendly_label_resolved_via_channel_directory(self):
|
||||
"""deliver: 'whatsapp:Alice (dm)' resolves to the real JID."""
|
||||
job = {"deliver": "whatsapp:Alice (dm)"}
|
||||
with patch(
|
||||
"gateway.channel_directory.resolve_channel_name",
|
||||
return_value="12345678901234@lid",
|
||||
):
|
||||
result = _resolve_delivery_target(job)
|
||||
assert result == {
|
||||
"platform": "whatsapp",
|
||||
"chat_id": "12345678901234@lid",
|
||||
"thread_id": None,
|
||||
}
|
||||
|
||||
def test_human_friendly_label_without_suffix_resolved(self):
|
||||
"""deliver: 'telegram:My Group' resolves without display suffix."""
|
||||
job = {"deliver": "telegram:My Group"}
|
||||
with patch(
|
||||
"gateway.channel_directory.resolve_channel_name",
|
||||
return_value="-1009999",
|
||||
):
|
||||
result = _resolve_delivery_target(job)
|
||||
assert result == {
|
||||
"platform": "telegram",
|
||||
"chat_id": "-1009999",
|
||||
"thread_id": None,
|
||||
}
|
||||
|
||||
def test_raw_id_not_mangled_when_directory_returns_none(self):
|
||||
"""deliver: 'whatsapp:12345@lid' passes through when directory has no match."""
|
||||
job = {"deliver": "whatsapp:12345@lid"}
|
||||
with patch(
|
||||
"gateway.channel_directory.resolve_channel_name",
|
||||
return_value=None,
|
||||
):
|
||||
result = _resolve_delivery_target(job)
|
||||
assert result == {
|
||||
"platform": "whatsapp",
|
||||
"chat_id": "12345@lid",
|
||||
"thread_id": None,
|
||||
}
|
||||
|
||||
def test_bare_platform_uses_matching_origin_chat(self):
|
||||
job = {
|
||||
"deliver": "telegram",
|
||||
@@ -209,32 +167,6 @@ class TestDeliverResultWrapping:
|
||||
sent_content = send_mock.call_args.kwargs.get("content") or send_mock.call_args[0][-1]
|
||||
assert "Cronjob Response: abc-123" in sent_content
|
||||
|
||||
def test_delivery_skips_wrapping_when_config_disabled(self):
|
||||
"""When cron.wrap_response is false, deliver raw content without header/footer."""
|
||||
from gateway.config import Platform
|
||||
|
||||
pconfig = MagicMock()
|
||||
pconfig.enabled = True
|
||||
mock_cfg = MagicMock()
|
||||
mock_cfg.platforms = {Platform.TELEGRAM: pconfig}
|
||||
|
||||
with patch("gateway.config.load_gateway_config", return_value=mock_cfg), \
|
||||
patch("tools.send_message_tool._send_to_platform", new=AsyncMock(return_value={"success": True})) as send_mock, \
|
||||
patch("cron.scheduler.load_config", return_value={"cron": {"wrap_response": False}}):
|
||||
job = {
|
||||
"id": "test-job",
|
||||
"name": "daily-report",
|
||||
"deliver": "origin",
|
||||
"origin": {"platform": "telegram", "chat_id": "123"},
|
||||
}
|
||||
_deliver_result(job, "Clean output only.")
|
||||
|
||||
send_mock.assert_called_once()
|
||||
sent_content = send_mock.call_args.kwargs.get("content") or send_mock.call_args[0][-1]
|
||||
assert sent_content == "Clean output only."
|
||||
assert "Cronjob Response" not in sent_content
|
||||
assert "The agent cannot see" not in sent_content
|
||||
|
||||
def test_no_mirror_to_session_call(self):
|
||||
"""Cron deliveries should NOT mirror into the gateway session."""
|
||||
from gateway.config import Platform
|
||||
|
||||
@@ -13,7 +13,7 @@ def _would_warn():
|
||||
"SIGNAL_ALLOWED_USERS", "SIGNAL_GROUP_ALLOWED_USERS",
|
||||
"EMAIL_ALLOWED_USERS",
|
||||
"SMS_ALLOWED_USERS", "MATTERMOST_ALLOWED_USERS",
|
||||
"MATRIX_ALLOWED_USERS", "DINGTALK_ALLOWED_USERS", "FEISHU_ALLOWED_USERS", "WECOM_ALLOWED_USERS",
|
||||
"MATRIX_ALLOWED_USERS", "DINGTALK_ALLOWED_USERS",
|
||||
"GATEWAY_ALLOWED_USERS")
|
||||
)
|
||||
_allow_all = os.getenv("GATEWAY_ALLOW_ALL_USERS", "").lower() in ("true", "1", "yes") or any(
|
||||
@@ -22,7 +22,7 @@ def _would_warn():
|
||||
"WHATSAPP_ALLOW_ALL_USERS", "SLACK_ALLOW_ALL_USERS",
|
||||
"SIGNAL_ALLOW_ALL_USERS", "EMAIL_ALLOW_ALL_USERS",
|
||||
"SMS_ALLOW_ALL_USERS", "MATTERMOST_ALLOW_ALL_USERS",
|
||||
"MATRIX_ALLOW_ALL_USERS", "DINGTALK_ALLOW_ALL_USERS", "FEISHU_ALLOW_ALL_USERS", "WECOM_ALLOW_ALL_USERS")
|
||||
"MATRIX_ALLOW_ALL_USERS", "DINGTALK_ALLOW_ALL_USERS")
|
||||
)
|
||||
return not _any_allowlist and not _allow_all
|
||||
|
||||
|
||||
@@ -15,7 +15,6 @@ class DummyTelegramAdapter(BasePlatformAdapter):
|
||||
super().__init__(PlatformConfig(enabled=True, token="fake-token"), Platform.TELEGRAM)
|
||||
self.sent = []
|
||||
self.typing = []
|
||||
self.processing_hooks = []
|
||||
|
||||
async def connect(self) -> bool:
|
||||
return True
|
||||
@@ -41,12 +40,6 @@ class DummyTelegramAdapter(BasePlatformAdapter):
|
||||
async def get_chat_info(self, chat_id: str):
|
||||
return {"id": chat_id}
|
||||
|
||||
async def on_processing_start(self, event: MessageEvent) -> None:
|
||||
self.processing_hooks.append(("start", event.message_id))
|
||||
|
||||
async def on_processing_complete(self, event: MessageEvent, success: bool) -> None:
|
||||
self.processing_hooks.append(("complete", event.message_id, success))
|
||||
|
||||
|
||||
def _make_event(chat_id: str, thread_id: str, message_id: str = "1") -> MessageEvent:
|
||||
return MessageEvent(
|
||||
@@ -140,83 +133,3 @@ class TestBasePlatformTopicSessions:
|
||||
"metadata": {"thread_id": "17585"},
|
||||
}
|
||||
]
|
||||
assert adapter.processing_hooks == [
|
||||
("start", "1"),
|
||||
("complete", "1", True),
|
||||
]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_process_message_background_marks_total_send_failure_unsuccessful(self):
|
||||
adapter = DummyTelegramAdapter()
|
||||
|
||||
async def handler(_event):
|
||||
await asyncio.sleep(0)
|
||||
return "ack"
|
||||
|
||||
async def failing_send(*_args, **_kwargs):
|
||||
return SendResult(success=False, error="send failed")
|
||||
|
||||
async def hold_typing(_chat_id, interval=2.0, metadata=None):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
adapter.set_message_handler(handler)
|
||||
adapter.send = failing_send
|
||||
adapter._keep_typing = hold_typing
|
||||
|
||||
event = _make_event("-1001", "17585")
|
||||
await adapter._process_message_background(event, build_session_key(event.source))
|
||||
|
||||
assert adapter.processing_hooks == [
|
||||
("start", "1"),
|
||||
("complete", "1", False),
|
||||
]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_process_message_background_marks_exception_unsuccessful(self):
|
||||
adapter = DummyTelegramAdapter()
|
||||
|
||||
async def handler(_event):
|
||||
await asyncio.sleep(0)
|
||||
raise RuntimeError("boom")
|
||||
|
||||
async def hold_typing(_chat_id, interval=2.0, metadata=None):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
adapter.set_message_handler(handler)
|
||||
adapter._keep_typing = hold_typing
|
||||
|
||||
event = _make_event("-1001", "17585")
|
||||
await adapter._process_message_background(event, build_session_key(event.source))
|
||||
|
||||
assert adapter.processing_hooks == [
|
||||
("start", "1"),
|
||||
("complete", "1", False),
|
||||
]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_process_message_background_marks_cancellation_unsuccessful(self):
|
||||
adapter = DummyTelegramAdapter()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def handler(_event):
|
||||
await release.wait()
|
||||
return "ack"
|
||||
|
||||
async def hold_typing(_chat_id, interval=2.0, metadata=None):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
adapter.set_message_handler(handler)
|
||||
adapter._keep_typing = hold_typing
|
||||
|
||||
event = _make_event("-1001", "17585")
|
||||
task = asyncio.create_task(adapter._process_message_background(event, build_session_key(event.source)))
|
||||
await asyncio.sleep(0)
|
||||
task.cancel()
|
||||
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
|
||||
assert adapter.processing_hooks == [
|
||||
("start", "1"),
|
||||
("complete", "1", False),
|
||||
]
|
||||
|
||||
@@ -1,15 +1,11 @@
|
||||
"""Tests for gateway configuration management."""
|
||||
|
||||
import os
|
||||
from unittest.mock import patch
|
||||
|
||||
from gateway.config import (
|
||||
GatewayConfig,
|
||||
HomeChannel,
|
||||
Platform,
|
||||
PlatformConfig,
|
||||
SessionResetPolicy,
|
||||
_apply_env_overrides,
|
||||
load_gateway_config,
|
||||
)
|
||||
|
||||
@@ -196,75 +192,3 @@ class TestLoadGatewayConfig:
|
||||
|
||||
assert config.unauthorized_dm_behavior == "ignore"
|
||||
assert config.platforms[Platform.WHATSAPP].extra["unauthorized_dm_behavior"] == "pair"
|
||||
|
||||
|
||||
class TestHomeChannelEnvOverrides:
|
||||
"""Home channel env vars should apply even when the platform was already
|
||||
configured via config.yaml (not just when credential env vars create it)."""
|
||||
|
||||
def test_existing_platform_configs_accept_home_channel_env_overrides(self):
|
||||
cases = [
|
||||
(
|
||||
Platform.SLACK,
|
||||
PlatformConfig(enabled=True, token="xoxb-from-config"),
|
||||
{"SLACK_HOME_CHANNEL": "C123", "SLACK_HOME_CHANNEL_NAME": "Ops"},
|
||||
("C123", "Ops"),
|
||||
),
|
||||
(
|
||||
Platform.SIGNAL,
|
||||
PlatformConfig(
|
||||
enabled=True,
|
||||
extra={"http_url": "http://localhost:9090", "account": "+15551234567"},
|
||||
),
|
||||
{"SIGNAL_HOME_CHANNEL": "+1555000", "SIGNAL_HOME_CHANNEL_NAME": "Phone"},
|
||||
("+1555000", "Phone"),
|
||||
),
|
||||
(
|
||||
Platform.MATTERMOST,
|
||||
PlatformConfig(
|
||||
enabled=True,
|
||||
token="mm-token",
|
||||
extra={"url": "https://mm.example.com"},
|
||||
),
|
||||
{"MATTERMOST_HOME_CHANNEL": "ch_abc123", "MATTERMOST_HOME_CHANNEL_NAME": "General"},
|
||||
("ch_abc123", "General"),
|
||||
),
|
||||
(
|
||||
Platform.MATRIX,
|
||||
PlatformConfig(
|
||||
enabled=True,
|
||||
token="syt_abc123",
|
||||
extra={"homeserver": "https://matrix.example.org"},
|
||||
),
|
||||
{"MATRIX_HOME_ROOM": "!room123:example.org", "MATRIX_HOME_ROOM_NAME": "Bot Room"},
|
||||
("!room123:example.org", "Bot Room"),
|
||||
),
|
||||
(
|
||||
Platform.EMAIL,
|
||||
PlatformConfig(
|
||||
enabled=True,
|
||||
extra={
|
||||
"address": "hermes@test.com",
|
||||
"imap_host": "imap.test.com",
|
||||
"smtp_host": "smtp.test.com",
|
||||
},
|
||||
),
|
||||
{"EMAIL_HOME_ADDRESS": "user@test.com", "EMAIL_HOME_ADDRESS_NAME": "Inbox"},
|
||||
("user@test.com", "Inbox"),
|
||||
),
|
||||
(
|
||||
Platform.SMS,
|
||||
PlatformConfig(enabled=True, api_key="token_abc"),
|
||||
{"SMS_HOME_CHANNEL": "+15559876543", "SMS_HOME_CHANNEL_NAME": "My Phone"},
|
||||
("+15559876543", "My Phone"),
|
||||
),
|
||||
]
|
||||
|
||||
for platform, platform_config, env, expected in cases:
|
||||
config = GatewayConfig(platforms={platform: platform_config})
|
||||
with patch.dict(os.environ, env, clear=True):
|
||||
_apply_env_overrides(config)
|
||||
|
||||
home = config.platforms[platform].home_channel
|
||||
assert home is not None, f"{platform.value}: home_channel should not be None"
|
||||
assert (home.chat_id, home.name) == expected, platform.value
|
||||
|
||||
@@ -1,170 +0,0 @@
|
||||
"""Tests for Discord message reactions tied to processing lifecycle hooks."""
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import MessageEvent, MessageType, SendResult
|
||||
from gateway.session import SessionSource, build_session_key
|
||||
|
||||
|
||||
def _ensure_discord_mock():
|
||||
if "discord" in sys.modules and hasattr(sys.modules["discord"], "__file__"):
|
||||
return
|
||||
|
||||
discord_mod = MagicMock()
|
||||
discord_mod.Intents.default.return_value = MagicMock()
|
||||
discord_mod.DMChannel = type("DMChannel", (), {})
|
||||
discord_mod.Thread = type("Thread", (), {})
|
||||
discord_mod.ForumChannel = type("ForumChannel", (), {})
|
||||
discord_mod.Interaction = object
|
||||
discord_mod.app_commands = SimpleNamespace(
|
||||
describe=lambda **kwargs: (lambda fn: fn),
|
||||
choices=lambda **kwargs: (lambda fn: fn),
|
||||
Choice=lambda **kwargs: SimpleNamespace(**kwargs),
|
||||
)
|
||||
|
||||
ext_mod = MagicMock()
|
||||
commands_mod = MagicMock()
|
||||
commands_mod.Bot = MagicMock
|
||||
ext_mod.commands = commands_mod
|
||||
|
||||
sys.modules.setdefault("discord", discord_mod)
|
||||
sys.modules.setdefault("discord.ext", ext_mod)
|
||||
sys.modules.setdefault("discord.ext.commands", commands_mod)
|
||||
|
||||
|
||||
_ensure_discord_mock()
|
||||
|
||||
from gateway.platforms.discord import DiscordAdapter # noqa: E402
|
||||
|
||||
|
||||
class FakeTree:
|
||||
def __init__(self):
|
||||
self.commands = {}
|
||||
|
||||
def command(self, *, name, description):
|
||||
def decorator(fn):
|
||||
self.commands[name] = fn
|
||||
return fn
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def adapter():
|
||||
config = PlatformConfig(enabled=True, token="***")
|
||||
adapter = DiscordAdapter(config)
|
||||
adapter._client = SimpleNamespace(
|
||||
tree=FakeTree(),
|
||||
get_channel=lambda _id: None,
|
||||
fetch_channel=AsyncMock(),
|
||||
user=SimpleNamespace(id=99999, name="HermesBot"),
|
||||
)
|
||||
return adapter
|
||||
|
||||
|
||||
def _make_event(message_id: str, raw_message) -> MessageEvent:
|
||||
return MessageEvent(
|
||||
text="hello",
|
||||
message_type=MessageType.TEXT,
|
||||
source=SessionSource(
|
||||
platform=Platform.DISCORD,
|
||||
chat_id="123",
|
||||
chat_type="dm",
|
||||
user_id="42",
|
||||
user_name="Jezza",
|
||||
),
|
||||
raw_message=raw_message,
|
||||
message_id=message_id,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_process_message_background_adds_and_swaps_reactions(adapter):
|
||||
raw_message = SimpleNamespace(
|
||||
add_reaction=AsyncMock(),
|
||||
remove_reaction=AsyncMock(),
|
||||
)
|
||||
|
||||
async def handler(_event):
|
||||
await asyncio.sleep(0)
|
||||
return "ack"
|
||||
|
||||
async def hold_typing(_chat_id, interval=2.0, metadata=None):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
adapter.set_message_handler(handler)
|
||||
adapter.send = AsyncMock(return_value=SendResult(success=True, message_id="999"))
|
||||
adapter._keep_typing = hold_typing
|
||||
|
||||
event = _make_event("1", raw_message)
|
||||
await adapter._process_message_background(event, build_session_key(event.source))
|
||||
|
||||
assert raw_message.add_reaction.await_args_list[0].args == ("👀",)
|
||||
assert raw_message.remove_reaction.await_args_list[0].args == ("👀", adapter._client.user)
|
||||
assert raw_message.add_reaction.await_args_list[1].args == ("✅",)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_interaction_backed_events_do_not_attempt_reactions(adapter):
|
||||
interaction = SimpleNamespace(guild_id=123456789)
|
||||
|
||||
async def handler(_event):
|
||||
await asyncio.sleep(0)
|
||||
return None
|
||||
|
||||
async def hold_typing(_chat_id, interval=2.0, metadata=None):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
adapter.set_message_handler(handler)
|
||||
adapter._add_reaction = AsyncMock()
|
||||
adapter._remove_reaction = AsyncMock()
|
||||
adapter._keep_typing = hold_typing
|
||||
|
||||
event = MessageEvent(
|
||||
text="/status",
|
||||
message_type=MessageType.COMMAND,
|
||||
source=SessionSource(
|
||||
platform=Platform.DISCORD,
|
||||
chat_id="123",
|
||||
chat_type="dm",
|
||||
user_id="42",
|
||||
user_name="Jezza",
|
||||
),
|
||||
raw_message=interaction,
|
||||
message_id="2",
|
||||
)
|
||||
|
||||
await adapter._process_message_background(event, build_session_key(event.source))
|
||||
|
||||
adapter._add_reaction.assert_not_awaited()
|
||||
adapter._remove_reaction.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_reaction_helper_failures_do_not_break_message_flow(adapter):
|
||||
raw_message = SimpleNamespace(
|
||||
add_reaction=AsyncMock(side_effect=[RuntimeError("no perms"), RuntimeError("no perms")]),
|
||||
remove_reaction=AsyncMock(side_effect=RuntimeError("no perms")),
|
||||
)
|
||||
|
||||
async def handler(_event):
|
||||
await asyncio.sleep(0)
|
||||
return "ack"
|
||||
|
||||
async def hold_typing(_chat_id, interval=2.0, metadata=None):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
adapter.set_message_handler(handler)
|
||||
adapter.send = AsyncMock(return_value=SendResult(success=True, message_id="999"))
|
||||
adapter._keep_typing = hold_typing
|
||||
|
||||
event = _make_event("3", raw_message)
|
||||
await adapter._process_message_background(event, build_session_key(event.source))
|
||||
|
||||
adapter.send.assert_awaited_once()
|
||||
@@ -1057,122 +1057,5 @@ class TestSendEmailStandalone(unittest.TestCase):
|
||||
self.assertIn("not configured", result["error"])
|
||||
|
||||
|
||||
class TestSmtpConnectionCleanup(unittest.TestCase):
|
||||
"""Verify SMTP connections are closed even when send_message raises."""
|
||||
|
||||
@patch.dict(os.environ, {
|
||||
"EMAIL_ADDRESS": "hermes@test.com",
|
||||
"EMAIL_PASSWORD": "secret",
|
||||
"EMAIL_IMAP_HOST": "imap.test.com",
|
||||
"EMAIL_SMTP_HOST": "smtp.test.com",
|
||||
"EMAIL_SMTP_PORT": "587",
|
||||
}, clear=False)
|
||||
def _make_adapter(self):
|
||||
from gateway.config import PlatformConfig
|
||||
from gateway.platforms.email import EmailAdapter
|
||||
return EmailAdapter(PlatformConfig(enabled=True))
|
||||
|
||||
@patch.dict(os.environ, {
|
||||
"EMAIL_ADDRESS": "hermes@test.com",
|
||||
"EMAIL_PASSWORD": "secret",
|
||||
"EMAIL_IMAP_HOST": "imap.test.com",
|
||||
"EMAIL_SMTP_HOST": "smtp.test.com",
|
||||
"EMAIL_SMTP_PORT": "587",
|
||||
}, clear=False)
|
||||
def test_smtp_quit_called_on_send_message_failure(self):
|
||||
"""SMTP quit() must be called even when send_message() raises."""
|
||||
adapter = self._make_adapter()
|
||||
mock_smtp = MagicMock()
|
||||
mock_smtp.send_message.side_effect = Exception("send failed")
|
||||
|
||||
with patch("smtplib.SMTP", return_value=mock_smtp):
|
||||
with self.assertRaises(Exception):
|
||||
adapter._send_email("user@test.com", "Hello")
|
||||
|
||||
mock_smtp.quit.assert_called_once()
|
||||
|
||||
@patch.dict(os.environ, {
|
||||
"EMAIL_ADDRESS": "hermes@test.com",
|
||||
"EMAIL_PASSWORD": "secret",
|
||||
"EMAIL_IMAP_HOST": "imap.test.com",
|
||||
"EMAIL_SMTP_HOST": "smtp.test.com",
|
||||
"EMAIL_SMTP_PORT": "587",
|
||||
}, clear=False)
|
||||
def test_smtp_close_called_when_quit_also_fails(self):
|
||||
"""If both send_message() and quit() fail, close() is the fallback."""
|
||||
adapter = self._make_adapter()
|
||||
mock_smtp = MagicMock()
|
||||
mock_smtp.send_message.side_effect = Exception("send failed")
|
||||
mock_smtp.quit.side_effect = Exception("quit failed")
|
||||
|
||||
with patch("smtplib.SMTP", return_value=mock_smtp):
|
||||
with self.assertRaises(Exception):
|
||||
adapter._send_email("user@test.com", "Hello")
|
||||
|
||||
mock_smtp.close.assert_called_once()
|
||||
|
||||
|
||||
class TestImapConnectionCleanup(unittest.TestCase):
|
||||
"""Verify IMAP connections are closed even when fetch raises."""
|
||||
|
||||
@patch.dict(os.environ, {
|
||||
"EMAIL_ADDRESS": "hermes@test.com",
|
||||
"EMAIL_PASSWORD": "secret",
|
||||
"EMAIL_IMAP_HOST": "imap.test.com",
|
||||
"EMAIL_IMAP_PORT": "993",
|
||||
"EMAIL_SMTP_HOST": "smtp.test.com",
|
||||
}, clear=False)
|
||||
def _make_adapter(self):
|
||||
from gateway.config import PlatformConfig
|
||||
from gateway.platforms.email import EmailAdapter
|
||||
return EmailAdapter(PlatformConfig(enabled=True))
|
||||
|
||||
@patch.dict(os.environ, {
|
||||
"EMAIL_ADDRESS": "hermes@test.com",
|
||||
"EMAIL_PASSWORD": "secret",
|
||||
"EMAIL_IMAP_HOST": "imap.test.com",
|
||||
"EMAIL_IMAP_PORT": "993",
|
||||
"EMAIL_SMTP_HOST": "smtp.test.com",
|
||||
}, clear=False)
|
||||
def test_imap_logout_called_on_uid_fetch_failure(self):
|
||||
"""IMAP logout() must be called even when uid fetch raises."""
|
||||
adapter = self._make_adapter()
|
||||
mock_imap = MagicMock()
|
||||
|
||||
def uid_handler(command, *args):
|
||||
if command == "search":
|
||||
return ("OK", [b"1"])
|
||||
if command == "fetch":
|
||||
raise Exception("fetch failed")
|
||||
return ("NO", [])
|
||||
|
||||
mock_imap.uid.side_effect = uid_handler
|
||||
|
||||
with patch("imaplib.IMAP4_SSL", return_value=mock_imap):
|
||||
results = adapter._fetch_new_messages()
|
||||
|
||||
self.assertEqual(results, [])
|
||||
mock_imap.logout.assert_called_once()
|
||||
|
||||
@patch.dict(os.environ, {
|
||||
"EMAIL_ADDRESS": "hermes@test.com",
|
||||
"EMAIL_PASSWORD": "secret",
|
||||
"EMAIL_IMAP_HOST": "imap.test.com",
|
||||
"EMAIL_IMAP_PORT": "993",
|
||||
"EMAIL_SMTP_HOST": "smtp.test.com",
|
||||
}, clear=False)
|
||||
def test_imap_logout_called_on_early_return(self):
|
||||
"""IMAP logout() must be called even when returning early (no unseen)."""
|
||||
adapter = self._make_adapter()
|
||||
mock_imap = MagicMock()
|
||||
mock_imap.uid.return_value = ("OK", [b""])
|
||||
|
||||
with patch("imaplib.IMAP4_SSL", return_value=mock_imap):
|
||||
results = adapter._fetch_new_messages()
|
||||
|
||||
self.assertEqual(results, [])
|
||||
mock_imap.logout.assert_called_once()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -29,18 +29,13 @@ class TestHookRegistryInit:
|
||||
assert reg._handlers == {}
|
||||
|
||||
|
||||
def _patch_no_builtins(reg):
|
||||
"""Suppress built-in hook registration so tests only exercise user-hook discovery."""
|
||||
return patch.object(reg, "_register_builtin_hooks")
|
||||
|
||||
|
||||
class TestDiscoverAndLoad:
|
||||
def test_loads_valid_hook(self, tmp_path):
|
||||
_create_hook(tmp_path, "my-hook", '["agent:start"]',
|
||||
"def handle(event_type, context):\n pass\n")
|
||||
|
||||
reg = HookRegistry()
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path), _patch_no_builtins(reg):
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path):
|
||||
reg.discover_and_load()
|
||||
|
||||
assert len(reg.loaded_hooks) == 1
|
||||
@@ -53,7 +48,7 @@ class TestDiscoverAndLoad:
|
||||
(hook_dir / "handler.py").write_text("def handle(e, c): pass\n")
|
||||
|
||||
reg = HookRegistry()
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path), _patch_no_builtins(reg):
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path):
|
||||
reg.discover_and_load()
|
||||
|
||||
assert len(reg.loaded_hooks) == 0
|
||||
@@ -64,7 +59,7 @@ class TestDiscoverAndLoad:
|
||||
(hook_dir / "HOOK.yaml").write_text("name: bad\nevents: ['agent:start']\n")
|
||||
|
||||
reg = HookRegistry()
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path), _patch_no_builtins(reg):
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path):
|
||||
reg.discover_and_load()
|
||||
|
||||
assert len(reg.loaded_hooks) == 0
|
||||
@@ -76,7 +71,7 @@ class TestDiscoverAndLoad:
|
||||
(hook_dir / "handler.py").write_text("def handle(e, c): pass\n")
|
||||
|
||||
reg = HookRegistry()
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path), _patch_no_builtins(reg):
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path):
|
||||
reg.discover_and_load()
|
||||
|
||||
assert len(reg.loaded_hooks) == 0
|
||||
@@ -88,14 +83,14 @@ class TestDiscoverAndLoad:
|
||||
(hook_dir / "handler.py").write_text("def something_else(): pass\n")
|
||||
|
||||
reg = HookRegistry()
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path), _patch_no_builtins(reg):
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path):
|
||||
reg.discover_and_load()
|
||||
|
||||
assert len(reg.loaded_hooks) == 0
|
||||
|
||||
def test_nonexistent_hooks_dir(self, tmp_path):
|
||||
reg = HookRegistry()
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path / "nonexistent"), _patch_no_builtins(reg):
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path / "nonexistent"):
|
||||
reg.discover_and_load()
|
||||
|
||||
assert len(reg.loaded_hooks) == 0
|
||||
@@ -107,7 +102,7 @@ class TestDiscoverAndLoad:
|
||||
"def handle(e, c): pass\n")
|
||||
|
||||
reg = HookRegistry()
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path), _patch_no_builtins(reg):
|
||||
with patch("gateway.hooks.HOOKS_DIR", tmp_path):
|
||||
reg.discover_and_load()
|
||||
|
||||
assert len(reg.loaded_hooks) == 2
|
||||
|
||||
@@ -1,340 +0,0 @@
|
||||
"""Tests for Matrix voice message support (MSC3245)."""
|
||||
import io
|
||||
|
||||
import pytest
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
nio = pytest.importorskip("nio", reason="matrix-nio not installed")
|
||||
|
||||
from gateway.platforms.base import MessageType
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Adapter helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _make_adapter():
|
||||
"""Create a MatrixAdapter with mocked config."""
|
||||
from gateway.platforms.matrix import MatrixAdapter
|
||||
from gateway.config import PlatformConfig
|
||||
|
||||
config = PlatformConfig(
|
||||
enabled=True,
|
||||
token="***",
|
||||
extra={
|
||||
"homeserver": "https://matrix.example.org",
|
||||
"user_id": "@bot:example.org",
|
||||
},
|
||||
)
|
||||
adapter = MatrixAdapter(config)
|
||||
return adapter
|
||||
|
||||
|
||||
def _make_room(room_id: str = "!test:example.org", member_count: int = 2):
|
||||
"""Create a mock Matrix room."""
|
||||
room = MagicMock()
|
||||
room.room_id = room_id
|
||||
room.member_count = member_count
|
||||
return room
|
||||
|
||||
|
||||
def _make_audio_event(
|
||||
event_id: str = "$audio_event",
|
||||
sender: str = "@alice:example.org",
|
||||
body: str = "Voice message",
|
||||
url: str = "mxc://example.org/abc123",
|
||||
is_voice: bool = False,
|
||||
mimetype: str = "audio/ogg",
|
||||
timestamp: float = 9999999999000, # ms
|
||||
):
|
||||
"""
|
||||
Create a mock RoomMessageAudio event that passes isinstance checks.
|
||||
|
||||
Args:
|
||||
is_voice: If True, adds org.matrix.msc3245.voice field to content
|
||||
"""
|
||||
import nio
|
||||
|
||||
# Build the source dict that nio events expose via .source
|
||||
content = {
|
||||
"msgtype": "m.audio",
|
||||
"body": body,
|
||||
"url": url,
|
||||
"info": {
|
||||
"mimetype": mimetype,
|
||||
},
|
||||
}
|
||||
|
||||
if is_voice:
|
||||
content["org.matrix.msc3245.voice"] = {}
|
||||
|
||||
# Create a real nio RoomMessageAudio-like object
|
||||
# We use MagicMock but configure __class__ to pass isinstance check
|
||||
event = MagicMock(spec=nio.RoomMessageAudio)
|
||||
event.event_id = event_id
|
||||
event.sender = sender
|
||||
event.body = body
|
||||
event.url = url
|
||||
event.server_timestamp = timestamp
|
||||
event.source = {
|
||||
"type": "m.room.message",
|
||||
"content": content,
|
||||
}
|
||||
# For MIME type extraction - needs to be a dict
|
||||
event.content = content
|
||||
|
||||
return event
|
||||
|
||||
|
||||
def _make_download_response(body: bytes = b"fake audio data"):
|
||||
"""Create a mock nio.MemoryDownloadResponse."""
|
||||
import nio
|
||||
resp = MagicMock()
|
||||
resp.body = body
|
||||
resp.__class__ = nio.MemoryDownloadResponse
|
||||
return resp
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests: MSC3245 Voice Detection (RED -> GREEN)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestMatrixVoiceMessageDetection:
|
||||
"""Test that MSC3245 voice messages are detected and tagged correctly."""
|
||||
|
||||
def setup_method(self):
|
||||
self.adapter = _make_adapter()
|
||||
self.adapter._user_id = "@bot:example.org"
|
||||
self.adapter._startup_ts = 0.0
|
||||
self.adapter._dm_rooms = {}
|
||||
self.adapter._message_handler = AsyncMock()
|
||||
# Mock _mxc_to_http to return a fake HTTP URL
|
||||
self.adapter._mxc_to_http = lambda url: f"https://matrix.example.org/_matrix/media/v3/download/{url[6:]}"
|
||||
# Mock client for authenticated download
|
||||
self.adapter._client = MagicMock()
|
||||
self.adapter._client.download = AsyncMock(return_value=_make_download_response())
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_voice_message_has_type_voice(self):
|
||||
"""Voice messages (with MSC3245 field) should be MessageType.VOICE."""
|
||||
room = _make_room()
|
||||
event = _make_audio_event(is_voice=True)
|
||||
|
||||
# Capture the MessageEvent passed to handle_message
|
||||
captured_event = None
|
||||
|
||||
async def capture(msg_event):
|
||||
nonlocal captured_event
|
||||
captured_event = msg_event
|
||||
|
||||
self.adapter.handle_message = capture
|
||||
|
||||
await self.adapter._on_room_message_media(room, event)
|
||||
|
||||
assert captured_event is not None, "No event was captured"
|
||||
assert captured_event.message_type == MessageType.VOICE, \
|
||||
f"Expected MessageType.VOICE, got {captured_event.message_type}"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_voice_message_has_local_path(self):
|
||||
"""Voice messages should have a local cached path in media_urls."""
|
||||
room = _make_room()
|
||||
event = _make_audio_event(is_voice=True)
|
||||
|
||||
captured_event = None
|
||||
|
||||
async def capture(msg_event):
|
||||
nonlocal captured_event
|
||||
captured_event = msg_event
|
||||
|
||||
self.adapter.handle_message = capture
|
||||
|
||||
await self.adapter._on_room_message_media(room, event)
|
||||
|
||||
assert captured_event is not None
|
||||
assert captured_event.media_urls is not None
|
||||
assert len(captured_event.media_urls) > 0
|
||||
# Should be a local path, not an HTTP URL
|
||||
assert not captured_event.media_urls[0].startswith("http"), \
|
||||
f"media_urls should contain local path, got {captured_event.media_urls[0]}"
|
||||
self.adapter._client.download.assert_awaited_once_with(mxc=event.url)
|
||||
assert captured_event.media_types == ["audio/ogg"]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_audio_without_msc3245_stays_audio_type(self):
|
||||
"""Regular audio uploads (no MSC3245 field) should remain MessageType.AUDIO."""
|
||||
room = _make_room()
|
||||
event = _make_audio_event(is_voice=False) # NOT a voice message
|
||||
|
||||
captured_event = None
|
||||
|
||||
async def capture(msg_event):
|
||||
nonlocal captured_event
|
||||
captured_event = msg_event
|
||||
|
||||
self.adapter.handle_message = capture
|
||||
|
||||
await self.adapter._on_room_message_media(room, event)
|
||||
|
||||
assert captured_event is not None
|
||||
assert captured_event.message_type == MessageType.AUDIO, \
|
||||
f"Expected MessageType.AUDIO for non-voice, got {captured_event.message_type}"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_regular_audio_has_http_url(self):
|
||||
"""Regular audio uploads should keep HTTP URL (not cached locally)."""
|
||||
room = _make_room()
|
||||
event = _make_audio_event(is_voice=False)
|
||||
|
||||
captured_event = None
|
||||
|
||||
async def capture(msg_event):
|
||||
nonlocal captured_event
|
||||
captured_event = msg_event
|
||||
|
||||
self.adapter.handle_message = capture
|
||||
|
||||
await self.adapter._on_room_message_media(room, event)
|
||||
|
||||
assert captured_event is not None
|
||||
assert captured_event.media_urls is not None
|
||||
# Should be HTTP URL, not local path
|
||||
assert captured_event.media_urls[0].startswith("http"), \
|
||||
f"Non-voice audio should have HTTP URL, got {captured_event.media_urls[0]}"
|
||||
self.adapter._client.download.assert_not_awaited()
|
||||
assert captured_event.media_types == ["audio/ogg"]
|
||||
|
||||
|
||||
class TestMatrixVoiceCacheFallback:
|
||||
"""Test graceful fallback when voice caching fails."""
|
||||
|
||||
def setup_method(self):
|
||||
self.adapter = _make_adapter()
|
||||
self.adapter._user_id = "@bot:example.org"
|
||||
self.adapter._startup_ts = 0.0
|
||||
self.adapter._dm_rooms = {}
|
||||
self.adapter._message_handler = AsyncMock()
|
||||
self.adapter._mxc_to_http = lambda url: f"https://matrix.example.org/_matrix/media/v3/download/{url[6:]}"
|
||||
self.adapter._client = MagicMock()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_voice_cache_failure_falls_back_to_http_url(self):
|
||||
"""If caching fails, voice message should still be delivered with HTTP URL."""
|
||||
room = _make_room()
|
||||
event = _make_audio_event(is_voice=True)
|
||||
|
||||
# Make download fail
|
||||
import nio
|
||||
error_resp = MagicMock()
|
||||
error_resp.__class__ = nio.DownloadError
|
||||
self.adapter._client.download = AsyncMock(return_value=error_resp)
|
||||
|
||||
captured_event = None
|
||||
|
||||
async def capture(msg_event):
|
||||
nonlocal captured_event
|
||||
captured_event = msg_event
|
||||
|
||||
self.adapter.handle_message = capture
|
||||
|
||||
await self.adapter._on_room_message_media(room, event)
|
||||
|
||||
assert captured_event is not None
|
||||
assert captured_event.media_urls is not None
|
||||
# Should fall back to HTTP URL
|
||||
assert captured_event.media_urls[0].startswith("http"), \
|
||||
f"Should fall back to HTTP URL on cache failure, got {captured_event.media_urls[0]}"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_voice_cache_exception_falls_back_to_http_url(self):
|
||||
"""Unexpected download exceptions should also fall back to HTTP URL."""
|
||||
room = _make_room()
|
||||
event = _make_audio_event(is_voice=True)
|
||||
|
||||
self.adapter._client.download = AsyncMock(side_effect=RuntimeError("boom"))
|
||||
|
||||
captured_event = None
|
||||
|
||||
async def capture(msg_event):
|
||||
nonlocal captured_event
|
||||
captured_event = msg_event
|
||||
|
||||
self.adapter.handle_message = capture
|
||||
|
||||
await self.adapter._on_room_message_media(room, event)
|
||||
|
||||
assert captured_event is not None
|
||||
assert captured_event.media_urls is not None
|
||||
assert captured_event.media_urls[0].startswith("http"), \
|
||||
f"Should fall back to HTTP URL on exception, got {captured_event.media_urls[0]}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tests: send_voice includes MSC3245 field (RED -> GREEN)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestMatrixSendVoiceMSC3245:
|
||||
"""Test that send_voice includes MSC3245 field for native voice rendering."""
|
||||
|
||||
def setup_method(self):
|
||||
self.adapter = _make_adapter()
|
||||
self.adapter._user_id = "@bot:example.org"
|
||||
# Mock client with successful upload
|
||||
self.adapter._client = MagicMock()
|
||||
self.upload_call = None
|
||||
|
||||
async def mock_upload(*args, **kwargs):
|
||||
self.upload_call = (args, kwargs)
|
||||
import nio
|
||||
resp = MagicMock()
|
||||
resp.content_uri = "mxc://example.org/uploaded"
|
||||
resp.__class__ = nio.UploadResponse
|
||||
return resp, None
|
||||
|
||||
self.adapter._client.upload = mock_upload
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_voice_includes_msc3245_field(self):
|
||||
"""send_voice should include org.matrix.msc3245.voice in message content."""
|
||||
import tempfile
|
||||
import os
|
||||
|
||||
# Create a temp audio file
|
||||
with tempfile.NamedTemporaryFile(suffix=".ogg", delete=False) as f:
|
||||
f.write(b"fake audio data")
|
||||
temp_path = f.name
|
||||
|
||||
try:
|
||||
# Capture the message content sent to room_send
|
||||
sent_content = None
|
||||
|
||||
async def mock_room_send(room_id, event_type, content):
|
||||
nonlocal sent_content
|
||||
sent_content = content
|
||||
resp = MagicMock()
|
||||
resp.event_id = "$sent_event"
|
||||
import nio
|
||||
resp.__class__ = nio.RoomSendResponse
|
||||
return resp
|
||||
|
||||
self.adapter._client.room_send = mock_room_send
|
||||
|
||||
await self.adapter.send_voice(
|
||||
chat_id="!room:example.org",
|
||||
audio_path=temp_path,
|
||||
caption="Test voice",
|
||||
)
|
||||
|
||||
assert sent_content is not None, "No message was sent"
|
||||
assert "org.matrix.msc3245.voice" in sent_content, \
|
||||
f"MSC3245 voice field missing from content: {sent_content.keys()}"
|
||||
assert sent_content["msgtype"] == "m.audio"
|
||||
assert sent_content["info"]["mimetype"] == "audio/ogg"
|
||||
assert self.upload_call is not None, "Expected upload() to be called"
|
||||
args, kwargs = self.upload_call
|
||||
assert isinstance(args[0], io.BytesIO)
|
||||
assert kwargs["content_type"] == "audio/ogg"
|
||||
assert kwargs["filename"].endswith(".ogg")
|
||||
|
||||
finally:
|
||||
os.unlink(temp_path)
|
||||
@@ -126,20 +126,9 @@ class TestAppMentionHandler:
|
||||
"user": "testbot",
|
||||
})
|
||||
|
||||
# Mock AsyncWebClient so multi-workspace auth_test is awaitable
|
||||
mock_web_client = AsyncMock()
|
||||
mock_web_client.auth_test = AsyncMock(return_value={
|
||||
"user_id": "U_BOT",
|
||||
"user": "testbot",
|
||||
"team_id": "T_FAKE",
|
||||
"team": "FakeTeam",
|
||||
})
|
||||
|
||||
with patch.object(_slack_mod, "AsyncApp", return_value=mock_app), \
|
||||
patch.object(_slack_mod, "AsyncWebClient", return_value=mock_web_client), \
|
||||
patch.object(_slack_mod, "AsyncSocketModeHandler", return_value=MagicMock()), \
|
||||
patch.dict(os.environ, {"SLACK_APP_TOKEN": "xapp-fake"}), \
|
||||
patch("gateway.status.acquire_scoped_lock", return_value=(True, None)), \
|
||||
patch("asyncio.create_task"):
|
||||
asyncio.run(adapter.connect())
|
||||
|
||||
|
||||
@@ -1,110 +0,0 @@
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
from gateway.config import Platform, PlatformConfig, load_gateway_config
|
||||
|
||||
|
||||
def _make_adapter(require_mention=None, free_response_chats=None, mention_patterns=None):
|
||||
from gateway.platforms.telegram import TelegramAdapter
|
||||
|
||||
extra = {}
|
||||
if require_mention is not None:
|
||||
extra["require_mention"] = require_mention
|
||||
if free_response_chats is not None:
|
||||
extra["free_response_chats"] = free_response_chats
|
||||
if mention_patterns is not None:
|
||||
extra["mention_patterns"] = mention_patterns
|
||||
|
||||
adapter = object.__new__(TelegramAdapter)
|
||||
adapter.platform = Platform.TELEGRAM
|
||||
adapter.config = PlatformConfig(enabled=True, token="***", extra=extra)
|
||||
adapter._bot = SimpleNamespace(id=999, username="hermes_bot")
|
||||
adapter._message_handler = AsyncMock()
|
||||
adapter._pending_text_batches = {}
|
||||
adapter._pending_text_batch_tasks = {}
|
||||
adapter._text_batch_delay_seconds = 0.01
|
||||
adapter._mention_patterns = adapter._compile_mention_patterns()
|
||||
return adapter
|
||||
|
||||
|
||||
def _group_message(text="hello", *, chat_id=-100, reply_to_bot=False, entities=None, caption=None, caption_entities=None):
|
||||
reply_to_message = None
|
||||
if reply_to_bot:
|
||||
reply_to_message = SimpleNamespace(from_user=SimpleNamespace(id=999))
|
||||
return SimpleNamespace(
|
||||
text=text,
|
||||
caption=caption,
|
||||
entities=entities or [],
|
||||
caption_entities=caption_entities or [],
|
||||
chat=SimpleNamespace(id=chat_id, type="group"),
|
||||
reply_to_message=reply_to_message,
|
||||
)
|
||||
|
||||
|
||||
def _mention_entity(text, mention="@hermes_bot"):
|
||||
offset = text.index(mention)
|
||||
return SimpleNamespace(type="mention", offset=offset, length=len(mention))
|
||||
|
||||
|
||||
def test_group_messages_can_be_opened_via_config():
|
||||
adapter = _make_adapter(require_mention=False)
|
||||
|
||||
assert adapter._should_process_message(_group_message("hello everyone")) is True
|
||||
|
||||
|
||||
def test_group_messages_can_require_direct_trigger_via_config():
|
||||
adapter = _make_adapter(require_mention=True)
|
||||
|
||||
assert adapter._should_process_message(_group_message("hello everyone")) is False
|
||||
assert adapter._should_process_message(_group_message("hi @hermes_bot", entities=[_mention_entity("hi @hermes_bot")])) is True
|
||||
assert adapter._should_process_message(_group_message("replying", reply_to_bot=True)) is True
|
||||
assert adapter._should_process_message(_group_message("/status"), is_command=True) is True
|
||||
|
||||
|
||||
def test_free_response_chats_bypass_mention_requirement():
|
||||
adapter = _make_adapter(require_mention=True, free_response_chats=["-200"])
|
||||
|
||||
assert adapter._should_process_message(_group_message("hello everyone", chat_id=-200)) is True
|
||||
assert adapter._should_process_message(_group_message("hello everyone", chat_id=-201)) is False
|
||||
|
||||
|
||||
def test_regex_mention_patterns_allow_custom_wake_words():
|
||||
adapter = _make_adapter(require_mention=True, mention_patterns=[r"^\s*chompy\b"])
|
||||
|
||||
assert adapter._should_process_message(_group_message("chompy status")) is True
|
||||
assert adapter._should_process_message(_group_message(" chompy help")) is True
|
||||
assert adapter._should_process_message(_group_message("hey chompy")) is False
|
||||
|
||||
|
||||
def test_invalid_regex_patterns_are_ignored():
|
||||
adapter = _make_adapter(require_mention=True, mention_patterns=[r"(", r"^\s*chompy\b"])
|
||||
|
||||
assert adapter._should_process_message(_group_message("chompy status")) is True
|
||||
assert adapter._should_process_message(_group_message("hello everyone")) is False
|
||||
|
||||
|
||||
def test_config_bridges_telegram_group_settings(monkeypatch, tmp_path):
|
||||
hermes_home = tmp_path / ".hermes"
|
||||
hermes_home.mkdir()
|
||||
(hermes_home / "config.yaml").write_text(
|
||||
"telegram:\n"
|
||||
" require_mention: true\n"
|
||||
" mention_patterns:\n"
|
||||
" - \"^\\\\s*chompy\\\\b\"\n"
|
||||
" free_response_chats:\n"
|
||||
" - \"-123\"\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
monkeypatch.setenv("HERMES_HOME", str(hermes_home))
|
||||
monkeypatch.delenv("TELEGRAM_REQUIRE_MENTION", raising=False)
|
||||
monkeypatch.delenv("TELEGRAM_MENTION_PATTERNS", raising=False)
|
||||
monkeypatch.delenv("TELEGRAM_FREE_RESPONSE_CHATS", raising=False)
|
||||
|
||||
config = load_gateway_config()
|
||||
|
||||
assert config is not None
|
||||
assert __import__("os").environ["TELEGRAM_REQUIRE_MENTION"] == "true"
|
||||
assert json.loads(__import__("os").environ["TELEGRAM_MENTION_PATTERNS"]) == [r"^\s*chompy\b"]
|
||||
assert __import__("os").environ["TELEGRAM_FREE_RESPONSE_CHATS"] == "-123"
|
||||
@@ -3,7 +3,6 @@ from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
import gateway.run as gateway_run
|
||||
from gateway.config import GatewayConfig, Platform, PlatformConfig
|
||||
from gateway.platforms.base import MessageEvent
|
||||
from gateway.session import SessionSource
|
||||
@@ -20,7 +19,7 @@ def _clear_auth_env(monkeypatch) -> None:
|
||||
"SMS_ALLOWED_USERS",
|
||||
"MATTERMOST_ALLOWED_USERS",
|
||||
"MATRIX_ALLOWED_USERS",
|
||||
"DINGTALK_ALLOWED_USERS", "FEISHU_ALLOWED_USERS", "WECOM_ALLOWED_USERS",
|
||||
"DINGTALK_ALLOWED_USERS",
|
||||
"GATEWAY_ALLOWED_USERS",
|
||||
"TELEGRAM_ALLOW_ALL_USERS",
|
||||
"DISCORD_ALLOW_ALL_USERS",
|
||||
@@ -31,7 +30,7 @@ def _clear_auth_env(monkeypatch) -> None:
|
||||
"SMS_ALLOW_ALL_USERS",
|
||||
"MATTERMOST_ALLOW_ALL_USERS",
|
||||
"MATRIX_ALLOW_ALL_USERS",
|
||||
"DINGTALK_ALLOW_ALL_USERS", "FEISHU_ALLOW_ALL_USERS", "WECOM_ALLOW_ALL_USERS",
|
||||
"DINGTALK_ALLOW_ALL_USERS",
|
||||
"GATEWAY_ALLOW_ALL_USERS",
|
||||
):
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
@@ -63,32 +62,6 @@ def _make_runner(platform: Platform, config: GatewayConfig):
|
||||
return runner, adapter
|
||||
|
||||
|
||||
def test_whatsapp_lid_user_matches_phone_allowlist_via_session_mapping(monkeypatch, tmp_path):
|
||||
_clear_auth_env(monkeypatch)
|
||||
monkeypatch.setenv("WHATSAPP_ALLOWED_USERS", "15550000001")
|
||||
monkeypatch.setattr(gateway_run, "_hermes_home", tmp_path)
|
||||
|
||||
session_dir = tmp_path / "whatsapp" / "session"
|
||||
session_dir.mkdir(parents=True)
|
||||
(session_dir / "lid-mapping-15550000001.json").write_text('"900000000000001"', encoding="utf-8")
|
||||
(session_dir / "lid-mapping-900000000000001_reverse.json").write_text('"15550000001"', encoding="utf-8")
|
||||
|
||||
runner, _adapter = _make_runner(
|
||||
Platform.WHATSAPP,
|
||||
GatewayConfig(platforms={Platform.WHATSAPP: PlatformConfig(enabled=True)}),
|
||||
)
|
||||
|
||||
source = SessionSource(
|
||||
platform=Platform.WHATSAPP,
|
||||
user_id="900000000000001@lid",
|
||||
chat_id="900000000000001@lid",
|
||||
user_name="tester",
|
||||
chat_type="dm",
|
||||
)
|
||||
|
||||
assert runner._is_user_authorized(source) is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unauthorized_dm_pairs_by_default(monkeypatch):
|
||||
_clear_auth_env(monkeypatch)
|
||||
|
||||
@@ -1,596 +0,0 @@
|
||||
"""Tests for the WeCom platform adapter."""
|
||||
|
||||
import base64
|
||||
import os
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import SendResult
|
||||
|
||||
|
||||
class TestWeComRequirements:
|
||||
def test_returns_false_without_aiohttp(self, monkeypatch):
|
||||
monkeypatch.setattr("gateway.platforms.wecom.AIOHTTP_AVAILABLE", False)
|
||||
monkeypatch.setattr("gateway.platforms.wecom.HTTPX_AVAILABLE", True)
|
||||
from gateway.platforms.wecom import check_wecom_requirements
|
||||
|
||||
assert check_wecom_requirements() is False
|
||||
|
||||
def test_returns_false_without_httpx(self, monkeypatch):
|
||||
monkeypatch.setattr("gateway.platforms.wecom.AIOHTTP_AVAILABLE", True)
|
||||
monkeypatch.setattr("gateway.platforms.wecom.HTTPX_AVAILABLE", False)
|
||||
from gateway.platforms.wecom import check_wecom_requirements
|
||||
|
||||
assert check_wecom_requirements() is False
|
||||
|
||||
def test_returns_true_when_available(self, monkeypatch):
|
||||
monkeypatch.setattr("gateway.platforms.wecom.AIOHTTP_AVAILABLE", True)
|
||||
monkeypatch.setattr("gateway.platforms.wecom.HTTPX_AVAILABLE", True)
|
||||
from gateway.platforms.wecom import check_wecom_requirements
|
||||
|
||||
assert check_wecom_requirements() is True
|
||||
|
||||
|
||||
class TestWeComAdapterInit:
|
||||
def test_reads_config_from_extra(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
config = PlatformConfig(
|
||||
enabled=True,
|
||||
extra={
|
||||
"bot_id": "cfg-bot",
|
||||
"secret": "cfg-secret",
|
||||
"websocket_url": "wss://custom.wecom.example/ws",
|
||||
"group_policy": "allowlist",
|
||||
"group_allow_from": ["group-1"],
|
||||
},
|
||||
)
|
||||
adapter = WeComAdapter(config)
|
||||
|
||||
assert adapter._bot_id == "cfg-bot"
|
||||
assert adapter._secret == "cfg-secret"
|
||||
assert adapter._ws_url == "wss://custom.wecom.example/ws"
|
||||
assert adapter._group_policy == "allowlist"
|
||||
assert adapter._group_allow_from == ["group-1"]
|
||||
|
||||
def test_falls_back_to_env_vars(self, monkeypatch):
|
||||
monkeypatch.setenv("WECOM_BOT_ID", "env-bot")
|
||||
monkeypatch.setenv("WECOM_SECRET", "env-secret")
|
||||
monkeypatch.setenv("WECOM_WEBSOCKET_URL", "wss://env.example/ws")
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
assert adapter._bot_id == "env-bot"
|
||||
assert adapter._secret == "env-secret"
|
||||
assert adapter._ws_url == "wss://env.example/ws"
|
||||
|
||||
|
||||
class TestWeComConnect:
|
||||
@pytest.mark.asyncio
|
||||
async def test_connect_records_missing_credentials(self, monkeypatch):
|
||||
import gateway.platforms.wecom as wecom_module
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
monkeypatch.setattr(wecom_module, "AIOHTTP_AVAILABLE", True)
|
||||
monkeypatch.setattr(wecom_module, "HTTPX_AVAILABLE", True)
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
|
||||
success = await adapter.connect()
|
||||
|
||||
assert success is False
|
||||
assert adapter.has_fatal_error is True
|
||||
assert adapter.fatal_error_code == "wecom_missing_credentials"
|
||||
assert "WECOM_BOT_ID" in (adapter.fatal_error_message or "")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_connect_records_handshake_failure_details(self, monkeypatch):
|
||||
import gateway.platforms.wecom as wecom_module
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
class DummyClient:
|
||||
async def aclose(self):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(wecom_module, "AIOHTTP_AVAILABLE", True)
|
||||
monkeypatch.setattr(wecom_module, "HTTPX_AVAILABLE", True)
|
||||
monkeypatch.setattr(
|
||||
wecom_module,
|
||||
"httpx",
|
||||
SimpleNamespace(AsyncClient=lambda **kwargs: DummyClient()),
|
||||
)
|
||||
|
||||
adapter = WeComAdapter(
|
||||
PlatformConfig(enabled=True, extra={"bot_id": "bot-1", "secret": "secret-1"})
|
||||
)
|
||||
adapter._open_connection = AsyncMock(side_effect=RuntimeError("invalid secret (errcode=40013)"))
|
||||
|
||||
success = await adapter.connect()
|
||||
|
||||
assert success is False
|
||||
assert adapter.has_fatal_error is True
|
||||
assert adapter.fatal_error_code == "wecom_connect_error"
|
||||
assert "invalid secret" in (adapter.fatal_error_message or "")
|
||||
|
||||
|
||||
class TestWeComReplyMode:
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_uses_passive_reply_stream_when_reply_context_exists(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter._reply_req_ids["msg-1"] = "req-1"
|
||||
adapter._send_reply_request = AsyncMock(
|
||||
return_value={"headers": {"req_id": "req-1"}, "errcode": 0}
|
||||
)
|
||||
|
||||
result = await adapter.send("chat-123", "hello from reply", reply_to="msg-1")
|
||||
|
||||
assert result.success is True
|
||||
adapter._send_reply_request.assert_awaited_once()
|
||||
args = adapter._send_reply_request.await_args.args
|
||||
assert args[0] == "req-1"
|
||||
assert args[1]["msgtype"] == "stream"
|
||||
assert args[1]["stream"]["finish"] is True
|
||||
assert args[1]["stream"]["content"] == "hello from reply"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_image_file_uses_passive_reply_media_when_reply_context_exists(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter._reply_req_ids["msg-1"] = "req-1"
|
||||
adapter._prepare_outbound_media = AsyncMock(
|
||||
return_value={
|
||||
"data": b"image-bytes",
|
||||
"content_type": "image/png",
|
||||
"file_name": "demo.png",
|
||||
"detected_type": "image",
|
||||
"final_type": "image",
|
||||
"rejected": False,
|
||||
"reject_reason": None,
|
||||
"downgraded": False,
|
||||
"downgrade_note": None,
|
||||
}
|
||||
)
|
||||
adapter._upload_media_bytes = AsyncMock(return_value={"media_id": "media-1", "type": "image"})
|
||||
adapter._send_reply_request = AsyncMock(
|
||||
return_value={"headers": {"req_id": "req-1"}, "errcode": 0}
|
||||
)
|
||||
|
||||
result = await adapter.send_image_file("chat-123", "/tmp/demo.png", reply_to="msg-1")
|
||||
|
||||
assert result.success is True
|
||||
adapter._send_reply_request.assert_awaited_once()
|
||||
args = adapter._send_reply_request.await_args.args
|
||||
assert args[0] == "req-1"
|
||||
assert args[1] == {"msgtype": "image", "image": {"media_id": "media-1"}}
|
||||
|
||||
|
||||
class TestExtractText:
|
||||
def test_extracts_plain_text(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
body = {
|
||||
"msgtype": "text",
|
||||
"text": {"content": " hello world "},
|
||||
}
|
||||
text, reply_text = WeComAdapter._extract_text(body)
|
||||
assert text == "hello world"
|
||||
assert reply_text is None
|
||||
|
||||
def test_extracts_mixed_text(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
body = {
|
||||
"msgtype": "mixed",
|
||||
"mixed": {
|
||||
"msg_item": [
|
||||
{"msgtype": "text", "text": {"content": "part1"}},
|
||||
{"msgtype": "image", "image": {"url": "https://example.com/x.png"}},
|
||||
{"msgtype": "text", "text": {"content": "part2"}},
|
||||
]
|
||||
},
|
||||
}
|
||||
text, _reply_text = WeComAdapter._extract_text(body)
|
||||
assert text == "part1\npart2"
|
||||
|
||||
def test_extracts_voice_and_quote(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
body = {
|
||||
"msgtype": "voice",
|
||||
"voice": {"content": "spoken text"},
|
||||
"quote": {"msgtype": "text", "text": {"content": "quoted"}},
|
||||
}
|
||||
text, reply_text = WeComAdapter._extract_text(body)
|
||||
assert text == "spoken text"
|
||||
assert reply_text == "quoted"
|
||||
|
||||
|
||||
class TestCallbackDispatch:
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("cmd", ["aibot_msg_callback", "aibot_callback"])
|
||||
async def test_dispatch_accepts_new_and_legacy_callback_cmds(self, cmd):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter._on_message = AsyncMock()
|
||||
|
||||
await adapter._dispatch_payload({"cmd": cmd, "headers": {"req_id": "req-1"}, "body": {}})
|
||||
|
||||
adapter._on_message.assert_awaited_once()
|
||||
|
||||
|
||||
class TestPolicyHelpers:
|
||||
def test_dm_allowlist(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(
|
||||
PlatformConfig(enabled=True, extra={"dm_policy": "allowlist", "allow_from": ["user-1"]})
|
||||
)
|
||||
assert adapter._is_dm_allowed("user-1") is True
|
||||
assert adapter._is_dm_allowed("user-2") is False
|
||||
|
||||
def test_group_allowlist_and_per_group_sender_allowlist(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(
|
||||
PlatformConfig(
|
||||
enabled=True,
|
||||
extra={
|
||||
"group_policy": "allowlist",
|
||||
"group_allow_from": ["group-1"],
|
||||
"groups": {"group-1": {"allow_from": ["user-1"]}},
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
assert adapter._is_group_allowed("group-1", "user-1") is True
|
||||
assert adapter._is_group_allowed("group-1", "user-2") is False
|
||||
assert adapter._is_group_allowed("group-2", "user-1") is False
|
||||
|
||||
|
||||
class TestMediaHelpers:
|
||||
def test_detect_wecom_media_type(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
assert WeComAdapter._detect_wecom_media_type("image/png") == "image"
|
||||
assert WeComAdapter._detect_wecom_media_type("video/mp4") == "video"
|
||||
assert WeComAdapter._detect_wecom_media_type("audio/amr") == "voice"
|
||||
assert WeComAdapter._detect_wecom_media_type("application/pdf") == "file"
|
||||
|
||||
def test_voice_non_amr_downgrades_to_file(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
result = WeComAdapter._apply_file_size_limits(128, "voice", "audio/mpeg")
|
||||
|
||||
assert result["final_type"] == "file"
|
||||
assert result["downgraded"] is True
|
||||
assert "AMR" in (result["downgrade_note"] or "")
|
||||
|
||||
def test_oversized_file_is_rejected(self):
|
||||
from gateway.platforms.wecom import ABSOLUTE_MAX_BYTES, WeComAdapter
|
||||
|
||||
result = WeComAdapter._apply_file_size_limits(ABSOLUTE_MAX_BYTES + 1, "file", "application/pdf")
|
||||
|
||||
assert result["rejected"] is True
|
||||
assert "20MB" in (result["reject_reason"] or "")
|
||||
|
||||
def test_decrypt_file_bytes_round_trip(self):
|
||||
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
plaintext = b"wecom-secret"
|
||||
key = os.urandom(32)
|
||||
pad_len = 32 - (len(plaintext) % 32)
|
||||
padded = plaintext + bytes([pad_len]) * pad_len
|
||||
encryptor = Cipher(algorithms.AES(key), modes.CBC(key[:16])).encryptor()
|
||||
encrypted = encryptor.update(padded) + encryptor.finalize()
|
||||
|
||||
decrypted = WeComAdapter._decrypt_file_bytes(encrypted, base64.b64encode(key).decode("ascii"))
|
||||
|
||||
assert decrypted == plaintext
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_load_outbound_media_rejects_placeholder_path(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
|
||||
with pytest.raises(ValueError, match="placeholder was not replaced"):
|
||||
await adapter._load_outbound_media("<path>")
|
||||
|
||||
|
||||
class TestMediaUpload:
|
||||
@pytest.mark.asyncio
|
||||
async def test_upload_media_bytes_uses_sdk_sequence(self, monkeypatch):
|
||||
import gateway.platforms.wecom as wecom_module
|
||||
from gateway.platforms.wecom import (
|
||||
APP_CMD_UPLOAD_MEDIA_CHUNK,
|
||||
APP_CMD_UPLOAD_MEDIA_FINISH,
|
||||
APP_CMD_UPLOAD_MEDIA_INIT,
|
||||
WeComAdapter,
|
||||
)
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
calls = []
|
||||
|
||||
async def fake_send_request(cmd, body, timeout=0):
|
||||
calls.append((cmd, body))
|
||||
if cmd == APP_CMD_UPLOAD_MEDIA_INIT:
|
||||
return {"errcode": 0, "body": {"upload_id": "upload-1"}}
|
||||
if cmd == APP_CMD_UPLOAD_MEDIA_CHUNK:
|
||||
return {"errcode": 0}
|
||||
if cmd == APP_CMD_UPLOAD_MEDIA_FINISH:
|
||||
return {
|
||||
"errcode": 0,
|
||||
"body": {
|
||||
"media_id": "media-1",
|
||||
"type": "file",
|
||||
"created_at": "2026-03-18T00:00:00Z",
|
||||
},
|
||||
}
|
||||
raise AssertionError(f"unexpected cmd {cmd}")
|
||||
|
||||
monkeypatch.setattr(wecom_module, "UPLOAD_CHUNK_SIZE", 4)
|
||||
adapter._send_request = fake_send_request
|
||||
|
||||
result = await adapter._upload_media_bytes(b"abcdefghij", "file", "demo.bin")
|
||||
|
||||
assert result["media_id"] == "media-1"
|
||||
assert [cmd for cmd, _body in calls] == [
|
||||
APP_CMD_UPLOAD_MEDIA_INIT,
|
||||
APP_CMD_UPLOAD_MEDIA_CHUNK,
|
||||
APP_CMD_UPLOAD_MEDIA_CHUNK,
|
||||
APP_CMD_UPLOAD_MEDIA_CHUNK,
|
||||
APP_CMD_UPLOAD_MEDIA_FINISH,
|
||||
]
|
||||
assert calls[1][1]["chunk_index"] == 0
|
||||
assert calls[2][1]["chunk_index"] == 1
|
||||
assert calls[3][1]["chunk_index"] == 2
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_remote_bytes_rejects_large_content_length(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
class FakeResponse:
|
||||
headers = {"content-length": "10"}
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
return None
|
||||
|
||||
def raise_for_status(self):
|
||||
return None
|
||||
|
||||
async def aiter_bytes(self):
|
||||
yield b"abc"
|
||||
|
||||
class FakeClient:
|
||||
def stream(self, method, url, headers=None):
|
||||
return FakeResponse()
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter._http_client = FakeClient()
|
||||
|
||||
with pytest.raises(ValueError, match="exceeds WeCom limit"):
|
||||
await adapter._download_remote_bytes("https://example.com/file.bin", max_bytes=4)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cache_media_decrypts_url_payload_before_writing(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
plaintext = b"secret document bytes"
|
||||
key = os.urandom(32)
|
||||
pad_len = 32 - (len(plaintext) % 32)
|
||||
padded = plaintext + bytes([pad_len]) * pad_len
|
||||
|
||||
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
|
||||
|
||||
encryptor = Cipher(algorithms.AES(key), modes.CBC(key[:16])).encryptor()
|
||||
encrypted = encryptor.update(padded) + encryptor.finalize()
|
||||
adapter._download_remote_bytes = AsyncMock(
|
||||
return_value=(
|
||||
encrypted,
|
||||
{
|
||||
"content-type": "application/octet-stream",
|
||||
"content-disposition": 'attachment; filename="secret.bin"',
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
cached = await adapter._cache_media(
|
||||
"file",
|
||||
{
|
||||
"url": "https://example.com/secret.bin",
|
||||
"aeskey": base64.b64encode(key).decode("ascii"),
|
||||
},
|
||||
)
|
||||
|
||||
assert cached is not None
|
||||
cached_path, content_type = cached
|
||||
assert Path(cached_path).read_bytes() == plaintext
|
||||
assert content_type == "application/octet-stream"
|
||||
|
||||
|
||||
class TestSend:
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_uses_proactive_payload(self):
|
||||
from gateway.platforms.wecom import APP_CMD_SEND, WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter._send_request = AsyncMock(return_value={"headers": {"req_id": "req-1"}, "errcode": 0})
|
||||
|
||||
result = await adapter.send("chat-123", "Hello WeCom")
|
||||
|
||||
assert result.success is True
|
||||
adapter._send_request.assert_awaited_once_with(
|
||||
APP_CMD_SEND,
|
||||
{
|
||||
"chatid": "chat-123",
|
||||
"msgtype": "markdown",
|
||||
"markdown": {"content": "Hello WeCom"},
|
||||
},
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_reports_wecom_errors(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter._send_request = AsyncMock(return_value={"errcode": 40001, "errmsg": "bad request"})
|
||||
|
||||
result = await adapter.send("chat-123", "Hello WeCom")
|
||||
|
||||
assert result.success is False
|
||||
assert "40001" in (result.error or "")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_image_falls_back_to_text_for_remote_url(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter._send_media_source = AsyncMock(return_value=SendResult(success=False, error="upload failed"))
|
||||
adapter.send = AsyncMock(return_value=SendResult(success=True, message_id="msg-1"))
|
||||
|
||||
result = await adapter.send_image("chat-123", "https://example.com/demo.png", caption="demo")
|
||||
|
||||
assert result.success is True
|
||||
adapter.send.assert_awaited_once_with(chat_id="chat-123", content="demo\nhttps://example.com/demo.png", reply_to=None)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_voice_sends_caption_and_downgrade_note(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter._prepare_outbound_media = AsyncMock(
|
||||
return_value={
|
||||
"data": b"voice-bytes",
|
||||
"content_type": "audio/mpeg",
|
||||
"file_name": "voice.mp3",
|
||||
"detected_type": "voice",
|
||||
"final_type": "file",
|
||||
"rejected": False,
|
||||
"reject_reason": None,
|
||||
"downgraded": True,
|
||||
"downgrade_note": "语音格式 audio/mpeg 不支持，企微仅支持 AMR 格式，已转为文件格式发送",
|
||||
}
|
||||
)
|
||||
adapter._upload_media_bytes = AsyncMock(return_value={"media_id": "media-1", "type": "file"})
|
||||
adapter._send_media_message = AsyncMock(return_value={"headers": {"req_id": "req-media"}, "errcode": 0})
|
||||
adapter.send = AsyncMock(return_value=SendResult(success=True, message_id="msg-1"))
|
||||
|
||||
result = await adapter.send_voice("chat-123", "/tmp/voice.mp3", caption="listen")
|
||||
|
||||
assert result.success is True
|
||||
adapter._send_media_message.assert_awaited_once_with("chat-123", "file", "media-1")
|
||||
assert adapter.send.await_count == 2
|
||||
adapter.send.assert_any_await(chat_id="chat-123", content="listen", reply_to=None)
|
||||
adapter.send.assert_any_await(
|
||||
chat_id="chat-123",
|
||||
content="ℹ️ 语音格式 audio/mpeg 不支持，企微仅支持 AMR 格式，已转为文件格式发送",
|
||||
reply_to=None,
|
||||
)
|
||||
|
||||
|
||||
class TestInboundMessages:
|
||||
@pytest.mark.asyncio
|
||||
async def test_on_message_builds_event(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter.handle_message = AsyncMock()
|
||||
adapter._extract_media = AsyncMock(return_value=(["/tmp/test.png"], ["image/png"]))
|
||||
|
||||
payload = {
|
||||
"cmd": "aibot_msg_callback",
|
||||
"headers": {"req_id": "req-1"},
|
||||
"body": {
|
||||
"msgid": "msg-1",
|
||||
"chatid": "group-1",
|
||||
"chattype": "group",
|
||||
"from": {"userid": "user-1"},
|
||||
"msgtype": "text",
|
||||
"text": {"content": "hello"},
|
||||
},
|
||||
}
|
||||
|
||||
await adapter._on_message(payload)
|
||||
|
||||
adapter.handle_message.assert_awaited_once()
|
||||
event = adapter.handle_message.await_args.args[0]
|
||||
assert event.text == "hello"
|
||||
assert event.source.chat_id == "group-1"
|
||||
assert event.source.user_id == "user-1"
|
||||
assert event.media_urls == ["/tmp/test.png"]
|
||||
assert event.media_types == ["image/png"]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_on_message_preserves_quote_context(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(PlatformConfig(enabled=True))
|
||||
adapter.handle_message = AsyncMock()
|
||||
adapter._extract_media = AsyncMock(return_value=([], []))
|
||||
|
||||
payload = {
|
||||
"cmd": "aibot_msg_callback",
|
||||
"headers": {"req_id": "req-1"},
|
||||
"body": {
|
||||
"msgid": "msg-1",
|
||||
"chatid": "group-1",
|
||||
"chattype": "group",
|
||||
"from": {"userid": "user-1"},
|
||||
"msgtype": "text",
|
||||
"text": {"content": "follow up"},
|
||||
"quote": {"msgtype": "text", "text": {"content": "quoted message"}},
|
||||
},
|
||||
}
|
||||
|
||||
await adapter._on_message(payload)
|
||||
|
||||
event = adapter.handle_message.await_args.args[0]
|
||||
assert event.reply_to_text == "quoted message"
|
||||
assert event.reply_to_message_id == "quote:msg-1"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_on_message_respects_group_policy(self):
|
||||
from gateway.platforms.wecom import WeComAdapter
|
||||
|
||||
adapter = WeComAdapter(
|
||||
PlatformConfig(
|
||||
enabled=True,
|
||||
extra={"group_policy": "allowlist", "group_allow_from": ["group-allowed"]},
|
||||
)
|
||||
)
|
||||
adapter.handle_message = AsyncMock()
|
||||
adapter._extract_media = AsyncMock(return_value=([], []))
|
||||
|
||||
payload = {
|
||||
"cmd": "aibot_callback",
|
||||
"headers": {"req_id": "req-1"},
|
||||
"body": {
|
||||
"msgid": "msg-1",
|
||||
"chatid": "group-blocked",
|
||||
"chattype": "group",
|
||||
"from": {"userid": "user-1"},
|
||||
"msgtype": "text",
|
||||
"text": {"content": "hello"},
|
||||
},
|
||||
}
|
||||
|
||||
await adapter._on_message(payload)
|
||||
adapter.handle_message.assert_not_awaited()
|
||||
|
||||
|
||||
class TestPlatformEnum:
|
||||
def test_wecom_in_platform_enum(self):
|
||||
assert Platform.WECOM.value == "wecom"
|
||||
@@ -63,7 +63,6 @@ def _make_adapter():
|
||||
adapter._background_tasks = set()
|
||||
adapter._auto_tts_disabled_chats = set()
|
||||
adapter._message_queue = asyncio.Queue()
|
||||
adapter._http_session = None
|
||||
return adapter
|
||||
|
||||
|
||||
@@ -220,7 +219,6 @@ class TestBridgeRuntimeFailure:
|
||||
fatal_handler = AsyncMock()
|
||||
adapter.set_fatal_error_handler(fatal_handler)
|
||||
adapter._running = True
|
||||
adapter._http_session = MagicMock() # Persistent session active
|
||||
mock_fh = MagicMock()
|
||||
adapter._bridge_log_fh = mock_fh
|
||||
|
||||
@@ -244,7 +242,6 @@ class TestBridgeRuntimeFailure:
|
||||
fatal_handler = AsyncMock()
|
||||
adapter.set_fatal_error_handler(fatal_handler)
|
||||
adapter._running = True
|
||||
adapter._http_session = MagicMock() # Persistent session active
|
||||
mock_fh = MagicMock()
|
||||
adapter._bridge_log_fh = mock_fh
|
||||
|
||||
@@ -420,83 +417,3 @@ class TestKillPortProcess:
|
||||
with patch("gateway.platforms.whatsapp._IS_WINDOWS", True), \
|
||||
patch("gateway.platforms.whatsapp.subprocess.run", side_effect=OSError("no netstat")):
|
||||
_kill_port_process(3000) # must not raise
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Persistent HTTP session lifecycle
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestHttpSessionLifecycle:
|
||||
"""Verify persistent aiohttp.ClientSession is created and cleaned up."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_session_closed_on_disconnect(self):
|
||||
"""disconnect() should close self._http_session."""
|
||||
adapter = _make_adapter()
|
||||
mock_session = AsyncMock()
|
||||
mock_session.closed = False
|
||||
adapter._http_session = mock_session
|
||||
adapter._poll_task = None
|
||||
adapter._bridge_process = None
|
||||
adapter._running = True
|
||||
adapter._session_lock_identity = None
|
||||
|
||||
await adapter.disconnect()
|
||||
|
||||
mock_session.close.assert_called_once()
|
||||
assert adapter._http_session is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_session_not_closed_when_already_closed(self):
|
||||
"""disconnect() should skip close() when session is already closed."""
|
||||
adapter = _make_adapter()
|
||||
mock_session = AsyncMock()
|
||||
mock_session.closed = True
|
||||
adapter._http_session = mock_session
|
||||
adapter._poll_task = None
|
||||
adapter._bridge_process = None
|
||||
adapter._running = True
|
||||
adapter._session_lock_identity = None
|
||||
|
||||
await adapter.disconnect()
|
||||
|
||||
mock_session.close.assert_not_called()
|
||||
assert adapter._http_session is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_poll_task_cancelled_on_disconnect(self):
|
||||
"""disconnect() should cancel the poll task."""
|
||||
adapter = _make_adapter()
|
||||
mock_task = MagicMock()
|
||||
mock_task.done.return_value = False
|
||||
mock_task.cancel = MagicMock()
|
||||
mock_future = asyncio.Future()
|
||||
mock_future.set_exception(asyncio.CancelledError())
|
||||
mock_task.__await__ = mock_future.__await__
|
||||
adapter._poll_task = mock_task
|
||||
adapter._http_session = None
|
||||
adapter._bridge_process = None
|
||||
adapter._running = True
|
||||
adapter._session_lock_identity = None
|
||||
|
||||
await adapter.disconnect()
|
||||
|
||||
mock_task.cancel.assert_called_once()
|
||||
assert adapter._poll_task is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_disconnect_skips_done_poll_task(self):
|
||||
"""disconnect() should not cancel an already-done poll task."""
|
||||
adapter = _make_adapter()
|
||||
mock_task = MagicMock()
|
||||
mock_task.done.return_value = True
|
||||
adapter._poll_task = mock_task
|
||||
adapter._http_session = None
|
||||
adapter._bridge_process = None
|
||||
adapter._running = True
|
||||
adapter._session_lock_identity = None
|
||||
|
||||
await adapter.disconnect()
|
||||
|
||||
mock_task.cancel.assert_not_called()
|
||||
assert adapter._poll_task is None
|
||||
|
||||
@@ -1,42 +0,0 @@
|
||||
"""Tests for the top-level `./hermes` launcher script."""
|
||||
|
||||
import runpy
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def test_launcher_delegates_to_argparse_entrypoint(monkeypatch):
|
||||
"""`./hermes` should use `hermes_cli.main`, not the legacy Fire wrapper."""
|
||||
launcher_path = Path(__file__).resolve().parents[2] / "hermes"
|
||||
called = []
|
||||
|
||||
fake_main_module = types.ModuleType("hermes_cli.main")
|
||||
|
||||
def fake_main():
|
||||
called.append("hermes_cli.main")
|
||||
|
||||
fake_main_module.main = fake_main
|
||||
monkeypatch.setitem(sys.modules, "hermes_cli.main", fake_main_module)
|
||||
|
||||
fake_cli_module = types.ModuleType("cli")
|
||||
|
||||
def legacy_cli_main(*args, **kwargs):
|
||||
raise AssertionError("launcher should not import cli.main")
|
||||
|
||||
fake_cli_module.main = legacy_cli_main
|
||||
monkeypatch.setitem(sys.modules, "cli", fake_cli_module)
|
||||
|
||||
fake_fire_module = types.ModuleType("fire")
|
||||
|
||||
def legacy_fire(*args, **kwargs):
|
||||
raise AssertionError("launcher should not invoke fire.Fire")
|
||||
|
||||
fake_fire_module.Fire = legacy_fire
|
||||
monkeypatch.setitem(sys.modules, "fire", fake_fire_module)
|
||||
|
||||
monkeypatch.setattr(sys, "argv", [str(launcher_path), "gateway", "status"])
|
||||
|
||||
runpy.run_path(str(launcher_path), run_name="__main__")
|
||||
|
||||
assert called == ["hermes_cli.main"]
|
||||
@@ -1,283 +0,0 @@
|
||||
"""Tests for tool token estimation and curses_ui status_fn support."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
# tiktoken is not in core/[all] deps — skip estimation tests when unavailable
|
||||
_has_tiktoken = True
|
||||
try:
|
||||
import tiktoken # noqa: F401
|
||||
except ImportError:
|
||||
_has_tiktoken = False
|
||||
|
||||
_needs_tiktoken = pytest.mark.skipif(not _has_tiktoken, reason="tiktoken not installed")
|
||||
|
||||
|
||||
# ─── Token Estimation Tests ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
@_needs_tiktoken
|
||||
def test_estimate_tool_tokens_returns_positive_counts():
|
||||
"""_estimate_tool_tokens should return a non-empty dict with positive values."""
|
||||
from hermes_cli.tools_config import _estimate_tool_tokens, _tool_token_cache
|
||||
|
||||
# Clear cache to force fresh computation
|
||||
import hermes_cli.tools_config as tc
|
||||
tc._tool_token_cache = None
|
||||
|
||||
tokens = _estimate_tool_tokens()
|
||||
|
||||
assert isinstance(tokens, dict)
|
||||
assert len(tokens) > 0
|
||||
for name, count in tokens.items():
|
||||
assert isinstance(name, str)
|
||||
assert isinstance(count, int)
|
||||
assert count > 0, f"Tool {name} has non-positive token count: {count}"
|
||||
|
||||
|
||||
@_needs_tiktoken
|
||||
def test_estimate_tool_tokens_is_cached():
|
||||
"""Second call should return the same cached dict object."""
|
||||
import hermes_cli.tools_config as tc
|
||||
tc._tool_token_cache = None
|
||||
|
||||
first = tc._estimate_tool_tokens()
|
||||
second = tc._estimate_tool_tokens()
|
||||
|
||||
assert first is second
|
||||
|
||||
|
||||
def test_estimate_tool_tokens_returns_empty_when_tiktoken_unavailable(monkeypatch):
|
||||
"""Graceful degradation when tiktoken cannot be imported."""
|
||||
import hermes_cli.tools_config as tc
|
||||
tc._tool_token_cache = None
|
||||
|
||||
import builtins
|
||||
real_import = builtins.__import__
|
||||
|
||||
def mock_import(name, *args, **kwargs):
|
||||
if name == "tiktoken":
|
||||
raise ImportError("mocked")
|
||||
return real_import(name, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(builtins, "__import__", mock_import)
|
||||
|
||||
result = tc._estimate_tool_tokens()
|
||||
|
||||
assert result == {}
|
||||
|
||||
# Reset cache for other tests
|
||||
tc._tool_token_cache = None
|
||||
|
||||
|
||||
@_needs_tiktoken
|
||||
def test_estimate_tool_tokens_covers_known_tools():
|
||||
"""Should include schemas for well-known tools like terminal, web_search."""
|
||||
import hermes_cli.tools_config as tc
|
||||
tc._tool_token_cache = None
|
||||
|
||||
tokens = tc._estimate_tool_tokens()
|
||||
|
||||
# These tools should always be discoverable
|
||||
for expected in ("terminal", "web_search", "read_file"):
|
||||
assert expected in tokens, f"Expected {expected!r} in token estimates"
|
||||
|
||||
|
||||
# ─── Status Function Tests ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_prompt_toolset_checklist_passes_status_fn(monkeypatch):
|
||||
"""_prompt_toolset_checklist should pass a status_fn to curses_checklist."""
|
||||
import hermes_cli.tools_config as tc
|
||||
|
||||
captured_kwargs = {}
|
||||
|
||||
def fake_checklist(title, items, selected, *, cancel_returns=None, status_fn=None):
|
||||
captured_kwargs["status_fn"] = status_fn
|
||||
captured_kwargs["title"] = title
|
||||
return selected # Return pre-selected unchanged
|
||||
|
||||
monkeypatch.setattr("hermes_cli.curses_ui.curses_checklist", fake_checklist)
|
||||
|
||||
tc._prompt_toolset_checklist("CLI", {"web", "terminal"})
|
||||
|
||||
assert "status_fn" in captured_kwargs
|
||||
# If tiktoken is available, status_fn should be set
|
||||
tokens = tc._estimate_tool_tokens()
|
||||
if tokens:
|
||||
assert captured_kwargs["status_fn"] is not None
|
||||
|
||||
|
||||
def test_status_fn_returns_formatted_token_count(monkeypatch):
|
||||
"""The status_fn should return a human-readable token count string."""
|
||||
import hermes_cli.tools_config as tc
|
||||
from hermes_cli.tools_config import CONFIGURABLE_TOOLSETS
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_checklist(title, items, selected, *, cancel_returns=None, status_fn=None):
|
||||
captured["status_fn"] = status_fn
|
||||
return selected
|
||||
|
||||
monkeypatch.setattr("hermes_cli.curses_ui.curses_checklist", fake_checklist)
|
||||
|
||||
tc._prompt_toolset_checklist("CLI", {"web", "terminal"})
|
||||
|
||||
status_fn = captured.get("status_fn")
|
||||
if status_fn is None:
|
||||
pytest.skip("tiktoken unavailable; status_fn not created")
|
||||
|
||||
# Find the indices for web and terminal
|
||||
idx_map = {ts_key: i for i, (ts_key, _, _) in enumerate(CONFIGURABLE_TOOLSETS)}
|
||||
|
||||
# Call status_fn with web + terminal selected
|
||||
result = status_fn({idx_map["web"], idx_map["terminal"]})
|
||||
assert "tokens" in result
|
||||
assert "Est. tool context" in result
|
||||
|
||||
|
||||
def test_status_fn_deduplicates_overlapping_tools(monkeypatch):
|
||||
"""When toolsets overlap (browser includes web_search), tokens should not double-count."""
|
||||
import hermes_cli.tools_config as tc
|
||||
from hermes_cli.tools_config import CONFIGURABLE_TOOLSETS
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_checklist(title, items, selected, *, cancel_returns=None, status_fn=None):
|
||||
captured["status_fn"] = status_fn
|
||||
return selected
|
||||
|
||||
monkeypatch.setattr("hermes_cli.curses_ui.curses_checklist", fake_checklist)
|
||||
|
||||
tc._prompt_toolset_checklist("CLI", {"web"})
|
||||
|
||||
status_fn = captured.get("status_fn")
|
||||
if status_fn is None:
|
||||
pytest.skip("tiktoken unavailable; status_fn not created")
|
||||
|
||||
idx_map = {ts_key: i for i, (ts_key, _, _) in enumerate(CONFIGURABLE_TOOLSETS)}
|
||||
|
||||
# web alone
|
||||
web_only = status_fn({idx_map["web"]})
|
||||
# browser includes web_search, so browser + web should not double-count web_search
|
||||
browser_only = status_fn({idx_map["browser"]})
|
||||
both = status_fn({idx_map["web"], idx_map["browser"]})
|
||||
|
||||
# Extract numeric token counts from strings like "~8.3k tokens" or "~350 tokens"
|
||||
import re
|
||||
|
||||
def parse_tokens(s):
|
||||
m = re.search(r"~([\d.]+)k?\s+tokens", s)
|
||||
if not m:
|
||||
return 0
|
||||
val = float(m.group(1))
|
||||
if "k" in s[m.start():m.end()]:
|
||||
val *= 1000
|
||||
return val
|
||||
|
||||
web_tok = parse_tokens(web_only)
|
||||
browser_tok = parse_tokens(browser_only)
|
||||
both_tok = parse_tokens(both)
|
||||
|
||||
# Both together should be LESS than naive sum (due to web_search dedup)
|
||||
naive_sum = web_tok + browser_tok
|
||||
assert both_tok < naive_sum, (
|
||||
f"Expected deduplication: web({web_tok}) + browser({browser_tok}) = {naive_sum} "
|
||||
f"but combined = {both_tok}"
|
||||
)
|
||||
|
||||
|
||||
def test_status_fn_empty_selection():
|
||||
"""Status function with no tools selected should return ~0 tokens."""
|
||||
import hermes_cli.tools_config as tc
|
||||
|
||||
tc._tool_token_cache = None
|
||||
tokens = tc._estimate_tool_tokens()
|
||||
if not tokens:
|
||||
pytest.skip("tiktoken unavailable")
|
||||
|
||||
from hermes_cli.tools_config import CONFIGURABLE_TOOLSETS
|
||||
from toolsets import resolve_toolset
|
||||
|
||||
ts_keys = [ts_key for ts_key, _, _ in CONFIGURABLE_TOOLSETS]
|
||||
|
||||
def status_fn(chosen: set) -> str:
|
||||
all_tools: set = set()
|
||||
for idx in chosen:
|
||||
all_tools.update(resolve_toolset(ts_keys[idx]))
|
||||
total = sum(tokens.get(name, 0) for name in all_tools)
|
||||
if total >= 1000:
|
||||
return f"Est. tool context: ~{total / 1000:.1f}k tokens"
|
||||
return f"Est. tool context: ~{total} tokens"
|
||||
|
||||
result = status_fn(set())
|
||||
assert "~0 tokens" in result
|
||||
|
||||
|
||||
# ─── Curses UI Status Bar Tests ──────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_curses_checklist_numbered_fallback_shows_status(monkeypatch, capsys):
|
||||
"""The numbered fallback should print the status_fn output."""
|
||||
from hermes_cli.curses_ui import _numbered_fallback
|
||||
|
||||
def my_status(chosen):
|
||||
return f"Selected {len(chosen)} items"
|
||||
|
||||
# Simulate user pressing Enter immediately (empty input → confirm)
|
||||
monkeypatch.setattr("builtins.input", lambda _prompt="": "")
|
||||
|
||||
result = _numbered_fallback(
|
||||
"Test title",
|
||||
["Item A", "Item B", "Item C"],
|
||||
{0, 2},
|
||||
{0, 2},
|
||||
status_fn=my_status,
|
||||
)
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "Selected 2 items" in captured.out
|
||||
assert result == {0, 2}
|
||||
|
||||
|
||||
def test_curses_checklist_numbered_fallback_without_status(monkeypatch, capsys):
|
||||
"""The numbered fallback should work fine without status_fn."""
|
||||
from hermes_cli.curses_ui import _numbered_fallback
|
||||
|
||||
monkeypatch.setattr("builtins.input", lambda _prompt="": "")
|
||||
|
||||
result = _numbered_fallback(
|
||||
"Test title",
|
||||
["Item A", "Item B"],
|
||||
{0},
|
||||
{0},
|
||||
)
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "Est. tool context" not in captured.out
|
||||
assert result == {0}
|
||||
|
||||
|
||||
# ─── Registry get_schema Tests ───────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_registry_get_schema_returns_schema():
|
||||
"""registry.get_schema() should return a tool's schema dict."""
|
||||
from tools.registry import registry
|
||||
|
||||
# Import to trigger discovery
|
||||
import model_tools # noqa: F401
|
||||
|
||||
schema = registry.get_schema("terminal")
|
||||
assert schema is not None
|
||||
assert "name" in schema
|
||||
assert schema["name"] == "terminal"
|
||||
assert "parameters" in schema
|
||||
|
||||
|
||||
def test_registry_get_schema_returns_none_for_unknown():
|
||||
"""registry.get_schema() should return None for unknown tools."""
|
||||
from tools.registry import registry
|
||||
|
||||
assert registry.get_schema("nonexistent_tool_xyz") is None
|
||||
@@ -1,427 +0,0 @@
|
||||
"""Tests for optional-skills/productivity/memento-flashcards/scripts/memento_cards.py"""
|
||||
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
# Add the scripts dir so we can import the module directly
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parents[2] / "optional-skills" / "productivity" / "memento-flashcards" / "scripts"
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
import memento_cards
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolated_data(tmp_path, monkeypatch):
|
||||
"""Redirect card storage to a temp directory for every test."""
|
||||
data_dir = tmp_path / "data"
|
||||
data_dir.mkdir()
|
||||
monkeypatch.setattr(memento_cards, "DATA_DIR", data_dir)
|
||||
monkeypatch.setattr(memento_cards, "CARDS_FILE", data_dir / "cards.json")
|
||||
return data_dir
|
||||
|
||||
|
||||
def _run(capsys, argv: list[str]) -> dict:
|
||||
"""Run main() with given argv and return parsed JSON output."""
|
||||
with mock.patch("sys.argv", ["memento_cards"] + argv):
|
||||
memento_cards.main()
|
||||
captured = capsys.readouterr()
|
||||
return json.loads(captured.out)
|
||||
|
||||
|
||||
# ── Add / List / Delete ──────────────────────────────────────────────────────
|
||||
|
||||
class TestCardCRUD:
|
||||
def test_add_creates_card(self, capsys):
|
||||
result = _run(capsys, ["add", "--question", "What is 2+2?", "--answer", "4", "--collection", "Math"])
|
||||
assert result["ok"] is True
|
||||
card = result["card"]
|
||||
assert card["question"] == "What is 2+2?"
|
||||
assert card["answer"] == "4"
|
||||
assert card["collection"] == "Math"
|
||||
assert card["status"] == "learning"
|
||||
assert card["ease_streak"] == 0
|
||||
uuid.UUID(card["id"]) # validates it's a real UUID
|
||||
|
||||
def test_add_default_collection(self, capsys):
|
||||
result = _run(capsys, ["add", "--question", "Q?", "--answer", "A"])
|
||||
assert result["card"]["collection"] == "General"
|
||||
|
||||
def test_list_all(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q1", "--answer", "A1", "--collection", "C1"])
|
||||
_run(capsys, ["add", "--question", "Q2", "--answer", "A2", "--collection", "C2"])
|
||||
result = _run(capsys, ["list"])
|
||||
assert result["count"] == 2
|
||||
|
||||
def test_list_by_collection(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q1", "--answer", "A1", "--collection", "C1"])
|
||||
_run(capsys, ["add", "--question", "Q2", "--answer", "A2", "--collection", "C2"])
|
||||
result = _run(capsys, ["list", "--collection", "C1"])
|
||||
assert result["count"] == 1
|
||||
assert result["cards"][0]["collection"] == "C1"
|
||||
|
||||
def test_list_by_status(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q1", "--answer", "A1"])
|
||||
result = _run(capsys, ["list", "--status", "learning"])
|
||||
assert result["count"] == 1
|
||||
result = _run(capsys, ["list", "--status", "retired"])
|
||||
assert result["count"] == 0
|
||||
|
||||
def test_delete_card(self, capsys):
|
||||
result = _run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = result["card"]["id"]
|
||||
del_result = _run(capsys, ["delete", "--id", card_id])
|
||||
assert del_result["ok"] is True
|
||||
assert del_result["deleted"] == card_id
|
||||
# Verify gone
|
||||
list_result = _run(capsys, ["list"])
|
||||
assert list_result["count"] == 0
|
||||
|
||||
def test_delete_nonexistent(self, capsys):
|
||||
with pytest.raises(SystemExit):
|
||||
_run(capsys, ["delete", "--id", "nonexistent"])
|
||||
|
||||
def test_delete_collection(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q1", "--answer", "A1", "--collection", "ToDelete"])
|
||||
_run(capsys, ["add", "--question", "Q2", "--answer", "A2", "--collection", "ToDelete"])
|
||||
_run(capsys, ["add", "--question", "Q3", "--answer", "A3", "--collection", "Keep"])
|
||||
result = _run(capsys, ["delete-collection", "--collection", "ToDelete"])
|
||||
assert result["ok"] is True
|
||||
assert result["deleted_count"] == 2
|
||||
list_result = _run(capsys, ["list"])
|
||||
assert list_result["count"] == 1
|
||||
assert list_result["cards"][0]["collection"] == "Keep"
|
||||
|
||||
|
||||
# ── Due Filtering ────────────────────────────────────────────────────────────
|
||||
|
||||
class TestDueFiltering:
|
||||
def test_new_card_is_due(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
result = _run(capsys, ["due"])
|
||||
assert result["count"] == 1
|
||||
|
||||
def test_future_card_not_due(self, capsys, monkeypatch):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
# Rate it good (pushes next_review_at to +3 days)
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
_run(capsys, ["rate", "--id", card_id, "--rating", "good"])
|
||||
result = _run(capsys, ["due"])
|
||||
assert result["count"] == 0
|
||||
|
||||
def test_retired_card_not_due(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
_run(capsys, ["rate", "--id", card_id, "--rating", "retire"])
|
||||
result = _run(capsys, ["due"])
|
||||
assert result["count"] == 0
|
||||
|
||||
def test_due_with_collection_filter(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q1", "--answer", "A1", "--collection", "C1"])
|
||||
_run(capsys, ["add", "--question", "Q2", "--answer", "A2", "--collection", "C2"])
|
||||
result = _run(capsys, ["due", "--collection", "C1"])
|
||||
assert result["count"] == 1
|
||||
assert result["cards"][0]["collection"] == "C1"
|
||||
|
||||
|
||||
# ── Rating and Rescheduling ──────────────────────────────────────────────────
|
||||
|
||||
class TestRating:
|
||||
def test_hard_adds_1_day(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
before = datetime.now(timezone.utc)
|
||||
result = _run(capsys, ["rate", "--id", card_id, "--rating", "hard"])
|
||||
after = datetime.now(timezone.utc)
|
||||
next_review = datetime.fromisoformat(result["card"]["next_review_at"])
|
||||
assert before + timedelta(days=1) <= next_review <= after + timedelta(days=1)
|
||||
assert result["card"]["ease_streak"] == 0
|
||||
|
||||
def test_good_adds_3_days(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
before = datetime.now(timezone.utc)
|
||||
result = _run(capsys, ["rate", "--id", card_id, "--rating", "good"])
|
||||
next_review = datetime.fromisoformat(result["card"]["next_review_at"])
|
||||
assert next_review >= before + timedelta(days=3)
|
||||
assert result["card"]["ease_streak"] == 0
|
||||
|
||||
def test_easy_adds_7_days_and_increments_streak(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
result = _run(capsys, ["rate", "--id", card_id, "--rating", "easy"])
|
||||
assert result["card"]["ease_streak"] == 1
|
||||
assert result["card"]["status"] == "learning"
|
||||
|
||||
def test_retire_sets_retired(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
result = _run(capsys, ["rate", "--id", card_id, "--rating", "retire"])
|
||||
assert result["card"]["status"] == "retired"
|
||||
assert result["card"]["ease_streak"] == 0
|
||||
|
||||
def test_auto_retire_after_3_easys(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
|
||||
# Force card to be due by manipulating next_review_at through rate
|
||||
for i in range(3):
|
||||
# Load and directly set next_review_at to now so it's ratable
|
||||
data = memento_cards._load()
|
||||
for c in data["cards"]:
|
||||
if c["id"] == card_id:
|
||||
c["next_review_at"] = memento_cards._iso(memento_cards._now())
|
||||
memento_cards._save(data)
|
||||
|
||||
result = _run(capsys, ["rate", "--id", card_id, "--rating", "easy"])
|
||||
|
||||
assert result["card"]["ease_streak"] == 3
|
||||
assert result["card"]["status"] == "retired"
|
||||
|
||||
def test_hard_resets_ease_streak(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
|
||||
# Easy twice
|
||||
for _ in range(2):
|
||||
data = memento_cards._load()
|
||||
for c in data["cards"]:
|
||||
if c["id"] == card_id:
|
||||
c["next_review_at"] = memento_cards._iso(memento_cards._now())
|
||||
memento_cards._save(data)
|
||||
_run(capsys, ["rate", "--id", card_id, "--rating", "easy"])
|
||||
|
||||
# Verify streak is 2
|
||||
check = _run(capsys, ["list"])
|
||||
assert check["cards"][0]["ease_streak"] == 2
|
||||
|
||||
# Hard resets
|
||||
data = memento_cards._load()
|
||||
for c in data["cards"]:
|
||||
if c["id"] == card_id:
|
||||
c["next_review_at"] = memento_cards._iso(memento_cards._now())
|
||||
memento_cards._save(data)
|
||||
result = _run(capsys, ["rate", "--id", card_id, "--rating", "hard"])
|
||||
assert result["card"]["ease_streak"] == 0
|
||||
assert result["card"]["status"] == "learning"
|
||||
|
||||
def test_rate_nonexistent_card(self, capsys):
|
||||
with pytest.raises(SystemExit):
|
||||
_run(capsys, ["rate", "--id", "nonexistent", "--rating", "easy"])
|
||||
|
||||
|
||||
# ── CSV Export/Import ────────────────────────────────────────────────────────
|
||||
|
||||
class TestCSV:
|
||||
def test_export_import_roundtrip(self, capsys, tmp_path):
|
||||
_run(capsys, ["add", "--question", "Q1", "--answer", "A1", "--collection", "C1"])
|
||||
_run(capsys, ["add", "--question", "Q2", "--answer", "A2", "--collection", "C2"])
|
||||
|
||||
csv_path = str(tmp_path / "export.csv")
|
||||
result = _run(capsys, ["export", "--output", csv_path])
|
||||
assert result["ok"] is True
|
||||
assert result["exported"] == 2
|
||||
|
||||
# Verify CSV content
|
||||
with open(csv_path, "r") as f:
|
||||
reader = csv.reader(f)
|
||||
rows = list(reader)
|
||||
assert len(rows) == 2
|
||||
assert rows[0] == ["Q1", "A1", "C1"]
|
||||
assert rows[1] == ["Q2", "A2", "C2"]
|
||||
|
||||
# Delete all and reimport
|
||||
data = memento_cards._load()
|
||||
data["cards"] = []
|
||||
memento_cards._save(data)
|
||||
|
||||
result = _run(capsys, ["import", "--file", csv_path, "--collection", "Fallback"])
|
||||
assert result["ok"] is True
|
||||
assert result["imported"] == 2
|
||||
|
||||
# Verify imported cards use CSV collection column
|
||||
list_result = _run(capsys, ["list"])
|
||||
collections = {c["collection"] for c in list_result["cards"]}
|
||||
assert collections == {"C1", "C2"}
|
||||
|
||||
def test_import_without_collection_column(self, capsys, tmp_path):
|
||||
csv_path = str(tmp_path / "no_col.csv")
|
||||
with open(csv_path, "w", newline="") as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(["Q1", "A1"])
|
||||
writer.writerow(["Q2", "A2"])
|
||||
|
||||
result = _run(capsys, ["import", "--file", csv_path, "--collection", "MyDeck"])
|
||||
assert result["imported"] == 2
|
||||
|
||||
list_result = _run(capsys, ["list"])
|
||||
assert all(c["collection"] == "MyDeck" for c in list_result["cards"])
|
||||
|
||||
def test_import_skips_empty_rows(self, capsys, tmp_path):
|
||||
csv_path = str(tmp_path / "sparse.csv")
|
||||
with open(csv_path, "w", newline="") as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(["Q1", "A1"])
|
||||
writer.writerow(["", ""]) # empty
|
||||
writer.writerow(["Q2"]) # only one column
|
||||
writer.writerow(["Q3", "A3"])
|
||||
|
||||
result = _run(capsys, ["import", "--file", csv_path, "--collection", "Test"])
|
||||
assert result["imported"] == 2
|
||||
|
||||
def test_import_nonexistent_file(self, capsys, tmp_path):
|
||||
with pytest.raises(SystemExit):
|
||||
_run(capsys, ["import", "--file", str(tmp_path / "nope.csv"), "--collection", "X"])
|
||||
|
||||
|
||||
# ── Quiz Batch Add ───────────────────────────────────────────────────────────
|
||||
|
||||
class TestQuizBatchAdd:
|
||||
def test_add_quiz_creates_cards(self, capsys):
|
||||
questions = json.dumps([
|
||||
{"question": "Q1?", "answer": "A1"},
|
||||
{"question": "Q2?", "answer": "A2"},
|
||||
])
|
||||
result = _run(capsys, ["add-quiz", "--video-id", "abc123", "--questions", questions, "--collection", "Quiz - Test"])
|
||||
assert result["ok"] is True
|
||||
assert result["created_count"] == 2
|
||||
for card in result["cards"]:
|
||||
assert card["video_id"] == "abc123"
|
||||
assert card["collection"] == "Quiz - Test"
|
||||
|
||||
def test_add_quiz_deduplicates_by_video_id(self, capsys):
|
||||
questions = json.dumps([{"question": "Q?", "answer": "A"}])
|
||||
_run(capsys, ["add-quiz", "--video-id", "dup1", "--questions", questions])
|
||||
result = _run(capsys, ["add-quiz", "--video-id", "dup1", "--questions", questions])
|
||||
assert result["ok"] is True
|
||||
assert result["skipped"] is True
|
||||
assert result["reason"] == "duplicate_video_id"
|
||||
# Only 1 card total (not 2)
|
||||
list_result = _run(capsys, ["list"])
|
||||
assert list_result["count"] == 1
|
||||
|
||||
def test_add_quiz_invalid_json(self, capsys):
|
||||
with pytest.raises(SystemExit):
|
||||
_run(capsys, ["add-quiz", "--video-id", "x", "--questions", "not json"])
|
||||
|
||||
|
||||
# ── Statistics ───────────────────────────────────────────────────────────────
|
||||
|
||||
class TestStats:
|
||||
def test_stats_empty(self, capsys):
|
||||
result = _run(capsys, ["stats"])
|
||||
assert result["total"] == 0
|
||||
assert result["learning"] == 0
|
||||
assert result["retired"] == 0
|
||||
assert result["due_now"] == 0
|
||||
|
||||
def test_stats_counts(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q1", "--answer", "A1", "--collection", "C1"])
|
||||
_run(capsys, ["add", "--question", "Q2", "--answer", "A2", "--collection", "C1"])
|
||||
_run(capsys, ["add", "--question", "Q3", "--answer", "A3", "--collection", "C2"])
|
||||
|
||||
# Retire one
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
_run(capsys, ["rate", "--id", card_id, "--rating", "retire"])
|
||||
|
||||
result = _run(capsys, ["stats"])
|
||||
assert result["total"] == 3
|
||||
assert result["learning"] == 2
|
||||
assert result["retired"] == 1
|
||||
assert result["due_now"] == 2 # 2 learning cards still due
|
||||
assert result["collections"] == {"C1": 2, "C2": 1}
|
||||
|
||||
|
||||
# ── Edge Cases ───────────────────────────────────────────────────────────────
|
||||
|
||||
class TestEdgeCases:
|
||||
def test_empty_deck_operations(self, capsys):
|
||||
"""Operations on empty deck shouldn't crash."""
|
||||
result = _run(capsys, ["due"])
|
||||
assert result["count"] == 0
|
||||
result = _run(capsys, ["list"])
|
||||
assert result["count"] == 0
|
||||
result = _run(capsys, ["stats"])
|
||||
assert result["total"] == 0
|
||||
|
||||
def test_corrupt_json_recovery(self, capsys):
|
||||
"""Corrupt JSON file should be treated as empty."""
|
||||
memento_cards.DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
with open(memento_cards.CARDS_FILE, "w") as f:
|
||||
f.write("{corrupted json...")
|
||||
result = _run(capsys, ["list"])
|
||||
assert result["count"] == 0
|
||||
# Can still add
|
||||
result = _run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
assert result["ok"] is True
|
||||
|
||||
def test_missing_cards_key_recovery(self, capsys):
|
||||
"""JSON without 'cards' key should be treated as empty."""
|
||||
memento_cards.DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
with open(memento_cards.CARDS_FILE, "w") as f:
|
||||
json.dump({"version": 1}, f)
|
||||
result = _run(capsys, ["list"])
|
||||
assert result["count"] == 0
|
||||
|
||||
def test_atomic_write_creates_dir(self, capsys):
|
||||
"""Data dir is created automatically if missing."""
|
||||
import shutil
|
||||
if memento_cards.DATA_DIR.exists():
|
||||
shutil.rmtree(memento_cards.DATA_DIR)
|
||||
result = _run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
assert result["ok"] is True
|
||||
assert memento_cards.CARDS_FILE.exists()
|
||||
|
||||
def test_delete_collection_empty(self, capsys):
|
||||
"""Deleting a nonexistent collection succeeds with 0 deleted."""
|
||||
result = _run(capsys, ["delete-collection", "--collection", "Nope"])
|
||||
assert result["ok"] is True
|
||||
assert result["deleted_count"] == 0
|
||||
|
||||
|
||||
# ── User Answer Tracking ────────────────────────────────────────────────────
|
||||
|
||||
class TestUserAnswer:
|
||||
def test_rate_stores_user_answer(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
result = _run(capsys, ["rate", "--id", card_id, "--rating", "easy",
|
||||
"--user-answer", "my answer"])
|
||||
assert result["card"]["last_user_answer"] == "my answer"
|
||||
|
||||
def test_rate_without_user_answer_keeps_null(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
result = _run(capsys, ["rate", "--id", card_id, "--rating", "easy"])
|
||||
assert result["card"]["last_user_answer"] is None
|
||||
|
||||
def test_new_card_has_last_user_answer_null(self, capsys):
|
||||
result = _run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
assert result["card"]["last_user_answer"] is None
|
||||
|
||||
def test_user_answer_persists_in_list(self, capsys):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
_run(capsys, ["rate", "--id", card_id, "--rating", "easy",
|
||||
"--user-answer", "my answer"])
|
||||
result = _run(capsys, ["list"])
|
||||
assert result["cards"][0]["last_user_answer"] == "my answer"
|
||||
|
||||
def test_export_excludes_user_answer(self, capsys, tmp_path):
|
||||
_run(capsys, ["add", "--question", "Q", "--answer", "A"])
|
||||
card_id = _run(capsys, ["list"])["cards"][0]["id"]
|
||||
_run(capsys, ["rate", "--id", card_id, "--rating", "easy",
|
||||
"--user-answer", "my answer"])
|
||||
csv_path = str(tmp_path / "export.csv")
|
||||
_run(capsys, ["export", "--output", csv_path])
|
||||
with open(csv_path) as f:
|
||||
rows = list(csv.reader(f))
|
||||
# CSV stays 3-column (question, answer, collection) — user_answer is internal only
|
||||
assert len(rows[0]) == 3
|
||||
@@ -1,128 +0,0 @@
|
||||
"""Tests for optional-skills/productivity/memento-flashcards/scripts/youtube_quiz.py"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parents[2] / "optional-skills" / "productivity" / "memento-flashcards" / "scripts"
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
import youtube_quiz
|
||||
|
||||
|
||||
def _run(capsys, argv: list[str]) -> dict:
|
||||
"""Run main() with given argv and return parsed JSON output."""
|
||||
with mock.patch("sys.argv", ["youtube_quiz"] + argv):
|
||||
youtube_quiz.main()
|
||||
captured = capsys.readouterr()
|
||||
return json.loads(captured.out)
|
||||
|
||||
|
||||
class TestNormalizeSegments:
|
||||
def test_basic(self):
|
||||
segments = [{"text": "hello "}, {"text": " world"}]
|
||||
assert youtube_quiz._normalize_segments(segments) == "hello world"
|
||||
|
||||
def test_empty_segments(self):
|
||||
assert youtube_quiz._normalize_segments([]) == ""
|
||||
|
||||
def test_whitespace_only(self):
|
||||
assert youtube_quiz._normalize_segments([{"text": " "}, {"text": " "}]) == ""
|
||||
|
||||
def test_collapses_multiple_spaces(self):
|
||||
segments = [{"text": "a b"}, {"text": "c d"}]
|
||||
assert youtube_quiz._normalize_segments(segments) == "a b c d"
|
||||
|
||||
|
||||
class TestFetchMissingDependency:
|
||||
def test_missing_youtube_transcript_api(self, capsys, monkeypatch):
|
||||
"""When youtube-transcript-api is not installed, report the error."""
|
||||
import builtins
|
||||
real_import = builtins.__import__
|
||||
|
||||
def mock_import(name, *args, **kwargs):
|
||||
if name == "youtube_transcript_api":
|
||||
raise ImportError("No module named 'youtube_transcript_api'")
|
||||
return real_import(name, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(builtins, "__import__", mock_import)
|
||||
|
||||
with pytest.raises(SystemExit) as exc_info:
|
||||
_run(capsys, ["fetch", "test123"])
|
||||
|
||||
captured = capsys.readouterr()
|
||||
result = json.loads(captured.out)
|
||||
assert result["ok"] is False
|
||||
assert result["error"] == "missing_dependency"
|
||||
assert "pip install" in result["message"]
|
||||
|
||||
|
||||
class TestFetchWithMockedAPI:
|
||||
def _make_mock_module(self, segments=None, raise_exc=None):
|
||||
"""Create a mock youtube_transcript_api module."""
|
||||
mock_module = mock.MagicMock()
|
||||
|
||||
mock_api_instance = mock.MagicMock()
|
||||
mock_module.YouTubeTranscriptApi.return_value = mock_api_instance
|
||||
|
||||
if raise_exc:
|
||||
mock_api_instance.fetch.side_effect = raise_exc
|
||||
else:
|
||||
raw_data = segments or [{"text": "Hello world"}]
|
||||
result = mock.MagicMock()
|
||||
result.to_raw_data.return_value = raw_data
|
||||
mock_api_instance.fetch.return_value = result
|
||||
|
||||
return mock_module
|
||||
|
||||
def test_successful_fetch(self, capsys):
|
||||
mock_mod = self._make_mock_module(
|
||||
segments=[{"text": "This is a test"}, {"text": "transcript segment"}]
|
||||
)
|
||||
with mock.patch.dict("sys.modules", {"youtube_transcript_api": mock_mod}):
|
||||
result = _run(capsys, ["fetch", "abc123"])
|
||||
|
||||
assert result["ok"] is True
|
||||
assert result["video_id"] == "abc123"
|
||||
assert "This is a test" in result["transcript"]
|
||||
assert "transcript segment" in result["transcript"]
|
||||
|
||||
def test_fetch_error(self, capsys):
|
||||
mock_mod = self._make_mock_module(raise_exc=Exception("Video unavailable"))
|
||||
with mock.patch.dict("sys.modules", {"youtube_transcript_api": mock_mod}):
|
||||
with pytest.raises(SystemExit):
|
||||
_run(capsys, ["fetch", "bad_id"])
|
||||
|
||||
captured = capsys.readouterr()
|
||||
result = json.loads(captured.out)
|
||||
assert result["ok"] is False
|
||||
assert result["error"] == "transcript_unavailable"
|
||||
|
||||
def test_empty_transcript(self, capsys):
|
||||
mock_mod = self._make_mock_module(segments=[{"text": ""}, {"text": " "}])
|
||||
with mock.patch.dict("sys.modules", {"youtube_transcript_api": mock_mod}):
|
||||
with pytest.raises(SystemExit):
|
||||
_run(capsys, ["fetch", "empty_vid"])
|
||||
|
||||
captured = capsys.readouterr()
|
||||
result = json.loads(captured.out)
|
||||
assert result["ok"] is False
|
||||
assert result["error"] == "empty_transcript"
|
||||
|
||||
def test_segments_without_to_raw_data(self, capsys):
|
||||
"""Handle plain list segments (no to_raw_data method)."""
|
||||
mock_mod = mock.MagicMock()
|
||||
mock_api = mock.MagicMock()
|
||||
mock_mod.YouTubeTranscriptApi.return_value = mock_api
|
||||
# Return a plain list (no to_raw_data attribute)
|
||||
mock_api.fetch.return_value = [{"text": "plain list"}]
|
||||
|
||||
with mock.patch.dict("sys.modules", {"youtube_transcript_api": mock_mod}):
|
||||
result = _run(capsys, ["fetch", "plain123"])
|
||||
|
||||
assert result["ok"] is True
|
||||
assert result["transcript"] == "plain list"
|
||||
@@ -266,8 +266,7 @@ class TestResolveProvider:
|
||||
|
||||
def test_auto_does_not_select_copilot_from_github_token(self, monkeypatch):
|
||||
monkeypatch.setenv("GITHUB_TOKEN", "gh-test-token")
|
||||
with pytest.raises(AuthError, match="No inference provider configured"):
|
||||
resolve_provider("auto")
|
||||
assert resolve_provider("auto") == "openrouter"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
|
||||
@@ -214,9 +214,8 @@ class TestStatusBarWidthSource:
|
||||
frags = cli_obj._get_status_bar_fragments()
|
||||
|
||||
total_text = "".join(text for _, text in frags)
|
||||
display_width = cli_obj._status_bar_display_width(total_text)
|
||||
assert display_width <= width + 4, ( # +4 for minor padding chars
|
||||
f"At width={width}, fragment total {display_width} cells overflows "
|
||||
assert len(total_text) <= width + 4, ( # +4 for minor padding chars
|
||||
f"At width={width}, fragment total {len(total_text)} chars overflows "
|
||||
f"({total_text!r})"
|
||||
)
|
||||
|
||||
|
||||
@@ -60,43 +60,34 @@ class TestToolsSlashList:
|
||||
|
||||
class TestToolsSlashDisableWithReset:
|
||||
|
||||
def test_disable_applies_directly_and_resets_session(self):
|
||||
"""Disable applies immediately (no confirmation prompt) and resets session."""
|
||||
def test_disable_confirms_then_resets_session(self):
|
||||
cli_obj = _make_cli(["web", "memory"])
|
||||
with patch("hermes_cli.tools_config.load_config",
|
||||
return_value={"platform_toolsets": {"cli": ["web", "memory"]}}), \
|
||||
patch("hermes_cli.tools_config.save_config"), \
|
||||
patch("hermes_cli.tools_config._get_platform_tools", return_value={"memory"}), \
|
||||
patch("hermes_cli.config.load_config", return_value={}), \
|
||||
patch.object(cli_obj, "new_session") as mock_reset:
|
||||
patch.object(cli_obj, "new_session") as mock_reset, \
|
||||
patch("builtins.input", return_value="y"):
|
||||
cli_obj._handle_tools_command("/tools disable web")
|
||||
mock_reset.assert_called_once()
|
||||
assert "web" not in cli_obj.enabled_toolsets
|
||||
|
||||
def test_disable_does_not_prompt_for_confirmation(self):
|
||||
"""Disable no longer uses input() — it applies directly."""
|
||||
def test_disable_cancelled_does_not_reset(self):
|
||||
cli_obj = _make_cli(["web", "memory"])
|
||||
with patch("hermes_cli.tools_config.load_config",
|
||||
return_value={"platform_toolsets": {"cli": ["web", "memory"]}}), \
|
||||
patch("hermes_cli.tools_config.save_config"), \
|
||||
patch("hermes_cli.tools_config._get_platform_tools", return_value={"memory"}), \
|
||||
patch("hermes_cli.config.load_config", return_value={}), \
|
||||
patch.object(cli_obj, "new_session"), \
|
||||
patch("builtins.input") as mock_input:
|
||||
with patch.object(cli_obj, "new_session") as mock_reset, \
|
||||
patch("builtins.input", return_value="n"):
|
||||
cli_obj._handle_tools_command("/tools disable web")
|
||||
mock_input.assert_not_called()
|
||||
mock_reset.assert_not_called()
|
||||
# Toolsets unchanged
|
||||
assert cli_obj.enabled_toolsets == {"web", "memory"}
|
||||
|
||||
def test_disable_always_resets_session(self):
|
||||
"""Even without a confirmation prompt, disable always resets the session."""
|
||||
def test_disable_eof_cancels(self):
|
||||
cli_obj = _make_cli(["web", "memory"])
|
||||
with patch("hermes_cli.tools_config.load_config",
|
||||
return_value={"platform_toolsets": {"cli": ["web", "memory"]}}), \
|
||||
patch("hermes_cli.tools_config.save_config"), \
|
||||
patch("hermes_cli.tools_config._get_platform_tools", return_value={"memory"}), \
|
||||
patch("hermes_cli.config.load_config", return_value={}), \
|
||||
patch.object(cli_obj, "new_session") as mock_reset:
|
||||
with patch.object(cli_obj, "new_session") as mock_reset, \
|
||||
patch("builtins.input", side_effect=EOFError):
|
||||
cli_obj._handle_tools_command("/tools disable web")
|
||||
mock_reset.assert_called_once()
|
||||
mock_reset.assert_not_called()
|
||||
|
||||
def test_disable_missing_name_prints_usage(self, capsys):
|
||||
cli_obj = _make_cli()
|
||||
@@ -110,15 +101,15 @@ class TestToolsSlashDisableWithReset:
|
||||
|
||||
class TestToolsSlashEnableWithReset:
|
||||
|
||||
def test_enable_applies_directly_and_resets_session(self):
|
||||
"""Enable applies immediately (no confirmation prompt) and resets session."""
|
||||
def test_enable_confirms_then_resets_session(self):
|
||||
cli_obj = _make_cli(["memory"])
|
||||
with patch("hermes_cli.tools_config.load_config",
|
||||
return_value={"platform_toolsets": {"cli": ["memory"]}}), \
|
||||
patch("hermes_cli.tools_config.save_config"), \
|
||||
patch("hermes_cli.tools_config._get_platform_tools", return_value={"memory", "web"}), \
|
||||
patch("hermes_cli.config.load_config", return_value={}), \
|
||||
patch.object(cli_obj, "new_session") as mock_reset:
|
||||
patch.object(cli_obj, "new_session") as mock_reset, \
|
||||
patch("builtins.input", return_value="y"):
|
||||
cli_obj._handle_tools_command("/tools enable web")
|
||||
mock_reset.assert_called_once()
|
||||
assert "web" in cli_obj.enabled_toolsets
|
||||
|
||||
@@ -33,7 +33,6 @@ def test_get_codex_model_ids_prioritizes_default_and_cache(tmp_path, monkeypatch
|
||||
assert "gpt-5.3-codex" in models
|
||||
# Non-codex-suffixed models are included when the cache says they're available
|
||||
assert "gpt-5.4" in models
|
||||
assert "gpt-5.4-mini" in models
|
||||
assert "gpt-5-hidden-codex" not in models
|
||||
|
||||
|
||||
@@ -65,7 +64,7 @@ def test_get_codex_model_ids_adds_forward_compat_models_from_templates(monkeypat
|
||||
|
||||
models = get_codex_model_ids(access_token="codex-access-token")
|
||||
|
||||
assert models == ["gpt-5.2-codex", "gpt-5.4-mini", "gpt-5.4", "gpt-5.3-codex", "gpt-5.3-codex-spark"]
|
||||
assert models == ["gpt-5.2-codex", "gpt-5.3-codex", "gpt-5.4", "gpt-5.3-codex-spark"]
|
||||
|
||||
|
||||
def test_model_command_uses_runtime_access_token_for_codex_list(monkeypatch):
|
||||
|
||||
@@ -25,8 +25,6 @@ def _make_agent_with_compressor() -> AIAgent:
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o",
|
||||
}
|
||||
agent._fallback_chain = [agent._fallback_model]
|
||||
agent._fallback_index = 0
|
||||
|
||||
# Context compressor with primary model values
|
||||
compressor = ContextCompressor(
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -150,11 +150,11 @@ class TestPluginsCommandDispatch:
|
||||
plugins_command(args)
|
||||
mock_list.assert_called_once()
|
||||
|
||||
@patch("hermes_cli.plugins_cmd.cmd_toggle")
|
||||
def test_none_falls_through_to_toggle(self, mock_toggle):
|
||||
@patch("hermes_cli.plugins_cmd.cmd_list")
|
||||
def test_none_falls_through_to_list(self, mock_list):
|
||||
args = self._make_args(None)
|
||||
plugins_command(args)
|
||||
mock_toggle.assert_called_once()
|
||||
mock_list.assert_called_once()
|
||||
|
||||
@patch("hermes_cli.plugins_cmd.cmd_install")
|
||||
def test_install_dispatches(self, mock_install):
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
"""Regression tests for packaging metadata in pyproject.toml."""
|
||||
|
||||
from pathlib import Path
|
||||
import tomllib
|
||||
|
||||
|
||||
def _load_optional_dependencies():
|
||||
pyproject_path = Path(__file__).resolve().parents[1] / "pyproject.toml"
|
||||
with pyproject_path.open("rb") as handle:
|
||||
project = tomllib.load(handle)["project"]
|
||||
return project["optional-dependencies"]
|
||||
|
||||
|
||||
def test_all_extra_includes_matrix_dependency():
|
||||
optional_dependencies = _load_optional_dependencies()
|
||||
|
||||
assert "matrix" in optional_dependencies
|
||||
assert "hermes-agent[matrix]" in optional_dependencies["all"]
|
||||
@@ -1,156 +0,0 @@
|
||||
"""Tests for ordered provider fallback chain (salvage of PR #1761).
|
||||
|
||||
Extends the single-fallback tests in test_fallback_model.py to cover
|
||||
the new list-based ``fallback_providers`` config format and chain
|
||||
advancement through multiple providers.
|
||||
"""
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from run_agent import AIAgent
|
||||
|
||||
|
||||
def _make_agent(fallback_model=None):
|
||||
"""Create a minimal AIAgent with optional fallback config."""
|
||||
with (
|
||||
patch("run_agent.get_tool_definitions", return_value=[]),
|
||||
patch("run_agent.check_toolset_requirements", return_value={}),
|
||||
patch("run_agent.OpenAI"),
|
||||
):
|
||||
agent = AIAgent(
|
||||
api_key="test-key",
|
||||
quiet_mode=True,
|
||||
skip_context_files=True,
|
||||
skip_memory=True,
|
||||
fallback_model=fallback_model,
|
||||
)
|
||||
agent.client = MagicMock()
|
||||
return agent
|
||||
|
||||
|
||||
def _mock_client(base_url="https://openrouter.ai/api/v1", api_key="fb-key"):
|
||||
mock = MagicMock()
|
||||
mock.base_url = base_url
|
||||
mock.api_key = api_key
|
||||
return mock
|
||||
|
||||
|
||||
# ── Chain initialisation ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestFallbackChainInit:
|
||||
def test_no_fallback(self):
|
||||
agent = _make_agent(fallback_model=None)
|
||||
assert agent._fallback_chain == []
|
||||
assert agent._fallback_index == 0
|
||||
assert agent._fallback_model is None
|
||||
|
||||
def test_single_dict_backwards_compat(self):
|
||||
fb = {"provider": "openai", "model": "gpt-4o"}
|
||||
agent = _make_agent(fallback_model=fb)
|
||||
assert agent._fallback_chain == [fb]
|
||||
assert agent._fallback_model == fb
|
||||
|
||||
def test_list_of_providers(self):
|
||||
fbs = [
|
||||
{"provider": "openai", "model": "gpt-4o"},
|
||||
{"provider": "zai", "model": "glm-4.7"},
|
||||
]
|
||||
agent = _make_agent(fallback_model=fbs)
|
||||
assert len(agent._fallback_chain) == 2
|
||||
assert agent._fallback_model == fbs[0]
|
||||
|
||||
def test_invalid_entries_filtered(self):
|
||||
fbs = [
|
||||
{"provider": "openai", "model": "gpt-4o"},
|
||||
{"provider": "", "model": "glm-4.7"},
|
||||
{"provider": "zai"},
|
||||
"not-a-dict",
|
||||
]
|
||||
agent = _make_agent(fallback_model=fbs)
|
||||
assert len(agent._fallback_chain) == 1
|
||||
assert agent._fallback_chain[0]["provider"] == "openai"
|
||||
|
||||
def test_empty_list(self):
|
||||
agent = _make_agent(fallback_model=[])
|
||||
assert agent._fallback_chain == []
|
||||
assert agent._fallback_model is None
|
||||
|
||||
def test_invalid_dict_no_provider(self):
|
||||
agent = _make_agent(fallback_model={"model": "gpt-4o"})
|
||||
assert agent._fallback_chain == []
|
||||
|
||||
|
||||
# ── Chain advancement ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TestFallbackChainAdvancement:
|
||||
def test_exhausted_returns_false(self):
|
||||
agent = _make_agent(fallback_model=None)
|
||||
assert agent._try_activate_fallback() is False
|
||||
|
||||
def test_advances_index(self):
|
||||
fbs = [
|
||||
{"provider": "openai", "model": "gpt-4o"},
|
||||
{"provider": "zai", "model": "glm-4.7"},
|
||||
]
|
||||
agent = _make_agent(fallback_model=fbs)
|
||||
with patch("agent.auxiliary_client.resolve_provider_client",
|
||||
return_value=(_mock_client(), "gpt-4o")):
|
||||
assert agent._try_activate_fallback() is True
|
||||
assert agent._fallback_index == 1
|
||||
assert agent.model == "gpt-4o"
|
||||
assert agent._fallback_activated is True
|
||||
|
||||
def test_second_fallback_works(self):
|
||||
fbs = [
|
||||
{"provider": "openai", "model": "gpt-4o"},
|
||||
{"provider": "zai", "model": "glm-4.7"},
|
||||
]
|
||||
agent = _make_agent(fallback_model=fbs)
|
||||
with patch("agent.auxiliary_client.resolve_provider_client",
|
||||
return_value=(_mock_client(), "resolved")):
|
||||
assert agent._try_activate_fallback() is True
|
||||
assert agent.model == "gpt-4o"
|
||||
assert agent._try_activate_fallback() is True
|
||||
assert agent.model == "glm-4.7"
|
||||
assert agent._fallback_index == 2
|
||||
|
||||
def test_all_exhausted_returns_false(self):
|
||||
fbs = [{"provider": "openai", "model": "gpt-4o"}]
|
||||
agent = _make_agent(fallback_model=fbs)
|
||||
with patch("agent.auxiliary_client.resolve_provider_client",
|
||||
return_value=(_mock_client(), "gpt-4o")):
|
||||
assert agent._try_activate_fallback() is True
|
||||
assert agent._try_activate_fallback() is False
|
||||
|
||||
def test_skips_unconfigured_provider_to_next(self):
|
||||
"""If resolve_provider_client returns None, skip to next in chain."""
|
||||
fbs = [
|
||||
{"provider": "broken", "model": "nope"},
|
||||
{"provider": "openai", "model": "gpt-4o"},
|
||||
]
|
||||
agent = _make_agent(fallback_model=fbs)
|
||||
with patch("agent.auxiliary_client.resolve_provider_client") as mock_rpc:
|
||||
mock_rpc.side_effect = [
|
||||
(None, None), # broken provider
|
||||
(_mock_client(), "gpt-4o"), # fallback succeeds
|
||||
]
|
||||
assert agent._try_activate_fallback() is True
|
||||
assert agent.model == "gpt-4o"
|
||||
assert agent._fallback_index == 2
|
||||
|
||||
def test_skips_provider_that_raises_to_next(self):
|
||||
"""If resolve_provider_client raises, skip to next in chain."""
|
||||
fbs = [
|
||||
{"provider": "broken", "model": "nope"},
|
||||
{"provider": "openai", "model": "gpt-4o"},
|
||||
]
|
||||
agent = _make_agent(fallback_model=fbs)
|
||||
with patch("agent.auxiliary_client.resolve_provider_client") as mock_rpc:
|
||||
mock_rpc.side_effect = [
|
||||
RuntimeError("auth failed"),
|
||||
(_mock_client(), "gpt-4o"),
|
||||
]
|
||||
assert agent._try_activate_fallback() is True
|
||||
assert agent.model == "gpt-4o"
|
||||
@@ -2507,8 +2507,6 @@ class TestFallbackAnthropicProvider:
|
||||
def test_fallback_to_anthropic_sets_api_mode(self, agent):
|
||||
agent._fallback_activated = False
|
||||
agent._fallback_model = {"provider": "anthropic", "model": "claude-sonnet-4-20250514"}
|
||||
agent._fallback_chain = [agent._fallback_model]
|
||||
agent._fallback_index = 0
|
||||
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://api.anthropic.com/v1"
|
||||
@@ -2530,8 +2528,6 @@ class TestFallbackAnthropicProvider:
|
||||
def test_fallback_to_anthropic_enables_prompt_caching(self, agent):
|
||||
agent._fallback_activated = False
|
||||
agent._fallback_model = {"provider": "anthropic", "model": "claude-sonnet-4-20250514"}
|
||||
agent._fallback_chain = [agent._fallback_model]
|
||||
agent._fallback_index = 0
|
||||
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://api.anthropic.com/v1"
|
||||
@@ -2549,8 +2545,6 @@ class TestFallbackAnthropicProvider:
|
||||
def test_fallback_to_openrouter_uses_openai_client(self, agent):
|
||||
agent._fallback_activated = False
|
||||
agent._fallback_model = {"provider": "openrouter", "model": "anthropic/claude-sonnet-4"}
|
||||
agent._fallback_chain = [agent._fallback_model]
|
||||
agent._fallback_index = 0
|
||||
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://openrouter.ai/api/v1"
|
||||
@@ -3244,8 +3238,6 @@ class TestFallbackSetsOAuthFlag:
|
||||
def test_fallback_to_anthropic_oauth_sets_flag(self, agent):
|
||||
agent._fallback_activated = False
|
||||
agent._fallback_model = {"provider": "anthropic", "model": "claude-sonnet-4-6"}
|
||||
agent._fallback_chain = [agent._fallback_model]
|
||||
agent._fallback_index = 0
|
||||
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://api.anthropic.com/v1"
|
||||
@@ -3267,8 +3259,6 @@ class TestFallbackSetsOAuthFlag:
|
||||
def test_fallback_to_anthropic_api_key_clears_flag(self, agent):
|
||||
agent._fallback_activated = False
|
||||
agent._fallback_model = {"provider": "anthropic", "model": "claude-sonnet-4-6"}
|
||||
agent._fallback_chain = [agent._fallback_model]
|
||||
agent._fallback_index = 0
|
||||
|
||||
mock_client = MagicMock()
|
||||
mock_client.base_url = "https://api.anthropic.com/v1"
|
||||
|
||||
@@ -339,16 +339,6 @@ class TestTeePattern:
|
||||
assert dangerous is True
|
||||
assert key is not None
|
||||
|
||||
def test_tee_custom_hermes_home_env(self):
|
||||
dangerous, key, desc = detect_dangerous_command("echo x | tee $HERMES_HOME/.env")
|
||||
assert dangerous is True
|
||||
assert key is not None
|
||||
|
||||
def test_tee_quoted_custom_hermes_home_env(self):
|
||||
dangerous, key, desc = detect_dangerous_command('echo x | tee "$HERMES_HOME/.env"')
|
||||
assert dangerous is True
|
||||
assert key is not None
|
||||
|
||||
def test_tee_tmp_safe(self):
|
||||
dangerous, key, desc = detect_dangerous_command("echo hello | tee /tmp/output.txt")
|
||||
assert dangerous is False
|
||||
@@ -384,30 +374,6 @@ class TestFindExecFullPathRm:
|
||||
assert key is None
|
||||
|
||||
|
||||
class TestSensitiveRedirectPattern:
|
||||
"""Detect shell redirection writes to sensitive user-managed paths."""
|
||||
|
||||
def test_redirect_to_custom_hermes_home_env(self):
|
||||
dangerous, key, desc = detect_dangerous_command("echo x > $HERMES_HOME/.env")
|
||||
assert dangerous is True
|
||||
assert key is not None
|
||||
|
||||
def test_append_to_home_ssh_authorized_keys(self):
|
||||
dangerous, key, desc = detect_dangerous_command("cat key >> $HOME/.ssh/authorized_keys")
|
||||
assert dangerous is True
|
||||
assert key is not None
|
||||
|
||||
def test_append_to_tilde_ssh_authorized_keys(self):
|
||||
dangerous, key, desc = detect_dangerous_command("cat key >> ~/.ssh/authorized_keys")
|
||||
assert dangerous is True
|
||||
assert key is not None
|
||||
|
||||
def test_redirect_to_safe_tmp_file(self):
|
||||
dangerous, key, desc = detect_dangerous_command("echo hello > /tmp/output.txt")
|
||||
assert dangerous is False
|
||||
assert key is None
|
||||
|
||||
|
||||
class TestPatternKeyUniqueness:
|
||||
"""Bug: pattern_key is derived by splitting on \\b and taking [1], so
|
||||
patterns starting with the same word (e.g. find -exec rm and find -delete)
|
||||
@@ -640,4 +606,3 @@ class TestNormalizationBypass:
|
||||
dangerous, key, desc = detect_dangerous_command(cmd)
|
||||
assert dangerous is False
|
||||
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user