Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| b0b9ef0c86 |
@@ -1,5 +1 @@
|
||||
watch_file pyproject.toml uv.lock
|
||||
watch_file ui-tui/package-lock.json ui-tui/package.json
|
||||
watch_file flake.nix flake.lock nix/devShell.nix nix/tui.nix nix/package.nix nix/python.nix
|
||||
|
||||
use flake
|
||||
|
||||
@@ -16,8 +16,13 @@ concurrency:
|
||||
|
||||
jobs:
|
||||
test:
|
||||
name: test (${{ matrix.group }}/4)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
group: [1, 2, 3, 4]
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
@@ -37,10 +42,11 @@ jobs:
|
||||
source .venv/bin/activate
|
||||
uv pip install -e ".[all,dev]"
|
||||
|
||||
- name: Run tests
|
||||
- name: Run tests (shard ${{ matrix.group }}/4)
|
||||
run: |
|
||||
source .venv/bin/activate
|
||||
python -m pytest tests/ -q --ignore=tests/integration --ignore=tests/e2e --tb=short -n auto
|
||||
python -m pytest tests/ -q --ignore=tests/integration --ignore=tests/e2e --tb=short \
|
||||
--splits 4 --group ${{ matrix.group }}
|
||||
env:
|
||||
# Ensure tests don't accidentally call real APIs
|
||||
OPENROUTER_API_KEY: ""
|
||||
|
||||
@@ -60,6 +60,5 @@ mini-swe-agent/
|
||||
|
||||
# Nix
|
||||
.direnv/
|
||||
.nix-stamps/
|
||||
result
|
||||
website/static/api/skills-index.json
|
||||
|
||||
@@ -56,19 +56,6 @@ hermes-agent/
|
||||
│ ├── run.py # Main loop, slash commands, message dispatch
|
||||
│ ├── session.py # SessionStore — conversation persistence
|
||||
│ └── platforms/ # Adapters: telegram, discord, slack, whatsapp, homeassistant, signal, qqbot
|
||||
├── ui-tui/ # Ink (React) terminal UI — `hermes --tui`
|
||||
│ ├── src/entry.tsx # TTY gate + render()
|
||||
│ ├── src/app.tsx # Main state machine and UI
|
||||
│ ├── src/gatewayClient.ts # Child process + JSON-RPC bridge
|
||||
│ ├── src/app/ # Decomposed app logic (event handler, slash handler, stores, hooks)
|
||||
│ ├── src/components/ # Ink components (branding, markdown, prompts, pickers, etc.)
|
||||
│ ├── src/hooks/ # useCompletion, useInputHistory, useQueue, useVirtualHistory
|
||||
│ └── src/lib/ # Pure helpers (history, osc52, text, rpc, messages)
|
||||
├── tui_gateway/ # Python JSON-RPC backend for the TUI
|
||||
│ ├── entry.py # stdio entrypoint
|
||||
│ ├── server.py # RPC handlers and session logic
|
||||
│ ├── render.py # Optional rich/ANSI bridge
|
||||
│ └── slash_worker.py # Persistent HermesCLI subprocess for slash commands
|
||||
├── acp_adapter/ # ACP server (VS Code / Zed / JetBrains integration)
|
||||
├── cron/ # Scheduler (jobs.py, scheduler.py)
|
||||
├── environments/ # RL training environments (Atropos)
|
||||
@@ -192,59 +179,6 @@ if canonical == "mycommand":
|
||||
|
||||
---
|
||||
|
||||
## TUI Architecture (ui-tui + tui_gateway)
|
||||
|
||||
The TUI is a full replacement for the classic (prompt_toolkit) CLI, activated via `hermes --tui` or `HERMES_TUI=1`.
|
||||
|
||||
### Process Model
|
||||
|
||||
```
|
||||
hermes --tui
|
||||
└─ Node (Ink) ──stdio JSON-RPC── Python (tui_gateway)
|
||||
│ └─ AIAgent + tools + sessions
|
||||
└─ renders transcript, composer, prompts, activity
|
||||
```
|
||||
|
||||
TypeScript owns the screen. Python owns sessions, tools, model calls, and slash command logic.
|
||||
|
||||
### Transport
|
||||
|
||||
Newline-delimited JSON-RPC over stdio. Requests from Ink, events from Python. See `tui_gateway/server.py` for the full method/event catalog.
|
||||
|
||||
### Key Surfaces
|
||||
|
||||
| Surface | Ink component | Gateway method |
|
||||
|---------|---------------|----------------|
|
||||
| Chat streaming | `app.tsx` + `messageLine.tsx` | `prompt.submit` → `message.delta/complete` |
|
||||
| Tool activity | `thinking.tsx` | `tool.start/progress/complete` |
|
||||
| Approvals | `prompts.tsx` | `approval.respond` ← `approval.request` |
|
||||
| Clarify/sudo/secret | `prompts.tsx`, `maskedPrompt.tsx` | `clarify/sudo/secret.respond` |
|
||||
| Session picker | `sessionPicker.tsx` | `session.list/resume` |
|
||||
| Slash commands | Local handler + fallthrough | `slash.exec` → `_SlashWorker`, `command.dispatch` |
|
||||
| Completions | `useCompletion` hook | `complete.slash`, `complete.path` |
|
||||
| Theming | `theme.ts` + `branding.tsx` | `gateway.ready` with skin data |
|
||||
|
||||
### Slash Command Flow
|
||||
|
||||
1. Built-in client commands (`/help`, `/quit`, `/clear`, `/resume`, `/copy`, `/paste`, etc.) handled locally in `app.tsx`
|
||||
2. Everything else → `slash.exec` (runs in persistent `_SlashWorker` subprocess) → `command.dispatch` fallback
|
||||
|
||||
### Dev Commands
|
||||
|
||||
```bash
|
||||
cd ui-tui
|
||||
npm install # first time
|
||||
npm run dev # watch mode (rebuilds hermes-ink + tsx --watch)
|
||||
npm start # production
|
||||
npm run build # full build (hermes-ink + tsc)
|
||||
npm run type-check # typecheck only (tsc --noEmit)
|
||||
npm run lint # eslint
|
||||
npm run fmt # prettier
|
||||
npm test # vitest
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding New Tools
|
||||
|
||||
Requires changes in **2 files**:
|
||||
@@ -524,45 +458,13 @@ def profile_env(tmp_path, monkeypatch):
|
||||
|
||||
## Testing
|
||||
|
||||
**ALWAYS use `scripts/run_tests.sh`** — do not call `pytest` directly. The script enforces
|
||||
hermetic environment parity with CI (unset credential vars, TZ=UTC, LANG=C.UTF-8,
|
||||
4 xdist workers matching GHA ubuntu-latest). Direct `pytest` on a 16+ core
|
||||
developer machine with API keys set diverges from CI in ways that have caused
|
||||
multiple "works locally, fails in CI" incidents (and the reverse).
|
||||
|
||||
```bash
|
||||
scripts/run_tests.sh # full suite, CI-parity
|
||||
scripts/run_tests.sh tests/gateway/ # one directory
|
||||
scripts/run_tests.sh tests/agent/test_foo.py::test_x # one test
|
||||
scripts/run_tests.sh -v --tb=long # pass-through pytest flags
|
||||
```
|
||||
|
||||
### Why the wrapper (and why the old "just call pytest" doesn't work)
|
||||
|
||||
Five real sources of local-vs-CI drift the script closes:
|
||||
|
||||
| | Without wrapper | With wrapper |
|
||||
|---|---|---|
|
||||
| Provider API keys | Whatever is in your env (auto-detects pool) | All `*_API_KEY`/`*_TOKEN`/etc. unset |
|
||||
| HOME / `~/.hermes/` | Your real config+auth.json | Temp dir per test |
|
||||
| Timezone | Local TZ (PDT etc.) | UTC |
|
||||
| Locale | Whatever is set | C.UTF-8 |
|
||||
| xdist workers | `-n auto` = all cores (20+ on a workstation) | `-n 4` matching CI |
|
||||
|
||||
`tests/conftest.py` also enforces points 1-4 as an autouse fixture so ANY pytest
|
||||
invocation (including IDE integrations) gets hermetic behavior — but the wrapper
|
||||
is belt-and-suspenders.
|
||||
|
||||
### Running without the wrapper (only if you must)
|
||||
|
||||
If you can't use the wrapper (e.g. on Windows or inside an IDE that shells
|
||||
pytest directly), at minimum activate the venv and pass `-n 4`:
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python -m pytest tests/ -q -n 4
|
||||
python -m pytest tests/ -q # Full suite (~3000 tests, ~3 min)
|
||||
python -m pytest tests/test_model_tools.py -q # Toolset resolution
|
||||
python -m pytest tests/test_cli_init.py -q # CLI config loading
|
||||
python -m pytest tests/gateway/ -q # Gateway tests
|
||||
python -m pytest tests/tools/ -q # Tool-level tests
|
||||
```
|
||||
|
||||
Worker count above 4 will surface test-ordering flakes that CI never sees.
|
||||
|
||||
Always run the full suite before pushing changes.
|
||||
|
||||
+10
-20
@@ -21,36 +21,26 @@ RUN useradd -u 10000 -m -d /opt/data hermes
|
||||
COPY --chmod=0755 --from=gosu_source /gosu /usr/local/bin/
|
||||
COPY --chmod=0755 --from=uv_source /usr/local/bin/uv /usr/local/bin/uvx /usr/local/bin/
|
||||
|
||||
COPY . /opt/hermes
|
||||
WORKDIR /opt/hermes
|
||||
|
||||
# ---------- Layer-cached dependency install ----------
|
||||
# Copy only package manifests first so npm install + Playwright are cached
|
||||
# unless the lockfiles themselves change.
|
||||
COPY package.json package-lock.json ./
|
||||
COPY scripts/whatsapp-bridge/package.json scripts/whatsapp-bridge/package-lock.json scripts/whatsapp-bridge/
|
||||
COPY web/package.json web/package-lock.json web/
|
||||
|
||||
# Install Node dependencies and Playwright as root (--with-deps needs apt)
|
||||
RUN npm install --prefer-offline --no-audit && \
|
||||
npx playwright install --with-deps chromium --only-shell && \
|
||||
(cd scripts/whatsapp-bridge && npm install --prefer-offline --no-audit) && \
|
||||
(cd web && npm install --prefer-offline --no-audit) && \
|
||||
cd /opt/hermes/scripts/whatsapp-bridge && \
|
||||
npm install --prefer-offline --no-audit && \
|
||||
npm cache clean --force
|
||||
|
||||
# ---------- Source code ----------
|
||||
# .dockerignore excludes node_modules, so the installs above survive.
|
||||
COPY --chown=hermes:hermes . .
|
||||
|
||||
# Build web dashboard (Vite outputs to hermes_cli/web_dist/)
|
||||
RUN cd web && npm run build
|
||||
|
||||
# ---------- Python virtualenv ----------
|
||||
RUN chown hermes:hermes /opt/hermes
|
||||
# Hand ownership to hermes user, then install Python deps in a virtualenv
|
||||
RUN chown -R hermes:hermes /opt/hermes
|
||||
USER hermes
|
||||
|
||||
RUN uv venv && \
|
||||
uv pip install --no-cache-dir -e ".[all]"
|
||||
|
||||
# ---------- Runtime ----------
|
||||
ENV HERMES_WEB_DIST=/opt/hermes/hermes_cli/web_dist
|
||||
USER root
|
||||
RUN chmod +x /opt/hermes/docker/entrypoint.sh
|
||||
|
||||
ENV HERMES_HOME=/opt/data
|
||||
VOLUME [ "/opt/data" ]
|
||||
ENTRYPOINT [ "/opt/hermes/docker/entrypoint.sh" ]
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
|
||||
**The self-improving AI agent built by [Nous Research](https://nousresearch.com).** It's the only agent with a built-in learning loop — it creates skills from experience, improves them during use, nudges itself to persist knowledge, searches its own past conversations, and builds a deepening model of who you are across sessions. Run it on a $5 VPS, a GPU cluster, or serverless infrastructure that costs nearly nothing when idle. It's not tied to your laptop — talk to it from Telegram while it works on a cloud VM.
|
||||
|
||||
Use any model you want — [Nous Portal](https://portal.nousresearch.com), [OpenRouter](https://openrouter.ai) (200+ models), [NVIDIA NIM](https://build.nvidia.com) (Nemotron), [Xiaomi MiMo](https://platform.xiaomimimo.com), [z.ai/GLM](https://z.ai), [Kimi/Moonshot](https://platform.moonshot.ai), [MiniMax](https://www.minimax.io), [Hugging Face](https://huggingface.co), OpenAI, or your own endpoint. Switch with `hermes model` — no code changes, no lock-in.
|
||||
Use any model you want — [Nous Portal](https://portal.nousresearch.com), [OpenRouter](https://openrouter.ai) (200+ models), [Xiaomi MiMo](https://platform.xiaomimimo.com), [z.ai/GLM](https://z.ai), [Kimi/Moonshot](https://platform.moonshot.ai), [MiniMax](https://www.minimax.io), [Hugging Face](https://huggingface.co), OpenAI, or your own endpoint. Switch with `hermes model` — no code changes, no lock-in.
|
||||
|
||||
<table>
|
||||
<tr><td><b>A real terminal interface</b></td><td>Full TUI with multiline editing, slash-command autocomplete, conversation history, interrupt-and-redirect, and streaming tool output.</td></tr>
|
||||
@@ -141,18 +141,11 @@ See `hermes claw migrate --help` for all options, or use the `openclaw-migration
|
||||
|
||||
We welcome contributions! See the [Contributing Guide](https://hermes-agent.nousresearch.com/docs/developer-guide/contributing) for development setup, code style, and PR process.
|
||||
|
||||
Quick start for contributors — clone and go with `setup-hermes.sh`:
|
||||
Quick start for contributors:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/NousResearch/hermes-agent.git
|
||||
cd hermes-agent
|
||||
./setup-hermes.sh # installs uv, creates venv, installs .[all], symlinks ~/.local/bin/hermes
|
||||
./hermes # auto-detects the venv, no need to `source` first
|
||||
```
|
||||
|
||||
Manual path (equivalent to the above):
|
||||
|
||||
```bash
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
uv venv venv --python 3.11
|
||||
source venv/bin/activate
|
||||
|
||||
+1
-20
@@ -49,7 +49,6 @@ def make_tool_progress_cb(
|
||||
session_id: str,
|
||||
loop: asyncio.AbstractEventLoop,
|
||||
tool_call_ids: Dict[str, Deque[str]],
|
||||
tool_call_meta: Dict[str, Dict[str, Any]],
|
||||
) -> Callable:
|
||||
"""Create a ``tool_progress_callback`` for AIAgent.
|
||||
|
||||
@@ -85,16 +84,6 @@ def make_tool_progress_cb(
|
||||
tool_call_ids[name] = queue
|
||||
queue.append(tc_id)
|
||||
|
||||
snapshot = None
|
||||
if name in {"write_file", "patch", "skill_manage"}:
|
||||
try:
|
||||
from agent.display import capture_local_edit_snapshot
|
||||
|
||||
snapshot = capture_local_edit_snapshot(name, args)
|
||||
except Exception:
|
||||
logger.debug("Failed to capture ACP edit snapshot for %s", name, exc_info=True)
|
||||
tool_call_meta[tc_id] = {"args": args, "snapshot": snapshot}
|
||||
|
||||
update = build_tool_start(tc_id, name, args)
|
||||
_send_update(conn, session_id, loop, update)
|
||||
|
||||
@@ -130,7 +119,6 @@ def make_step_cb(
|
||||
session_id: str,
|
||||
loop: asyncio.AbstractEventLoop,
|
||||
tool_call_ids: Dict[str, Deque[str]],
|
||||
tool_call_meta: Dict[str, Dict[str, Any]],
|
||||
) -> Callable:
|
||||
"""Create a ``step_callback`` for AIAgent.
|
||||
|
||||
@@ -144,12 +132,10 @@ def make_step_cb(
|
||||
for tool_info in prev_tools:
|
||||
tool_name = None
|
||||
result = None
|
||||
function_args = None
|
||||
|
||||
if isinstance(tool_info, dict):
|
||||
tool_name = tool_info.get("name") or tool_info.get("function_name")
|
||||
result = tool_info.get("result") or tool_info.get("output")
|
||||
function_args = tool_info.get("arguments") or tool_info.get("args")
|
||||
elif isinstance(tool_info, str):
|
||||
tool_name = tool_info
|
||||
|
||||
@@ -159,13 +145,8 @@ def make_step_cb(
|
||||
tool_call_ids[tool_name] = queue
|
||||
if tool_name and queue:
|
||||
tc_id = queue.popleft()
|
||||
meta = tool_call_meta.pop(tc_id, {})
|
||||
update = build_tool_complete(
|
||||
tc_id,
|
||||
tool_name,
|
||||
result=str(result) if result is not None else None,
|
||||
function_args=function_args or meta.get("args"),
|
||||
snapshot=meta.get("snapshot"),
|
||||
tc_id, tool_name, result=str(result) if result is not None else None
|
||||
)
|
||||
_send_update(conn, session_id, loop, update)
|
||||
if not queue:
|
||||
|
||||
+30
-148
@@ -26,7 +26,6 @@ from acp.schema import (
|
||||
McpServerHttp,
|
||||
McpServerSse,
|
||||
McpServerStdio,
|
||||
ModelInfo,
|
||||
NewSessionResponse,
|
||||
PromptResponse,
|
||||
ResumeSessionResponse,
|
||||
@@ -37,7 +36,6 @@ from acp.schema import (
|
||||
SessionCapabilities,
|
||||
SessionForkCapabilities,
|
||||
SessionListCapabilities,
|
||||
SessionModelState,
|
||||
SessionResumeCapabilities,
|
||||
SessionInfo,
|
||||
TextContentBlock,
|
||||
@@ -149,98 +147,6 @@ class HermesACPAgent(acp.Agent):
|
||||
self._conn = conn
|
||||
logger.info("ACP client connected")
|
||||
|
||||
@staticmethod
|
||||
def _encode_model_choice(provider: str | None, model: str | None) -> str:
|
||||
"""Encode a model selection so ACP clients can keep provider context."""
|
||||
raw_model = str(model or "").strip()
|
||||
if not raw_model:
|
||||
return ""
|
||||
raw_provider = str(provider or "").strip().lower()
|
||||
if not raw_provider:
|
||||
return raw_model
|
||||
return f"{raw_provider}:{raw_model}"
|
||||
|
||||
def _build_model_state(self, state: SessionState) -> SessionModelState | None:
|
||||
"""Return the ACP model selector payload for editors like Zed."""
|
||||
model = str(state.model or getattr(state.agent, "model", "") or "").strip()
|
||||
provider = getattr(state.agent, "provider", None) or detect_provider() or "openrouter"
|
||||
|
||||
try:
|
||||
from hermes_cli.models import curated_models_for_provider, normalize_provider, provider_label
|
||||
|
||||
normalized_provider = normalize_provider(provider)
|
||||
provider_name = provider_label(normalized_provider)
|
||||
available_models: list[ModelInfo] = []
|
||||
seen_ids: set[str] = set()
|
||||
|
||||
for model_id, description in curated_models_for_provider(normalized_provider):
|
||||
rendered_model = str(model_id or "").strip()
|
||||
if not rendered_model:
|
||||
continue
|
||||
choice_id = self._encode_model_choice(normalized_provider, rendered_model)
|
||||
if choice_id in seen_ids:
|
||||
continue
|
||||
desc_parts = [f"Provider: {provider_name}"]
|
||||
if description:
|
||||
desc_parts.append(str(description).strip())
|
||||
if rendered_model == model:
|
||||
desc_parts.append("current")
|
||||
available_models.append(
|
||||
ModelInfo(
|
||||
model_id=choice_id,
|
||||
name=rendered_model,
|
||||
description=" • ".join(part for part in desc_parts if part),
|
||||
)
|
||||
)
|
||||
seen_ids.add(choice_id)
|
||||
|
||||
current_model_id = self._encode_model_choice(normalized_provider, model)
|
||||
if current_model_id and current_model_id not in seen_ids:
|
||||
available_models.insert(
|
||||
0,
|
||||
ModelInfo(
|
||||
model_id=current_model_id,
|
||||
name=model,
|
||||
description=f"Provider: {provider_name} • current",
|
||||
),
|
||||
)
|
||||
|
||||
if available_models:
|
||||
return SessionModelState(
|
||||
available_models=available_models,
|
||||
current_model_id=current_model_id or available_models[0].model_id,
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Could not build ACP model state", exc_info=True)
|
||||
|
||||
if not model:
|
||||
return None
|
||||
|
||||
fallback_choice = self._encode_model_choice(provider, model)
|
||||
return SessionModelState(
|
||||
available_models=[ModelInfo(model_id=fallback_choice, name=model)],
|
||||
current_model_id=fallback_choice,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _resolve_model_selection(raw_model: str, current_provider: str) -> tuple[str, str]:
|
||||
"""Resolve ``provider:model`` input into the provider and normalized model id."""
|
||||
target_provider = current_provider
|
||||
new_model = raw_model.strip()
|
||||
|
||||
try:
|
||||
from hermes_cli.models import detect_provider_for_model, parse_model_input
|
||||
|
||||
target_provider, new_model = parse_model_input(new_model, current_provider)
|
||||
if target_provider == current_provider:
|
||||
detected = detect_provider_for_model(new_model, current_provider)
|
||||
if detected:
|
||||
target_provider, new_model = detected
|
||||
except Exception:
|
||||
logger.debug("Provider detection failed, using model as-is", exc_info=True)
|
||||
|
||||
return target_provider, new_model
|
||||
|
||||
async def _register_session_mcp_servers(
|
||||
self,
|
||||
state: SessionState,
|
||||
@@ -367,10 +273,7 @@ class HermesACPAgent(acp.Agent):
|
||||
await self._register_session_mcp_servers(state, mcp_servers)
|
||||
logger.info("New session %s (cwd=%s)", state.session_id, cwd)
|
||||
self._schedule_available_commands_update(state.session_id)
|
||||
return NewSessionResponse(
|
||||
session_id=state.session_id,
|
||||
models=self._build_model_state(state),
|
||||
)
|
||||
return NewSessionResponse(session_id=state.session_id)
|
||||
|
||||
async def load_session(
|
||||
self,
|
||||
@@ -386,7 +289,7 @@ class HermesACPAgent(acp.Agent):
|
||||
await self._register_session_mcp_servers(state, mcp_servers)
|
||||
logger.info("Loaded session %s", session_id)
|
||||
self._schedule_available_commands_update(session_id)
|
||||
return LoadSessionResponse(models=self._build_model_state(state))
|
||||
return LoadSessionResponse()
|
||||
|
||||
async def resume_session(
|
||||
self,
|
||||
@@ -402,7 +305,7 @@ class HermesACPAgent(acp.Agent):
|
||||
await self._register_session_mcp_servers(state, mcp_servers)
|
||||
logger.info("Resumed session %s", state.session_id)
|
||||
self._schedule_available_commands_update(state.session_id)
|
||||
return ResumeSessionResponse(models=self._build_model_state(state))
|
||||
return ResumeSessionResponse()
|
||||
|
||||
async def cancel(self, session_id: str, **kwargs: Any) -> None:
|
||||
state = self.session_manager.get_session(session_id)
|
||||
@@ -437,20 +340,11 @@ class HermesACPAgent(acp.Agent):
|
||||
cwd: str | None = None,
|
||||
**kwargs: Any,
|
||||
) -> ListSessionsResponse:
|
||||
infos = self.session_manager.list_sessions(cwd=cwd)
|
||||
sessions = []
|
||||
for s in infos:
|
||||
updated_at = s.get("updated_at")
|
||||
if updated_at is not None and not isinstance(updated_at, str):
|
||||
updated_at = str(updated_at)
|
||||
sessions.append(
|
||||
SessionInfo(
|
||||
session_id=s["session_id"],
|
||||
cwd=s["cwd"],
|
||||
title=s.get("title"),
|
||||
updated_at=updated_at,
|
||||
)
|
||||
)
|
||||
infos = self.session_manager.list_sessions()
|
||||
sessions = [
|
||||
SessionInfo(session_id=s["session_id"], cwd=s["cwd"])
|
||||
for s in infos
|
||||
]
|
||||
return ListSessionsResponse(sessions=sessions)
|
||||
|
||||
# ---- Prompt (core) ------------------------------------------------------
|
||||
@@ -495,13 +389,12 @@ class HermesACPAgent(acp.Agent):
|
||||
state.cancel_event.clear()
|
||||
|
||||
tool_call_ids: dict[str, Deque[str]] = defaultdict(deque)
|
||||
tool_call_meta: dict[str, dict[str, Any]] = {}
|
||||
previous_approval_cb = None
|
||||
|
||||
if conn:
|
||||
tool_progress_cb = make_tool_progress_cb(conn, session_id, loop, tool_call_ids, tool_call_meta)
|
||||
tool_progress_cb = make_tool_progress_cb(conn, session_id, loop, tool_call_ids)
|
||||
thinking_cb = make_thinking_cb(conn, session_id, loop)
|
||||
step_cb = make_step_cb(conn, session_id, loop, tool_call_ids, tool_call_meta)
|
||||
step_cb = make_step_cb(conn, session_id, loop, tool_call_ids)
|
||||
message_cb = make_message_cb(conn, session_id, loop)
|
||||
approval_cb = make_approval_callback(conn.request_permission, loop, session_id)
|
||||
else:
|
||||
@@ -556,19 +449,6 @@ class HermesACPAgent(acp.Agent):
|
||||
self.session_manager.save_session(session_id)
|
||||
|
||||
final_response = result.get("final_response", "")
|
||||
if final_response:
|
||||
try:
|
||||
from agent.title_generator import maybe_auto_title
|
||||
|
||||
maybe_auto_title(
|
||||
self.session_manager._get_db(),
|
||||
session_id,
|
||||
user_text,
|
||||
final_response,
|
||||
state.history,
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Failed to auto-title ACP session %s", session_id, exc_info=True)
|
||||
if final_response and conn:
|
||||
update = acp.update_agent_message_text(final_response)
|
||||
await conn.session_update(session_id, update)
|
||||
@@ -676,15 +556,27 @@ class HermesACPAgent(acp.Agent):
|
||||
provider = getattr(state.agent, "provider", None) or "auto"
|
||||
return f"Current model: {model}\nProvider: {provider}"
|
||||
|
||||
new_model = args.strip()
|
||||
target_provider = None
|
||||
current_provider = getattr(state.agent, "provider", None) or "openrouter"
|
||||
target_provider, new_model = self._resolve_model_selection(args, current_provider)
|
||||
|
||||
# Auto-detect provider for the requested model
|
||||
try:
|
||||
from hermes_cli.models import parse_model_input, detect_provider_for_model
|
||||
target_provider, new_model = parse_model_input(new_model, current_provider)
|
||||
if target_provider == current_provider:
|
||||
detected = detect_provider_for_model(new_model, current_provider)
|
||||
if detected:
|
||||
target_provider, new_model = detected
|
||||
except Exception:
|
||||
logger.debug("Provider detection failed, using model as-is", exc_info=True)
|
||||
|
||||
state.model = new_model
|
||||
state.agent = self.session_manager._make_agent(
|
||||
session_id=state.session_id,
|
||||
cwd=state.cwd,
|
||||
model=new_model,
|
||||
requested_provider=target_provider,
|
||||
requested_provider=target_provider or current_provider,
|
||||
)
|
||||
self.session_manager.save_session(state.session_id)
|
||||
provider_label = getattr(state.agent, "provider", None) or target_provider or current_provider
|
||||
@@ -786,30 +678,20 @@ class HermesACPAgent(acp.Agent):
|
||||
"""Switch the model for a session (called by ACP protocol)."""
|
||||
state = self.session_manager.get_session(session_id)
|
||||
if state:
|
||||
state.model = model_id
|
||||
current_provider = getattr(state.agent, "provider", None)
|
||||
requested_provider, resolved_model = self._resolve_model_selection(
|
||||
model_id,
|
||||
current_provider or "openrouter",
|
||||
)
|
||||
state.model = resolved_model
|
||||
provider_changed = bool(current_provider and requested_provider != current_provider)
|
||||
current_base_url = None if provider_changed else getattr(state.agent, "base_url", None)
|
||||
current_api_mode = None if provider_changed else getattr(state.agent, "api_mode", None)
|
||||
current_base_url = getattr(state.agent, "base_url", None)
|
||||
current_api_mode = getattr(state.agent, "api_mode", None)
|
||||
state.agent = self.session_manager._make_agent(
|
||||
session_id=session_id,
|
||||
cwd=state.cwd,
|
||||
model=resolved_model,
|
||||
requested_provider=requested_provider,
|
||||
model=model_id,
|
||||
requested_provider=current_provider,
|
||||
base_url=current_base_url,
|
||||
api_mode=current_api_mode,
|
||||
)
|
||||
self.session_manager.save_session(session_id)
|
||||
logger.info(
|
||||
"Session %s: model switched to %s via provider %s",
|
||||
session_id,
|
||||
resolved_model,
|
||||
requested_provider,
|
||||
)
|
||||
logger.info("Session %s: model switched to %s", session_id, model_id)
|
||||
return SetSessionModelResponse()
|
||||
logger.warning("Session %s: model switch requested for missing session", session_id)
|
||||
return None
|
||||
|
||||
+34
-127
@@ -13,12 +13,8 @@ from hermes_constants import get_hermes_home
|
||||
import copy
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from dataclasses import dataclass, field
|
||||
from threading import Lock
|
||||
from typing import Any, Dict, List, Optional
|
||||
@@ -26,64 +22,6 @@ from typing import Any, Dict, List, Optional
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _normalize_cwd_for_compare(cwd: str | None) -> str:
|
||||
raw = str(cwd or ".").strip()
|
||||
if not raw:
|
||||
raw = "."
|
||||
expanded = os.path.expanduser(raw)
|
||||
|
||||
# Normalize Windows drive paths into the equivalent WSL mount form so
|
||||
# ACP history filters match the same workspace across Windows and WSL.
|
||||
match = re.match(r"^([A-Za-z]):[\\/](.*)$", expanded)
|
||||
if match:
|
||||
drive = match.group(1).lower()
|
||||
tail = match.group(2).replace("\\", "/")
|
||||
expanded = f"/mnt/{drive}/{tail}"
|
||||
elif re.match(r"^/mnt/[A-Za-z]/", expanded):
|
||||
expanded = f"/mnt/{expanded[5].lower()}/{expanded[7:]}"
|
||||
|
||||
return os.path.normpath(expanded)
|
||||
|
||||
|
||||
def _build_session_title(title: Any, preview: Any, cwd: str | None) -> str:
|
||||
explicit = str(title or "").strip()
|
||||
if explicit:
|
||||
return explicit
|
||||
preview_text = str(preview or "").strip()
|
||||
if preview_text:
|
||||
return preview_text
|
||||
leaf = os.path.basename(str(cwd or "").rstrip("/\\"))
|
||||
return leaf or "New thread"
|
||||
|
||||
|
||||
def _format_updated_at(value: Any) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, str) and value.strip():
|
||||
return value
|
||||
try:
|
||||
return datetime.fromtimestamp(float(value), tz=timezone.utc).isoformat()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _updated_at_sort_key(value: Any) -> float:
|
||||
if value is None:
|
||||
return float("-inf")
|
||||
if isinstance(value, (int, float)):
|
||||
return float(value)
|
||||
raw = str(value).strip()
|
||||
if not raw:
|
||||
return float("-inf")
|
||||
try:
|
||||
return datetime.fromisoformat(raw.replace("Z", "+00:00")).timestamp()
|
||||
except Exception:
|
||||
try:
|
||||
return float(raw)
|
||||
except Exception:
|
||||
return float("-inf")
|
||||
|
||||
|
||||
def _acp_stderr_print(*args, **kwargs) -> None:
|
||||
"""Best-effort human-readable output sink for ACP stdio sessions.
|
||||
|
||||
@@ -224,78 +162,47 @@ class SessionManager:
|
||||
logger.info("Forked ACP session %s -> %s", session_id, new_id)
|
||||
return state
|
||||
|
||||
def list_sessions(self, cwd: str | None = None) -> List[Dict[str, Any]]:
|
||||
def list_sessions(self) -> List[Dict[str, Any]]:
|
||||
"""Return lightweight info dicts for all sessions (memory + database)."""
|
||||
normalized_cwd = _normalize_cwd_for_compare(cwd) if cwd else None
|
||||
db = self._get_db()
|
||||
persisted_rows: dict[str, dict[str, Any]] = {}
|
||||
|
||||
if db is not None:
|
||||
try:
|
||||
for row in db.list_sessions_rich(source="acp", limit=1000):
|
||||
persisted_rows[str(row["id"])] = dict(row)
|
||||
except Exception:
|
||||
logger.debug("Failed to load ACP sessions from DB", exc_info=True)
|
||||
|
||||
# Collect in-memory sessions first.
|
||||
with self._lock:
|
||||
seen_ids = set(self._sessions.keys())
|
||||
results = []
|
||||
for s in self._sessions.values():
|
||||
history_len = len(s.history)
|
||||
if history_len <= 0:
|
||||
continue
|
||||
if normalized_cwd and _normalize_cwd_for_compare(s.cwd) != normalized_cwd:
|
||||
continue
|
||||
persisted = persisted_rows.get(s.session_id, {})
|
||||
preview = next(
|
||||
(
|
||||
str(msg.get("content") or "").strip()
|
||||
for msg in s.history
|
||||
if msg.get("role") == "user" and str(msg.get("content") or "").strip()
|
||||
),
|
||||
persisted.get("preview") or "",
|
||||
)
|
||||
results.append(
|
||||
{
|
||||
"session_id": s.session_id,
|
||||
"cwd": s.cwd,
|
||||
"model": s.model,
|
||||
"history_len": history_len,
|
||||
"title": _build_session_title(persisted.get("title"), preview, s.cwd),
|
||||
"updated_at": _format_updated_at(
|
||||
persisted.get("last_active") or persisted.get("started_at") or time.time()
|
||||
),
|
||||
}
|
||||
)
|
||||
results = [
|
||||
{
|
||||
"session_id": s.session_id,
|
||||
"cwd": s.cwd,
|
||||
"model": s.model,
|
||||
"history_len": len(s.history),
|
||||
}
|
||||
for s in self._sessions.values()
|
||||
]
|
||||
|
||||
# Merge any persisted sessions not currently in memory.
|
||||
for sid, row in persisted_rows.items():
|
||||
if sid in seen_ids:
|
||||
continue
|
||||
message_count = int(row.get("message_count") or 0)
|
||||
if message_count <= 0:
|
||||
continue
|
||||
# Extract cwd from model_config JSON.
|
||||
session_cwd = "."
|
||||
mc = row.get("model_config")
|
||||
if mc:
|
||||
try:
|
||||
session_cwd = json.loads(mc).get("cwd", ".")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
if normalized_cwd and _normalize_cwd_for_compare(session_cwd) != normalized_cwd:
|
||||
continue
|
||||
results.append({
|
||||
"session_id": sid,
|
||||
"cwd": session_cwd,
|
||||
"model": row.get("model") or "",
|
||||
"history_len": message_count,
|
||||
"title": _build_session_title(row.get("title"), row.get("preview"), session_cwd),
|
||||
"updated_at": _format_updated_at(row.get("last_active") or row.get("started_at")),
|
||||
})
|
||||
db = self._get_db()
|
||||
if db is not None:
|
||||
try:
|
||||
rows = db.search_sessions(source="acp", limit=1000)
|
||||
for row in rows:
|
||||
sid = row["id"]
|
||||
if sid in seen_ids:
|
||||
continue
|
||||
# Extract cwd from model_config JSON.
|
||||
cwd = "."
|
||||
mc = row.get("model_config")
|
||||
if mc:
|
||||
try:
|
||||
cwd = json.loads(mc).get("cwd", ".")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
results.append({
|
||||
"session_id": sid,
|
||||
"cwd": cwd,
|
||||
"model": row.get("model") or "",
|
||||
"history_len": row.get("message_count") or 0,
|
||||
})
|
||||
except Exception:
|
||||
logger.debug("Failed to list ACP sessions from DB", exc_info=True)
|
||||
|
||||
results.sort(key=lambda item: _updated_at_sort_key(item.get("updated_at")), reverse=True)
|
||||
return results
|
||||
|
||||
def update_cwd(self, session_id: str, cwd: str) -> Optional[SessionState]:
|
||||
|
||||
+9
-174
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import uuid
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
@@ -97,170 +96,6 @@ def build_tool_title(tool_name: str, args: Dict[str, Any]) -> str:
|
||||
return tool_name
|
||||
|
||||
|
||||
def _build_patch_mode_content(patch_text: str) -> List[Any]:
|
||||
"""Parse V4A patch mode input into ACP diff blocks when possible."""
|
||||
if not patch_text:
|
||||
return [acp.tool_content(acp.text_block(""))]
|
||||
|
||||
try:
|
||||
from tools.patch_parser import OperationType, parse_v4a_patch
|
||||
|
||||
operations, error = parse_v4a_patch(patch_text)
|
||||
if error or not operations:
|
||||
return [acp.tool_content(acp.text_block(patch_text))]
|
||||
|
||||
content: List[Any] = []
|
||||
for op in operations:
|
||||
if op.operation == OperationType.UPDATE:
|
||||
old_chunks: list[str] = []
|
||||
new_chunks: list[str] = []
|
||||
for hunk in op.hunks:
|
||||
old_lines = [line.content for line in hunk.lines if line.prefix in (" ", "-")]
|
||||
new_lines = [line.content for line in hunk.lines if line.prefix in (" ", "+")]
|
||||
if old_lines or new_lines:
|
||||
old_chunks.append("\n".join(old_lines))
|
||||
new_chunks.append("\n".join(new_lines))
|
||||
|
||||
old_text = "\n...\n".join(chunk for chunk in old_chunks if chunk)
|
||||
new_text = "\n...\n".join(chunk for chunk in new_chunks if chunk)
|
||||
if old_text or new_text:
|
||||
content.append(
|
||||
acp.tool_diff_content(
|
||||
path=op.file_path,
|
||||
old_text=old_text or None,
|
||||
new_text=new_text or "",
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
if op.operation == OperationType.ADD:
|
||||
added_lines = [line.content for hunk in op.hunks for line in hunk.lines if line.prefix == "+"]
|
||||
content.append(
|
||||
acp.tool_diff_content(
|
||||
path=op.file_path,
|
||||
new_text="\n".join(added_lines),
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
if op.operation == OperationType.DELETE:
|
||||
content.append(
|
||||
acp.tool_diff_content(
|
||||
path=op.file_path,
|
||||
old_text=f"Delete file: {op.file_path}",
|
||||
new_text="",
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
if op.operation == OperationType.MOVE:
|
||||
content.append(
|
||||
acp.tool_content(acp.text_block(f"Move file: {op.file_path} -> {op.new_path}"))
|
||||
)
|
||||
|
||||
return content or [acp.tool_content(acp.text_block(patch_text))]
|
||||
except Exception:
|
||||
return [acp.tool_content(acp.text_block(patch_text))]
|
||||
|
||||
|
||||
def _strip_diff_prefix(path: str) -> str:
|
||||
raw = str(path or "").strip()
|
||||
if raw.startswith(("a/", "b/")):
|
||||
return raw[2:]
|
||||
return raw
|
||||
|
||||
|
||||
def _parse_unified_diff_content(diff_text: str) -> List[Any]:
|
||||
"""Convert unified diff text into ACP diff content blocks."""
|
||||
if not diff_text:
|
||||
return []
|
||||
|
||||
content: List[Any] = []
|
||||
current_old_path: Optional[str] = None
|
||||
current_new_path: Optional[str] = None
|
||||
old_lines: list[str] = []
|
||||
new_lines: list[str] = []
|
||||
|
||||
def _flush() -> None:
|
||||
nonlocal current_old_path, current_new_path, old_lines, new_lines
|
||||
if current_old_path is None and current_new_path is None:
|
||||
return
|
||||
path = current_new_path if current_new_path and current_new_path != "/dev/null" else current_old_path
|
||||
if not path or path == "/dev/null":
|
||||
current_old_path = None
|
||||
current_new_path = None
|
||||
old_lines = []
|
||||
new_lines = []
|
||||
return
|
||||
content.append(
|
||||
acp.tool_diff_content(
|
||||
path=_strip_diff_prefix(path),
|
||||
old_text="\n".join(old_lines) if old_lines else None,
|
||||
new_text="\n".join(new_lines),
|
||||
)
|
||||
)
|
||||
current_old_path = None
|
||||
current_new_path = None
|
||||
old_lines = []
|
||||
new_lines = []
|
||||
|
||||
for line in diff_text.splitlines():
|
||||
if line.startswith("--- "):
|
||||
_flush()
|
||||
current_old_path = line[4:].strip()
|
||||
continue
|
||||
if line.startswith("+++ "):
|
||||
current_new_path = line[4:].strip()
|
||||
continue
|
||||
if line.startswith("@@"):
|
||||
continue
|
||||
if current_old_path is None and current_new_path is None:
|
||||
continue
|
||||
if line.startswith("+"):
|
||||
new_lines.append(line[1:])
|
||||
elif line.startswith("-"):
|
||||
old_lines.append(line[1:])
|
||||
elif line.startswith(" "):
|
||||
shared = line[1:]
|
||||
old_lines.append(shared)
|
||||
new_lines.append(shared)
|
||||
|
||||
_flush()
|
||||
return content
|
||||
|
||||
|
||||
def _build_tool_complete_content(
|
||||
tool_name: str,
|
||||
result: Optional[str],
|
||||
*,
|
||||
function_args: Optional[Dict[str, Any]] = None,
|
||||
snapshot: Any = None,
|
||||
) -> List[Any]:
|
||||
"""Build structured ACP completion content, falling back to plain text."""
|
||||
display_result = result or ""
|
||||
if len(display_result) > 5000:
|
||||
display_result = display_result[:4900] + f"\n... ({len(result)} chars total, truncated)"
|
||||
|
||||
if tool_name in {"write_file", "patch", "skill_manage"}:
|
||||
try:
|
||||
from agent.display import extract_edit_diff
|
||||
|
||||
diff_text = extract_edit_diff(
|
||||
tool_name,
|
||||
result,
|
||||
function_args=function_args,
|
||||
snapshot=snapshot,
|
||||
)
|
||||
if isinstance(diff_text, str) and diff_text.strip():
|
||||
diff_content = _parse_unified_diff_content(diff_text)
|
||||
if diff_content:
|
||||
return diff_content
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return [acp.tool_content(acp.text_block(display_result))]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Build ACP content objects for tool-call events
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -284,8 +119,9 @@ def build_tool_start(
|
||||
new = arguments.get("new_string", "")
|
||||
content = [acp.tool_diff_content(path=path, new_text=new, old_text=old)]
|
||||
else:
|
||||
# Patch mode — show the patch content as text
|
||||
patch_text = arguments.get("patch", "")
|
||||
content = _build_patch_mode_content(patch_text)
|
||||
content = [acp.tool_content(acp.text_block(patch_text))]
|
||||
return acp.start_tool_call(
|
||||
tool_call_id, title, kind=kind, content=content, locations=locations,
|
||||
raw_input=arguments,
|
||||
@@ -342,17 +178,16 @@ def build_tool_complete(
|
||||
tool_call_id: str,
|
||||
tool_name: str,
|
||||
result: Optional[str] = None,
|
||||
function_args: Optional[Dict[str, Any]] = None,
|
||||
snapshot: Any = None,
|
||||
) -> ToolCallProgress:
|
||||
"""Create a ToolCallUpdate (progress) event for a completed tool call."""
|
||||
kind = get_tool_kind(tool_name)
|
||||
content = _build_tool_complete_content(
|
||||
tool_name,
|
||||
result,
|
||||
function_args=function_args,
|
||||
snapshot=snapshot,
|
||||
)
|
||||
|
||||
# Truncate very large results for the UI
|
||||
display_result = result or ""
|
||||
if len(display_result) > 5000:
|
||||
display_result = display_result[:4900] + f"\n... ({len(result)} chars total, truncated)"
|
||||
|
||||
content = [acp.tool_content(acp.text_block(display_result))]
|
||||
return acp.update_tool_call(
|
||||
tool_call_id,
|
||||
kind=kind,
|
||||
|
||||
+33
-88
@@ -94,54 +94,6 @@ def _normalize_aux_provider(provider: Optional[str]) -> str:
|
||||
return "custom"
|
||||
return _PROVIDER_ALIASES.get(normalized, normalized)
|
||||
|
||||
|
||||
_FIXED_TEMPERATURE_MODELS: Dict[str, float] = {
|
||||
"kimi-for-coding": 0.6,
|
||||
}
|
||||
|
||||
# Moonshot's kimi-for-coding endpoint (api.kimi.com/coding) documents:
|
||||
# "k2.5 model will use a fixed value 1.0, non-thinking mode will use a fixed
|
||||
# value 0.6. Any other value will result in an error." The same lock applies
|
||||
# to the other k2.* models served on that endpoint. Enumerated explicitly so
|
||||
# non-coding siblings like `kimi-k2-instruct` (variable temperature, served on
|
||||
# the standard chat API and third parties) are NOT clamped.
|
||||
# Source: https://platform.kimi.ai/docs/guide/kimi-k2-5-quickstart
|
||||
_KIMI_INSTANT_MODELS: frozenset = frozenset({
|
||||
"kimi-k2.5",
|
||||
"kimi-k2-turbo-preview",
|
||||
"kimi-k2-0905-preview",
|
||||
})
|
||||
_KIMI_THINKING_MODELS: frozenset = frozenset({
|
||||
"kimi-k2-thinking",
|
||||
"kimi-k2-thinking-turbo",
|
||||
})
|
||||
|
||||
|
||||
def _fixed_temperature_for_model(model: Optional[str]) -> Optional[float]:
|
||||
"""Return a required temperature override for models with strict contracts.
|
||||
|
||||
Moonshot's kimi-for-coding endpoint rejects any non-approved temperature on
|
||||
the k2.5 family. Non-thinking variants require exactly 0.6; thinking
|
||||
variants require 1.0. An optional ``vendor/`` prefix (e.g.
|
||||
``moonshotai/kimi-k2.5``) is tolerated for aggregator routings.
|
||||
|
||||
Returns ``None`` for every other model, including ``kimi-k2-instruct*``
|
||||
which is the separate non-coding K2 family with variable temperature.
|
||||
"""
|
||||
normalized = (model or "").strip().lower()
|
||||
fixed = _FIXED_TEMPERATURE_MODELS.get(normalized)
|
||||
if fixed is not None:
|
||||
logger.debug("Forcing temperature=%s for model %r (fixed map)", fixed, model)
|
||||
return fixed
|
||||
bare = normalized.rsplit("/", 1)[-1]
|
||||
if bare in _KIMI_THINKING_MODELS:
|
||||
logger.debug("Forcing temperature=1.0 for kimi thinking model %r", model)
|
||||
return 1.0
|
||||
if bare in _KIMI_INSTANT_MODELS:
|
||||
logger.debug("Forcing temperature=0.6 for kimi instant model %r", model)
|
||||
return 0.6
|
||||
return None
|
||||
|
||||
# Default auxiliary models for direct API-key providers (cheap/fast for side tasks)
|
||||
_API_KEY_PROVIDER_AUX_MODELS: Dict[str, str] = {
|
||||
"gemini": "gemini-3-flash-preview",
|
||||
@@ -1112,6 +1064,8 @@ _AUTO_PROVIDER_LABELS = {
|
||||
"_resolve_api_key_provider": "api-key",
|
||||
}
|
||||
|
||||
_AGGREGATOR_PROVIDERS = frozenset({"openrouter", "nous"})
|
||||
|
||||
_MAIN_RUNTIME_FIELDS = ("provider", "model", "base_url", "api_key", "api_mode")
|
||||
|
||||
|
||||
@@ -1242,15 +1196,11 @@ def _resolve_auto(main_runtime: Optional[Dict[str, Any]] = None) -> Tuple[Option
|
||||
"""Full auto-detection chain.
|
||||
|
||||
Priority:
|
||||
1. User's main provider + main model, regardless of provider type.
|
||||
This means auxiliary tasks (compression, vision, web extraction,
|
||||
session search, etc.) use the same model the user configured for
|
||||
chat. Users on OpenRouter/Nous get their chosen chat model; users
|
||||
on DeepSeek/ZAI/Alibaba get theirs; etc. Running aux tasks on the
|
||||
user's picked model keeps behavior predictable — no surprise
|
||||
switches to a cheap fallback model for side tasks.
|
||||
2. OpenRouter → Nous → custom → Codex → API-key providers (fallback
|
||||
chain, only used when the main provider has no working client).
|
||||
1. If the user's main provider is NOT an aggregator (OpenRouter / Nous),
|
||||
use their main provider + main model directly. This ensures users on
|
||||
Alibaba, DeepSeek, ZAI, etc. get auxiliary tasks handled by the same
|
||||
provider they already have credentials for — no OpenRouter key needed.
|
||||
2. OpenRouter → Nous → custom → Codex → API-key providers (original chain).
|
||||
"""
|
||||
global auxiliary_is_nous, _stale_base_url_warned
|
||||
auxiliary_is_nous = False # Reset — _try_nous() will set True if it wins
|
||||
@@ -1280,16 +1230,11 @@ def _resolve_auto(main_runtime: Optional[Dict[str, Any]] = None) -> Tuple[Option
|
||||
)
|
||||
_stale_base_url_warned = True
|
||||
|
||||
# ── Step 1: main provider + main model → use them directly ──
|
||||
#
|
||||
# This is the primary aux backend for every user. "auto" means
|
||||
# "use my main chat model for side tasks as well" — including users
|
||||
# on aggregators (OpenRouter, Nous) who previously got routed to a
|
||||
# cheap provider-side default. Explicit per-task overrides set via
|
||||
# config.yaml (auxiliary.<task>.provider) still win over this.
|
||||
# ── Step 1: non-aggregator main provider → use main model directly ──
|
||||
main_provider = runtime_provider or _read_main_provider()
|
||||
main_model = runtime_model or _read_main_model()
|
||||
if (main_provider and main_model
|
||||
and main_provider not in _AGGREGATOR_PROVIDERS
|
||||
and main_provider not in ("auto", "")):
|
||||
resolved_provider = main_provider
|
||||
explicit_base_url = None
|
||||
@@ -1648,6 +1593,7 @@ def resolve_provider_client(
|
||||
from hermes_cli.models import copilot_default_headers
|
||||
|
||||
headers.update(copilot_default_headers())
|
||||
|
||||
client = OpenAI(api_key=api_key, base_url=base_url,
|
||||
**({"default_headers": headers} if headers else {}))
|
||||
|
||||
@@ -1871,31 +1817,34 @@ def resolve_vision_provider_client(
|
||||
|
||||
if requested == "auto":
|
||||
# Vision auto-detection order:
|
||||
# 1. User's main provider + main model (including aggregators).
|
||||
# _PROVIDER_VISION_MODELS provides per-provider vision model
|
||||
# overrides when the provider has a dedicated multimodal model
|
||||
# that differs from the chat model (e.g. xiaomi → mimo-v2-omni,
|
||||
# zai → glm-5v-turbo).
|
||||
# 2. OpenRouter (vision-capable aggregator fallback)
|
||||
# 3. Nous Portal (vision-capable aggregator fallback)
|
||||
# 1. Active provider + model (user's main chat config)
|
||||
# 2. OpenRouter (known vision-capable default model)
|
||||
# 3. Nous Portal (known vision-capable default model)
|
||||
# 4. Stop
|
||||
main_provider = _read_main_provider()
|
||||
main_model = _read_main_model()
|
||||
if main_provider and main_provider not in ("auto", ""):
|
||||
vision_model = _PROVIDER_VISION_MODELS.get(main_provider, main_model)
|
||||
rpc_client, rpc_model = resolve_provider_client(
|
||||
main_provider, vision_model,
|
||||
api_mode=resolved_api_mode)
|
||||
if rpc_client is not None:
|
||||
logger.info(
|
||||
"Vision auto-detect: using main provider %s (%s)",
|
||||
main_provider, rpc_model or vision_model,
|
||||
)
|
||||
return _finalize(
|
||||
main_provider, rpc_client, rpc_model or vision_model)
|
||||
if main_provider in _VISION_AUTO_PROVIDER_ORDER:
|
||||
# Known strict backend — use its defaults.
|
||||
sync_client, default_model = _resolve_strict_vision_backend(main_provider)
|
||||
if sync_client is not None:
|
||||
return _finalize(main_provider, sync_client, default_model)
|
||||
else:
|
||||
# Exotic provider (DeepSeek, Alibaba, Xiaomi, named custom, etc.)
|
||||
# Use provider-specific vision model if available, otherwise main model.
|
||||
vision_model = _PROVIDER_VISION_MODELS.get(main_provider, main_model)
|
||||
rpc_client, rpc_model = resolve_provider_client(
|
||||
main_provider, vision_model,
|
||||
api_mode=resolved_api_mode)
|
||||
if rpc_client is not None:
|
||||
logger.info(
|
||||
"Vision auto-detect: using active provider %s (%s)",
|
||||
main_provider, rpc_model or vision_model,
|
||||
)
|
||||
return _finalize(
|
||||
main_provider, rpc_client, rpc_model or vision_model)
|
||||
|
||||
# Fall back through aggregators (uses their dedicated vision model,
|
||||
# not the user's main model) when main provider has no client.
|
||||
# Fall back through aggregators.
|
||||
for candidate in _VISION_AUTO_PROVIDER_ORDER:
|
||||
if candidate == main_provider:
|
||||
continue # already tried above
|
||||
@@ -2344,10 +2293,6 @@ def _build_call_kwargs(
|
||||
"timeout": timeout,
|
||||
}
|
||||
|
||||
fixed_temperature = _fixed_temperature_for_model(model)
|
||||
if fixed_temperature is not None:
|
||||
temperature = fixed_temperature
|
||||
|
||||
# Opus 4.7+ rejects any non-default temperature/top_p/top_k — silently
|
||||
# drop here so auxiliary callers that hardcode temperature (e.g. 0.3 on
|
||||
# flush_memories, 0 on structured-JSON extraction) don't 400 the moment
|
||||
|
||||
@@ -63,52 +63,6 @@ _CHARS_PER_TOKEN = 4
|
||||
_SUMMARY_FAILURE_COOLDOWN_SECONDS = 600
|
||||
|
||||
|
||||
def _truncate_tool_call_args_json(args: str, head_chars: int = 200) -> str:
|
||||
"""Shrink long string values inside a tool-call arguments JSON blob while
|
||||
preserving JSON validity.
|
||||
|
||||
The ``function.arguments`` field on a tool call is a JSON-encoded string
|
||||
passed through to the LLM provider; downstream providers strictly
|
||||
validate it and return a non-retryable 400 when it is not well-formed.
|
||||
An earlier implementation sliced the raw JSON at a fixed byte offset and
|
||||
appended ``...[truncated]`` — which routinely produced strings like::
|
||||
|
||||
{"path": "/foo/bar", "content": "# long markdown
|
||||
...[truncated]
|
||||
|
||||
i.e. an unterminated string and a missing closing brace. MiniMax, for
|
||||
example, rejects this with ``invalid function arguments json string``
|
||||
and the session gets stuck re-sending the same broken history on every
|
||||
turn. See issue #11762 for the observed loop.
|
||||
|
||||
This helper parses the arguments, shrinks long string leaves inside the
|
||||
parsed structure, and re-serialises. Non-string values (paths, ints,
|
||||
booleans) are preserved intact. If the arguments are not valid JSON
|
||||
to begin with — some model backends use non-JSON tool arguments — the
|
||||
original string is returned unchanged rather than replaced with
|
||||
something neither we nor the backend can parse.
|
||||
"""
|
||||
try:
|
||||
parsed = json.loads(args)
|
||||
except (ValueError, TypeError):
|
||||
return args
|
||||
|
||||
def _shrink(obj: Any) -> Any:
|
||||
if isinstance(obj, str):
|
||||
if len(obj) > head_chars:
|
||||
return obj[:head_chars] + "...[truncated]"
|
||||
return obj
|
||||
if isinstance(obj, dict):
|
||||
return {k: _shrink(v) for k, v in obj.items()}
|
||||
if isinstance(obj, list):
|
||||
return [_shrink(v) for v in obj]
|
||||
return obj
|
||||
|
||||
shrunken = _shrink(parsed)
|
||||
# ensure_ascii=False preserves CJK/emoji instead of bloating with \uXXXX
|
||||
return json.dumps(shrunken, ensure_ascii=False)
|
||||
|
||||
|
||||
def _summarize_tool_result(tool_name: str, tool_args: str, tool_content: str) -> str:
|
||||
"""Create an informative 1-line summary of a tool call + result.
|
||||
|
||||
@@ -495,11 +449,6 @@ class ContextCompressor(ContextEngine):
|
||||
# Pass 3: Truncate large tool_call arguments in assistant messages
|
||||
# outside the protected tail. write_file with 50KB content, for
|
||||
# example, survives pruning entirely without this.
|
||||
#
|
||||
# The shrinking is done inside the parsed JSON structure so the
|
||||
# result remains valid JSON — otherwise downstream providers 400
|
||||
# on every subsequent turn until the broken call falls out of
|
||||
# the window. See ``_truncate_tool_call_args_json`` docstring.
|
||||
for i in range(prune_boundary):
|
||||
msg = result[i]
|
||||
if msg.get("role") != "assistant" or not msg.get("tool_calls"):
|
||||
@@ -510,10 +459,8 @@ class ContextCompressor(ContextEngine):
|
||||
if isinstance(tc, dict):
|
||||
args = tc.get("function", {}).get("arguments", "")
|
||||
if len(args) > 500:
|
||||
new_args = _truncate_tool_call_args_json(args)
|
||||
if new_args != args:
|
||||
tc = {**tc, "function": {**tc["function"], "arguments": new_args}}
|
||||
modified = True
|
||||
tc = {**tc, "function": {**tc["function"], "arguments": args[:200] + "...[truncated]"}}
|
||||
modified = True
|
||||
new_tcs.append(tc)
|
||||
if modified:
|
||||
result[i] = {**msg, "tool_calls": new_tcs}
|
||||
|
||||
+120
-28
@@ -22,6 +22,8 @@ from hermes_cli.auth import (
|
||||
_auth_store_lock,
|
||||
_codex_access_token_is_expiring,
|
||||
_decode_jwt_claims,
|
||||
_import_codex_cli_tokens,
|
||||
_write_codex_cli_tokens,
|
||||
_load_auth_store,
|
||||
_load_provider_state,
|
||||
_resolve_kimi_base_url,
|
||||
@@ -455,6 +457,39 @@ class CredentialPool:
|
||||
logger.debug("Failed to sync from credentials file: %s", exc)
|
||||
return entry
|
||||
|
||||
def _sync_codex_entry_from_cli(self, entry: PooledCredential) -> PooledCredential:
|
||||
"""Sync an openai-codex pool entry from ~/.codex/auth.json if tokens differ.
|
||||
|
||||
OpenAI OAuth refresh tokens are single-use and rotate on every refresh.
|
||||
When the Codex CLI (or another Hermes profile) refreshes its token,
|
||||
the pool entry's refresh_token becomes stale. This method detects that
|
||||
by comparing against ~/.codex/auth.json and syncing the fresh pair.
|
||||
"""
|
||||
if self.provider != "openai-codex":
|
||||
return entry
|
||||
try:
|
||||
cli_tokens = _import_codex_cli_tokens()
|
||||
if not cli_tokens:
|
||||
return entry
|
||||
cli_refresh = cli_tokens.get("refresh_token", "")
|
||||
cli_access = cli_tokens.get("access_token", "")
|
||||
if cli_refresh and cli_refresh != entry.refresh_token:
|
||||
logger.debug("Pool entry %s: syncing tokens from ~/.codex/auth.json (refresh token changed)", entry.id)
|
||||
updated = replace(
|
||||
entry,
|
||||
access_token=cli_access,
|
||||
refresh_token=cli_refresh,
|
||||
last_status=None,
|
||||
last_status_at=None,
|
||||
last_error_code=None,
|
||||
)
|
||||
self._replace_entry(entry, updated)
|
||||
self._persist()
|
||||
return updated
|
||||
except Exception as exc:
|
||||
logger.debug("Failed to sync from ~/.codex/auth.json: %s", exc)
|
||||
return entry
|
||||
|
||||
def _sync_device_code_entry_to_auth_store(self, entry: PooledCredential) -> None:
|
||||
"""Write refreshed pool entry tokens back to auth.json providers.
|
||||
|
||||
@@ -550,6 +585,13 @@ class CredentialPool:
|
||||
except Exception as wexc:
|
||||
logger.debug("Failed to write refreshed token to credentials file: %s", wexc)
|
||||
elif self.provider == "openai-codex":
|
||||
# Proactively sync from ~/.codex/auth.json before refresh.
|
||||
# The Codex CLI (or another Hermes profile) may have already
|
||||
# consumed our refresh_token. Syncing first avoids a
|
||||
# "refresh_token_reused" error when the CLI has a newer pair.
|
||||
synced = self._sync_codex_entry_from_cli(entry)
|
||||
if synced is not entry:
|
||||
entry = synced
|
||||
refreshed = auth_mod.refresh_codex_oauth_pure(
|
||||
entry.access_token,
|
||||
entry.refresh_token,
|
||||
@@ -635,6 +677,45 @@ class CredentialPool:
|
||||
# Credentials file had a valid (non-expired) token — use it directly
|
||||
logger.debug("Credentials file has valid token, using without refresh")
|
||||
return synced
|
||||
# For openai-codex: the refresh_token may have been consumed by
|
||||
# the Codex CLI between our proactive sync and the refresh call.
|
||||
# Re-sync and retry once.
|
||||
if self.provider == "openai-codex":
|
||||
synced = self._sync_codex_entry_from_cli(entry)
|
||||
if synced.refresh_token != entry.refresh_token:
|
||||
logger.debug("Retrying Codex refresh with synced token from ~/.codex/auth.json")
|
||||
try:
|
||||
refreshed = auth_mod.refresh_codex_oauth_pure(
|
||||
synced.access_token,
|
||||
synced.refresh_token,
|
||||
)
|
||||
updated = replace(
|
||||
synced,
|
||||
access_token=refreshed["access_token"],
|
||||
refresh_token=refreshed["refresh_token"],
|
||||
last_refresh=refreshed.get("last_refresh"),
|
||||
last_status=STATUS_OK,
|
||||
last_status_at=None,
|
||||
last_error_code=None,
|
||||
)
|
||||
self._replace_entry(synced, updated)
|
||||
self._persist()
|
||||
self._sync_device_code_entry_to_auth_store(updated)
|
||||
try:
|
||||
_write_codex_cli_tokens(
|
||||
updated.access_token,
|
||||
updated.refresh_token,
|
||||
last_refresh=updated.last_refresh,
|
||||
)
|
||||
except Exception as wexc:
|
||||
logger.debug("Failed to write refreshed Codex tokens to CLI file (retry): %s", wexc)
|
||||
return updated
|
||||
except Exception as retry_exc:
|
||||
logger.debug("Codex retry refresh also failed: %s", retry_exc)
|
||||
elif not self._entry_needs_refresh(synced):
|
||||
logger.debug("Codex CLI has valid token, using without refresh")
|
||||
self._sync_device_code_entry_to_auth_store(synced)
|
||||
return synced
|
||||
self._mark_exhausted(entry, None)
|
||||
return None
|
||||
|
||||
@@ -653,6 +734,17 @@ class CredentialPool:
|
||||
# _seed_from_singletons() on the next load_pool() sees fresh state
|
||||
# instead of re-seeding stale/consumed tokens.
|
||||
self._sync_device_code_entry_to_auth_store(updated)
|
||||
# Write refreshed tokens back to ~/.codex/auth.json so Codex CLI
|
||||
# and VS Code don't hit "refresh_token_reused" on their next refresh.
|
||||
if self.provider == "openai-codex":
|
||||
try:
|
||||
_write_codex_cli_tokens(
|
||||
updated.access_token,
|
||||
updated.refresh_token,
|
||||
last_refresh=updated.last_refresh,
|
||||
)
|
||||
except Exception as wexc:
|
||||
logger.debug("Failed to write refreshed Codex tokens to CLI file: %s", wexc)
|
||||
return updated
|
||||
|
||||
def _entry_needs_refresh(self, entry: PooledCredential) -> bool:
|
||||
@@ -698,6 +790,16 @@ class CredentialPool:
|
||||
if synced is not entry:
|
||||
entry = synced
|
||||
cleared_any = True
|
||||
# For openai-codex entries, sync from ~/.codex/auth.json before
|
||||
# any status/refresh checks. This picks up tokens refreshed by
|
||||
# the Codex CLI or another Hermes profile.
|
||||
if (self.provider == "openai-codex"
|
||||
and entry.last_status == STATUS_EXHAUSTED
|
||||
and entry.refresh_token):
|
||||
synced = self._sync_codex_entry_from_cli(entry)
|
||||
if synced is not entry:
|
||||
entry = synced
|
||||
cleared_any = True
|
||||
if entry.last_status == STATUS_EXHAUSTED:
|
||||
exhausted_until = _exhausted_until(entry)
|
||||
if exhausted_until is not None and now < exhausted_until:
|
||||
@@ -1028,14 +1130,6 @@ def _seed_from_singletons(provider: str, entries: List[PooledCredential]) -> Tup
|
||||
state = _load_provider_state(auth_store, "nous")
|
||||
if state:
|
||||
active_sources.add("device_code")
|
||||
# Prefer a user-supplied label embedded in the singleton state
|
||||
# (set by persist_nous_credentials(label=...) when the user ran
|
||||
# `hermes auth add nous --label <name>`). Fall back to the
|
||||
# auto-derived token fingerprint for logins that didn't supply one.
|
||||
custom_label = str(state.get("label") or "").strip()
|
||||
seeded_label = custom_label or label_from_token(
|
||||
state.get("access_token", ""), "device_code"
|
||||
)
|
||||
changed |= _upsert_entry(
|
||||
entries,
|
||||
provider,
|
||||
@@ -1054,7 +1148,7 @@ def _seed_from_singletons(provider: str, entries: List[PooledCredential]) -> Tup
|
||||
"agent_key": state.get("agent_key"),
|
||||
"agent_key_expires_at": state.get("agent_key_expires_at"),
|
||||
"tls": state.get("tls") if isinstance(state.get("tls"), dict) else None,
|
||||
"label": seeded_label,
|
||||
"label": label_from_token(state.get("access_token", ""), "device_code"),
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1114,27 +1208,25 @@ def _seed_from_singletons(provider: str, entries: List[PooledCredential]) -> Tup
|
||||
logger.debug("Qwen OAuth token seed failed: %s", exc)
|
||||
|
||||
elif provider == "openai-codex":
|
||||
# Respect user suppression — `hermes auth remove openai-codex` marks
|
||||
# the device_code source as suppressed so it won't be re-seeded from
|
||||
# the Hermes auth store. Without this gate the removal is instantly
|
||||
# undone on the next load_pool() call.
|
||||
codex_suppressed = False
|
||||
try:
|
||||
from hermes_cli.auth import is_source_suppressed
|
||||
codex_suppressed = is_source_suppressed(provider, "device_code")
|
||||
except ImportError:
|
||||
pass
|
||||
if codex_suppressed:
|
||||
return changed, active_sources
|
||||
|
||||
state = _load_provider_state(auth_store, "openai-codex")
|
||||
tokens = state.get("tokens") if isinstance(state, dict) else None
|
||||
# Hermes owns its own Codex auth state — we do NOT auto-import from
|
||||
# ~/.codex/auth.json at pool-load time. OAuth refresh tokens are
|
||||
# single-use, so sharing them with Codex CLI / VS Code causes
|
||||
# refresh_token_reused race failures. Users who want to adopt
|
||||
# existing Codex CLI credentials get a one-time, explicit prompt
|
||||
# via `hermes auth openai-codex`.
|
||||
# Fallback: import from Codex CLI (~/.codex/auth.json) if Hermes auth
|
||||
# store has no tokens. This mirrors resolve_codex_runtime_credentials()
|
||||
# so that load_pool() and list_authenticated_providers() detect tokens
|
||||
# that only exist in the Codex CLI shared file.
|
||||
if not (isinstance(tokens, dict) and tokens.get("access_token")):
|
||||
try:
|
||||
from hermes_cli.auth import _import_codex_cli_tokens, _save_codex_tokens
|
||||
cli_tokens = _import_codex_cli_tokens()
|
||||
if cli_tokens:
|
||||
logger.info("Importing Codex CLI tokens into Hermes auth store.")
|
||||
_save_codex_tokens(cli_tokens)
|
||||
# Re-read state after import
|
||||
auth_store = _load_auth_store()
|
||||
state = _load_provider_state(auth_store, "openai-codex")
|
||||
tokens = state.get("tokens") if isinstance(state, dict) else None
|
||||
except Exception as exc:
|
||||
logger.debug("Codex CLI token import failed: %s", exc)
|
||||
if isinstance(tokens, dict) and tokens.get("access_token"):
|
||||
active_sources.add("device_code")
|
||||
changed |= _upsert_entry(
|
||||
|
||||
@@ -747,149 +747,18 @@ class GeminiCloudCodeClient:
|
||||
|
||||
|
||||
def _gemini_http_error(response: httpx.Response) -> CodeAssistError:
|
||||
"""Translate an httpx response into a CodeAssistError with rich metadata.
|
||||
|
||||
Parses Google's error envelope (``{"error": {"code", "message", "status",
|
||||
"details": [...]}}``) so the agent's error classifier can reason about
|
||||
the failure — ``status_code`` enables the rate_limit / auth classification
|
||||
paths, and ``response`` lets the main loop honor ``Retry-After`` just
|
||||
like it does for OpenAI SDK exceptions.
|
||||
|
||||
Also lifts a few recognizable Google conditions into human-readable
|
||||
messages so the user sees something better than a 500-char JSON dump:
|
||||
|
||||
MODEL_CAPACITY_EXHAUSTED → "Gemini model capacity exhausted for
|
||||
<model>. This is a Google-side throttle..."
|
||||
RESOURCE_EXHAUSTED w/o reason → quota-style message
|
||||
404 → "Model <name> not found at cloudcode-pa..."
|
||||
"""
|
||||
status = response.status_code
|
||||
|
||||
# Parse the body once, surviving any weird encodings.
|
||||
body_text = ""
|
||||
body_json: Dict[str, Any] = {}
|
||||
try:
|
||||
body_text = response.text
|
||||
body = response.text[:500]
|
||||
except Exception:
|
||||
body_text = ""
|
||||
if body_text:
|
||||
try:
|
||||
parsed = json.loads(body_text)
|
||||
if isinstance(parsed, dict):
|
||||
body_json = parsed
|
||||
except (ValueError, TypeError):
|
||||
body_json = {}
|
||||
|
||||
# Dig into Google's error envelope. Shape is:
|
||||
# {"error": {"code": 429, "message": "...", "status": "RESOURCE_EXHAUSTED",
|
||||
# "details": [{"@type": ".../ErrorInfo", "reason": "MODEL_CAPACITY_EXHAUSTED",
|
||||
# "metadata": {...}},
|
||||
# {"@type": ".../RetryInfo", "retryDelay": "30s"}]}}
|
||||
err_obj = body_json.get("error") if isinstance(body_json, dict) else None
|
||||
if not isinstance(err_obj, dict):
|
||||
err_obj = {}
|
||||
err_status = str(err_obj.get("status") or "").strip()
|
||||
err_message = str(err_obj.get("message") or "").strip()
|
||||
err_details_list = err_obj.get("details") if isinstance(err_obj.get("details"), list) else []
|
||||
|
||||
# Extract google.rpc.ErrorInfo reason + metadata. There may be more
|
||||
# than one ErrorInfo (rare), so we pick the first one with a reason.
|
||||
error_reason = ""
|
||||
error_metadata: Dict[str, Any] = {}
|
||||
retry_delay_seconds: Optional[float] = None
|
||||
for detail in err_details_list:
|
||||
if not isinstance(detail, dict):
|
||||
continue
|
||||
type_url = str(detail.get("@type") or "")
|
||||
if not error_reason and type_url.endswith("/google.rpc.ErrorInfo"):
|
||||
reason = detail.get("reason")
|
||||
if isinstance(reason, str) and reason:
|
||||
error_reason = reason
|
||||
md = detail.get("metadata")
|
||||
if isinstance(md, dict):
|
||||
error_metadata = md
|
||||
elif retry_delay_seconds is None and type_url.endswith("/google.rpc.RetryInfo"):
|
||||
# retryDelay is a google.protobuf.Duration string like "30s" or "1.5s".
|
||||
delay_raw = detail.get("retryDelay")
|
||||
if isinstance(delay_raw, str) and delay_raw.endswith("s"):
|
||||
try:
|
||||
retry_delay_seconds = float(delay_raw[:-1])
|
||||
except ValueError:
|
||||
pass
|
||||
elif isinstance(delay_raw, (int, float)):
|
||||
retry_delay_seconds = float(delay_raw)
|
||||
|
||||
# Fall back to the Retry-After header if the body didn't include RetryInfo.
|
||||
if retry_delay_seconds is None:
|
||||
try:
|
||||
header_val = response.headers.get("Retry-After") or response.headers.get("retry-after")
|
||||
except Exception:
|
||||
header_val = None
|
||||
if header_val:
|
||||
try:
|
||||
retry_delay_seconds = float(header_val)
|
||||
except (TypeError, ValueError):
|
||||
retry_delay_seconds = None
|
||||
|
||||
# Classify the error code. ``code_assist_rate_limited`` stays the default
|
||||
# for 429s; a more specific reason tag helps downstream callers (e.g. tests,
|
||||
# logs) without changing the rate_limit classification path.
|
||||
body = ""
|
||||
# Let run_agent's retry logic see auth errors as rotatable via `api_key`
|
||||
code = f"code_assist_http_{status}"
|
||||
if status == 401:
|
||||
code = "code_assist_unauthorized"
|
||||
elif status == 429:
|
||||
code = "code_assist_rate_limited"
|
||||
if error_reason == "MODEL_CAPACITY_EXHAUSTED":
|
||||
code = "code_assist_capacity_exhausted"
|
||||
|
||||
# Build a human-readable message. Keep the status + a raw-body tail for
|
||||
# debugging, but lead with a friendlier summary when we recognize the
|
||||
# Google signal.
|
||||
model_hint = ""
|
||||
if isinstance(error_metadata, dict):
|
||||
model_hint = str(error_metadata.get("model") or error_metadata.get("modelId") or "").strip()
|
||||
|
||||
if status == 429 and error_reason == "MODEL_CAPACITY_EXHAUSTED":
|
||||
target = model_hint or "this Gemini model"
|
||||
message = (
|
||||
f"Gemini capacity exhausted for {target} (Google-side throttle, "
|
||||
f"not a Hermes issue). Try a different Gemini model or set a "
|
||||
f"fallback_providers entry to a non-Gemini provider."
|
||||
)
|
||||
if retry_delay_seconds is not None:
|
||||
message += f" Google suggests retrying in {retry_delay_seconds:g}s."
|
||||
elif status == 429 and err_status == "RESOURCE_EXHAUSTED":
|
||||
message = (
|
||||
f"Gemini quota exhausted ({err_message or 'RESOURCE_EXHAUSTED'}). "
|
||||
f"Check /gquota for remaining daily requests."
|
||||
)
|
||||
if retry_delay_seconds is not None:
|
||||
message += f" Retry suggested in {retry_delay_seconds:g}s."
|
||||
elif status == 404:
|
||||
# Google returns 404 when a model has been retired or renamed.
|
||||
target = model_hint or (err_message or "model")
|
||||
message = (
|
||||
f"Code Assist 404: {target} is not available at "
|
||||
f"cloudcode-pa.googleapis.com. It may have been renamed or "
|
||||
f"retired. Check hermes_cli/models.py for the current list."
|
||||
)
|
||||
elif err_message:
|
||||
# Generic fallback with the parsed message.
|
||||
message = f"Code Assist HTTP {status} ({err_status or 'error'}): {err_message}"
|
||||
else:
|
||||
# Last-ditch fallback — raw body snippet.
|
||||
message = f"Code Assist returned HTTP {status}: {body_text[:500]}"
|
||||
|
||||
return CodeAssistError(
|
||||
message,
|
||||
f"Code Assist returned HTTP {status}: {body}",
|
||||
code=code,
|
||||
status_code=status,
|
||||
response=response,
|
||||
retry_after=retry_delay_seconds,
|
||||
details={
|
||||
"status": err_status,
|
||||
"reason": error_reason,
|
||||
"metadata": error_metadata,
|
||||
"message": err_message,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -68,45 +68,9 @@ _ONBOARDING_POLL_INTERVAL_SECONDS = 5.0
|
||||
|
||||
|
||||
class CodeAssistError(RuntimeError):
|
||||
"""Exception raised by the Code Assist (``cloudcode-pa``) integration.
|
||||
|
||||
Carries HTTP status / response / retry-after metadata so the agent's
|
||||
``error_classifier._extract_status_code`` and the main loop's Retry-After
|
||||
handling (which walks ``error.response.headers``) pick up the right
|
||||
signals. Without these, 429s from the OAuth path look like opaque
|
||||
``RuntimeError`` and skip the rate-limit path.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
*,
|
||||
code: str = "code_assist_error",
|
||||
status_code: Optional[int] = None,
|
||||
response: Any = None,
|
||||
retry_after: Optional[float] = None,
|
||||
details: Optional[Dict[str, Any]] = None,
|
||||
) -> None:
|
||||
def __init__(self, message: str, *, code: str = "code_assist_error") -> None:
|
||||
super().__init__(message)
|
||||
self.code = code
|
||||
# ``status_code`` is picked up by ``agent.error_classifier._extract_status_code``
|
||||
# so a 429 from Code Assist classifies as FailoverReason.rate_limit and
|
||||
# triggers the main loop's fallback_providers chain the same way SDK
|
||||
# errors do.
|
||||
self.status_code = status_code
|
||||
# ``response`` is the underlying ``httpx.Response`` (or a shim with a
|
||||
# ``.headers`` mapping and ``.json()`` method). The main loop reads
|
||||
# ``error.response.headers["Retry-After"]`` to honor Google's retry
|
||||
# hints when the backend throttles us.
|
||||
self.response = response
|
||||
# Parsed ``Retry-After`` seconds (kept separately for convenience —
|
||||
# Google returns retry hints in both the header and the error body's
|
||||
# ``google.rpc.RetryInfo`` details, and we pick whichever we found).
|
||||
self.retry_after = retry_after
|
||||
# Parsed structured error details from the Google error envelope
|
||||
# (e.g. ``{"reason": "MODEL_CAPACITY_EXHAUSTED", "status": "RESOURCE_EXHAUSTED"}``).
|
||||
# Useful for logging and for tests that want to assert on specifics.
|
||||
self.details = details or {}
|
||||
|
||||
|
||||
class ProjectIdRequiredError(CodeAssistError):
|
||||
|
||||
@@ -38,7 +38,6 @@ _PROVIDER_PREFIXES: frozenset[str] = frozenset({
|
||||
"mimo", "xiaomi-mimo",
|
||||
"arcee-ai", "arceeai",
|
||||
"xai", "x-ai", "x.ai", "grok",
|
||||
"nvidia", "nim", "nvidia-nim", "nemotron",
|
||||
"qwen-portal",
|
||||
})
|
||||
|
||||
@@ -125,6 +124,7 @@ DEFAULT_CONTEXT_LENGTHS = {
|
||||
"gemini": 1048576,
|
||||
# Gemma (open models served via AI Studio)
|
||||
"gemma-4-31b": 256000,
|
||||
"gemma-4-26b": 256000,
|
||||
"gemma-3": 131072,
|
||||
"gemma": 8192, # fallback for older gemma models
|
||||
# DeepSeek
|
||||
@@ -158,8 +158,6 @@ DEFAULT_CONTEXT_LENGTHS = {
|
||||
"grok": 131072, # catch-all (grok-beta, unknown grok-*)
|
||||
# Kimi
|
||||
"kimi": 262144,
|
||||
# Nemotron — NVIDIA's open-weights series (128K context across all sizes)
|
||||
"nemotron": 131072,
|
||||
# Arcee
|
||||
"trinity": 262144,
|
||||
# OpenRouter
|
||||
@@ -242,7 +240,6 @@ _URL_TO_PROVIDER: Dict[str, str] = {
|
||||
"api.fireworks.ai": "fireworks",
|
||||
"opencode.ai": "opencode-go",
|
||||
"api.x.ai": "xai",
|
||||
"integrate.api.nvidia.com": "nvidia",
|
||||
"api.xiaomimimo.com": "xiaomi",
|
||||
"xiaomimimo.com": "xiaomi",
|
||||
"ollama.com": "ollama-cloud",
|
||||
|
||||
+3
-43
@@ -420,10 +420,7 @@ def list_provider_models(provider: str) -> List[str]:
|
||||
models = _get_provider_models(provider)
|
||||
if models is None:
|
||||
return []
|
||||
return [
|
||||
mid for mid in models.keys()
|
||||
if not _should_hide_from_provider_catalog(provider, mid)
|
||||
]
|
||||
return list(models.keys())
|
||||
|
||||
|
||||
# Patterns that indicate non-agentic or noise models (TTS, embedding,
|
||||
@@ -435,43 +432,6 @@ _NOISE_PATTERNS: re.Pattern = re.compile(
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
# Google's live Gemini catalogs currently include a mix of stale slugs and
|
||||
# Gemma models whose TPM quotas are too small for normal Hermes agent traffic.
|
||||
# Keep capability metadata available for direct/manual use, but hide these from
|
||||
# the Gemini model catalogs we surface in setup and model selection.
|
||||
_GOOGLE_HIDDEN_MODELS = frozenset({
|
||||
# Low-TPM Gemma models that trip Google input-token quota walls under
|
||||
# agent-style traffic despite advertising large context windows.
|
||||
"gemma-4-31b-it",
|
||||
"gemma-4-26b-it",
|
||||
"gemma-4-26b-a4b-it",
|
||||
"gemma-3-1b",
|
||||
"gemma-3-1b-it",
|
||||
"gemma-3-2b",
|
||||
"gemma-3-2b-it",
|
||||
"gemma-3-4b",
|
||||
"gemma-3-4b-it",
|
||||
"gemma-3-12b",
|
||||
"gemma-3-12b-it",
|
||||
"gemma-3-27b",
|
||||
"gemma-3-27b-it",
|
||||
# Stale/retired Google slugs that still surface through models.dev-backed
|
||||
# Gemini selection but 404 on the current Google endpoints.
|
||||
"gemini-1.5-flash",
|
||||
"gemini-1.5-pro",
|
||||
"gemini-1.5-flash-8b",
|
||||
"gemini-2.0-flash",
|
||||
"gemini-2.0-flash-lite",
|
||||
})
|
||||
|
||||
|
||||
def _should_hide_from_provider_catalog(provider: str, model_id: str) -> bool:
|
||||
provider_lower = (provider or "").strip().lower()
|
||||
model_lower = (model_id or "").strip().lower()
|
||||
if provider_lower in {"gemini", "google"} and model_lower in _GOOGLE_HIDDEN_MODELS:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def list_agentic_models(provider: str) -> List[str]:
|
||||
"""Return model IDs suitable for agentic use from models.dev.
|
||||
@@ -488,8 +448,6 @@ def list_agentic_models(provider: str) -> List[str]:
|
||||
for mid, entry in models.items():
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
if _should_hide_from_provider_catalog(provider, mid):
|
||||
continue
|
||||
if not entry.get("tool_call", False):
|
||||
continue
|
||||
if _NOISE_PATTERNS.search(mid):
|
||||
@@ -624,3 +582,5 @@ def get_model_info(
|
||||
return _parse_model_info(mid, mdata, mdev_id)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
|
||||
@@ -654,7 +654,7 @@ def build_skills_system_prompt(
|
||||
):
|
||||
continue
|
||||
skills_by_category.setdefault(category, []).append(
|
||||
(frontmatter_name, entry.get("description", ""))
|
||||
(skill_name, entry.get("description", ""))
|
||||
)
|
||||
category_descriptions = {
|
||||
str(k): str(v)
|
||||
@@ -679,7 +679,7 @@ def build_skills_system_prompt(
|
||||
):
|
||||
continue
|
||||
skills_by_category.setdefault(entry["category"], []).append(
|
||||
(entry["frontmatter_name"], entry["description"])
|
||||
(skill_name, entry["description"])
|
||||
)
|
||||
|
||||
# Read category-level DESCRIPTION.md files
|
||||
@@ -722,10 +722,9 @@ def build_skills_system_prompt(
|
||||
continue
|
||||
entry = _build_snapshot_entry(skill_file, ext_dir, frontmatter, desc)
|
||||
skill_name = entry["skill_name"]
|
||||
frontmatter_name = entry["frontmatter_name"]
|
||||
if frontmatter_name in seen_skill_names:
|
||||
if skill_name in seen_skill_names:
|
||||
continue
|
||||
if frontmatter_name in disabled or skill_name in disabled:
|
||||
if entry["frontmatter_name"] in disabled or skill_name in disabled:
|
||||
continue
|
||||
if not _skill_should_show(
|
||||
extract_skill_conditions(frontmatter),
|
||||
@@ -733,9 +732,9 @@ def build_skills_system_prompt(
|
||||
available_toolsets,
|
||||
):
|
||||
continue
|
||||
seen_skill_names.add(frontmatter_name)
|
||||
seen_skill_names.add(skill_name)
|
||||
skills_by_category.setdefault(entry["category"], []).append(
|
||||
(frontmatter_name, entry["description"])
|
||||
(skill_name, entry["description"])
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug("Error reading external skill %s: %s", skill_file, e)
|
||||
|
||||
@@ -24,7 +24,6 @@ model:
|
||||
# "minimax" - MiniMax global (requires: MINIMAX_API_KEY)
|
||||
# "minimax-cn" - MiniMax China (requires: MINIMAX_CN_API_KEY)
|
||||
# "huggingface" - Hugging Face Inference (requires: HF_TOKEN)
|
||||
# "nvidia" - NVIDIA NIM / build.nvidia.com (requires: NVIDIA_API_KEY)
|
||||
# "xiaomi" - Xiaomi MiMo (requires: XIAOMI_API_KEY)
|
||||
# "arcee" - Arcee AI Trinity models (requires: ARCEEAI_API_KEY)
|
||||
# "ollama-cloud" - Ollama Cloud (requires: OLLAMA_API_KEY — https://ollama.com/settings)
|
||||
|
||||
@@ -18,8 +18,6 @@ import os
|
||||
import shutil
|
||||
import sys
|
||||
import json
|
||||
import re
|
||||
import base64
|
||||
import atexit
|
||||
import tempfile
|
||||
import time
|
||||
@@ -80,76 +78,6 @@ _project_env = Path(__file__).parent / '.env'
|
||||
load_hermes_dotenv(hermes_home=_hermes_home, project_env=_project_env)
|
||||
|
||||
|
||||
_REASONING_TAGS = (
|
||||
"REASONING_SCRATCHPAD",
|
||||
"think",
|
||||
"thinking",
|
||||
"reasoning",
|
||||
"thought",
|
||||
)
|
||||
|
||||
|
||||
def _strip_reasoning_tags(text: str) -> str:
|
||||
"""Remove reasoning/thinking blocks from displayed text.
|
||||
|
||||
Handles every case:
|
||||
* Closed pairs ``<tag>…</tag>`` (case-insensitive, multi-line).
|
||||
* Unterminated open tags that run to end-of-text (e.g. truncated
|
||||
generations on NIM/MiniMax where the close tag is dropped).
|
||||
* Stray orphan close tags (``stuff</think>answer``) left behind by
|
||||
partial-content dumps.
|
||||
|
||||
Covers the variants emitted by reasoning models today: ``<think>``,
|
||||
``<thinking>``, ``<reasoning>``, ``<REASONING_SCRATCHPAD>``, and
|
||||
``<thought>`` (Gemma 4). Must stay in sync with
|
||||
``run_agent.py::_strip_think_blocks`` and the stream consumer's
|
||||
``_OPEN_THINK_TAGS`` / ``_CLOSE_THINK_TAGS`` tuples.
|
||||
"""
|
||||
cleaned = text
|
||||
for tag in _REASONING_TAGS:
|
||||
# Closed pair — case-insensitive so <THINK>…</THINK> is handled too.
|
||||
cleaned = re.sub(
|
||||
rf"<{tag}>.*?</{tag}>\s*",
|
||||
"",
|
||||
cleaned,
|
||||
flags=re.DOTALL | re.IGNORECASE,
|
||||
)
|
||||
# Unterminated open tag — strip from the tag to end of text.
|
||||
cleaned = re.sub(
|
||||
rf"<{tag}>.*$",
|
||||
"",
|
||||
cleaned,
|
||||
flags=re.DOTALL | re.IGNORECASE,
|
||||
)
|
||||
# Stray orphan close tag left behind by partial dumps.
|
||||
cleaned = re.sub(
|
||||
rf"</{tag}>\s*",
|
||||
"",
|
||||
cleaned,
|
||||
flags=re.IGNORECASE,
|
||||
)
|
||||
return cleaned.strip()
|
||||
|
||||
|
||||
def _assistant_content_as_text(content: Any) -> str:
|
||||
if content is None:
|
||||
return ""
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
parts = [
|
||||
str(part.get("text", ""))
|
||||
for part in content
|
||||
if isinstance(part, dict) and part.get("type") == "text"
|
||||
]
|
||||
return "\n".join(p for p in parts if p)
|
||||
return str(content)
|
||||
|
||||
|
||||
def _assistant_copy_text(content: Any) -> str:
|
||||
return _strip_reasoning_tags(_assistant_content_as_text(content))
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Configuration Loading
|
||||
# =============================================================================
|
||||
@@ -1244,10 +1172,6 @@ def _resolve_attachment_path(raw_path: str) -> Path | None:
|
||||
return None
|
||||
|
||||
expanded = os.path.expandvars(os.path.expanduser(token))
|
||||
if os.name != "nt":
|
||||
normalized = expanded.replace("\\", "/")
|
||||
if len(normalized) >= 3 and normalized[1] == ":" and normalized[2] == "/" and normalized[0].isalpha():
|
||||
expanded = f"/mnt/{normalized[0].lower()}/{normalized[3:]}"
|
||||
path = Path(expanded)
|
||||
if not path.is_absolute():
|
||||
base_dir = Path(os.getenv("TERMINAL_CWD", os.getcwd()))
|
||||
@@ -1330,12 +1254,10 @@ def _detect_file_drop(user_input: str) -> "dict | None":
|
||||
or stripped.startswith("~")
|
||||
or stripped.startswith("./")
|
||||
or stripped.startswith("../")
|
||||
or (len(stripped) >= 3 and stripped[1] == ":" and stripped[2] in ("\\", "/") and stripped[0].isalpha())
|
||||
or stripped.startswith('"/')
|
||||
or stripped.startswith('"~')
|
||||
or stripped.startswith("'/")
|
||||
or stripped.startswith("'~")
|
||||
or (len(stripped) >= 4 and stripped[0] in ("'", '"') and stripped[2] == ":" and stripped[3] in ("\\", "/") and stripped[1].isalpha())
|
||||
)
|
||||
if not starts_like_path:
|
||||
return None
|
||||
@@ -1810,7 +1732,7 @@ class HermesCLI:
|
||||
mcp_names = set((CLI_CONFIG.get("mcp_servers") or {}).keys())
|
||||
invalid = [t for t in toolsets if not validate_toolset(t) and t not in mcp_names]
|
||||
if invalid:
|
||||
self._console_print(f"[bold red]Warning: Unknown toolsets: {', '.join(invalid)}[/]")
|
||||
self.console.print(f"[bold red]Warning: Unknown toolsets: {', '.join(invalid)}[/]")
|
||||
|
||||
# Filesystem checkpoints: CLI flag > config
|
||||
cp_cfg = CLI_CONFIG.get("checkpoints", {})
|
||||
@@ -2102,35 +2024,20 @@ class HermesCLI:
|
||||
|
||||
def _spinner_widget_height(self, width: Optional[int] = None) -> int:
|
||||
"""Return the visible height for the spinner/status text line above the status bar."""
|
||||
spinner_line = self._render_spinner_text()
|
||||
if not spinner_line:
|
||||
if not getattr(self, "_spinner_text", ""):
|
||||
return 0
|
||||
if self._use_minimal_tui_chrome(width=width):
|
||||
return 0
|
||||
# Compute how many lines the spinner text needs when wrapped.
|
||||
# The rendered text is " {emoji} {label} ({elapsed})" — about
|
||||
# len(_spinner_text) + 16 chars for indent + timer suffix.
|
||||
width = width or self._get_tui_terminal_width()
|
||||
if width and width > 10:
|
||||
import math
|
||||
text_width = self._status_bar_display_width(spinner_line)
|
||||
return max(1, math.ceil(text_width / width))
|
||||
text_len = len(self._spinner_text) + 16 # indent + timer
|
||||
return max(1, math.ceil(text_len / width))
|
||||
return 1
|
||||
|
||||
def _render_spinner_text(self) -> str:
|
||||
"""Return the live spinner/status text exactly as rendered in the TUI."""
|
||||
txt = getattr(self, "_spinner_text", "")
|
||||
if not txt:
|
||||
return ""
|
||||
t0 = getattr(self, "_tool_start_time", 0) or 0
|
||||
if t0 > 0:
|
||||
import time as _time
|
||||
elapsed = _time.monotonic() - t0
|
||||
if elapsed >= 60:
|
||||
_m, _s = int(elapsed // 60), int(elapsed % 60)
|
||||
elapsed_str = f"{_m}m {_s}s"
|
||||
else:
|
||||
elapsed_str = f"{elapsed:.1f}s"
|
||||
return f" {txt} ({elapsed_str})"
|
||||
return f" {txt}"
|
||||
|
||||
def _get_voice_status_fragments(self, width: Optional[int] = None):
|
||||
"""Return the voice status bar fragments for the interactive TUI."""
|
||||
width = width or self._get_tui_terminal_width()
|
||||
@@ -2261,7 +2168,7 @@ class HermesCLI:
|
||||
normalized_model = normalize_model_for_provider(current_model, resolved_provider)
|
||||
if normalized_model and normalized_model != current_model:
|
||||
if not self._model_is_default:
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f"[yellow]⚠️ Normalized model '{current_model}' to '{normalized_model}' for {resolved_provider}.[/]"
|
||||
)
|
||||
self.model = normalized_model
|
||||
@@ -2277,7 +2184,7 @@ class HermesCLI:
|
||||
canonical = normalize_copilot_model_id(current_model, api_key=self.api_key)
|
||||
if canonical and canonical != current_model:
|
||||
if not self._model_is_default:
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f"[yellow]⚠️ Normalized Copilot model '{current_model}' to '{canonical}'.[/]"
|
||||
)
|
||||
self.model = canonical
|
||||
@@ -2299,7 +2206,7 @@ class HermesCLI:
|
||||
canonical = normalize_opencode_model_id(resolved_provider, current_model)
|
||||
if canonical and canonical != current_model:
|
||||
if not self._model_is_default:
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f"[yellow]⚠️ Stripped provider prefix from '{current_model}'; using '{canonical}' for {resolved_provider}.[/]"
|
||||
)
|
||||
self.model = canonical
|
||||
@@ -2321,7 +2228,7 @@ class HermesCLI:
|
||||
if "/" in current_model:
|
||||
slug = current_model.split("/", 1)[1]
|
||||
if not self._model_is_default:
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f"[yellow]⚠️ Stripped provider prefix from '{current_model}'; "
|
||||
f"using '{slug}' for OpenAI Codex.[/]"
|
||||
)
|
||||
@@ -3070,7 +2977,7 @@ class HermesCLI:
|
||||
use_compact = self.compact or term_width < 80
|
||||
|
||||
if use_compact:
|
||||
self._console_print(_build_compact_banner())
|
||||
self.console.print(_build_compact_banner())
|
||||
self._show_status()
|
||||
else:
|
||||
# Get tools for display
|
||||
@@ -3095,25 +3002,25 @@ class HermesCLI:
|
||||
|
||||
# Warn about very low context lengths (common with local servers)
|
||||
if ctx_len and ctx_len <= 8192:
|
||||
self._console_print()
|
||||
self._console_print(
|
||||
self.console.print()
|
||||
self.console.print(
|
||||
f"[yellow]⚠️ Context length is only {ctx_len:,} tokens — "
|
||||
f"this is likely too low for agent use with tools.[/]"
|
||||
)
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
"[dim] Hermes needs 16k–32k minimum. Tool schemas + system prompt alone use ~4k–8k.[/]"
|
||||
)
|
||||
base_url = getattr(self, "base_url", "") or ""
|
||||
if "11434" in base_url or "ollama" in base_url.lower():
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
"[dim] Ollama fix: OLLAMA_CONTEXT_LENGTH=32768 ollama serve[/]"
|
||||
)
|
||||
elif "1234" in base_url:
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
"[dim] LM Studio fix: Set context length in model settings → reload model[/]"
|
||||
)
|
||||
else:
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
"[dim] Fix: Set model.context_length in config.yaml, or increase your server's context setting[/]"
|
||||
)
|
||||
|
||||
@@ -3122,20 +3029,20 @@ class HermesCLI:
|
||||
|
||||
model_name = getattr(self, "model", "") or ""
|
||||
if is_nous_hermes_non_agentic(model_name):
|
||||
self._console_print()
|
||||
self._console_print(
|
||||
self.console.print()
|
||||
self.console.print(
|
||||
"[bold yellow]⚠ Nous Research Hermes 3 & 4 models are NOT agentic and are not "
|
||||
"designed for use with Hermes Agent.[/]"
|
||||
)
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
"[dim] They lack tool-calling capabilities required for agent workflows. "
|
||||
"Consider using an agentic model (Claude, GPT, Gemini, DeepSeek, etc.).[/]"
|
||||
)
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
"[dim] Switch with: /model sonnet or /model gpt5[/]"
|
||||
)
|
||||
|
||||
self._console_print()
|
||||
self.console.print()
|
||||
|
||||
def _preload_resumed_session(self) -> bool:
|
||||
"""Load a resumed session's history from the DB early (before first chat).
|
||||
@@ -3153,10 +3060,10 @@ class HermesCLI:
|
||||
|
||||
session_meta = self._session_db.get_session(self.session_id)
|
||||
if not session_meta:
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f"[bold red]Session not found: {self.session_id}[/]"
|
||||
)
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
"[dim]Use a session ID from a previous CLI run "
|
||||
"(hermes sessions list).[/]"
|
||||
)
|
||||
@@ -3171,7 +3078,7 @@ class HermesCLI:
|
||||
if session_meta.get("title"):
|
||||
title_part = f' "{session_meta["title"]}"'
|
||||
accent_color = _accent_hex()
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f"[{accent_color}]↻ Resumed session [bold]{self.session_id}[/bold]"
|
||||
f"{title_part} "
|
||||
f"({msg_count} user message{'s' if msg_count != 1 else ''}, "
|
||||
@@ -3179,7 +3086,7 @@ class HermesCLI:
|
||||
)
|
||||
else:
|
||||
accent_color = _accent_hex()
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f"[{accent_color}]Session {self.session_id} found but has no "
|
||||
f"messages. Starting fresh.[/]"
|
||||
)
|
||||
@@ -3218,6 +3125,21 @@ class HermesCLI:
|
||||
MAX_ASST_LEN = 200 # truncate assistant text
|
||||
MAX_ASST_LINES = 3 # max lines of assistant text
|
||||
|
||||
def _strip_reasoning(text: str) -> str:
|
||||
"""Remove <REASONING_SCRATCHPAD>...</REASONING_SCRATCHPAD> blocks
|
||||
from displayed text (reasoning model internal thoughts)."""
|
||||
import re
|
||||
cleaned = re.sub(
|
||||
r"<REASONING_SCRATCHPAD>.*?</REASONING_SCRATCHPAD>\s*",
|
||||
"", text, flags=re.DOTALL,
|
||||
)
|
||||
# Also strip unclosed reasoning tags at the end
|
||||
cleaned = re.sub(
|
||||
r"<REASONING_SCRATCHPAD>.*$",
|
||||
"", cleaned, flags=re.DOTALL,
|
||||
)
|
||||
return cleaned.strip()
|
||||
|
||||
# Collect displayable entries (skip system, tool-result messages)
|
||||
entries = [] # list of (role, display_text)
|
||||
_last_asst_idx = None # index of last assistant entry
|
||||
@@ -3249,7 +3171,7 @@ class HermesCLI:
|
||||
|
||||
elif role == "assistant":
|
||||
text = "" if content is None else str(content)
|
||||
text = _strip_reasoning_tags(text)
|
||||
text = _strip_reasoning(text)
|
||||
parts = []
|
||||
full_parts = [] # un-truncated version
|
||||
if text:
|
||||
@@ -3354,7 +3276,7 @@ class HermesCLI:
|
||||
padding=(0, 1),
|
||||
style=_history_text_c,
|
||||
)
|
||||
self._console_print(panel)
|
||||
self.console.print(panel)
|
||||
|
||||
def _try_attach_clipboard_image(self) -> bool:
|
||||
"""Check clipboard for an image and attach it if found.
|
||||
@@ -3588,26 +3510,6 @@ class HermesCLI:
|
||||
killed = process_registry.kill_all()
|
||||
print(f" ✅ Stopped {killed} process(es).")
|
||||
|
||||
def _handle_agents_command(self):
|
||||
"""Handle /agents — show background processes and agent status."""
|
||||
from tools.process_registry import format_uptime_short, process_registry
|
||||
|
||||
processes = process_registry.list_sessions()
|
||||
running = [p for p in processes if p.get("status") == "running"]
|
||||
finished = [p for p in processes if p.get("status") != "running"]
|
||||
|
||||
_cprint(f" Running processes: {len(running)}")
|
||||
for p in running:
|
||||
cmd = p.get("command", "")[:80]
|
||||
up = format_uptime_short(p.get("uptime_seconds", 0))
|
||||
_cprint(f" {p.get('session_id', '?')} · {up} · {cmd}")
|
||||
|
||||
if finished:
|
||||
_cprint(f" Recently finished: {len(finished)}")
|
||||
|
||||
agent_running = getattr(self, "_agent_running", False)
|
||||
_cprint(f" Agent: {'running' if agent_running else 'idle'}")
|
||||
|
||||
def _handle_paste_command(self):
|
||||
"""Handle /paste — explicitly check clipboard for an image.
|
||||
|
||||
@@ -3633,61 +3535,6 @@ class HermesCLI:
|
||||
else:
|
||||
_cprint(f" {_DIM}(._.) No image found in clipboard{_RST}")
|
||||
|
||||
def _write_osc52_clipboard(self, text: str) -> None:
|
||||
"""Copy *text* to terminal clipboard via OSC 52."""
|
||||
payload = base64.b64encode(text.encode("utf-8")).decode("ascii")
|
||||
seq = f"\x1b]52;c;{payload}\x07"
|
||||
out = getattr(self, "_app", None)
|
||||
output = getattr(out, "output", None) if out else None
|
||||
if output and hasattr(output, "write_raw"):
|
||||
output.write_raw(seq)
|
||||
output.flush()
|
||||
return
|
||||
if output and hasattr(output, "write"):
|
||||
output.write(seq)
|
||||
output.flush()
|
||||
return
|
||||
sys.stdout.write(seq)
|
||||
sys.stdout.flush()
|
||||
|
||||
def _handle_copy_command(self, cmd_original: str) -> None:
|
||||
"""Handle /copy [number] — copy assistant output to clipboard."""
|
||||
parts = cmd_original.split(maxsplit=1)
|
||||
arg = parts[1].strip() if len(parts) > 1 else ""
|
||||
|
||||
assistant = [m for m in self.conversation_history if m.get("role") == "assistant"]
|
||||
if not assistant:
|
||||
_cprint(" Nothing to copy yet.")
|
||||
return
|
||||
|
||||
if arg:
|
||||
try:
|
||||
idx = int(arg) - 1
|
||||
except ValueError:
|
||||
_cprint(" Usage: /copy [number]")
|
||||
return
|
||||
if idx < 0 or idx >= len(assistant):
|
||||
_cprint(f" Invalid response number. Use 1-{len(assistant)}.")
|
||||
return
|
||||
else:
|
||||
idx = len(assistant) - 1
|
||||
while idx >= 0 and not _assistant_copy_text(assistant[idx].get("content")):
|
||||
idx -= 1
|
||||
if idx < 0:
|
||||
_cprint(" Nothing to copy in assistant responses yet.")
|
||||
return
|
||||
|
||||
text = _assistant_copy_text(assistant[idx].get("content"))
|
||||
if not text:
|
||||
_cprint(" Nothing to copy in that assistant response.")
|
||||
return
|
||||
|
||||
try:
|
||||
self._write_osc52_clipboard(text)
|
||||
_cprint(f" Copied assistant response #{idx + 1} to clipboard")
|
||||
except Exception as e:
|
||||
_cprint(f" Clipboard copy failed: {e}")
|
||||
|
||||
def _handle_image_command(self, cmd_original: str):
|
||||
"""Handle /image <path> — attach a local image file for the next prompt."""
|
||||
raw_args = (cmd_original.split(None, 1)[1].strip() if " " in cmd_original else "")
|
||||
@@ -3790,14 +3637,14 @@ class HermesCLI:
|
||||
api_key_missing = [u for u in unavailable if u["missing_vars"]]
|
||||
|
||||
if api_key_missing:
|
||||
self._console_print()
|
||||
self._console_print("[yellow]⚠️ Some tools disabled (missing API keys):[/]")
|
||||
self.console.print()
|
||||
self.console.print("[yellow]⚠️ Some tools disabled (missing API keys):[/]")
|
||||
for item in api_key_missing:
|
||||
tools_str = ", ".join(item["tools"][:2]) # Show first 2 tools
|
||||
if len(item["tools"]) > 2:
|
||||
tools_str += f", +{len(item['tools'])-2} more"
|
||||
self._console_print(f" [dim]• {item['name']}[/] [dim italic]({', '.join(item['missing_vars'])})[/]")
|
||||
self._console_print("[dim] Run 'hermes setup' to configure[/]")
|
||||
self.console.print(f" [dim]• {item['name']}[/] [dim italic]({', '.join(item['missing_vars'])})[/]")
|
||||
self.console.print("[dim] Run 'hermes setup' to configure[/]")
|
||||
except Exception:
|
||||
pass # Don't crash on import errors
|
||||
|
||||
@@ -3824,7 +3671,7 @@ class HermesCLI:
|
||||
skin = get_active_skin()
|
||||
separator_color = skin.get_color("banner_dim", "#B8860B")
|
||||
accent_color = skin.get_color("ui_accent", "#FFBF00")
|
||||
label_color = skin.get_color("ui_label", "#DAA520")
|
||||
label_color = skin.get_color("ui_label", "#4dd0e1")
|
||||
except Exception:
|
||||
separator_color, accent_color, label_color = "#B8860B", "#FFBF00", "cyan"
|
||||
toolsets_info = ""
|
||||
@@ -3835,7 +3682,7 @@ class HermesCLI:
|
||||
if self._provider_source:
|
||||
provider_info += f" [dim {separator_color}]·[/] [dim]auth: {self._provider_source}[/]"
|
||||
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f" {api_indicator} [{accent_color}]{model_short}[/] "
|
||||
f"[dim {separator_color}]·[/] [bold {label_color}]{tool_count} tools[/]"
|
||||
f"{toolsets_info}{provider_info}"
|
||||
@@ -3892,7 +3739,7 @@ class HermesCLI:
|
||||
f"Tokens: {total_tokens:,}",
|
||||
f"Agent Running: {'Yes' if is_running else 'No'}",
|
||||
])
|
||||
self._console_print("\n".join(lines), highlight=False, markup=False)
|
||||
self.console.print("\n".join(lines), highlight=False, markup=False)
|
||||
|
||||
def _fast_command_available(self) -> bool:
|
||||
try:
|
||||
@@ -4667,34 +4514,6 @@ class HermesCLI:
|
||||
self._restore_modal_input_snapshot()
|
||||
self._invalidate(min_interval=0.0)
|
||||
|
||||
@staticmethod
|
||||
def _compute_model_picker_viewport(
|
||||
selected: int,
|
||||
scroll_offset: int,
|
||||
n: int,
|
||||
term_rows: int,
|
||||
reserved_below: int = 6,
|
||||
panel_chrome: int = 6,
|
||||
min_visible: int = 3,
|
||||
) -> tuple[int, int]:
|
||||
"""Resolve (scroll_offset, visible) for the /model picker viewport.
|
||||
|
||||
``reserved_below`` matches the approval / clarify panels — input area,
|
||||
status bar, and separators below the panel. ``panel_chrome`` covers
|
||||
this panel's own borders + blanks + hint row. The remaining rows hold
|
||||
the scrollable list, with the offset slid to keep ``selected`` on screen.
|
||||
"""
|
||||
max_visible = max(min_visible, term_rows - reserved_below - panel_chrome)
|
||||
if n <= max_visible:
|
||||
return 0, n
|
||||
visible = max_visible
|
||||
if selected < scroll_offset:
|
||||
scroll_offset = selected
|
||||
elif selected >= scroll_offset + visible:
|
||||
scroll_offset = selected - visible + 1
|
||||
scroll_offset = max(0, min(scroll_offset, n - visible))
|
||||
return scroll_offset, visible
|
||||
|
||||
def _apply_model_switch_result(self, result, persist_global: bool) -> None:
|
||||
if not result.success:
|
||||
_cprint(f" ✗ {result.error_message}")
|
||||
@@ -5090,15 +4909,8 @@ class HermesCLI:
|
||||
|
||||
print(" To change model or provider, use: hermes model")
|
||||
|
||||
def _output_console(self):
|
||||
"""Use prompt_toolkit-safe Rich rendering once the TUI is live."""
|
||||
if getattr(self, "_app", None):
|
||||
return ChatConsole()
|
||||
return self.console
|
||||
|
||||
def _console_print(self, *args, **kwargs):
|
||||
"""Print through the active command-safe console."""
|
||||
self._output_console().print(*args, **kwargs)
|
||||
|
||||
|
||||
@staticmethod
|
||||
def _resolve_personality_prompt(value) -> str:
|
||||
@@ -5118,14 +4930,14 @@ class HermesCLI:
|
||||
from agent.google_oauth import get_valid_access_token, GoogleOAuthError, load_credentials
|
||||
from agent.google_code_assist import retrieve_user_quota, CodeAssistError
|
||||
except ImportError as exc:
|
||||
self._console_print(f" [red]Gemini modules unavailable: {exc}[/]")
|
||||
self.console.print(f" [red]Gemini modules unavailable: {exc}[/]")
|
||||
return
|
||||
|
||||
try:
|
||||
access_token = get_valid_access_token()
|
||||
except GoogleOAuthError as exc:
|
||||
self._console_print(f" [yellow]{exc}[/]")
|
||||
self._console_print(" Run [bold]/model[/] and pick 'Google Gemini (OAuth)' to sign in.")
|
||||
self.console.print(f" [yellow]{exc}[/]")
|
||||
self.console.print(" Run [bold]/model[/] and pick 'Google Gemini (OAuth)' to sign in.")
|
||||
return
|
||||
|
||||
creds = load_credentials()
|
||||
@@ -5134,18 +4946,18 @@ class HermesCLI:
|
||||
try:
|
||||
buckets = retrieve_user_quota(access_token, project_id=project_id)
|
||||
except CodeAssistError as exc:
|
||||
self._console_print(f" [red]Quota lookup failed:[/] {exc}")
|
||||
self.console.print(f" [red]Quota lookup failed:[/] {exc}")
|
||||
return
|
||||
|
||||
if not buckets:
|
||||
self._console_print(" [dim]No quota buckets reported (account may be on legacy/unmetered tier).[/]")
|
||||
self.console.print(" [dim]No quota buckets reported (account may be on legacy/unmetered tier).[/]")
|
||||
return
|
||||
|
||||
# Sort for stable display, group by model
|
||||
buckets.sort(key=lambda b: (b.model_id, b.token_type))
|
||||
self._console_print()
|
||||
self._console_print(f" [bold]Gemini Code Assist quota[/] (project: {project_id or '(auto / free-tier)'})")
|
||||
self._console_print()
|
||||
self.console.print()
|
||||
self.console.print(f" [bold]Gemini Code Assist quota[/] (project: {project_id or '(auto / free-tier)'})")
|
||||
self.console.print()
|
||||
for b in buckets:
|
||||
pct = max(0.0, min(1.0, b.remaining_fraction))
|
||||
width = 20
|
||||
@@ -5155,8 +4967,8 @@ class HermesCLI:
|
||||
header = b.model_id
|
||||
if b.token_type:
|
||||
header += f" [{b.token_type}]"
|
||||
self._console_print(f" {header:40s} {bar} {pct_str}")
|
||||
self._console_print()
|
||||
self.console.print(f" {header:40s} {bar} {pct_str}")
|
||||
self.console.print()
|
||||
|
||||
def _handle_personality_command(self, cmd: str):
|
||||
"""Handle the /personality command to set predefined personalities."""
|
||||
@@ -5604,7 +5416,7 @@ class HermesCLI:
|
||||
_tip_color = get_active_skin().get_color("banner_dim", "#B8860B")
|
||||
except Exception:
|
||||
_tip_color = "#B8860B"
|
||||
self._console_print(f"[dim {_tip_color}]✦ Tip: {_tip}[/]")
|
||||
self.console.print(f"[dim {_tip_color}]✦ Tip: {_tip}[/]")
|
||||
except Exception:
|
||||
pass
|
||||
elif canonical == "history":
|
||||
@@ -5698,7 +5510,7 @@ class HermesCLI:
|
||||
elif canonical == "statusbar":
|
||||
self._status_bar_visible = not self._status_bar_visible
|
||||
state = "visible" if self._status_bar_visible else "hidden"
|
||||
self._console_print(f" Status bar {state}")
|
||||
self.console.print(f" Status bar {state}")
|
||||
elif canonical == "verbose":
|
||||
self._toggle_verbose()
|
||||
elif canonical == "yolo":
|
||||
@@ -5713,8 +5525,6 @@ class HermesCLI:
|
||||
self._show_usage()
|
||||
elif canonical == "insights":
|
||||
self._show_insights(cmd_original)
|
||||
elif canonical == "copy":
|
||||
self._handle_copy_command(cmd_original)
|
||||
elif canonical == "debug":
|
||||
self._handle_debug_command()
|
||||
elif canonical == "paste":
|
||||
@@ -5758,8 +5568,6 @@ class HermesCLI:
|
||||
self._handle_snapshot_command(cmd_original)
|
||||
elif canonical == "stop":
|
||||
self._handle_stop_command()
|
||||
elif canonical == "agents":
|
||||
self._handle_agents_command()
|
||||
elif canonical == "background":
|
||||
self._handle_background_command(cmd_original)
|
||||
elif canonical == "btw":
|
||||
@@ -5776,30 +5584,6 @@ class HermesCLI:
|
||||
_cprint(f" Queued for the next turn: {payload[:80]}{'...' if len(payload) > 80 else ''}")
|
||||
else:
|
||||
_cprint(f" Queued: {payload[:80]}{'...' if len(payload) > 80 else ''}")
|
||||
elif canonical == "steer":
|
||||
# Inject a message after the next tool call without interrupting.
|
||||
# If the agent is actively running, push the text into the agent's
|
||||
# pending_steer slot — the drain hook in _execute_tool_calls_*
|
||||
# will append it to the next tool result's content. If no agent
|
||||
# is running, fall back to queue semantics (same as /queue).
|
||||
parts = cmd_original.split(None, 1)
|
||||
payload = parts[1].strip() if len(parts) > 1 else ""
|
||||
if not payload:
|
||||
_cprint(" Usage: /steer <prompt>")
|
||||
elif self._agent_running and self.agent is not None and hasattr(self.agent, "steer"):
|
||||
try:
|
||||
accepted = self.agent.steer(payload)
|
||||
except Exception as exc:
|
||||
_cprint(f" Steer failed: {exc}")
|
||||
else:
|
||||
if accepted:
|
||||
_cprint(f" ⏩ Steer queued — arrives after the next tool call: {payload[:80]}{'...' if len(payload) > 80 else ''}")
|
||||
else:
|
||||
_cprint(" Steer rejected (empty payload).")
|
||||
else:
|
||||
# No active run — treat as a normal next-turn message.
|
||||
self._pending_input.put(payload)
|
||||
_cprint(f" No agent running; queued as next turn: {payload[:80]}{'...' if len(payload) > 80 else ''}")
|
||||
elif canonical == "skin":
|
||||
self._handle_skin_command(cmd_original)
|
||||
elif canonical == "voice":
|
||||
@@ -5821,15 +5605,15 @@ class HermesCLI:
|
||||
)
|
||||
output = result.stdout.strip() or result.stderr.strip()
|
||||
if output:
|
||||
self._console_print(_rich_text_from_ansi(output))
|
||||
self.console.print(_rich_text_from_ansi(output))
|
||||
else:
|
||||
self._console_print("[dim]Command returned no output[/]")
|
||||
self.console.print("[dim]Command returned no output[/]")
|
||||
except subprocess.TimeoutExpired:
|
||||
self._console_print("[bold red]Quick command timed out (30s)[/]")
|
||||
self.console.print("[bold red]Quick command timed out (30s)[/]")
|
||||
except Exception as e:
|
||||
self._console_print(f"[bold red]Quick command error: {e}[/]")
|
||||
self.console.print(f"[bold red]Quick command error: {e}[/]")
|
||||
else:
|
||||
self._console_print(f"[bold red]Quick command '{base_cmd}' has no command defined[/]")
|
||||
self.console.print(f"[bold red]Quick command '{base_cmd}' has no command defined[/]")
|
||||
elif qcmd.get("type") == "alias":
|
||||
target = qcmd.get("target", "").strip()
|
||||
if target:
|
||||
@@ -5838,9 +5622,9 @@ class HermesCLI:
|
||||
aliased_command = f"{target} {user_args}".strip()
|
||||
return self.process_command(aliased_command)
|
||||
else:
|
||||
self._console_print(f"[bold red]Quick command '{base_cmd}' has no target defined[/]")
|
||||
self.console.print(f"[bold red]Quick command '{base_cmd}' has no target defined[/]")
|
||||
else:
|
||||
self._console_print(f"[bold red]Quick command '{base_cmd}' has unsupported type (supported: 'exec', 'alias')[/]")
|
||||
self.console.print(f"[bold red]Quick command '{base_cmd}' has unsupported type (supported: 'exec', 'alias')[/]")
|
||||
# Check for plugin-registered slash commands
|
||||
elif base_cmd.lstrip("/") in _get_plugin_cmd_handler_names():
|
||||
from hermes_cli.plugins import get_plugin_command_handler
|
||||
@@ -7097,7 +6881,8 @@ class HermesCLI:
|
||||
)
|
||||
raise RuntimeError(
|
||||
"Voice mode requires sounddevice and numpy.\n"
|
||||
f"Install with: {sys.executable} -m pip install sounddevice numpy"
|
||||
"Install with: pip install sounddevice numpy\n"
|
||||
"Or: pip install hermes-agent[voice]"
|
||||
)
|
||||
if not reqs.get("stt_available", reqs.get("stt_key_set")):
|
||||
raise RuntimeError(
|
||||
@@ -7373,7 +7158,8 @@ class HermesCLI:
|
||||
_cprint(f" {_DIM}Then install/update the Termux:API Android app for microphone capture{_RST}")
|
||||
_cprint(f" {_BOLD}Option 2: pkg install python-numpy portaudio && python -m pip install sounddevice{_RST}")
|
||||
else:
|
||||
_cprint(f"\n {_BOLD}Install: {sys.executable} -m pip install {' '.join(reqs['missing_packages'])}{_RST}")
|
||||
_cprint(f"\n {_BOLD}Install: pip install {' '.join(reqs['missing_packages'])}{_RST}")
|
||||
_cprint(f" {_DIM}Or: pip install hermes-agent[voice]{_RST}")
|
||||
return
|
||||
|
||||
with self._voice_lock:
|
||||
@@ -8324,15 +8110,7 @@ class HermesCLI:
|
||||
else:
|
||||
print(f"\n⚡ Sending after interrupt: '{preview}'")
|
||||
self._pending_input.put(combined)
|
||||
|
||||
# If a /steer was left over (agent finished before another tool
|
||||
# batch could absorb it), deliver it as the next user turn.
|
||||
_leftover_steer = result.get("pending_steer") if result else None
|
||||
if _leftover_steer and hasattr(self, '_pending_input'):
|
||||
preview = _leftover_steer[:60] + ("..." if len(_leftover_steer) > 60 else "")
|
||||
print(f"\n⏩ Delivering leftover /steer as next turn: '{preview}'")
|
||||
self._pending_input.put(_leftover_steer)
|
||||
|
||||
|
||||
return response
|
||||
|
||||
except Exception as e:
|
||||
@@ -8610,7 +8388,7 @@ class HermesCLI:
|
||||
except Exception:
|
||||
_welcome_text = "Welcome to Hermes Agent! Type your message or /help for commands."
|
||||
_welcome_color = "#FFF8DC"
|
||||
self._console_print(f"[{_welcome_color}]{_welcome_text}[/]")
|
||||
self.console.print(f"[{_welcome_color}]{_welcome_text}[/]")
|
||||
# Show a random tip to help users discover features
|
||||
try:
|
||||
from hermes_cli.tips import get_random_tip
|
||||
@@ -8619,16 +8397,16 @@ class HermesCLI:
|
||||
_tip_color = _welcome_skin.get_color("banner_dim", "#B8860B")
|
||||
except Exception:
|
||||
_tip_color = "#B8860B"
|
||||
self._console_print(f"[dim {_tip_color}]✦ Tip: {_tip}[/]")
|
||||
self.console.print(f"[dim {_tip_color}]✦ Tip: {_tip}[/]")
|
||||
except Exception:
|
||||
pass # Tips are non-critical — never break startup
|
||||
if self.preloaded_skills and not self._startup_skills_line_shown:
|
||||
skills_label = ", ".join(self.preloaded_skills)
|
||||
self._console_print(
|
||||
self.console.print(
|
||||
f"[bold {_accent_hex()}]Activated skills:[/] {skills_label}"
|
||||
)
|
||||
self._startup_skills_line_shown = True
|
||||
self._console_print()
|
||||
self.console.print()
|
||||
|
||||
# State for async operation
|
||||
self._agent_running = False
|
||||
@@ -8750,7 +8528,6 @@ class HermesCLI:
|
||||
# --- /model picker modal ---
|
||||
if self._model_picker_state:
|
||||
self._handle_model_picker_selection()
|
||||
event.app.current_buffer.reset()
|
||||
event.app.invalidate()
|
||||
return
|
||||
|
||||
@@ -8916,13 +8693,6 @@ class HermesCLI:
|
||||
state["selected"] = min(max_idx, state.get("selected", 0) + 1)
|
||||
event.app.invalidate()
|
||||
|
||||
@kb.add('escape', filter=Condition(lambda: bool(self._model_picker_state)), eager=True)
|
||||
def model_picker_escape(event):
|
||||
"""ESC closes the /model picker."""
|
||||
self._close_model_picker()
|
||||
event.app.current_buffer.reset()
|
||||
event.app.invalidate()
|
||||
|
||||
# --- History navigation: up/down browse history in normal input mode ---
|
||||
# The TextArea is multiline, so by default up/down only move the cursor.
|
||||
# Buffer.auto_up/auto_down handle both: cursor movement when multi-line,
|
||||
@@ -9431,10 +9201,21 @@ class HermesCLI:
|
||||
return cli_ref._agent_spacer_height()
|
||||
|
||||
def get_spinner_text():
|
||||
spinner_line = cli_ref._render_spinner_text()
|
||||
if not spinner_line:
|
||||
txt = cli_ref._spinner_text
|
||||
if not txt:
|
||||
return []
|
||||
return [('class:hint', spinner_line)]
|
||||
# Append live elapsed timer when a tool is running
|
||||
t0 = cli_ref._tool_start_time
|
||||
if t0 > 0:
|
||||
import time as _time
|
||||
elapsed = _time.monotonic() - t0
|
||||
if elapsed >= 60:
|
||||
_m, _s = int(elapsed // 60), int(elapsed % 60)
|
||||
elapsed_str = f"{_m}m {_s}s"
|
||||
else:
|
||||
elapsed_str = f"{elapsed:.1f}s"
|
||||
return [('class:hint', f' {txt} ({elapsed_str})')]
|
||||
return [('class:hint', f' {txt}')]
|
||||
|
||||
def get_spinner_height():
|
||||
return cli_ref._spinner_widget_height()
|
||||
@@ -9713,22 +9494,6 @@ class HermesCLI:
|
||||
|
||||
box_width = _panel_box_width(title, [hint] + choices, min_width=46, max_width=84)
|
||||
inner_text_width = max(8, box_width - 6)
|
||||
selected = state.get("selected", 0)
|
||||
|
||||
# Scrolling viewport: the panel renders into a Window with no max
|
||||
# height, so without limiting visible items the bottom border and
|
||||
# any items past the available terminal rows get clipped on long
|
||||
# provider catalogs (e.g. Ollama Cloud's 36+ models).
|
||||
try:
|
||||
from prompt_toolkit.application import get_app
|
||||
term_rows = get_app().output.get_size().rows
|
||||
except Exception:
|
||||
term_rows = shutil.get_terminal_size((100, 24)).lines
|
||||
scroll_offset, visible = HermesCLI._compute_model_picker_viewport(
|
||||
selected, state.get("_scroll_offset", 0), len(choices), term_rows,
|
||||
)
|
||||
state["_scroll_offset"] = scroll_offset
|
||||
|
||||
lines = []
|
||||
lines.append(('class:clarify-border', '╭─ '))
|
||||
lines.append(('class:clarify-title', title))
|
||||
@@ -9736,8 +9501,8 @@ class HermesCLI:
|
||||
_append_blank_panel_line(lines, 'class:clarify-border', box_width)
|
||||
_append_panel_line(lines, 'class:clarify-border', 'class:clarify-hint', hint, box_width)
|
||||
_append_blank_panel_line(lines, 'class:clarify-border', box_width)
|
||||
for idx in range(scroll_offset, scroll_offset + visible):
|
||||
choice = choices[idx]
|
||||
selected = state.get("selected", 0)
|
||||
for idx, choice in enumerate(choices):
|
||||
style = 'class:clarify-selected' if idx == selected else 'class:clarify-choice'
|
||||
prefix = '❯ ' if idx == selected else ' '
|
||||
for wrapped in _wrap_panel_text(prefix + choice, inner_text_width, subsequent_indent=' '):
|
||||
@@ -10142,36 +9907,8 @@ class HermesCLI:
|
||||
|
||||
# Register signal handlers for graceful shutdown on SSH disconnect / SIGTERM
|
||||
def _signal_handler(signum, frame):
|
||||
"""Handle SIGHUP/SIGTERM by triggering graceful cleanup.
|
||||
|
||||
Calls ``self.agent.interrupt()`` first so the agent daemon
|
||||
thread's poll loop sees the per-thread interrupt and kills the
|
||||
tool's subprocess group via ``_kill_process`` (os.killpg).
|
||||
Without this, the main thread dies from KeyboardInterrupt and
|
||||
the daemon thread is killed with it — before it can run one
|
||||
more poll iteration to clean up the subprocess, which was
|
||||
spawned with ``os.setsid`` and therefore survives as an orphan
|
||||
with PPID=1.
|
||||
|
||||
Grace window (``HERMES_SIGTERM_GRACE``, default 1.5 s) gives
|
||||
the daemon time to: detect the interrupt (next 200 ms poll) →
|
||||
call _kill_process (SIGTERM + 1 s wait + SIGKILL if needed) →
|
||||
return from _wait_for_process. ``time.sleep`` releases the
|
||||
GIL so the daemon actually runs during the window.
|
||||
"""
|
||||
"""Handle SIGHUP/SIGTERM by triggering graceful cleanup."""
|
||||
logger.debug("Received signal %s, triggering graceful shutdown", signum)
|
||||
try:
|
||||
if getattr(self, "agent", None) and getattr(self, "_agent_running", False):
|
||||
self.agent.interrupt(f"received signal {signum}")
|
||||
import time as _t
|
||||
try:
|
||||
_grace = float(os.getenv("HERMES_SIGTERM_GRACE", "1.5"))
|
||||
except (TypeError, ValueError):
|
||||
_grace = 1.5
|
||||
if _grace > 0:
|
||||
_t.sleep(_grace)
|
||||
except Exception:
|
||||
pass # never block signal handling
|
||||
raise KeyboardInterrupt()
|
||||
|
||||
try:
|
||||
@@ -10474,45 +10211,6 @@ def main(
|
||||
|
||||
# Register cleanup for single-query mode (interactive mode registers in run())
|
||||
atexit.register(_run_cleanup)
|
||||
|
||||
# Also install signal handlers in single-query / `-q` mode. Interactive
|
||||
# mode registers its own inside HermesCLI.run(), but `-q` runs
|
||||
# cli.agent.run_conversation() below and AIAgent spawns worker threads
|
||||
# for tools — so when SIGTERM arrives on the main thread, raising
|
||||
# KeyboardInterrupt only unwinds the main thread, not the worker
|
||||
# running _wait_for_process. Python then exits, the child subprocess
|
||||
# (spawned with os.setsid, its own process group) is reparented to
|
||||
# init and keeps running as an orphan.
|
||||
#
|
||||
# Fix: route SIGTERM/SIGHUP through agent.interrupt() which sets the
|
||||
# per-thread interrupt flag the worker's poll loop checks every 200 ms.
|
||||
# Give the worker a grace window to call _kill_process (SIGTERM to the
|
||||
# process group, then SIGKILL after 1 s), then raise KeyboardInterrupt
|
||||
# so main unwinds normally. HERMES_SIGTERM_GRACE overrides the 1.5 s
|
||||
# default for debugging.
|
||||
def _signal_handler_q(signum, frame):
|
||||
logger.debug("Received signal %s in single-query mode", signum)
|
||||
try:
|
||||
_agent = getattr(cli, "agent", None)
|
||||
if _agent is not None:
|
||||
_agent.interrupt(f"received signal {signum}")
|
||||
import time as _t
|
||||
try:
|
||||
_grace = float(os.getenv("HERMES_SIGTERM_GRACE", "1.5"))
|
||||
except (TypeError, ValueError):
|
||||
_grace = 1.5
|
||||
if _grace > 0:
|
||||
_t.sleep(_grace)
|
||||
except Exception:
|
||||
pass # never block signal handling
|
||||
raise KeyboardInterrupt()
|
||||
try:
|
||||
import signal as _signal
|
||||
_signal.signal(_signal.SIGTERM, _signal_handler_q)
|
||||
if hasattr(_signal, "SIGHUP"):
|
||||
_signal.signal(_signal.SIGHUP, _signal_handler_q)
|
||||
except Exception:
|
||||
pass # signal handler may fail in restricted environments
|
||||
|
||||
# Handle single query mode
|
||||
if query or image:
|
||||
|
||||
+111
-254
@@ -27,7 +27,7 @@ except ImportError:
|
||||
except ImportError:
|
||||
msvcrt = None
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
from typing import Optional
|
||||
|
||||
# Add parent directory to path for imports BEFORE repo-level imports.
|
||||
# Without this, standalone invocations (e.g. after `hermes update` reloads
|
||||
@@ -49,33 +49,6 @@ _KNOWN_DELIVERY_PLATFORMS = frozenset({
|
||||
"qqbot",
|
||||
})
|
||||
|
||||
# Platforms that support a configured cron/notification home target, mapped to
|
||||
# the environment variable used by gateway setup/runtime config.
|
||||
_HOME_TARGET_ENV_VARS = {
|
||||
"matrix": "MATRIX_HOME_ROOM",
|
||||
"telegram": "TELEGRAM_HOME_CHANNEL",
|
||||
"discord": "DISCORD_HOME_CHANNEL",
|
||||
"slack": "SLACK_HOME_CHANNEL",
|
||||
"signal": "SIGNAL_HOME_CHANNEL",
|
||||
"mattermost": "MATTERMOST_HOME_CHANNEL",
|
||||
"sms": "SMS_HOME_CHANNEL",
|
||||
"email": "EMAIL_HOME_ADDRESS",
|
||||
"dingtalk": "DINGTALK_HOME_CHANNEL",
|
||||
"feishu": "FEISHU_HOME_CHANNEL",
|
||||
"wecom": "WECOM_HOME_CHANNEL",
|
||||
"weixin": "WEIXIN_HOME_CHANNEL",
|
||||
"bluebubbles": "BLUEBUBBLES_HOME_CHANNEL",
|
||||
"qqbot": "QQBOT_HOME_CHANNEL",
|
||||
}
|
||||
|
||||
# Legacy env var names kept for back-compat. Each entry is the current
|
||||
# primary env var → the previous name. _get_home_target_chat_id falls
|
||||
# back to the legacy name if the primary is unset, so users who set the
|
||||
# old name before the rename keep working until they migrate.
|
||||
_LEGACY_HOME_TARGET_ENV_VARS = {
|
||||
"QQBOT_HOME_CHANNEL": "QQ_HOME_CHANNEL",
|
||||
}
|
||||
|
||||
from cron.jobs import get_due_jobs, mark_job_run, save_job_output, advance_next_run
|
||||
|
||||
# Sentinel: when a cron agent has nothing new to report, it can start its
|
||||
@@ -103,28 +76,15 @@ def _resolve_origin(job: dict) -> Optional[dict]:
|
||||
return None
|
||||
|
||||
|
||||
def _get_home_target_chat_id(platform_name: str) -> str:
|
||||
"""Return the configured home target chat/room ID for a delivery platform."""
|
||||
env_var = _HOME_TARGET_ENV_VARS.get(platform_name.lower())
|
||||
if not env_var:
|
||||
return ""
|
||||
value = os.getenv(env_var, "")
|
||||
if not value:
|
||||
legacy = _LEGACY_HOME_TARGET_ENV_VARS.get(env_var)
|
||||
if legacy:
|
||||
value = os.getenv(legacy, "")
|
||||
return value
|
||||
|
||||
|
||||
def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[dict]:
|
||||
"""Resolve one concrete auto-delivery target for a cron job."""
|
||||
|
||||
def _resolve_delivery_target(job: dict) -> Optional[dict]:
|
||||
"""Resolve the concrete auto-delivery target for a cron job, if any."""
|
||||
deliver = job.get("deliver", "local")
|
||||
origin = _resolve_origin(job)
|
||||
|
||||
if deliver_value == "local":
|
||||
if deliver == "local":
|
||||
return None
|
||||
|
||||
if deliver_value == "origin":
|
||||
if deliver == "origin":
|
||||
if origin:
|
||||
return {
|
||||
"platform": origin["platform"],
|
||||
@@ -133,8 +93,8 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
}
|
||||
# Origin missing (e.g. job created via API/script) — try each
|
||||
# platform's home channel as a fallback instead of silently dropping.
|
||||
for platform_name in _HOME_TARGET_ENV_VARS:
|
||||
chat_id = _get_home_target_chat_id(platform_name)
|
||||
for platform_name in ("matrix", "telegram", "discord", "slack", "bluebubbles"):
|
||||
chat_id = os.getenv(f"{platform_name.upper()}_HOME_CHANNEL", "")
|
||||
if chat_id:
|
||||
logger.info(
|
||||
"Job '%s' has deliver=origin but no origin; falling back to %s home channel",
|
||||
@@ -148,8 +108,8 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
}
|
||||
return None
|
||||
|
||||
if ":" in deliver_value:
|
||||
platform_name, rest = deliver_value.split(":", 1)
|
||||
if ":" in deliver:
|
||||
platform_name, rest = deliver.split(":", 1)
|
||||
platform_key = platform_name.lower()
|
||||
|
||||
from tools.send_message_tool import _parse_target_ref
|
||||
@@ -179,7 +139,7 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
"thread_id": thread_id,
|
||||
}
|
||||
|
||||
platform_name = deliver_value
|
||||
platform_name = deliver
|
||||
if origin and origin.get("platform") == platform_name:
|
||||
return {
|
||||
"platform": platform_name,
|
||||
@@ -189,7 +149,7 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
|
||||
if platform_name.lower() not in _KNOWN_DELIVERY_PLATFORMS:
|
||||
return None
|
||||
chat_id = _get_home_target_chat_id(platform_name)
|
||||
chat_id = os.getenv(f"{platform_name.upper()}_HOME_CHANNEL", "")
|
||||
if not chat_id:
|
||||
return None
|
||||
|
||||
@@ -200,30 +160,6 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
}
|
||||
|
||||
|
||||
def _resolve_delivery_targets(job: dict) -> List[dict]:
|
||||
"""Resolve all concrete auto-delivery targets for a cron job (supports comma-separated deliver)."""
|
||||
deliver = job.get("deliver", "local")
|
||||
if deliver == "local":
|
||||
return []
|
||||
parts = [p.strip() for p in str(deliver).split(",") if p.strip()]
|
||||
seen = set()
|
||||
targets = []
|
||||
for part in parts:
|
||||
target = _resolve_single_delivery_target(job, part)
|
||||
if target:
|
||||
key = (target["platform"].lower(), str(target["chat_id"]), target.get("thread_id"))
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
targets.append(target)
|
||||
return targets
|
||||
|
||||
|
||||
def _resolve_delivery_target(job: dict) -> Optional[dict]:
|
||||
"""Resolve the concrete auto-delivery target for a cron job, if any."""
|
||||
targets = _resolve_delivery_targets(job)
|
||||
return targets[0] if targets else None
|
||||
|
||||
|
||||
# Media extension sets — keep in sync with gateway/platforms/base.py:_process_message_background
|
||||
_AUDIO_EXTS = frozenset({'.ogg', '.opus', '.mp3', '.wav', '.m4a'})
|
||||
_VIDEO_EXTS = frozenset({'.mp4', '.mov', '.avi', '.mkv', '.webm', '.3gp'})
|
||||
@@ -264,7 +200,7 @@ def _send_media_via_adapter(adapter, chat_id: str, media_files: list, metadata:
|
||||
|
||||
def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Optional[str]:
|
||||
"""
|
||||
Deliver job output to the configured target(s) (origin chat, specific platform, etc.).
|
||||
Deliver job output to the configured target (origin chat, specific platform, etc.).
|
||||
|
||||
When ``adapters`` and ``loop`` are provided (gateway is running), tries to
|
||||
use the live adapter first — this supports E2EE rooms (e.g. Matrix) where
|
||||
@@ -273,14 +209,33 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Option
|
||||
|
||||
Returns None on success, or an error string on failure.
|
||||
"""
|
||||
targets = _resolve_delivery_targets(job)
|
||||
if not targets:
|
||||
target = _resolve_delivery_target(job)
|
||||
if not target:
|
||||
if job.get("deliver", "local") != "local":
|
||||
msg = f"no delivery target resolved for deliver={job.get('deliver', 'local')}"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
return None # local-only jobs don't deliver — not a failure
|
||||
|
||||
platform_name = target["platform"]
|
||||
chat_id = target["chat_id"]
|
||||
thread_id = target.get("thread_id")
|
||||
|
||||
# Diagnostic: log thread_id for topic-aware delivery debugging
|
||||
origin = job.get("origin") or {}
|
||||
origin_thread = origin.get("thread_id")
|
||||
if origin_thread and not thread_id:
|
||||
logger.warning(
|
||||
"Job '%s': origin has thread_id=%s but delivery target lost it "
|
||||
"(deliver=%s, target=%s)",
|
||||
job["id"], origin_thread, job.get("deliver", "local"), target,
|
||||
)
|
||||
elif thread_id:
|
||||
logger.debug(
|
||||
"Job '%s': delivering to %s:%s thread_id=%s",
|
||||
job["id"], platform_name, chat_id, thread_id,
|
||||
)
|
||||
|
||||
from tools.send_message_tool import _send_to_platform
|
||||
from gateway.config import load_gateway_config, Platform
|
||||
|
||||
@@ -303,6 +258,24 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Option
|
||||
"bluebubbles": Platform.BLUEBUBBLES,
|
||||
"qqbot": Platform.QQBOT,
|
||||
}
|
||||
platform = platform_map.get(platform_name.lower())
|
||||
if not platform:
|
||||
msg = f"unknown platform '{platform_name}'"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
try:
|
||||
config = load_gateway_config()
|
||||
except Exception as e:
|
||||
msg = f"failed to load gateway config: {e}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
pconfig = config.platforms.get(platform)
|
||||
if not pconfig or not pconfig.enabled:
|
||||
msg = f"platform '{platform_name}' not configured/enabled"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
# Optionally wrap the content with a header/footer so the user knows this
|
||||
# is a cron delivery. Wrapping is on by default; set cron.wrap_response: false
|
||||
@@ -331,117 +304,67 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Option
|
||||
from gateway.platforms.base import BasePlatformAdapter
|
||||
media_files, cleaned_delivery_content = BasePlatformAdapter.extract_media(delivery_content)
|
||||
|
||||
# Prefer the live adapter when the gateway is running — this supports E2EE
|
||||
# rooms (e.g. Matrix) where the standalone HTTP path cannot encrypt.
|
||||
runtime_adapter = (adapters or {}).get(platform)
|
||||
if runtime_adapter is not None and loop is not None and getattr(loop, "is_running", lambda: False)():
|
||||
send_metadata = {"thread_id": thread_id} if thread_id else None
|
||||
try:
|
||||
# Send cleaned text (MEDIA tags stripped) — not the raw content
|
||||
text_to_send = cleaned_delivery_content.strip()
|
||||
adapter_ok = True
|
||||
if text_to_send:
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
runtime_adapter.send(chat_id, text_to_send, metadata=send_metadata),
|
||||
loop,
|
||||
)
|
||||
send_result = future.result(timeout=60)
|
||||
if send_result and not getattr(send_result, "success", True):
|
||||
err = getattr(send_result, "error", "unknown")
|
||||
logger.warning(
|
||||
"Job '%s': live adapter send to %s:%s failed (%s), falling back to standalone",
|
||||
job["id"], platform_name, chat_id, err,
|
||||
)
|
||||
adapter_ok = False # fall through to standalone path
|
||||
|
||||
# Send extracted media files as native attachments via the live adapter
|
||||
if adapter_ok and media_files:
|
||||
_send_media_via_adapter(runtime_adapter, chat_id, media_files, send_metadata, loop, job)
|
||||
|
||||
if adapter_ok:
|
||||
logger.info("Job '%s': delivered to %s:%s via live adapter", job["id"], platform_name, chat_id)
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Job '%s': live adapter delivery to %s:%s failed (%s), falling back to standalone",
|
||||
job["id"], platform_name, chat_id, e,
|
||||
)
|
||||
|
||||
# Standalone path: run the async send in a fresh event loop (safe from any thread)
|
||||
coro = _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files)
|
||||
try:
|
||||
config = load_gateway_config()
|
||||
result = asyncio.run(coro)
|
||||
except RuntimeError:
|
||||
# asyncio.run() checks for a running loop before awaiting the coroutine;
|
||||
# when it raises, the original coro was never started — close it to
|
||||
# prevent "coroutine was never awaited" RuntimeWarning, then retry in a
|
||||
# fresh thread that has no running loop.
|
||||
coro.close()
|
||||
import concurrent.futures
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
|
||||
future = pool.submit(asyncio.run, _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files))
|
||||
result = future.result(timeout=30)
|
||||
except Exception as e:
|
||||
msg = f"failed to load gateway config: {e}"
|
||||
msg = f"delivery to {platform_name}:{chat_id} failed: {e}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
delivery_errors = []
|
||||
if result and result.get("error"):
|
||||
msg = f"delivery error: {result['error']}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
for target in targets:
|
||||
platform_name = target["platform"]
|
||||
chat_id = target["chat_id"]
|
||||
thread_id = target.get("thread_id")
|
||||
|
||||
# Diagnostic: log thread_id for topic-aware delivery debugging
|
||||
origin = job.get("origin") or {}
|
||||
origin_thread = origin.get("thread_id")
|
||||
if origin_thread and not thread_id:
|
||||
logger.warning(
|
||||
"Job '%s': origin has thread_id=%s but delivery target lost it "
|
||||
"(deliver=%s, target=%s)",
|
||||
job["id"], origin_thread, job.get("deliver", "local"), target,
|
||||
)
|
||||
elif thread_id:
|
||||
logger.debug(
|
||||
"Job '%s': delivering to %s:%s thread_id=%s",
|
||||
job["id"], platform_name, chat_id, thread_id,
|
||||
)
|
||||
|
||||
platform = platform_map.get(platform_name.lower())
|
||||
if not platform:
|
||||
msg = f"unknown platform '{platform_name}'"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
delivery_errors.append(msg)
|
||||
continue
|
||||
|
||||
# Prefer the live adapter when the gateway is running — this supports E2EE
|
||||
# rooms (e.g. Matrix) where the standalone HTTP path cannot encrypt.
|
||||
runtime_adapter = (adapters or {}).get(platform)
|
||||
delivered = False
|
||||
if runtime_adapter is not None and loop is not None and getattr(loop, "is_running", lambda: False)():
|
||||
send_metadata = {"thread_id": thread_id} if thread_id else None
|
||||
try:
|
||||
# Send cleaned text (MEDIA tags stripped) — not the raw content
|
||||
text_to_send = cleaned_delivery_content.strip()
|
||||
adapter_ok = True
|
||||
if text_to_send:
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
runtime_adapter.send(chat_id, text_to_send, metadata=send_metadata),
|
||||
loop,
|
||||
)
|
||||
send_result = future.result(timeout=60)
|
||||
if send_result and not getattr(send_result, "success", True):
|
||||
err = getattr(send_result, "error", "unknown")
|
||||
logger.warning(
|
||||
"Job '%s': live adapter send to %s:%s failed (%s), falling back to standalone",
|
||||
job["id"], platform_name, chat_id, err,
|
||||
)
|
||||
adapter_ok = False # fall through to standalone path
|
||||
|
||||
# Send extracted media files as native attachments via the live adapter
|
||||
if adapter_ok and media_files:
|
||||
_send_media_via_adapter(runtime_adapter, chat_id, media_files, send_metadata, loop, job)
|
||||
|
||||
if adapter_ok:
|
||||
logger.info("Job '%s': delivered to %s:%s via live adapter", job["id"], platform_name, chat_id)
|
||||
delivered = True
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Job '%s': live adapter delivery to %s:%s failed (%s), falling back to standalone",
|
||||
job["id"], platform_name, chat_id, e,
|
||||
)
|
||||
|
||||
if not delivered:
|
||||
pconfig = config.platforms.get(platform)
|
||||
if not pconfig or not pconfig.enabled:
|
||||
msg = f"platform '{platform_name}' not configured/enabled"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
delivery_errors.append(msg)
|
||||
continue
|
||||
|
||||
# Standalone path: run the async send in a fresh event loop (safe from any thread)
|
||||
coro = _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files)
|
||||
try:
|
||||
result = asyncio.run(coro)
|
||||
except RuntimeError:
|
||||
# asyncio.run() checks for a running loop before awaiting the coroutine;
|
||||
# when it raises, the original coro was never started — close it to
|
||||
# prevent "coroutine was never awaited" RuntimeWarning, then retry in a
|
||||
# fresh thread that has no running loop.
|
||||
coro.close()
|
||||
import concurrent.futures
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
|
||||
future = pool.submit(asyncio.run, _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files))
|
||||
result = future.result(timeout=30)
|
||||
except Exception as e:
|
||||
msg = f"delivery to {platform_name}:{chat_id} failed: {e}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
delivery_errors.append(msg)
|
||||
continue
|
||||
|
||||
if result and result.get("error"):
|
||||
msg = f"delivery error: {result['error']}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
delivery_errors.append(msg)
|
||||
continue
|
||||
|
||||
logger.info("Job '%s': delivered to %s:%s", job["id"], platform_name, chat_id)
|
||||
|
||||
if delivery_errors:
|
||||
return "; ".join(delivery_errors)
|
||||
logger.info("Job '%s': delivered to %s:%s", job["id"], platform_name, chat_id)
|
||||
return None
|
||||
|
||||
|
||||
@@ -564,53 +487,15 @@ def _run_job_script(script_path: str) -> tuple[bool, str]:
|
||||
return False, f"Script execution failed: {exc}"
|
||||
|
||||
|
||||
def _parse_wake_gate(script_output: str) -> bool:
|
||||
"""Parse the last non-empty stdout line of a cron job's pre-check script
|
||||
as a wake gate.
|
||||
|
||||
The convention (ported from nanoclaw #1232): if the last stdout line is
|
||||
JSON like ``{"wakeAgent": false}``, the agent is skipped entirely — no
|
||||
LLM run, no delivery. Any other output (non-JSON, missing flag, gate
|
||||
absent, or ``wakeAgent: true``) means wake the agent normally.
|
||||
|
||||
Returns True if the agent should wake, False to skip.
|
||||
"""
|
||||
if not script_output:
|
||||
return True
|
||||
stripped_lines = [line for line in script_output.splitlines() if line.strip()]
|
||||
if not stripped_lines:
|
||||
return True
|
||||
last_line = stripped_lines[-1].strip()
|
||||
try:
|
||||
gate = json.loads(last_line)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return True
|
||||
if not isinstance(gate, dict):
|
||||
return True
|
||||
return gate.get("wakeAgent", True) is not False
|
||||
|
||||
|
||||
def _build_job_prompt(job: dict, prerun_script: Optional[tuple] = None) -> str:
|
||||
"""Build the effective prompt for a cron job, optionally loading one or more skills first.
|
||||
|
||||
Args:
|
||||
job: The cron job dict.
|
||||
prerun_script: Optional ``(success, stdout)`` from a script that has
|
||||
already been executed by the caller (e.g. for a wake-gate check).
|
||||
When provided, the script is not re-executed and the cached
|
||||
result is used for prompt injection. When omitted, the script
|
||||
(if any) runs inline as before.
|
||||
"""
|
||||
def _build_job_prompt(job: dict) -> str:
|
||||
"""Build the effective prompt for a cron job, optionally loading one or more skills first."""
|
||||
prompt = job.get("prompt", "")
|
||||
skills = job.get("skills")
|
||||
|
||||
# Run data-collection script if configured, inject output as context.
|
||||
script_path = job.get("script")
|
||||
if script_path:
|
||||
if prerun_script is not None:
|
||||
success, script_output = prerun_script
|
||||
else:
|
||||
success, script_output = _run_job_script(script_path)
|
||||
success, script_output = _run_job_script(script_path)
|
||||
if success:
|
||||
if script_output:
|
||||
prompt = (
|
||||
@@ -712,41 +597,13 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||
|
||||
job_id = job["id"]
|
||||
job_name = job["name"]
|
||||
|
||||
# Wake-gate: if this job has a pre-check script, run it BEFORE building
|
||||
# the prompt so a ``{"wakeAgent": false}`` response can short-circuit
|
||||
# the whole agent run. We pass the result into _build_job_prompt so
|
||||
# the script is only executed once.
|
||||
prerun_script = None
|
||||
script_path = job.get("script")
|
||||
if script_path:
|
||||
prerun_script = _run_job_script(script_path)
|
||||
_ran_ok, _script_output = prerun_script
|
||||
if _ran_ok and not _parse_wake_gate(_script_output):
|
||||
logger.info(
|
||||
"Job '%s' (ID: %s): wakeAgent=false, skipping agent run",
|
||||
job_name, job_id,
|
||||
)
|
||||
silent_doc = (
|
||||
f"# Cron Job: {job_name}\n\n"
|
||||
f"**Job ID:** {job_id}\n"
|
||||
f"**Run Time:** {_hermes_now().strftime('%Y-%m-%d %H:%M:%S')}\n\n"
|
||||
"Script gate returned `wakeAgent=false` — agent skipped.\n"
|
||||
)
|
||||
return True, silent_doc, SILENT_MARKER, None
|
||||
|
||||
prompt = _build_job_prompt(job, prerun_script=prerun_script)
|
||||
prompt = _build_job_prompt(job)
|
||||
origin = _resolve_origin(job)
|
||||
_cron_session_id = f"cron_{job_id}_{_hermes_now().strftime('%Y%m%d_%H%M%S')}"
|
||||
|
||||
logger.info("Running job '%s' (ID: %s)", job_name, job_id)
|
||||
logger.info("Prompt: %s", prompt[:100])
|
||||
|
||||
# Mark this as a cron session so the approval system can apply cron_mode.
|
||||
# This env var is process-wide and persists for the lifetime of the
|
||||
# scheduler process — every job this process runs is a cron job.
|
||||
os.environ["HERMES_CRON_SESSION"] = "1"
|
||||
|
||||
try:
|
||||
# Inject origin context so the agent's send_message tool knows the chat.
|
||||
# Must be INSIDE the try block so the finally cleanup always runs.
|
||||
|
||||
@@ -0,0 +1,228 @@
|
||||
# Hermes Agent — ACP (Agent Client Protocol) Setup Guide
|
||||
|
||||
Hermes Agent supports the **Agent Client Protocol (ACP)**, allowing it to run as
|
||||
a coding agent inside your editor. ACP lets your IDE send tasks to Hermes, and
|
||||
Hermes responds with file edits, terminal commands, and explanations — all shown
|
||||
natively in the editor UI.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Hermes Agent installed and configured (`hermes setup` completed)
|
||||
- An API key / provider set up in `~/.hermes/.env` or via `hermes login`
|
||||
- Python 3.11+
|
||||
|
||||
Install the ACP extra:
|
||||
|
||||
```bash
|
||||
pip install -e ".[acp]"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## VS Code Setup
|
||||
|
||||
### 1. Install the ACP Client extension
|
||||
|
||||
Open VS Code and install **ACP Client** from the marketplace:
|
||||
|
||||
- Press `Ctrl+Shift+X` (or `Cmd+Shift+X` on macOS)
|
||||
- Search for **"ACP Client"**
|
||||
- Click **Install**
|
||||
|
||||
Or install from the command line:
|
||||
|
||||
```bash
|
||||
code --install-extension anysphere.acp-client
|
||||
```
|
||||
|
||||
### 2. Configure settings.json
|
||||
|
||||
Open your VS Code settings (`Ctrl+,` → click the `{}` icon for JSON) and add:
|
||||
|
||||
```json
|
||||
{
|
||||
"acpClient.agents": [
|
||||
{
|
||||
"name": "hermes-agent",
|
||||
"registryDir": "/path/to/hermes-agent/acp_registry"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Replace `/path/to/hermes-agent` with the actual path to your Hermes Agent
|
||||
installation (e.g. `~/.hermes/hermes-agent`).
|
||||
|
||||
Alternatively, if `hermes` is on your PATH, the ACP Client can discover it
|
||||
automatically via the registry directory.
|
||||
|
||||
### 3. Restart VS Code
|
||||
|
||||
After configuring, restart VS Code. You should see **Hermes Agent** appear in
|
||||
the ACP agent picker in the chat/agent panel.
|
||||
|
||||
---
|
||||
|
||||
## Zed Setup
|
||||
|
||||
Zed has built-in ACP support.
|
||||
|
||||
### 1. Configure Zed settings
|
||||
|
||||
Open Zed settings (`Cmd+,` on macOS or `Ctrl+,` on Linux) and add to your
|
||||
`settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"agent_servers": {
|
||||
"hermes-agent": {
|
||||
"type": "custom",
|
||||
"command": "hermes",
|
||||
"args": ["acp"],
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Restart Zed
|
||||
|
||||
Hermes Agent will appear in the agent panel. Select it and start a conversation.
|
||||
|
||||
---
|
||||
|
||||
## JetBrains Setup (IntelliJ, PyCharm, WebStorm, etc.)
|
||||
|
||||
### 1. Install the ACP plugin
|
||||
|
||||
- Open **Settings** → **Plugins** → **Marketplace**
|
||||
- Search for **"ACP"** or **"Agent Client Protocol"**
|
||||
- Install and restart the IDE
|
||||
|
||||
### 2. Configure the agent
|
||||
|
||||
- Open **Settings** → **Tools** → **ACP Agents**
|
||||
- Click **+** to add a new agent
|
||||
- Set the registry directory to your `acp_registry/` folder:
|
||||
`/path/to/hermes-agent/acp_registry`
|
||||
- Click **OK**
|
||||
|
||||
### 3. Use the agent
|
||||
|
||||
Open the ACP panel (usually in the right sidebar) and select **Hermes Agent**.
|
||||
|
||||
---
|
||||
|
||||
## What You Will See
|
||||
|
||||
Once connected, your editor provides a native interface to Hermes Agent:
|
||||
|
||||
### Chat Panel
|
||||
A conversational interface where you can describe tasks, ask questions, and
|
||||
give instructions. Hermes responds with explanations and actions.
|
||||
|
||||
### File Diffs
|
||||
When Hermes edits files, you see standard diffs in the editor. You can:
|
||||
- **Accept** individual changes
|
||||
- **Reject** changes you don't want
|
||||
- **Review** the full diff before applying
|
||||
|
||||
### Terminal Commands
|
||||
When Hermes needs to run shell commands (builds, tests, installs), the editor
|
||||
shows them in an integrated terminal. Depending on your settings:
|
||||
- Commands may run automatically
|
||||
- Or you may be prompted to **approve** each command
|
||||
|
||||
### Approval Flow
|
||||
For potentially destructive operations, the editor will prompt you for
|
||||
approval before Hermes proceeds. This includes:
|
||||
- File deletions
|
||||
- Shell commands
|
||||
- Git operations
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
Hermes Agent under ACP uses the **same configuration** as the CLI:
|
||||
|
||||
- **API keys / providers**: `~/.hermes/.env`
|
||||
- **Agent config**: `~/.hermes/config.yaml`
|
||||
- **Skills**: `~/.hermes/skills/`
|
||||
- **Sessions**: `~/.hermes/state.db`
|
||||
|
||||
You can run `hermes setup` to configure providers, or edit `~/.hermes/.env`
|
||||
directly.
|
||||
|
||||
### Changing the model
|
||||
|
||||
Edit `~/.hermes/config.yaml`:
|
||||
|
||||
```yaml
|
||||
model: openrouter/nous/hermes-3-llama-3.1-70b
|
||||
```
|
||||
|
||||
Or set the `HERMES_MODEL` environment variable.
|
||||
|
||||
### Toolsets
|
||||
|
||||
ACP sessions use the curated `hermes-acp` toolset by default. It is designed for editor workflows and intentionally excludes things like messaging delivery, cronjob management, and audio-first UX features.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Agent doesn't appear in the editor
|
||||
|
||||
1. **Check the registry path** — make sure the `acp_registry/` directory path
|
||||
in your editor settings is correct and contains `agent.json`.
|
||||
2. **Check `hermes` is on PATH** — run `which hermes` in a terminal. If not
|
||||
found, you may need to activate your virtualenv or add it to PATH.
|
||||
3. **Restart the editor** after changing settings.
|
||||
|
||||
### Agent starts but errors immediately
|
||||
|
||||
1. Run `hermes doctor` to check your configuration.
|
||||
2. Check that you have a valid API key: `hermes status`
|
||||
3. Try running `hermes acp` directly in a terminal to see error output.
|
||||
|
||||
### "Module not found" errors
|
||||
|
||||
Make sure you installed the ACP extra:
|
||||
|
||||
```bash
|
||||
pip install -e ".[acp]"
|
||||
```
|
||||
|
||||
### Slow responses
|
||||
|
||||
- ACP streams responses, so you should see incremental output. If the agent
|
||||
appears stuck, check your network connection and API provider status.
|
||||
- Some providers have rate limits. Try switching to a different model/provider.
|
||||
|
||||
### Permission denied for terminal commands
|
||||
|
||||
If the editor blocks terminal commands, check your ACP Client extension
|
||||
settings for auto-approval or manual-approval preferences.
|
||||
|
||||
### Logs
|
||||
|
||||
Hermes logs are written to stderr when running in ACP mode. Check:
|
||||
- VS Code: **Output** panel → select **ACP Client** or **Hermes Agent**
|
||||
- Zed: **View** → **Toggle Terminal** and check the process output
|
||||
- JetBrains: **Event Log** or the ACP tool window
|
||||
|
||||
You can also enable verbose logging:
|
||||
|
||||
```bash
|
||||
HERMES_LOG_LEVEL=DEBUG hermes acp
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Further Reading
|
||||
|
||||
- [ACP Specification](https://github.com/anysphere/acp)
|
||||
- [Hermes Agent Documentation](https://github.com/NousResearch/hermes-agent)
|
||||
- Run `hermes --help` for all CLI options
|
||||
@@ -0,0 +1,698 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>honcho-integration-spec</title>
|
||||
<style>
|
||||
:root {
|
||||
--bg: #0b0e14;
|
||||
--bg-surface: #11151c;
|
||||
--bg-elevated: #181d27;
|
||||
--bg-code: #0d1018;
|
||||
--fg: #c9d1d9;
|
||||
--fg-bright: #e6edf3;
|
||||
--fg-muted: #6e7681;
|
||||
--fg-subtle: #484f58;
|
||||
--accent: #7eb8f6;
|
||||
--accent-dim: #3d6ea5;
|
||||
--accent-glow: rgba(126, 184, 246, 0.08);
|
||||
--green: #7ee6a8;
|
||||
--green-dim: #2ea04f;
|
||||
--orange: #e6a855;
|
||||
--red: #f47067;
|
||||
--purple: #bc8cff;
|
||||
--cyan: #56d4dd;
|
||||
--border: #21262d;
|
||||
--border-subtle: #161b22;
|
||||
--radius: 6px;
|
||||
--font-sans: 'New York', ui-serif, 'Iowan Old Style', 'Apple Garamond', Baskerville, 'Times New Roman', 'Noto Emoji', serif;
|
||||
--font-mono: 'Departure Mono', 'Noto Emoji', monospace;
|
||||
}
|
||||
|
||||
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
|
||||
html { scroll-behavior: smooth; scroll-padding-top: 2rem; }
|
||||
body {
|
||||
font-family: var(--font-sans);
|
||||
background: var(--bg);
|
||||
color: var(--fg);
|
||||
line-height: 1.7;
|
||||
font-size: 15px;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
}
|
||||
|
||||
.container { max-width: 860px; margin: 0 auto; padding: 3rem 2rem 6rem; }
|
||||
|
||||
.hero {
|
||||
text-align: center;
|
||||
padding: 4rem 0 3rem;
|
||||
border-bottom: 1px solid var(--border);
|
||||
margin-bottom: 3rem;
|
||||
}
|
||||
.hero h1 { font-family: var(--font-mono); font-size: 2.2rem; font-weight: 700; color: var(--fg-bright); letter-spacing: -0.03em; margin-bottom: 0.5rem; }
|
||||
.hero h1 span { color: var(--accent); }
|
||||
.hero .subtitle { font-family: var(--font-sans); color: var(--fg-muted); font-size: 0.92rem; max-width: 560px; margin: 0 auto; line-height: 1.6; }
|
||||
.hero .meta { margin-top: 1.5rem; display: flex; justify-content: center; gap: 1.5rem; flex-wrap: wrap; }
|
||||
.hero .meta span { font-size: 0.8rem; color: var(--fg-subtle); font-family: var(--font-mono); }
|
||||
|
||||
.toc { background: var(--bg-surface); border: 1px solid var(--border); border-radius: var(--radius); padding: 1.5rem 2rem; margin-bottom: 3rem; }
|
||||
.toc h2 { font-size: 0.75rem; text-transform: uppercase; letter-spacing: 0.1em; color: var(--fg-muted); margin-bottom: 1rem; }
|
||||
.toc ol { list-style: none; counter-reset: toc; columns: 2; column-gap: 2rem; }
|
||||
.toc li { counter-increment: toc; break-inside: avoid; margin-bottom: 0.35rem; }
|
||||
.toc li::before { content: counter(toc, decimal-leading-zero) " "; color: var(--fg-subtle); font-family: var(--font-mono); font-size: 0.75rem; margin-right: 0.25rem; }
|
||||
.toc a { font-family: var(--font-mono); color: var(--fg); text-decoration: none; font-size: 0.82rem; transition: color 0.15s; }
|
||||
.toc a:hover { color: var(--accent); }
|
||||
|
||||
section { margin-bottom: 4rem; }
|
||||
section + section { padding-top: 1rem; }
|
||||
|
||||
h2 { font-family: var(--font-mono); font-size: 1.3rem; font-weight: 700; color: var(--fg-bright); letter-spacing: -0.01em; margin-bottom: 1.25rem; padding-bottom: 0.5rem; border-bottom: 1px solid var(--border); }
|
||||
h3 { font-family: var(--font-mono); font-size: 1rem; font-weight: 600; color: var(--fg-bright); margin-top: 2rem; margin-bottom: 0.75rem; }
|
||||
h4 { font-family: var(--font-mono); font-size: 0.9rem; font-weight: 600; color: var(--accent); margin-top: 1.5rem; margin-bottom: 0.5rem; }
|
||||
|
||||
p { margin-bottom: 1rem; font-size: 0.95rem; line-height: 1.75; }
|
||||
strong { color: var(--fg-bright); font-weight: 600; }
|
||||
a { color: var(--accent); text-decoration: none; }
|
||||
a:hover { text-decoration: underline; }
|
||||
|
||||
ul, ol { margin-bottom: 1rem; padding-left: 1.5rem; font-size: 0.93rem; line-height: 1.7; }
|
||||
li { margin-bottom: 0.35rem; }
|
||||
li::marker { color: var(--fg-subtle); }
|
||||
|
||||
.table-wrap { overflow-x: auto; margin-bottom: 1.5rem; }
|
||||
table { width: 100%; border-collapse: collapse; font-size: 0.88rem; }
|
||||
th, td { text-align: left; padding: 0.6rem 1rem; border-bottom: 1px solid var(--border-subtle); }
|
||||
th { font-family: var(--font-mono); font-size: 0.72rem; text-transform: uppercase; letter-spacing: 0.06em; color: var(--fg-muted); background: var(--bg-surface); border-bottom-color: var(--border); white-space: nowrap; }
|
||||
td { font-family: var(--font-sans); font-size: 0.88rem; color: var(--fg); }
|
||||
tr:hover td { background: var(--accent-glow); }
|
||||
td code { background: var(--bg-elevated); padding: 0.15em 0.4em; border-radius: 3px; font-family: var(--font-mono); font-size: 0.82em; color: var(--cyan); }
|
||||
|
||||
pre { background: var(--bg-code); border: 1px solid var(--border); border-radius: var(--radius); padding: 1.25rem 1.5rem; overflow-x: auto; margin-bottom: 1.5rem; font-family: var(--font-mono); font-size: 0.82rem; line-height: 1.65; color: var(--fg); }
|
||||
pre code { background: none; padding: 0; color: inherit; font-size: inherit; }
|
||||
code { font-family: var(--font-mono); font-size: 0.85em; }
|
||||
p code, li code { background: var(--bg-elevated); padding: 0.15em 0.4em; border-radius: 3px; color: var(--cyan); font-size: 0.85em; }
|
||||
|
||||
.kw { color: var(--purple); }
|
||||
.str { color: var(--green); }
|
||||
.cm { color: var(--fg-subtle); font-style: italic; }
|
||||
.num { color: var(--orange); }
|
||||
.key { color: var(--accent); }
|
||||
|
||||
.mermaid { margin: 1.5rem 0 2rem; text-align: center; }
|
||||
.mermaid svg { max-width: 100%; height: auto; }
|
||||
|
||||
.callout { font-family: var(--font-sans); background: var(--bg-surface); border-left: 3px solid var(--accent-dim); border-radius: 0 var(--radius) var(--radius) 0; padding: 1rem 1.25rem; margin-bottom: 1.5rem; font-size: 0.88rem; color: var(--fg-muted); line-height: 1.6; }
|
||||
.callout strong { font-family: var(--font-mono); color: var(--fg-bright); }
|
||||
.callout.success { border-left-color: var(--green-dim); }
|
||||
.callout.warn { border-left-color: var(--orange); }
|
||||
|
||||
.badge { display: inline-block; font-family: var(--font-mono); font-size: 0.65rem; font-weight: 600; text-transform: uppercase; letter-spacing: 0.05em; padding: 0.2em 0.6em; border-radius: 3px; vertical-align: middle; margin-left: 0.4rem; }
|
||||
.badge-done { background: var(--green-dim); color: #fff; }
|
||||
.badge-wip { background: var(--orange); color: #0b0e14; }
|
||||
.badge-todo { background: var(--fg-subtle); color: var(--fg); }
|
||||
|
||||
.checklist { list-style: none; padding-left: 0; }
|
||||
.checklist li { padding-left: 1.5rem; position: relative; margin-bottom: 0.5rem; }
|
||||
.checklist li::before { position: absolute; left: 0; font-family: var(--font-mono); font-size: 0.85rem; }
|
||||
.checklist li.done { color: var(--fg-muted); }
|
||||
.checklist li.done::before { content: "\2713"; color: var(--green); }
|
||||
.checklist li.todo::before { content: "\25CB"; color: var(--fg-subtle); }
|
||||
.checklist li.wip::before { content: "\25D4"; color: var(--orange); }
|
||||
|
||||
.compare { display: grid; grid-template-columns: 1fr 1fr; gap: 1rem; margin-bottom: 2rem; }
|
||||
.compare-card { background: var(--bg-surface); border: 1px solid var(--border); border-radius: var(--radius); padding: 1.25rem; }
|
||||
.compare-card h4 { margin-top: 0; font-size: 0.82rem; }
|
||||
.compare-card.after { border-color: var(--accent-dim); }
|
||||
.compare-card ul { font-family: var(--font-mono); padding-left: 1.25rem; font-size: 0.8rem; }
|
||||
|
||||
hr { border: none; border-top: 1px solid var(--border); margin: 3rem 0; }
|
||||
|
||||
.progress-bar { position: fixed; top: 0; left: 0; height: 2px; background: var(--accent); z-index: 999; transition: width 0.1s linear; }
|
||||
|
||||
@media (max-width: 640px) {
|
||||
.container { padding: 2rem 1rem 4rem; }
|
||||
.hero h1 { font-size: 1.6rem; }
|
||||
.toc ol { columns: 1; }
|
||||
.compare { grid-template-columns: 1fr; }
|
||||
table { font-size: 0.8rem; }
|
||||
th, td { padding: 0.4rem 0.6rem; }
|
||||
}
|
||||
</style>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link href="https://fonts.googleapis.com/css2?family=Noto+Emoji&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
@font-face {
|
||||
font-family: 'Departure Mono';
|
||||
src: url('https://cdn.jsdelivr.net/gh/rektdeckard/departure-mono@latest/fonts/DepartureMono-Regular.woff2') format('woff2');
|
||||
font-weight: normal;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<div class="progress-bar" id="progress"></div>
|
||||
|
||||
<div class="container">
|
||||
|
||||
<header class="hero">
|
||||
<h1>honcho<span>-integration-spec</span></h1>
|
||||
<p class="subtitle">Comparison of Hermes Agent vs. openclaw-honcho — and a porting spec for bringing Hermes patterns into other Honcho integrations.</p>
|
||||
<div class="meta">
|
||||
<span>hermes-agent / openclaw-honcho</span>
|
||||
<span>Python + TypeScript</span>
|
||||
<span>2026-03-09</span>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<nav class="toc">
|
||||
<h2>Contents</h2>
|
||||
<ol>
|
||||
<li><a href="#overview">Overview</a></li>
|
||||
<li><a href="#architecture">Architecture comparison</a></li>
|
||||
<li><a href="#diff-table">Diff table</a></li>
|
||||
<li><a href="#patterns">Hermes patterns to port</a></li>
|
||||
<li><a href="#spec-async">Spec: async prefetch</a></li>
|
||||
<li><a href="#spec-reasoning">Spec: dynamic reasoning level</a></li>
|
||||
<li><a href="#spec-modes">Spec: per-peer memory modes</a></li>
|
||||
<li><a href="#spec-identity">Spec: AI peer identity formation</a></li>
|
||||
<li><a href="#spec-sessions">Spec: session naming strategies</a></li>
|
||||
<li><a href="#spec-cli">Spec: CLI surface injection</a></li>
|
||||
<li><a href="#openclaw-checklist">openclaw-honcho checklist</a></li>
|
||||
<li><a href="#nanobot-checklist">nanobot-honcho checklist</a></li>
|
||||
</ol>
|
||||
</nav>
|
||||
|
||||
<!-- OVERVIEW -->
|
||||
<section id="overview">
|
||||
<h2>Overview</h2>
|
||||
|
||||
<p>Two independent Honcho integrations have been built for two different agent runtimes: <strong>Hermes Agent</strong> (Python, baked into the runner) and <strong>openclaw-honcho</strong> (TypeScript plugin via hook/tool API). Both use the same Honcho peer paradigm — dual peer model, <code>session.context()</code>, <code>peer.chat()</code> — but they made different tradeoffs at every layer.</p>
|
||||
|
||||
<p>This document maps those tradeoffs and defines a porting spec: a set of Hermes-originated patterns, each stated as an integration-agnostic interface, that any Honcho integration can adopt regardless of runtime or language.</p>
|
||||
|
||||
<div class="callout">
|
||||
<strong>Scope</strong> Both integrations work correctly today. This spec is about the delta — patterns in Hermes that are worth propagating and patterns in openclaw-honcho that Hermes should eventually adopt. The spec is additive, not prescriptive.
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- ARCHITECTURE -->
|
||||
<section id="architecture">
|
||||
<h2>Architecture comparison</h2>
|
||||
|
||||
<h3>Hermes: baked-in runner</h3>
|
||||
<p>Honcho is initialised directly inside <code>AIAgent.__init__</code>. There is no plugin boundary. Session management, context injection, async prefetch, and CLI surface are all first-class concerns of the runner. Context is injected once per session (baked into <code>_cached_system_prompt</code>) and never re-fetched mid-session — this maximises prefix cache hits at the LLM provider.</p>
|
||||
|
||||
<div class="mermaid">
|
||||
%%{init: {'theme': 'dark', 'themeVariables': { 'primaryColor': '#1f3150', 'primaryTextColor': '#c9d1d9', 'primaryBorderColor': '#3d6ea5', 'lineColor': '#3d6ea5', 'secondaryColor': '#162030', 'tertiaryColor': '#11151c' }}}%%
|
||||
flowchart TD
|
||||
U["user message"] --> P["_honcho_prefetch()<br/>(reads cache — no HTTP)"]
|
||||
P --> SP["_build_system_prompt()<br/>(first turn only, cached)"]
|
||||
SP --> LLM["LLM call"]
|
||||
LLM --> R["response"]
|
||||
R --> FP["_honcho_fire_prefetch()<br/>(daemon threads, turn end)"]
|
||||
FP --> C1["prefetch_context() thread"]
|
||||
FP --> C2["prefetch_dialectic() thread"]
|
||||
C1 --> CACHE["_context_cache / _dialectic_cache"]
|
||||
C2 --> CACHE
|
||||
|
||||
style U fill:#162030,stroke:#3d6ea5,color:#c9d1d9
|
||||
style P fill:#1f3150,stroke:#3d6ea5,color:#c9d1d9
|
||||
style SP fill:#1f3150,stroke:#3d6ea5,color:#c9d1d9
|
||||
style LLM fill:#162030,stroke:#3d6ea5,color:#c9d1d9
|
||||
style R fill:#162030,stroke:#3d6ea5,color:#c9d1d9
|
||||
style FP fill:#2a1a40,stroke:#bc8cff,color:#c9d1d9
|
||||
style C1 fill:#2a1a40,stroke:#bc8cff,color:#c9d1d9
|
||||
style C2 fill:#2a1a40,stroke:#bc8cff,color:#c9d1d9
|
||||
style CACHE fill:#11151c,stroke:#484f58,color:#6e7681
|
||||
</div>
|
||||
|
||||
<h3>openclaw-honcho: hook-based plugin</h3>
|
||||
<p>The plugin registers hooks against OpenClaw's event bus. Context is fetched synchronously inside <code>before_prompt_build</code> on every turn. Message capture happens in <code>agent_end</code>. The multi-agent hierarchy is tracked via <code>subagent_spawned</code>. This model is correct but every turn pays a blocking Honcho round-trip before the LLM call can begin.</p>
|
||||
|
||||
<div class="mermaid">
|
||||
%%{init: {'theme': 'dark', 'themeVariables': { 'primaryColor': '#1f3150', 'primaryTextColor': '#c9d1d9', 'primaryBorderColor': '#3d6ea5', 'lineColor': '#3d6ea5', 'secondaryColor': '#162030', 'tertiaryColor': '#11151c' }}}%%
|
||||
flowchart TD
|
||||
U2["user message"] --> BPB["before_prompt_build<br/>(BLOCKING HTTP — every turn)"]
|
||||
BPB --> CTX["session.context()"]
|
||||
CTX --> SP2["system prompt assembled"]
|
||||
SP2 --> LLM2["LLM call"]
|
||||
LLM2 --> R2["response"]
|
||||
R2 --> AE["agent_end hook"]
|
||||
AE --> SAVE["session.addMessages()<br/>session.setMetadata()"]
|
||||
|
||||
style U2 fill:#162030,stroke:#3d6ea5,color:#c9d1d9
|
||||
style BPB fill:#3a1515,stroke:#f47067,color:#c9d1d9
|
||||
style CTX fill:#3a1515,stroke:#f47067,color:#c9d1d9
|
||||
style SP2 fill:#1f3150,stroke:#3d6ea5,color:#c9d1d9
|
||||
style LLM2 fill:#162030,stroke:#3d6ea5,color:#c9d1d9
|
||||
style R2 fill:#162030,stroke:#3d6ea5,color:#c9d1d9
|
||||
style AE fill:#162030,stroke:#3d6ea5,color:#c9d1d9
|
||||
style SAVE fill:#11151c,stroke:#484f58,color:#6e7681
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- DIFF TABLE -->
|
||||
<section id="diff-table">
|
||||
<h2>Diff table</h2>
|
||||
|
||||
<div class="table-wrap">
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Dimension</th>
|
||||
<th>Hermes Agent</th>
|
||||
<th>openclaw-honcho</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><strong>Context injection timing</strong></td>
|
||||
<td>Once per session (cached). Zero HTTP on response path after turn 1.</td>
|
||||
<td>Every turn, blocking. Fresh context per turn but adds latency.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Prefetch strategy</strong></td>
|
||||
<td>Daemon threads fire at turn end; consumed next turn from cache.</td>
|
||||
<td>None. Blocking call at prompt-build time.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Dialectic (peer.chat)</strong></td>
|
||||
<td>Prefetched async; result injected into system prompt next turn.</td>
|
||||
<td>On-demand via <code>honcho_recall</code> / <code>honcho_analyze</code> tools.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Reasoning level</strong></td>
|
||||
<td>Dynamic: scales with message length. Floor = config default. Cap = "high".</td>
|
||||
<td>Fixed per tool: recall=minimal, analyze=medium.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Memory modes</strong></td>
|
||||
<td><code>user_memory_mode</code> / <code>agent_memory_mode</code>: hybrid / honcho / local.</td>
|
||||
<td>None. Always writes to Honcho.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Write frequency</strong></td>
|
||||
<td>async (background queue), turn, session, N turns.</td>
|
||||
<td>After every agent_end (no control).</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>AI peer identity</strong></td>
|
||||
<td><code>observe_me=True</code>, <code>seed_ai_identity()</code>, <code>get_ai_representation()</code>, SOUL.md → AI peer.</td>
|
||||
<td>Agent files uploaded to agent peer at setup. No ongoing self-observation seeding.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Context scope</strong></td>
|
||||
<td>User peer + AI peer representation, both injected.</td>
|
||||
<td>User peer (owner) representation + conversation summary. <code>peerPerspective</code> on context call.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Session naming</strong></td>
|
||||
<td>per-directory / global / manual map / title-based.</td>
|
||||
<td>Derived from platform session key.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Multi-agent</strong></td>
|
||||
<td>Single-agent only.</td>
|
||||
<td>Parent observer hierarchy via <code>subagent_spawned</code>.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Tool surface</strong></td>
|
||||
<td>Single <code>query_user_context</code> tool (on-demand dialectic).</td>
|
||||
<td>6 tools: session, profile, search, context (fast) + recall, analyze (LLM).</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Platform metadata</strong></td>
|
||||
<td>Not stripped.</td>
|
||||
<td>Explicitly stripped before Honcho storage.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Message dedup</strong></td>
|
||||
<td>None (sends on every save cycle).</td>
|
||||
<td><code>lastSavedIndex</code> in session metadata prevents re-sending.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>CLI surface in prompt</strong></td>
|
||||
<td>Management commands injected into system prompt. Agent knows its own CLI.</td>
|
||||
<td>Not injected.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>AI peer name in identity</strong></td>
|
||||
<td>Replaces "Hermes Agent" in DEFAULT_AGENT_IDENTITY when configured.</td>
|
||||
<td>Not implemented.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>QMD / local file search</strong></td>
|
||||
<td>Not implemented.</td>
|
||||
<td>Passthrough tools when QMD backend configured.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>Workspace metadata</strong></td>
|
||||
<td>Not implemented.</td>
|
||||
<td><code>agentPeerMap</code> in workspace metadata tracks agent→peer ID.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- PATTERNS -->
|
||||
<section id="patterns">
|
||||
<h2>Hermes patterns to port</h2>
|
||||
|
||||
<p>Six patterns from Hermes are worth adopting in any Honcho integration. They are described below as integration-agnostic interfaces — the implementation will differ per runtime, but the contract is the same.</p>
|
||||
|
||||
<div class="compare">
|
||||
<div class="compare-card">
|
||||
<h4>Patterns Hermes contributes</h4>
|
||||
<ul>
|
||||
<li>Async prefetch (zero-latency)</li>
|
||||
<li>Dynamic reasoning level</li>
|
||||
<li>Per-peer memory modes</li>
|
||||
<li>AI peer identity formation</li>
|
||||
<li>Session naming strategies</li>
|
||||
<li>CLI surface injection</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="compare-card after">
|
||||
<h4>Patterns openclaw contributes back</h4>
|
||||
<ul>
|
||||
<li>lastSavedIndex dedup</li>
|
||||
<li>Platform metadata stripping</li>
|
||||
<li>Multi-agent observer hierarchy</li>
|
||||
<li>peerPerspective on context()</li>
|
||||
<li>Tiered tool surface (fast/LLM)</li>
|
||||
<li>Workspace agentPeerMap</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- SPEC: ASYNC PREFETCH -->
|
||||
<section id="spec-async">
|
||||
<h2>Spec: async prefetch</h2>
|
||||
|
||||
<h3>Problem</h3>
|
||||
<p>Calling <code>session.context()</code> and <code>peer.chat()</code> synchronously before each LLM call adds 200–800ms of Honcho round-trip latency to every turn. Users experience this as the agent "thinking slowly."</p>
|
||||
|
||||
<h3>Pattern</h3>
|
||||
<p>Fire both calls as non-blocking background work at the <strong>end</strong> of each turn. Store results in a per-session cache keyed by session ID. At the <strong>start</strong> of the next turn, pop from cache — the HTTP is already done. First turn is cold (empty cache); all subsequent turns are zero-latency on the response path.</p>
|
||||
|
||||
<h3>Interface contract</h3>
|
||||
<pre><code><span class="cm">// TypeScript (openclaw / nanobot plugin shape)</span>
|
||||
|
||||
<span class="kw">interface</span> <span class="key">AsyncPrefetch</span> {
|
||||
<span class="cm">// Fire context + dialectic fetches at turn end. Non-blocking.</span>
|
||||
firePrefetch(sessionId: <span class="str">string</span>, userMessage: <span class="str">string</span>): <span class="kw">void</span>;
|
||||
|
||||
<span class="cm">// Pop cached results at turn start. Returns empty if cache is cold.</span>
|
||||
popContextResult(sessionId: <span class="str">string</span>): ContextResult | <span class="kw">null</span>;
|
||||
popDialecticResult(sessionId: <span class="str">string</span>): <span class="str">string</span> | <span class="kw">null</span>;
|
||||
}
|
||||
|
||||
<span class="kw">type</span> <span class="key">ContextResult</span> = {
|
||||
representation: <span class="str">string</span>;
|
||||
card: <span class="str">string</span>[];
|
||||
aiRepresentation?: <span class="str">string</span>; <span class="cm">// AI peer context if enabled</span>
|
||||
summary?: <span class="str">string</span>; <span class="cm">// conversation summary if fetched</span>
|
||||
};</code></pre>
|
||||
|
||||
<h3>Implementation notes</h3>
|
||||
<ul>
|
||||
<li>Python: <code>threading.Thread(daemon=True)</code>. Write to <code>dict[session_id, result]</code> — GIL makes this safe for simple writes.</li>
|
||||
<li>TypeScript: <code>Promise</code> stored in <code>Map<string, Promise<ContextResult>></code>. Await at pop time. If not resolved yet, skip (return null) — do not block.</li>
|
||||
<li>The pop is destructive: clears the cache entry after reading so stale data never accumulates.</li>
|
||||
<li>Prefetch should also fire on first turn (even though it won't be consumed until turn 2) — this ensures turn 2 is never cold.</li>
|
||||
</ul>
|
||||
|
||||
<h3>openclaw-honcho adoption</h3>
|
||||
<p>Move <code>session.context()</code> from <code>before_prompt_build</code> to a post-<code>agent_end</code> background task. Store result in <code>state.contextCache</code>. In <code>before_prompt_build</code>, read from cache instead of calling Honcho. If cache is empty (turn 1), inject nothing — the prompt is still valid without Honcho context on the first turn.</p>
|
||||
</section>
|
||||
|
||||
<!-- SPEC: DYNAMIC REASONING LEVEL -->
|
||||
<section id="spec-reasoning">
|
||||
<h2>Spec: dynamic reasoning level</h2>
|
||||
|
||||
<h3>Problem</h3>
|
||||
<p>Honcho's dialectic endpoint supports reasoning levels from <code>minimal</code> to <code>max</code>. A fixed level per tool wastes budget on simple queries and under-serves complex ones.</p>
|
||||
|
||||
<h3>Pattern</h3>
|
||||
<p>Select the reasoning level dynamically based on the user's message. Use the configured default as a floor. Bump by message length. Cap auto-selection at <code>high</code> — never select <code>max</code> automatically.</p>
|
||||
|
||||
<h3>Interface contract</h3>
|
||||
<pre><code><span class="cm">// Shared helper — identical logic in any language</span>
|
||||
|
||||
<span class="kw">const</span> LEVELS = [<span class="str">"minimal"</span>, <span class="str">"low"</span>, <span class="str">"medium"</span>, <span class="str">"high"</span>, <span class="str">"max"</span>];
|
||||
|
||||
<span class="kw">function</span> <span class="key">dynamicReasoningLevel</span>(
|
||||
query: <span class="str">string</span>,
|
||||
configDefault: <span class="str">string</span> = <span class="str">"low"</span>
|
||||
): <span class="str">string</span> {
|
||||
<span class="kw">const</span> baseIdx = Math.max(<span class="num">0</span>, LEVELS.indexOf(configDefault));
|
||||
<span class="kw">const</span> n = query.length;
|
||||
<span class="kw">const</span> bump = n < <span class="num">120</span> ? <span class="num">0</span> : n < <span class="num">400</span> ? <span class="num">1</span> : <span class="num">2</span>;
|
||||
<span class="kw">return</span> LEVELS[Math.min(baseIdx + bump, <span class="num">3</span>)]; <span class="cm">// cap at "high" (idx 3)</span>
|
||||
}</code></pre>
|
||||
|
||||
<h3>Config key</h3>
|
||||
<p>Add a <code>dialecticReasoningLevel</code> config field (string, default <code>"low"</code>). This sets the floor. Users can raise or lower it. The dynamic bump always applies on top.</p>
|
||||
|
||||
<h3>openclaw-honcho adoption</h3>
|
||||
<p>Apply in <code>honcho_recall</code> and <code>honcho_analyze</code>: replace the fixed <code>reasoningLevel</code> with the dynamic selector. <code>honcho_recall</code> should use floor <code>"minimal"</code> and <code>honcho_analyze</code> floor <code>"medium"</code> — both still bump with message length.</p>
|
||||
</section>
|
||||
|
||||
<!-- SPEC: PER-PEER MEMORY MODES -->
|
||||
<section id="spec-modes">
|
||||
<h2>Spec: per-peer memory modes</h2>
|
||||
|
||||
<h3>Problem</h3>
|
||||
<p>Users want independent control over whether user context and agent context are written locally, to Honcho, or both. A single <code>memoryMode</code> shorthand is not granular enough.</p>
|
||||
|
||||
<h3>Pattern</h3>
|
||||
<p>Three modes per peer: <code>hybrid</code> (write both local + Honcho), <code>honcho</code> (Honcho only, disable local files), <code>local</code> (local files only, skip Honcho sync for this peer). Two orthogonal axes: user peer and agent peer.</p>
|
||||
|
||||
<h3>Config schema</h3>
|
||||
<pre><code><span class="cm">// ~/.openclaw/openclaw.json (or ~/.nanobot/config.json)</span>
|
||||
{
|
||||
<span class="str">"plugins"</span>: {
|
||||
<span class="str">"openclaw-honcho"</span>: {
|
||||
<span class="str">"config"</span>: {
|
||||
<span class="str">"apiKey"</span>: <span class="str">"..."</span>,
|
||||
<span class="str">"memoryMode"</span>: <span class="str">"hybrid"</span>, <span class="cm">// shorthand: both peers</span>
|
||||
<span class="str">"userMemoryMode"</span>: <span class="str">"honcho"</span>, <span class="cm">// override for user peer</span>
|
||||
<span class="str">"agentMemoryMode"</span>: <span class="str">"hybrid"</span> <span class="cm">// override for agent peer</span>
|
||||
}
|
||||
}
|
||||
}
|
||||
}</code></pre>
|
||||
|
||||
<h3>Resolution order</h3>
|
||||
<ol>
|
||||
<li>Per-peer field (<code>userMemoryMode</code> / <code>agentMemoryMode</code>) — wins if present.</li>
|
||||
<li>Shorthand <code>memoryMode</code> — applies to both peers as default.</li>
|
||||
<li>Hardcoded default: <code>"hybrid"</code>.</li>
|
||||
</ol>
|
||||
|
||||
<h3>Effect on Honcho sync</h3>
|
||||
<ul>
|
||||
<li><code>userMemoryMode=local</code>: skip adding user peer messages to Honcho.</li>
|
||||
<li><code>agentMemoryMode=local</code>: skip adding assistant peer messages to Honcho.</li>
|
||||
<li>Both local: skip <code>session.addMessages()</code> entirely.</li>
|
||||
<li><code>userMemoryMode=honcho</code>: disable local USER.md writes.</li>
|
||||
<li><code>agentMemoryMode=honcho</code>: disable local MEMORY.md / SOUL.md writes.</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<!-- SPEC: AI PEER IDENTITY -->
|
||||
<section id="spec-identity">
|
||||
<h2>Spec: AI peer identity formation</h2>
|
||||
|
||||
<h3>Problem</h3>
|
||||
<p>Honcho builds the user's representation organically by observing what the user says. The same mechanism exists for the AI peer — but only if <code>observe_me=True</code> is set for the agent peer. Without it, the agent peer accumulates nothing and Honcho's AI-side model never forms.</p>
|
||||
|
||||
<p>Additionally, existing persona files (SOUL.md, IDENTITY.md) should seed the AI peer's Honcho representation at first activation, rather than waiting for it to emerge from scratch.</p>
|
||||
|
||||
<h3>Part A: observe_me=True for agent peer</h3>
|
||||
<pre><code><span class="cm">// TypeScript — in session.addPeers() call</span>
|
||||
<span class="kw">await</span> session.addPeers([
|
||||
[ownerPeer.id, { observeMe: <span class="kw">true</span>, observeOthers: <span class="kw">false</span> }],
|
||||
[agentPeer.id, { observeMe: <span class="kw">true</span>, observeOthers: <span class="kw">true</span> }], <span class="cm">// was false</span>
|
||||
]);</code></pre>
|
||||
|
||||
<p>This is a one-line change but foundational. Without it, Honcho's AI peer representation stays empty regardless of what the agent says.</p>
|
||||
|
||||
<h3>Part B: seedAiIdentity()</h3>
|
||||
<pre><code><span class="kw">async function</span> <span class="key">seedAiIdentity</span>(
|
||||
session: HonchoSession,
|
||||
agentPeer: Peer,
|
||||
content: <span class="str">string</span>,
|
||||
source: <span class="str">string</span>
|
||||
): Promise<<span class="kw">boolean</span>> {
|
||||
<span class="kw">const</span> wrapped = [
|
||||
<span class="str">`<ai_identity_seed>`</span>,
|
||||
<span class="str">`<source>${source}</source>`</span>,
|
||||
<span class="str">``</span>,
|
||||
content.trim(),
|
||||
<span class="str">`</ai_identity_seed>`</span>,
|
||||
].join(<span class="str">"\n"</span>);
|
||||
|
||||
<span class="kw">await</span> agentPeer.addMessage(<span class="str">"assistant"</span>, wrapped);
|
||||
<span class="kw">return true</span>;
|
||||
}</code></pre>
|
||||
|
||||
<h3>Part C: migrate agent files at setup</h3>
|
||||
<p>During <code>openclaw honcho setup</code>, upload agent-self files (SOUL.md, IDENTITY.md, AGENTS.md, BOOTSTRAP.md) to the agent peer using <code>seedAiIdentity()</code> instead of <code>session.uploadFile()</code>. This routes the content through Honcho's observation pipeline rather than the file store.</p>
|
||||
|
||||
<h3>Part D: AI peer name in identity</h3>
|
||||
<p>When the agent has a configured name (non-default), inject it into the agent's self-identity prefix. In OpenClaw this means adding to the injected system prompt section:</p>
|
||||
<pre><code><span class="cm">// In context hook return value</span>
|
||||
<span class="kw">return</span> {
|
||||
systemPrompt: [
|
||||
agentName ? <span class="str">`You are ${agentName}.`</span> : <span class="str">""</span>,
|
||||
<span class="str">"## User Memory Context"</span>,
|
||||
...sections,
|
||||
].filter(Boolean).join(<span class="str">"\n\n"</span>)
|
||||
};</code></pre>
|
||||
|
||||
<h3>CLI surface: honcho identity subcommand</h3>
|
||||
<pre><code>openclaw honcho identity <file> <span class="cm"># seed from file</span>
|
||||
openclaw honcho identity --show <span class="cm"># show current AI peer representation</span></code></pre>
|
||||
</section>
|
||||
|
||||
<!-- SPEC: SESSION NAMING -->
|
||||
<section id="spec-sessions">
|
||||
<h2>Spec: session naming strategies</h2>
|
||||
|
||||
<h3>Problem</h3>
|
||||
<p>When Honcho is used across multiple projects or directories, a single global session means every project shares the same context. Per-directory sessions provide isolation without requiring users to name sessions manually.</p>
|
||||
|
||||
<h3>Strategies</h3>
|
||||
<div class="table-wrap">
|
||||
<table>
|
||||
<thead><tr><th>Strategy</th><th>Session key</th><th>When to use</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td><code>per-directory</code></td><td>basename of CWD</td><td>Default. Each project gets its own session.</td></tr>
|
||||
<tr><td><code>global</code></td><td>fixed string <code>"global"</code></td><td>Single cross-project session.</td></tr>
|
||||
<tr><td>manual map</td><td>user-configured per path</td><td><code>sessions</code> config map overrides directory basename.</td></tr>
|
||||
<tr><td>title-based</td><td>sanitized session title</td><td>When agent supports named sessions; title set mid-conversation.</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<h3>Config schema</h3>
|
||||
<pre><code>{
|
||||
<span class="str">"sessionStrategy"</span>: <span class="str">"per-directory"</span>, <span class="cm">// "per-directory" | "global"</span>
|
||||
<span class="str">"sessionPeerPrefix"</span>: <span class="kw">false</span>, <span class="cm">// prepend peer name to session key</span>
|
||||
<span class="str">"sessions"</span>: { <span class="cm">// manual overrides</span>
|
||||
<span class="str">"/home/user/projects/foo"</span>: <span class="str">"foo-project"</span>
|
||||
}
|
||||
}</code></pre>
|
||||
|
||||
<h3>CLI surface</h3>
|
||||
<pre><code>openclaw honcho sessions <span class="cm"># list all mappings</span>
|
||||
openclaw honcho map <name> <span class="cm"># map cwd to session name</span>
|
||||
openclaw honcho map <span class="cm"># no-arg = list mappings</span></code></pre>
|
||||
|
||||
<p>Resolution order: manual map wins → session title → directory basename → platform key.</p>
|
||||
</section>
|
||||
|
||||
<!-- SPEC: CLI SURFACE INJECTION -->
|
||||
<section id="spec-cli">
|
||||
<h2>Spec: CLI surface injection</h2>
|
||||
|
||||
<h3>Problem</h3>
|
||||
<p>When a user asks "how do I change my memory settings?" or "what Honcho commands are available?" the agent either hallucinates or says it doesn't know. The agent should know its own management interface.</p>
|
||||
|
||||
<h3>Pattern</h3>
|
||||
<p>When Honcho is active, append a compact command reference to the system prompt. The agent can cite these commands directly instead of guessing.</p>
|
||||
|
||||
<pre><code><span class="cm">// In context hook, append to systemPrompt</span>
|
||||
<span class="kw">const</span> honchoSection = [
|
||||
<span class="str">"# Honcho memory integration"</span>,
|
||||
<span class="str">`Active. Session: ${sessionKey}. Mode: ${mode}.`</span>,
|
||||
<span class="str">"Management commands:"</span>,
|
||||
<span class="str">" openclaw honcho status — show config + connection"</span>,
|
||||
<span class="str">" openclaw honcho mode [hybrid|honcho|local] — show or set memory mode"</span>,
|
||||
<span class="str">" openclaw honcho sessions — list session mappings"</span>,
|
||||
<span class="str">" openclaw honcho map <name> — map directory to session"</span>,
|
||||
<span class="str">" openclaw honcho identity [file] [--show] — seed or show AI identity"</span>,
|
||||
<span class="str">" openclaw honcho setup — full interactive wizard"</span>,
|
||||
].join(<span class="str">"\n"</span>);</code></pre>
|
||||
|
||||
<div class="callout warn">
|
||||
<strong>Keep it compact.</strong> This section is injected every turn. Keep it under 300 chars of context. List commands, not explanations — the agent can explain them on request.
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- OPENCLAW CHECKLIST -->
|
||||
<section id="openclaw-checklist">
|
||||
<h2>openclaw-honcho checklist</h2>
|
||||
|
||||
<p>Ordered by impact. Each item maps to a spec section above.</p>
|
||||
|
||||
<ul class="checklist">
|
||||
<li class="todo"><strong>Async prefetch</strong> — move <code>session.context()</code> out of <code>before_prompt_build</code> into post-<code>agent_end</code> background Promise. Pop from cache at prompt build. (<a href="#spec-async">spec</a>)</li>
|
||||
<li class="todo"><strong>observe_me=True for agent peer</strong> — one-line change in <code>session.addPeers()</code> config for agent peer. (<a href="#spec-identity">spec</a>)</li>
|
||||
<li class="todo"><strong>Dynamic reasoning level</strong> — add <code>dynamicReasoningLevel()</code> helper; apply in <code>honcho_recall</code> and <code>honcho_analyze</code>. Add <code>dialecticReasoningLevel</code> to config schema. (<a href="#spec-reasoning">spec</a>)</li>
|
||||
<li class="todo"><strong>Per-peer memory modes</strong> — add <code>userMemoryMode</code> / <code>agentMemoryMode</code> to config; gate Honcho sync and local writes accordingly. (<a href="#spec-modes">spec</a>)</li>
|
||||
<li class="todo"><strong>seedAiIdentity()</strong> — add helper; apply during setup migration for SOUL.md / IDENTITY.md instead of <code>session.uploadFile()</code>. (<a href="#spec-identity">spec</a>)</li>
|
||||
<li class="todo"><strong>Session naming strategies</strong> — add <code>sessionStrategy</code>, <code>sessions</code> map, <code>sessionPeerPrefix</code> to config; implement resolution function. (<a href="#spec-sessions">spec</a>)</li>
|
||||
<li class="todo"><strong>CLI surface injection</strong> — append command reference to <code>before_prompt_build</code> return value when Honcho is active. (<a href="#spec-cli">spec</a>)</li>
|
||||
<li class="todo"><strong>honcho identity subcommand</strong> — add <code>openclaw honcho identity</code> CLI command. (<a href="#spec-identity">spec</a>)</li>
|
||||
<li class="todo"><strong>AI peer name injection</strong> — if <code>aiPeer</code> name configured, prepend to injected system prompt. (<a href="#spec-identity">spec</a>)</li>
|
||||
<li class="todo"><strong>honcho mode / honcho sessions / honcho map</strong> — CLI parity with Hermes. (<a href="#spec-sessions">spec</a>)</li>
|
||||
</ul>
|
||||
|
||||
<div class="callout success">
|
||||
<strong>Already done in openclaw-honcho (do not re-implement):</strong> lastSavedIndex dedup, platform metadata stripping, multi-agent parent observer hierarchy, peerPerspective on context(), tiered tool surface (fast/LLM), workspace agentPeerMap, QMD passthrough, self-hosted Honcho support.
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- NANOBOT CHECKLIST -->
|
||||
<section id="nanobot-checklist">
|
||||
<h2>nanobot-honcho checklist</h2>
|
||||
|
||||
<p>nanobot-honcho is a greenfield integration. Start from openclaw-honcho's architecture (hook-based, dual peer) and apply all Hermes patterns from day one rather than retrofitting. Priority order:</p>
|
||||
|
||||
<h3>Phase 1 — core correctness</h3>
|
||||
<ul class="checklist">
|
||||
<li class="todo">Dual peer model (owner + agent peer), both with <code>observe_me=True</code></li>
|
||||
<li class="todo">Message capture at turn end with <code>lastSavedIndex</code> dedup</li>
|
||||
<li class="todo">Platform metadata stripping before Honcho storage</li>
|
||||
<li class="todo">Async prefetch from day one — do not implement blocking context injection</li>
|
||||
<li class="todo">Legacy file migration at first activation (USER.md → owner peer, SOUL.md → <code>seedAiIdentity()</code>)</li>
|
||||
</ul>
|
||||
|
||||
<h3>Phase 2 — configuration</h3>
|
||||
<ul class="checklist">
|
||||
<li class="todo">Config schema: <code>apiKey</code>, <code>workspaceId</code>, <code>baseUrl</code>, <code>memoryMode</code>, <code>userMemoryMode</code>, <code>agentMemoryMode</code>, <code>dialecticReasoningLevel</code>, <code>sessionStrategy</code>, <code>sessions</code></li>
|
||||
<li class="todo">Per-peer memory mode gating</li>
|
||||
<li class="todo">Dynamic reasoning level</li>
|
||||
<li class="todo">Session naming strategies</li>
|
||||
</ul>
|
||||
|
||||
<h3>Phase 3 — tools and CLI</h3>
|
||||
<ul class="checklist">
|
||||
<li class="todo">Tool surface: <code>honcho_profile</code>, <code>honcho_recall</code>, <code>honcho_analyze</code>, <code>honcho_search</code>, <code>honcho_context</code></li>
|
||||
<li class="todo">CLI: <code>setup</code>, <code>status</code>, <code>sessions</code>, <code>map</code>, <code>mode</code>, <code>identity</code></li>
|
||||
<li class="todo">CLI surface injection into system prompt</li>
|
||||
<li class="todo">AI peer name wired into agent identity</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
</div>
|
||||
|
||||
<script type="module">
|
||||
import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs';
|
||||
mermaid.initialize({ startOnLoad: true, securityLevel: 'loose', fontFamily: 'Departure Mono, Noto Emoji, monospace' });
|
||||
</script>
|
||||
<script>
|
||||
window.addEventListener('scroll', () => {
|
||||
const bar = document.getElementById('progress');
|
||||
const max = document.documentElement.scrollHeight - window.innerHeight;
|
||||
bar.style.width = (max > 0 ? (window.scrollY / max) * 100 : 0) + '%';
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,377 @@
|
||||
# honcho-integration-spec
|
||||
|
||||
Comparison of Hermes Agent vs. openclaw-honcho — and a porting spec for bringing Hermes patterns into other Honcho integrations.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Two independent Honcho integrations have been built for two different agent runtimes: **Hermes Agent** (Python, baked into the runner) and **openclaw-honcho** (TypeScript plugin via hook/tool API). Both use the same Honcho peer paradigm — dual peer model, `session.context()`, `peer.chat()` — but they made different tradeoffs at every layer.
|
||||
|
||||
This document maps those tradeoffs and defines a porting spec: a set of Hermes-originated patterns, each stated as an integration-agnostic interface, that any Honcho integration can adopt regardless of runtime or language.
|
||||
|
||||
> **Scope** Both integrations work correctly today. This spec is about the delta — patterns in Hermes that are worth propagating and patterns in openclaw-honcho that Hermes should eventually adopt. The spec is additive, not prescriptive.
|
||||
|
||||
---
|
||||
|
||||
## Architecture comparison
|
||||
|
||||
### Hermes: baked-in runner
|
||||
|
||||
Honcho is initialised directly inside `AIAgent.__init__`. There is no plugin boundary. Session management, context injection, async prefetch, and CLI surface are all first-class concerns of the runner. Context is injected once per session (baked into `_cached_system_prompt`) and never re-fetched mid-session — this maximises prefix cache hits at the LLM provider.
|
||||
|
||||
Turn flow:
|
||||
|
||||
```
|
||||
user message
|
||||
→ _honcho_prefetch() (reads cache — no HTTP)
|
||||
→ _build_system_prompt() (first turn only, cached)
|
||||
→ LLM call
|
||||
→ response
|
||||
→ _honcho_fire_prefetch() (daemon threads, turn end)
|
||||
→ prefetch_context() thread ──┐
|
||||
→ prefetch_dialectic() thread ─┴→ _context_cache / _dialectic_cache
|
||||
```
|
||||
|
||||
### openclaw-honcho: hook-based plugin
|
||||
|
||||
The plugin registers hooks against OpenClaw's event bus. Context is fetched synchronously inside `before_prompt_build` on every turn. Message capture happens in `agent_end`. The multi-agent hierarchy is tracked via `subagent_spawned`. This model is correct but every turn pays a blocking Honcho round-trip before the LLM call can begin.
|
||||
|
||||
Turn flow:
|
||||
|
||||
```
|
||||
user message
|
||||
→ before_prompt_build (BLOCKING HTTP — every turn)
|
||||
→ session.context()
|
||||
→ system prompt assembled
|
||||
→ LLM call
|
||||
→ response
|
||||
→ agent_end hook
|
||||
→ session.addMessages()
|
||||
→ session.setMetadata()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Diff table
|
||||
|
||||
| Dimension | Hermes Agent | openclaw-honcho |
|
||||
|---|---|---|
|
||||
| **Context injection timing** | Once per session (cached). Zero HTTP on response path after turn 1. | Every turn, blocking. Fresh context per turn but adds latency. |
|
||||
| **Prefetch strategy** | Daemon threads fire at turn end; consumed next turn from cache. | None. Blocking call at prompt-build time. |
|
||||
| **Dialectic (peer.chat)** | Prefetched async; result injected into system prompt next turn. | On-demand via `honcho_recall` / `honcho_analyze` tools. |
|
||||
| **Reasoning level** | Dynamic: scales with message length. Floor = config default. Cap = "high". | Fixed per tool: recall=minimal, analyze=medium. |
|
||||
| **Memory modes** | `user_memory_mode` / `agent_memory_mode`: hybrid / honcho / local. | None. Always writes to Honcho. |
|
||||
| **Write frequency** | async (background queue), turn, session, N turns. | After every agent_end (no control). |
|
||||
| **AI peer identity** | `observe_me=True`, `seed_ai_identity()`, `get_ai_representation()`, SOUL.md → AI peer. | Agent files uploaded to agent peer at setup. No ongoing self-observation. |
|
||||
| **Context scope** | User peer + AI peer representation, both injected. | User peer (owner) representation + conversation summary. `peerPerspective` on context call. |
|
||||
| **Session naming** | per-directory / global / manual map / title-based. | Derived from platform session key. |
|
||||
| **Multi-agent** | Single-agent only. | Parent observer hierarchy via `subagent_spawned`. |
|
||||
| **Tool surface** | Single `query_user_context` tool (on-demand dialectic). | 6 tools: session, profile, search, context (fast) + recall, analyze (LLM). |
|
||||
| **Platform metadata** | Not stripped. | Explicitly stripped before Honcho storage. |
|
||||
| **Message dedup** | None. | `lastSavedIndex` in session metadata prevents re-sending. |
|
||||
| **CLI surface in prompt** | Management commands injected into system prompt. Agent knows its own CLI. | Not injected. |
|
||||
| **AI peer name in identity** | Replaces "Hermes Agent" in DEFAULT_AGENT_IDENTITY when configured. | Not implemented. |
|
||||
| **QMD / local file search** | Not implemented. | Passthrough tools when QMD backend configured. |
|
||||
| **Workspace metadata** | Not implemented. | `agentPeerMap` in workspace metadata tracks agent→peer ID. |
|
||||
|
||||
---
|
||||
|
||||
## Patterns
|
||||
|
||||
Six patterns from Hermes are worth adopting in any Honcho integration. Each is described as an integration-agnostic interface.
|
||||
|
||||
**Hermes contributes:**
|
||||
- Async prefetch (zero-latency)
|
||||
- Dynamic reasoning level
|
||||
- Per-peer memory modes
|
||||
- AI peer identity formation
|
||||
- Session naming strategies
|
||||
- CLI surface injection
|
||||
|
||||
**openclaw-honcho contributes back (Hermes should adopt):**
|
||||
- `lastSavedIndex` dedup
|
||||
- Platform metadata stripping
|
||||
- Multi-agent observer hierarchy
|
||||
- `peerPerspective` on `context()`
|
||||
- Tiered tool surface (fast/LLM)
|
||||
- Workspace `agentPeerMap`
|
||||
|
||||
---
|
||||
|
||||
## Spec: async prefetch
|
||||
|
||||
### Problem
|
||||
|
||||
Calling `session.context()` and `peer.chat()` synchronously before each LLM call adds 200–800ms of Honcho round-trip latency to every turn.
|
||||
|
||||
### Pattern
|
||||
|
||||
Fire both calls as non-blocking background work at the **end** of each turn. Store results in a per-session cache keyed by session ID. At the **start** of the next turn, pop from cache — the HTTP is already done. First turn is cold (empty cache); all subsequent turns are zero-latency on the response path.
|
||||
|
||||
### Interface contract
|
||||
|
||||
```typescript
|
||||
interface AsyncPrefetch {
|
||||
// Fire context + dialectic fetches at turn end. Non-blocking.
|
||||
firePrefetch(sessionId: string, userMessage: string): void;
|
||||
|
||||
// Pop cached results at turn start. Returns empty if cache is cold.
|
||||
popContextResult(sessionId: string): ContextResult | null;
|
||||
popDialecticResult(sessionId: string): string | null;
|
||||
}
|
||||
|
||||
type ContextResult = {
|
||||
representation: string;
|
||||
card: string[];
|
||||
aiRepresentation?: string; // AI peer context if enabled
|
||||
summary?: string; // conversation summary if fetched
|
||||
};
|
||||
```
|
||||
|
||||
### Implementation notes
|
||||
|
||||
- **Python:** `threading.Thread(daemon=True)`. Write to `dict[session_id, result]` — GIL makes this safe for simple writes.
|
||||
- **TypeScript:** `Promise` stored in `Map<string, Promise<ContextResult>>`. Await at pop time. If not resolved yet, return null — do not block.
|
||||
- The pop is destructive: clears the cache entry after reading so stale data never accumulates.
|
||||
- Prefetch should also fire on first turn (even though it won't be consumed until turn 2).
|
||||
|
||||
### openclaw-honcho adoption
|
||||
|
||||
Move `session.context()` from `before_prompt_build` to a post-`agent_end` background task. Store result in `state.contextCache`. In `before_prompt_build`, read from cache instead of calling Honcho. If cache is empty (turn 1), inject nothing — the prompt is still valid without Honcho context on the first turn.
|
||||
|
||||
---
|
||||
|
||||
## Spec: dynamic reasoning level
|
||||
|
||||
### Problem
|
||||
|
||||
Honcho's dialectic endpoint supports reasoning levels from `minimal` to `max`. A fixed level per tool wastes budget on simple queries and under-serves complex ones.
|
||||
|
||||
### Pattern
|
||||
|
||||
Select the reasoning level dynamically based on the user's message. Use the configured default as a floor. Bump by message length. Cap auto-selection at `high` — never select `max` automatically.
|
||||
|
||||
### Logic
|
||||
|
||||
```
|
||||
< 120 chars → default (typically "low")
|
||||
120–400 chars → one level above default (cap at "high")
|
||||
> 400 chars → two levels above default (cap at "high")
|
||||
```
|
||||
|
||||
### Config key
|
||||
|
||||
Add `dialecticReasoningLevel` (string, default `"low"`). This sets the floor. The dynamic bump always applies on top.
|
||||
|
||||
### openclaw-honcho adoption
|
||||
|
||||
Apply in `honcho_recall` and `honcho_analyze`: replace fixed `reasoningLevel` with the dynamic selector. `honcho_recall` uses floor `"minimal"`, `honcho_analyze` uses floor `"medium"` — both still bump with message length.
|
||||
|
||||
---
|
||||
|
||||
## Spec: per-peer memory modes
|
||||
|
||||
### Problem
|
||||
|
||||
Users want independent control over whether user context and agent context are written locally, to Honcho, or both.
|
||||
|
||||
### Modes
|
||||
|
||||
| Mode | Effect |
|
||||
|---|---|
|
||||
| `hybrid` | Write to both local files and Honcho (default) |
|
||||
| `honcho` | Honcho only — disable corresponding local file writes |
|
||||
| `local` | Local files only — skip Honcho sync for this peer |
|
||||
|
||||
### Config schema
|
||||
|
||||
```json
|
||||
{
|
||||
"memoryMode": "hybrid",
|
||||
"userMemoryMode": "honcho",
|
||||
"agentMemoryMode": "hybrid"
|
||||
}
|
||||
```
|
||||
|
||||
Resolution order: per-peer field wins → shorthand `memoryMode` → default `"hybrid"`.
|
||||
|
||||
### Effect on Honcho sync
|
||||
|
||||
- `userMemoryMode=local`: skip adding user peer messages to Honcho
|
||||
- `agentMemoryMode=local`: skip adding assistant peer messages to Honcho
|
||||
- Both local: skip `session.addMessages()` entirely
|
||||
- `userMemoryMode=honcho`: disable local USER.md writes
|
||||
- `agentMemoryMode=honcho`: disable local MEMORY.md / SOUL.md writes
|
||||
|
||||
---
|
||||
|
||||
## Spec: AI peer identity formation
|
||||
|
||||
### Problem
|
||||
|
||||
Honcho builds the user's representation organically by observing what the user says. The same mechanism exists for the AI peer — but only if `observe_me=True` is set for the agent peer. Without it, the agent peer accumulates nothing.
|
||||
|
||||
Additionally, existing persona files (SOUL.md, IDENTITY.md) should seed the AI peer's Honcho representation at first activation.
|
||||
|
||||
### Part A: observe_me=True for agent peer
|
||||
|
||||
```typescript
|
||||
await session.addPeers([
|
||||
[ownerPeer.id, { observeMe: true, observeOthers: false }],
|
||||
[agentPeer.id, { observeMe: true, observeOthers: true }], // was false
|
||||
]);
|
||||
```
|
||||
|
||||
One-line change. Foundational. Without it, the AI peer representation stays empty regardless of what the agent says.
|
||||
|
||||
### Part B: seedAiIdentity()
|
||||
|
||||
```typescript
|
||||
async function seedAiIdentity(
|
||||
agentPeer: Peer,
|
||||
content: string,
|
||||
source: string
|
||||
): Promise<boolean> {
|
||||
const wrapped = [
|
||||
`<ai_identity_seed>`,
|
||||
`<source>${source}</source>`,
|
||||
``,
|
||||
content.trim(),
|
||||
`</ai_identity_seed>`,
|
||||
].join("\n");
|
||||
|
||||
await agentPeer.addMessage("assistant", wrapped);
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Part C: migrate agent files at setup
|
||||
|
||||
During `honcho setup`, upload agent-self files (SOUL.md, IDENTITY.md, AGENTS.md) to the agent peer via `seedAiIdentity()` instead of `session.uploadFile()`. This routes content through Honcho's observation pipeline.
|
||||
|
||||
### Part D: AI peer name in identity
|
||||
|
||||
When the agent has a configured name, prepend it to the injected system prompt:
|
||||
|
||||
```typescript
|
||||
const namePrefix = agentName ? `You are ${agentName}.\n\n` : "";
|
||||
return { systemPrompt: namePrefix + "## User Memory Context\n\n" + sections };
|
||||
```
|
||||
|
||||
### CLI surface
|
||||
|
||||
```
|
||||
honcho identity <file> # seed from file
|
||||
honcho identity --show # show current AI peer representation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Spec: session naming strategies
|
||||
|
||||
### Problem
|
||||
|
||||
A single global session means every project shares the same Honcho context. Per-directory sessions provide isolation without requiring users to name sessions manually.
|
||||
|
||||
### Strategies
|
||||
|
||||
| Strategy | Session key | When to use |
|
||||
|---|---|---|
|
||||
| `per-directory` | basename of CWD | Default. Each project gets its own session. |
|
||||
| `global` | fixed string `"global"` | Single cross-project session. |
|
||||
| manual map | user-configured per path | `sessions` config map overrides directory basename. |
|
||||
| title-based | sanitized session title | When agent supports named sessions set mid-conversation. |
|
||||
|
||||
### Config schema
|
||||
|
||||
```json
|
||||
{
|
||||
"sessionStrategy": "per-directory",
|
||||
"sessionPeerPrefix": false,
|
||||
"sessions": {
|
||||
"/home/user/projects/foo": "foo-project"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### CLI surface
|
||||
|
||||
```
|
||||
honcho sessions # list all mappings
|
||||
honcho map <name> # map cwd to session name
|
||||
honcho map # no-arg = list mappings
|
||||
```
|
||||
|
||||
Resolution order: manual map → session title → directory basename → platform key.
|
||||
|
||||
---
|
||||
|
||||
## Spec: CLI surface injection
|
||||
|
||||
### Problem
|
||||
|
||||
When a user asks "how do I change my memory settings?" the agent either hallucinates or says it doesn't know. The agent should know its own management interface.
|
||||
|
||||
### Pattern
|
||||
|
||||
When Honcho is active, append a compact command reference to the system prompt. Keep it under 300 chars.
|
||||
|
||||
```
|
||||
# Honcho memory integration
|
||||
Active. Session: {sessionKey}. Mode: {mode}.
|
||||
Management commands:
|
||||
honcho status — show config + connection
|
||||
honcho mode [hybrid|honcho|local] — show or set memory mode
|
||||
honcho sessions — list session mappings
|
||||
honcho map <name> — map directory to session
|
||||
honcho identity [file] [--show] — seed or show AI identity
|
||||
honcho setup — full interactive wizard
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## openclaw-honcho checklist
|
||||
|
||||
Ordered by impact:
|
||||
|
||||
- [ ] **Async prefetch** — move `session.context()` out of `before_prompt_build` into post-`agent_end` background Promise
|
||||
- [ ] **observe_me=True for agent peer** — one-line change in `session.addPeers()`
|
||||
- [ ] **Dynamic reasoning level** — add helper; apply in `honcho_recall` and `honcho_analyze`; add `dialecticReasoningLevel` to config
|
||||
- [ ] **Per-peer memory modes** — add `userMemoryMode` / `agentMemoryMode` to config; gate Honcho sync and local writes
|
||||
- [ ] **seedAiIdentity()** — add helper; use during setup migration for SOUL.md / IDENTITY.md
|
||||
- [ ] **Session naming strategies** — add `sessionStrategy`, `sessions` map, `sessionPeerPrefix`
|
||||
- [ ] **CLI surface injection** — append command reference to `before_prompt_build` return value
|
||||
- [ ] **honcho identity subcommand** — seed from file or `--show` current representation
|
||||
- [ ] **AI peer name injection** — if `aiPeer` name configured, prepend to injected system prompt
|
||||
- [ ] **honcho mode / sessions / map** — CLI parity with Hermes
|
||||
|
||||
Already done in openclaw-honcho (do not re-implement): `lastSavedIndex` dedup, platform metadata stripping, multi-agent parent observer, `peerPerspective` on `context()`, tiered tool surface, workspace `agentPeerMap`, QMD passthrough, self-hosted Honcho.
|
||||
|
||||
---
|
||||
|
||||
## nanobot-honcho checklist
|
||||
|
||||
Greenfield integration. Start from openclaw-honcho's architecture and apply all Hermes patterns from day one.
|
||||
|
||||
### Phase 1 — core correctness
|
||||
|
||||
- [ ] Dual peer model (owner + agent peer), both with `observe_me=True`
|
||||
- [ ] Message capture at turn end with `lastSavedIndex` dedup
|
||||
- [ ] Platform metadata stripping before Honcho storage
|
||||
- [ ] Async prefetch from day one — do not implement blocking context injection
|
||||
- [ ] Legacy file migration at first activation (USER.md → owner peer, SOUL.md → `seedAiIdentity()`)
|
||||
|
||||
### Phase 2 — configuration
|
||||
|
||||
- [ ] Config schema: `apiKey`, `workspaceId`, `baseUrl`, `memoryMode`, `userMemoryMode`, `agentMemoryMode`, `dialecticReasoningLevel`, `sessionStrategy`, `sessions`
|
||||
- [ ] Per-peer memory mode gating
|
||||
- [ ] Dynamic reasoning level
|
||||
- [ ] Session naming strategies
|
||||
|
||||
### Phase 3 — tools and CLI
|
||||
|
||||
- [ ] Tool surface: `honcho_profile`, `honcho_recall`, `honcho_analyze`, `honcho_search`, `honcho_context`
|
||||
- [ ] CLI: `setup`, `status`, `sessions`, `map`, `mode`, `identity`
|
||||
- [ ] CLI surface injection into system prompt
|
||||
- [ ] AI peer name wired into agent identity
|
||||
@@ -0,0 +1,142 @@
|
||||
# Migrating from OpenClaw to Hermes Agent
|
||||
|
||||
This guide covers how to import your OpenClaw settings, memories, skills, and API keys into Hermes Agent.
|
||||
|
||||
## Three Ways to Migrate
|
||||
|
||||
### 1. Automatic (during first-time setup)
|
||||
|
||||
When you run `hermes setup` for the first time and Hermes detects `~/.openclaw`, it automatically offers to import your OpenClaw data before configuration begins. Just accept the prompt and everything is handled for you.
|
||||
|
||||
### 2. CLI Command (quick, scriptable)
|
||||
|
||||
```bash
|
||||
hermes claw migrate # Preview then migrate (always shows preview first)
|
||||
hermes claw migrate --dry-run # Preview only, no changes
|
||||
hermes claw migrate --preset user-data # Migrate without API keys/secrets
|
||||
hermes claw migrate --yes # Skip confirmation prompt
|
||||
```
|
||||
|
||||
The migration always shows a full preview of what will be imported before making any changes. You review the preview and confirm before anything is written.
|
||||
|
||||
**All options:**
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--source PATH` | Path to OpenClaw directory (default: `~/.openclaw`) |
|
||||
| `--dry-run` | Preview only — no files are modified |
|
||||
| `--preset {user-data,full}` | Migration preset (default: `full`). `user-data` excludes secrets |
|
||||
| `--overwrite` | Overwrite existing files (default: skip conflicts) |
|
||||
| `--migrate-secrets` | Include allowlisted secrets (auto-enabled with `full` preset) |
|
||||
| `--workspace-target PATH` | Copy workspace instructions (AGENTS.md) to this absolute path |
|
||||
| `--skill-conflict {skip,overwrite,rename}` | How to handle skill name conflicts (default: `skip`) |
|
||||
| `--yes`, `-y` | Skip confirmation prompts |
|
||||
|
||||
### 3. Agent-Guided (interactive, with previews)
|
||||
|
||||
Ask the agent to run the migration for you:
|
||||
|
||||
```
|
||||
> Migrate my OpenClaw setup to Hermes
|
||||
```
|
||||
|
||||
The agent will use the `openclaw-migration` skill to:
|
||||
1. Run a preview first to show what would change
|
||||
2. Ask about conflict resolution (SOUL.md, skills, etc.)
|
||||
3. Let you choose between `user-data` and `full` presets
|
||||
4. Execute the migration with your choices
|
||||
5. Print a detailed summary of what was migrated
|
||||
|
||||
## What Gets Migrated
|
||||
|
||||
### `user-data` preset
|
||||
| Item | Source | Destination |
|
||||
|------|--------|-------------|
|
||||
| SOUL.md | `~/.openclaw/workspace/SOUL.md` | `~/.hermes/SOUL.md` |
|
||||
| Memory entries | `~/.openclaw/workspace/MEMORY.md` | `~/.hermes/memories/MEMORY.md` |
|
||||
| User profile | `~/.openclaw/workspace/USER.md` | `~/.hermes/memories/USER.md` |
|
||||
| Skills | `~/.openclaw/workspace/skills/` | `~/.hermes/skills/openclaw-imports/` |
|
||||
| Command allowlist | `~/.openclaw/workspace/exec_approval_patterns.yaml` | Merged into `~/.hermes/config.yaml` |
|
||||
| Messaging settings | `~/.openclaw/config.yaml` (TELEGRAM_ALLOWED_USERS, MESSAGING_CWD) | `~/.hermes/.env` |
|
||||
| TTS assets | `~/.openclaw/workspace/tts/` | `~/.hermes/tts/` |
|
||||
|
||||
Workspace files are also checked at `workspace.default/` and `workspace-main/` as fallback paths (OpenClaw renamed `workspace/` to `workspace-main/` in recent versions).
|
||||
|
||||
### `full` preset (adds to `user-data`)
|
||||
| Item | Source | Destination |
|
||||
|------|--------|-------------|
|
||||
| Telegram bot token | `openclaw.json` channels config | `~/.hermes/.env` |
|
||||
| OpenRouter API key | `.env`, `openclaw.json`, or `openclaw.json["env"]` | `~/.hermes/.env` |
|
||||
| OpenAI API key | `.env`, `openclaw.json`, or `openclaw.json["env"]` | `~/.hermes/.env` |
|
||||
| Anthropic API key | `.env`, `openclaw.json`, or `openclaw.json["env"]` | `~/.hermes/.env` |
|
||||
| ElevenLabs API key | `.env`, `openclaw.json`, or `openclaw.json["env"]` | `~/.hermes/.env` |
|
||||
|
||||
API keys are searched across four sources: inline config values, `~/.openclaw/.env`, the `openclaw.json` `"env"` sub-object, and per-agent auth profiles.
|
||||
|
||||
Only allowlisted secrets are ever imported. Other credentials are skipped and reported.
|
||||
|
||||
## OpenClaw Schema Compatibility
|
||||
|
||||
The migration handles both old and current OpenClaw config layouts:
|
||||
|
||||
- **Channel tokens**: Reads from flat paths (`channels.telegram.botToken`) and the newer `accounts.default` layout (`channels.telegram.accounts.default.botToken`)
|
||||
- **TTS provider**: OpenClaw renamed "edge" to "microsoft" — both are recognized and mapped to Hermes' "edge"
|
||||
- **Provider API types**: Both short (`openai`, `anthropic`) and hyphenated (`openai-completions`, `anthropic-messages`, `google-generative-ai`) values are mapped correctly
|
||||
- **thinkingDefault**: All enum values are handled including newer ones (`minimal`, `xhigh`, `adaptive`)
|
||||
- **Matrix**: Uses `accessToken` field (not `botToken`)
|
||||
- **SecretRef formats**: Plain strings, env templates (`${VAR}`), and `source: "env"` SecretRefs are resolved. `source: "file"` and `source: "exec"` SecretRefs produce a warning — add those keys manually after migration.
|
||||
|
||||
## Conflict Handling
|
||||
|
||||
By default, the migration **will not overwrite** existing Hermes data:
|
||||
|
||||
- **SOUL.md** — skipped if one already exists in `~/.hermes/`
|
||||
- **Memory entries** — skipped if memories already exist (to avoid duplicates)
|
||||
- **Skills** — skipped if a skill with the same name already exists
|
||||
- **API keys** — skipped if the key is already set in `~/.hermes/.env`
|
||||
|
||||
To overwrite conflicts, use `--overwrite`. The migration creates backups before overwriting.
|
||||
|
||||
For skills, you can also use `--skill-conflict rename` to import conflicting skills under a new name (e.g., `skill-name-imported`).
|
||||
|
||||
## Migration Report
|
||||
|
||||
Every migration produces a report showing:
|
||||
- **Migrated items** — what was successfully imported
|
||||
- **Conflicts** — items skipped because they already exist
|
||||
- **Skipped items** — items not found in the source
|
||||
- **Errors** — items that failed to import
|
||||
|
||||
For executed migrations, the full report is saved to `~/.hermes/migration/openclaw/<timestamp>/`.
|
||||
|
||||
## Post-Migration Notes
|
||||
|
||||
- **Skills require a new session** — imported skills take effect after restarting your agent or starting a new chat.
|
||||
- **WhatsApp requires re-pairing** — WhatsApp uses QR-code pairing, not token-based auth. Run `hermes whatsapp` to pair.
|
||||
- **Archive cleanup** — after migration, you'll be offered to rename `~/.openclaw/` to `.openclaw.pre-migration/` to prevent state confusion. You can also run `hermes claw cleanup` later.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "OpenClaw directory not found"
|
||||
The migration looks for `~/.openclaw` by default, then tries `~/.clawdbot` and `~/.moltbot`. If your OpenClaw is installed elsewhere, use `--source`:
|
||||
```bash
|
||||
hermes claw migrate --source /path/to/.openclaw
|
||||
```
|
||||
|
||||
### "Migration script not found"
|
||||
The migration script ships with Hermes Agent. If you installed via pip (not git clone), the `optional-skills/` directory may not be present. Install the skill from the Skills Hub:
|
||||
```bash
|
||||
hermes skills install openclaw-migration
|
||||
```
|
||||
|
||||
### Memory overflow
|
||||
If your OpenClaw MEMORY.md or USER.md exceeds Hermes' character limits, excess entries are exported to an overflow file in the migration report directory. You can manually review and add the most important ones.
|
||||
|
||||
### API keys not found
|
||||
Keys might be stored in different places depending on your OpenClaw setup:
|
||||
- `~/.openclaw/.env` file
|
||||
- Inline in `openclaw.json` under `models.providers.*.apiKey`
|
||||
- In `openclaw.json` under the `"env"` or `"env.vars"` sub-objects
|
||||
- In `~/.openclaw/agents/main/agent/auth-profiles.json`
|
||||
|
||||
The migration checks all four. If keys use `source: "file"` or `source: "exec"` SecretRefs, they can't be resolved automatically — add them via `hermes config set`.
|
||||
@@ -0,0 +1,608 @@
|
||||
# Pricing Accuracy Architecture
|
||||
|
||||
Date: 2026-03-16
|
||||
|
||||
## Goal
|
||||
|
||||
Hermes should only show dollar costs when they are backed by an official source for the user's actual billing path.
|
||||
|
||||
This design replaces the current static, heuristic pricing flow in:
|
||||
|
||||
- `run_agent.py`
|
||||
- `agent/usage_pricing.py`
|
||||
- `agent/insights.py`
|
||||
- `cli.py`
|
||||
|
||||
with a provider-aware pricing system that:
|
||||
|
||||
- handles cache billing correctly
|
||||
- distinguishes `actual` vs `estimated` vs `included` vs `unknown`
|
||||
- reconciles post-hoc costs when providers expose authoritative billing data
|
||||
- supports direct providers, OpenRouter, subscriptions, enterprise pricing, and custom endpoints
|
||||
|
||||
## Problems In The Current Design
|
||||
|
||||
Current Hermes behavior has four structural issues:
|
||||
|
||||
1. It stores only `prompt_tokens` and `completion_tokens`, which is insufficient for providers that bill cache reads and cache writes separately.
|
||||
2. It uses a static model price table and fuzzy heuristics, which can drift from current official pricing.
|
||||
3. It assumes public API list pricing matches the user's real billing path.
|
||||
4. It has no distinction between live estimates and reconciled billed cost.
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. Normalize usage before pricing.
|
||||
2. Never fold cached tokens into plain input cost.
|
||||
3. Track certainty explicitly.
|
||||
4. Treat the billing path as part of the model identity.
|
||||
5. Prefer official machine-readable sources over scraped docs.
|
||||
6. Use post-hoc provider cost APIs when available.
|
||||
7. Show `n/a` rather than inventing precision.
|
||||
|
||||
## High-Level Architecture
|
||||
|
||||
The new system has four layers:
|
||||
|
||||
1. `usage_normalization`
|
||||
Converts raw provider usage into a canonical usage record.
|
||||
2. `pricing_source_resolution`
|
||||
Determines the billing path, source of truth, and applicable pricing source.
|
||||
3. `cost_estimation_and_reconciliation`
|
||||
Produces an immediate estimate when possible, then replaces or annotates it with actual billed cost later.
|
||||
4. `presentation`
|
||||
`/usage`, `/insights`, and the status bar display cost with certainty metadata.
|
||||
|
||||
## Canonical Usage Record
|
||||
|
||||
Add a canonical usage model that every provider path maps into before any pricing math happens.
|
||||
|
||||
Suggested structure:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class CanonicalUsage:
|
||||
provider: str
|
||||
billing_provider: str
|
||||
model: str
|
||||
billing_route: str
|
||||
|
||||
input_tokens: int = 0
|
||||
output_tokens: int = 0
|
||||
cache_read_tokens: int = 0
|
||||
cache_write_tokens: int = 0
|
||||
reasoning_tokens: int = 0
|
||||
request_count: int = 1
|
||||
|
||||
raw_usage: dict[str, Any] | None = None
|
||||
raw_usage_fields: dict[str, str] | None = None
|
||||
computed_fields: set[str] | None = None
|
||||
|
||||
provider_request_id: str | None = None
|
||||
provider_generation_id: str | None = None
|
||||
provider_response_id: str | None = None
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `input_tokens` means non-cached input only.
|
||||
- `cache_read_tokens` and `cache_write_tokens` are never merged into `input_tokens`.
|
||||
- `output_tokens` excludes cache metrics.
|
||||
- `reasoning_tokens` is telemetry unless a provider officially bills it separately.
|
||||
|
||||
This is the same normalization pattern used by `opencode`, extended with provenance and reconciliation ids.
|
||||
|
||||
## Provider Normalization Rules
|
||||
|
||||
### OpenAI Direct
|
||||
|
||||
Source usage fields:
|
||||
|
||||
- `prompt_tokens`
|
||||
- `completion_tokens`
|
||||
- `prompt_tokens_details.cached_tokens`
|
||||
|
||||
Normalization:
|
||||
|
||||
- `cache_read_tokens = cached_tokens`
|
||||
- `input_tokens = prompt_tokens - cached_tokens`
|
||||
- `cache_write_tokens = 0` unless OpenAI exposes it in the relevant route
|
||||
- `output_tokens = completion_tokens`
|
||||
|
||||
### Anthropic Direct
|
||||
|
||||
Source usage fields:
|
||||
|
||||
- `input_tokens`
|
||||
- `output_tokens`
|
||||
- `cache_read_input_tokens`
|
||||
- `cache_creation_input_tokens`
|
||||
|
||||
Normalization:
|
||||
|
||||
- `input_tokens = input_tokens`
|
||||
- `output_tokens = output_tokens`
|
||||
- `cache_read_tokens = cache_read_input_tokens`
|
||||
- `cache_write_tokens = cache_creation_input_tokens`
|
||||
|
||||
### OpenRouter
|
||||
|
||||
Estimate-time usage normalization should use the response usage payload with the same rules as the underlying provider when possible.
|
||||
|
||||
Reconciliation-time records should also store:
|
||||
|
||||
- OpenRouter generation id
|
||||
- native token fields when available
|
||||
- `total_cost`
|
||||
- `cache_discount`
|
||||
- `upstream_inference_cost`
|
||||
- `is_byok`
|
||||
|
||||
### Gemini / Vertex
|
||||
|
||||
Use official Gemini or Vertex usage fields where available.
|
||||
|
||||
If cached content tokens are exposed:
|
||||
|
||||
- map them to `cache_read_tokens`
|
||||
|
||||
If a route exposes no cache creation metric:
|
||||
|
||||
- store `cache_write_tokens = 0`
|
||||
- preserve the raw usage payload for later extension
|
||||
|
||||
### DeepSeek And Other Direct Providers
|
||||
|
||||
Normalize only the fields that are officially exposed.
|
||||
|
||||
If a provider does not expose cache buckets:
|
||||
|
||||
- do not infer them unless the provider explicitly documents how to derive them
|
||||
|
||||
### Subscription / Included-Cost Routes
|
||||
|
||||
These still use the canonical usage model.
|
||||
|
||||
Tokens are tracked normally. Cost depends on billing mode, not on whether usage exists.
|
||||
|
||||
## Billing Route Model
|
||||
|
||||
Hermes must stop keying pricing solely by `model`.
|
||||
|
||||
Introduce a billing route descriptor:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class BillingRoute:
|
||||
provider: str
|
||||
base_url: str | None
|
||||
model: str
|
||||
billing_mode: str
|
||||
organization_hint: str | None = None
|
||||
```
|
||||
|
||||
`billing_mode` values:
|
||||
|
||||
- `official_cost_api`
|
||||
- `official_generation_api`
|
||||
- `official_models_api`
|
||||
- `official_docs_snapshot`
|
||||
- `subscription_included`
|
||||
- `user_override`
|
||||
- `custom_contract`
|
||||
- `unknown`
|
||||
|
||||
Examples:
|
||||
|
||||
- OpenAI direct API with Costs API access: `official_cost_api`
|
||||
- Anthropic direct API with Usage & Cost API access: `official_cost_api`
|
||||
- OpenRouter request before reconciliation: `official_models_api`
|
||||
- OpenRouter request after generation lookup: `official_generation_api`
|
||||
- GitHub Copilot style subscription route: `subscription_included`
|
||||
- local OpenAI-compatible server: `unknown`
|
||||
- enterprise contract with configured rates: `custom_contract`
|
||||
|
||||
## Cost Status Model
|
||||
|
||||
Every displayed cost should have:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class CostResult:
|
||||
amount_usd: Decimal | None
|
||||
status: Literal["actual", "estimated", "included", "unknown"]
|
||||
source: Literal[
|
||||
"provider_cost_api",
|
||||
"provider_generation_api",
|
||||
"provider_models_api",
|
||||
"official_docs_snapshot",
|
||||
"user_override",
|
||||
"custom_contract",
|
||||
"none",
|
||||
]
|
||||
label: str
|
||||
fetched_at: datetime | None
|
||||
pricing_version: str | None
|
||||
notes: list[str]
|
||||
```
|
||||
|
||||
Presentation rules:
|
||||
|
||||
- `actual`: show dollar amount as final
|
||||
- `estimated`: show dollar amount with estimate labeling
|
||||
- `included`: show `included` or `$0.00 (included)` depending on UX choice
|
||||
- `unknown`: show `n/a`
|
||||
|
||||
## Official Source Hierarchy
|
||||
|
||||
Resolve cost using this order:
|
||||
|
||||
1. Request-level or account-level official billed cost
|
||||
2. Official machine-readable model pricing
|
||||
3. Official docs snapshot
|
||||
4. User override or custom contract
|
||||
5. Unknown
|
||||
|
||||
The system must never skip to a lower level if a higher-confidence source exists for the current billing route.
|
||||
|
||||
## Provider-Specific Truth Rules
|
||||
|
||||
### OpenAI Direct
|
||||
|
||||
Preferred truth:
|
||||
|
||||
1. Costs API for reconciled spend
|
||||
2. Official pricing page for live estimate
|
||||
|
||||
### Anthropic Direct
|
||||
|
||||
Preferred truth:
|
||||
|
||||
1. Usage & Cost API for reconciled spend
|
||||
2. Official pricing docs for live estimate
|
||||
|
||||
### OpenRouter
|
||||
|
||||
Preferred truth:
|
||||
|
||||
1. `GET /api/v1/generation` for reconciled `total_cost`
|
||||
2. `GET /api/v1/models` pricing for live estimate
|
||||
|
||||
Do not use underlying provider public pricing as the source of truth for OpenRouter billing.
|
||||
|
||||
### Gemini / Vertex
|
||||
|
||||
Preferred truth:
|
||||
|
||||
1. official billing export or billing API for reconciled spend when available for the route
|
||||
2. official pricing docs for estimate
|
||||
|
||||
### DeepSeek
|
||||
|
||||
Preferred truth:
|
||||
|
||||
1. official machine-readable cost source if available in the future
|
||||
2. official pricing docs snapshot today
|
||||
|
||||
### Subscription-Included Routes
|
||||
|
||||
Preferred truth:
|
||||
|
||||
1. explicit route config marking the model as included in subscription
|
||||
|
||||
These should display `included`, not an API list-price estimate.
|
||||
|
||||
### Custom Endpoint / Local Model
|
||||
|
||||
Preferred truth:
|
||||
|
||||
1. user override
|
||||
2. custom contract config
|
||||
3. unknown
|
||||
|
||||
These should default to `unknown`.
|
||||
|
||||
## Pricing Catalog
|
||||
|
||||
Replace the current `MODEL_PRICING` dict with a richer pricing catalog.
|
||||
|
||||
Suggested record:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class PricingEntry:
|
||||
provider: str
|
||||
route_pattern: str
|
||||
model_pattern: str
|
||||
|
||||
input_cost_per_million: Decimal | None = None
|
||||
output_cost_per_million: Decimal | None = None
|
||||
cache_read_cost_per_million: Decimal | None = None
|
||||
cache_write_cost_per_million: Decimal | None = None
|
||||
request_cost: Decimal | None = None
|
||||
image_cost: Decimal | None = None
|
||||
|
||||
source: str = "official_docs_snapshot"
|
||||
source_url: str | None = None
|
||||
fetched_at: datetime | None = None
|
||||
pricing_version: str | None = None
|
||||
```
|
||||
|
||||
The catalog should be route-aware:
|
||||
|
||||
- `openai:gpt-5`
|
||||
- `anthropic:claude-opus-4-6`
|
||||
- `openrouter:anthropic/claude-opus-4.6`
|
||||
- `copilot:gpt-4o`
|
||||
|
||||
This avoids conflating direct-provider billing with aggregator billing.
|
||||
|
||||
## Pricing Sync Architecture
|
||||
|
||||
Introduce a pricing sync subsystem instead of manually maintaining a single hardcoded table.
|
||||
|
||||
Suggested modules:
|
||||
|
||||
- `agent/pricing/catalog.py`
|
||||
- `agent/pricing/sources.py`
|
||||
- `agent/pricing/sync.py`
|
||||
- `agent/pricing/reconcile.py`
|
||||
- `agent/pricing/types.py`
|
||||
|
||||
### Sync Sources
|
||||
|
||||
- OpenRouter models API
|
||||
- official provider docs snapshots where no API exists
|
||||
- user overrides from config
|
||||
|
||||
### Sync Output
|
||||
|
||||
Cache pricing entries locally with:
|
||||
|
||||
- source URL
|
||||
- fetch timestamp
|
||||
- version/hash
|
||||
- confidence/source type
|
||||
|
||||
### Sync Frequency
|
||||
|
||||
- startup warm cache
|
||||
- background refresh every 6 to 24 hours depending on source
|
||||
- manual `hermes pricing sync`
|
||||
|
||||
## Reconciliation Architecture
|
||||
|
||||
Live requests may produce only an estimate initially. Hermes should reconcile them later when a provider exposes actual billed cost.
|
||||
|
||||
Suggested flow:
|
||||
|
||||
1. Agent call completes.
|
||||
2. Hermes stores canonical usage plus reconciliation ids.
|
||||
3. Hermes computes an immediate estimate if a pricing source exists.
|
||||
4. A reconciliation worker fetches actual cost when supported.
|
||||
5. Session and message records are updated with `actual` cost.
|
||||
|
||||
This can run:
|
||||
|
||||
- inline for cheap lookups
|
||||
- asynchronously for delayed provider accounting
|
||||
|
||||
## Persistence Changes
|
||||
|
||||
Session storage should stop storing only aggregate prompt/completion totals.
|
||||
|
||||
Add fields for both usage and cost certainty:
|
||||
|
||||
- `input_tokens`
|
||||
- `output_tokens`
|
||||
- `cache_read_tokens`
|
||||
- `cache_write_tokens`
|
||||
- `reasoning_tokens`
|
||||
- `estimated_cost_usd`
|
||||
- `actual_cost_usd`
|
||||
- `cost_status`
|
||||
- `cost_source`
|
||||
- `pricing_version`
|
||||
- `billing_provider`
|
||||
- `billing_mode`
|
||||
|
||||
If schema expansion is too large for one PR, add a new pricing events table:
|
||||
|
||||
```text
|
||||
session_cost_events
|
||||
id
|
||||
session_id
|
||||
request_id
|
||||
provider
|
||||
model
|
||||
billing_mode
|
||||
input_tokens
|
||||
output_tokens
|
||||
cache_read_tokens
|
||||
cache_write_tokens
|
||||
estimated_cost_usd
|
||||
actual_cost_usd
|
||||
cost_status
|
||||
cost_source
|
||||
pricing_version
|
||||
created_at
|
||||
updated_at
|
||||
```
|
||||
|
||||
## Hermes Touchpoints
|
||||
|
||||
### `run_agent.py`
|
||||
|
||||
Current responsibility:
|
||||
|
||||
- parse raw provider usage
|
||||
- update session token counters
|
||||
|
||||
New responsibility:
|
||||
|
||||
- build `CanonicalUsage`
|
||||
- update canonical counters
|
||||
- store reconciliation ids
|
||||
- emit usage event to pricing subsystem
|
||||
|
||||
### `agent/usage_pricing.py`
|
||||
|
||||
Current responsibility:
|
||||
|
||||
- static lookup table
|
||||
- direct cost arithmetic
|
||||
|
||||
New responsibility:
|
||||
|
||||
- move or replace with pricing catalog facade
|
||||
- no fuzzy model-family heuristics
|
||||
- no direct pricing without billing-route context
|
||||
|
||||
### `cli.py`
|
||||
|
||||
Current responsibility:
|
||||
|
||||
- compute session cost directly from prompt/completion totals
|
||||
|
||||
New responsibility:
|
||||
|
||||
- display `CostResult`
|
||||
- show status badges:
|
||||
- `actual`
|
||||
- `estimated`
|
||||
- `included`
|
||||
- `n/a`
|
||||
|
||||
### `agent/insights.py`
|
||||
|
||||
Current responsibility:
|
||||
|
||||
- recompute historical estimates from static pricing
|
||||
|
||||
New responsibility:
|
||||
|
||||
- aggregate stored pricing events
|
||||
- prefer actual cost over estimate
|
||||
- surface estimates only when reconciliation is unavailable
|
||||
|
||||
## UX Rules
|
||||
|
||||
### Status Bar
|
||||
|
||||
Show one of:
|
||||
|
||||
- `$1.42`
|
||||
- `~$1.42`
|
||||
- `included`
|
||||
- `cost n/a`
|
||||
|
||||
Where:
|
||||
|
||||
- `$1.42` means `actual`
|
||||
- `~$1.42` means `estimated`
|
||||
- `included` means subscription-backed or explicitly zero-cost route
|
||||
- `cost n/a` means unknown
|
||||
|
||||
### `/usage`
|
||||
|
||||
Show:
|
||||
|
||||
- token buckets
|
||||
- estimated cost
|
||||
- actual cost if available
|
||||
- cost status
|
||||
- pricing source
|
||||
|
||||
### `/insights`
|
||||
|
||||
Aggregate:
|
||||
|
||||
- actual cost totals
|
||||
- estimated-only totals
|
||||
- unknown-cost sessions count
|
||||
- included-cost sessions count
|
||||
|
||||
## Config And Overrides
|
||||
|
||||
Add user-configurable pricing overrides in config:
|
||||
|
||||
```yaml
|
||||
pricing:
|
||||
mode: hybrid
|
||||
sync_on_startup: true
|
||||
sync_interval_hours: 12
|
||||
overrides:
|
||||
- provider: openrouter
|
||||
model: anthropic/claude-opus-4.6
|
||||
billing_mode: custom_contract
|
||||
input_cost_per_million: 4.25
|
||||
output_cost_per_million: 22.0
|
||||
cache_read_cost_per_million: 0.5
|
||||
cache_write_cost_per_million: 6.0
|
||||
included_routes:
|
||||
- provider: copilot
|
||||
model: "*"
|
||||
- provider: codex-subscription
|
||||
model: "*"
|
||||
```
|
||||
|
||||
Overrides must win over catalog defaults for the matching billing route.
|
||||
|
||||
## Rollout Plan
|
||||
|
||||
### Phase 1
|
||||
|
||||
- add canonical usage model
|
||||
- split cache token buckets in `run_agent.py`
|
||||
- stop pricing cache-inflated prompt totals
|
||||
- preserve current UI with improved backend math
|
||||
|
||||
### Phase 2
|
||||
|
||||
- add route-aware pricing catalog
|
||||
- integrate OpenRouter models API sync
|
||||
- add `estimated` vs `included` vs `unknown`
|
||||
|
||||
### Phase 3
|
||||
|
||||
- add reconciliation for OpenRouter generation cost
|
||||
- add actual cost persistence
|
||||
- update `/insights` to prefer actual cost
|
||||
|
||||
### Phase 4
|
||||
|
||||
- add direct OpenAI and Anthropic reconciliation paths
|
||||
- add user overrides and contract pricing
|
||||
- add pricing sync CLI command
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
Add tests for:
|
||||
|
||||
- OpenAI cached token subtraction
|
||||
- Anthropic cache read/write separation
|
||||
- OpenRouter estimated vs actual reconciliation
|
||||
- subscription-backed models showing `included`
|
||||
- custom endpoints showing `n/a`
|
||||
- override precedence
|
||||
- stale catalog fallback behavior
|
||||
|
||||
Current tests that assume heuristic pricing should be replaced with route-aware expectations.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- exact enterprise billing reconstruction without an official source or user override
|
||||
- backfilling perfect historical cost for old sessions that lack cache bucket data
|
||||
- scraping arbitrary provider web pages at request time
|
||||
|
||||
## Recommendation
|
||||
|
||||
Do not expand the existing `MODEL_PRICING` dict.
|
||||
|
||||
That path cannot satisfy the product requirement. Hermes should instead migrate to:
|
||||
|
||||
- canonical usage normalization
|
||||
- route-aware pricing sources
|
||||
- estimate-then-reconcile cost lifecycle
|
||||
- explicit certainty states in the UI
|
||||
|
||||
This is the minimum architecture that makes the statement "Hermes pricing is backed by official sources where possible, and otherwise clearly labeled" defensible.
|
||||
@@ -0,0 +1,97 @@
|
||||
# ============================================================================
|
||||
# Hermes Agent — Example Skin Template
|
||||
# ============================================================================
|
||||
#
|
||||
# Copy this file to ~/.hermes/skins/<name>.yaml to create a custom skin.
|
||||
# All fields are optional — missing values inherit from the default skin.
|
||||
# Activate with: /skin <name> or display.skin: <name> in config.yaml
|
||||
#
|
||||
# See hermes_cli/skin_engine.py for the full schema reference.
|
||||
# ============================================================================
|
||||
|
||||
# Required: unique skin name (used in /skin command and config)
|
||||
name: example
|
||||
description: An example custom skin — copy and modify this template
|
||||
|
||||
# ── Colors ──────────────────────────────────────────────────────────────────
|
||||
# Hex color values for Rich markup. These control the CLI's visual palette.
|
||||
colors:
|
||||
# Banner panel (the startup welcome box)
|
||||
banner_border: "#CD7F32" # Panel border
|
||||
banner_title: "#FFD700" # Panel title text
|
||||
banner_accent: "#FFBF00" # Section headers (Available Tools, Skills, etc.)
|
||||
banner_dim: "#B8860B" # Dim/muted text (separators, model info)
|
||||
banner_text: "#FFF8DC" # Body text (tool names, skill names)
|
||||
|
||||
# UI elements
|
||||
ui_accent: "#FFBF00" # General accent color
|
||||
ui_label: "#4dd0e1" # Labels
|
||||
ui_ok: "#4caf50" # Success indicators
|
||||
ui_error: "#ef5350" # Error indicators
|
||||
ui_warn: "#ffa726" # Warning indicators
|
||||
|
||||
# Input area
|
||||
prompt: "#FFF8DC" # Prompt text color
|
||||
input_rule: "#CD7F32" # Horizontal rule around input
|
||||
|
||||
# Response box
|
||||
response_border: "#FFD700" # Response box border (ANSI color)
|
||||
|
||||
# Session display
|
||||
session_label: "#DAA520" # Session label
|
||||
session_border: "#8B8682" # Session ID dim color
|
||||
|
||||
# TUI surfaces
|
||||
status_bar_bg: "#1a1a2e" # Status / usage bar background
|
||||
voice_status_bg: "#1a1a2e" # Voice-mode badge background
|
||||
completion_menu_bg: "#1a1a2e" # Completion list background
|
||||
completion_menu_current_bg: "#333355" # Active completion row background
|
||||
completion_menu_meta_bg: "#1a1a2e" # Completion meta column background
|
||||
completion_menu_meta_current_bg: "#333355" # Active completion meta background
|
||||
|
||||
# ── Spinner ─────────────────────────────────────────────────────────────────
|
||||
# Customize the animated spinner shown during API calls and tool execution.
|
||||
spinner:
|
||||
# Faces shown while waiting for the API response
|
||||
waiting_faces:
|
||||
- "(。◕‿◕。)"
|
||||
- "(◕‿◕✿)"
|
||||
- "٩(◕‿◕。)۶"
|
||||
|
||||
# Faces shown during extended thinking/reasoning
|
||||
thinking_faces:
|
||||
- "(。•́︿•̀。)"
|
||||
- "(◔_◔)"
|
||||
- "(¬‿¬)"
|
||||
|
||||
# Verbs used in spinner messages (e.g., "pondering your request...")
|
||||
thinking_verbs:
|
||||
- "pondering"
|
||||
- "contemplating"
|
||||
- "musing"
|
||||
- "ruminating"
|
||||
|
||||
# Optional: left/right decorations around the spinner
|
||||
# Each entry is a [left, right] pair. Omit entirely for no wings.
|
||||
# wings:
|
||||
# - ["⟪⚔", "⚔⟫"]
|
||||
# - ["⟪▲", "▲⟫"]
|
||||
|
||||
# ── Branding ────────────────────────────────────────────────────────────────
|
||||
# Text strings used throughout the CLI interface.
|
||||
branding:
|
||||
agent_name: "Hermes Agent" # Banner title, about display
|
||||
welcome: "Welcome! Type your message or /help for commands."
|
||||
goodbye: "Goodbye! ⚕" # Exit message
|
||||
response_label: " ⚕ Hermes " # Response box header label
|
||||
prompt_symbol: "❯ " # Input prompt symbol
|
||||
help_header: "(^_^)? Available Commands" # /help header text
|
||||
|
||||
# ── Tool Output ─────────────────────────────────────────────────────────────
|
||||
# Character used as the prefix for tool output lines.
|
||||
# Default is "┊" (thin dotted vertical line). Some alternatives:
|
||||
# "╎" (light triple dash vertical)
|
||||
# "▏" (left one-eighth block)
|
||||
# "│" (box drawing light vertical)
|
||||
# "┃" (box drawing heavy vertical)
|
||||
tool_prefix: "┊"
|
||||
@@ -0,0 +1,329 @@
|
||||
# Container-Aware CLI Review Fixes Spec
|
||||
|
||||
**PR:** NousResearch/hermes-agent#7543
|
||||
**Review:** cursor[bot] bugbot review (4094049442) + two prior rounds
|
||||
**Date:** 2026-04-12
|
||||
**Branch:** `feat/container-aware-cli-clean`
|
||||
|
||||
## Review Issues Summary
|
||||
|
||||
Six issues were raised across three bugbot review rounds. Three were fixed in intermediate commits (38277a6a, 726cf90f). This spec addresses remaining design concerns surfaced by those reviews and simplifies the implementation based on interview decisions.
|
||||
|
||||
| # | Issue | Severity | Status |
|
||||
|---|-------|----------|--------|
|
||||
| 1 | `os.execvp` retry loop unreachable | Medium | Fixed in 79e8cd12 (switched to subprocess.run) |
|
||||
| 2 | Redundant `shutil.which("sudo")` | Medium | Fixed in 38277a6a (reuses `sudo` var) |
|
||||
| 3 | Missing `chown -h` on symlink update | Low | Fixed in 38277a6a |
|
||||
| 4 | Container routing after `parse_args()` | High | Fixed in 726cf90f |
|
||||
| 5 | Hardcoded `/home/${user}` | Medium | Fixed in 726cf90f |
|
||||
| 6 | Group membership not gated on `container.enable` | Low | Fixed in 726cf90f |
|
||||
|
||||
The mechanical fixes are in place but the overall design needs revision. The retry loop, error swallowing, and process model have deeper issues than what the bugbot flagged.
|
||||
|
||||
---
|
||||
|
||||
## Spec: Revised `_exec_in_container`
|
||||
|
||||
### Design Principles
|
||||
|
||||
1. **Let it crash.** No silent fallbacks. If `.container-mode` exists but something goes wrong, the error propagates naturally (Python traceback). The only case where container routing is skipped is when `.container-mode` doesn't exist or `HERMES_DEV=1`.
|
||||
2. **No retries.** Probe once for sudo, exec once. If it fails, docker/podman's stderr reaches the user verbatim.
|
||||
3. **Completely transparent.** No error wrapping, no prefixes, no spinners. Docker's output goes straight through.
|
||||
4. **`os.execvp` on the happy path.** Replace the Python process entirely so there's no idle parent during interactive sessions. Note: `execvp` never returns on success (process is replaced) and raises `OSError` on failure (it does not return a value). The container process's exit code becomes the process exit code by definition — no explicit propagation needed.
|
||||
5. **One human-readable exception to "let it crash".** `subprocess.TimeoutExpired` from the sudo probe gets a specific catch with a readable message, since a raw traceback for "your Docker daemon is slow" is confusing. All other exceptions propagate naturally.
|
||||
|
||||
### Execution Flow
|
||||
|
||||
```
|
||||
1. get_container_exec_info()
|
||||
- HERMES_DEV=1 → return None (skip routing)
|
||||
- Inside container → return None (skip routing)
|
||||
- .container-mode doesn't exist → return None (skip routing)
|
||||
- .container-mode exists → parse and return dict
|
||||
- .container-mode exists but malformed/unreadable → LET IT CRASH (no try/except)
|
||||
|
||||
2. _exec_in_container(container_info, sys.argv[1:])
|
||||
a. shutil.which(backend) → if None, print "{backend} not found on PATH" and sys.exit(1)
|
||||
b. Sudo probe: subprocess.run([runtime, "inspect", "--format", "ok", container_name], timeout=15)
|
||||
- If succeeds → needs_sudo = False
|
||||
- If fails → try subprocess.run([sudo, "-n", runtime, "inspect", ...], timeout=15)
|
||||
- If succeeds → needs_sudo = True
|
||||
- If fails → print error with sudoers hint (including why -n is required) and sys.exit(1)
|
||||
- If TimeoutExpired → catch specifically, print human-readable message about slow daemon
|
||||
c. Build exec_cmd: [sudo? + runtime, "exec", tty_flags, "-u", exec_user, env_flags, container, hermes_bin, *cli_args]
|
||||
d. os.execvp(exec_cmd[0], exec_cmd)
|
||||
- On success: process is replaced — Python is gone, container exit code IS the process exit code
|
||||
- On OSError: let it crash (natural traceback)
|
||||
```
|
||||
|
||||
### Changes to `hermes_cli/main.py`
|
||||
|
||||
#### `_exec_in_container` — rewrite
|
||||
|
||||
Remove:
|
||||
- The entire retry loop (`max_retries`, `for attempt in range(...)`)
|
||||
- Spinner logic (`"Waiting for container..."`, dots)
|
||||
- Exit code classification (125/126/127 handling)
|
||||
- `subprocess.run` for the exec call (keep it only for the sudo probe)
|
||||
- Special TTY vs non-TTY retry counts
|
||||
- The `time` import (no longer needed)
|
||||
|
||||
Change:
|
||||
- Use `os.execvp(exec_cmd[0], exec_cmd)` as the final call
|
||||
- Keep the `subprocess` import only for the sudo probe
|
||||
- Keep TTY detection for the `-it` vs `-i` flag
|
||||
- Keep env var forwarding (TERM, COLORTERM, LANG, LC_ALL)
|
||||
- Keep the sudo probe as-is (it's the one "smart" part)
|
||||
- Bump probe `timeout` from 5s to 15s — cold podman on a loaded machine needs headroom
|
||||
- Catch `subprocess.TimeoutExpired` specifically on both probe calls — print a readable message about the daemon being unresponsive instead of a raw traceback
|
||||
- Expand the sudoers hint error message to explain *why* `-n` (non-interactive) is required: a password prompt would hang the CLI or break piped commands
|
||||
|
||||
The function becomes roughly:
|
||||
|
||||
```python
|
||||
def _exec_in_container(container_info: dict, cli_args: list):
|
||||
"""Replace the current process with a command inside the managed container.
|
||||
|
||||
Probes whether sudo is needed (rootful containers), then os.execvp
|
||||
into the container. If exec fails, the OS error propagates naturally.
|
||||
"""
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
backend = container_info["backend"]
|
||||
container_name = container_info["container_name"]
|
||||
exec_user = container_info["exec_user"]
|
||||
hermes_bin = container_info["hermes_bin"]
|
||||
|
||||
runtime = shutil.which(backend)
|
||||
if not runtime:
|
||||
print(f"Error: {backend} not found on PATH. Cannot route to container.",
|
||||
file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Probe whether we need sudo to see the rootful container.
|
||||
# Timeout is 15s — cold podman on a loaded machine can take a while.
|
||||
# TimeoutExpired is caught specifically for a human-readable message;
|
||||
# all other exceptions propagate naturally.
|
||||
needs_sudo = False
|
||||
sudo = None
|
||||
try:
|
||||
probe = subprocess.run(
|
||||
[runtime, "inspect", "--format", "ok", container_name],
|
||||
capture_output=True, text=True, timeout=15,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
print(
|
||||
f"Error: timed out waiting for {backend} to respond.\n"
|
||||
f"The {backend} daemon may be unresponsive or starting up.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
if probe.returncode != 0:
|
||||
sudo = shutil.which("sudo")
|
||||
if sudo:
|
||||
try:
|
||||
probe2 = subprocess.run(
|
||||
[sudo, "-n", runtime, "inspect", "--format", "ok", container_name],
|
||||
capture_output=True, text=True, timeout=15,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
print(
|
||||
f"Error: timed out waiting for sudo {backend} to respond.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
if probe2.returncode == 0:
|
||||
needs_sudo = True
|
||||
else:
|
||||
print(
|
||||
f"Error: container '{container_name}' not found via {backend}.\n"
|
||||
f"\n"
|
||||
f"The NixOS service runs the container as root. Your user cannot\n"
|
||||
f"see it because {backend} uses per-user namespaces.\n"
|
||||
f"\n"
|
||||
f"Fix: grant passwordless sudo for {backend}. The -n (non-interactive)\n"
|
||||
f"flag is required because the CLI calls sudo non-interactively —\n"
|
||||
f"a password prompt would hang or break piped commands:\n"
|
||||
f"\n"
|
||||
f' security.sudo.extraRules = [{{\n'
|
||||
f' users = [ "{os.getenv("USER", "your-user")}" ];\n'
|
||||
f' commands = [{{ command = "{runtime}"; options = [ "NOPASSWD" ]; }}];\n'
|
||||
f' }}];\n'
|
||||
f"\n"
|
||||
f"Or run: sudo hermes {' '.join(cli_args)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
else:
|
||||
print(
|
||||
f"Error: container '{container_name}' not found via {backend}.\n"
|
||||
f"The container may be running under root. Try: sudo hermes {' '.join(cli_args)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
is_tty = sys.stdin.isatty()
|
||||
tty_flags = ["-it"] if is_tty else ["-i"]
|
||||
|
||||
env_flags = []
|
||||
for var in ("TERM", "COLORTERM", "LANG", "LC_ALL"):
|
||||
val = os.environ.get(var)
|
||||
if val:
|
||||
env_flags.extend(["-e", f"{var}={val}"])
|
||||
|
||||
cmd_prefix = [sudo, "-n", runtime] if needs_sudo else [runtime]
|
||||
exec_cmd = (
|
||||
cmd_prefix + ["exec"]
|
||||
+ tty_flags
|
||||
+ ["-u", exec_user]
|
||||
+ env_flags
|
||||
+ [container_name, hermes_bin]
|
||||
+ cli_args
|
||||
)
|
||||
|
||||
# execvp replaces this process entirely — it never returns on success.
|
||||
# On failure it raises OSError, which propagates naturally.
|
||||
os.execvp(exec_cmd[0], exec_cmd)
|
||||
```
|
||||
|
||||
#### Container routing call site in `main()` — remove try/except
|
||||
|
||||
Current:
|
||||
```python
|
||||
try:
|
||||
from hermes_cli.config import get_container_exec_info
|
||||
container_info = get_container_exec_info()
|
||||
if container_info:
|
||||
_exec_in_container(container_info, sys.argv[1:])
|
||||
sys.exit(1) # exec failed if we reach here
|
||||
except SystemExit:
|
||||
raise
|
||||
except Exception:
|
||||
pass # Container routing unavailable, proceed locally
|
||||
```
|
||||
|
||||
Revised:
|
||||
```python
|
||||
from hermes_cli.config import get_container_exec_info
|
||||
container_info = get_container_exec_info()
|
||||
if container_info:
|
||||
_exec_in_container(container_info, sys.argv[1:])
|
||||
# Unreachable: os.execvp never returns on success (process is replaced)
|
||||
# and raises OSError on failure (which propagates as a traceback).
|
||||
# This line exists only as a defensive assertion.
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
No try/except. If `.container-mode` doesn't exist, `get_container_exec_info()` returns `None` and we skip routing. If it exists but is broken, the exception propagates with a natural traceback.
|
||||
|
||||
Note: `sys.exit(1)` after `_exec_in_container` is dead code in all paths — `os.execvp` either replaces the process or raises. It's kept as a belt-and-suspenders assertion with a comment marking it unreachable, not as actual error handling.
|
||||
|
||||
### Changes to `hermes_cli/config.py`
|
||||
|
||||
#### `get_container_exec_info` — remove inner try/except
|
||||
|
||||
Current code catches `(OSError, IOError)` and returns `None`. This silently hides permission errors, corrupt files, etc.
|
||||
|
||||
Change: Remove the try/except around file reading. Keep the early returns for `HERMES_DEV=1` and `_is_inside_container()`. The `FileNotFoundError` from `open()` when `.container-mode` doesn't exist should still return `None` (this is the "container mode not enabled" case). All other exceptions propagate.
|
||||
|
||||
```python
|
||||
def get_container_exec_info() -> Optional[dict]:
|
||||
if os.environ.get("HERMES_DEV") == "1":
|
||||
return None
|
||||
if _is_inside_container():
|
||||
return None
|
||||
|
||||
container_mode_file = get_hermes_home() / ".container-mode"
|
||||
|
||||
try:
|
||||
with open(container_mode_file, "r") as f:
|
||||
# ... parse key=value lines ...
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
# All other exceptions (PermissionError, malformed data, etc.) propagate
|
||||
|
||||
return { ... }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Spec: NixOS Module Changes
|
||||
|
||||
### Symlink creation — simplify to two branches
|
||||
|
||||
Current: 4 branches (symlink exists, directory exists, other file, doesn't exist).
|
||||
|
||||
Revised: 2 branches.
|
||||
|
||||
```bash
|
||||
if [ -d "${symlinkPath}" ] && [ ! -L "${symlinkPath}" ]; then
|
||||
# Real directory — back it up, then create symlink
|
||||
_backup="${symlinkPath}.bak.$(date +%s)"
|
||||
echo "hermes-agent: backing up existing ${symlinkPath} to $_backup"
|
||||
mv "${symlinkPath}" "$_backup"
|
||||
fi
|
||||
# For everything else (symlink, doesn't exist, etc.) — just force-create
|
||||
ln -sfn "${target}" "${symlinkPath}"
|
||||
chown -h ${user}:${cfg.group} "${symlinkPath}"
|
||||
```
|
||||
|
||||
`ln -sfn` handles: existing symlink (replaces), doesn't exist (creates), and after the `mv` above (creates). The only case that needs special handling is a real directory, because `ln -sfn` cannot atomically replace a directory.
|
||||
|
||||
Note: there is a theoretical race between the `[ -d ... ]` check and the `mv` (something could create/remove the directory in between). In practice this is a NixOS activation script running as root during `nixos-rebuild switch` — no other process should be touching `~/.hermes` at that moment. Not worth adding locking for.
|
||||
|
||||
### Sudoers — document, don't auto-configure
|
||||
|
||||
Do NOT add `security.sudo.extraRules` to the module. Document the sudoers requirement in the module's description/comments and in the error message the CLI prints when sudo probe fails.
|
||||
|
||||
### Group membership gating — keep as-is
|
||||
|
||||
The fix in 726cf90f (`cfg.container.enable && cfg.container.hostUsers != []`) is correct. Leftover group membership when container mode is disabled is harmless. No cleanup needed.
|
||||
|
||||
---
|
||||
|
||||
## Spec: Test Rewrite
|
||||
|
||||
The existing test file (`tests/hermes_cli/test_container_aware_cli.py`) has 16 tests. With the simplified exec model, several are obsolete.
|
||||
|
||||
### Tests to keep (update as needed)
|
||||
|
||||
- `test_is_inside_container_dockerenv` — unchanged
|
||||
- `test_is_inside_container_containerenv` — unchanged
|
||||
- `test_is_inside_container_cgroup_docker` — unchanged
|
||||
- `test_is_inside_container_false_on_host` — unchanged
|
||||
- `test_get_container_exec_info_returns_metadata` — unchanged
|
||||
- `test_get_container_exec_info_none_inside_container` — unchanged
|
||||
- `test_get_container_exec_info_none_without_file` — unchanged
|
||||
- `test_get_container_exec_info_skipped_when_hermes_dev` — unchanged
|
||||
- `test_get_container_exec_info_not_skipped_when_hermes_dev_zero` — unchanged
|
||||
- `test_get_container_exec_info_defaults` — unchanged
|
||||
- `test_get_container_exec_info_docker_backend` — unchanged
|
||||
|
||||
### Tests to add
|
||||
|
||||
- `test_get_container_exec_info_crashes_on_permission_error` — verify that `PermissionError` propagates (no silent `None` return)
|
||||
- `test_exec_in_container_calls_execvp` — verify `os.execvp` is called with correct args (runtime, tty flags, user, env, container, binary, cli args)
|
||||
- `test_exec_in_container_sudo_probe_sets_prefix` — verify that when first probe fails and sudo probe succeeds, `os.execvp` is called with `sudo -n` prefix
|
||||
- `test_exec_in_container_no_runtime_hard_fails` — keep existing, verify `sys.exit(1)` when `shutil.which` returns None
|
||||
- `test_exec_in_container_non_tty_uses_i_only` — update to check `os.execvp` args instead of `subprocess.run` args
|
||||
- `test_exec_in_container_probe_timeout_prints_message` — verify that `subprocess.TimeoutExpired` from the probe produces a human-readable error and `sys.exit(1)`, not a raw traceback
|
||||
- `test_exec_in_container_container_not_running_no_sudo` — verify the path where runtime exists (`shutil.which` returns a path) but probe returns non-zero and no sudo is available. Should print the "container may be running under root" error. This is distinct from `no_runtime_hard_fails` which covers `shutil.which` returning None.
|
||||
|
||||
### Tests to delete
|
||||
|
||||
- `test_exec_in_container_tty_retries_on_container_failure` — retry loop removed
|
||||
- `test_exec_in_container_non_tty_retries_silently_exits_126` — retry loop removed
|
||||
- `test_exec_in_container_propagates_hermes_exit_code` — no subprocess.run to check exit codes; execvp replaces the process. Note: exit code propagation still works correctly — when `os.execvp` succeeds, the container's process *becomes* this process, so its exit code is the process exit code by OS semantics. No application code needed, no test needed. A comment in the function docstring documents this intent for future readers.
|
||||
|
||||
---
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Auto-configuring sudoers rules in the NixOS module
|
||||
- Any changes to `get_container_exec_info` parsing logic beyond the try/except narrowing
|
||||
- Changes to `.container-mode` file format
|
||||
- Changes to the `HERMES_DEV=1` bypass
|
||||
- Changes to container detection logic (`_is_inside_container`)
|
||||
Generated
-21
@@ -36,26 +36,6 @@
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"npm-lockfile-fix": {
|
||||
"inputs": {
|
||||
"nixpkgs": [
|
||||
"nixpkgs"
|
||||
]
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1775903712,
|
||||
"narHash": "sha256-2GV79U6iVH4gKAPWYrxUReB0S41ty/Y3dBLquU8AlaA=",
|
||||
"owner": "jeslie0",
|
||||
"repo": "npm-lockfile-fix",
|
||||
"rev": "c6093acb0c0548e0f9b8b3d82918823721930fe8",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "jeslie0",
|
||||
"repo": "npm-lockfile-fix",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"pyproject-build-systems": {
|
||||
"inputs": {
|
||||
"nixpkgs": [
|
||||
@@ -144,7 +124,6 @@
|
||||
"inputs": {
|
||||
"flake-parts": "flake-parts",
|
||||
"nixpkgs": "nixpkgs",
|
||||
"npm-lockfile-fix": "npm-lockfile-fix",
|
||||
"pyproject-build-systems": "pyproject-build-systems",
|
||||
"pyproject-nix": "pyproject-nix_2",
|
||||
"uv2nix": "uv2nix_2"
|
||||
|
||||
@@ -19,20 +19,11 @@
|
||||
url = "github:pyproject-nix/build-system-pkgs";
|
||||
inputs.nixpkgs.follows = "nixpkgs";
|
||||
};
|
||||
npm-lockfile-fix = {
|
||||
url = "github:jeslie0/npm-lockfile-fix";
|
||||
inputs.nixpkgs.follows = "nixpkgs";
|
||||
};
|
||||
};
|
||||
|
||||
outputs =
|
||||
inputs:
|
||||
outputs = inputs:
|
||||
inputs.flake-parts.lib.mkFlake { inherit inputs; } {
|
||||
systems = [
|
||||
"x86_64-linux"
|
||||
"aarch64-linux"
|
||||
"aarch64-darwin"
|
||||
];
|
||||
systems = [ "x86_64-linux" "aarch64-linux" "aarch64-darwin" ];
|
||||
|
||||
imports = [
|
||||
./nix/packages.nix
|
||||
|
||||
@@ -100,7 +100,7 @@ def build_channel_directory(adapters: Dict[Any, Any]) -> Dict[str, Any]:
|
||||
|
||||
|
||||
def _build_discord(adapter) -> List[Dict[str, str]]:
|
||||
"""Enumerate all text channels and forum channels the Discord bot can see."""
|
||||
"""Enumerate all text channels the Discord bot can see."""
|
||||
channels = []
|
||||
client = getattr(adapter, "_client", None)
|
||||
if not client:
|
||||
@@ -119,15 +119,6 @@ def _build_discord(adapter) -> List[Dict[str, str]]:
|
||||
"guild": guild.name,
|
||||
"type": "channel",
|
||||
})
|
||||
# Forum channels (type 15) — creating a message auto-spawns a thread post.
|
||||
forums = getattr(guild, "forum_channels", None) or []
|
||||
for ch in forums:
|
||||
channels.append({
|
||||
"id": str(ch.id),
|
||||
"name": ch.name,
|
||||
"guild": guild.name,
|
||||
"type": "forum",
|
||||
})
|
||||
# Also include DM-capable users we've interacted with is not
|
||||
# feasible via guild enumeration; those come from sessions.
|
||||
|
||||
@@ -200,15 +191,6 @@ def load_directory() -> Dict[str, Any]:
|
||||
return {"updated_at": None, "platforms": {}}
|
||||
|
||||
|
||||
def lookup_channel_type(platform_name: str, chat_id: str) -> Optional[str]:
|
||||
"""Return the channel ``type`` string (e.g. ``"channel"``, ``"forum"``) for *chat_id*, or *None* if unknown."""
|
||||
directory = load_directory()
|
||||
for ch in directory.get("platforms", {}).get(platform_name, []):
|
||||
if ch.get("id") == chat_id:
|
||||
return ch.get("type")
|
||||
return None
|
||||
|
||||
|
||||
def resolve_channel_name(platform_name: str, name: str) -> Optional[str]:
|
||||
"""
|
||||
Resolve a human-friendly channel name to a numeric ID.
|
||||
|
||||
+2
-89
@@ -258,13 +258,6 @@ class GatewayConfig:
|
||||
# Streaming configuration
|
||||
streaming: StreamingConfig = field(default_factory=StreamingConfig)
|
||||
|
||||
# Session store pruning: drop SessionEntry records older than this many
|
||||
# days from the in-memory dict and sessions.json. Keeps the store from
|
||||
# growing unbounded in gateways serving many chats/threads/users over
|
||||
# months. Pruning is invisible to users — if they resume, they get a
|
||||
# fresh session exactly as if the reset policy had fired. 0 = disabled.
|
||||
session_store_max_age_days: int = 90
|
||||
|
||||
def get_connected_platforms(self) -> List[Platform]:
|
||||
"""Return list of platforms that are enabled and configured."""
|
||||
connected = []
|
||||
@@ -314,14 +307,6 @@ class GatewayConfig:
|
||||
# QQBot uses extra dict for app credentials
|
||||
elif platform == Platform.QQBOT and config.extra.get("app_id") and config.extra.get("client_secret"):
|
||||
connected.append(platform)
|
||||
# DingTalk uses client_id/client_secret from config.extra or env vars
|
||||
elif platform == Platform.DINGTALK and (
|
||||
config.extra.get("client_id") or os.getenv("DINGTALK_CLIENT_ID")
|
||||
) and (
|
||||
config.extra.get("client_secret") or os.getenv("DINGTALK_CLIENT_SECRET")
|
||||
):
|
||||
connected.append(platform)
|
||||
|
||||
return connected
|
||||
|
||||
def get_home_channel(self, platform: Platform) -> Optional[HomeChannel]:
|
||||
@@ -372,7 +357,6 @@ class GatewayConfig:
|
||||
"thread_sessions_per_user": self.thread_sessions_per_user,
|
||||
"unauthorized_dm_behavior": self.unauthorized_dm_behavior,
|
||||
"streaming": self.streaming.to_dict(),
|
||||
"session_store_max_age_days": self.session_store_max_age_days,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
@@ -420,13 +404,6 @@ class GatewayConfig:
|
||||
"pair",
|
||||
)
|
||||
|
||||
try:
|
||||
session_store_max_age_days = int(data.get("session_store_max_age_days", 90))
|
||||
if session_store_max_age_days < 0:
|
||||
session_store_max_age_days = 0
|
||||
except (TypeError, ValueError):
|
||||
session_store_max_age_days = 90
|
||||
|
||||
return cls(
|
||||
platforms=platforms,
|
||||
default_reset_policy=default_policy,
|
||||
@@ -441,7 +418,6 @@ class GatewayConfig:
|
||||
thread_sessions_per_user=_coerce_bool(thread_sessions_per_user, False),
|
||||
unauthorized_dm_behavior=unauthorized_dm_behavior,
|
||||
streaming=StreamingConfig.from_dict(data.get("streaming", {})),
|
||||
session_store_max_age_days=session_store_max_age_days,
|
||||
)
|
||||
|
||||
def get_unauthorized_dm_behavior(self, platform: Optional[Platform] = None) -> str:
|
||||
@@ -641,20 +617,6 @@ def load_gateway_config() -> GatewayConfig:
|
||||
if isinstance(ntc, list):
|
||||
ntc = ",".join(str(v) for v in ntc)
|
||||
os.environ["DISCORD_NO_THREAD_CHANNELS"] = str(ntc)
|
||||
# allow_mentions: granular control over what the bot can ping.
|
||||
# Safe defaults (no @everyone/roles) are applied in the adapter;
|
||||
# these YAML keys only override when set and let users opt back
|
||||
# into unsafe modes (e.g. roles=true) if they actually want it.
|
||||
allow_mentions_cfg = discord_cfg.get("allow_mentions")
|
||||
if isinstance(allow_mentions_cfg, dict):
|
||||
for yaml_key, env_key in (
|
||||
("everyone", "DISCORD_ALLOW_MENTION_EVERYONE"),
|
||||
("roles", "DISCORD_ALLOW_MENTION_ROLES"),
|
||||
("users", "DISCORD_ALLOW_MENTION_USERS"),
|
||||
("replied_user", "DISCORD_ALLOW_MENTION_REPLIED_USER"),
|
||||
):
|
||||
if yaml_key in allow_mentions_cfg and not os.getenv(env_key):
|
||||
os.environ[env_key] = str(allow_mentions_cfg[yaml_key]).lower()
|
||||
|
||||
# Telegram settings → env vars (env vars take precedence)
|
||||
telegram_cfg = yaml_cfg.get("telegram", {})
|
||||
@@ -701,24 +663,6 @@ def load_gateway_config() -> GatewayConfig:
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["WHATSAPP_FREE_RESPONSE_CHATS"] = str(frc)
|
||||
|
||||
# DingTalk settings → env vars (env vars take precedence)
|
||||
dingtalk_cfg = yaml_cfg.get("dingtalk", {})
|
||||
if isinstance(dingtalk_cfg, dict):
|
||||
if "require_mention" in dingtalk_cfg and not os.getenv("DINGTALK_REQUIRE_MENTION"):
|
||||
os.environ["DINGTALK_REQUIRE_MENTION"] = str(dingtalk_cfg["require_mention"]).lower()
|
||||
if "mention_patterns" in dingtalk_cfg and not os.getenv("DINGTALK_MENTION_PATTERNS"):
|
||||
os.environ["DINGTALK_MENTION_PATTERNS"] = json.dumps(dingtalk_cfg["mention_patterns"])
|
||||
frc = dingtalk_cfg.get("free_response_chats")
|
||||
if frc is not None and not os.getenv("DINGTALK_FREE_RESPONSE_CHATS"):
|
||||
if isinstance(frc, list):
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["DINGTALK_FREE_RESPONSE_CHATS"] = str(frc)
|
||||
allowed = dingtalk_cfg.get("allowed_users")
|
||||
if allowed is not None and not os.getenv("DINGTALK_ALLOWED_USERS"):
|
||||
if isinstance(allowed, list):
|
||||
allowed = ",".join(str(v) for v in allowed)
|
||||
os.environ["DINGTALK_ALLOWED_USERS"] = str(allowed)
|
||||
|
||||
# Matrix settings → env vars (env vars take precedence)
|
||||
matrix_cfg = yaml_cfg.get("matrix", {})
|
||||
if isinstance(matrix_cfg, dict):
|
||||
@@ -1062,25 +1006,6 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
if webhook_secret:
|
||||
config.platforms[Platform.WEBHOOK].extra["secret"] = webhook_secret
|
||||
|
||||
# DingTalk
|
||||
dingtalk_client_id = os.getenv("DINGTALK_CLIENT_ID")
|
||||
dingtalk_client_secret = os.getenv("DINGTALK_CLIENT_SECRET")
|
||||
if dingtalk_client_id and dingtalk_client_secret:
|
||||
if Platform.DINGTALK not in config.platforms:
|
||||
config.platforms[Platform.DINGTALK] = PlatformConfig()
|
||||
config.platforms[Platform.DINGTALK].enabled = True
|
||||
config.platforms[Platform.DINGTALK].extra.update({
|
||||
"client_id": dingtalk_client_id,
|
||||
"client_secret": dingtalk_client_secret,
|
||||
})
|
||||
dingtalk_home = os.getenv("DINGTALK_HOME_CHANNEL")
|
||||
if dingtalk_home:
|
||||
config.platforms[Platform.DINGTALK].home_channel = HomeChannel(
|
||||
platform=Platform.DINGTALK,
|
||||
chat_id=dingtalk_home,
|
||||
name=os.getenv("DINGTALK_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# Feishu / Lark
|
||||
feishu_app_id = os.getenv("FEISHU_APP_ID")
|
||||
feishu_app_secret = os.getenv("FEISHU_APP_SECRET")
|
||||
@@ -1229,24 +1154,12 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
qq_group_allowed = os.getenv("QQ_GROUP_ALLOWED_USERS", "").strip()
|
||||
if qq_group_allowed:
|
||||
extra["group_allow_from"] = qq_group_allowed
|
||||
qq_home = os.getenv("QQBOT_HOME_CHANNEL", "").strip()
|
||||
qq_home_name_env = "QQBOT_HOME_CHANNEL_NAME"
|
||||
if not qq_home:
|
||||
# Back-compat: accept the pre-rename name and log a one-time warning.
|
||||
legacy_home = os.getenv("QQ_HOME_CHANNEL", "").strip()
|
||||
if legacy_home:
|
||||
qq_home = legacy_home
|
||||
qq_home_name_env = "QQ_HOME_CHANNEL_NAME"
|
||||
import logging
|
||||
logging.getLogger(__name__).warning(
|
||||
"QQ_HOME_CHANNEL is deprecated; rename to QQBOT_HOME_CHANNEL "
|
||||
"in your .env for consistency with the platform key."
|
||||
)
|
||||
qq_home = os.getenv("QQ_HOME_CHANNEL", "").strip()
|
||||
if qq_home:
|
||||
config.platforms[Platform.QQBOT].home_channel = HomeChannel(
|
||||
platform=Platform.QQBOT,
|
||||
chat_id=qq_home,
|
||||
name=os.getenv("QQBOT_HOME_CHANNEL_NAME") or os.getenv(qq_home_name_env, "Home"),
|
||||
name=os.getenv("QQ_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# Session settings
|
||||
|
||||
@@ -669,15 +669,6 @@ class MessageEvent:
|
||||
# Original platform data
|
||||
raw_message: Any = None
|
||||
message_id: Optional[str] = None
|
||||
|
||||
# Platform-specific update identifier. For Telegram this is the
|
||||
# ``update_id`` from the PTB Update wrapper; other platforms currently
|
||||
# ignore it. Used by ``/restart`` to record the triggering update so the
|
||||
# new gateway can advance the Telegram offset past it and avoid processing
|
||||
# the same ``/restart`` twice if PTB's graceful-shutdown ACK times out
|
||||
# ("Error while calling `get_updates` one more time to mark all fetched
|
||||
# updates" in gateway.log).
|
||||
platform_update_id: Optional[int] = None
|
||||
|
||||
# Media attachments
|
||||
# media_urls: local file paths (for vision tool access)
|
||||
@@ -1054,40 +1045,16 @@ class BasePlatformAdapter(ABC):
|
||||
"""
|
||||
pass
|
||||
|
||||
# Default: the adapter treats ``finalize=True`` on edit_message as a
|
||||
# no-op and is happy to have the stream consumer skip redundant final
|
||||
# edits. Subclasses that *require* an explicit finalize call to close
|
||||
# out the message lifecycle (e.g. rich card / AI assistant surfaces
|
||||
# such as DingTalk AI Cards) override this to True (class attribute or
|
||||
# property) so the stream consumer knows not to short-circuit.
|
||||
REQUIRES_EDIT_FINALIZE: bool = False
|
||||
|
||||
async def edit_message(
|
||||
self,
|
||||
chat_id: str,
|
||||
message_id: str,
|
||||
content: str,
|
||||
*,
|
||||
finalize: bool = False,
|
||||
) -> SendResult:
|
||||
"""
|
||||
Edit a previously sent message. Optional — platforms that don't
|
||||
support editing return success=False and callers fall back to
|
||||
sending a new message.
|
||||
|
||||
``finalize`` signals that this is the last edit in a streaming
|
||||
sequence. Most platforms (Telegram, Slack, Discord, Matrix,
|
||||
etc.) treat it as a no-op because their edit APIs have no notion
|
||||
of message lifecycle state — an edit is an edit. Platforms that
|
||||
render streaming updates with a distinct "in progress" state and
|
||||
require explicit closure (e.g. rich card / AI assistant surfaces
|
||||
such as DingTalk AI Cards) use it to finalize the message and
|
||||
transition the UI out of the streaming indicator — those should
|
||||
also set ``REQUIRES_EDIT_FINALIZE = True`` so callers route a
|
||||
final edit through even when content is unchanged. Callers
|
||||
should set ``finalize=True`` on the final edit of a streamed
|
||||
response (typically when ``got_done`` fires in the stream
|
||||
consumer) and leave it ``False`` on intermediate edits.
|
||||
"""
|
||||
return SendResult(success=False, error="Not supported")
|
||||
|
||||
@@ -1612,9 +1579,7 @@ class BasePlatformAdapter(ABC):
|
||||
# session lifecycle and its cleanup races with the running task
|
||||
# (see PR #4926).
|
||||
cmd = event.get_command()
|
||||
from hermes_cli.commands import should_bypass_active_session
|
||||
|
||||
if should_bypass_active_session(cmd):
|
||||
if cmd in ("approve", "deny", "status", "stop", "new", "reset", "background", "restart", "queue", "q"):
|
||||
logger.debug(
|
||||
"[%s] Command '/%s' bypassing active-session guard for %s",
|
||||
self.name, cmd, session_key,
|
||||
@@ -1926,18 +1891,9 @@ class BasePlatformAdapter(ABC):
|
||||
if session_key in self._pending_messages:
|
||||
pending_event = self._pending_messages.pop(session_key)
|
||||
logger.debug("[%s] Processing queued message from interrupt", self.name)
|
||||
# Keep the _active_sessions entry live across the turn chain
|
||||
# and only CLEAR the interrupt Event — do NOT delete the entry.
|
||||
# If we deleted here, a concurrent inbound message arriving
|
||||
# during the awaits below would pass the Level-1 guard, spawn
|
||||
# its own _process_message_background, and run simultaneously
|
||||
# with the recursive drain below. Two agents on one
|
||||
# session_key = duplicate responses, duplicate tool calls.
|
||||
# Clearing the Event keeps the guard live so follow-ups take
|
||||
# the busy-handler path (queue + interrupt) as intended.
|
||||
_active = self._active_sessions.get(session_key)
|
||||
if _active is not None:
|
||||
_active.clear()
|
||||
# Clean up current session before processing pending
|
||||
if session_key in self._active_sessions:
|
||||
del self._active_sessions[session_key]
|
||||
typing_task.cancel()
|
||||
try:
|
||||
await typing_task
|
||||
@@ -1995,34 +1951,6 @@ class BasePlatformAdapter(ABC):
|
||||
await self.stop_typing(event.source.chat_id)
|
||||
except Exception:
|
||||
pass
|
||||
# Late-arrival drain: a message may have arrived during the
|
||||
# cleanup awaits above (typing_task cancel, stop_typing). Such
|
||||
# messages passed the Level-1 guard (entry still live, Event
|
||||
# possibly set) and landed in _pending_messages via the
|
||||
# busy-handler path. Without this block, we would delete the
|
||||
# active-session entry and the queued message would be silently
|
||||
# dropped (user never gets a reply).
|
||||
late_pending = self._pending_messages.pop(session_key, None)
|
||||
if late_pending is not None:
|
||||
logger.debug(
|
||||
"[%s] Late-arrival pending message during cleanup — spawning drain task",
|
||||
self.name,
|
||||
)
|
||||
_active = self._active_sessions.get(session_key)
|
||||
if _active is not None:
|
||||
_active.clear()
|
||||
drain_task = asyncio.create_task(
|
||||
self._process_message_background(late_pending, session_key)
|
||||
)
|
||||
try:
|
||||
self._background_tasks.add(drain_task)
|
||||
drain_task.add_done_callback(self._background_tasks.discard)
|
||||
except TypeError:
|
||||
# Tests stub create_task() with non-hashable sentinels; tolerate.
|
||||
pass
|
||||
# Leave _active_sessions[session_key] populated — the drain
|
||||
# task's own lifecycle will clean it up.
|
||||
return
|
||||
# Clean up session tracking
|
||||
if session_key in self._active_sessions:
|
||||
del self._active_sessions[session_key]
|
||||
@@ -2063,7 +1991,6 @@ class BasePlatformAdapter(ABC):
|
||||
chat_topic: Optional[str] = None,
|
||||
user_id_alt: Optional[str] = None,
|
||||
chat_id_alt: Optional[str] = None,
|
||||
is_bot: bool = False,
|
||||
) -> SessionSource:
|
||||
"""Helper to build a SessionSource for this platform."""
|
||||
# Normalize empty topic to None
|
||||
@@ -2080,7 +2007,6 @@ class BasePlatformAdapter(ABC):
|
||||
chat_topic=chat_topic.strip() if chat_topic else None,
|
||||
user_id_alt=user_id_alt,
|
||||
chat_id_alt=chat_id_alt,
|
||||
is_bot=is_bot,
|
||||
)
|
||||
|
||||
@abstractmethod
|
||||
|
||||
+69
-1084
File diff suppressed because it is too large
Load Diff
+92
-580
@@ -51,9 +51,7 @@ from gateway.platforms.base import (
|
||||
ProcessingOutcome,
|
||||
SendResult,
|
||||
cache_image_from_url,
|
||||
cache_image_from_bytes,
|
||||
cache_audio_from_url,
|
||||
cache_audio_from_bytes,
|
||||
cache_document_from_bytes,
|
||||
SUPPORTED_DOCUMENT_TYPES,
|
||||
)
|
||||
@@ -82,41 +80,6 @@ def check_discord_requirements() -> bool:
|
||||
return DISCORD_AVAILABLE
|
||||
|
||||
|
||||
def _build_allowed_mentions():
|
||||
"""Build Discord ``AllowedMentions`` with safe defaults, overridable via env.
|
||||
|
||||
Discord bots default to parsing ``@everyone``, ``@here``, role pings, and
|
||||
user pings when ``allowed_mentions`` is unset on the client — any LLM
|
||||
output or echoed user content that contains ``@everyone`` would therefore
|
||||
ping the whole server. We explicitly deny ``@everyone`` and role pings
|
||||
by default and keep user / replied-user pings enabled so normal
|
||||
conversation still works.
|
||||
|
||||
Override via environment variables (or ``discord.allow_mentions.*`` in
|
||||
config.yaml):
|
||||
|
||||
DISCORD_ALLOW_MENTION_EVERYONE default false — @everyone + @here
|
||||
DISCORD_ALLOW_MENTION_ROLES default false — @role pings
|
||||
DISCORD_ALLOW_MENTION_USERS default true — @user pings
|
||||
DISCORD_ALLOW_MENTION_REPLIED_USER default true — reply-ping author
|
||||
"""
|
||||
if not DISCORD_AVAILABLE:
|
||||
return None
|
||||
|
||||
def _b(name: str, default: bool) -> bool:
|
||||
raw = os.getenv(name, "").strip().lower()
|
||||
if not raw:
|
||||
return default
|
||||
return raw in ("true", "1", "yes", "on")
|
||||
|
||||
return discord.AllowedMentions(
|
||||
everyone=_b("DISCORD_ALLOW_MENTION_EVERYONE", False),
|
||||
roles=_b("DISCORD_ALLOW_MENTION_ROLES", False),
|
||||
users=_b("DISCORD_ALLOW_MENTION_USERS", True),
|
||||
replied_user=_b("DISCORD_ALLOW_MENTION_REPLIED_USER", True),
|
||||
)
|
||||
|
||||
|
||||
class VoiceReceiver:
|
||||
"""Captures and decodes voice audio from a Discord voice channel.
|
||||
|
||||
@@ -495,7 +458,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
self._client: Optional[commands.Bot] = None
|
||||
self._ready_event = asyncio.Event()
|
||||
self._allowed_user_ids: set = set() # For button approval authorization
|
||||
self._allowed_role_ids: set = set() # For DISCORD_ALLOWED_ROLES filtering
|
||||
# Voice channel state (per-guild)
|
||||
self._voice_clients: Dict[int, Any] = {} # guild_id -> VoiceClient
|
||||
# Text batching: merge rapid successive messages (Telegram-style)
|
||||
@@ -574,15 +536,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
if uid.strip()
|
||||
}
|
||||
|
||||
# Parse DISCORD_ALLOWED_ROLES — comma-separated role IDs.
|
||||
# Users with ANY of these roles can interact with the bot.
|
||||
roles_env = os.getenv("DISCORD_ALLOWED_ROLES", "")
|
||||
if roles_env:
|
||||
self._allowed_role_ids = {
|
||||
int(rid.strip()) for rid in roles_env.split(",")
|
||||
if rid.strip().isdigit()
|
||||
}
|
||||
|
||||
# Set up intents.
|
||||
# Message Content is required for normal text replies.
|
||||
# Server Members is only needed when the allowlist contains usernames
|
||||
@@ -594,10 +547,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
intents.message_content = True
|
||||
intents.dm_messages = True
|
||||
intents.guild_messages = True
|
||||
intents.members = (
|
||||
any(not entry.isdigit() for entry in self._allowed_user_ids)
|
||||
or bool(self._allowed_role_ids) # Need members intent for role lookup
|
||||
)
|
||||
intents.members = any(not entry.isdigit() for entry in self._allowed_user_ids)
|
||||
intents.voice_states = True
|
||||
|
||||
# Resolve proxy (DISCORD_PROXY > generic env vars > macOS system proxy)
|
||||
@@ -606,15 +556,10 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
if proxy_url:
|
||||
logger.info("[%s] Using proxy for Discord: %s", self.name, proxy_url)
|
||||
|
||||
# Create bot — proxy= for HTTP, connector= for SOCKS.
|
||||
# allowed_mentions is set with safe defaults (no @everyone/roles)
|
||||
# so LLM output or echoed user content can't ping the whole
|
||||
# server; override per DISCORD_ALLOW_MENTION_* env vars or the
|
||||
# discord.allow_mentions.* block in config.yaml.
|
||||
# Create bot — proxy= for HTTP, connector= for SOCKS
|
||||
self._client = commands.Bot(
|
||||
command_prefix="!", # Not really used, we handle raw messages
|
||||
intents=intents,
|
||||
allowed_mentions=_build_allowed_mentions(),
|
||||
**proxy_kwargs_for_bot(proxy_url),
|
||||
)
|
||||
adapter_self = self # capture for closure
|
||||
@@ -649,13 +594,14 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
if message.type not in (discord.MessageType.default, discord.MessageType.reply):
|
||||
return
|
||||
|
||||
# Check if the message author is in the allowed user list
|
||||
if not self._is_allowed_user(str(message.author.id)):
|
||||
return
|
||||
|
||||
# Bot message filtering (DISCORD_ALLOW_BOTS):
|
||||
# "none" — ignore all other bots (default)
|
||||
# "mentions" — accept bot messages only when they @mention us
|
||||
# "all" — accept all bot messages
|
||||
# Must run BEFORE the user allowlist check so that bots
|
||||
# permitted by DISCORD_ALLOW_BOTS are not rejected for
|
||||
# not being in DISCORD_ALLOWED_USERS (fixes #4466).
|
||||
if getattr(message.author, "bot", False):
|
||||
allow_bots = os.getenv("DISCORD_ALLOW_BOTS", "none").lower().strip()
|
||||
if allow_bots == "none":
|
||||
@@ -663,12 +609,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
elif allow_bots == "mentions":
|
||||
if not self._client.user or self._client.user not in message.mentions:
|
||||
return
|
||||
# "all" falls through; bot is permitted — skip the
|
||||
# human-user allowlist below (bots aren't in it).
|
||||
else:
|
||||
# Non-bot: enforce the configured user/role allowlists.
|
||||
if not self._is_allowed_user(str(message.author.id), message.author):
|
||||
return
|
||||
# "all" falls through to handle_message
|
||||
|
||||
# Multi-agent filtering: if the message mentions specific bots
|
||||
# but NOT this bot, the sender is talking to another agent —
|
||||
@@ -857,9 +798,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
|
||||
When metadata contains a thread_id, the message is sent to that
|
||||
thread instead of the parent channel identified by chat_id.
|
||||
|
||||
Forum channels (type 15) reject direct messages — a thread post is
|
||||
created automatically.
|
||||
"""
|
||||
if not self._client:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
@@ -885,10 +823,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
if not channel:
|
||||
return SendResult(success=False, error=f"Channel {chat_id} not found")
|
||||
|
||||
# Forum channels reject channel.send() — create a thread post instead.
|
||||
if self._is_forum_parent(channel):
|
||||
return await self._send_to_forum(channel, content)
|
||||
|
||||
# Format and split message if needed
|
||||
formatted = self.format_message(content)
|
||||
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
|
||||
@@ -899,10 +833,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
if reply_to and self._reply_to_mode != "off":
|
||||
try:
|
||||
ref_msg = await channel.fetch_message(int(reply_to))
|
||||
if hasattr(ref_msg, "to_reference"):
|
||||
reference = ref_msg.to_reference(fail_if_not_exists=False)
|
||||
else:
|
||||
reference = ref_msg
|
||||
reference = ref_msg
|
||||
except Exception as e:
|
||||
logger.debug("Could not fetch reply-to message: %s", e)
|
||||
|
||||
@@ -920,20 +851,14 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
err_text = str(e)
|
||||
if (
|
||||
chunk_reference is not None
|
||||
and (
|
||||
(
|
||||
"error code: 50035" in err_text
|
||||
and "Cannot reply to a system message" in err_text
|
||||
)
|
||||
or "error code: 10008" in err_text
|
||||
)
|
||||
and "error code: 50035" in err_text
|
||||
and "Cannot reply to a system message" in err_text
|
||||
):
|
||||
logger.warning(
|
||||
"[%s] Reply target %s rejected the reply reference; retrying send without reply reference",
|
||||
"[%s] Reply target %s is a Discord system message; retrying send without reply reference",
|
||||
self.name,
|
||||
reply_to,
|
||||
)
|
||||
reference = None
|
||||
msg = await channel.send(
|
||||
content=chunk,
|
||||
reference=None,
|
||||
@@ -952,120 +877,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
logger.error("[%s] Failed to send Discord message: %s", self.name, e, exc_info=True)
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
async def _send_to_forum(self, forum_channel: Any, content: str) -> SendResult:
|
||||
"""Create a thread post in a forum channel with the message as starter content.
|
||||
|
||||
Forum channels (type 15) don't support direct messages. Instead we
|
||||
POST to /channels/{forum_id}/threads with a thread name derived from
|
||||
the first line of the message. Any follow-up chunk failures are
|
||||
reported in ``raw_response['warnings']`` so the caller can surface
|
||||
partial-send issues.
|
||||
"""
|
||||
from tools.send_message_tool import _derive_forum_thread_name
|
||||
|
||||
formatted = self.format_message(content)
|
||||
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
|
||||
|
||||
thread_name = _derive_forum_thread_name(content)
|
||||
|
||||
starter_content = chunks[0] if chunks else thread_name
|
||||
|
||||
try:
|
||||
thread = await forum_channel.create_thread(
|
||||
name=thread_name,
|
||||
content=starter_content,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error("[%s] Failed to create forum thread in %s: %s", self.name, forum_channel.id, e)
|
||||
return SendResult(success=False, error=f"Forum thread creation failed: {e}")
|
||||
|
||||
thread_channel = thread if hasattr(thread, "send") else getattr(thread, "thread", None)
|
||||
thread_id = str(getattr(thread_channel, "id", getattr(thread, "id", "")))
|
||||
starter_msg = getattr(thread, "message", None)
|
||||
message_id = str(getattr(starter_msg, "id", thread_id)) if starter_msg else thread_id
|
||||
|
||||
# Send remaining chunks into the newly created thread. Track any
|
||||
# per-chunk failures so the caller sees partial-send outcomes.
|
||||
message_ids = [message_id]
|
||||
warnings: list[str] = []
|
||||
for chunk in chunks[1:]:
|
||||
try:
|
||||
msg = await thread_channel.send(content=chunk)
|
||||
message_ids.append(str(msg.id))
|
||||
except Exception as e:
|
||||
warning = f"Failed to send follow-up chunk to forum thread {thread_id}: {e}"
|
||||
logger.warning("[%s] %s", self.name, warning)
|
||||
warnings.append(warning)
|
||||
|
||||
raw_response: Dict[str, Any] = {"message_ids": message_ids, "thread_id": thread_id}
|
||||
if warnings:
|
||||
raw_response["warnings"] = warnings
|
||||
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=message_ids[0],
|
||||
raw_response=raw_response,
|
||||
)
|
||||
|
||||
async def _forum_post_file(
|
||||
self,
|
||||
forum_channel: Any,
|
||||
*,
|
||||
thread_name: Optional[str] = None,
|
||||
content: str = "",
|
||||
file: Any = None,
|
||||
files: Optional[list] = None,
|
||||
) -> SendResult:
|
||||
"""Create a forum thread whose starter message carries file attachments.
|
||||
|
||||
Used by the send_voice / send_image_file / send_document paths when
|
||||
the target channel is a forum (type 15). ``create_thread`` on a
|
||||
ForumChannel accepts the same file/files/content kwargs as
|
||||
``channel.send``, creating the thread and starter message atomically.
|
||||
"""
|
||||
from tools.send_message_tool import _derive_forum_thread_name
|
||||
|
||||
if not thread_name:
|
||||
# Prefer the text content, fall back to the first attached
|
||||
# filename, fall back to the generic default.
|
||||
hint = content or ""
|
||||
if not hint.strip():
|
||||
if file is not None:
|
||||
hint = getattr(file, "filename", "") or ""
|
||||
elif files:
|
||||
hint = getattr(files[0], "filename", "") or ""
|
||||
thread_name = _derive_forum_thread_name(hint) if hint.strip() else "New Post"
|
||||
|
||||
kwargs: Dict[str, Any] = {"name": thread_name}
|
||||
if content:
|
||||
kwargs["content"] = content
|
||||
if file is not None:
|
||||
kwargs["file"] = file
|
||||
if files:
|
||||
kwargs["files"] = files
|
||||
|
||||
try:
|
||||
thread = await forum_channel.create_thread(**kwargs)
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
"[%s] Failed to create forum thread with file in %s: %s",
|
||||
self.name,
|
||||
getattr(forum_channel, "id", "?"),
|
||||
e,
|
||||
)
|
||||
return SendResult(success=False, error=f"Forum thread creation failed: {e}")
|
||||
|
||||
thread_channel = thread if hasattr(thread, "send") else getattr(thread, "thread", None)
|
||||
thread_id = str(getattr(thread_channel, "id", getattr(thread, "id", "")))
|
||||
starter_msg = getattr(thread, "message", None)
|
||||
message_id = str(getattr(starter_msg, "id", thread_id)) if starter_msg else thread_id
|
||||
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=message_id,
|
||||
raw_response={"thread_id": thread_id},
|
||||
)
|
||||
|
||||
async def edit_message(
|
||||
self,
|
||||
chat_id: str,
|
||||
@@ -1096,11 +907,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
caption: Optional[str] = None,
|
||||
file_name: Optional[str] = None,
|
||||
) -> SendResult:
|
||||
"""Send a local file as a Discord attachment.
|
||||
|
||||
Forum channels (type 15) get a new thread whose starter message
|
||||
carries the file — they reject direct POST /messages.
|
||||
"""
|
||||
"""Send a local file as a Discord attachment."""
|
||||
if not self._client:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
|
||||
@@ -1113,12 +920,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
filename = file_name or os.path.basename(file_path)
|
||||
with open(file_path, "rb") as fh:
|
||||
file = discord.File(fh, filename=filename)
|
||||
if self._is_forum_parent(channel):
|
||||
return await self._forum_post_file(
|
||||
channel,
|
||||
content=(caption or "").strip(),
|
||||
file=file,
|
||||
)
|
||||
msg = await channel.send(content=caption if caption else None, file=file)
|
||||
return SendResult(success=True, message_id=str(msg.id))
|
||||
|
||||
@@ -1167,18 +968,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
with open(audio_path, "rb") as f:
|
||||
file_data = f.read()
|
||||
|
||||
# Forum channels (type 15) reject direct POST /messages — the
|
||||
# native voice flag path also targets /messages so it would fail
|
||||
# too. Create a thread post with the audio as the starter
|
||||
# attachment instead.
|
||||
if self._is_forum_parent(channel):
|
||||
forum_file = discord.File(io.BytesIO(file_data), filename=filename)
|
||||
return await self._forum_post_file(
|
||||
channel,
|
||||
content=(caption or "").strip(),
|
||||
file=forum_file,
|
||||
)
|
||||
|
||||
# Try sending as a native voice message via raw API (flags=8192).
|
||||
try:
|
||||
import base64
|
||||
@@ -1521,48 +1310,11 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _is_allowed_user(self, user_id: str, author=None) -> bool:
|
||||
"""Check if user is allowed via DISCORD_ALLOWED_USERS or DISCORD_ALLOWED_ROLES.
|
||||
|
||||
Uses OR semantics: if the user matches EITHER allowlist, they're allowed.
|
||||
If both allowlists are empty, everyone is allowed (backwards compatible).
|
||||
When author is a Member, checks .roles directly; otherwise falls back
|
||||
to scanning the bot's mutual guilds for a Member record.
|
||||
"""
|
||||
# ``getattr`` fallbacks here guard against test fixtures that build
|
||||
# an adapter via ``object.__new__(DiscordAdapter)`` and skip __init__
|
||||
# (see AGENTS.md pitfall #17 — same pattern as gateway.run).
|
||||
allowed_users = getattr(self, "_allowed_user_ids", set())
|
||||
allowed_roles = getattr(self, "_allowed_role_ids", set())
|
||||
has_users = bool(allowed_users)
|
||||
has_roles = bool(allowed_roles)
|
||||
if not has_users and not has_roles:
|
||||
def _is_allowed_user(self, user_id: str) -> bool:
|
||||
"""Check if user is in DISCORD_ALLOWED_USERS."""
|
||||
if not self._allowed_user_ids:
|
||||
return True
|
||||
# Check user ID allowlist
|
||||
if has_users and user_id in allowed_users:
|
||||
return True
|
||||
# Check role allowlist
|
||||
if has_roles:
|
||||
# Try direct role check from Member object
|
||||
direct_roles = getattr(author, "roles", None) if author is not None else None
|
||||
if direct_roles:
|
||||
if any(getattr(r, "id", None) in allowed_roles for r in direct_roles):
|
||||
return True
|
||||
# Fallback: scan mutual guilds for member's roles
|
||||
if self._client is not None:
|
||||
try:
|
||||
uid_int = int(user_id)
|
||||
except (TypeError, ValueError):
|
||||
uid_int = None
|
||||
if uid_int is not None:
|
||||
for guild in self._client.guilds:
|
||||
m = guild.get_member(uid_int)
|
||||
if m is None:
|
||||
continue
|
||||
m_roles = getattr(m, "roles", None) or []
|
||||
if any(getattr(r, "id", None) in allowed_roles for r in m_roles):
|
||||
return True
|
||||
return False
|
||||
return user_id in self._allowed_user_ids
|
||||
|
||||
async def send_image_file(
|
||||
self,
|
||||
@@ -1631,13 +1383,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
import io
|
||||
file = discord.File(io.BytesIO(image_data), filename=f"image.{ext}")
|
||||
|
||||
if self._is_forum_parent(channel):
|
||||
return await self._forum_post_file(
|
||||
channel,
|
||||
content=(caption or "").strip(),
|
||||
file=file,
|
||||
)
|
||||
|
||||
msg = await channel.send(
|
||||
content=caption if caption else None,
|
||||
file=file,
|
||||
@@ -1700,13 +1445,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
import io
|
||||
file = discord.File(io.BytesIO(animation_data), filename="animation.gif")
|
||||
|
||||
if self._is_forum_parent(channel):
|
||||
return await self._forum_post_file(
|
||||
channel,
|
||||
content=(caption or "").strip(),
|
||||
file=file,
|
||||
)
|
||||
|
||||
msg = await channel.send(
|
||||
content=caption if caption else None,
|
||||
file=file,
|
||||
@@ -1933,24 +1671,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
the "thinking..." indicator is replaced with that text; otherwise it
|
||||
is deleted so the channel isn't cluttered.
|
||||
"""
|
||||
# Log the invoker so ghost-command reports can be triaged. Discord
|
||||
# native slash invocations are always user-initiated (no bot can fire
|
||||
# them), but mobile autocomplete / keyboard shortcuts / other users
|
||||
# in the same channel are easy to miss in post-mortems.
|
||||
try:
|
||||
_user = interaction.user
|
||||
_chan_id = getattr(interaction.channel, "id", None) or getattr(interaction, "channel_id", None)
|
||||
logger.info(
|
||||
"[Discord] slash '%s' invoked by user=%s id=%s channel=%s guild=%s",
|
||||
command_text,
|
||||
getattr(_user, "name", "?"),
|
||||
getattr(_user, "id", "?"),
|
||||
_chan_id,
|
||||
getattr(interaction, "guild_id", None),
|
||||
)
|
||||
except Exception:
|
||||
pass # logging must never block command dispatch
|
||||
|
||||
await interaction.response.defer(ephemeral=True)
|
||||
event = self._build_slash_event(interaction, command_text)
|
||||
await self.handle_message(event)
|
||||
@@ -2012,11 +1732,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
async def slash_stop(interaction: discord.Interaction):
|
||||
await self._run_simple_slash(interaction, "/stop", "Stop requested~")
|
||||
|
||||
@tree.command(name="steer", description="Inject a message after the next tool call (no interrupt)")
|
||||
@discord.app_commands.describe(prompt="Text to inject into the agent's next tool result")
|
||||
async def slash_steer(interaction: discord.Interaction, prompt: str):
|
||||
await self._run_simple_slash(interaction, f"/steer {prompt}".strip())
|
||||
|
||||
@tree.command(name="compress", description="Compress conversation context")
|
||||
async def slash_compress(interaction: discord.Interaction):
|
||||
await self._run_simple_slash(interaction, "/compress")
|
||||
@@ -2189,23 +1904,12 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
self._register_skill_group(tree)
|
||||
|
||||
def _register_skill_group(self, tree) -> None:
|
||||
"""Register a single ``/skill`` command with autocomplete on the name.
|
||||
"""Register a ``/skill`` command group with category subcommand groups.
|
||||
|
||||
Discord enforces an ~8000-byte per-command payload limit. The older
|
||||
nested layout (``/skill <category> <name>``) registered one giant
|
||||
command whose serialized payload grew linearly with the skill
|
||||
catalog — with the default ~75 skills the payload was ~14 KB and
|
||||
``tree.sync()`` rejected the entire slash-command batch (issues
|
||||
#11321, #10259, #11385, #10261, #10214).
|
||||
|
||||
Autocomplete options are fetched dynamically by Discord when the
|
||||
user types — they do NOT count against the per-command registration
|
||||
budget. So we register ONE flat ``/skill`` command with
|
||||
``name: str`` (autocompleted) and ``args: str = ""``. This scales
|
||||
to thousands of skills with no size math, no splitting, and no
|
||||
hidden skills. The slash picker also becomes more discoverable —
|
||||
Discord live-filters by the user's typed prefix against both the
|
||||
skill name and its description.
|
||||
Skills are organized by their directory category under ``SKILLS_DIR``.
|
||||
Each category becomes a subcommand group; root-level skills become
|
||||
direct subcommands. Discord supports 25 subcommand groups × 25
|
||||
subcommands each = 625 skills — well beyond the old 100-command cap.
|
||||
"""
|
||||
try:
|
||||
from hermes_cli.commands import discord_skill_commands_by_category
|
||||
@@ -2216,97 +1920,68 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Reuse the existing collector for consistent filtering
|
||||
# (per-platform disabled, hub-excluded, name clamping), then
|
||||
# flatten — the category grouping was only useful for the
|
||||
# nested layout.
|
||||
categories, uncategorized, hidden = discord_skill_commands_by_category(
|
||||
reserved_names=existing_names,
|
||||
)
|
||||
entries: list[tuple[str, str, str]] = list(uncategorized)
|
||||
for cat_skills in categories.values():
|
||||
entries.extend(cat_skills)
|
||||
|
||||
if not entries:
|
||||
if not categories and not uncategorized:
|
||||
return
|
||||
|
||||
# Stable alphabetical order so the autocomplete suggestion
|
||||
# list is predictable across restarts.
|
||||
entries.sort(key=lambda t: t[0])
|
||||
|
||||
# name -> (description, cmd_key) — used by both the autocomplete
|
||||
# callback and the handler for O(1) dispatch.
|
||||
skill_lookup: dict[str, tuple[str, str]] = {
|
||||
n: (d, k) for n, d, k in entries
|
||||
}
|
||||
|
||||
async def _autocomplete_name(
|
||||
interaction: "discord.Interaction", current: str,
|
||||
) -> list:
|
||||
"""Filter skills by the user's typed prefix.
|
||||
|
||||
Matches both the skill name and its description so
|
||||
"/skill pdf" surfaces skills whose description mentions
|
||||
PDFs even if the name doesn't. Discord caps this list at
|
||||
25 entries per query.
|
||||
"""
|
||||
q = (current or "").strip().lower()
|
||||
choices: list = []
|
||||
for name, desc, _key in entries:
|
||||
if not q or q in name.lower() or (desc and q in desc.lower()):
|
||||
if desc:
|
||||
label = f"{name} — {desc}"
|
||||
else:
|
||||
label = name
|
||||
# Discord's Choice.name is capped at 100 chars.
|
||||
if len(label) > 100:
|
||||
label = label[:97] + "..."
|
||||
choices.append(
|
||||
discord.app_commands.Choice(name=label, value=name)
|
||||
)
|
||||
if len(choices) >= 25:
|
||||
break
|
||||
return choices
|
||||
|
||||
@discord.app_commands.describe(
|
||||
name="Which skill to run",
|
||||
args="Optional arguments for the skill",
|
||||
)
|
||||
@discord.app_commands.autocomplete(name=_autocomplete_name)
|
||||
async def _skill_handler(
|
||||
interaction: "discord.Interaction", name: str, args: str = "",
|
||||
):
|
||||
entry = skill_lookup.get(name)
|
||||
if not entry:
|
||||
await interaction.response.send_message(
|
||||
f"Unknown skill: `{name}`. Start typing for "
|
||||
f"autocomplete suggestions.",
|
||||
ephemeral=True,
|
||||
)
|
||||
return
|
||||
_desc, cmd_key = entry
|
||||
await self._run_simple_slash(
|
||||
interaction, f"{cmd_key} {args}".strip()
|
||||
)
|
||||
|
||||
cmd = discord.app_commands.Command(
|
||||
skill_group = discord.app_commands.Group(
|
||||
name="skill",
|
||||
description="Run a Hermes skill",
|
||||
callback=_skill_handler,
|
||||
)
|
||||
tree.add_command(cmd)
|
||||
|
||||
# ── Helper: build a callback for a skill command key ──
|
||||
def _make_handler(_key: str):
|
||||
@discord.app_commands.describe(args="Optional arguments for the skill")
|
||||
async def _handler(interaction: discord.Interaction, args: str = ""):
|
||||
await self._run_simple_slash(interaction, f"{_key} {args}".strip())
|
||||
_handler.__name__ = f"skill_{_key.lstrip('/').replace('-', '_')}"
|
||||
return _handler
|
||||
|
||||
# ── Uncategorized (root-level) skills → direct subcommands ──
|
||||
for discord_name, description, cmd_key in uncategorized:
|
||||
cmd = discord.app_commands.Command(
|
||||
name=discord_name,
|
||||
description=description or f"Run the {discord_name} skill",
|
||||
callback=_make_handler(cmd_key),
|
||||
)
|
||||
skill_group.add_command(cmd)
|
||||
|
||||
# ── Category subcommand groups ──
|
||||
for cat_name in sorted(categories):
|
||||
cat_desc = f"{cat_name.replace('-', ' ').title()} skills"
|
||||
if len(cat_desc) > 100:
|
||||
cat_desc = cat_desc[:97] + "..."
|
||||
cat_group = discord.app_commands.Group(
|
||||
name=cat_name,
|
||||
description=cat_desc,
|
||||
parent=skill_group,
|
||||
)
|
||||
for discord_name, description, cmd_key in categories[cat_name]:
|
||||
cmd = discord.app_commands.Command(
|
||||
name=discord_name,
|
||||
description=description or f"Run the {discord_name} skill",
|
||||
callback=_make_handler(cmd_key),
|
||||
)
|
||||
cat_group.add_command(cmd)
|
||||
|
||||
tree.add_command(skill_group)
|
||||
|
||||
total = sum(len(v) for v in categories.values()) + len(uncategorized)
|
||||
logger.info(
|
||||
"[%s] Registered /skill command with %d skill(s) via autocomplete",
|
||||
self.name, len(entries),
|
||||
"[%s] Registered /skill group: %d skill(s) across %d categories"
|
||||
" + %d uncategorized",
|
||||
self.name, total, len(categories), len(uncategorized),
|
||||
)
|
||||
if hidden:
|
||||
logger.info(
|
||||
"[%s] %d skill(s) filtered out of /skill (name clamp / reserved)",
|
||||
logger.warning(
|
||||
"[%s] %d skill(s) not registered (Discord subcommand limits)",
|
||||
self.name, hidden,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("[%s] Failed to register /skill command: %s", self.name, exc)
|
||||
logger.warning("[%s] Failed to register /skill group: %s", self.name, exc)
|
||||
|
||||
def _build_slash_event(self, interaction: discord.Interaction, text: str) -> MessageEvent:
|
||||
"""Build a MessageEvent from a Discord slash command interaction."""
|
||||
@@ -2465,26 +2140,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
from gateway.platforms.base import resolve_channel_prompt
|
||||
return resolve_channel_prompt(self.config.extra, channel_id, parent_id)
|
||||
|
||||
def _discord_require_mention(self) -> bool:
|
||||
"""Return whether Discord channel messages require a bot mention."""
|
||||
configured = self.config.extra.get("require_mention")
|
||||
if configured is not None:
|
||||
if isinstance(configured, str):
|
||||
return configured.lower() not in ("false", "0", "no", "off")
|
||||
return bool(configured)
|
||||
return os.getenv("DISCORD_REQUIRE_MENTION", "true").lower() not in ("false", "0", "no", "off")
|
||||
|
||||
def _discord_free_response_channels(self) -> set:
|
||||
"""Return Discord channel IDs where no bot mention is required."""
|
||||
raw = self.config.extra.get("free_response_channels")
|
||||
if raw is None:
|
||||
raw = os.getenv("DISCORD_FREE_RESPONSE_CHANNELS", "")
|
||||
if isinstance(raw, list):
|
||||
return {str(part).strip() for part in raw if str(part).strip()}
|
||||
if isinstance(raw, str) and raw.strip():
|
||||
return {part.strip() for part in raw.split(",") if part.strip()}
|
||||
return set()
|
||||
|
||||
def _thread_parent_channel(self, channel: Any) -> Any:
|
||||
"""Return the parent text channel when invoked from a thread."""
|
||||
return getattr(channel, "parent", None) or channel
|
||||
@@ -2587,15 +2242,8 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
|
||||
Returns the created thread object, or ``None`` on failure.
|
||||
"""
|
||||
# Build a short thread name from the message. Strip Discord mention
|
||||
# syntax (users / roles / channels) so thread titles don't end up
|
||||
# showing raw <@id>, <@&id>, or <#id> markers — the ID isn't
|
||||
# meaningful to humans glancing at the thread list (#6336).
|
||||
# Build a short thread name from the message
|
||||
content = (message.content or "").strip()
|
||||
# <@123>, <@!123>, <@&123>, <#123> — collapse to empty; normalize spaces.
|
||||
content = re.sub(r"<@[!&]?\d+>", "", content)
|
||||
content = re.sub(r"<#\d+>", "", content)
|
||||
content = re.sub(r"\s+", " ", content).strip()
|
||||
thread_name = content[:80] if content else "Hermes"
|
||||
if len(content) > 80:
|
||||
thread_name = thread_name[:77] + "..."
|
||||
@@ -2603,25 +2251,9 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
try:
|
||||
thread = await message.create_thread(name=thread_name, auto_archive_duration=1440)
|
||||
return thread
|
||||
except Exception as direct_error:
|
||||
display_name = getattr(getattr(message, "author", None), "display_name", None) or "unknown user"
|
||||
reason = f"Auto-threaded from mention by {display_name}"
|
||||
try:
|
||||
seed_msg = await message.channel.send(f"\U0001f9f5 Thread created by Hermes: **{thread_name}**")
|
||||
thread = await seed_msg.create_thread(
|
||||
name=thread_name,
|
||||
auto_archive_duration=1440,
|
||||
reason=reason,
|
||||
)
|
||||
return thread
|
||||
except Exception as fallback_error:
|
||||
logger.warning(
|
||||
"[%s] Auto-thread creation failed. Direct error: %s. Fallback error: %s",
|
||||
self.name,
|
||||
direct_error,
|
||||
fallback_error,
|
||||
)
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.warning("[%s] Auto-thread creation failed: %s", self.name, e)
|
||||
return None
|
||||
|
||||
async def send_exec_approval(
|
||||
self, chat_id: str, command: str, session_key: str,
|
||||
@@ -2808,124 +2440,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
return f"{parent_name} / {thread_name}"
|
||||
return thread_name
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Attachment download helpers
|
||||
#
|
||||
# Discord attachments (images / audio / documents) are fetched via the
|
||||
# authenticated bot session whenever the Attachment object exposes
|
||||
# ``read()``. That sidesteps two classes of bug that hit the older
|
||||
# plain-HTTP path:
|
||||
#
|
||||
# 1. ``cdn.discordapp.com`` URLs increasingly require bot auth on
|
||||
# download — unauthenticated httpx sees 403 Forbidden.
|
||||
# (issue #8242)
|
||||
# 2. Some user environments (VPNs, corporate DNS, tunnels) resolve
|
||||
# ``cdn.discordapp.com`` to private-looking IPs that our
|
||||
# ``is_safe_url`` guard classifies as SSRF risks. Routing the
|
||||
# fetch through discord.py's own HTTP client handles DNS
|
||||
# internally so our guard isn't consulted for the attachment
|
||||
# path. (issue #6587)
|
||||
#
|
||||
# If ``att.read()`` is unavailable (unexpected object shape / test
|
||||
# stub) or the bot session fetch fails, we fall back to the existing
|
||||
# SSRF-gated URL downloaders. The fallback keeps defense-in-depth
|
||||
# against any future Discord payload-schema drift that could slip a
|
||||
# non-CDN URL into the ``att.url`` field. (issue #11345)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _read_attachment_bytes(self, att) -> Optional[bytes]:
|
||||
"""Read an attachment via discord.py's authenticated bot session.
|
||||
|
||||
Returns the raw bytes on success, or ``None`` if ``att`` doesn't
|
||||
expose a callable ``read()`` or the read itself fails. Callers
|
||||
should treat ``None`` as a signal to fall back to the URL-based
|
||||
downloaders.
|
||||
"""
|
||||
reader = getattr(att, "read", None)
|
||||
if reader is None or not callable(reader):
|
||||
return None
|
||||
try:
|
||||
return await reader()
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"[Discord] Authenticated attachment read failed for %s: %s",
|
||||
getattr(att, "filename", None) or getattr(att, "url", "<unknown>"),
|
||||
e,
|
||||
)
|
||||
return None
|
||||
|
||||
async def _cache_discord_image(self, att, ext: str) -> str:
|
||||
"""Cache a Discord image attachment to local disk.
|
||||
|
||||
Primary path: ``att.read()`` + ``cache_image_from_bytes``
|
||||
(authenticated, no SSRF gate).
|
||||
|
||||
Fallback: ``cache_image_from_url`` (plain httpx, SSRF-gated).
|
||||
"""
|
||||
raw_bytes = await self._read_attachment_bytes(att)
|
||||
if raw_bytes is not None:
|
||||
try:
|
||||
return cache_image_from_bytes(raw_bytes, ext=ext)
|
||||
except Exception as e:
|
||||
logger.debug(
|
||||
"[Discord] cache_image_from_bytes rejected att.read() data; falling back to URL: %s",
|
||||
e,
|
||||
)
|
||||
return await cache_image_from_url(att.url, ext=ext)
|
||||
|
||||
async def _cache_discord_audio(self, att, ext: str) -> str:
|
||||
"""Cache a Discord audio attachment to local disk.
|
||||
|
||||
Primary path: ``att.read()`` + ``cache_audio_from_bytes``
|
||||
(authenticated, no SSRF gate).
|
||||
|
||||
Fallback: ``cache_audio_from_url`` (plain httpx, SSRF-gated).
|
||||
"""
|
||||
raw_bytes = await self._read_attachment_bytes(att)
|
||||
if raw_bytes is not None:
|
||||
try:
|
||||
return cache_audio_from_bytes(raw_bytes, ext=ext)
|
||||
except Exception as e:
|
||||
logger.debug(
|
||||
"[Discord] cache_audio_from_bytes failed; falling back to URL: %s",
|
||||
e,
|
||||
)
|
||||
return await cache_audio_from_url(att.url, ext=ext)
|
||||
|
||||
async def _cache_discord_document(self, att, ext: str) -> bytes:
|
||||
"""Download a Discord document attachment and return the raw bytes.
|
||||
|
||||
Primary path: ``att.read()`` (authenticated, no SSRF gate).
|
||||
|
||||
Fallback: SSRF-gated ``aiohttp`` download. This closes the gap
|
||||
where the old document path made raw ``aiohttp.ClientSession``
|
||||
requests with no safety check (#11345). The caller is responsible
|
||||
for passing the returned bytes to ``cache_document_from_bytes``
|
||||
(and, where applicable, for injecting text content).
|
||||
"""
|
||||
raw_bytes = await self._read_attachment_bytes(att)
|
||||
if raw_bytes is not None:
|
||||
return raw_bytes
|
||||
|
||||
# Fallback: SSRF-gated URL download.
|
||||
if not is_safe_url(att.url):
|
||||
raise ValueError(
|
||||
f"Blocked unsafe attachment URL (SSRF protection): {att.url}"
|
||||
)
|
||||
import aiohttp
|
||||
from gateway.platforms.base import resolve_proxy_url, proxy_kwargs_for_aiohttp
|
||||
_proxy = resolve_proxy_url(platform_env_var="DISCORD_PROXY")
|
||||
_sess_kw, _req_kw = proxy_kwargs_for_aiohttp(_proxy)
|
||||
async with aiohttp.ClientSession(**_sess_kw) as session:
|
||||
async with session.get(
|
||||
att.url,
|
||||
timeout=aiohttp.ClientTimeout(total=30),
|
||||
**_req_kw,
|
||||
) as resp:
|
||||
if resp.status != 200:
|
||||
raise Exception(f"HTTP {resp.status}")
|
||||
return await resp.read()
|
||||
|
||||
async def _handle_message(self, message: DiscordMessage) -> None:
|
||||
"""Handle incoming Discord messages."""
|
||||
# In server channels (not DMs), require the bot to be @mentioned
|
||||
@@ -2968,11 +2482,12 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
logger.debug("[%s] Ignoring message in ignored channel: %s", self.name, channel_ids)
|
||||
return
|
||||
|
||||
free_channels = self._discord_free_response_channels()
|
||||
free_channels_raw = os.getenv("DISCORD_FREE_RESPONSE_CHANNELS", "")
|
||||
free_channels = {ch.strip() for ch in free_channels_raw.split(",") if ch.strip()}
|
||||
if parent_channel_id:
|
||||
channel_ids.add(parent_channel_id)
|
||||
|
||||
require_mention = self._discord_require_mention()
|
||||
require_mention = os.getenv("DISCORD_REQUIRE_MENTION", "true").lower() not in ("false", "0", "no")
|
||||
# Voice-linked text channels act as free-response while voice is active.
|
||||
# Only the exact bound channel gets the exemption, not sibling threads.
|
||||
voice_linked_ids = {str(ch_id) for ch_id in self._voice_text_channels.values()}
|
||||
@@ -3000,10 +2515,9 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
if not is_thread and not isinstance(message.channel, discord.DMChannel):
|
||||
no_thread_channels_raw = os.getenv("DISCORD_NO_THREAD_CHANNELS", "")
|
||||
no_thread_channels = {ch.strip() for ch in no_thread_channels_raw.split(",") if ch.strip()}
|
||||
skip_thread = bool(channel_ids & no_thread_channels) or is_free_channel
|
||||
skip_thread = bool(channel_ids & no_thread_channels)
|
||||
auto_thread = os.getenv("DISCORD_AUTO_THREAD", "true").lower() in ("true", "1", "yes")
|
||||
is_reply_message = getattr(message, "type", None) == discord.MessageType.reply
|
||||
if auto_thread and not skip_thread and not is_voice_linked_channel and not is_reply_message:
|
||||
if auto_thread and not skip_thread and not is_voice_linked_channel:
|
||||
thread = await self._auto_create_thread(message)
|
||||
if thread:
|
||||
is_thread = True
|
||||
@@ -3064,7 +2578,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
user_name=message.author.display_name,
|
||||
thread_id=thread_id,
|
||||
chat_topic=chat_topic,
|
||||
is_bot=getattr(message.author, "bot", False),
|
||||
)
|
||||
|
||||
# Build media URLs -- download image attachments to local cache so the
|
||||
@@ -3080,7 +2593,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
ext = "." + content_type.split("/")[-1].split(";")[0]
|
||||
if ext not in (".jpg", ".jpeg", ".png", ".gif", ".webp"):
|
||||
ext = ".jpg"
|
||||
cached_path = await self._cache_discord_image(att, ext)
|
||||
cached_path = await cache_image_from_url(att.url, ext=ext)
|
||||
media_urls.append(cached_path)
|
||||
media_types.append(content_type)
|
||||
print(f"[Discord] Cached user image: {cached_path}", flush=True)
|
||||
@@ -3094,7 +2607,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
ext = "." + content_type.split("/")[-1].split(";")[0]
|
||||
if ext not in (".ogg", ".mp3", ".wav", ".webm", ".m4a"):
|
||||
ext = ".ogg"
|
||||
cached_path = await self._cache_discord_audio(att, ext)
|
||||
cached_path = await cache_audio_from_url(att.url, ext=ext)
|
||||
media_urls.append(cached_path)
|
||||
media_types.append(content_type)
|
||||
print(f"[Discord] Cached user audio: {cached_path}", flush=True)
|
||||
@@ -3125,7 +2638,19 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
)
|
||||
else:
|
||||
try:
|
||||
raw_bytes = await self._cache_discord_document(att, ext)
|
||||
import aiohttp
|
||||
from gateway.platforms.base import resolve_proxy_url, proxy_kwargs_for_aiohttp
|
||||
_proxy = resolve_proxy_url(platform_env_var="DISCORD_PROXY")
|
||||
_sess_kw, _req_kw = proxy_kwargs_for_aiohttp(_proxy)
|
||||
async with aiohttp.ClientSession(**_sess_kw) as session:
|
||||
async with session.get(
|
||||
att.url,
|
||||
timeout=aiohttp.ClientTimeout(total=30),
|
||||
**_req_kw,
|
||||
) as resp:
|
||||
if resp.status != 200:
|
||||
raise Exception(f"HTTP {resp.status}")
|
||||
raw_bytes = await resp.read()
|
||||
cached_path = cache_document_from_bytes(
|
||||
raw_bytes, att.filename or f"document{ext}"
|
||||
)
|
||||
@@ -3265,20 +2790,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
"[Discord] Flushing text batch %s (%d chars)",
|
||||
key, len(event.text or ""),
|
||||
)
|
||||
# Shield the downstream dispatch so that a subsequent chunk
|
||||
# arriving while handle_message is mid-flight cannot cancel
|
||||
# the running agent turn. _enqueue_text_event always cancels
|
||||
# the prior flush task when a new chunk lands; without this
|
||||
# shield, CancelledError would propagate from our task down
|
||||
# into handle_message → the agent's streaming request,
|
||||
# aborting the response the user was waiting on. The new
|
||||
# chunk is handled by the fresh flush task regardless.
|
||||
await asyncio.shield(self.handle_message(event))
|
||||
except asyncio.CancelledError:
|
||||
# Only reached if cancel landed before the pop — the shielded
|
||||
# handle_message is unaffected either way. Let the task exit
|
||||
# cleanly so the finally block cleans up.
|
||||
pass
|
||||
await self.handle_message(event)
|
||||
finally:
|
||||
if self._pending_text_batch_tasks.get(key) is current_task:
|
||||
self._pending_text_batch_tasks.pop(key, None)
|
||||
|
||||
@@ -1228,10 +1228,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
.register_p2_im_chat_member_bot_deleted_v1(self._on_bot_removed_from_chat)
|
||||
.register_p2_im_chat_access_event_bot_p2p_chat_entered_v1(self._on_p2p_chat_entered)
|
||||
.register_p2_im_message_recalled_v1(self._on_message_recalled)
|
||||
.register_p2_customized_event(
|
||||
"drive.notice.comment_add_v1",
|
||||
self._on_drive_comment_event,
|
||||
)
|
||||
.build()
|
||||
)
|
||||
|
||||
@@ -1969,25 +1965,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
def _on_message_recalled(self, data: Any) -> None:
|
||||
logger.debug("[Feishu] Message recalled by user")
|
||||
|
||||
def _on_drive_comment_event(self, data: Any) -> None:
|
||||
"""Handle drive document comment notification (drive.notice.comment_add_v1).
|
||||
|
||||
Delegates to :mod:`gateway.platforms.feishu_comment` for parsing,
|
||||
logging, and reaction. Scheduling follows the same
|
||||
``run_coroutine_threadsafe`` pattern used by ``_on_message_event``.
|
||||
"""
|
||||
from gateway.platforms.feishu_comment import handle_drive_comment_event
|
||||
|
||||
loop = self._loop
|
||||
if not self._loop_accepts_callbacks(loop):
|
||||
logger.warning("[Feishu] Dropping drive comment event before adapter loop is ready")
|
||||
return
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
handle_drive_comment_event(self._client, data, self_open_id=self._bot_open_id),
|
||||
loop,
|
||||
)
|
||||
future.add_done_callback(self._log_background_failure)
|
||||
|
||||
def _on_reaction_event(self, event_type: str, data: Any) -> None:
|
||||
"""Route user reactions on bot messages as synthetic text events."""
|
||||
event = getattr(data, "event", None)
|
||||
@@ -2613,8 +2590,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
self._on_reaction_event(event_type, data)
|
||||
elif event_type == "card.action.trigger":
|
||||
self._on_card_action_trigger(data)
|
||||
elif event_type == "drive.notice.comment_add_v1":
|
||||
self._on_drive_comment_event(data)
|
||||
else:
|
||||
logger.debug("[Feishu] Ignoring webhook event type: %s", event_type or "unknown")
|
||||
return web.json_response({"code": 0, "msg": "ok"})
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,429 +0,0 @@
|
||||
"""
|
||||
Feishu document comment access-control rules.
|
||||
|
||||
3-tier rule resolution: exact doc > wildcard "*" > top-level > code defaults.
|
||||
Each field (enabled/policy/allow_from) falls back independently.
|
||||
Config: ~/.hermes/feishu_comment_rules.json (mtime-cached, hot-reload).
|
||||
Pairing store: ~/.hermes/feishu_comment_pairing.json.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Paths
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# Uses the canonical ``get_hermes_home()`` helper (HERMES_HOME-aware and
|
||||
# profile-safe). Resolved at import time; this module is lazy-imported by
|
||||
# the Feishu comment event handler, which runs long after profile overrides
|
||||
# have been applied, so freezing paths here is safe.
|
||||
|
||||
RULES_FILE = get_hermes_home() / "feishu_comment_rules.json"
|
||||
PAIRING_FILE = get_hermes_home() / "feishu_comment_pairing.json"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Data models
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_VALID_POLICIES = ("allowlist", "pairing")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CommentDocumentRule:
|
||||
"""Per-document rule. ``None`` means 'inherit from lower tier'."""
|
||||
enabled: Optional[bool] = None
|
||||
policy: Optional[str] = None
|
||||
allow_from: Optional[frozenset] = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CommentsConfig:
|
||||
"""Top-level comment access config."""
|
||||
enabled: bool = True
|
||||
policy: str = "pairing"
|
||||
allow_from: frozenset = field(default_factory=frozenset)
|
||||
documents: Dict[str, CommentDocumentRule] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ResolvedCommentRule:
|
||||
"""Fully resolved rule after field-by-field fallback."""
|
||||
enabled: bool
|
||||
policy: str
|
||||
allow_from: frozenset
|
||||
match_source: str # e.g. "exact:docx:xxx" | "wildcard" | "top" | "default"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Mtime-cached file loading
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _MtimeCache:
|
||||
"""Generic mtime-based file cache. ``stat()`` per access, re-read only on change."""
|
||||
|
||||
def __init__(self, path: Path):
|
||||
self._path = path
|
||||
self._mtime: float = 0.0
|
||||
self._data: Optional[dict] = None
|
||||
|
||||
def load(self) -> dict:
|
||||
try:
|
||||
st = self._path.stat()
|
||||
mtime = st.st_mtime
|
||||
except FileNotFoundError:
|
||||
self._mtime = 0.0
|
||||
self._data = {}
|
||||
return {}
|
||||
|
||||
if mtime == self._mtime and self._data is not None:
|
||||
return self._data
|
||||
|
||||
try:
|
||||
with open(self._path, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, dict):
|
||||
data = {}
|
||||
except (json.JSONDecodeError, OSError):
|
||||
logger.warning("[Feishu-Rules] Failed to read %s, using empty config", self._path)
|
||||
data = {}
|
||||
|
||||
self._mtime = mtime
|
||||
self._data = data
|
||||
return data
|
||||
|
||||
|
||||
_rules_cache = _MtimeCache(RULES_FILE)
|
||||
_pairing_cache = _MtimeCache(PAIRING_FILE)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _parse_frozenset(raw: Any) -> Optional[frozenset]:
|
||||
"""Parse a list of strings into a frozenset; return None if key absent."""
|
||||
if raw is None:
|
||||
return None
|
||||
if isinstance(raw, (list, tuple)):
|
||||
return frozenset(str(u).strip() for u in raw if str(u).strip())
|
||||
return None
|
||||
|
||||
|
||||
def _parse_document_rule(raw: dict) -> CommentDocumentRule:
|
||||
enabled = raw.get("enabled")
|
||||
if enabled is not None:
|
||||
enabled = bool(enabled)
|
||||
policy = raw.get("policy")
|
||||
if policy is not None:
|
||||
policy = str(policy).strip().lower()
|
||||
if policy not in _VALID_POLICIES:
|
||||
policy = None
|
||||
allow_from = _parse_frozenset(raw.get("allow_from"))
|
||||
return CommentDocumentRule(enabled=enabled, policy=policy, allow_from=allow_from)
|
||||
|
||||
|
||||
def load_config() -> CommentsConfig:
|
||||
"""Load comment rules from disk (mtime-cached)."""
|
||||
raw = _rules_cache.load()
|
||||
if not raw:
|
||||
return CommentsConfig()
|
||||
|
||||
documents: Dict[str, CommentDocumentRule] = {}
|
||||
raw_docs = raw.get("documents", {})
|
||||
if isinstance(raw_docs, dict):
|
||||
for key, rule_raw in raw_docs.items():
|
||||
if isinstance(rule_raw, dict):
|
||||
documents[str(key)] = _parse_document_rule(rule_raw)
|
||||
|
||||
policy = str(raw.get("policy", "pairing")).strip().lower()
|
||||
if policy not in _VALID_POLICIES:
|
||||
policy = "pairing"
|
||||
|
||||
return CommentsConfig(
|
||||
enabled=raw.get("enabled", True),
|
||||
policy=policy,
|
||||
allow_from=_parse_frozenset(raw.get("allow_from")) or frozenset(),
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Rule resolution (§8.4 field-by-field fallback)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def has_wiki_keys(cfg: CommentsConfig) -> bool:
|
||||
"""Check if any document rule key starts with 'wiki:'."""
|
||||
return any(k.startswith("wiki:") for k in cfg.documents)
|
||||
|
||||
|
||||
def resolve_rule(
|
||||
cfg: CommentsConfig,
|
||||
file_type: str,
|
||||
file_token: str,
|
||||
wiki_token: str = "",
|
||||
) -> ResolvedCommentRule:
|
||||
"""Resolve effective rule: exact doc → wiki key → wildcard → top-level → defaults."""
|
||||
exact_key = f"{file_type}:{file_token}"
|
||||
|
||||
exact = cfg.documents.get(exact_key)
|
||||
exact_src = f"exact:{exact_key}"
|
||||
if exact is None and wiki_token:
|
||||
wiki_key = f"wiki:{wiki_token}"
|
||||
exact = cfg.documents.get(wiki_key)
|
||||
exact_src = f"exact:{wiki_key}"
|
||||
|
||||
wildcard = cfg.documents.get("*")
|
||||
|
||||
layers = []
|
||||
if exact is not None:
|
||||
layers.append((exact, exact_src))
|
||||
if wildcard is not None:
|
||||
layers.append((wildcard, "wildcard"))
|
||||
|
||||
def _pick(field_name: str):
|
||||
for layer, source in layers:
|
||||
val = getattr(layer, field_name)
|
||||
if val is not None:
|
||||
return val, source
|
||||
return getattr(cfg, field_name), "top"
|
||||
|
||||
enabled, en_src = _pick("enabled")
|
||||
policy, pol_src = _pick("policy")
|
||||
allow_from, _ = _pick("allow_from")
|
||||
|
||||
# match_source = highest-priority tier that contributed any field
|
||||
priority_order = {"exact": 0, "wildcard": 1, "top": 2}
|
||||
best_src = min(
|
||||
[en_src, pol_src],
|
||||
key=lambda s: priority_order.get(s.split(":")[0], 3),
|
||||
)
|
||||
|
||||
return ResolvedCommentRule(
|
||||
enabled=enabled,
|
||||
policy=policy,
|
||||
allow_from=allow_from,
|
||||
match_source=best_src,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pairing store
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _load_pairing_approved() -> set:
|
||||
"""Return set of approved user open_ids (mtime-cached)."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if isinstance(approved, dict):
|
||||
return set(approved.keys())
|
||||
if isinstance(approved, list):
|
||||
return set(str(u) for u in approved if u)
|
||||
return set()
|
||||
|
||||
|
||||
def _save_pairing(data: dict) -> None:
|
||||
PAIRING_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = PAIRING_FILE.with_suffix(".tmp")
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
tmp.replace(PAIRING_FILE)
|
||||
# Invalidate cache so next load picks up change
|
||||
_pairing_cache._mtime = 0.0
|
||||
_pairing_cache._data = None
|
||||
|
||||
|
||||
def pairing_add(user_open_id: str) -> bool:
|
||||
"""Add a user to the pairing-approved list. Returns True if newly added."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if not isinstance(approved, dict):
|
||||
approved = {}
|
||||
if user_open_id in approved:
|
||||
return False
|
||||
approved[user_open_id] = {"approved_at": time.time()}
|
||||
data["approved"] = approved
|
||||
_save_pairing(data)
|
||||
return True
|
||||
|
||||
|
||||
def pairing_remove(user_open_id: str) -> bool:
|
||||
"""Remove a user from the pairing-approved list. Returns True if removed."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if not isinstance(approved, dict):
|
||||
return False
|
||||
if user_open_id not in approved:
|
||||
return False
|
||||
del approved[user_open_id]
|
||||
data["approved"] = approved
|
||||
_save_pairing(data)
|
||||
return True
|
||||
|
||||
|
||||
def pairing_list() -> Dict[str, Any]:
|
||||
"""Return the approved dict {user_open_id: {approved_at: ...}}."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
return dict(approved) if isinstance(approved, dict) else {}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Access check (public API for feishu_comment.py)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def is_user_allowed(rule: ResolvedCommentRule, user_open_id: str) -> bool:
|
||||
"""Check if user passes the resolved rule's policy gate."""
|
||||
if user_open_id in rule.allow_from:
|
||||
return True
|
||||
if rule.policy == "pairing":
|
||||
return user_open_id in _load_pairing_approved()
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _print_status() -> None:
|
||||
cfg = load_config()
|
||||
print(f"Rules file: {RULES_FILE}")
|
||||
print(f" exists: {RULES_FILE.exists()}")
|
||||
print(f"Pairing file: {PAIRING_FILE}")
|
||||
print(f" exists: {PAIRING_FILE.exists()}")
|
||||
print()
|
||||
print(f"Top-level:")
|
||||
print(f" enabled: {cfg.enabled}")
|
||||
print(f" policy: {cfg.policy}")
|
||||
print(f" allow_from: {sorted(cfg.allow_from) if cfg.allow_from else '[]'}")
|
||||
print()
|
||||
if cfg.documents:
|
||||
print(f"Document rules ({len(cfg.documents)}):")
|
||||
for key, rule in sorted(cfg.documents.items()):
|
||||
parts = []
|
||||
if rule.enabled is not None:
|
||||
parts.append(f"enabled={rule.enabled}")
|
||||
if rule.policy is not None:
|
||||
parts.append(f"policy={rule.policy}")
|
||||
if rule.allow_from is not None:
|
||||
parts.append(f"allow_from={sorted(rule.allow_from)}")
|
||||
print(f" [{key}] {', '.join(parts) if parts else '(empty — inherits all)'}")
|
||||
else:
|
||||
print("Document rules: (none)")
|
||||
print()
|
||||
approved = pairing_list()
|
||||
print(f"Pairing approved ({len(approved)}):")
|
||||
for uid, meta in sorted(approved.items()):
|
||||
ts = meta.get("approved_at", 0)
|
||||
print(f" {uid} (approved_at={ts})")
|
||||
|
||||
|
||||
def _do_check(doc_key: str, user_open_id: str) -> None:
|
||||
cfg = load_config()
|
||||
parts = doc_key.split(":", 1)
|
||||
if len(parts) != 2:
|
||||
print(f"Error: doc_key must be 'fileType:fileToken', got '{doc_key}'")
|
||||
return
|
||||
file_type, file_token = parts
|
||||
rule = resolve_rule(cfg, file_type, file_token)
|
||||
allowed = is_user_allowed(rule, user_open_id)
|
||||
print(f"Document: {doc_key}")
|
||||
print(f"User: {user_open_id}")
|
||||
print(f"Resolved rule:")
|
||||
print(f" enabled: {rule.enabled}")
|
||||
print(f" policy: {rule.policy}")
|
||||
print(f" allow_from: {sorted(rule.allow_from) if rule.allow_from else '[]'}")
|
||||
print(f" match_source: {rule.match_source}")
|
||||
print(f"Result: {'ALLOWED' if allowed else 'DENIED'}")
|
||||
|
||||
|
||||
def _main() -> int:
|
||||
import sys
|
||||
|
||||
try:
|
||||
from hermes_cli.env_loader import load_hermes_dotenv
|
||||
load_hermes_dotenv()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
usage = (
|
||||
"Usage: python -m gateway.platforms.feishu_comment_rules <command> [args]\n"
|
||||
"\n"
|
||||
"Commands:\n"
|
||||
" status Show rules config and pairing state\n"
|
||||
" check <fileType:token> <user> Simulate access check\n"
|
||||
" pairing add <user_open_id> Add user to pairing-approved list\n"
|
||||
" pairing remove <user_open_id> Remove user from pairing-approved list\n"
|
||||
" pairing list List pairing-approved users\n"
|
||||
"\n"
|
||||
f"Rules config file: {RULES_FILE}\n"
|
||||
" Edit this JSON file directly to configure policies and document rules.\n"
|
||||
" Changes take effect on the next comment event (no restart needed).\n"
|
||||
)
|
||||
|
||||
args = sys.argv[1:]
|
||||
if not args:
|
||||
print(usage)
|
||||
return 1
|
||||
|
||||
cmd = args[0]
|
||||
|
||||
if cmd == "status":
|
||||
_print_status()
|
||||
|
||||
elif cmd == "check":
|
||||
if len(args) < 3:
|
||||
print("Usage: check <fileType:fileToken> <user_open_id>")
|
||||
return 1
|
||||
_do_check(args[1], args[2])
|
||||
|
||||
elif cmd == "pairing":
|
||||
if len(args) < 2:
|
||||
print("Usage: pairing <add|remove|list> [args]")
|
||||
return 1
|
||||
sub = args[1]
|
||||
if sub == "add":
|
||||
if len(args) < 3:
|
||||
print("Usage: pairing add <user_open_id>")
|
||||
return 1
|
||||
if pairing_add(args[2]):
|
||||
print(f"Added: {args[2]}")
|
||||
else:
|
||||
print(f"Already approved: {args[2]}")
|
||||
elif sub == "remove":
|
||||
if len(args) < 3:
|
||||
print("Usage: pairing remove <user_open_id>")
|
||||
return 1
|
||||
if pairing_remove(args[2]):
|
||||
print(f"Removed: {args[2]}")
|
||||
else:
|
||||
print(f"Not in approved list: {args[2]}")
|
||||
elif sub == "list":
|
||||
approved = pairing_list()
|
||||
if not approved:
|
||||
print("(no approved users)")
|
||||
for uid, meta in sorted(approved.items()):
|
||||
print(f" {uid} approved_at={meta.get('approved_at', '?')}")
|
||||
else:
|
||||
print(f"Unknown pairing subcommand: {sub}")
|
||||
return 1
|
||||
else:
|
||||
print(f"Unknown command: {cmd}\n")
|
||||
print(usage)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
sys.exit(_main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,57 +0,0 @@
|
||||
"""
|
||||
QQBot platform package.
|
||||
|
||||
Re-exports the main adapter symbols from ``adapter.py`` (the original
|
||||
``qqbot.py``) so that **all existing import paths remain unchanged**::
|
||||
|
||||
from gateway.platforms.qqbot import QQAdapter # works
|
||||
from gateway.platforms.qqbot import check_qq_requirements # works
|
||||
|
||||
New modules:
|
||||
- ``constants`` — shared constants (API URLs, timeouts, message types)
|
||||
- ``utils`` — User-Agent builder, config helpers
|
||||
- ``crypto`` — AES-256-GCM key generation and decryption
|
||||
- ``onboard`` — QR-code scan-to-configure flow
|
||||
"""
|
||||
|
||||
# -- Adapter (original qqbot.py) ------------------------------------------
|
||||
from .adapter import ( # noqa: F401
|
||||
QQAdapter,
|
||||
QQCloseError,
|
||||
check_qq_requirements,
|
||||
_coerce_list,
|
||||
_ssrf_redirect_guard,
|
||||
)
|
||||
|
||||
# -- Onboard (QR-code scan-to-configure) -----------------------------------
|
||||
from .onboard import ( # noqa: F401
|
||||
BindStatus,
|
||||
create_bind_task,
|
||||
poll_bind_result,
|
||||
build_connect_url,
|
||||
)
|
||||
from .crypto import decrypt_secret, generate_bind_key # noqa: F401
|
||||
|
||||
# -- Utils -----------------------------------------------------------------
|
||||
from .utils import build_user_agent, get_api_headers, coerce_list # noqa: F401
|
||||
|
||||
__all__ = [
|
||||
# adapter
|
||||
"QQAdapter",
|
||||
"QQCloseError",
|
||||
"check_qq_requirements",
|
||||
"_coerce_list",
|
||||
"_ssrf_redirect_guard",
|
||||
# onboard
|
||||
"BindStatus",
|
||||
"create_bind_task",
|
||||
"poll_bind_result",
|
||||
"build_connect_url",
|
||||
# crypto
|
||||
"decrypt_secret",
|
||||
"generate_bind_key",
|
||||
# utils
|
||||
"build_user_agent",
|
||||
"get_api_headers",
|
||||
"coerce_list",
|
||||
]
|
||||
@@ -1,74 +0,0 @@
|
||||
"""QQBot package-level constants shared across adapter, onboard, and other modules."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQBot adapter version — bump on functional changes to the adapter package.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
QQBOT_VERSION = "1.1.0"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# API endpoints
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# The portal domain is configurable via QQ_API_HOST for corporate proxies
|
||||
# or test environments. Default: q.qq.com (production).
|
||||
PORTAL_HOST = os.getenv("QQ_PORTAL_HOST", "q.qq.com")
|
||||
|
||||
API_BASE = "https://api.sgroup.qq.com"
|
||||
TOKEN_URL = "https://bots.qq.com/app/getAppAccessToken"
|
||||
GATEWAY_URL_PATH = "/gateway"
|
||||
|
||||
# QR-code onboard endpoints (on the portal host)
|
||||
ONBOARD_CREATE_PATH = "/lite/create_bind_task"
|
||||
ONBOARD_POLL_PATH = "/lite/poll_bind_result"
|
||||
QR_URL_TEMPLATE = (
|
||||
"https://q.qq.com/qqbot/openclaw/connect.html"
|
||||
"?task_id={task_id}&_wv=2&source=hermes"
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Timeouts & retry
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
DEFAULT_API_TIMEOUT = 30.0
|
||||
FILE_UPLOAD_TIMEOUT = 120.0
|
||||
CONNECT_TIMEOUT_SECONDS = 20.0
|
||||
|
||||
RECONNECT_BACKOFF = [2, 5, 10, 30, 60]
|
||||
MAX_RECONNECT_ATTEMPTS = 100
|
||||
RATE_LIMIT_DELAY = 60 # seconds
|
||||
QUICK_DISCONNECT_THRESHOLD = 5.0 # seconds
|
||||
MAX_QUICK_DISCONNECT_COUNT = 3
|
||||
|
||||
ONBOARD_POLL_INTERVAL = 2.0 # seconds between poll_bind_result calls
|
||||
ONBOARD_API_TIMEOUT = 10.0
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Message limits
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MAX_MESSAGE_LENGTH = 4000
|
||||
DEDUP_WINDOW_SECONDS = 300
|
||||
DEDUP_MAX_SIZE = 1000
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQ Bot message types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MSG_TYPE_TEXT = 0
|
||||
MSG_TYPE_MARKDOWN = 2
|
||||
MSG_TYPE_MEDIA = 7
|
||||
MSG_TYPE_INPUT_NOTIFY = 6
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQ Bot file media types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MEDIA_TYPE_IMAGE = 1
|
||||
MEDIA_TYPE_VIDEO = 2
|
||||
MEDIA_TYPE_VOICE = 3
|
||||
MEDIA_TYPE_FILE = 4
|
||||
@@ -1,45 +0,0 @@
|
||||
"""AES-256-GCM utilities for QQBot scan-to-configure credential decryption."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import os
|
||||
|
||||
|
||||
def generate_bind_key() -> str:
|
||||
"""Generate a 256-bit random AES key and return it as base64.
|
||||
|
||||
The key is passed to ``create_bind_task`` so the server can encrypt
|
||||
the bot's *client_secret* before returning it. Only this CLI holds
|
||||
the key, ensuring the secret never travels in plaintext.
|
||||
"""
|
||||
return base64.b64encode(os.urandom(32)).decode()
|
||||
|
||||
|
||||
def decrypt_secret(encrypted_base64: str, key_base64: str) -> str:
|
||||
"""Decrypt a base64-encoded AES-256-GCM ciphertext.
|
||||
|
||||
Ciphertext layout (after base64-decoding)::
|
||||
|
||||
IV (12 bytes) ‖ ciphertext (N bytes) ‖ AuthTag (16 bytes)
|
||||
|
||||
Args:
|
||||
encrypted_base64: The ``bot_encrypt_secret`` value from
|
||||
``poll_bind_result``.
|
||||
key_base64: The base64 AES key generated by
|
||||
:func:`generate_bind_key`.
|
||||
|
||||
Returns:
|
||||
The decrypted *client_secret* as a UTF-8 string.
|
||||
"""
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
|
||||
key = base64.b64decode(key_base64)
|
||||
raw = base64.b64decode(encrypted_base64)
|
||||
|
||||
iv = raw[:12]
|
||||
ciphertext_with_tag = raw[12:] # AESGCM expects ciphertext + tag concatenated
|
||||
|
||||
aesgcm = AESGCM(key)
|
||||
plaintext = aesgcm.decrypt(iv, ciphertext_with_tag, None)
|
||||
return plaintext.decode("utf-8")
|
||||
@@ -1,124 +0,0 @@
|
||||
"""
|
||||
QQBot scan-to-configure (QR code onboard) module.
|
||||
|
||||
Calls the ``q.qq.com`` ``create_bind_task`` / ``poll_bind_result`` APIs to
|
||||
generate a QR-code URL and poll for scan completion. On success the caller
|
||||
receives the bot's *app_id*, *client_secret* (decrypted locally), and the
|
||||
scanner's *user_openid* — enough to fully configure the QQBot gateway.
|
||||
|
||||
Reference: https://bot.q.qq.com/wiki/develop/api-v2/
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from enum import IntEnum
|
||||
from typing import Tuple
|
||||
from urllib.parse import quote
|
||||
|
||||
from .constants import (
|
||||
ONBOARD_API_TIMEOUT,
|
||||
ONBOARD_CREATE_PATH,
|
||||
ONBOARD_POLL_PATH,
|
||||
PORTAL_HOST,
|
||||
QR_URL_TEMPLATE,
|
||||
)
|
||||
from .crypto import generate_bind_key
|
||||
from .utils import get_api_headers
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bind status
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class BindStatus(IntEnum):
|
||||
"""Status codes returned by ``poll_bind_result``."""
|
||||
|
||||
NONE = 0
|
||||
PENDING = 1
|
||||
COMPLETED = 2
|
||||
EXPIRED = 3
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def create_bind_task(
|
||||
timeout: float = ONBOARD_API_TIMEOUT,
|
||||
) -> Tuple[str, str]:
|
||||
"""Create a bind task and return *(task_id, aes_key_base64)*.
|
||||
|
||||
The AES key is generated locally and sent to the server so it can
|
||||
encrypt the bot credentials before returning them.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the API returns a non-zero ``retcode``.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
url = f"https://{PORTAL_HOST}{ONBOARD_CREATE_PATH}"
|
||||
key = generate_bind_key()
|
||||
|
||||
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
|
||||
resp = await client.post(url, json={"key": key}, headers=get_api_headers())
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
if data.get("retcode") != 0:
|
||||
raise RuntimeError(data.get("msg", "create_bind_task failed"))
|
||||
|
||||
task_id = data.get("data", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise RuntimeError("create_bind_task: missing task_id in response")
|
||||
|
||||
logger.debug("create_bind_task ok: task_id=%s", task_id)
|
||||
return task_id, key
|
||||
|
||||
|
||||
async def poll_bind_result(
|
||||
task_id: str,
|
||||
timeout: float = ONBOARD_API_TIMEOUT,
|
||||
) -> Tuple[BindStatus, str, str, str]:
|
||||
"""Poll the bind result for *task_id*.
|
||||
|
||||
Returns:
|
||||
A 4-tuple of ``(status, bot_appid, bot_encrypt_secret, user_openid)``.
|
||||
|
||||
* ``bot_encrypt_secret`` is AES-256-GCM encrypted — decrypt it with
|
||||
:func:`~gateway.platforms.qqbot.crypto.decrypt_secret` using the
|
||||
key from :func:`create_bind_task`.
|
||||
* ``user_openid`` is the OpenID of the person who scanned the code
|
||||
(available when ``status == COMPLETED``).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the API returns a non-zero ``retcode``.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
url = f"https://{PORTAL_HOST}{ONBOARD_POLL_PATH}"
|
||||
|
||||
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
|
||||
resp = await client.post(url, json={"task_id": task_id}, headers=get_api_headers())
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
if data.get("retcode") != 0:
|
||||
raise RuntimeError(data.get("msg", "poll_bind_result failed"))
|
||||
|
||||
d = data.get("data", {})
|
||||
return (
|
||||
BindStatus(d.get("status", 0)),
|
||||
str(d.get("bot_appid", "")),
|
||||
d.get("bot_encrypt_secret", ""),
|
||||
d.get("user_openid", ""),
|
||||
)
|
||||
|
||||
|
||||
def build_connect_url(task_id: str) -> str:
|
||||
"""Build the QR-code target URL for a given *task_id*."""
|
||||
return QR_URL_TEMPLATE.format(task_id=quote(task_id))
|
||||
@@ -1,71 +0,0 @@
|
||||
"""QQBot shared utilities — User-Agent, HTTP helpers, config coercion."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import platform
|
||||
import sys
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from .constants import QQBOT_VERSION
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# User-Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _get_hermes_version() -> str:
|
||||
"""Return the hermes-agent package version, or 'dev' if unavailable."""
|
||||
try:
|
||||
from importlib.metadata import version
|
||||
return version("hermes-agent")
|
||||
except Exception:
|
||||
return "dev"
|
||||
|
||||
|
||||
def build_user_agent() -> str:
|
||||
"""Build a descriptive User-Agent string.
|
||||
|
||||
Format::
|
||||
|
||||
QQBotAdapter/<qqbot_version> (Python/<py_version>; <os>; Hermes/<hermes_version>)
|
||||
|
||||
Example::
|
||||
|
||||
QQBotAdapter/1.0.0 (Python/3.11.15; darwin; Hermes/0.9.0)
|
||||
"""
|
||||
py_version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
|
||||
os_name = platform.system().lower()
|
||||
hermes_version = _get_hermes_version()
|
||||
return f"QQBotAdapter/{QQBOT_VERSION} (Python/{py_version}; {os_name}; Hermes/{hermes_version})"
|
||||
|
||||
|
||||
def get_api_headers() -> Dict[str, str]:
|
||||
"""Return standard HTTP headers for QQBot API requests.
|
||||
|
||||
Includes ``Content-Type``, ``Accept``, and a dynamic ``User-Agent``.
|
||||
``q.qq.com`` requires ``Accept: application/json`` — without it,
|
||||
the server returns a JavaScript anti-bot challenge page.
|
||||
"""
|
||||
return {
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json",
|
||||
"User-Agent": build_user_agent(),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def coerce_list(value: Any) -> List[str]:
|
||||
"""Coerce config values into a trimmed string list.
|
||||
|
||||
Accepts comma-separated strings, lists, tuples, sets, or single values.
|
||||
"""
|
||||
if value is None:
|
||||
return []
|
||||
if isinstance(value, str):
|
||||
return [item.strip() for item in value.split(",") if item.strip()]
|
||||
if isinstance(value, (list, tuple, set)):
|
||||
return [str(item).strip() for item in value if str(item).strip()]
|
||||
return [str(value).strip()] if str(value).strip() else []
|
||||
@@ -160,14 +160,6 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
self._sse_task: Optional[asyncio.Task] = None
|
||||
self._health_monitor_task: Optional[asyncio.Task] = None
|
||||
self._typing_tasks: Dict[str, asyncio.Task] = {}
|
||||
# Per-chat typing-indicator backoff. When signal-cli reports
|
||||
# NETWORK_FAILURE (recipient offline / unroutable), base.py's
|
||||
# _keep_typing refresh loop would otherwise hammer sendTyping every
|
||||
# ~2s indefinitely, producing WARNING-level log spam and pointless
|
||||
# RPC traffic. We track consecutive failures per chat and skip the
|
||||
# RPC during a cooldown window instead.
|
||||
self._typing_failures: Dict[str, int] = {}
|
||||
self._typing_skip_until: Dict[str, float] = {}
|
||||
self._running = False
|
||||
self._last_sse_activity = 0.0
|
||||
self._sse_response: Optional[httpx.Response] = None
|
||||
@@ -556,22 +548,8 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
# JSON-RPC Communication
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _rpc(
|
||||
self,
|
||||
method: str,
|
||||
params: dict,
|
||||
rpc_id: str = None,
|
||||
*,
|
||||
log_failures: bool = True,
|
||||
) -> Any:
|
||||
"""Send a JSON-RPC 2.0 request to signal-cli daemon.
|
||||
|
||||
When ``log_failures=False``, error and exception paths log at DEBUG
|
||||
instead of WARNING — used by the typing-indicator path to silence
|
||||
repeated NETWORK_FAILURE spam for unreachable recipients while
|
||||
still preserving visibility for the first occurrence and for
|
||||
unrelated RPCs.
|
||||
"""
|
||||
async def _rpc(self, method: str, params: dict, rpc_id: str = None) -> Any:
|
||||
"""Send a JSON-RPC 2.0 request to signal-cli daemon."""
|
||||
if not self.client:
|
||||
logger.warning("Signal: RPC called but client not connected")
|
||||
return None
|
||||
@@ -596,19 +574,13 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
data = resp.json()
|
||||
|
||||
if "error" in data:
|
||||
if log_failures:
|
||||
logger.warning("Signal RPC error (%s): %s", method, data["error"])
|
||||
else:
|
||||
logger.debug("Signal RPC error (%s): %s", method, data["error"])
|
||||
logger.warning("Signal RPC error (%s): %s", method, data["error"])
|
||||
return None
|
||||
|
||||
return data.get("result")
|
||||
|
||||
except Exception as e:
|
||||
if log_failures:
|
||||
logger.warning("Signal RPC %s failed: %s", method, e)
|
||||
else:
|
||||
logger.debug("Signal RPC %s failed: %s", method, e)
|
||||
logger.warning("Signal RPC %s failed: %s", method, e)
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -655,28 +627,7 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
self._recent_sent_timestamps.pop()
|
||||
|
||||
async def send_typing(self, chat_id: str, metadata=None) -> None:
|
||||
"""Send a typing indicator.
|
||||
|
||||
base.py's ``_keep_typing`` refresh loop calls this every ~2s while
|
||||
the agent is processing. If signal-cli returns NETWORK_FAILURE for
|
||||
this recipient (offline, unroutable, group membership lost, etc.)
|
||||
the unmitigated behaviour is: a WARNING log every 2 seconds for as
|
||||
long as the agent keeps running. Instead we:
|
||||
|
||||
- silence the WARNING after the first consecutive failure (subsequent
|
||||
attempts log at DEBUG) so transport issues are still visible once
|
||||
but don't flood the log,
|
||||
- skip the RPC entirely during an exponential cooldown window once
|
||||
three consecutive failures have happened, so we stop hammering
|
||||
signal-cli with requests it can't deliver.
|
||||
|
||||
A successful sendTyping clears the counters.
|
||||
"""
|
||||
now = time.monotonic()
|
||||
skip_until = self._typing_skip_until.get(chat_id, 0.0)
|
||||
if now < skip_until:
|
||||
return
|
||||
|
||||
"""Send a typing indicator."""
|
||||
params: Dict[str, Any] = {
|
||||
"account": self.account,
|
||||
}
|
||||
@@ -686,26 +637,7 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
else:
|
||||
params["recipient"] = [chat_id]
|
||||
|
||||
fails = self._typing_failures.get(chat_id, 0)
|
||||
result = await self._rpc(
|
||||
"sendTyping",
|
||||
params,
|
||||
rpc_id="typing",
|
||||
log_failures=(fails == 0),
|
||||
)
|
||||
|
||||
if result is None:
|
||||
fails += 1
|
||||
self._typing_failures[chat_id] = fails
|
||||
# After 3 consecutive failures, back off exponentially (16s,
|
||||
# 32s, 60s cap) to stop spamming signal-cli for a recipient
|
||||
# that clearly isn't reachable right now.
|
||||
if fails >= 3:
|
||||
backoff = min(60.0, 16.0 * (2 ** (fails - 3)))
|
||||
self._typing_skip_until[chat_id] = now + backoff
|
||||
else:
|
||||
self._typing_failures.pop(chat_id, None)
|
||||
self._typing_skip_until.pop(chat_id, None)
|
||||
await self._rpc("sendTyping", params, rpc_id="typing")
|
||||
|
||||
async def send_image(
|
||||
self,
|
||||
@@ -857,10 +789,6 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
# Reset per-chat typing backoff state so the next agent turn starts
|
||||
# fresh rather than inheriting a cooldown from a prior conversation.
|
||||
self._typing_failures.pop(chat_id, None)
|
||||
self._typing_skip_until.pop(chat_id, None)
|
||||
|
||||
async def stop_typing(self, chat_id: str) -> None:
|
||||
"""Public interface for stopping typing — called by base adapter's
|
||||
|
||||
@@ -118,84 +118,6 @@ def _strip_mdv2(text: str) -> str:
|
||||
return cleaned
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Markdown table → code block conversion
|
||||
# ---------------------------------------------------------------------------
|
||||
# Telegram's MarkdownV2 has no table syntax — '|' is just an escaped literal,
|
||||
# so pipe tables render as noisy backslash-pipe text with no alignment.
|
||||
# Wrapping the table in a fenced code block makes Telegram render it as
|
||||
# monospace preformatted text with columns intact.
|
||||
|
||||
# Matches a GFM table delimiter row: optional outer pipes, cells containing
|
||||
# only dashes (with optional leading/trailing colons for alignment) separated
|
||||
# by '|'. Requires at least one internal '|' so lone '---' horizontal rules
|
||||
# are NOT matched.
|
||||
_TABLE_SEPARATOR_RE = re.compile(
|
||||
r'^\s*\|?\s*:?-+:?\s*(?:\|\s*:?-+:?\s*){1,}\|?\s*$'
|
||||
)
|
||||
|
||||
|
||||
def _is_table_row(line: str) -> bool:
|
||||
"""Return True if *line* could plausibly be a table data row."""
|
||||
stripped = line.strip()
|
||||
return bool(stripped) and '|' in stripped
|
||||
|
||||
|
||||
def _wrap_markdown_tables(text: str) -> str:
|
||||
"""Wrap GFM-style pipe tables in ``` fences so Telegram renders them.
|
||||
|
||||
Detected by a row containing '|' immediately followed by a delimiter
|
||||
row matching :data:`_TABLE_SEPARATOR_RE`. Subsequent pipe-containing
|
||||
non-blank lines are consumed as the table body and included in the
|
||||
wrapped block. Tables inside existing fenced code blocks are left
|
||||
alone.
|
||||
"""
|
||||
if '|' not in text or '-' not in text:
|
||||
return text
|
||||
|
||||
lines = text.split('\n')
|
||||
out: list[str] = []
|
||||
in_fence = False
|
||||
i = 0
|
||||
while i < len(lines):
|
||||
line = lines[i]
|
||||
stripped = line.lstrip()
|
||||
|
||||
# Track existing fenced code blocks — never touch content inside.
|
||||
if stripped.startswith('```'):
|
||||
in_fence = not in_fence
|
||||
out.append(line)
|
||||
i += 1
|
||||
continue
|
||||
if in_fence:
|
||||
out.append(line)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# Look for a header row (contains '|') immediately followed by a
|
||||
# delimiter row.
|
||||
if (
|
||||
'|' in line
|
||||
and i + 1 < len(lines)
|
||||
and _TABLE_SEPARATOR_RE.match(lines[i + 1])
|
||||
):
|
||||
table_block = [line, lines[i + 1]]
|
||||
j = i + 2
|
||||
while j < len(lines) and _is_table_row(lines[j]):
|
||||
table_block.append(lines[j])
|
||||
j += 1
|
||||
out.append('```')
|
||||
out.extend(table_block)
|
||||
out.append('```')
|
||||
i = j
|
||||
continue
|
||||
|
||||
out.append(line)
|
||||
i += 1
|
||||
|
||||
return '\n'.join(out)
|
||||
|
||||
|
||||
class TelegramAdapter(BasePlatformAdapter):
|
||||
"""
|
||||
Telegram bot adapter.
|
||||
@@ -1994,12 +1916,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
|
||||
text = content
|
||||
|
||||
# 0) Pre-wrap GFM-style pipe tables in ``` fences. Telegram can't
|
||||
# render tables natively, but fenced code blocks render as
|
||||
# monospace preformatted text with columns intact. The wrapped
|
||||
# tables then flow through step (1) below as protected regions.
|
||||
text = _wrap_markdown_tables(text)
|
||||
|
||||
# 1) Protect fenced code blocks (``` ... ```)
|
||||
# Per MarkdownV2 spec, \ and ` inside pre/code must be escaped.
|
||||
def _protect_fenced(m):
|
||||
@@ -2326,7 +2242,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
if not self._should_process_message(update.message):
|
||||
return
|
||||
|
||||
event = self._build_message_event(update.message, MessageType.TEXT, update_id=update.update_id)
|
||||
event = self._build_message_event(update.message, MessageType.TEXT)
|
||||
event.text = self._clean_bot_trigger_text(event.text)
|
||||
self._enqueue_text_event(event)
|
||||
|
||||
@@ -2337,7 +2253,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
if not self._should_process_message(update.message, is_command=True):
|
||||
return
|
||||
|
||||
event = self._build_message_event(update.message, MessageType.COMMAND, update_id=update.update_id)
|
||||
event = self._build_message_event(update.message, MessageType.COMMAND)
|
||||
await self.handle_message(event)
|
||||
|
||||
async def _handle_location_message(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
||||
@@ -2373,7 +2289,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
parts.append(f"Map: https://www.google.com/maps/search/?api=1&query={lat},{lon}")
|
||||
parts.append("Ask what they'd like to find nearby (restaurants, cafes, etc.) and any preferences.")
|
||||
|
||||
event = self._build_message_event(msg, MessageType.LOCATION, update_id=update.update_id)
|
||||
event = self._build_message_event(msg, MessageType.LOCATION)
|
||||
event.text = "\n".join(parts)
|
||||
await self.handle_message(event)
|
||||
|
||||
@@ -2524,7 +2440,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
else:
|
||||
msg_type = MessageType.DOCUMENT
|
||||
|
||||
event = self._build_message_event(msg, msg_type, update_id=update.update_id)
|
||||
event = self._build_message_event(msg, msg_type)
|
||||
|
||||
# Add caption as text
|
||||
if msg.caption:
|
||||
@@ -2863,19 +2779,8 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
self.name, cache_key, thread_id,
|
||||
)
|
||||
|
||||
def _build_message_event(
|
||||
self,
|
||||
message: Message,
|
||||
msg_type: MessageType,
|
||||
update_id: Optional[int] = None,
|
||||
) -> MessageEvent:
|
||||
"""Build a MessageEvent from a Telegram message.
|
||||
|
||||
``update_id`` is the ``Update.update_id`` from PTB; passing it through
|
||||
lets ``/restart`` record the triggering offset so the new gateway
|
||||
process can advance past it (prevents ``/restart`` being re-delivered
|
||||
when PTB's graceful-shutdown ACK fails).
|
||||
"""
|
||||
def _build_message_event(self, message: Message, msg_type: MessageType) -> MessageEvent:
|
||||
"""Build a MessageEvent from a Telegram message."""
|
||||
chat = message.chat
|
||||
user = message.from_user
|
||||
|
||||
@@ -2926,8 +2831,8 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
chat_id=str(chat.id),
|
||||
chat_name=chat.title or (chat.full_name if hasattr(chat, "full_name") else None),
|
||||
chat_type=chat_type,
|
||||
user_id=str(user.id) if user else (str(chat.id) if chat_type == "dm" else None),
|
||||
user_name=user.full_name if user else (chat.full_name if hasattr(chat, "full_name") and chat_type == "dm" else None),
|
||||
user_id=str(user.id) if user else None,
|
||||
user_name=user.full_name if user else None,
|
||||
thread_id=thread_id_str,
|
||||
chat_topic=chat_topic,
|
||||
)
|
||||
@@ -2954,7 +2859,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
source=source,
|
||||
raw_message=message,
|
||||
message_id=str(message.message_id),
|
||||
platform_update_id=update_id,
|
||||
reply_to_message_id=reply_to_id,
|
||||
reply_to_text=reply_to_text,
|
||||
auto_skill=topic_skill,
|
||||
|
||||
+10
-41
@@ -180,8 +180,6 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
self._text_batch_split_delay_seconds = float(os.getenv("HERMES_WECOM_TEXT_BATCH_SPLIT_DELAY_SECONDS", "2.0"))
|
||||
self._pending_text_batches: Dict[str, MessageEvent] = {}
|
||||
self._pending_text_batch_tasks: Dict[str, asyncio.Task] = {}
|
||||
self._device_id = uuid.uuid4().hex
|
||||
self._last_chat_req_ids: Dict[str, str] = {}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Connection lifecycle
|
||||
@@ -279,11 +277,7 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
{
|
||||
"cmd": APP_CMD_SUBSCRIBE,
|
||||
"headers": {"req_id": req_id},
|
||||
"body": {
|
||||
"bot_id": self._bot_id,
|
||||
"secret": self._secret,
|
||||
"device_id": self._device_id,
|
||||
},
|
||||
"body": {"bot_id": self._bot_id, "secret": self._secret},
|
||||
}
|
||||
)
|
||||
|
||||
@@ -502,11 +496,6 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
logger.debug("[%s] DM sender %s blocked by policy", self.name, sender_id)
|
||||
return
|
||||
|
||||
# Cache the inbound req_id after policy checks so proactive sends to
|
||||
# this chat can fall back to APP_CMD_RESPONSE (required for groups —
|
||||
# WeCom AI Bots cannot initiate APP_CMD_SEND in group chats).
|
||||
self._remember_chat_req_id(chat_id, self._payload_req_id(payload))
|
||||
|
||||
text, reply_text = self._extract_text(body)
|
||||
media_urls, media_types = await self._extract_media(body)
|
||||
message_type = self._derive_message_type(body, text, media_types)
|
||||
@@ -858,23 +847,6 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
while len(self._reply_req_ids) > DEDUP_MAX_SIZE:
|
||||
self._reply_req_ids.pop(next(iter(self._reply_req_ids)))
|
||||
|
||||
def _remember_chat_req_id(self, chat_id: str, req_id: str) -> None:
|
||||
"""Cache the most recent inbound req_id per chat.
|
||||
|
||||
Used as a fallback reply target when we need to send into a group
|
||||
without an explicit ``reply_to`` — WeCom AI Bots are blocked from
|
||||
APP_CMD_SEND in groups and must use APP_CMD_RESPONSE bound to some
|
||||
prior req_id. Bounded like _reply_req_ids so long-running gateways
|
||||
don't leak memory across many chats.
|
||||
"""
|
||||
normalized_chat_id = str(chat_id or "").strip()
|
||||
normalized_req_id = str(req_id or "").strip()
|
||||
if not normalized_chat_id or not normalized_req_id:
|
||||
return
|
||||
self._last_chat_req_ids[normalized_chat_id] = normalized_req_id
|
||||
while len(self._last_chat_req_ids) > DEDUP_MAX_SIZE:
|
||||
self._last_chat_req_ids.pop(next(iter(self._last_chat_req_ids)))
|
||||
|
||||
def _reply_req_id_for_message(self, reply_to: Optional[str]) -> Optional[str]:
|
||||
normalized = str(reply_to or "").strip()
|
||||
if not normalized or normalized.startswith("quote:"):
|
||||
@@ -1191,15 +1163,19 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
self._raise_for_wecom_error(response, "send media message")
|
||||
return response
|
||||
|
||||
async def _send_reply_markdown(self, reply_req_id: str, content: str) -> Dict[str, Any]:
|
||||
async def _send_reply_stream(self, reply_req_id: str, content: str) -> Dict[str, Any]:
|
||||
response = await self._send_reply_request(
|
||||
reply_req_id,
|
||||
{
|
||||
"msgtype": "markdown",
|
||||
"markdown": {"content": content[:self.MAX_MESSAGE_LENGTH]},
|
||||
"msgtype": "stream",
|
||||
"stream": {
|
||||
"id": self._new_req_id("stream"),
|
||||
"finish": True,
|
||||
"content": content[:self.MAX_MESSAGE_LENGTH],
|
||||
},
|
||||
},
|
||||
)
|
||||
self._raise_for_wecom_error(response, "send reply markdown")
|
||||
self._raise_for_wecom_error(response, "send reply stream")
|
||||
return response
|
||||
|
||||
async def _send_reply_media_message(
|
||||
@@ -1259,9 +1235,6 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error=prepared["reject_reason"])
|
||||
|
||||
reply_req_id = self._reply_req_id_for_message(reply_to)
|
||||
if not reply_req_id and chat_id in self._last_chat_req_ids:
|
||||
reply_req_id = self._last_chat_req_ids[chat_id]
|
||||
|
||||
try:
|
||||
upload_result = await self._upload_media_bytes(
|
||||
prepared["data"],
|
||||
@@ -1329,12 +1302,8 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
|
||||
try:
|
||||
reply_req_id = self._reply_req_id_for_message(reply_to)
|
||||
|
||||
if not reply_req_id and chat_id in self._last_chat_req_ids:
|
||||
reply_req_id = self._last_chat_req_ids[chat_id]
|
||||
|
||||
if reply_req_id:
|
||||
response = await self._send_reply_markdown(reply_req_id, content)
|
||||
response = await self._send_reply_stream(reply_req_id, content)
|
||||
else:
|
||||
response = await self._send_request(
|
||||
APP_CMD_SEND,
|
||||
|
||||
+86
-310
@@ -28,7 +28,7 @@ import uuid
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
from urllib.parse import quote, urlparse
|
||||
from urllib.parse import quote
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -96,28 +96,6 @@ MEDIA_VIDEO = 2
|
||||
MEDIA_FILE = 3
|
||||
MEDIA_VOICE = 4
|
||||
|
||||
_LIVE_ADAPTERS: Dict[str, Any] = {}
|
||||
|
||||
|
||||
def _make_ssl_connector() -> Optional["aiohttp.TCPConnector"]:
|
||||
"""Return a TCPConnector with a certifi CA bundle, or None if certifi is unavailable.
|
||||
|
||||
Tencent's iLink server (``ilinkai.weixin.qq.com``) is not verifiable against
|
||||
some system CA stores (notably Homebrew's OpenSSL on macOS Apple Silicon).
|
||||
When ``certifi`` is installed, use its Mozilla CA bundle to guarantee
|
||||
verification. Otherwise fall back to aiohttp's default (which honors
|
||||
``SSL_CERT_FILE`` env var via ``trust_env=True``).
|
||||
"""
|
||||
try:
|
||||
import ssl
|
||||
import certifi
|
||||
except ImportError:
|
||||
return None
|
||||
if not AIOHTTP_AVAILABLE:
|
||||
return None
|
||||
ssl_ctx = ssl.create_default_context(cafile=certifi.where())
|
||||
return aiohttp.TCPConnector(ssl=ssl_ctx)
|
||||
|
||||
ITEM_TEXT = 1
|
||||
ITEM_IMAGE = 2
|
||||
ITEM_VOICE = 3
|
||||
@@ -420,12 +398,7 @@ async def _send_message(
|
||||
text: str,
|
||||
context_token: Optional[str],
|
||||
client_id: str,
|
||||
) -> Dict[str, Any]:
|
||||
"""Send a text message via iLink sendmessage API.
|
||||
|
||||
Returns the raw API response dict (may contain error codes like
|
||||
``errcode: -14`` for session expiry that the caller can inspect).
|
||||
"""
|
||||
) -> None:
|
||||
if not text or not text.strip():
|
||||
raise ValueError("_send_message: text must not be empty")
|
||||
message: Dict[str, Any] = {
|
||||
@@ -438,7 +411,7 @@ async def _send_message(
|
||||
}
|
||||
if context_token:
|
||||
message["context_token"] = context_token
|
||||
return await _api_post(
|
||||
await _api_post(
|
||||
session,
|
||||
base_url=base_url,
|
||||
endpoint=EP_SEND_MESSAGE,
|
||||
@@ -560,39 +533,6 @@ async def _download_bytes(
|
||||
return await response.read()
|
||||
|
||||
|
||||
_WEIXIN_CDN_ALLOWLIST: frozenset[str] = frozenset(
|
||||
{
|
||||
"novac2c.cdn.weixin.qq.com",
|
||||
"ilinkai.weixin.qq.com",
|
||||
"wx.qlogo.cn",
|
||||
"thirdwx.qlogo.cn",
|
||||
"res.wx.qq.com",
|
||||
"mmbiz.qpic.cn",
|
||||
"mmbiz.qlogo.cn",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _assert_weixin_cdn_url(url: str) -> None:
|
||||
"""Raise ValueError if *url* does not point at a known WeChat CDN host."""
|
||||
try:
|
||||
parsed = urlparse(url)
|
||||
scheme = parsed.scheme.lower()
|
||||
host = parsed.hostname or ""
|
||||
except Exception as exc: # noqa: BLE001
|
||||
raise ValueError(f"Unparseable media URL: {url!r}") from exc
|
||||
|
||||
if scheme not in ("http", "https"):
|
||||
raise ValueError(
|
||||
f"Media URL has disallowed scheme {scheme!r}; only http/https are permitted."
|
||||
)
|
||||
if host not in _WEIXIN_CDN_ALLOWLIST:
|
||||
raise ValueError(
|
||||
f"Media URL host {host!r} is not in the WeChat CDN allowlist. "
|
||||
"Refusing to fetch to prevent SSRF."
|
||||
)
|
||||
|
||||
|
||||
def _media_reference(item: Dict[str, Any], key: str) -> Dict[str, Any]:
|
||||
return (item.get(key) or {}).get("media") or {}
|
||||
|
||||
@@ -613,7 +553,6 @@ async def _download_and_decrypt_media(
|
||||
timeout_seconds=timeout_seconds,
|
||||
)
|
||||
elif full_url:
|
||||
_assert_weixin_cdn_url(full_url)
|
||||
raw = await _download_bytes(session, url=full_url, timeout_seconds=timeout_seconds)
|
||||
else:
|
||||
raise RuntimeError("media item had neither encrypt_query_param nor full_url")
|
||||
@@ -684,31 +623,42 @@ def _rewrite_table_block_for_weixin(lines: List[str]) -> str:
|
||||
def _normalize_markdown_blocks(content: str) -> str:
|
||||
lines = content.splitlines()
|
||||
result: List[str] = []
|
||||
i = 0
|
||||
in_code_block = False
|
||||
blank_run = 0
|
||||
|
||||
for raw_line in lines:
|
||||
line = raw_line.rstrip()
|
||||
if _FENCE_RE.match(line.strip()):
|
||||
while i < len(lines):
|
||||
line = lines[i].rstrip()
|
||||
fence_match = _FENCE_RE.match(line.strip())
|
||||
if fence_match:
|
||||
in_code_block = not in_code_block
|
||||
result.append(line)
|
||||
blank_run = 0
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if in_code_block:
|
||||
result.append(line)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if not line.strip():
|
||||
blank_run += 1
|
||||
if blank_run <= 1:
|
||||
result.append("")
|
||||
if (
|
||||
i + 1 < len(lines)
|
||||
and "|" in lines[i]
|
||||
and _TABLE_RULE_RE.match(lines[i + 1].rstrip())
|
||||
):
|
||||
table_lines = [lines[i].rstrip(), lines[i + 1].rstrip()]
|
||||
i += 2
|
||||
while i < len(lines) and "|" in lines[i]:
|
||||
table_lines.append(lines[i].rstrip())
|
||||
i += 1
|
||||
result.append(_rewrite_table_block_for_weixin(table_lines))
|
||||
continue
|
||||
|
||||
blank_run = 0
|
||||
result.append(line)
|
||||
result.append(_MARKDOWN_LINK_RE.sub(r"\1 (\2)", _rewrite_headers_for_weixin(line)))
|
||||
i += 1
|
||||
|
||||
return "\n".join(result).strip()
|
||||
normalized = "\n".join(item.rstrip() for item in result)
|
||||
normalized = re.sub(r"\n{3,}", "\n\n", normalized)
|
||||
return normalized.strip()
|
||||
|
||||
|
||||
def _split_markdown_blocks(content: str) -> List[str]:
|
||||
@@ -754,8 +704,8 @@ def _split_delivery_units_for_weixin(content: str) -> List[str]:
|
||||
|
||||
Weixin can render Markdown, but chat readability is better when top-level
|
||||
line breaks become separate messages. Keep fenced code blocks intact and
|
||||
attach indented continuation lines to the previous top-level line so nested
|
||||
list items do not get torn apart.
|
||||
attach indented continuation lines to the previous top-level line so
|
||||
transformed tables/lists do not get torn apart.
|
||||
"""
|
||||
units: List[str] = []
|
||||
|
||||
@@ -797,9 +747,7 @@ def _looks_like_chatty_line_for_weixin(line: str) -> bool:
|
||||
return False
|
||||
if line.startswith((" ", "\t")):
|
||||
return False
|
||||
if stripped.startswith((">", "-", "*", "【", "#", "|")):
|
||||
return False
|
||||
if _TABLE_RULE_RE.match(stripped):
|
||||
if stripped.startswith((">", "-", "*", "【")):
|
||||
return False
|
||||
if re.match(r"^\*\*[^*]+\*\*$", stripped):
|
||||
return False
|
||||
@@ -809,12 +757,10 @@ def _looks_like_chatty_line_for_weixin(line: str) -> bool:
|
||||
|
||||
|
||||
def _looks_like_heading_line_for_weixin(line: str) -> bool:
|
||||
"""Return True when a short line behaves like a heading."""
|
||||
"""Return True when a short line behaves like a plain-text heading."""
|
||||
stripped = line.strip()
|
||||
if not stripped:
|
||||
return False
|
||||
if _HEADER_RE.match(stripped):
|
||||
return True
|
||||
return len(stripped) <= 24 and stripped.endswith((":", ":"))
|
||||
|
||||
|
||||
@@ -989,7 +935,7 @@ async def qr_login(
|
||||
if not AIOHTTP_AVAILABLE:
|
||||
raise RuntimeError("aiohttp is required for Weixin QR login")
|
||||
|
||||
async with aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector()) as session:
|
||||
async with aiohttp.ClientSession(trust_env=True) as session:
|
||||
try:
|
||||
qr_resp = await _api_get(
|
||||
session,
|
||||
@@ -1007,10 +953,6 @@ async def qr_login(
|
||||
logger.error("weixin: QR response missing qrcode")
|
||||
return None
|
||||
|
||||
# qrcode_url is the full scannable liteapp URL; qrcode_value is just the hex token
|
||||
# WeChat needs to scan the full URL, not the raw hex string
|
||||
qr_scan_data = qrcode_url if qrcode_url else qrcode_value
|
||||
|
||||
print("\n请使用微信扫描以下二维码:")
|
||||
if qrcode_url:
|
||||
print(qrcode_url)
|
||||
@@ -1018,11 +960,11 @@ async def qr_login(
|
||||
import qrcode
|
||||
|
||||
qr = qrcode.QRCode()
|
||||
qr.add_data(qr_scan_data)
|
||||
qr.add_data(qrcode_url or qrcode_value)
|
||||
qr.make(fit=True)
|
||||
qr.print_ascii(invert=True)
|
||||
except Exception as _qr_exc:
|
||||
print(f"(终端二维码渲染失败: {_qr_exc},请直接打开上面的二维码链接)")
|
||||
except Exception:
|
||||
print("(终端二维码渲染失败,请直接打开上面的二维码链接)")
|
||||
|
||||
deadline = time.time() + timeout_seconds
|
||||
current_base_url = ILINK_BASE_URL
|
||||
@@ -1068,17 +1010,8 @@ async def qr_login(
|
||||
)
|
||||
qrcode_value = str(qr_resp.get("qrcode") or "")
|
||||
qrcode_url = str(qr_resp.get("qrcode_img_content") or "")
|
||||
qr_scan_data = qrcode_url if qrcode_url else qrcode_value
|
||||
if qrcode_url:
|
||||
print(qrcode_url)
|
||||
try:
|
||||
import qrcode as _qrcode
|
||||
qr = _qrcode.QRCode()
|
||||
qr.add_data(qr_scan_data)
|
||||
qr.make(fit=True)
|
||||
qr.print_ascii(invert=True)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as exc:
|
||||
logger.error("weixin: QR refresh failed: %s", exc)
|
||||
return None
|
||||
@@ -1126,8 +1059,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
self._hermes_home = hermes_home
|
||||
self._token_store = ContextTokenStore(hermes_home)
|
||||
self._typing_cache = TypingTicketCache()
|
||||
self._poll_session: Optional[aiohttp.ClientSession] = None
|
||||
self._send_session: Optional[aiohttp.ClientSession] = None
|
||||
self._session: Optional[aiohttp.ClientSession] = None
|
||||
self._poll_task: Optional[asyncio.Task] = None
|
||||
self._dedup = MessageDeduplicator(ttl_seconds=MESSAGE_DEDUP_TTL_SECONDS)
|
||||
|
||||
@@ -1202,17 +1134,14 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
except Exception as exc:
|
||||
logger.debug("[%s] Token lock unavailable (non-fatal): %s", self.name, exc)
|
||||
|
||||
self._poll_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
||||
self._send_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
||||
self._session = aiohttp.ClientSession(trust_env=True)
|
||||
self._token_store.restore(self._account_id)
|
||||
self._poll_task = asyncio.create_task(self._poll_loop(), name="weixin-poll")
|
||||
self._mark_connected()
|
||||
_LIVE_ADAPTERS[self._token] = self
|
||||
logger.info("[%s] Connected account=%s base=%s", self.name, _safe_id(self._account_id), self._base_url)
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
_LIVE_ADAPTERS.pop(self._token, None)
|
||||
self._running = False
|
||||
if self._poll_task and not self._poll_task.done():
|
||||
self._poll_task.cancel()
|
||||
@@ -1221,18 +1150,15 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._poll_task = None
|
||||
if self._poll_session and not self._poll_session.closed:
|
||||
await self._poll_session.close()
|
||||
self._poll_session = None
|
||||
if self._send_session and not self._send_session.closed:
|
||||
await self._send_session.close()
|
||||
self._send_session = None
|
||||
if self._session and not self._session.closed:
|
||||
await self._session.close()
|
||||
self._session = None
|
||||
self._release_platform_lock()
|
||||
self._mark_disconnected()
|
||||
logger.info("[%s] Disconnected", self.name)
|
||||
|
||||
async def _poll_loop(self) -> None:
|
||||
assert self._poll_session is not None
|
||||
assert self._session is not None
|
||||
sync_buf = _load_sync_buf(self._hermes_home, self._account_id)
|
||||
timeout_ms = LONG_POLL_TIMEOUT_MS
|
||||
consecutive_failures = 0
|
||||
@@ -1240,7 +1166,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
while self._running:
|
||||
try:
|
||||
response = await _get_updates(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
sync_buf=sync_buf,
|
||||
@@ -1297,7 +1223,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
logger.error("[%s] unhandled inbound error from=%s: %s", self.name, _safe_id(message.get("from_user_id")), exc, exc_info=True)
|
||||
|
||||
async def _process_message(self, message: Dict[str, Any]) -> None:
|
||||
assert self._poll_session is not None
|
||||
assert self._session is not None
|
||||
sender_id = str(message.get("from_user_id") or "").strip()
|
||||
if not sender_id:
|
||||
return
|
||||
@@ -1390,7 +1316,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
media = _media_reference(item, "image_item")
|
||||
try:
|
||||
data = await _download_and_decrypt_media(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
encrypted_query_param=media.get("encrypt_query_param"),
|
||||
aes_key_b64=(item.get("image_item") or {}).get("aeskey")
|
||||
@@ -1408,7 +1334,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
media = _media_reference(item, "video_item")
|
||||
try:
|
||||
data = await _download_and_decrypt_media(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
encrypted_query_param=media.get("encrypt_query_param"),
|
||||
aes_key_b64=media.get("aes_key"),
|
||||
@@ -1427,7 +1353,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
mime = _mime_from_filename(filename)
|
||||
try:
|
||||
data = await _download_and_decrypt_media(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
encrypted_query_param=media.get("encrypt_query_param"),
|
||||
aes_key_b64=media.get("aes_key"),
|
||||
@@ -1446,7 +1372,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
return None
|
||||
try:
|
||||
data = await _download_and_decrypt_media(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
encrypted_query_param=media.get("encrypt_query_param"),
|
||||
aes_key_b64=media.get("aes_key"),
|
||||
@@ -1459,13 +1385,13 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
return None
|
||||
|
||||
async def _maybe_fetch_typing_ticket(self, user_id: str, context_token: Optional[str]) -> None:
|
||||
if not self._poll_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return
|
||||
if self._typing_cache.get(user_id):
|
||||
return
|
||||
try:
|
||||
response = await _get_config(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
user_id=user_id,
|
||||
@@ -1490,19 +1416,12 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
context_token: Optional[str],
|
||||
client_id: str,
|
||||
) -> None:
|
||||
"""Send a single text chunk with per-chunk retry and backoff.
|
||||
|
||||
On session-expired errors (errcode -14), automatically retries
|
||||
*without* ``context_token`` — iLink accepts tokenless sends as a
|
||||
degraded fallback, which keeps cron-initiated push messages working
|
||||
even when no user message has refreshed the session recently.
|
||||
"""
|
||||
"""Send a single text chunk with per-chunk retry and backoff."""
|
||||
last_error: Optional[Exception] = None
|
||||
retried_without_token = False
|
||||
for attempt in range(self._send_chunk_retries + 1):
|
||||
try:
|
||||
resp = await _send_message(
|
||||
self._send_session,
|
||||
await _send_message(
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to=chat_id,
|
||||
@@ -1510,31 +1429,6 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
context_token=context_token,
|
||||
client_id=client_id,
|
||||
)
|
||||
# Check iLink response for session-expired error
|
||||
if resp and isinstance(resp, dict):
|
||||
ret = resp.get("ret")
|
||||
errcode = resp.get("errcode")
|
||||
if (ret is not None and ret not in (0,)) or (errcode is not None and errcode not in (0,)):
|
||||
is_session_expired = (
|
||||
ret == SESSION_EXPIRED_ERRCODE
|
||||
or errcode == SESSION_EXPIRED_ERRCODE
|
||||
)
|
||||
# Session expired — strip token and retry once
|
||||
if is_session_expired and not retried_without_token and context_token:
|
||||
retried_without_token = True
|
||||
context_token = None
|
||||
self._token_store._cache.pop(
|
||||
self._token_store._key(self._account_id, chat_id), None
|
||||
)
|
||||
logger.warning(
|
||||
"[%s] session expired for %s; retrying without context_token",
|
||||
self.name, _safe_id(chat_id),
|
||||
)
|
||||
continue
|
||||
errmsg = resp.get("errmsg") or resp.get("msg") or "unknown error"
|
||||
raise RuntimeError(
|
||||
f"iLink sendmessage error: ret={ret} errcode={errcode} errmsg={errmsg}"
|
||||
)
|
||||
return
|
||||
except Exception as exc:
|
||||
last_error = exc
|
||||
@@ -1562,48 +1456,12 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
context_token = self._token_store.get(self._account_id, chat_id)
|
||||
last_message_id: Optional[str] = None
|
||||
|
||||
# Extract MEDIA: tags and bare local file paths before text delivery.
|
||||
media_files, cleaned_content = self.extract_media(content)
|
||||
_, image_cleaned = self.extract_images(cleaned_content)
|
||||
local_files, final_content = self.extract_local_files(image_cleaned)
|
||||
|
||||
_AUDIO_EXTS = {".ogg", ".opus", ".mp3", ".wav", ".m4a"}
|
||||
_VIDEO_EXTS = {".mp4", ".mov", ".avi", ".mkv", ".webm", ".3gp"}
|
||||
_IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".webp", ".gif"}
|
||||
|
||||
async def _deliver_media(path: str, is_voice: bool = False) -> None:
|
||||
ext = Path(path).suffix.lower()
|
||||
if is_voice or ext in _AUDIO_EXTS:
|
||||
await self.send_voice(chat_id=chat_id, audio_path=path, metadata=metadata)
|
||||
elif ext in _VIDEO_EXTS:
|
||||
await self.send_video(chat_id=chat_id, video_path=path, metadata=metadata)
|
||||
elif ext in _IMAGE_EXTS:
|
||||
await self.send_image_file(chat_id=chat_id, image_path=path, metadata=metadata)
|
||||
else:
|
||||
await self.send_document(chat_id=chat_id, file_path=path, metadata=metadata)
|
||||
|
||||
try:
|
||||
# Deliver extracted MEDIA: attachments first.
|
||||
for media_path, is_voice in media_files:
|
||||
try:
|
||||
await _deliver_media(media_path, is_voice)
|
||||
except Exception as exc:
|
||||
logger.warning("[%s] media delivery failed for %s: %s", self.name, media_path, exc)
|
||||
|
||||
# Deliver bare local file paths.
|
||||
for file_path in local_files:
|
||||
try:
|
||||
await _deliver_media(file_path, is_voice=False)
|
||||
except Exception as exc:
|
||||
logger.warning("[%s] local file delivery failed for %s: %s", self.name, file_path, exc)
|
||||
|
||||
# Deliver text content.
|
||||
chunks = [c for c in self._split_text(self.format_message(final_content)) if c and c.strip()]
|
||||
chunks = [c for c in self._split_text(self.format_message(content)) if c and c.strip()]
|
||||
for idx, chunk in enumerate(chunks):
|
||||
client_id = f"hermes-weixin-{uuid.uuid4().hex}"
|
||||
await self._send_text_chunk(
|
||||
@@ -1621,14 +1479,14 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
async def send_typing(self, chat_id: str, metadata: Optional[Dict[str, Any]] = None) -> None:
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return
|
||||
typing_ticket = self._typing_cache.get(chat_id)
|
||||
if not typing_ticket:
|
||||
return
|
||||
try:
|
||||
await _send_typing(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to_user_id=chat_id,
|
||||
@@ -1639,14 +1497,14 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
logger.debug("[%s] typing start failed for %s: %s", self.name, _safe_id(chat_id), exc)
|
||||
|
||||
async def stop_typing(self, chat_id: str) -> None:
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return
|
||||
typing_ticket = self._typing_cache.get(chat_id)
|
||||
if not typing_ticket:
|
||||
return
|
||||
try:
|
||||
await _send_typing(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to_user_id=chat_id,
|
||||
@@ -1684,35 +1542,24 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
async def send_image_file(
|
||||
self,
|
||||
chat_id: str,
|
||||
image_path: str,
|
||||
caption: Optional[str] = None,
|
||||
path: str,
|
||||
caption: str = "",
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
del reply_to, kwargs
|
||||
return await self.send_document(
|
||||
chat_id=chat_id,
|
||||
file_path=image_path,
|
||||
caption=caption,
|
||||
metadata=metadata,
|
||||
)
|
||||
return await self.send_document(chat_id, file_path=path, caption=caption, metadata=metadata)
|
||||
|
||||
async def send_document(
|
||||
self,
|
||||
chat_id: str,
|
||||
file_path: str,
|
||||
caption: Optional[str] = None,
|
||||
file_name: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
caption: str = "",
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
del file_name, reply_to, metadata, kwargs
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
try:
|
||||
message_id = await self._send_file(chat_id, file_path, caption or "")
|
||||
message_id = await self._send_file(chat_id, file_path, caption)
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
except Exception as exc:
|
||||
logger.error("[%s] send_document failed to=%s: %s", self.name, _safe_id(chat_id), exc)
|
||||
@@ -1726,7 +1573,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
try:
|
||||
message_id = await self._send_file(chat_id, video_path, caption or "")
|
||||
@@ -1743,24 +1590,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
if not self._send_session or not self._token:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
|
||||
# Native outbound Weixin voice bubbles are not proven-working in the
|
||||
# upstream reference implementation. Prefer a reliable file attachment
|
||||
# fallback so users at least receive playable audio, even for .silk.
|
||||
fallback_caption = caption or "[voice message as attachment]"
|
||||
try:
|
||||
message_id = await self._send_file(
|
||||
chat_id,
|
||||
audio_path,
|
||||
fallback_caption,
|
||||
force_file_attachment=True,
|
||||
)
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
except Exception as exc:
|
||||
logger.error("[%s] send_voice failed to=%s: %s", self.name, _safe_id(chat_id), exc)
|
||||
return SendResult(success=False, error=str(exc))
|
||||
return await self.send_document(chat_id, audio_path, caption=caption or "", metadata=metadata)
|
||||
|
||||
async def _download_remote_media(self, url: str) -> str:
|
||||
from tools.url_safety import is_safe_url
|
||||
@@ -1768,8 +1598,8 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
if not is_safe_url(url):
|
||||
raise ValueError(f"Blocked unsafe URL (SSRF protection): {url}")
|
||||
|
||||
assert self._send_session is not None
|
||||
async with self._send_session.get(url, timeout=aiohttp.ClientTimeout(total=30)) as response:
|
||||
assert self._session is not None
|
||||
async with self._session.get(url, timeout=aiohttp.ClientTimeout(total=30)) as response:
|
||||
response.raise_for_status()
|
||||
data = await response.read()
|
||||
suffix = Path(url.split("?", 1)[0]).suffix or ".bin"
|
||||
@@ -1777,22 +1607,16 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
handle.write(data)
|
||||
return handle.name
|
||||
|
||||
async def _send_file(
|
||||
self,
|
||||
chat_id: str,
|
||||
path: str,
|
||||
caption: str,
|
||||
force_file_attachment: bool = False,
|
||||
) -> str:
|
||||
assert self._send_session is not None and self._token is not None
|
||||
async def _send_file(self, chat_id: str, path: str, caption: str) -> str:
|
||||
assert self._session is not None and self._token is not None
|
||||
plaintext = Path(path).read_bytes()
|
||||
media_type, item_builder = self._outbound_media_builder(path, force_file_attachment=force_file_attachment)
|
||||
media_type, item_builder = self._outbound_media_builder(path)
|
||||
filekey = secrets.token_hex(16)
|
||||
aes_key = secrets.token_bytes(16)
|
||||
rawsize = len(plaintext)
|
||||
rawfilemd5 = hashlib.md5(plaintext).hexdigest()
|
||||
upload_response = await _get_upload_url(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to_user_id=chat_id,
|
||||
@@ -1818,34 +1642,30 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
raise RuntimeError(f"getUploadUrl returned neither upload_param nor upload_full_url: {upload_response}")
|
||||
|
||||
encrypted_query_param = await _upload_ciphertext(
|
||||
self._send_session,
|
||||
self._session,
|
||||
ciphertext=ciphertext,
|
||||
upload_url=upload_url,
|
||||
)
|
||||
|
||||
context_token = self._token_store.get(self._account_id, chat_id)
|
||||
# The iLink API expects aes_key as base64(hex_string), not base64(raw_bytes).
|
||||
# Sending base64(raw_bytes) causes images to show as grey boxes on the
|
||||
# receiver side because the decryption key doesn't match.
|
||||
aes_key_for_api = base64.b64encode(aes_key.hex().encode("ascii")).decode("ascii")
|
||||
item_kwargs = {
|
||||
"encrypt_query_param": encrypted_query_param,
|
||||
"aes_key_for_api": aes_key_for_api,
|
||||
"ciphertext_size": len(ciphertext),
|
||||
"plaintext_size": rawsize,
|
||||
"filename": Path(path).name,
|
||||
"rawfilemd5": rawfilemd5,
|
||||
}
|
||||
if media_type == MEDIA_VOICE and path.endswith(".silk"):
|
||||
item_kwargs["encode_type"] = 6
|
||||
item_kwargs["sample_rate"] = 24000
|
||||
item_kwargs["bits_per_sample"] = 16
|
||||
media_item = item_builder(**item_kwargs)
|
||||
media_item = item_builder(
|
||||
encrypt_query_param=encrypted_query_param,
|
||||
aes_key_for_api=aes_key_for_api,
|
||||
ciphertext_size=len(ciphertext),
|
||||
plaintext_size=rawsize,
|
||||
filename=Path(path).name,
|
||||
rawfilemd5=rawfilemd5,
|
||||
)
|
||||
|
||||
last_message_id = None
|
||||
if caption:
|
||||
last_message_id = f"hermes-weixin-{uuid.uuid4().hex}"
|
||||
await _send_message(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to=chat_id,
|
||||
@@ -1856,7 +1676,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
|
||||
last_message_id = f"hermes-weixin-{uuid.uuid4().hex}"
|
||||
await _api_post(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
endpoint=EP_SEND_MESSAGE,
|
||||
payload={
|
||||
@@ -1875,7 +1695,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
)
|
||||
return last_message_id
|
||||
|
||||
def _outbound_media_builder(self, path: str, force_file_attachment: bool = False):
|
||||
def _outbound_media_builder(self, path: str):
|
||||
mime = mimetypes.guess_type(path)[0] or "application/octet-stream"
|
||||
if mime.startswith("image/"):
|
||||
return MEDIA_IMAGE, lambda **kw: {
|
||||
@@ -1903,7 +1723,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
"video_md5": kw.get("rawfilemd5", ""),
|
||||
},
|
||||
}
|
||||
if path.endswith(".silk") and not force_file_attachment:
|
||||
if mime.startswith("audio/") or path.endswith(".silk"):
|
||||
return MEDIA_VOICE, lambda **kw: {
|
||||
"type": ITEM_VOICE,
|
||||
"voice_item": {
|
||||
@@ -1912,25 +1732,9 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
"aes_key": kw["aes_key_for_api"],
|
||||
"encrypt_type": 1,
|
||||
},
|
||||
"encode_type": kw.get("encode_type"),
|
||||
"bits_per_sample": kw.get("bits_per_sample"),
|
||||
"sample_rate": kw.get("sample_rate"),
|
||||
"playtime": kw.get("playtime", 0),
|
||||
},
|
||||
}
|
||||
if mime.startswith("audio/"):
|
||||
return MEDIA_FILE, lambda **kw: {
|
||||
"type": ITEM_FILE,
|
||||
"file_item": {
|
||||
"media": {
|
||||
"encrypt_query_param": kw["encrypt_query_param"],
|
||||
"aes_key": kw["aes_key_for_api"],
|
||||
"encrypt_type": 1,
|
||||
},
|
||||
"file_name": kw["filename"],
|
||||
"len": str(kw["plaintext_size"]),
|
||||
},
|
||||
}
|
||||
return MEDIA_FILE, lambda **kw: {
|
||||
"type": ITEM_FILE,
|
||||
"file_item": {
|
||||
@@ -1980,34 +1784,7 @@ async def send_weixin_direct(
|
||||
token_store.restore(account_id)
|
||||
context_token = token_store.get(account_id, chat_id)
|
||||
|
||||
live_adapter = _LIVE_ADAPTERS.get(resolved_token)
|
||||
send_session = getattr(live_adapter, '_send_session', None)
|
||||
if live_adapter is not None and send_session is not None and not send_session.closed:
|
||||
last_result: Optional[SendResult] = None
|
||||
cleaned = live_adapter.format_message(message)
|
||||
if cleaned:
|
||||
last_result = await live_adapter.send(chat_id, cleaned)
|
||||
if not last_result.success:
|
||||
return {"error": f"Weixin send failed: {last_result.error}"}
|
||||
|
||||
for media_path, _is_voice in media_files or []:
|
||||
ext = Path(media_path).suffix.lower()
|
||||
if ext in {".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp"}:
|
||||
last_result = await live_adapter.send_image_file(chat_id, media_path)
|
||||
else:
|
||||
last_result = await live_adapter.send_document(chat_id, media_path)
|
||||
if not last_result.success:
|
||||
return {"error": f"Weixin media send failed: {last_result.error}"}
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"platform": "weixin",
|
||||
"chat_id": chat_id,
|
||||
"message_id": last_result.message_id if last_result else None,
|
||||
"context_token_used": bool(context_token),
|
||||
}
|
||||
|
||||
async with aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector()) as session:
|
||||
async with aiohttp.ClientSession(trust_env=True) as session:
|
||||
adapter = WeixinAdapter(
|
||||
PlatformConfig(
|
||||
enabled=True,
|
||||
@@ -2020,7 +1797,6 @@ async def send_weixin_direct(
|
||||
},
|
||||
)
|
||||
)
|
||||
adapter._send_session = session
|
||||
adapter._session = session
|
||||
adapter._token = resolved_token
|
||||
adapter._account_id = account_id
|
||||
|
||||
+31
-749
File diff suppressed because it is too large
Load Diff
+3
-156
@@ -82,7 +82,6 @@ class SessionSource:
|
||||
chat_topic: Optional[str] = None # Channel topic/description (Discord, Slack)
|
||||
user_id_alt: Optional[str] = None # Signal UUID (alternative to phone number)
|
||||
chat_id_alt: Optional[str] = None # Signal group internal ID
|
||||
is_bot: bool = False # True when the message author is a bot/webhook (Discord)
|
||||
|
||||
@property
|
||||
def description(self) -> str:
|
||||
@@ -377,19 +376,7 @@ class SessionEntry:
|
||||
# this session (create a new session_id) so the user starts fresh.
|
||||
# Set by /stop to break stuck-resume loops (#7536).
|
||||
suspended: bool = False
|
||||
|
||||
# When True the session was interrupted by a gateway restart/shutdown
|
||||
# drain timeout, but recovery is still expected. Unlike ``suspended``,
|
||||
# ``resume_pending`` preserves the existing session_id on next access —
|
||||
# the user stays on the same transcript and the agent auto-continues
|
||||
# from where it left off. Cleared after the next successful turn.
|
||||
# Escalation to ``suspended`` is handled by the existing
|
||||
# ``.restart_failure_counts`` stuck-loop counter (#7536), not by a
|
||||
# parallel counter on this entry.
|
||||
resume_pending: bool = False
|
||||
resume_reason: Optional[str] = None # e.g. "restart_timeout"
|
||||
last_resume_marked_at: Optional[datetime] = None
|
||||
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
result = {
|
||||
"session_key": self.session_key,
|
||||
@@ -409,13 +396,6 @@ class SessionEntry:
|
||||
"cost_status": self.cost_status,
|
||||
"memory_flushed": self.memory_flushed,
|
||||
"suspended": self.suspended,
|
||||
"resume_pending": self.resume_pending,
|
||||
"resume_reason": self.resume_reason,
|
||||
"last_resume_marked_at": (
|
||||
self.last_resume_marked_at.isoformat()
|
||||
if self.last_resume_marked_at
|
||||
else None
|
||||
),
|
||||
}
|
||||
if self.origin:
|
||||
result["origin"] = self.origin.to_dict()
|
||||
@@ -433,15 +413,7 @@ class SessionEntry:
|
||||
platform = Platform(data["platform"])
|
||||
except ValueError as e:
|
||||
logger.debug("Unknown platform value %r: %s", data["platform"], e)
|
||||
|
||||
last_resume_marked_at = None
|
||||
_lrma = data.get("last_resume_marked_at")
|
||||
if _lrma:
|
||||
try:
|
||||
last_resume_marked_at = datetime.fromisoformat(_lrma)
|
||||
except (TypeError, ValueError):
|
||||
last_resume_marked_at = None
|
||||
|
||||
|
||||
return cls(
|
||||
session_key=data["session_key"],
|
||||
session_id=data["session_id"],
|
||||
@@ -461,9 +433,6 @@ class SessionEntry:
|
||||
cost_status=data.get("cost_status", "unknown"),
|
||||
memory_flushed=data.get("memory_flushed", False),
|
||||
suspended=data.get("suspended", False),
|
||||
resume_pending=data.get("resume_pending", False),
|
||||
resume_reason=data.get("resume_reason"),
|
||||
last_resume_marked_at=last_resume_marked_at,
|
||||
)
|
||||
|
||||
|
||||
@@ -740,23 +709,9 @@ class SessionStore:
|
||||
entry = self._entries[session_key]
|
||||
|
||||
# Auto-reset sessions marked as suspended (e.g. after /stop
|
||||
# broke a stuck loop — #7536). ``suspended`` is the hard
|
||||
# forced-wipe signal and always wins over ``resume_pending``,
|
||||
# so repeated interrupted restarts that escalate via the
|
||||
# existing ``.restart_failure_counts`` stuck-loop counter
|
||||
# still converge to a clean slate.
|
||||
# broke a stuck loop — #7536).
|
||||
if entry.suspended:
|
||||
reset_reason = "suspended"
|
||||
elif entry.resume_pending:
|
||||
# Restart-interrupted session: preserve the session_id
|
||||
# and return the existing entry so the transcript
|
||||
# reloads intact. ``resume_pending`` is cleared after
|
||||
# the NEXT successful turn completes (not here), which
|
||||
# means a re-interrupted retry keeps trying — the
|
||||
# stuck-loop counter handles terminal escalation.
|
||||
entry.updated_at = now
|
||||
self._save()
|
||||
return entry
|
||||
else:
|
||||
reset_reason = self._should_reset(entry, source)
|
||||
if not reset_reason:
|
||||
@@ -846,106 +801,6 @@ class SessionStore:
|
||||
return True
|
||||
return False
|
||||
|
||||
def mark_resume_pending(
|
||||
self,
|
||||
session_key: str,
|
||||
reason: str = "restart_timeout",
|
||||
) -> bool:
|
||||
"""Mark a session as resumable after a restart interruption.
|
||||
|
||||
Unlike ``suspend_session()``, this preserves the existing
|
||||
``session_id`` and the transcript. The next call to
|
||||
``get_or_create_session()`` for this key returns the same entry
|
||||
so the user auto-resumes on the same conversation lane.
|
||||
|
||||
Returns True if the session existed and was marked.
|
||||
"""
|
||||
with self._lock:
|
||||
self._ensure_loaded_locked()
|
||||
if session_key in self._entries:
|
||||
entry = self._entries[session_key]
|
||||
# Never override an explicit ``suspended`` — that is a hard
|
||||
# forced-wipe signal (from /stop or stuck-loop escalation).
|
||||
if entry.suspended:
|
||||
return False
|
||||
entry.resume_pending = True
|
||||
entry.resume_reason = reason
|
||||
entry.last_resume_marked_at = _now()
|
||||
self._save()
|
||||
return True
|
||||
return False
|
||||
|
||||
def clear_resume_pending(self, session_key: str) -> bool:
|
||||
"""Clear the resume-pending flag after a successful resumed turn.
|
||||
|
||||
Called from the gateway after ``run_conversation()`` returns a
|
||||
final response for a session that had ``resume_pending=True``,
|
||||
signalling that recovery succeeded.
|
||||
|
||||
Returns True if a flag was cleared.
|
||||
"""
|
||||
with self._lock:
|
||||
self._ensure_loaded_locked()
|
||||
entry = self._entries.get(session_key)
|
||||
if entry is None or not entry.resume_pending:
|
||||
return False
|
||||
entry.resume_pending = False
|
||||
entry.resume_reason = None
|
||||
entry.last_resume_marked_at = None
|
||||
self._save()
|
||||
return True
|
||||
|
||||
def prune_old_entries(self, max_age_days: int) -> int:
|
||||
"""Drop SessionEntry records older than max_age_days.
|
||||
|
||||
Pruning is based on ``updated_at`` (last activity), not ``created_at``.
|
||||
A session that's been active within the window is kept regardless of
|
||||
how old it is. Entries marked ``suspended`` are kept — the user
|
||||
explicitly paused them for later resume. Entries held by an active
|
||||
process (via has_active_processes_fn) are also kept so long-running
|
||||
background work isn't orphaned.
|
||||
|
||||
Pruning is functionally identical to a natural reset-policy expiry:
|
||||
the transcript in SQLite stays, but the session_key → session_id
|
||||
mapping is dropped and the user starts a fresh session on return.
|
||||
|
||||
``max_age_days <= 0`` disables pruning; returns 0 immediately.
|
||||
Returns the number of entries removed.
|
||||
"""
|
||||
if max_age_days is None or max_age_days <= 0:
|
||||
return 0
|
||||
from datetime import timedelta
|
||||
|
||||
cutoff = _now() - timedelta(days=max_age_days)
|
||||
removed_keys: list[str] = []
|
||||
|
||||
with self._lock:
|
||||
self._ensure_loaded_locked()
|
||||
for key, entry in list(self._entries.items()):
|
||||
if entry.suspended:
|
||||
continue
|
||||
# Never prune sessions with an active background process
|
||||
# attached — the user may still be waiting on output.
|
||||
if self._has_active_processes_fn is not None:
|
||||
try:
|
||||
if self._has_active_processes_fn(entry.session_id):
|
||||
continue
|
||||
except Exception:
|
||||
pass
|
||||
if entry.updated_at < cutoff:
|
||||
removed_keys.append(key)
|
||||
for key in removed_keys:
|
||||
self._entries.pop(key, None)
|
||||
if removed_keys:
|
||||
self._save()
|
||||
|
||||
if removed_keys:
|
||||
logger.info(
|
||||
"SessionStore pruned %d entries older than %d days",
|
||||
len(removed_keys), max_age_days,
|
||||
)
|
||||
return len(removed_keys)
|
||||
|
||||
def suspend_recently_active(self, max_age_seconds: int = 120) -> int:
|
||||
"""Mark recently-active sessions as suspended.
|
||||
|
||||
@@ -954,12 +809,6 @@ class SessionStore:
|
||||
(#7536). Only suspends sessions updated within *max_age_seconds*
|
||||
to avoid resetting long-idle sessions that are harmless to resume.
|
||||
Returns the number of sessions that were suspended.
|
||||
|
||||
Entries flagged ``resume_pending=True`` are skipped — those were
|
||||
marked intentionally by the drain-timeout path as recoverable.
|
||||
Terminal escalation for genuinely stuck ``resume_pending`` sessions
|
||||
is handled by the existing ``.restart_failure_counts`` stuck-loop
|
||||
counter, which runs after this method on startup.
|
||||
"""
|
||||
from datetime import timedelta
|
||||
|
||||
@@ -968,8 +817,6 @@ class SessionStore:
|
||||
with self._lock:
|
||||
self._ensure_loaded_locked()
|
||||
for entry in self._entries.values():
|
||||
if entry.resume_pending:
|
||||
continue
|
||||
if not entry.suspended and entry.updated_at >= cutoff:
|
||||
entry.suspended = True
|
||||
count += 1
|
||||
|
||||
+11
-159
@@ -188,8 +188,8 @@ def _write_json_file(path: Path, payload: dict[str, Any]) -> None:
|
||||
path.write_text(json.dumps(payload))
|
||||
|
||||
|
||||
def _read_pid_record(pid_path: Optional[Path] = None) -> Optional[dict]:
|
||||
pid_path = pid_path or _get_pid_path()
|
||||
def _read_pid_record() -> Optional[dict]:
|
||||
pid_path = _get_pid_path()
|
||||
if not pid_path.exists():
|
||||
return None
|
||||
|
||||
@@ -212,18 +212,6 @@ def _read_pid_record(pid_path: Optional[Path] = None) -> Optional[dict]:
|
||||
return None
|
||||
|
||||
|
||||
def _cleanup_invalid_pid_path(pid_path: Path, *, cleanup_stale: bool) -> None:
|
||||
if not cleanup_stale:
|
||||
return
|
||||
try:
|
||||
if pid_path == _get_pid_path():
|
||||
remove_pid_file()
|
||||
else:
|
||||
pid_path.unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def write_pid_file() -> None:
|
||||
"""Write the current process PID and metadata to the gateway PID file."""
|
||||
_write_json_file(_get_pid_path(), _build_pid_record())
|
||||
@@ -425,179 +413,43 @@ def release_all_scoped_locks() -> int:
|
||||
return removed
|
||||
|
||||
|
||||
# ── --replace takeover marker ─────────────────────────────────────────
|
||||
#
|
||||
# When a new gateway starts with ``--replace``, it SIGTERMs the existing
|
||||
# gateway so it can take over the bot token. PR #5646 made SIGTERM exit
|
||||
# the gateway with code 1 so ``Restart=on-failure`` can revive it after
|
||||
# unexpected kills — but that also means a --replace takeover target
|
||||
# exits 1, which tricks systemd into reviving it 30 seconds later,
|
||||
# starting a flap loop against the replacer when both services are
|
||||
# enabled in the user's systemd (e.g. ``hermes.service`` + ``hermes-
|
||||
# gateway.service``).
|
||||
#
|
||||
# The takeover marker breaks the loop: the replacer writes a short-lived
|
||||
# file naming the target PID + start_time BEFORE sending SIGTERM.
|
||||
# The target's shutdown handler reads the marker and, if it names
|
||||
# this process, treats the SIGTERM as a planned takeover and exits 0.
|
||||
# The marker is unlinked after the target has consumed it, so a stale
|
||||
# marker left by a crashed replacer can grief at most one future
|
||||
# shutdown on the same PID — and only within _TAKEOVER_MARKER_TTL_S.
|
||||
|
||||
_TAKEOVER_MARKER_FILENAME = ".gateway-takeover.json"
|
||||
_TAKEOVER_MARKER_TTL_S = 60 # Marker older than this is treated as stale
|
||||
|
||||
|
||||
def _get_takeover_marker_path() -> Path:
|
||||
"""Return the path to the --replace takeover marker file."""
|
||||
home = get_hermes_home()
|
||||
return home / _TAKEOVER_MARKER_FILENAME
|
||||
|
||||
|
||||
def write_takeover_marker(target_pid: int) -> bool:
|
||||
"""Record that ``target_pid`` is being replaced by the current process.
|
||||
|
||||
Captures the target's ``start_time`` so that PID reuse after the
|
||||
target exits cannot later match the marker. Also records the
|
||||
replacer's PID and a UTC timestamp for TTL-based staleness checks.
|
||||
|
||||
Returns True on successful write, False on any failure. The caller
|
||||
should proceed with the SIGTERM even if the write fails (the marker
|
||||
is a best-effort signal, not a correctness requirement).
|
||||
"""
|
||||
try:
|
||||
target_start_time = _get_process_start_time(target_pid)
|
||||
record = {
|
||||
"target_pid": target_pid,
|
||||
"target_start_time": target_start_time,
|
||||
"replacer_pid": os.getpid(),
|
||||
"written_at": _utc_now_iso(),
|
||||
}
|
||||
_write_json_file(_get_takeover_marker_path(), record)
|
||||
return True
|
||||
except (OSError, PermissionError):
|
||||
return False
|
||||
|
||||
|
||||
def consume_takeover_marker_for_self() -> bool:
|
||||
"""Check & unlink the takeover marker if it names the current process.
|
||||
|
||||
Returns True only when a valid (non-stale) marker names this PID +
|
||||
start_time. A returning True indicates the current SIGTERM is a
|
||||
planned --replace takeover; the caller should exit 0 instead of
|
||||
signalling ``_signal_initiated_shutdown``.
|
||||
|
||||
Always unlinks the marker on match (and on detected staleness) so
|
||||
subsequent unrelated signals don't re-trigger.
|
||||
"""
|
||||
path = _get_takeover_marker_path()
|
||||
record = _read_json_file(path)
|
||||
if not record:
|
||||
return False
|
||||
|
||||
# Any malformed or stale marker → drop it and return False
|
||||
try:
|
||||
target_pid = int(record["target_pid"])
|
||||
target_start_time = record.get("target_start_time")
|
||||
written_at = record.get("written_at") or ""
|
||||
except (KeyError, TypeError, ValueError):
|
||||
try:
|
||||
path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
return False
|
||||
|
||||
# TTL guard: a stale marker older than _TAKEOVER_MARKER_TTL_S is ignored.
|
||||
stale = False
|
||||
try:
|
||||
written_dt = datetime.fromisoformat(written_at)
|
||||
age = (datetime.now(timezone.utc) - written_dt).total_seconds()
|
||||
if age > _TAKEOVER_MARKER_TTL_S:
|
||||
stale = True
|
||||
except (TypeError, ValueError):
|
||||
stale = True # Unparseable timestamp — treat as stale
|
||||
|
||||
if stale:
|
||||
try:
|
||||
path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
return False
|
||||
|
||||
# Does the marker name THIS process?
|
||||
our_pid = os.getpid()
|
||||
our_start_time = _get_process_start_time(our_pid)
|
||||
matches = (
|
||||
target_pid == our_pid
|
||||
and target_start_time is not None
|
||||
and our_start_time is not None
|
||||
and target_start_time == our_start_time
|
||||
)
|
||||
|
||||
# Consume the marker whether it matched or not — a marker that doesn't
|
||||
# match our identity is stale-for-us anyway.
|
||||
try:
|
||||
path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
return matches
|
||||
|
||||
|
||||
def clear_takeover_marker() -> None:
|
||||
"""Remove the takeover marker unconditionally. Safe to call repeatedly."""
|
||||
try:
|
||||
_get_takeover_marker_path().unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def get_running_pid(
|
||||
pid_path: Optional[Path] = None,
|
||||
*,
|
||||
cleanup_stale: bool = True,
|
||||
) -> Optional[int]:
|
||||
def get_running_pid() -> Optional[int]:
|
||||
"""Return the PID of a running gateway instance, or ``None``.
|
||||
|
||||
Checks the PID file and verifies the process is actually alive.
|
||||
Cleans up stale PID files automatically.
|
||||
"""
|
||||
resolved_pid_path = pid_path or _get_pid_path()
|
||||
record = _read_pid_record(resolved_pid_path)
|
||||
record = _read_pid_record()
|
||||
if not record:
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
try:
|
||||
pid = int(record["pid"])
|
||||
except (KeyError, TypeError, ValueError):
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
try:
|
||||
os.kill(pid, 0) # signal 0 = existence check, no actual signal sent
|
||||
except (ProcessLookupError, PermissionError):
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
recorded_start = record.get("start_time")
|
||||
current_start = _get_process_start_time(pid)
|
||||
if recorded_start is not None and current_start is not None and current_start != recorded_start:
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
if not _looks_like_gateway_process(pid):
|
||||
if not _record_looks_like_gateway(record):
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
return pid
|
||||
|
||||
|
||||
def is_gateway_running(
|
||||
pid_path: Optional[Path] = None,
|
||||
*,
|
||||
cleanup_stale: bool = True,
|
||||
) -> bool:
|
||||
def is_gateway_running() -> bool:
|
||||
"""Check if the gateway daemon is currently running."""
|
||||
return get_running_pid(pid_path, cleanup_stale=cleanup_stale) is not None
|
||||
return get_running_pid() is not None
|
||||
|
||||
@@ -100,14 +100,6 @@ class GatewayStreamConsumer:
|
||||
self._flood_strikes = 0 # Consecutive flood-control edit failures
|
||||
self._current_edit_interval = self.cfg.edit_interval # Adaptive backoff
|
||||
self._final_response_sent = False
|
||||
# Cache adapter lifecycle capability: only platforms that need an
|
||||
# explicit finalize call (e.g. DingTalk AI Cards) force us to make
|
||||
# a redundant final edit. Everyone else keeps the fast path.
|
||||
# Use ``is True`` (not ``bool(...)``) so MagicMock attribute access
|
||||
# in tests doesn't incorrectly enable this path.
|
||||
self._adapter_requires_finalize: bool = (
|
||||
getattr(adapter, "REQUIRES_EDIT_FINALIZE", False) is True
|
||||
)
|
||||
|
||||
# Think-block filter state (mirrors CLI's _stream_delta tag suppression)
|
||||
self._in_think_block = False
|
||||
@@ -369,16 +361,7 @@ class GatewayStreamConsumer:
|
||||
if not got_done and not got_segment_break and commentary_text is None:
|
||||
display_text += self.cfg.cursor
|
||||
|
||||
# Segment break: finalize the current message so platforms
|
||||
# that need explicit closure (e.g. DingTalk AI Cards) don't
|
||||
# leave the previous segment stuck in a loading state when
|
||||
# the next segment (tool progress, next chunk) creates a
|
||||
# new message below it. got_done has its own finalize
|
||||
# path below so we don't finalize here for it.
|
||||
current_update_visible = await self._send_or_edit(
|
||||
display_text,
|
||||
finalize=got_segment_break,
|
||||
)
|
||||
current_update_visible = await self._send_or_edit(display_text)
|
||||
self._last_edit_time = time.monotonic()
|
||||
|
||||
if got_done:
|
||||
@@ -389,22 +372,10 @@ class GatewayStreamConsumer:
|
||||
if self._accumulated:
|
||||
if self._fallback_final_send:
|
||||
await self._send_fallback_final(self._accumulated)
|
||||
elif (
|
||||
current_update_visible
|
||||
and not self._adapter_requires_finalize
|
||||
):
|
||||
# Mid-stream edit above already delivered the
|
||||
# final accumulated content. Skip the redundant
|
||||
# final edit — but only for adapters that don't
|
||||
# need an explicit finalize signal.
|
||||
elif current_update_visible:
|
||||
self._final_response_sent = True
|
||||
elif self._message_id:
|
||||
# Either the mid-stream edit didn't run (no
|
||||
# visible update this tick) OR the adapter needs
|
||||
# explicit finalize=True to close the stream.
|
||||
self._final_response_sent = await self._send_or_edit(
|
||||
self._accumulated, finalize=True,
|
||||
)
|
||||
self._final_response_sent = await self._send_or_edit(self._accumulated)
|
||||
elif not self._already_sent:
|
||||
self._final_response_sent = await self._send_or_edit(self._accumulated)
|
||||
return
|
||||
@@ -430,21 +401,6 @@ class GatewayStreamConsumer:
|
||||
# a real string like "msg_1", not "__no_edit__", so that case
|
||||
# still resets and creates a fresh segment as intended.)
|
||||
if got_segment_break:
|
||||
# If the segment-break edit failed to deliver the
|
||||
# accumulated content (flood control that has not yet
|
||||
# promoted to fallback mode, or fallback mode itself),
|
||||
# _accumulated still holds pre-boundary text the user
|
||||
# never saw. Flush that tail as a continuation message
|
||||
# before the reset below wipes _accumulated — otherwise
|
||||
# text generated before the tool boundary is silently
|
||||
# dropped (issue #8124).
|
||||
if (
|
||||
self._accumulated
|
||||
and not current_update_visible
|
||||
and self._message_id
|
||||
and self._message_id != "__no_edit__"
|
||||
):
|
||||
await self._flush_segment_tail_on_edit_failure()
|
||||
self._reset_segment_state(preserve_no_edit=True)
|
||||
|
||||
await asyncio.sleep(0.05) # Small yield to not busy-loop
|
||||
@@ -635,39 +591,6 @@ class GatewayStreamConsumer:
|
||||
err_lower = err.lower()
|
||||
return "flood" in err_lower or "retry after" in err_lower or "rate" in err_lower
|
||||
|
||||
async def _flush_segment_tail_on_edit_failure(self) -> None:
|
||||
"""Deliver un-sent tail content before a segment-break reset.
|
||||
|
||||
When an edit fails (flood control, transport error) and a tool
|
||||
boundary arrives before the next retry, ``_accumulated`` holds text
|
||||
that was generated but never shown to the user. Without this flush,
|
||||
the segment reset would discard that tail and leave a frozen cursor
|
||||
in the partial message.
|
||||
|
||||
Sends the tail that sits after the last successfully-delivered
|
||||
prefix as a new message, and best-effort strips the stuck cursor
|
||||
from the previous partial message.
|
||||
"""
|
||||
if not self._fallback_final_send:
|
||||
await self._try_strip_cursor()
|
||||
visible = self._fallback_prefix or self._visible_prefix()
|
||||
tail = self._accumulated
|
||||
if visible and tail.startswith(visible):
|
||||
tail = tail[len(visible):].lstrip()
|
||||
tail = self._clean_for_display(tail)
|
||||
if not tail.strip():
|
||||
return
|
||||
try:
|
||||
result = await self.adapter.send(
|
||||
chat_id=self.chat_id,
|
||||
content=tail,
|
||||
metadata=self.metadata,
|
||||
)
|
||||
if result.success:
|
||||
self._already_sent = True
|
||||
except Exception as e:
|
||||
logger.error("Segment-break tail flush error: %s", e)
|
||||
|
||||
async def _try_strip_cursor(self) -> None:
|
||||
"""Best-effort edit to remove the cursor from the last visible message.
|
||||
|
||||
@@ -710,15 +633,12 @@ class GatewayStreamConsumer:
|
||||
logger.error("Commentary send error: %s", e)
|
||||
return False
|
||||
|
||||
async def _send_or_edit(self, text: str, *, finalize: bool = False) -> bool:
|
||||
async def _send_or_edit(self, text: str) -> bool:
|
||||
"""Send or edit the streaming message.
|
||||
|
||||
Returns True if the text was successfully delivered (sent or edited),
|
||||
False otherwise. Callers like the overflow split loop use this to
|
||||
decide whether to advance past the delivered chunk.
|
||||
|
||||
``finalize`` is True when this is the last edit in a streaming
|
||||
sequence.
|
||||
"""
|
||||
# Strip MEDIA: directives so they don't appear as visible text.
|
||||
# Media files are delivered as native attachments after the stream
|
||||
@@ -752,22 +672,14 @@ class GatewayStreamConsumer:
|
||||
try:
|
||||
if self._message_id is not None:
|
||||
if self._edit_supported:
|
||||
# Skip if text is identical to what we last sent.
|
||||
# Exception: adapters that require an explicit finalize
|
||||
# call (REQUIRES_EDIT_FINALIZE) must still receive the
|
||||
# finalize=True edit even when content is unchanged, so
|
||||
# their streaming UI can transition out of the in-
|
||||
# progress state. Everyone else short-circuits.
|
||||
if text == self._last_sent_text and not (
|
||||
finalize and self._adapter_requires_finalize
|
||||
):
|
||||
# Skip if text is identical to what we last sent
|
||||
if text == self._last_sent_text:
|
||||
return True
|
||||
# Edit existing message
|
||||
result = await self.adapter.edit_message(
|
||||
chat_id=self.chat_id,
|
||||
message_id=self._message_id,
|
||||
content=text,
|
||||
finalize=finalize,
|
||||
)
|
||||
if result.success:
|
||||
self._already_sent = True
|
||||
|
||||
+68
-92
@@ -233,14 +233,6 @@ PROVIDER_REGISTRY: Dict[str, ProviderConfig] = {
|
||||
api_key_env_vars=("XAI_API_KEY",),
|
||||
base_url_env_var="XAI_BASE_URL",
|
||||
),
|
||||
"nvidia": ProviderConfig(
|
||||
id="nvidia",
|
||||
name="NVIDIA NIM",
|
||||
auth_type="api_key",
|
||||
inference_base_url="https://integrate.api.nvidia.com/v1",
|
||||
api_key_env_vars=("NVIDIA_API_KEY",),
|
||||
base_url_env_var="NVIDIA_BASE_URL",
|
||||
),
|
||||
"ai-gateway": ProviderConfig(
|
||||
id="ai-gateway",
|
||||
name="Vercel AI Gateway",
|
||||
@@ -781,28 +773,6 @@ def is_source_suppressed(provider_id: str, source: str) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def unsuppress_credential_source(provider_id: str, source: str) -> bool:
|
||||
"""Clear a suppression marker so the source will be re-seeded on the next load.
|
||||
|
||||
Returns True if a marker was cleared, False if no marker existed.
|
||||
"""
|
||||
with _auth_store_lock():
|
||||
auth_store = _load_auth_store()
|
||||
suppressed = auth_store.get("suppressed_sources")
|
||||
if not isinstance(suppressed, dict):
|
||||
return False
|
||||
provider_list = suppressed.get(provider_id)
|
||||
if not isinstance(provider_list, list) or source not in provider_list:
|
||||
return False
|
||||
provider_list.remove(source)
|
||||
if not provider_list:
|
||||
suppressed.pop(provider_id, None)
|
||||
if not suppressed:
|
||||
auth_store.pop("suppressed_sources", None)
|
||||
_save_auth_store(auth_store)
|
||||
return True
|
||||
|
||||
|
||||
def get_provider_auth_state(provider_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""Return persisted auth state for a provider, or None."""
|
||||
auth_store = _load_auth_store()
|
||||
@@ -1434,6 +1404,49 @@ def _read_codex_tokens(*, _lock: bool = True) -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _write_codex_cli_tokens(
|
||||
access_token: str,
|
||||
refresh_token: str,
|
||||
*,
|
||||
last_refresh: Optional[str] = None,
|
||||
) -> None:
|
||||
"""Write refreshed tokens back to ~/.codex/auth.json.
|
||||
|
||||
OpenAI OAuth refresh tokens are single-use and rotate on every refresh.
|
||||
When Hermes refreshes a token it consumes the old refresh_token; if we
|
||||
don't write the new pair back, the Codex CLI (or VS Code extension) will
|
||||
fail with ``refresh_token_reused`` on its next refresh attempt.
|
||||
|
||||
This mirrors the Anthropic write-back to ~/.claude/.credentials.json
|
||||
via ``_write_claude_code_credentials()``.
|
||||
"""
|
||||
codex_home = os.getenv("CODEX_HOME", "").strip()
|
||||
if not codex_home:
|
||||
codex_home = str(Path.home() / ".codex")
|
||||
auth_path = Path(codex_home).expanduser() / "auth.json"
|
||||
try:
|
||||
existing: Dict[str, Any] = {}
|
||||
if auth_path.is_file():
|
||||
existing = json.loads(auth_path.read_text(encoding="utf-8"))
|
||||
if not isinstance(existing, dict):
|
||||
existing = {}
|
||||
|
||||
tokens_dict = existing.get("tokens")
|
||||
if not isinstance(tokens_dict, dict):
|
||||
tokens_dict = {}
|
||||
tokens_dict["access_token"] = access_token
|
||||
tokens_dict["refresh_token"] = refresh_token
|
||||
existing["tokens"] = tokens_dict
|
||||
if last_refresh is not None:
|
||||
existing["last_refresh"] = last_refresh
|
||||
|
||||
auth_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
auth_path.write_text(json.dumps(existing, indent=2), encoding="utf-8")
|
||||
auth_path.chmod(0o600)
|
||||
except (OSError, IOError) as exc:
|
||||
logger.debug("Failed to write refreshed tokens to %s: %s", auth_path, exc)
|
||||
|
||||
|
||||
def _save_codex_tokens(tokens: Dict[str, str], last_refresh: str = None) -> None:
|
||||
"""Save Codex OAuth tokens to Hermes auth store (~/.hermes/auth.json)."""
|
||||
if last_refresh is None:
|
||||
@@ -1501,11 +1514,6 @@ def refresh_codex_oauth_pure(
|
||||
"then run `hermes auth` to re-authenticate."
|
||||
)
|
||||
relogin_required = True
|
||||
# A 401/403 from the token endpoint always means the refresh token
|
||||
# is invalid/expired — force relogin even if the body error code
|
||||
# wasn't one of the known strings above.
|
||||
if response.status_code in (401, 403) and not relogin_required:
|
||||
relogin_required = True
|
||||
raise AuthError(
|
||||
message,
|
||||
provider="openai-codex",
|
||||
@@ -1561,6 +1569,12 @@ def _refresh_codex_auth_tokens(
|
||||
updated_tokens["refresh_token"] = refreshed["refresh_token"]
|
||||
|
||||
_save_codex_tokens(updated_tokens)
|
||||
# Write back to ~/.codex/auth.json so Codex CLI / VS Code stay in sync.
|
||||
_write_codex_cli_tokens(
|
||||
refreshed["access_token"],
|
||||
refreshed["refresh_token"],
|
||||
last_refresh=refreshed.get("last_refresh"),
|
||||
)
|
||||
return updated_tokens
|
||||
|
||||
|
||||
@@ -1605,7 +1619,25 @@ def resolve_codex_runtime_credentials(
|
||||
refresh_skew_seconds: int = CODEX_ACCESS_TOKEN_REFRESH_SKEW_SECONDS,
|
||||
) -> Dict[str, Any]:
|
||||
"""Resolve runtime credentials from Hermes's own Codex token store."""
|
||||
data = _read_codex_tokens()
|
||||
try:
|
||||
data = _read_codex_tokens()
|
||||
except AuthError as orig_err:
|
||||
# Only attempt migration when there are NO tokens stored at all
|
||||
# (code == "codex_auth_missing"), not when tokens exist but are invalid.
|
||||
if orig_err.code != "codex_auth_missing":
|
||||
raise
|
||||
|
||||
# Migration: user had Codex as active provider with old storage (~/.codex/).
|
||||
cli_tokens = _import_codex_cli_tokens()
|
||||
if cli_tokens:
|
||||
logger.info("Migrating Codex credentials from ~/.codex/ to Hermes auth store")
|
||||
print("⚠️ Migrating Codex credentials to Hermes's own auth store.")
|
||||
print(" This avoids conflicts with Codex CLI and VS Code.")
|
||||
print(" Run `hermes auth` to create a fully independent session.\n")
|
||||
_save_codex_tokens(cli_tokens)
|
||||
data = _read_codex_tokens()
|
||||
else:
|
||||
raise
|
||||
tokens = dict(data["tokens"])
|
||||
access_token = str(tokens.get("access_token", "") or "").strip()
|
||||
refresh_timeout_seconds = float(os.getenv("HERMES_CODEX_REFRESH_TIMEOUT_SECONDS", "20"))
|
||||
@@ -2097,62 +2129,6 @@ def refresh_nous_oauth_from_state(
|
||||
)
|
||||
|
||||
|
||||
NOUS_DEVICE_CODE_SOURCE = "device_code"
|
||||
|
||||
|
||||
def persist_nous_credentials(
|
||||
creds: Dict[str, Any],
|
||||
*,
|
||||
label: Optional[str] = None,
|
||||
):
|
||||
"""Persist minted Nous OAuth credentials as the singleton provider state
|
||||
and ensure the credential pool is in sync.
|
||||
|
||||
Nous credentials are read at runtime from two independent locations:
|
||||
|
||||
- ``providers.nous``: singleton state read by
|
||||
``resolve_nous_runtime_credentials()`` during 401 recovery and by
|
||||
``_seed_from_singletons()`` during pool load.
|
||||
- ``credential_pool.nous``: used by the runtime ``pool.select()`` path.
|
||||
|
||||
Historically ``hermes auth add nous`` wrote a ``manual:device_code`` pool
|
||||
entry only, skipping ``providers.nous``. When the 24h agent_key TTL
|
||||
expired, the recovery path read the empty singleton state and raised
|
||||
``AuthError`` silently (``logger.debug`` at INFO level).
|
||||
|
||||
This helper writes ``providers.nous`` then calls ``load_pool("nous")`` so
|
||||
``_seed_from_singletons`` materialises the canonical ``device_code`` pool
|
||||
entry from the singleton. Re-running login upserts the same entry in
|
||||
place; the pool never accumulates duplicate device_code rows.
|
||||
|
||||
``label`` is an optional user-chosen display name (from
|
||||
``hermes auth add nous --label <name>``). It gets embedded in the
|
||||
singleton state so that ``_seed_from_singletons`` uses it as the pool
|
||||
entry's label on every subsequent ``load_pool("nous")`` instead of the
|
||||
auto-derived token fingerprint. When ``None``, the auto-derived label
|
||||
via ``label_from_token`` is used (unchanged default behaviour).
|
||||
|
||||
Returns the upserted :class:`PooledCredential` entry (or ``None`` if
|
||||
seeding somehow produced no match — shouldn't happen).
|
||||
"""
|
||||
from agent.credential_pool import load_pool
|
||||
|
||||
state = dict(creds)
|
||||
if label and str(label).strip():
|
||||
state["label"] = str(label).strip()
|
||||
|
||||
with _auth_store_lock():
|
||||
auth_store = _load_auth_store()
|
||||
_save_provider_state(auth_store, "nous", state)
|
||||
_save_auth_store(auth_store)
|
||||
|
||||
pool = load_pool("nous")
|
||||
return next(
|
||||
(e for e in pool.entries() if e.source == NOUS_DEVICE_CODE_SOURCE),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def resolve_nous_runtime_credentials(
|
||||
*,
|
||||
min_key_ttl_seconds: int = DEFAULT_AGENT_KEY_MIN_TTL_SECONDS,
|
||||
|
||||
+13
-39
@@ -217,21 +217,22 @@ def auth_add_command(args) -> None:
|
||||
ca_bundle=getattr(args, "ca_bundle", None),
|
||||
min_key_ttl_seconds=max(60, int(getattr(args, "min_key_ttl_seconds", 5 * 60))),
|
||||
)
|
||||
# Honor `--label <name>` so nous matches other providers' UX. The
|
||||
# helper embeds this into providers.nous so that label_from_token
|
||||
# doesn't overwrite it on every subsequent load_pool("nous").
|
||||
custom_label = (getattr(args, "label", None) or "").strip() or None
|
||||
entry = auth_mod.persist_nous_credentials(creds, label=custom_label)
|
||||
shown_label = entry.label if entry is not None else label_from_token(
|
||||
creds.get("access_token", ""), _oauth_default_label(provider, 1),
|
||||
label = (getattr(args, "label", None) or "").strip() or label_from_token(
|
||||
creds.get("access_token", ""),
|
||||
_oauth_default_label(provider, len(pool.entries()) + 1),
|
||||
)
|
||||
print(f'Saved {provider} OAuth device-code credentials: "{shown_label}"')
|
||||
entry = PooledCredential.from_dict(provider, {
|
||||
**creds,
|
||||
"label": label,
|
||||
"auth_type": AUTH_TYPE_OAUTH,
|
||||
"source": f"{SOURCE_MANUAL}:device_code",
|
||||
"base_url": creds.get("inference_base_url"),
|
||||
})
|
||||
pool.add_entry(entry)
|
||||
print(f'Added {provider} OAuth credential #{len(pool.entries())}: "{entry.label}"')
|
||||
return
|
||||
|
||||
if provider == "openai-codex":
|
||||
# Clear any existing suppression marker so a re-link after `hermes auth
|
||||
# remove openai-codex` works without the new tokens being skipped.
|
||||
auth_mod.unsuppress_credential_source(provider, "device_code")
|
||||
creds = auth_mod._codex_device_code_login()
|
||||
label = (getattr(args, "label", None) or "").strip() or label_from_token(
|
||||
creds["tokens"]["access_token"],
|
||||
@@ -351,34 +352,7 @@ def auth_remove_command(args) -> None:
|
||||
# If this was a singleton-seeded credential (OAuth device_code, hermes_pkce),
|
||||
# clear the underlying auth store / credential file so it doesn't get
|
||||
# re-seeded on the next load_pool() call.
|
||||
elif provider == "openai-codex" and (
|
||||
removed.source == "device_code" or removed.source.endswith(":device_code")
|
||||
):
|
||||
# Codex tokens live in TWO places: the Hermes auth store and
|
||||
# ~/.codex/auth.json (the Codex CLI shared file). On every refresh,
|
||||
# refresh_codex_oauth_pure() writes to both. So clearing only the
|
||||
# Hermes auth store is not enough — _seed_from_singletons() will
|
||||
# auto-import from ~/.codex/auth.json on the next load_pool() and
|
||||
# the removal is instantly undone. Mark the source as suppressed
|
||||
# so auto-import is skipped; leave ~/.codex/auth.json untouched so
|
||||
# the Codex CLI itself keeps working.
|
||||
from hermes_cli.auth import (
|
||||
_load_auth_store, _save_auth_store, _auth_store_lock,
|
||||
suppress_credential_source,
|
||||
)
|
||||
with _auth_store_lock():
|
||||
auth_store = _load_auth_store()
|
||||
providers_dict = auth_store.get("providers")
|
||||
if isinstance(providers_dict, dict) and provider in providers_dict:
|
||||
del providers_dict[provider]
|
||||
_save_auth_store(auth_store)
|
||||
print(f"Cleared {provider} OAuth tokens from auth store")
|
||||
suppress_credential_source(provider, "device_code")
|
||||
print("Suppressed openai-codex device_code source — it will not be re-seeded.")
|
||||
print("Note: Codex CLI credentials still live in ~/.codex/auth.json")
|
||||
print("Run `hermes auth add openai-codex` to re-enable if needed.")
|
||||
|
||||
elif removed.source == "device_code" and provider == "nous":
|
||||
elif removed.source == "device_code" and provider in ("openai-codex", "nous"):
|
||||
from hermes_cli.auth import (
|
||||
_load_auth_store, _save_auth_store, _auth_store_lock,
|
||||
)
|
||||
|
||||
+70
-119
@@ -7,8 +7,8 @@ CLI tools that ship with the platform (or are commonly installed).
|
||||
|
||||
Platform support:
|
||||
macOS — osascript (always available), pngpaste (if installed)
|
||||
Windows — PowerShell via WinForms, Get-Clipboard, file-drop fallback
|
||||
WSL2 — powershell.exe via WinForms, Get-Clipboard, file-drop fallback
|
||||
Windows — PowerShell via .NET System.Windows.Forms.Clipboard
|
||||
WSL2 — powershell.exe via .NET System.Windows.Forms.Clipboard
|
||||
Linux — wl-paste (Wayland), xclip (X11)
|
||||
"""
|
||||
|
||||
@@ -46,11 +46,10 @@ def has_clipboard_image() -> bool:
|
||||
return _macos_has_image()
|
||||
if sys.platform == "win32":
|
||||
return _windows_has_image()
|
||||
# Match _linux_save fallthrough order: WSL → Wayland → X11
|
||||
if _is_wsl() and _wsl_has_image():
|
||||
return True
|
||||
if os.environ.get("WAYLAND_DISPLAY") and _wayland_has_image():
|
||||
return True
|
||||
if _is_wsl():
|
||||
return _wsl_has_image()
|
||||
if os.environ.get("WAYLAND_DISPLAY"):
|
||||
return _wayland_has_image()
|
||||
return _xclip_has_image()
|
||||
|
||||
|
||||
@@ -136,114 +135,6 @@ _PS_EXTRACT_IMAGE = (
|
||||
"[System.Convert]::ToBase64String($ms.ToArray())"
|
||||
)
|
||||
|
||||
_PS_CHECK_IMAGE_GET_CLIPBOARD = (
|
||||
"try { "
|
||||
"$img = Get-Clipboard -Format Image -ErrorAction Stop;"
|
||||
"if ($null -ne $img) { 'True' } else { 'False' }"
|
||||
"} catch { 'False' }"
|
||||
)
|
||||
|
||||
_PS_EXTRACT_IMAGE_GET_CLIPBOARD = (
|
||||
"try { "
|
||||
"Add-Type -AssemblyName System.Drawing;"
|
||||
"Add-Type -AssemblyName PresentationCore;"
|
||||
"Add-Type -AssemblyName WindowsBase;"
|
||||
"$img = Get-Clipboard -Format Image -ErrorAction Stop;"
|
||||
"if ($null -eq $img) { exit 1 }"
|
||||
"$ms = New-Object System.IO.MemoryStream;"
|
||||
"if ($img -is [System.Drawing.Image]) {"
|
||||
"$img.Save($ms, [System.Drawing.Imaging.ImageFormat]::Png)"
|
||||
"} elseif ($img -is [System.Windows.Media.Imaging.BitmapSource]) {"
|
||||
"$enc = New-Object System.Windows.Media.Imaging.PngBitmapEncoder;"
|
||||
"$enc.Frames.Add([System.Windows.Media.Imaging.BitmapFrame]::Create($img));"
|
||||
"$enc.Save($ms)"
|
||||
"} else { exit 2 }"
|
||||
"[System.Convert]::ToBase64String($ms.ToArray())"
|
||||
"} catch { exit 1 }"
|
||||
)
|
||||
|
||||
_FILEDROP_IMAGE_EXTS = "'.png','.jpg','.jpeg','.gif','.webp','.bmp','.tiff','.tif'"
|
||||
|
||||
_PS_CHECK_FILEDROP_IMAGE = (
|
||||
"try { "
|
||||
"$files = Get-Clipboard -Format FileDropList -ErrorAction Stop;"
|
||||
f"$exts = @({_FILEDROP_IMAGE_EXTS});"
|
||||
"$hit = $files | Where-Object { $exts -contains ([System.IO.Path]::GetExtension($_).ToLowerInvariant()) } | Select-Object -First 1;"
|
||||
"if ($null -ne $hit) { 'True' } else { 'False' }"
|
||||
"} catch { 'False' }"
|
||||
)
|
||||
|
||||
_PS_EXTRACT_FILEDROP_IMAGE = (
|
||||
"try { "
|
||||
"$files = Get-Clipboard -Format FileDropList -ErrorAction Stop;"
|
||||
f"$exts = @({_FILEDROP_IMAGE_EXTS});"
|
||||
"$hit = $files | Where-Object { $exts -contains ([System.IO.Path]::GetExtension($_).ToLowerInvariant()) } | Select-Object -First 1;"
|
||||
"if ($null -eq $hit) { exit 1 }"
|
||||
"[System.Convert]::ToBase64String([System.IO.File]::ReadAllBytes($hit))"
|
||||
"} catch { exit 1 }"
|
||||
)
|
||||
|
||||
_POWERSHELL_HAS_IMAGE_SCRIPTS = (
|
||||
_PS_CHECK_IMAGE,
|
||||
_PS_CHECK_IMAGE_GET_CLIPBOARD,
|
||||
_PS_CHECK_FILEDROP_IMAGE,
|
||||
)
|
||||
|
||||
_POWERSHELL_EXTRACT_IMAGE_SCRIPTS = (
|
||||
_PS_EXTRACT_IMAGE,
|
||||
_PS_EXTRACT_IMAGE_GET_CLIPBOARD,
|
||||
_PS_EXTRACT_FILEDROP_IMAGE,
|
||||
)
|
||||
|
||||
|
||||
def _run_powershell(exe: str, script: str, timeout: int) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
[exe, "-NoProfile", "-NonInteractive", "-Command", script],
|
||||
capture_output=True, text=True, timeout=timeout,
|
||||
)
|
||||
|
||||
|
||||
def _write_base64_image(dest: Path, b64_data: str) -> bool:
|
||||
image_bytes = base64.b64decode(b64_data, validate=True)
|
||||
dest.write_bytes(image_bytes)
|
||||
return dest.exists() and dest.stat().st_size > 0
|
||||
|
||||
|
||||
def _powershell_has_image(exe: str, *, timeout: int, label: str) -> bool:
|
||||
for script in _POWERSHELL_HAS_IMAGE_SCRIPTS:
|
||||
try:
|
||||
r = _run_powershell(exe, script, timeout=timeout)
|
||||
if r.returncode == 0 and "True" in r.stdout:
|
||||
return True
|
||||
except FileNotFoundError:
|
||||
logger.debug("%s not found — clipboard unavailable", exe)
|
||||
return False
|
||||
except Exception as e:
|
||||
logger.debug("%s clipboard image check failed: %s", label, e)
|
||||
return False
|
||||
|
||||
|
||||
def _powershell_save_image(exe: str, dest: Path, *, timeout: int, label: str) -> bool:
|
||||
for script in _POWERSHELL_EXTRACT_IMAGE_SCRIPTS:
|
||||
try:
|
||||
r = _run_powershell(exe, script, timeout=timeout)
|
||||
if r.returncode != 0:
|
||||
continue
|
||||
|
||||
b64_data = r.stdout.strip()
|
||||
if not b64_data:
|
||||
continue
|
||||
|
||||
if _write_base64_image(dest, b64_data):
|
||||
return True
|
||||
except FileNotFoundError:
|
||||
logger.debug("%s not found — clipboard unavailable", exe)
|
||||
return False
|
||||
except Exception as e:
|
||||
logger.debug("%s clipboard image extraction failed: %s", label, e)
|
||||
dest.unlink(missing_ok=True)
|
||||
return False
|
||||
|
||||
|
||||
# ── Native Windows ────────────────────────────────────────────────────────
|
||||
|
||||
@@ -284,7 +175,15 @@ def _windows_has_image() -> bool:
|
||||
ps = _get_ps_exe()
|
||||
if ps is None:
|
||||
return False
|
||||
return _powershell_has_image(ps, timeout=5, label="Windows")
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[ps, "-NoProfile", "-NonInteractive", "-Command", _PS_CHECK_IMAGE],
|
||||
capture_output=True, text=True, timeout=5,
|
||||
)
|
||||
return r.returncode == 0 and "True" in r.stdout
|
||||
except Exception as e:
|
||||
logger.debug("Windows clipboard image check failed: %s", e)
|
||||
return False
|
||||
|
||||
|
||||
def _windows_save(dest: Path) -> bool:
|
||||
@@ -293,7 +192,26 @@ def _windows_save(dest: Path) -> bool:
|
||||
if ps is None:
|
||||
logger.debug("No PowerShell found — Windows clipboard image paste unavailable")
|
||||
return False
|
||||
return _powershell_save_image(ps, dest, timeout=15, label="Windows")
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[ps, "-NoProfile", "-NonInteractive", "-Command", _PS_EXTRACT_IMAGE],
|
||||
capture_output=True, text=True, timeout=15,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
return False
|
||||
|
||||
b64_data = r.stdout.strip()
|
||||
if not b64_data:
|
||||
return False
|
||||
|
||||
png_bytes = base64.b64decode(b64_data)
|
||||
dest.write_bytes(png_bytes)
|
||||
return dest.exists() and dest.stat().st_size > 0
|
||||
|
||||
except Exception as e:
|
||||
logger.debug("Windows clipboard image extraction failed: %s", e)
|
||||
dest.unlink(missing_ok=True)
|
||||
return False
|
||||
|
||||
|
||||
# ── Linux ────────────────────────────────────────────────────────────────
|
||||
@@ -317,12 +235,45 @@ def _linux_save(dest: Path) -> bool:
|
||||
|
||||
def _wsl_has_image() -> bool:
|
||||
"""Check if Windows clipboard has an image (via powershell.exe)."""
|
||||
return _powershell_has_image("powershell.exe", timeout=8, label="WSL")
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["powershell.exe", "-NoProfile", "-NonInteractive", "-Command",
|
||||
_PS_CHECK_IMAGE],
|
||||
capture_output=True, text=True, timeout=8,
|
||||
)
|
||||
return r.returncode == 0 and "True" in r.stdout
|
||||
except FileNotFoundError:
|
||||
logger.debug("powershell.exe not found — WSL clipboard unavailable")
|
||||
except Exception as e:
|
||||
logger.debug("WSL clipboard check failed: %s", e)
|
||||
return False
|
||||
|
||||
|
||||
def _wsl_save(dest: Path) -> bool:
|
||||
"""Extract clipboard image via powershell.exe → base64 → decode to PNG."""
|
||||
return _powershell_save_image("powershell.exe", dest, timeout=15, label="WSL")
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["powershell.exe", "-NoProfile", "-NonInteractive", "-Command",
|
||||
_PS_EXTRACT_IMAGE],
|
||||
capture_output=True, text=True, timeout=15,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
return False
|
||||
|
||||
b64_data = r.stdout.strip()
|
||||
if not b64_data:
|
||||
return False
|
||||
|
||||
png_bytes = base64.b64decode(b64_data)
|
||||
dest.write_bytes(png_bytes)
|
||||
return dest.exists() and dest.stat().st_size > 0
|
||||
|
||||
except FileNotFoundError:
|
||||
logger.debug("powershell.exe not found — WSL clipboard unavailable")
|
||||
except Exception as e:
|
||||
logger.debug("WSL clipboard extraction failed: %s", e)
|
||||
dest.unlink(missing_ok=True)
|
||||
return False
|
||||
|
||||
|
||||
# ── Wayland (wl-paste) ──────────────────────────────────────────────────
|
||||
|
||||
+7
-112
@@ -87,12 +87,8 @@ COMMAND_REGISTRY: list[CommandDef] = [
|
||||
aliases=("bg",), args_hint="<prompt>"),
|
||||
CommandDef("btw", "Ephemeral side question using session context (no tools, not persisted)", "Session",
|
||||
args_hint="<question>"),
|
||||
CommandDef("agents", "Show active agents and running tasks", "Session",
|
||||
aliases=("tasks",)),
|
||||
CommandDef("queue", "Queue a prompt for the next turn (doesn't interrupt)", "Session",
|
||||
aliases=("q",), args_hint="<prompt>"),
|
||||
CommandDef("steer", "Inject a message after the next tool call without interrupting", "Session",
|
||||
args_hint="<prompt>"),
|
||||
CommandDef("status", "Show session info", "Session"),
|
||||
CommandDef("profile", "Show active profile name and home directory", "Info"),
|
||||
CommandDef("sethome", "Set this chat as the home channel", "Session",
|
||||
@@ -103,7 +99,7 @@ COMMAND_REGISTRY: list[CommandDef] = [
|
||||
# Configuration
|
||||
CommandDef("config", "Show current configuration", "Configuration",
|
||||
cli_only=True),
|
||||
CommandDef("model", "Switch model for this session", "Configuration", args_hint="[model] [--provider name] [--global]"),
|
||||
CommandDef("model", "Switch model for this session", "Configuration", args_hint="[model] [--global]"),
|
||||
CommandDef("provider", "Show available providers and current provider",
|
||||
"Configuration"),
|
||||
CommandDef("gquota", "Show Google Gemini Code Assist quota usage", "Info"),
|
||||
@@ -124,7 +120,7 @@ COMMAND_REGISTRY: list[CommandDef] = [
|
||||
args_hint="[normal|fast|status]",
|
||||
subcommands=("normal", "fast", "status", "on", "off")),
|
||||
CommandDef("skin", "Show or change the display skin/theme", "Configuration",
|
||||
args_hint="[name]"),
|
||||
cli_only=True, args_hint="[name]"),
|
||||
CommandDef("voice", "Toggle voice mode", "Configuration",
|
||||
args_hint="[on|off|tts|status]", subcommands=("on", "off", "tts", "status")),
|
||||
|
||||
@@ -159,9 +155,7 @@ COMMAND_REGISTRY: list[CommandDef] = [
|
||||
args_hint="[days]"),
|
||||
CommandDef("platforms", "Show gateway/messaging platform status", "Info",
|
||||
cli_only=True, aliases=("gateway",)),
|
||||
CommandDef("copy", "Copy the last assistant response to clipboard", "Info",
|
||||
cli_only=True, args_hint="[number]"),
|
||||
CommandDef("paste", "Attach clipboard image from your clipboard", "Info",
|
||||
CommandDef("paste", "Check clipboard for an image and attach it", "Info",
|
||||
cli_only=True),
|
||||
CommandDef("image", "Attach a local image file for your next prompt", "Info",
|
||||
cli_only=True, args_hint="<path>"),
|
||||
@@ -260,53 +254,6 @@ GATEWAY_KNOWN_COMMANDS: frozenset[str] = frozenset(
|
||||
)
|
||||
|
||||
|
||||
# Commands with explicit Level-2 running-agent handlers in gateway/run.py.
|
||||
# Listed here for introspection / tests; semantically a subset of
|
||||
# "all resolvable commands" — which is the real bypass set (see
|
||||
# should_bypass_active_session below).
|
||||
ACTIVE_SESSION_BYPASS_COMMANDS: frozenset[str] = frozenset(
|
||||
{
|
||||
"agents",
|
||||
"approve",
|
||||
"background",
|
||||
"commands",
|
||||
"deny",
|
||||
"help",
|
||||
"new",
|
||||
"profile",
|
||||
"queue",
|
||||
"restart",
|
||||
"status",
|
||||
"steer",
|
||||
"stop",
|
||||
"update",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def should_bypass_active_session(command_name: str | None) -> bool:
|
||||
"""Return True for any resolvable slash command.
|
||||
|
||||
Rationale: every gateway-registered slash command either has a
|
||||
specific Level-2 handler in gateway/run.py (/stop, /new, /model,
|
||||
/approve, etc.) or reaches the running-agent catch-all that returns
|
||||
a "busy — wait or /stop first" response. In both paths the command
|
||||
is dispatched, not queued.
|
||||
|
||||
Queueing is always wrong for a recognized slash command because the
|
||||
safety net in gateway.run discards any command text that reaches
|
||||
the pending queue — which meant a mid-run /model (or /reasoning,
|
||||
/voice, /insights, /title, /resume, /retry, /undo, /compress,
|
||||
/usage, /provider, /reload-mcp, /sethome, /reset) would silently
|
||||
interrupt the agent AND get discarded, producing a zero-char
|
||||
response. See issue #5057 / PRs #6252, #10370, #4665.
|
||||
|
||||
ACTIVE_SESSION_BYPASS_COMMANDS remains the subset of commands with
|
||||
explicit Level-2 handlers; the rest fall through to the catch-all.
|
||||
"""
|
||||
return resolve_command(command_name) is not None if command_name else False
|
||||
|
||||
|
||||
def _resolve_config_gates() -> set[str]:
|
||||
"""Return canonical names of commands whose ``gateway_config_gate`` is truthy.
|
||||
|
||||
@@ -1097,51 +1044,6 @@ class SlashCommandCompleter(Completer):
|
||||
display_meta=f"{fp} {meta}" if meta else fp,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _skin_completions(sub_text: str, sub_lower: str):
|
||||
"""Yield completions for /skin from available skins."""
|
||||
try:
|
||||
from hermes_cli.skin_engine import list_skins
|
||||
for s in list_skins():
|
||||
name = s["name"]
|
||||
if name.startswith(sub_lower) and name != sub_lower:
|
||||
yield Completion(
|
||||
name,
|
||||
start_position=-len(sub_text),
|
||||
display=name,
|
||||
display_meta=s.get("description", "") or s.get("source", ""),
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def _personality_completions(sub_text: str, sub_lower: str):
|
||||
"""Yield completions for /personality from configured personalities."""
|
||||
try:
|
||||
from hermes_cli.config import load_config
|
||||
personalities = load_config().get("agent", {}).get("personalities", {})
|
||||
if "none".startswith(sub_lower) and "none" != sub_lower:
|
||||
yield Completion(
|
||||
"none",
|
||||
start_position=-len(sub_text),
|
||||
display="none",
|
||||
display_meta="clear personality overlay",
|
||||
)
|
||||
for name, prompt in personalities.items():
|
||||
if name.startswith(sub_lower) and name != sub_lower:
|
||||
if isinstance(prompt, dict):
|
||||
meta = prompt.get("description") or prompt.get("system_prompt", "")[:50]
|
||||
else:
|
||||
meta = str(prompt)[:50]
|
||||
yield Completion(
|
||||
name,
|
||||
start_position=-len(sub_text),
|
||||
display=name,
|
||||
display_meta=meta,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _model_completions(self, sub_text: str, sub_lower: str):
|
||||
"""Yield completions for /model from config aliases + built-in aliases."""
|
||||
seen = set()
|
||||
@@ -1196,17 +1098,10 @@ class SlashCommandCompleter(Completer):
|
||||
sub_text = parts[1] if len(parts) > 1 else ""
|
||||
sub_lower = sub_text.lower()
|
||||
|
||||
# Dynamic completions for commands with runtime lists
|
||||
if " " not in sub_text:
|
||||
if base_cmd == "/model":
|
||||
yield from self._model_completions(sub_text, sub_lower)
|
||||
return
|
||||
if base_cmd == "/skin":
|
||||
yield from self._skin_completions(sub_text, sub_lower)
|
||||
return
|
||||
if base_cmd == "/personality":
|
||||
yield from self._personality_completions(sub_text, sub_lower)
|
||||
return
|
||||
# Dynamic model alias completions for /model
|
||||
if " " not in sub_text and base_cmd == "/model":
|
||||
yield from self._model_completions(sub_text, sub_lower)
|
||||
return
|
||||
|
||||
# Static subcommand completions
|
||||
if " " not in sub_text and base_cmd in SUBCOMMANDS and self._command_allowed(base_cmd):
|
||||
|
||||
+10
-146
@@ -12,7 +12,6 @@ This module provides:
|
||||
- hermes config wizard - Re-run setup wizard
|
||||
"""
|
||||
|
||||
import copy
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
@@ -27,7 +26,6 @@ from typing import Dict, Any, Optional, List, Tuple
|
||||
|
||||
_IS_WINDOWS = platform.system() == "Windows"
|
||||
_ENV_VAR_NAME_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
|
||||
_LAST_EXPANDED_CONFIG_BY_PATH: Dict[str, Any] = {}
|
||||
# Env var names written to .env that aren't in OPTIONAL_ENV_VARS
|
||||
# (managed by setup/provider flows directly).
|
||||
_EXTRA_ENV_KEYS = frozenset({
|
||||
@@ -46,8 +44,7 @@ _EXTRA_ENV_KEYS = frozenset({
|
||||
"WEIXIN_HOME_CHANNEL", "WEIXIN_HOME_CHANNEL_NAME", "WEIXIN_DM_POLICY", "WEIXIN_GROUP_POLICY",
|
||||
"WEIXIN_ALLOWED_USERS", "WEIXIN_GROUP_ALLOWED_USERS", "WEIXIN_ALLOW_ALL_USERS",
|
||||
"BLUEBUBBLES_SERVER_URL", "BLUEBUBBLES_PASSWORD",
|
||||
"QQ_APP_ID", "QQ_CLIENT_SECRET", "QQBOT_HOME_CHANNEL", "QQBOT_HOME_CHANNEL_NAME",
|
||||
"QQ_HOME_CHANNEL", "QQ_HOME_CHANNEL_NAME", # legacy aliases (pre-rename, still read for back-compat)
|
||||
"QQ_APP_ID", "QQ_CLIENT_SECRET", "QQ_HOME_CHANNEL", "QQ_HOME_CHANNEL_NAME",
|
||||
"QQ_ALLOWED_USERS", "QQ_GROUP_ALLOWED_USERS", "QQ_ALLOW_ALL_USERS", "QQ_MARKDOWN_SUPPORT",
|
||||
"QQ_STT_API_KEY", "QQ_STT_BASE_URL", "QQ_STT_MODEL",
|
||||
"TERMINAL_ENV", "TERMINAL_SSH_KEY", "TERMINAL_SSH_PORT",
|
||||
@@ -420,7 +417,6 @@ DEFAULT_CONFIG = {
|
||||
"command_timeout": 30, # Timeout for browser commands in seconds (screenshot, navigate, etc.)
|
||||
"record_sessions": False, # Auto-record browser sessions as WebM videos
|
||||
"allow_private_urls": False, # Allow navigating to private/internal IPs (localhost, 192.168.x.x, etc.)
|
||||
"cdp_url": "", # Optional persistent CDP endpoint for attaching to an existing Chromium/Chrome
|
||||
"camofox": {
|
||||
# When true, Hermes sends a stable profile-scoped userId to Camofox
|
||||
# so the server maps it to a persistent Firefox profile automatically.
|
||||
@@ -541,13 +537,6 @@ DEFAULT_CONFIG = {
|
||||
"api_key": "",
|
||||
"timeout": 30,
|
||||
},
|
||||
"title_generation": {
|
||||
"provider": "auto",
|
||||
"model": "",
|
||||
"base_url": "",
|
||||
"api_key": "",
|
||||
"timeout": 30,
|
||||
},
|
||||
},
|
||||
|
||||
"display": {
|
||||
@@ -737,14 +726,9 @@ DEFAULT_CONFIG = {
|
||||
# manual — always prompt the user (default)
|
||||
# smart — use auxiliary LLM to auto-approve low-risk commands, prompt for high-risk
|
||||
# off — skip all approval prompts (equivalent to --yolo)
|
||||
#
|
||||
# cron_mode — what to do when a cron job hits a dangerous command:
|
||||
# deny — block the command and let the agent find another way (default, safe)
|
||||
# approve — auto-approve all dangerous commands in cron jobs
|
||||
"approvals": {
|
||||
"mode": "manual",
|
||||
"timeout": 60,
|
||||
"cron_mode": "deny",
|
||||
},
|
||||
|
||||
# Permanently allowed dangerous command patterns (added via "always" approval)
|
||||
@@ -776,20 +760,6 @@ DEFAULT_CONFIG = {
|
||||
"wrap_response": True,
|
||||
},
|
||||
|
||||
# execute_code settings — controls the tool used for programmatic tool calls.
|
||||
"code_execution": {
|
||||
# Execution mode:
|
||||
# project (default) — scripts run in the session's working directory
|
||||
# with the active virtualenv/conda env's python, so project deps
|
||||
# (pandas, torch, project packages) and relative paths resolve.
|
||||
# strict — scripts run in an isolated temp directory with
|
||||
# hermes-agent's own python (sys.executable). Maximum isolation
|
||||
# and reproducibility; project deps and relative paths won't work.
|
||||
# Env scrubbing (strips *_API_KEY, *_TOKEN, *_SECRET, ...) and the
|
||||
# tool whitelist apply identically in both modes.
|
||||
"mode": "project",
|
||||
},
|
||||
|
||||
# Logging — controls file logging to ~/.hermes/logs/.
|
||||
# agent.log captures INFO+ (all agent activity); errors.log captures WARNING+.
|
||||
"logging": {
|
||||
@@ -807,7 +777,7 @@ DEFAULT_CONFIG = {
|
||||
},
|
||||
|
||||
# Config schema version - bump this when adding new required fields
|
||||
"_config_version": 19,
|
||||
"_config_version": 18,
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
@@ -891,22 +861,6 @@ OPTIONAL_ENV_VARS = {
|
||||
"category": "provider",
|
||||
"advanced": True,
|
||||
},
|
||||
"NVIDIA_API_KEY": {
|
||||
"description": "NVIDIA NIM API key (build.nvidia.com or local NIM endpoint)",
|
||||
"prompt": "NVIDIA NIM API key",
|
||||
"url": "https://build.nvidia.com/",
|
||||
"password": True,
|
||||
"category": "provider",
|
||||
"advanced": True,
|
||||
},
|
||||
"NVIDIA_BASE_URL": {
|
||||
"description": "NVIDIA NIM base URL override (e.g. http://localhost:8000/v1 for local NIM)",
|
||||
"prompt": "NVIDIA NIM base URL (leave empty for default)",
|
||||
"url": None,
|
||||
"password": False,
|
||||
"category": "provider",
|
||||
"advanced": True,
|
||||
},
|
||||
"GLM_API_KEY": {
|
||||
"description": "Z.AI / GLM API key (also recognized as ZAI_API_KEY / Z_AI_API_KEY)",
|
||||
"prompt": "Z.AI / GLM API key",
|
||||
@@ -1564,12 +1518,12 @@ OPTIONAL_ENV_VARS = {
|
||||
"prompt": "Allow All QQ Users",
|
||||
"category": "messaging",
|
||||
},
|
||||
"QQBOT_HOME_CHANNEL": {
|
||||
"QQ_HOME_CHANNEL": {
|
||||
"description": "Default QQ channel/group for cron delivery and notifications",
|
||||
"prompt": "QQ Home Channel",
|
||||
"category": "messaging",
|
||||
},
|
||||
"QQBOT_HOME_CHANNEL_NAME": {
|
||||
"QQ_HOME_CHANNEL_NAME": {
|
||||
"description": "Display name for the QQ home channel",
|
||||
"prompt": "QQ Home Channel Name",
|
||||
"category": "messaging",
|
||||
@@ -2656,85 +2610,6 @@ def _expand_env_vars(obj):
|
||||
return obj
|
||||
|
||||
|
||||
def _items_by_unique_name(items):
|
||||
"""Return a name-indexed dict only when all items have unique string names."""
|
||||
if not isinstance(items, list):
|
||||
return None
|
||||
indexed = {}
|
||||
for item in items:
|
||||
if not isinstance(item, dict) or not isinstance(item.get("name"), str):
|
||||
return None
|
||||
name = item["name"]
|
||||
if name in indexed:
|
||||
return None
|
||||
indexed[name] = item
|
||||
return indexed
|
||||
|
||||
|
||||
def _preserve_env_ref_templates(current, raw, loaded_expanded=None):
|
||||
"""Restore raw ``${VAR}`` templates when a value is otherwise unchanged.
|
||||
|
||||
``load_config()`` expands env refs for runtime use. When a caller later
|
||||
persists that config after modifying some unrelated setting, keep the
|
||||
original on-disk template instead of writing the expanded plaintext
|
||||
secret back to ``config.yaml``.
|
||||
|
||||
Prefer preserving the raw template when ``current`` still matches either
|
||||
the value previously returned by ``load_config()`` for this config path or
|
||||
the current environment expansion of ``raw``. This handles env-var
|
||||
rotation between load and save while still treating mixed literal/template
|
||||
string edits as caller-owned once their rendered value diverges.
|
||||
"""
|
||||
if isinstance(current, str) and isinstance(raw, str) and re.search(r"\${[^}]+}", raw):
|
||||
if current == raw:
|
||||
return raw
|
||||
if isinstance(loaded_expanded, str) and current == loaded_expanded:
|
||||
return raw
|
||||
if _expand_env_vars(raw) == current:
|
||||
return raw
|
||||
return current
|
||||
|
||||
if isinstance(current, dict) and isinstance(raw, dict):
|
||||
return {
|
||||
key: _preserve_env_ref_templates(
|
||||
value,
|
||||
raw.get(key),
|
||||
loaded_expanded.get(key) if isinstance(loaded_expanded, dict) else None,
|
||||
)
|
||||
for key, value in current.items()
|
||||
}
|
||||
|
||||
if isinstance(current, list) and isinstance(raw, list):
|
||||
# Prefer matching named config objects (e.g. custom_providers) by name
|
||||
# so harmless reordering doesn't drop the original template. If names
|
||||
# are duplicated, fall back to positional matching instead of silently
|
||||
# shadowing one entry.
|
||||
current_by_name = _items_by_unique_name(current)
|
||||
raw_by_name = _items_by_unique_name(raw)
|
||||
loaded_by_name = _items_by_unique_name(loaded_expanded)
|
||||
if current_by_name is not None and raw_by_name is not None:
|
||||
return [
|
||||
_preserve_env_ref_templates(
|
||||
item,
|
||||
raw_by_name.get(item.get("name")),
|
||||
loaded_by_name.get(item.get("name")) if loaded_by_name is not None else None,
|
||||
)
|
||||
for item in current
|
||||
]
|
||||
return [
|
||||
_preserve_env_ref_templates(
|
||||
item,
|
||||
raw[index] if index < len(raw) else None,
|
||||
loaded_expanded[index]
|
||||
if isinstance(loaded_expanded, list) and index < len(loaded_expanded)
|
||||
else None,
|
||||
)
|
||||
for index, item in enumerate(current)
|
||||
]
|
||||
|
||||
return current
|
||||
|
||||
|
||||
def _normalize_root_model_keys(config: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Move stale root-level provider/base_url into model section.
|
||||
|
||||
@@ -2802,6 +2677,7 @@ def read_raw_config() -> Dict[str, Any]:
|
||||
|
||||
def load_config() -> Dict[str, Any]:
|
||||
"""Load configuration from ~/.hermes/config.yaml."""
|
||||
import copy
|
||||
ensure_hermes_home()
|
||||
config_path = get_config_path()
|
||||
|
||||
@@ -2822,11 +2698,8 @@ def load_config() -> Dict[str, Any]:
|
||||
config = _deep_merge(config, user_config)
|
||||
except Exception as e:
|
||||
print(f"Warning: Failed to load config: {e}")
|
||||
|
||||
normalized = _normalize_root_model_keys(_normalize_max_turns_config(config))
|
||||
expanded = _expand_env_vars(normalized)
|
||||
_LAST_EXPANDED_CONFIG_BY_PATH[str(config_path)] = copy.deepcopy(expanded)
|
||||
return expanded
|
||||
|
||||
return _expand_env_vars(_normalize_root_model_keys(_normalize_max_turns_config(config)))
|
||||
|
||||
|
||||
_SECURITY_COMMENT = """
|
||||
@@ -2861,7 +2734,7 @@ _FALLBACK_COMMENT = """
|
||||
# minimax (MINIMAX_API_KEY) — MiniMax
|
||||
# minimax-cn (MINIMAX_CN_API_KEY) — MiniMax (China)
|
||||
#
|
||||
# For custom OpenAI-compatible endpoints, add base_url and key_env.
|
||||
# For custom OpenAI-compatible endpoints, add base_url and api_key_env.
|
||||
#
|
||||
# fallback_model:
|
||||
# provider: openrouter
|
||||
@@ -2905,7 +2778,7 @@ _COMMENTED_SECTIONS = """
|
||||
# minimax (MINIMAX_API_KEY) — MiniMax
|
||||
# minimax-cn (MINIMAX_CN_API_KEY) — MiniMax (China)
|
||||
#
|
||||
# For custom OpenAI-compatible endpoints, add base_url and key_env.
|
||||
# For custom OpenAI-compatible endpoints, add base_url and api_key_env.
|
||||
#
|
||||
# fallback_model:
|
||||
# provider: openrouter
|
||||
@@ -2935,15 +2808,7 @@ def save_config(config: Dict[str, Any]):
|
||||
|
||||
ensure_hermes_home()
|
||||
config_path = get_config_path()
|
||||
current_normalized = _normalize_root_model_keys(_normalize_max_turns_config(config))
|
||||
normalized = current_normalized
|
||||
raw_existing = _normalize_root_model_keys(_normalize_max_turns_config(read_raw_config()))
|
||||
if raw_existing:
|
||||
normalized = _preserve_env_ref_templates(
|
||||
normalized,
|
||||
raw_existing,
|
||||
_LAST_EXPANDED_CONFIG_BY_PATH.get(str(config_path)),
|
||||
)
|
||||
normalized = _normalize_root_model_keys(_normalize_max_turns_config(config))
|
||||
|
||||
# Build optional commented-out sections for features that are off by
|
||||
# default or only relevant when explicitly configured.
|
||||
@@ -2961,7 +2826,6 @@ def save_config(config: Dict[str, Any]):
|
||||
extra_content="".join(parts) if parts else None,
|
||||
)
|
||||
_secure_file(config_path)
|
||||
_LAST_EXPANDED_CONFIG_BY_PATH[str(config_path)] = copy.deepcopy(current_normalized)
|
||||
|
||||
|
||||
def load_env() -> Dict[str, str]:
|
||||
|
||||
+29
-137
@@ -6,10 +6,7 @@ Currently supports:
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
@@ -34,119 +31,6 @@ _MAX_LOG_BYTES = 512_000
|
||||
_AUTO_DELETE_SECONDS = 21600
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pending-deletion tracking (replaces the old fork-and-sleep subprocess).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _pending_file() -> Path:
|
||||
"""Path to ``~/.hermes/pastes/pending.json``.
|
||||
|
||||
Each entry: ``{"url": "...", "expire_at": <unix_ts>}``. Scheduled
|
||||
DELETEs used to be handled by spawning a detached Python process per
|
||||
paste that slept for 6 hours; those accumulated forever if the user
|
||||
ran ``hermes debug share`` repeatedly. We now persist the schedule
|
||||
to disk and sweep expired entries on the next debug invocation.
|
||||
"""
|
||||
return get_hermes_home() / "pastes" / "pending.json"
|
||||
|
||||
|
||||
def _load_pending() -> list[dict]:
|
||||
path = _pending_file()
|
||||
if not path.exists():
|
||||
return []
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
if isinstance(data, list):
|
||||
# Filter to well-formed entries only
|
||||
return [
|
||||
e for e in data
|
||||
if isinstance(e, dict) and "url" in e and "expire_at" in e
|
||||
]
|
||||
except (OSError, ValueError, json.JSONDecodeError):
|
||||
pass
|
||||
return []
|
||||
|
||||
|
||||
def _save_pending(entries: list[dict]) -> None:
|
||||
path = _pending_file()
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_suffix(".json.tmp")
|
||||
tmp.write_text(json.dumps(entries, indent=2), encoding="utf-8")
|
||||
os.replace(tmp, path)
|
||||
except OSError:
|
||||
# Non-fatal — worst case the user has to run ``hermes debug delete``
|
||||
# manually.
|
||||
pass
|
||||
|
||||
|
||||
def _record_pending(urls: list[str], delay_seconds: int = _AUTO_DELETE_SECONDS) -> None:
|
||||
"""Record *urls* for deletion at ``now + delay_seconds``.
|
||||
|
||||
Only paste.rs URLs are recorded (dpaste.com auto-expires). Entries
|
||||
are merged into any existing pending.json.
|
||||
"""
|
||||
paste_rs_urls = [u for u in urls if _extract_paste_id(u)]
|
||||
if not paste_rs_urls:
|
||||
return
|
||||
|
||||
entries = _load_pending()
|
||||
# Dedupe by URL: keep the later expire_at if same URL appears twice
|
||||
by_url: dict[str, float] = {e["url"]: float(e["expire_at"]) for e in entries}
|
||||
expire_at = time.time() + delay_seconds
|
||||
for u in paste_rs_urls:
|
||||
by_url[u] = max(expire_at, by_url.get(u, 0.0))
|
||||
merged = [{"url": u, "expire_at": ts} for u, ts in by_url.items()]
|
||||
_save_pending(merged)
|
||||
|
||||
|
||||
def _sweep_expired_pastes(now: Optional[float] = None) -> tuple[int, int]:
|
||||
"""Synchronously DELETE any pending pastes whose ``expire_at`` has passed.
|
||||
|
||||
Returns ``(deleted, remaining)``. Best-effort: failed deletes stay in
|
||||
the pending file and will be retried on the next sweep. Silent —
|
||||
intended to be called from every ``hermes debug`` invocation with
|
||||
minimal noise.
|
||||
"""
|
||||
entries = _load_pending()
|
||||
if not entries:
|
||||
return (0, 0)
|
||||
|
||||
current = time.time() if now is None else now
|
||||
deleted = 0
|
||||
remaining: list[dict] = []
|
||||
|
||||
for entry in entries:
|
||||
try:
|
||||
expire_at = float(entry.get("expire_at", 0))
|
||||
except (TypeError, ValueError):
|
||||
continue # drop malformed entries
|
||||
if expire_at > current:
|
||||
remaining.append(entry)
|
||||
continue
|
||||
|
||||
url = entry.get("url", "")
|
||||
try:
|
||||
if delete_paste(url):
|
||||
deleted += 1
|
||||
continue
|
||||
except Exception:
|
||||
# Network hiccup, 404 (already gone), etc. — drop the entry
|
||||
# after a grace period; don't retry forever.
|
||||
pass
|
||||
|
||||
# Retain failed deletes for up to 24h past expiration, then give up.
|
||||
if expire_at + 86400 > current:
|
||||
remaining.append(entry)
|
||||
else:
|
||||
deleted += 1 # count as reaped (paste.rs will GC eventually)
|
||||
|
||||
if deleted:
|
||||
_save_pending(remaining)
|
||||
|
||||
return (deleted, len(remaining))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Privacy / delete helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -206,19 +90,37 @@ def delete_paste(url: str) -> bool:
|
||||
|
||||
|
||||
def _schedule_auto_delete(urls: list[str], delay_seconds: int = _AUTO_DELETE_SECONDS):
|
||||
"""Record *urls* for deletion ``delay_seconds`` from now.
|
||||
"""Spawn a detached process to delete paste.rs pastes after *delay_seconds*.
|
||||
|
||||
Previously this spawned a detached Python subprocess per call that slept
|
||||
for 6 hours and then issued DELETE requests. Those subprocesses leaked —
|
||||
every ``hermes debug share`` invocation added ~20 MB of resident Python
|
||||
interpreters that never exited until the sleep completed.
|
||||
|
||||
The replacement is stateless: we append to ``~/.hermes/pastes/pending.json``
|
||||
and rely on opportunistic sweeps (``_sweep_expired_pastes``) called from
|
||||
every ``hermes debug`` invocation. If the user never runs ``hermes debug``
|
||||
again, paste.rs's own retention policy handles cleanup.
|
||||
The child process is fully detached (``start_new_session=True``) so it
|
||||
survives the parent exiting (important for CLI mode). Only paste.rs
|
||||
URLs are attempted — dpaste.com pastes auto-expire on their own.
|
||||
"""
|
||||
_record_pending(urls, delay_seconds=delay_seconds)
|
||||
import subprocess
|
||||
|
||||
paste_rs_urls = [u for u in urls if _extract_paste_id(u)]
|
||||
if not paste_rs_urls:
|
||||
return
|
||||
|
||||
# Build a tiny inline Python script. No imports beyond stdlib.
|
||||
url_list = ", ".join(f'"{u}"' for u in paste_rs_urls)
|
||||
script = (
|
||||
"import time, urllib.request; "
|
||||
f"time.sleep({delay_seconds}); "
|
||||
f"[urllib.request.urlopen(urllib.request.Request(u, method='DELETE', "
|
||||
f"headers={{'User-Agent': 'hermes-agent/auto-delete'}}), timeout=15) "
|
||||
f"for u in [{url_list}]]"
|
||||
)
|
||||
|
||||
try:
|
||||
subprocess.Popen(
|
||||
[sys.executable, "-c", script],
|
||||
start_new_session=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
except Exception:
|
||||
pass # Best-effort; manual delete still available.
|
||||
|
||||
|
||||
def _delete_hint(url: str) -> str:
|
||||
@@ -553,16 +455,6 @@ def run_debug_delete(args):
|
||||
|
||||
def run_debug(args):
|
||||
"""Route debug subcommands."""
|
||||
# Opportunistic sweep of expired pastes on every ``hermes debug`` call.
|
||||
# Replaces the old per-paste sleeping subprocess that used to leak as
|
||||
# one orphaned Python interpreter per scheduled deletion. Silent and
|
||||
# best-effort — any failure is swallowed so ``hermes debug`` stays
|
||||
# reliable even when offline.
|
||||
try:
|
||||
_sweep_expired_pastes()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
subcmd = getattr(args, "debug_command", None)
|
||||
if subcmd == "share":
|
||||
run_debug_share(args)
|
||||
|
||||
@@ -1,294 +0,0 @@
|
||||
"""
|
||||
DingTalk Device Flow authorization.
|
||||
|
||||
Implements the same 3-step registration flow as dingtalk-openclaw-connector:
|
||||
1. POST /app/registration/init → get nonce
|
||||
2. POST /app/registration/begin → get device_code + verification_uri_complete
|
||||
3. POST /app/registration/poll → poll until SUCCESS → get client_id + client_secret
|
||||
|
||||
The verification_uri_complete is rendered as a QR code in the terminal so the
|
||||
user can scan it with DingTalk to authorize, yielding AppKey + AppSecret
|
||||
automatically.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import logging
|
||||
from typing import Optional, Tuple
|
||||
|
||||
import requests
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ── Configuration ──────────────────────────────────────────────────────────
|
||||
|
||||
REGISTRATION_BASE_URL = os.environ.get(
|
||||
"DINGTALK_REGISTRATION_BASE_URL", "https://oapi.dingtalk.com"
|
||||
).rstrip("/")
|
||||
|
||||
REGISTRATION_SOURCE = os.environ.get("DINGTALK_REGISTRATION_SOURCE", "openClaw")
|
||||
|
||||
|
||||
# ── API helpers ────────────────────────────────────────────────────────────
|
||||
|
||||
class RegistrationError(Exception):
|
||||
"""Raised when a DingTalk registration API call fails."""
|
||||
|
||||
|
||||
def _api_post(path: str, payload: dict) -> dict:
|
||||
"""POST to the registration API and return the parsed JSON body."""
|
||||
url = f"{REGISTRATION_BASE_URL}{path}"
|
||||
try:
|
||||
resp = requests.post(url, json=payload, timeout=15)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
except requests.RequestException as exc:
|
||||
raise RegistrationError(f"Network error calling {url}: {exc}") from exc
|
||||
|
||||
errcode = data.get("errcode", -1)
|
||||
if errcode != 0:
|
||||
errmsg = data.get("errmsg", "unknown error")
|
||||
raise RegistrationError(f"API error [{path}]: {errmsg} (errcode={errcode})")
|
||||
return data
|
||||
|
||||
|
||||
# ── Core flow ──────────────────────────────────────────────────────────────
|
||||
|
||||
def begin_registration() -> dict:
|
||||
"""Start a device-flow registration.
|
||||
|
||||
Returns a dict with keys:
|
||||
device_code, verification_uri_complete, expires_in, interval
|
||||
"""
|
||||
# Step 1: init → nonce
|
||||
init_data = _api_post("/app/registration/init", {"source": REGISTRATION_SOURCE})
|
||||
nonce = str(init_data.get("nonce", "")).strip()
|
||||
if not nonce:
|
||||
raise RegistrationError("init response missing nonce")
|
||||
|
||||
# Step 2: begin → device_code, verification_uri_complete
|
||||
begin_data = _api_post("/app/registration/begin", {"nonce": nonce})
|
||||
device_code = str(begin_data.get("device_code", "")).strip()
|
||||
verification_uri_complete = str(begin_data.get("verification_uri_complete", "")).strip()
|
||||
if not device_code:
|
||||
raise RegistrationError("begin response missing device_code")
|
||||
if not verification_uri_complete:
|
||||
raise RegistrationError("begin response missing verification_uri_complete")
|
||||
|
||||
return {
|
||||
"device_code": device_code,
|
||||
"verification_uri_complete": verification_uri_complete,
|
||||
"expires_in": int(begin_data.get("expires_in", 7200)),
|
||||
"interval": max(int(begin_data.get("interval", 3)), 2),
|
||||
}
|
||||
|
||||
|
||||
def poll_registration(device_code: str) -> dict:
|
||||
"""Poll the registration status once.
|
||||
|
||||
Returns a dict with keys: status, client_id?, client_secret?, fail_reason?
|
||||
"""
|
||||
data = _api_post("/app/registration/poll", {"device_code": device_code})
|
||||
status_raw = str(data.get("status", "")).strip().upper()
|
||||
if status_raw not in ("WAITING", "SUCCESS", "FAIL", "EXPIRED"):
|
||||
status_raw = "UNKNOWN"
|
||||
return {
|
||||
"status": status_raw,
|
||||
"client_id": str(data.get("client_id", "")).strip() or None,
|
||||
"client_secret": str(data.get("client_secret", "")).strip() or None,
|
||||
"fail_reason": str(data.get("fail_reason", "")).strip() or None,
|
||||
}
|
||||
|
||||
|
||||
def wait_for_registration_success(
|
||||
device_code: str,
|
||||
interval: int = 3,
|
||||
expires_in: int = 7200,
|
||||
on_waiting: Optional[callable] = None,
|
||||
) -> Tuple[str, str]:
|
||||
"""Block until the registration succeeds or times out.
|
||||
|
||||
Returns (client_id, client_secret).
|
||||
"""
|
||||
deadline = time.monotonic() + expires_in
|
||||
retry_window = 120 # 2 minutes for transient errors
|
||||
retry_start = 0.0
|
||||
|
||||
while time.monotonic() < deadline:
|
||||
time.sleep(interval)
|
||||
try:
|
||||
result = poll_registration(device_code)
|
||||
except RegistrationError:
|
||||
if retry_start == 0:
|
||||
retry_start = time.monotonic()
|
||||
if time.monotonic() - retry_start < retry_window:
|
||||
continue
|
||||
raise
|
||||
|
||||
status = result["status"]
|
||||
if status == "WAITING":
|
||||
retry_start = 0
|
||||
if on_waiting:
|
||||
on_waiting()
|
||||
continue
|
||||
if status == "SUCCESS":
|
||||
cid = result["client_id"]
|
||||
csecret = result["client_secret"]
|
||||
if not cid or not csecret:
|
||||
raise RegistrationError("authorization succeeded but credentials are missing")
|
||||
return cid, csecret
|
||||
# FAIL / EXPIRED / UNKNOWN
|
||||
if retry_start == 0:
|
||||
retry_start = time.monotonic()
|
||||
if time.monotonic() - retry_start < retry_window:
|
||||
continue
|
||||
reason = result.get("fail_reason") or status
|
||||
raise RegistrationError(f"authorization failed: {reason}")
|
||||
|
||||
raise RegistrationError("authorization timed out, please retry")
|
||||
|
||||
|
||||
# ── QR code rendering ─────────────────────────────────────────────────────
|
||||
|
||||
def _ensure_qrcode_installed() -> bool:
|
||||
"""Try to import qrcode; if missing, auto-install it via pip/uv."""
|
||||
try:
|
||||
import qrcode # noqa: F401
|
||||
return True
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
import subprocess
|
||||
|
||||
# Try uv first (Hermes convention), then pip
|
||||
for cmd in (
|
||||
[sys.executable, "-m", "uv", "pip", "install", "qrcode"],
|
||||
[sys.executable, "-m", "pip", "install", "-q", "qrcode"],
|
||||
):
|
||||
try:
|
||||
subprocess.check_call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
import qrcode # noqa: F401,F811
|
||||
return True
|
||||
except (subprocess.CalledProcessError, ImportError, FileNotFoundError):
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
def render_qr_to_terminal(url: str) -> bool:
|
||||
"""Render *url* as a compact QR code in the terminal.
|
||||
|
||||
Returns True if the QR code was printed, False if the library is missing.
|
||||
"""
|
||||
try:
|
||||
import qrcode
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
qr = qrcode.QRCode(
|
||||
version=1,
|
||||
error_correction=qrcode.constants.ERROR_CORRECT_L,
|
||||
box_size=1,
|
||||
border=1,
|
||||
)
|
||||
qr.add_data(url)
|
||||
qr.make(fit=True)
|
||||
|
||||
# Use half-block characters for compact rendering (2 rows per character)
|
||||
matrix = qr.get_matrix()
|
||||
rows = len(matrix)
|
||||
lines: list[str] = []
|
||||
|
||||
TOP_HALF = "\u2580" # ▀
|
||||
BOTTOM_HALF = "\u2584" # ▄
|
||||
FULL_BLOCK = "\u2588" # █
|
||||
EMPTY = " "
|
||||
|
||||
for r in range(0, rows, 2):
|
||||
line_chars: list[str] = []
|
||||
for c in range(len(matrix[r])):
|
||||
top = matrix[r][c]
|
||||
bottom = matrix[r + 1][c] if r + 1 < rows else False
|
||||
if top and bottom:
|
||||
line_chars.append(FULL_BLOCK)
|
||||
elif top:
|
||||
line_chars.append(TOP_HALF)
|
||||
elif bottom:
|
||||
line_chars.append(BOTTOM_HALF)
|
||||
else:
|
||||
line_chars.append(EMPTY)
|
||||
lines.append(" " + "".join(line_chars))
|
||||
|
||||
print("\n".join(lines))
|
||||
return True
|
||||
|
||||
|
||||
# ── High-level entry point for the setup wizard ───────────────────────────
|
||||
|
||||
def dingtalk_qr_auth() -> Optional[Tuple[str, str]]:
|
||||
"""Run the interactive QR-code device-flow authorization.
|
||||
|
||||
Returns (client_id, client_secret) on success, or None if the user
|
||||
cancelled or the flow failed.
|
||||
"""
|
||||
from hermes_cli.setup import print_info, print_success, print_warning, print_error
|
||||
|
||||
print()
|
||||
print_info(" Initializing DingTalk device authorization...")
|
||||
print_info(" Note: the scan page is branded 'OpenClaw' — DingTalk's")
|
||||
print_info(" ecosystem onboarding bridge. Safe to use.")
|
||||
|
||||
try:
|
||||
reg = begin_registration()
|
||||
except RegistrationError as exc:
|
||||
print_error(f" Authorization init failed: {exc}")
|
||||
return None
|
||||
|
||||
url = reg["verification_uri_complete"]
|
||||
|
||||
# Ensure qrcode library is available (auto-install if missing)
|
||||
if not _ensure_qrcode_installed():
|
||||
print_warning(" qrcode library install failed, will show link only.")
|
||||
|
||||
print()
|
||||
print_info(" Please scan the QR code below with DingTalk to authorize:")
|
||||
print()
|
||||
|
||||
if not render_qr_to_terminal(url):
|
||||
print_warning(f" QR code render failed, please open the link below to authorize:")
|
||||
|
||||
print()
|
||||
print_info(f" Or open this link manually: {url}")
|
||||
print()
|
||||
print_info(" Waiting for QR scan authorization... (timeout: 2 hours)")
|
||||
|
||||
dot_count = 0
|
||||
|
||||
def _on_waiting():
|
||||
nonlocal dot_count
|
||||
dot_count += 1
|
||||
if dot_count % 10 == 0:
|
||||
sys.stdout.write(".")
|
||||
sys.stdout.flush()
|
||||
|
||||
try:
|
||||
client_id, client_secret = wait_for_registration_success(
|
||||
device_code=reg["device_code"],
|
||||
interval=reg["interval"],
|
||||
expires_in=reg["expires_in"],
|
||||
on_waiting=_on_waiting,
|
||||
)
|
||||
except RegistrationError as exc:
|
||||
print()
|
||||
print_error(f" Authorization failed: {exc}")
|
||||
return None
|
||||
|
||||
print()
|
||||
print_success(" QR scan authorization successful!")
|
||||
print_success(f" Client ID: {client_id}")
|
||||
print_success(f" Client Secret: {client_secret[:8]}{'*' * (len(client_secret) - 8)}")
|
||||
|
||||
return client_id, client_secret
|
||||
@@ -825,7 +825,6 @@ def run_doctor(args):
|
||||
("Arcee AI", ("ARCEEAI_API_KEY",), "https://api.arcee.ai/api/v1/models", "ARCEE_BASE_URL", True),
|
||||
("DeepSeek", ("DEEPSEEK_API_KEY",), "https://api.deepseek.com/v1/models", "DEEPSEEK_BASE_URL", True),
|
||||
("Hugging Face", ("HF_TOKEN",), "https://router.huggingface.co/v1/models", "HF_BASE_URL", True),
|
||||
("NVIDIA NIM", ("NVIDIA_API_KEY",), "https://integrate.api.nvidia.com/v1/models", "NVIDIA_BASE_URL", True),
|
||||
("Alibaba/DashScope", ("DASHSCOPE_API_KEY",), "https://dashscope-intl.aliyuncs.com/compatible-mode/v1/models", "DASHSCOPE_BASE_URL", True),
|
||||
# MiniMax: the /anthropic endpoint doesn't support /models, but the /v1 endpoint does.
|
||||
("MiniMax", ("MINIMAX_API_KEY",), "https://api.minimax.io/v1/models", "MINIMAX_BASE_URL", True),
|
||||
@@ -895,8 +894,8 @@ def run_doctor(args):
|
||||
_model_count = len(_br_resp.get("modelSummaries", []))
|
||||
print(f"\r {color('✓', Colors.GREEN)} {_label} {color(f'({_auth_var}, {_region}, {_model_count} models)', Colors.DIM)} ")
|
||||
except ImportError:
|
||||
print(f"\r {color('⚠', Colors.YELLOW)} {_label} {color(f'(boto3 not installed — {sys.executable} -m pip install boto3)', Colors.DIM)} ")
|
||||
issues.append(f"Install boto3 for Bedrock: {sys.executable} -m pip install boto3")
|
||||
print(f"\r {color('⚠', Colors.YELLOW)} {_label} {color('(boto3 not installed — pip install hermes-agent[bedrock])', Colors.DIM)} ")
|
||||
issues.append("Install boto3 for Bedrock: pip install hermes-agent[bedrock]")
|
||||
except Exception as _e:
|
||||
_err_name = type(_e).__name__
|
||||
print(f"\r {color('⚠', Colors.YELLOW)} {_label} {color(f'({_err_name}: {_e})', Colors.DIM)} ")
|
||||
|
||||
+35
-15
@@ -43,20 +43,41 @@ def _redact(value: str) -> str:
|
||||
|
||||
def _gateway_status() -> str:
|
||||
"""Return a short gateway status string."""
|
||||
try:
|
||||
from hermes_cli.gateway import get_gateway_runtime_snapshot
|
||||
|
||||
snapshot = get_gateway_runtime_snapshot()
|
||||
if snapshot.running:
|
||||
mode = snapshot.manager
|
||||
if snapshot.has_process_service_mismatch:
|
||||
mode = "manual"
|
||||
return f"running ({mode}, pid {snapshot.gateway_pids[0]})"
|
||||
if snapshot.service_installed and not snapshot.service_running:
|
||||
return f"stopped ({snapshot.manager})"
|
||||
return f"stopped ({snapshot.manager})"
|
||||
except Exception:
|
||||
return "unknown" if sys.platform.startswith(("linux", "darwin")) else "N/A"
|
||||
if sys.platform.startswith("linux"):
|
||||
from hermes_constants import is_container
|
||||
if is_container():
|
||||
try:
|
||||
from hermes_cli.gateway import find_gateway_pids
|
||||
pids = find_gateway_pids()
|
||||
if pids:
|
||||
return f"running (docker, pid {pids[0]})"
|
||||
return "stopped (docker)"
|
||||
except Exception:
|
||||
return "stopped (docker)"
|
||||
try:
|
||||
from hermes_cli.gateway import get_service_name
|
||||
svc = get_service_name()
|
||||
except Exception:
|
||||
svc = "hermes-gateway"
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["systemctl", "--user", "is-active", svc],
|
||||
capture_output=True, text=True, timeout=5,
|
||||
)
|
||||
return "running (systemd)" if r.stdout.strip() == "active" else "stopped"
|
||||
except Exception:
|
||||
return "unknown"
|
||||
elif sys.platform == "darwin":
|
||||
try:
|
||||
from hermes_cli.gateway import get_launchd_label
|
||||
r = subprocess.run(
|
||||
["launchctl", "list", get_launchd_label()],
|
||||
capture_output=True, text=True, timeout=5,
|
||||
)
|
||||
return "loaded (launchd)" if r.returncode == 0 else "not loaded"
|
||||
except Exception:
|
||||
return "unknown"
|
||||
return "N/A"
|
||||
|
||||
|
||||
def _count_skills(hermes_home: Path) -> int:
|
||||
@@ -275,7 +296,6 @@ def run_dump(args):
|
||||
("DEEPSEEK_API_KEY", "deepseek"),
|
||||
("DASHSCOPE_API_KEY", "dashscope"),
|
||||
("HF_TOKEN", "huggingface"),
|
||||
("NVIDIA_API_KEY", "nvidia"),
|
||||
("AI_GATEWAY_API_KEY", "ai_gateway"),
|
||||
("OPENCODE_ZEN_API_KEY", "opencode_zen"),
|
||||
("OPENCODE_GO_API_KEY", "opencode_go"),
|
||||
|
||||
+34
-691
@@ -10,7 +10,6 @@ import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT = Path(__file__).parent.parent.resolve()
|
||||
@@ -42,23 +41,6 @@ from hermes_cli.colors import Colors, color
|
||||
# Process Management (for manual gateway runs)
|
||||
# =============================================================================
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GatewayRuntimeSnapshot:
|
||||
manager: str
|
||||
service_installed: bool = False
|
||||
service_running: bool = False
|
||||
gateway_pids: tuple[int, ...] = ()
|
||||
service_scope: str | None = None
|
||||
|
||||
@property
|
||||
def running(self) -> bool:
|
||||
return self.service_running or bool(self.gateway_pids)
|
||||
|
||||
@property
|
||||
def has_process_service_mismatch(self) -> bool:
|
||||
return self.service_installed and self.running and not self.service_running
|
||||
|
||||
def _get_service_pids() -> set:
|
||||
"""Return PIDs currently managed by systemd or launchd gateway services.
|
||||
|
||||
@@ -175,22 +157,20 @@ def _request_gateway_self_restart(pid: int) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _append_unique_pid(pids: list[int], pid: int | None, exclude_pids: set[int]) -> None:
|
||||
if pid is None or pid <= 0:
|
||||
return
|
||||
if pid == os.getpid() or pid in exclude_pids or pid in pids:
|
||||
return
|
||||
pids.append(pid)
|
||||
def find_gateway_pids(exclude_pids: set | None = None, all_profiles: bool = False) -> list:
|
||||
"""Find PIDs of running gateway processes.
|
||||
|
||||
|
||||
def _scan_gateway_pids(exclude_pids: set[int], all_profiles: bool = False) -> list[int]:
|
||||
"""Best-effort process-table scan for gateway PIDs.
|
||||
|
||||
This supplements the profile-scoped PID file so status views can still spot
|
||||
a live gateway when the PID file is stale/missing, and ``--all`` sweeps can
|
||||
discover gateways outside the current profile.
|
||||
Args:
|
||||
exclude_pids: PIDs to exclude from the result (e.g. service-managed
|
||||
PIDs that should not be killed during a stale-process sweep).
|
||||
all_profiles: When ``True``, return gateway PIDs across **all**
|
||||
profiles (the pre-7923 global behaviour). ``hermes update``
|
||||
needs this because a code update affects every profile.
|
||||
When ``False`` (default), only PIDs belonging to the current
|
||||
Hermes profile are returned.
|
||||
"""
|
||||
pids: list[int] = []
|
||||
_exclude = exclude_pids or set()
|
||||
pids = [pid for pid in _get_service_pids() if pid not in _exclude]
|
||||
patterns = [
|
||||
"hermes_cli.main gateway",
|
||||
"hermes_cli.main --profile",
|
||||
@@ -223,24 +203,20 @@ def _scan_gateway_pids(exclude_pids: set[int], all_profiles: bool = False) -> li
|
||||
if is_windows():
|
||||
result = subprocess.run(
|
||||
["wmic", "process", "get", "ProcessId,CommandLine", "/FORMAT:LIST"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
capture_output=True, text=True, timeout=10
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
current_cmd = ""
|
||||
for line in result.stdout.split("\n"):
|
||||
for line in result.stdout.split('\n'):
|
||||
line = line.strip()
|
||||
if line.startswith("CommandLine="):
|
||||
current_cmd = line[len("CommandLine="):]
|
||||
elif line.startswith("ProcessId="):
|
||||
pid_str = line[len("ProcessId="):]
|
||||
if any(p in current_cmd for p in patterns) and (
|
||||
all_profiles or _matches_current_profile(current_cmd)
|
||||
):
|
||||
if any(p in current_cmd for p in patterns) and (all_profiles or _matches_current_profile(current_cmd)):
|
||||
try:
|
||||
_append_unique_pid(pids, int(pid_str), exclude_pids)
|
||||
pid = int(pid_str)
|
||||
if pid != os.getpid() and pid not in pids and pid not in _exclude:
|
||||
pids.append(pid)
|
||||
except ValueError:
|
||||
pass
|
||||
current_cmd = ""
|
||||
@@ -251,11 +227,9 @@ def _scan_gateway_pids(exclude_pids: set[int], all_profiles: bool = False) -> li
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
for line in result.stdout.split("\n"):
|
||||
for line in result.stdout.split('\n'):
|
||||
stripped = line.strip()
|
||||
if not stripped or "grep" in stripped:
|
||||
if not stripped or 'grep' in stripped:
|
||||
continue
|
||||
|
||||
pid = None
|
||||
@@ -277,137 +251,16 @@ def _scan_gateway_pids(exclude_pids: set[int], all_profiles: bool = False) -> li
|
||||
|
||||
if pid is None:
|
||||
continue
|
||||
if any(pattern in command for pattern in patterns) and (
|
||||
all_profiles or _matches_current_profile(command)
|
||||
):
|
||||
_append_unique_pid(pids, pid, exclude_pids)
|
||||
if pid == os.getpid() or pid in pids or pid in _exclude:
|
||||
continue
|
||||
if any(pattern in command for pattern in patterns) and (all_profiles or _matches_current_profile(command)):
|
||||
pids.append(pid)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return []
|
||||
pass
|
||||
|
||||
return pids
|
||||
|
||||
|
||||
def find_gateway_pids(exclude_pids: set | None = None, all_profiles: bool = False) -> list:
|
||||
"""Find PIDs of running gateway processes.
|
||||
|
||||
Args:
|
||||
exclude_pids: PIDs to exclude from the result (e.g. service-managed
|
||||
PIDs that should not be killed during a stale-process sweep).
|
||||
all_profiles: When ``True``, return gateway PIDs across **all**
|
||||
profiles (the pre-7923 global behaviour). ``hermes update``
|
||||
needs this because a code update affects every profile.
|
||||
When ``False`` (default), only PIDs belonging to the current
|
||||
Hermes profile are returned.
|
||||
"""
|
||||
_exclude = set(exclude_pids or set())
|
||||
pids: list[int] = []
|
||||
if not all_profiles:
|
||||
try:
|
||||
from gateway.status import get_running_pid
|
||||
|
||||
_append_unique_pid(pids, get_running_pid(), _exclude)
|
||||
except Exception:
|
||||
pass
|
||||
for pid in _get_service_pids():
|
||||
_append_unique_pid(pids, pid, _exclude)
|
||||
for pid in _scan_gateway_pids(_exclude, all_profiles=all_profiles):
|
||||
_append_unique_pid(pids, pid, _exclude)
|
||||
return pids
|
||||
|
||||
|
||||
def _probe_systemd_service_running(system: bool = False) -> tuple[bool, bool]:
|
||||
selected_system = _select_systemd_scope(system)
|
||||
unit_exists = get_systemd_unit_path(system=selected_system).exists()
|
||||
if not unit_exists:
|
||||
return selected_system, False
|
||||
try:
|
||||
result = _run_systemctl(
|
||||
["is-active", get_service_name()],
|
||||
system=selected_system,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
except (RuntimeError, subprocess.TimeoutExpired):
|
||||
return selected_system, False
|
||||
return selected_system, result.stdout.strip() == "active"
|
||||
|
||||
|
||||
def _probe_launchd_service_running() -> bool:
|
||||
if not get_launchd_plist_path().exists():
|
||||
return False
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["launchctl", "list", get_launchd_label()],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
return False
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def get_gateway_runtime_snapshot(system: bool = False) -> GatewayRuntimeSnapshot:
|
||||
"""Return a unified view of gateway liveness for the current profile."""
|
||||
gateway_pids = tuple(find_gateway_pids())
|
||||
if is_termux():
|
||||
return GatewayRuntimeSnapshot(
|
||||
manager="Termux / manual process",
|
||||
gateway_pids=gateway_pids,
|
||||
)
|
||||
|
||||
from hermes_constants import is_container
|
||||
|
||||
if is_linux() and is_container():
|
||||
return GatewayRuntimeSnapshot(
|
||||
manager="docker (foreground)",
|
||||
gateway_pids=gateway_pids,
|
||||
)
|
||||
|
||||
if supports_systemd_services():
|
||||
selected_system, service_running = _probe_systemd_service_running(system=system)
|
||||
scope_label = _service_scope_label(selected_system)
|
||||
return GatewayRuntimeSnapshot(
|
||||
manager=f"systemd ({scope_label})",
|
||||
service_installed=get_systemd_unit_path(system=selected_system).exists(),
|
||||
service_running=service_running,
|
||||
gateway_pids=gateway_pids,
|
||||
service_scope=scope_label,
|
||||
)
|
||||
|
||||
if is_macos():
|
||||
return GatewayRuntimeSnapshot(
|
||||
manager="launchd",
|
||||
service_installed=get_launchd_plist_path().exists(),
|
||||
service_running=_probe_launchd_service_running(),
|
||||
gateway_pids=gateway_pids,
|
||||
service_scope="launchd",
|
||||
)
|
||||
|
||||
return GatewayRuntimeSnapshot(
|
||||
manager="manual process",
|
||||
gateway_pids=gateway_pids,
|
||||
)
|
||||
|
||||
|
||||
def _format_gateway_pids(pids: tuple[int, ...] | list[int], *, limit: int | None = 3) -> str:
|
||||
rendered = [str(pid) for pid in pids[:limit] if pid > 0] if limit is not None else [str(pid) for pid in pids if pid > 0]
|
||||
if limit is not None and len(pids) > limit:
|
||||
rendered.append("...")
|
||||
return ", ".join(rendered)
|
||||
|
||||
|
||||
def _print_gateway_process_mismatch(snapshot: GatewayRuntimeSnapshot) -> None:
|
||||
if not snapshot.has_process_service_mismatch:
|
||||
return
|
||||
print()
|
||||
print("⚠ Gateway process is running for this profile, but the service is not active")
|
||||
print(f" PID(s): {_format_gateway_pids(snapshot.gateway_pids, limit=None)}")
|
||||
print(" This is usually a manual foreground/tmux/nohup run, so `hermes gateway`")
|
||||
print(" can refuse to start another copy until this process stops.")
|
||||
|
||||
|
||||
def kill_gateway_processes(force: bool = False, exclude_pids: set | None = None,
|
||||
all_profiles: bool = False) -> int:
|
||||
"""Kill any running gateway processes. Returns count killed.
|
||||
@@ -487,44 +340,25 @@ def _wsl_systemd_operational() -> bool:
|
||||
WSL2 with ``systemd=true`` in wsl.conf has working systemd.
|
||||
WSL2 without it (or WSL1) does not — systemctl commands fail.
|
||||
"""
|
||||
return _systemd_operational(system=True)
|
||||
|
||||
|
||||
def _systemd_operational(system: bool = False) -> bool:
|
||||
"""Return True when the requested systemd scope is usable."""
|
||||
try:
|
||||
result = _run_systemctl(
|
||||
["is-system-running"],
|
||||
system=system,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
result = subprocess.run(
|
||||
["systemctl", "is-system-running"],
|
||||
capture_output=True, text=True, timeout=5,
|
||||
)
|
||||
# "running", "degraded", "starting" all mean systemd is PID 1
|
||||
status = result.stdout.strip().lower()
|
||||
return status in ("running", "degraded", "starting", "initializing")
|
||||
except (RuntimeError, subprocess.TimeoutExpired, OSError):
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired, OSError):
|
||||
return False
|
||||
|
||||
|
||||
def _container_systemd_operational() -> bool:
|
||||
"""Return True when a container exposes working user or system systemd."""
|
||||
if _systemd_operational(system=False):
|
||||
return True
|
||||
if _systemd_operational(system=True):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def supports_systemd_services() -> bool:
|
||||
if not is_linux() or is_termux():
|
||||
if not is_linux() or is_termux() or is_container():
|
||||
return False
|
||||
if shutil.which("systemctl") is None:
|
||||
return False
|
||||
if is_wsl():
|
||||
return _wsl_systemd_operational()
|
||||
if is_container():
|
||||
return _container_systemd_operational()
|
||||
return True
|
||||
|
||||
|
||||
@@ -687,195 +521,6 @@ def has_conflicting_systemd_units() -> bool:
|
||||
return len(get_installed_systemd_scopes()) > 1
|
||||
|
||||
|
||||
# Legacy service names from older Hermes installs that predate the
|
||||
# hermes-gateway rename. Kept as an explicit allowlist (NOT a glob) so
|
||||
# profile units (hermes-gateway-*.service) and unrelated third-party
|
||||
# "hermes" units are never matched.
|
||||
_LEGACY_SERVICE_NAMES: tuple[str, ...] = ("hermes.service",)
|
||||
|
||||
# ExecStart content markers that identify a unit as running our gateway.
|
||||
# A legacy unit is only flagged when its file contains one of these.
|
||||
_LEGACY_UNIT_EXECSTART_MARKERS: tuple[str, ...] = (
|
||||
"hermes_cli.main gateway",
|
||||
"hermes_cli/main.py gateway",
|
||||
"gateway/run.py",
|
||||
" hermes gateway ",
|
||||
"/hermes gateway ",
|
||||
)
|
||||
|
||||
|
||||
def _legacy_unit_search_paths() -> list[tuple[bool, Path]]:
|
||||
"""Return ``[(is_system, base_dir), ...]`` — directories to scan for legacy units.
|
||||
|
||||
Factored out so tests can monkeypatch the search roots without touching
|
||||
real filesystem paths.
|
||||
"""
|
||||
return [
|
||||
(False, Path.home() / ".config" / "systemd" / "user"),
|
||||
(True, Path("/etc/systemd/system")),
|
||||
]
|
||||
|
||||
|
||||
def _find_legacy_hermes_units() -> list[tuple[str, Path, bool]]:
|
||||
"""Return ``[(unit_name, unit_path, is_system)]`` for legacy Hermes gateway units.
|
||||
|
||||
Detects unit files installed by older Hermes versions that used a
|
||||
different service name (e.g. ``hermes.service`` before the rename to
|
||||
``hermes-gateway.service``). When both a legacy unit and the current
|
||||
``hermes-gateway.service`` are active, they fight over the same bot
|
||||
token — the PR #5646 signal-recovery change turns this into a 30-second
|
||||
SIGTERM flap loop.
|
||||
|
||||
Safety guards:
|
||||
|
||||
* Explicit allowlist of legacy names (no globbing). Profile units such
|
||||
as ``hermes-gateway-coder.service`` and unrelated third-party
|
||||
``hermes-*`` services are never matched.
|
||||
* ExecStart content check — only flag units that invoke our gateway
|
||||
entrypoint. A user-created ``hermes.service`` running an unrelated
|
||||
binary is left untouched.
|
||||
* Results are returned purely for caller inspection; this function
|
||||
never mutates or removes anything.
|
||||
"""
|
||||
results: list[tuple[str, Path, bool]] = []
|
||||
for is_system, base in _legacy_unit_search_paths():
|
||||
for name in _LEGACY_SERVICE_NAMES:
|
||||
unit_path = base / name
|
||||
try:
|
||||
if not unit_path.exists():
|
||||
continue
|
||||
text = unit_path.read_text(encoding="utf-8", errors="ignore")
|
||||
except (OSError, PermissionError):
|
||||
continue
|
||||
if not any(marker in text for marker in _LEGACY_UNIT_EXECSTART_MARKERS):
|
||||
# Not our gateway — leave alone
|
||||
continue
|
||||
results.append((name, unit_path, is_system))
|
||||
return results
|
||||
|
||||
|
||||
def has_legacy_hermes_units() -> bool:
|
||||
"""Return True when any legacy Hermes gateway unit files exist."""
|
||||
return bool(_find_legacy_hermes_units())
|
||||
|
||||
|
||||
def print_legacy_unit_warning() -> None:
|
||||
"""Warn about legacy Hermes gateway unit files if any are installed.
|
||||
|
||||
Idempotent: prints nothing when no legacy units are detected. Safe to
|
||||
call from any status/install/setup path.
|
||||
"""
|
||||
legacy = _find_legacy_hermes_units()
|
||||
if not legacy:
|
||||
return
|
||||
print_warning("Legacy Hermes gateway unit(s) detected from an older install:")
|
||||
for name, path, is_system in legacy:
|
||||
scope = "system" if is_system else "user"
|
||||
print_info(f" {path} ({scope} scope)")
|
||||
print_info(" These run alongside the current hermes-gateway service and")
|
||||
print_info(" cause SIGTERM flap loops — both try to use the same bot token.")
|
||||
print_info(" Remove them with:")
|
||||
print_info(" hermes gateway migrate-legacy")
|
||||
|
||||
|
||||
def remove_legacy_hermes_units(
|
||||
interactive: bool = True,
|
||||
dry_run: bool = False,
|
||||
) -> tuple[int, list[Path]]:
|
||||
"""Stop, disable, and remove legacy Hermes gateway unit files.
|
||||
|
||||
Iterates over whatever ``_find_legacy_hermes_units()`` returns — which is
|
||||
an explicit allowlist of legacy names (not a glob). Profile units and
|
||||
unrelated third-party services are never touched.
|
||||
|
||||
Args:
|
||||
interactive: When True, prompt before removing. When False, remove
|
||||
without asking (used when another prompt has already confirmed,
|
||||
e.g. from the install flow).
|
||||
dry_run: When True, list what would be removed and return.
|
||||
|
||||
Returns:
|
||||
``(removed_count, remaining_paths)`` — remaining includes units we
|
||||
couldn't remove (typically system-scope when not running as root).
|
||||
"""
|
||||
legacy = _find_legacy_hermes_units()
|
||||
if not legacy:
|
||||
print("No legacy Hermes gateway units found.")
|
||||
return 0, []
|
||||
|
||||
user_units = [(n, p) for n, p, is_sys in legacy if not is_sys]
|
||||
system_units = [(n, p) for n, p, is_sys in legacy if is_sys]
|
||||
|
||||
print()
|
||||
print("Legacy Hermes gateway unit(s) found:")
|
||||
for name, path, is_system in legacy:
|
||||
scope = "system" if is_system else "user"
|
||||
print(f" {path} ({scope} scope)")
|
||||
print()
|
||||
|
||||
if dry_run:
|
||||
print("(dry-run — nothing removed)")
|
||||
return 0, [p for _, p, _ in legacy]
|
||||
|
||||
if interactive and not prompt_yes_no("Remove these legacy units?", True):
|
||||
print("Skipped. Run again with: hermes gateway migrate-legacy")
|
||||
return 0, [p for _, p, _ in legacy]
|
||||
|
||||
removed = 0
|
||||
remaining: list[Path] = []
|
||||
|
||||
# User-scope removal
|
||||
for name, path in user_units:
|
||||
try:
|
||||
_run_systemctl(["stop", name], system=False, check=False, timeout=90)
|
||||
_run_systemctl(["disable", name], system=False, check=False, timeout=30)
|
||||
path.unlink(missing_ok=True)
|
||||
print(f" ✓ Removed {path}")
|
||||
removed += 1
|
||||
except (OSError, RuntimeError) as e:
|
||||
print(f" ⚠ Could not remove {path}: {e}")
|
||||
remaining.append(path)
|
||||
|
||||
if user_units:
|
||||
try:
|
||||
_run_systemctl(["daemon-reload"], system=False, check=False, timeout=30)
|
||||
except RuntimeError:
|
||||
pass
|
||||
|
||||
# System-scope removal (needs root)
|
||||
if system_units:
|
||||
if os.geteuid() != 0:
|
||||
print()
|
||||
print_warning("System-scope legacy units require root to remove.")
|
||||
print_info(" Re-run with: sudo hermes gateway migrate-legacy")
|
||||
for _, path in system_units:
|
||||
remaining.append(path)
|
||||
else:
|
||||
for name, path in system_units:
|
||||
try:
|
||||
_run_systemctl(["stop", name], system=True, check=False, timeout=90)
|
||||
_run_systemctl(["disable", name], system=True, check=False, timeout=30)
|
||||
path.unlink(missing_ok=True)
|
||||
print(f" ✓ Removed {path}")
|
||||
removed += 1
|
||||
except (OSError, RuntimeError) as e:
|
||||
print(f" ⚠ Could not remove {path}: {e}")
|
||||
remaining.append(path)
|
||||
|
||||
try:
|
||||
_run_systemctl(["daemon-reload"], system=True, check=False, timeout=30)
|
||||
except RuntimeError:
|
||||
pass
|
||||
|
||||
print()
|
||||
if remaining:
|
||||
print_warning(f"{len(remaining)} legacy unit(s) still present — see messages above.")
|
||||
else:
|
||||
print_success(f"Removed {removed} legacy unit(s).")
|
||||
|
||||
return removed, remaining
|
||||
|
||||
|
||||
def print_systemd_scope_conflict_warning() -> None:
|
||||
scopes = get_installed_systemd_scopes()
|
||||
if len(scopes) < 2:
|
||||
@@ -1409,19 +1054,6 @@ def systemd_install(force: bool = False, system: bool = False, run_as_user: str
|
||||
if system:
|
||||
_require_root_for_system_service("install")
|
||||
|
||||
# Offer to remove legacy units (hermes.service from pre-rename installs)
|
||||
# before installing the new hermes-gateway.service. If both remain, they
|
||||
# flap-fight for the Telegram bot token on every gateway startup.
|
||||
# Only removes units matching _LEGACY_SERVICE_NAMES + our ExecStart
|
||||
# signature — profile units are never touched.
|
||||
if has_legacy_hermes_units():
|
||||
print()
|
||||
print_legacy_unit_warning()
|
||||
print()
|
||||
if prompt_yes_no("Remove the legacy unit(s) before installing?", True):
|
||||
remove_legacy_hermes_units(interactive=False)
|
||||
print()
|
||||
|
||||
unit_path = get_systemd_unit_path(system=system)
|
||||
scope_flag = " --system" if system else ""
|
||||
|
||||
@@ -1460,7 +1092,6 @@ def systemd_install(force: bool = False, system: bool = False, run_as_user: str
|
||||
_ensure_linger_enabled()
|
||||
|
||||
print_systemd_scope_conflict_warning()
|
||||
print_legacy_unit_warning()
|
||||
|
||||
|
||||
def systemd_uninstall(system: bool = False):
|
||||
@@ -1584,10 +1215,6 @@ def systemd_status(deep: bool = False, system: bool = False):
|
||||
print_systemd_scope_conflict_warning()
|
||||
print()
|
||||
|
||||
if has_legacy_hermes_units():
|
||||
print_legacy_unit_warning()
|
||||
print()
|
||||
|
||||
if not systemd_unit_is_current(system=system):
|
||||
print("⚠ Installed gateway service definition is outdated")
|
||||
print(f" Run: {'sudo ' if system else ''}hermes gateway restart{scope_flag} # auto-refreshes the unit")
|
||||
@@ -2371,7 +1998,7 @@ _PLATFORMS = [
|
||||
{"name": "QQ_ALLOWED_USERS", "prompt": "Allowed user OpenIDs (comma-separated, leave empty for open access)", "password": False,
|
||||
"is_allowlist": True,
|
||||
"help": "Optional — restrict DM access to specific user OpenIDs."},
|
||||
{"name": "QQBOT_HOME_CHANNEL", "prompt": "Home channel (user/group OpenID for cron delivery, or empty)", "password": False,
|
||||
{"name": "QQ_HOME_CHANNEL", "prompt": "Home channel (user/group OpenID for cron delivery, or empty)", "password": False,
|
||||
"help": "OpenID to deliver cron results and notifications to."},
|
||||
],
|
||||
},
|
||||
@@ -2584,62 +2211,9 @@ def _setup_sms():
|
||||
|
||||
|
||||
def _setup_dingtalk():
|
||||
"""Configure DingTalk — QR scan (recommended) or manual credential entry."""
|
||||
from hermes_cli.setup import (
|
||||
prompt_choice, prompt_yes_no, print_info, print_success, print_warning,
|
||||
)
|
||||
|
||||
"""Configure DingTalk via the standard platform setup."""
|
||||
dingtalk_platform = next(p for p in _PLATFORMS if p["key"] == "dingtalk")
|
||||
emoji = dingtalk_platform["emoji"]
|
||||
label = dingtalk_platform["label"]
|
||||
|
||||
print()
|
||||
print(color(f" ─── {emoji} {label} Setup ───", Colors.CYAN))
|
||||
|
||||
existing = get_env_value("DINGTALK_CLIENT_ID")
|
||||
if existing:
|
||||
print()
|
||||
print_success(f"{label} is already configured (Client ID: {existing}).")
|
||||
if not prompt_yes_no(f" Reconfigure {label}?", False):
|
||||
return
|
||||
|
||||
print()
|
||||
method = prompt_choice(
|
||||
" Choose setup method",
|
||||
[
|
||||
"QR Code Scan (Recommended, auto-obtain Client ID and Client Secret)",
|
||||
"Manual Input (Client ID and Client Secret)",
|
||||
],
|
||||
default=0,
|
||||
)
|
||||
|
||||
if method == 0:
|
||||
# ── QR-code device-flow authorization ──
|
||||
try:
|
||||
from hermes_cli.dingtalk_auth import dingtalk_qr_auth
|
||||
except ImportError as exc:
|
||||
print_warning(f" QR auth module failed to load ({exc}), falling back to manual input.")
|
||||
_setup_standard_platform(dingtalk_platform)
|
||||
return
|
||||
|
||||
result = dingtalk_qr_auth()
|
||||
if result is None:
|
||||
print_warning(" QR auth incomplete, falling back to manual input.")
|
||||
_setup_standard_platform(dingtalk_platform)
|
||||
return
|
||||
|
||||
client_id, client_secret = result
|
||||
save_env_value("DINGTALK_CLIENT_ID", client_id)
|
||||
save_env_value("DINGTALK_CLIENT_SECRET", client_secret)
|
||||
save_env_value("DINGTALK_ALLOW_ALL_USERS", "true")
|
||||
print()
|
||||
print_success(f"{emoji} {label} configured via QR scan!")
|
||||
else:
|
||||
# ── Manual entry ──
|
||||
_setup_standard_platform(dingtalk_platform)
|
||||
# Also enable allow-all by default for convenience
|
||||
if get_env_value("DINGTALK_CLIENT_ID"):
|
||||
save_env_value("DINGTALK_ALLOW_ALL_USERS", "true")
|
||||
_setup_standard_platform(dingtalk_platform)
|
||||
|
||||
|
||||
def _setup_wecom():
|
||||
@@ -2998,215 +2572,6 @@ def _setup_feishu():
|
||||
print_info(f" Bot: {bot_name}")
|
||||
|
||||
|
||||
def _setup_qqbot():
|
||||
"""Interactive setup for QQ Bot — scan-to-configure or manual credentials."""
|
||||
print()
|
||||
print(color(" ─── 🐧 QQ Bot Setup ───", Colors.CYAN))
|
||||
|
||||
existing_app_id = get_env_value("QQ_APP_ID")
|
||||
existing_secret = get_env_value("QQ_CLIENT_SECRET")
|
||||
if existing_app_id and existing_secret:
|
||||
print()
|
||||
print_success("QQ Bot is already configured.")
|
||||
if not prompt_yes_no(" Reconfigure QQ Bot?", False):
|
||||
return
|
||||
|
||||
# ── Choose setup method ──
|
||||
print()
|
||||
method_choices = [
|
||||
"Scan QR code to add bot automatically (recommended)",
|
||||
"Enter existing App ID and App Secret manually",
|
||||
]
|
||||
method_idx = prompt_choice(" How would you like to set up QQ Bot?", method_choices, 0)
|
||||
|
||||
credentials = None
|
||||
used_qr = False
|
||||
|
||||
if method_idx == 0:
|
||||
# ── QR scan-to-configure ──
|
||||
try:
|
||||
credentials = _qqbot_qr_flow()
|
||||
except KeyboardInterrupt:
|
||||
print()
|
||||
print_warning(" QQ Bot setup cancelled.")
|
||||
return
|
||||
if credentials:
|
||||
used_qr = True
|
||||
if not credentials:
|
||||
print_info(" QR setup did not complete. Continuing with manual input.")
|
||||
|
||||
# ── Manual credential input ──
|
||||
if not credentials:
|
||||
print()
|
||||
print_info(" Go to https://q.qq.com to register a QQ Bot application.")
|
||||
print_info(" Note your App ID and App Secret from the application page.")
|
||||
print()
|
||||
app_id = prompt(" App ID", password=False)
|
||||
if not app_id:
|
||||
print_warning(" Skipped — QQ Bot won't work without an App ID.")
|
||||
return
|
||||
app_secret = prompt(" App Secret", password=True)
|
||||
if not app_secret:
|
||||
print_warning(" Skipped — QQ Bot won't work without an App Secret.")
|
||||
return
|
||||
credentials = {"app_id": app_id.strip(), "client_secret": app_secret.strip(), "user_openid": ""}
|
||||
|
||||
# ── Save core credentials ──
|
||||
save_env_value("QQ_APP_ID", credentials["app_id"])
|
||||
save_env_value("QQ_CLIENT_SECRET", credentials["client_secret"])
|
||||
|
||||
user_openid = credentials.get("user_openid", "")
|
||||
|
||||
# ── DM security policy ──
|
||||
print()
|
||||
access_choices = [
|
||||
"Use DM pairing approval (recommended)",
|
||||
"Allow all direct messages",
|
||||
"Only allow listed user OpenIDs",
|
||||
]
|
||||
access_idx = prompt_choice(" How should direct messages be authorized?", access_choices, 0)
|
||||
if access_idx == 0:
|
||||
save_env_value("QQ_ALLOW_ALL_USERS", "false")
|
||||
if user_openid:
|
||||
print()
|
||||
if prompt_yes_no(f" Add yourself ({user_openid}) to the allow list?", True):
|
||||
save_env_value("QQ_ALLOWED_USERS", user_openid)
|
||||
print_success(f" Allow list set to {user_openid}")
|
||||
else:
|
||||
save_env_value("QQ_ALLOWED_USERS", "")
|
||||
else:
|
||||
save_env_value("QQ_ALLOWED_USERS", "")
|
||||
print_success(" DM pairing enabled.")
|
||||
print_info(" Unknown users can request access; approve with `hermes pairing approve`.")
|
||||
elif access_idx == 1:
|
||||
save_env_value("QQ_ALLOW_ALL_USERS", "true")
|
||||
save_env_value("QQ_ALLOWED_USERS", "")
|
||||
print_warning(" Open DM access enabled for QQ Bot.")
|
||||
else:
|
||||
default_allow = user_openid or ""
|
||||
allowlist = prompt(" Allowed user OpenIDs (comma-separated)", default_allow, password=False).replace(" ", "")
|
||||
save_env_value("QQ_ALLOW_ALL_USERS", "false")
|
||||
save_env_value("QQ_ALLOWED_USERS", allowlist)
|
||||
print_success(" Allowlist saved.")
|
||||
|
||||
# ── Home channel ──
|
||||
if user_openid:
|
||||
print()
|
||||
if prompt_yes_no(f" Use your QQ user ID ({user_openid}) as the home channel?", True):
|
||||
save_env_value("QQBOT_HOME_CHANNEL", user_openid)
|
||||
print_success(f" Home channel set to {user_openid}")
|
||||
else:
|
||||
print()
|
||||
home_channel = prompt(" Home channel OpenID (for cron/notifications, or empty)", password=False)
|
||||
if home_channel:
|
||||
save_env_value("QQBOT_HOME_CHANNEL", home_channel.strip())
|
||||
print_success(f" Home channel set to {home_channel.strip()}")
|
||||
|
||||
print()
|
||||
print_success("🐧 QQ Bot configured!")
|
||||
print_info(f" App ID: {credentials['app_id']}")
|
||||
|
||||
|
||||
def _qqbot_render_qr(url: str) -> bool:
|
||||
"""Try to render a QR code in the terminal. Returns True if successful."""
|
||||
try:
|
||||
import qrcode as _qr
|
||||
qr = _qr.QRCode(border=1,error_correction=_qr.constants.ERROR_CORRECT_L)
|
||||
qr.add_data(url)
|
||||
qr.make(fit=True)
|
||||
qr.print_ascii(invert=True)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _qqbot_qr_flow():
|
||||
"""Run the QR-code scan-to-configure flow.
|
||||
|
||||
Returns a dict with app_id, client_secret, user_openid on success,
|
||||
or None on failure/cancel.
|
||||
"""
|
||||
try:
|
||||
from gateway.platforms.qqbot import (
|
||||
create_bind_task, poll_bind_result, build_connect_url,
|
||||
decrypt_secret, BindStatus,
|
||||
)
|
||||
from gateway.platforms.qqbot.constants import ONBOARD_POLL_INTERVAL
|
||||
except Exception as exc:
|
||||
print_error(f" QQBot onboard import failed: {exc}")
|
||||
return None
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
MAX_REFRESHES = 3
|
||||
refresh_count = 0
|
||||
|
||||
while refresh_count <= MAX_REFRESHES:
|
||||
loop = asyncio.new_event_loop()
|
||||
|
||||
# ── Create bind task ──
|
||||
try:
|
||||
task_id, aes_key = loop.run_until_complete(create_bind_task())
|
||||
except Exception as e:
|
||||
print_warning(f" Failed to create bind task: {e}")
|
||||
loop.close()
|
||||
return None
|
||||
|
||||
url = build_connect_url(task_id)
|
||||
|
||||
# ── Display QR code + URL ──
|
||||
print()
|
||||
if _qqbot_render_qr(url):
|
||||
print(f" Scan the QR code above, or open this URL directly:\n {url}")
|
||||
else:
|
||||
print(f" Open this URL in QQ on your phone:\n {url}")
|
||||
print_info(" Tip: pip install qrcode to show a scannable QR code here")
|
||||
|
||||
# ── Poll loop (silent — keep QR visible at bottom) ──
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
status, app_id, encrypted_secret, user_openid = loop.run_until_complete(
|
||||
poll_bind_result(task_id)
|
||||
)
|
||||
except Exception:
|
||||
time.sleep(ONBOARD_POLL_INTERVAL)
|
||||
continue
|
||||
|
||||
if status == BindStatus.COMPLETED:
|
||||
client_secret = decrypt_secret(encrypted_secret, aes_key)
|
||||
print()
|
||||
print_success(f" QR scan complete! (App ID: {app_id})")
|
||||
if user_openid:
|
||||
print_info(f" Scanner's OpenID: {user_openid}")
|
||||
return {
|
||||
"app_id": app_id,
|
||||
"client_secret": client_secret,
|
||||
"user_openid": user_openid,
|
||||
}
|
||||
|
||||
if status == BindStatus.EXPIRED:
|
||||
refresh_count += 1
|
||||
if refresh_count > MAX_REFRESHES:
|
||||
print()
|
||||
print_warning(f" QR code expired {MAX_REFRESHES} times — giving up.")
|
||||
return None
|
||||
print()
|
||||
print_warning(f" QR code expired, refreshing... ({refresh_count}/{MAX_REFRESHES})")
|
||||
loop.close()
|
||||
break # outer while creates a new task
|
||||
|
||||
time.sleep(ONBOARD_POLL_INTERVAL)
|
||||
except KeyboardInterrupt:
|
||||
loop.close()
|
||||
raise
|
||||
finally:
|
||||
loop.close()
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _setup_signal():
|
||||
"""Interactive setup for Signal messenger."""
|
||||
import shutil
|
||||
@@ -3344,10 +2709,6 @@ def gateway_setup():
|
||||
print_systemd_scope_conflict_warning()
|
||||
print()
|
||||
|
||||
if supports_systemd_services() and has_legacy_hermes_units():
|
||||
print_legacy_unit_warning()
|
||||
print()
|
||||
|
||||
if service_installed and service_running:
|
||||
print_success("Gateway service is installed and running.")
|
||||
elif service_installed:
|
||||
@@ -3388,12 +2749,8 @@ def gateway_setup():
|
||||
_setup_signal()
|
||||
elif platform["key"] == "weixin":
|
||||
_setup_weixin()
|
||||
elif platform["key"] == "dingtalk":
|
||||
_setup_dingtalk()
|
||||
elif platform["key"] == "feishu":
|
||||
_setup_feishu()
|
||||
elif platform["key"] == "qqbot":
|
||||
_setup_qqbot()
|
||||
else:
|
||||
_setup_standard_platform(platform)
|
||||
|
||||
@@ -3753,18 +3110,15 @@ def gateway_command(args):
|
||||
elif subcmd == "status":
|
||||
deep = getattr(args, 'deep', False)
|
||||
system = getattr(args, 'system', False)
|
||||
snapshot = get_gateway_runtime_snapshot(system=system)
|
||||
|
||||
# Check for service first
|
||||
if supports_systemd_services() and (get_systemd_unit_path(system=False).exists() or get_systemd_unit_path(system=True).exists()):
|
||||
systemd_status(deep, system=system)
|
||||
_print_gateway_process_mismatch(snapshot)
|
||||
elif is_macos() and get_launchd_plist_path().exists():
|
||||
launchd_status(deep)
|
||||
_print_gateway_process_mismatch(snapshot)
|
||||
else:
|
||||
# Check for manually running processes
|
||||
pids = list(snapshot.gateway_pids)
|
||||
pids = find_gateway_pids()
|
||||
if pids:
|
||||
print(f"✓ Gateway is running (PID: {', '.join(map(str, pids))})")
|
||||
print(" (Running manually, not as a system service)")
|
||||
@@ -3805,14 +3159,3 @@ def gateway_command(args):
|
||||
else:
|
||||
print(" hermes gateway install # Install as user service")
|
||||
print(" sudo hermes gateway install --system # Install as boot-time system service")
|
||||
|
||||
elif subcmd == "migrate-legacy":
|
||||
# Stop, disable, and remove legacy Hermes gateway unit files from
|
||||
# pre-rename installs (e.g. hermes.service). Profile units and
|
||||
# unrelated third-party services are never touched.
|
||||
dry_run = getattr(args, 'dry_run', False)
|
||||
yes = getattr(args, 'yes', False)
|
||||
if not supports_systemd_services() and not is_macos():
|
||||
print("Legacy unit migration only applies to systemd-based Linux hosts.")
|
||||
return
|
||||
remove_legacy_hermes_units(interactive=not yes, dry_run=dry_run)
|
||||
|
||||
+688
-2586
File diff suppressed because it is too large
Load Diff
@@ -374,26 +374,7 @@ def normalize_model_for_provider(model_input: str, target_provider: str) -> str:
|
||||
return bare
|
||||
return _dots_to_hyphens(bare)
|
||||
|
||||
# --- Copilot / Copilot ACP: delegate to the Copilot-specific
|
||||
# normalizer. It knows about the alias table (vendor-prefix
|
||||
# stripping for Anthropic/OpenAI, dash-to-dot repair for Claude)
|
||||
# and live-catalog lookups. Without this, vendor-prefixed or
|
||||
# dash-notation Claude IDs survive to the Copilot API and hit
|
||||
# HTTP 400 "model_not_supported". See issue #6879.
|
||||
if provider in {"copilot", "copilot-acp"}:
|
||||
try:
|
||||
from hermes_cli.models import normalize_copilot_model_id
|
||||
|
||||
normalized = normalize_copilot_model_id(name)
|
||||
if normalized:
|
||||
return normalized
|
||||
except Exception:
|
||||
# Fall through to the generic strip-vendor behaviour below
|
||||
# if the Copilot-specific path is unavailable for any reason.
|
||||
pass
|
||||
|
||||
# --- Copilot / Copilot ACP / openai-codex fallback:
|
||||
# strip matching provider prefix, keep dots ---
|
||||
# --- Copilot: strip matching provider prefix, keep dots ---
|
||||
if provider in _STRIP_VENDOR_ONLY_PROVIDERS:
|
||||
stripped = _strip_matching_provider_prefix(name, provider)
|
||||
if stripped == name and name.startswith("openai/"):
|
||||
|
||||
@@ -692,12 +692,12 @@ def switch_model(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
)
|
||||
except Exception as e:
|
||||
except Exception:
|
||||
validation = {
|
||||
"accepted": False,
|
||||
"persist": False,
|
||||
"accepted": True,
|
||||
"persist": True,
|
||||
"recognized": False,
|
||||
"message": f"Could not validate `{new_model}`: {e}",
|
||||
"message": None,
|
||||
}
|
||||
|
||||
if not validation.get("accepted"):
|
||||
|
||||
+31
-58
@@ -26,8 +26,7 @@ COPILOT_REASONING_EFFORTS_O_SERIES = ["low", "medium", "high"]
|
||||
# Fallback OpenRouter snapshot used when the live catalog is unavailable.
|
||||
# (model_id, display description shown in menus)
|
||||
OPENROUTER_MODELS: list[tuple[str, str]] = [
|
||||
("moonshotai/kimi-k2.5", "recommended"),
|
||||
("anthropic/claude-opus-4.7", ""),
|
||||
("anthropic/claude-opus-4.7", "recommended"),
|
||||
("anthropic/claude-opus-4.6", ""),
|
||||
("anthropic/claude-sonnet-4.6", ""),
|
||||
("qwen/qwen3.6-plus", ""),
|
||||
@@ -50,6 +49,7 @@ OPENROUTER_MODELS: list[tuple[str, str]] = [
|
||||
("z-ai/glm-5.1", ""),
|
||||
("z-ai/glm-5v-turbo", ""),
|
||||
("z-ai/glm-5-turbo", ""),
|
||||
("moonshotai/kimi-k2.5", ""),
|
||||
("x-ai/grok-4.20", ""),
|
||||
("nvidia/nemotron-3-super-120b-a12b", ""),
|
||||
("nvidia/nemotron-3-super-120b-a12b:free", "free"),
|
||||
@@ -75,7 +75,6 @@ def _codex_curated_models() -> list[str]:
|
||||
|
||||
_PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
"nous": [
|
||||
"moonshotai/kimi-k2.5",
|
||||
"xiaomi/mimo-v2-pro",
|
||||
"anthropic/claude-opus-4.7",
|
||||
"anthropic/claude-opus-4.6",
|
||||
@@ -97,6 +96,7 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
"z-ai/glm-5.1",
|
||||
"z-ai/glm-5v-turbo",
|
||||
"z-ai/glm-5-turbo",
|
||||
"moonshotai/kimi-k2.5",
|
||||
"x-ai/grok-4.20-beta",
|
||||
"nvidia/nemotron-3-super-120b-a12b",
|
||||
"nvidia/nemotron-3-super-120b-a12b:free",
|
||||
@@ -133,6 +133,9 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
"gemini-2.5-pro",
|
||||
"gemini-2.5-flash",
|
||||
"gemini-2.5-flash-lite",
|
||||
# Gemma open models (also served via AI Studio)
|
||||
"gemma-4-31b-it",
|
||||
"gemma-4-26b-it",
|
||||
],
|
||||
"google-gemini-cli": [
|
||||
"gemini-2.5-pro",
|
||||
@@ -152,23 +155,9 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
"grok-4.20-reasoning",
|
||||
"grok-4-1-fast-reasoning",
|
||||
],
|
||||
"nvidia": [
|
||||
# NVIDIA flagship reasoning models
|
||||
"nvidia/nemotron-3-super-120b-a12b",
|
||||
"nvidia/nemotron-3-nano-30b-a3b",
|
||||
"nvidia/llama-3.3-nemotron-super-49b-v1.5",
|
||||
# Third-party agentic models hosted on build.nvidia.com
|
||||
# (map to OpenRouter defaults — users get familiar picks on NIM)
|
||||
"qwen/qwen3.5-397b-a17b",
|
||||
"deepseek-ai/deepseek-v3.2",
|
||||
"moonshotai/kimi-k2.5",
|
||||
"minimaxai/minimax-m2.5",
|
||||
"z-ai/glm5",
|
||||
"openai/gpt-oss-120b",
|
||||
],
|
||||
"kimi-coding": [
|
||||
"kimi-k2.5",
|
||||
"kimi-for-coding",
|
||||
"kimi-k2.5",
|
||||
"kimi-k2-thinking",
|
||||
"kimi-k2-thinking-turbo",
|
||||
"kimi-k2-turbo-preview",
|
||||
@@ -223,7 +212,6 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
"trinity-mini",
|
||||
],
|
||||
"opencode-zen": [
|
||||
"kimi-k2.5",
|
||||
"gpt-5.4-pro",
|
||||
"gpt-5.4",
|
||||
"gpt-5.3-codex",
|
||||
@@ -255,15 +243,16 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
"glm-5",
|
||||
"glm-4.7",
|
||||
"glm-4.6",
|
||||
"kimi-k2.5",
|
||||
"kimi-k2-thinking",
|
||||
"kimi-k2",
|
||||
"qwen3-coder",
|
||||
"big-pickle",
|
||||
],
|
||||
"opencode-go": [
|
||||
"kimi-k2.5",
|
||||
"glm-5.1",
|
||||
"glm-5",
|
||||
"kimi-k2.5",
|
||||
"mimo-v2-pro",
|
||||
"mimo-v2-omni",
|
||||
"minimax-m2.7",
|
||||
@@ -296,21 +285,21 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
# to https://dashscope-intl.aliyuncs.com/compatible-mode/v1 (OpenAI-compat)
|
||||
# or https://dashscope-intl.aliyuncs.com/apps/anthropic (Anthropic-compat).
|
||||
"alibaba": [
|
||||
"kimi-k2.5",
|
||||
"qwen3.5-plus",
|
||||
"qwen3-coder-plus",
|
||||
"qwen3-coder-next",
|
||||
# Third-party models available on coding-intl
|
||||
"glm-5",
|
||||
"glm-4.7",
|
||||
"kimi-k2.5",
|
||||
"MiniMax-M2.5",
|
||||
],
|
||||
# Curated HF model list — only agentic models that map to OpenRouter defaults.
|
||||
"huggingface": [
|
||||
"moonshotai/Kimi-K2.5",
|
||||
"Qwen/Qwen3.5-397B-A17B",
|
||||
"Qwen/Qwen3.5-35B-A3B",
|
||||
"deepseek-ai/DeepSeek-V3.2",
|
||||
"moonshotai/Kimi-K2.5",
|
||||
"MiniMaxAI/MiniMax-M2.5",
|
||||
"zai-org/GLM-5",
|
||||
"XiaomiMiMo/MiMo-V2-Flash",
|
||||
@@ -547,7 +536,6 @@ CANONICAL_PROVIDERS: list[ProviderEntry] = [
|
||||
ProviderEntry("anthropic", "Anthropic", "Anthropic (Claude models — API key or Claude Code)"),
|
||||
ProviderEntry("openai-codex", "OpenAI Codex", "OpenAI Codex"),
|
||||
ProviderEntry("xiaomi", "Xiaomi MiMo", "Xiaomi MiMo (MiMo-V2 models — pro, omni, flash)"),
|
||||
ProviderEntry("nvidia", "NVIDIA NIM", "NVIDIA NIM (Nemotron models — build.nvidia.com or local NIM)"),
|
||||
ProviderEntry("qwen-oauth", "Qwen OAuth (Portal)", "Qwen OAuth (reuses local Qwen CLI login)"),
|
||||
ProviderEntry("copilot", "GitHub Copilot", "GitHub Copilot (uses GITHUB_TOKEN or gh auth token)"),
|
||||
ProviderEntry("copilot-acp", "GitHub Copilot ACP", "GitHub Copilot ACP (spawns `copilot --acp --stdio`)"),
|
||||
@@ -630,10 +618,6 @@ _PROVIDER_ALIASES = {
|
||||
"grok": "xai",
|
||||
"x-ai": "xai",
|
||||
"x.ai": "xai",
|
||||
"nim": "nvidia",
|
||||
"nvidia-nim": "nvidia",
|
||||
"build-nvidia": "nvidia",
|
||||
"nemotron": "nvidia",
|
||||
"ollama": "custom", # bare "ollama" = local; use "ollama-cloud" for cloud
|
||||
"ollama_cloud": "ollama-cloud",
|
||||
}
|
||||
@@ -1504,19 +1488,6 @@ _COPILOT_MODEL_ALIASES = {
|
||||
"anthropic/claude-sonnet-4.6": "claude-sonnet-4.6",
|
||||
"anthropic/claude-sonnet-4.5": "claude-sonnet-4.5",
|
||||
"anthropic/claude-haiku-4.5": "claude-haiku-4.5",
|
||||
# Dash-notation fallbacks: Hermes' default Claude IDs elsewhere use
|
||||
# hyphens (anthropic native format), but Copilot's API only accepts
|
||||
# dot-notation. Accept both so users who configure copilot + a
|
||||
# default hyphenated Claude model don't hit HTTP 400
|
||||
# "model_not_supported". See issue #6879.
|
||||
"claude-opus-4-6": "claude-opus-4.6",
|
||||
"claude-sonnet-4-6": "claude-sonnet-4.6",
|
||||
"claude-sonnet-4-5": "claude-sonnet-4.5",
|
||||
"claude-haiku-4-5": "claude-haiku-4.5",
|
||||
"anthropic/claude-opus-4-6": "claude-opus-4.6",
|
||||
"anthropic/claude-sonnet-4-6": "claude-sonnet-4.6",
|
||||
"anthropic/claude-sonnet-4-5": "claude-sonnet-4.5",
|
||||
"anthropic/claude-haiku-4-5": "claude-haiku-4.5",
|
||||
}
|
||||
|
||||
|
||||
@@ -2048,8 +2019,8 @@ def validate_requested_model(
|
||||
)
|
||||
|
||||
return {
|
||||
"accepted": False,
|
||||
"persist": False,
|
||||
"accepted": True,
|
||||
"persist": True,
|
||||
"recognized": False,
|
||||
"message": message,
|
||||
}
|
||||
@@ -2062,8 +2033,8 @@ def validate_requested_model(
|
||||
message += f"\n If this server expects `/v1`, try base URL: `{probe.get('suggested_base_url')}`"
|
||||
|
||||
return {
|
||||
"accepted": False,
|
||||
"persist": False,
|
||||
"accepted": True,
|
||||
"persist": True,
|
||||
"recognized": False,
|
||||
"message": message,
|
||||
}
|
||||
@@ -2097,11 +2068,12 @@ def validate_requested_model(
|
||||
if suggestions:
|
||||
suggestion_text = "\n Similar models: " + ", ".join(f"`{s}`" for s in suggestions)
|
||||
return {
|
||||
"accepted": False,
|
||||
"persist": False,
|
||||
"accepted": True,
|
||||
"persist": True,
|
||||
"recognized": False,
|
||||
"message": (
|
||||
f"Model `{requested}` was not found in the OpenAI Codex model listing."
|
||||
f"Note: `{requested}` was not found in the OpenAI Codex model listing. "
|
||||
f"It may still work if your account has access to it."
|
||||
f"{suggestion_text}"
|
||||
),
|
||||
}
|
||||
@@ -2140,15 +2112,16 @@ def validate_requested_model(
|
||||
if suggestions:
|
||||
suggestion_text = "\n Similar models: " + ", ".join(f"`{s}`" for s in suggestions)
|
||||
|
||||
return {
|
||||
"accepted": False,
|
||||
"persist": False,
|
||||
"recognized": False,
|
||||
"message": (
|
||||
f"Model `{requested}` was not found in this provider's model listing."
|
||||
f"{suggestion_text}"
|
||||
),
|
||||
}
|
||||
return {
|
||||
"accepted": True,
|
||||
"persist": True,
|
||||
"recognized": False,
|
||||
"message": (
|
||||
f"Note: `{requested}` was not found in this provider's model listing. "
|
||||
f"It may still work if your plan supports it."
|
||||
f"{suggestion_text}"
|
||||
),
|
||||
}
|
||||
|
||||
# api_models is None — couldn't reach API. Accept and persist,
|
||||
# but warn so typos don't silently break things.
|
||||
@@ -2190,8 +2163,8 @@ def validate_requested_model(
|
||||
|
||||
provider_label = _PROVIDER_LABELS.get(normalized, normalized)
|
||||
return {
|
||||
"accepted": False,
|
||||
"persist": False,
|
||||
"accepted": True,
|
||||
"persist": True,
|
||||
"recognized": False,
|
||||
"message": (
|
||||
f"Could not reach the {provider_label} API to validate `{requested}`. "
|
||||
|
||||
+12
-3
@@ -300,10 +300,19 @@ def _read_config_model(profile_dir: Path) -> tuple:
|
||||
|
||||
def _check_gateway_running(profile_dir: Path) -> bool:
|
||||
"""Check if a gateway is running for a given profile directory."""
|
||||
pid_file = profile_dir / "gateway.pid"
|
||||
if not pid_file.exists():
|
||||
return False
|
||||
try:
|
||||
from gateway.status import get_running_pid
|
||||
return get_running_pid(profile_dir / "gateway.pid", cleanup_stale=False) is not None
|
||||
except Exception:
|
||||
raw = pid_file.read_text().strip()
|
||||
if not raw:
|
||||
return False
|
||||
data = json.loads(raw) if raw.startswith("{") else {"pid": int(raw)}
|
||||
pid = int(data["pid"])
|
||||
os.kill(pid, 0) # existence check
|
||||
return True
|
||||
except (json.JSONDecodeError, KeyError, ValueError, TypeError,
|
||||
ProcessLookupError, PermissionError, OSError):
|
||||
return False
|
||||
|
||||
|
||||
|
||||
@@ -137,11 +137,6 @@ HERMES_OVERLAYS: Dict[str, HermesOverlay] = {
|
||||
base_url_override="https://api.x.ai/v1",
|
||||
base_url_env_var="XAI_BASE_URL",
|
||||
),
|
||||
"nvidia": HermesOverlay(
|
||||
transport="openai_chat",
|
||||
base_url_override="https://integrate.api.nvidia.com/v1",
|
||||
base_url_env_var="NVIDIA_BASE_URL",
|
||||
),
|
||||
"xiaomi": HermesOverlay(
|
||||
transport="openai_chat",
|
||||
base_url_env_var="XIAOMI_BASE_URL",
|
||||
@@ -196,12 +191,6 @@ ALIASES: Dict[str, str] = {
|
||||
"x.ai": "xai",
|
||||
"grok": "xai",
|
||||
|
||||
# nvidia
|
||||
"nim": "nvidia",
|
||||
"nvidia-nim": "nvidia",
|
||||
"build-nvidia": "nvidia",
|
||||
"nemotron": "nvidia",
|
||||
|
||||
# kimi-for-coding (models.dev ID)
|
||||
"kimi": "kimi-for-coding",
|
||||
"kimi-coding": "kimi-for-coding",
|
||||
|
||||
+55
-15
@@ -91,6 +91,7 @@ _DEFAULT_PROVIDER_MODELS = {
|
||||
"gemini": [
|
||||
"gemini-3.1-pro-preview", "gemini-3-flash-preview", "gemini-3.1-flash-lite-preview",
|
||||
"gemini-2.5-pro", "gemini-2.5-flash", "gemini-2.5-flash-lite",
|
||||
"gemma-4-31b-it", "gemma-4-26b-it",
|
||||
],
|
||||
"zai": ["glm-5.1", "glm-5", "glm-4.7", "glm-4.5", "glm-4.5-flash"],
|
||||
"kimi-coding": ["kimi-k2.5", "kimi-k2-thinking", "kimi-k2-turbo-preview"],
|
||||
@@ -1460,9 +1461,7 @@ def setup_agent_settings(config: dict):
|
||||
)
|
||||
print_info("Maximum tool-calling iterations per conversation.")
|
||||
print_info("Higher = more complex tasks, but costs more tokens.")
|
||||
print_info(
|
||||
f"Press Enter to keep {current_max}. Use 90 for most tasks or 150+ for open exploration."
|
||||
)
|
||||
print_info("Default is 90, which works for most tasks. Use 150+ for open exploration.")
|
||||
|
||||
max_iter_str = prompt("Max iterations", current_max)
|
||||
try:
|
||||
@@ -2006,6 +2005,52 @@ def _setup_wecom_callback():
|
||||
_gw_setup()
|
||||
|
||||
|
||||
def _setup_qqbot():
|
||||
"""Configure QQ Bot gateway."""
|
||||
print_header("QQ Bot")
|
||||
existing = get_env_value("QQ_APP_ID")
|
||||
if existing:
|
||||
print_info("QQ Bot: already configured")
|
||||
if not prompt_yes_no("Reconfigure QQ Bot?", False):
|
||||
return
|
||||
|
||||
print_info("Connects Hermes to QQ via the Official QQ Bot API (v2).")
|
||||
print_info(" Requires a QQ Bot application at q.qq.com")
|
||||
print_info(" Reference: https://bot.q.qq.com/wiki/develop/api-v2/")
|
||||
print()
|
||||
|
||||
app_id = prompt("QQ Bot App ID")
|
||||
if not app_id:
|
||||
print_warning("App ID is required — skipping QQ Bot setup")
|
||||
return
|
||||
save_env_value("QQ_APP_ID", app_id.strip())
|
||||
|
||||
client_secret = prompt("QQ Bot App Secret", password=True)
|
||||
if not client_secret:
|
||||
print_warning("App Secret is required — skipping QQ Bot setup")
|
||||
return
|
||||
save_env_value("QQ_CLIENT_SECRET", client_secret)
|
||||
print_success("QQ Bot credentials saved")
|
||||
|
||||
print()
|
||||
print_info("🔒 Security: Restrict who can DM your bot")
|
||||
print_info(" Use QQ user OpenIDs (found in event payloads)")
|
||||
print()
|
||||
allowed_users = prompt("Allowed user OpenIDs (comma-separated, leave empty for open access)")
|
||||
if allowed_users:
|
||||
save_env_value("QQ_ALLOWED_USERS", allowed_users.replace(" ", ""))
|
||||
print_success("QQ Bot allowlist configured")
|
||||
else:
|
||||
print_info("⚠️ No allowlist set — anyone can DM the bot!")
|
||||
|
||||
print()
|
||||
print_info("📬 Home Channel: OpenID for cron job delivery and notifications.")
|
||||
home_channel = prompt("Home channel OpenID (leave empty to set later)")
|
||||
if home_channel:
|
||||
save_env_value("QQ_HOME_CHANNEL", home_channel)
|
||||
|
||||
print()
|
||||
print_success("QQ Bot configured!")
|
||||
|
||||
|
||||
def _setup_bluebubbles():
|
||||
@@ -2074,9 +2119,12 @@ def _setup_bluebubbles():
|
||||
|
||||
|
||||
def _setup_qqbot():
|
||||
"""Configure QQ Bot (Official API v2) via gateway setup."""
|
||||
from hermes_cli.gateway import _setup_qqbot as _gateway_setup_qqbot
|
||||
_gateway_setup_qqbot()
|
||||
"""Configure QQ Bot (Official API v2) via standard platform setup."""
|
||||
from hermes_cli.gateway import _PLATFORMS
|
||||
qq_platform = next((p for p in _PLATFORMS if p["key"] == "qqbot"), None)
|
||||
if qq_platform:
|
||||
from hermes_cli.gateway import _setup_standard_platform
|
||||
_setup_standard_platform(qq_platform)
|
||||
|
||||
|
||||
def _setup_webhooks():
|
||||
@@ -2216,9 +2264,7 @@ def setup_gateway(config: dict):
|
||||
missing_home.append("Slack")
|
||||
if get_env_value("BLUEBUBBLES_SERVER_URL") and not get_env_value("BLUEBUBBLES_HOME_CHANNEL"):
|
||||
missing_home.append("BlueBubbles")
|
||||
if get_env_value("QQ_APP_ID") and not (
|
||||
get_env_value("QQBOT_HOME_CHANNEL") or get_env_value("QQ_HOME_CHANNEL")
|
||||
):
|
||||
if get_env_value("QQ_APP_ID") and not get_env_value("QQ_HOME_CHANNEL"):
|
||||
missing_home.append("QQBot")
|
||||
|
||||
if missing_home:
|
||||
@@ -2243,10 +2289,8 @@ def setup_gateway(config: dict):
|
||||
_is_service_running,
|
||||
supports_systemd_services,
|
||||
has_conflicting_systemd_units,
|
||||
has_legacy_hermes_units,
|
||||
install_linux_gateway_from_setup,
|
||||
print_systemd_scope_conflict_warning,
|
||||
print_legacy_unit_warning,
|
||||
systemd_start,
|
||||
systemd_restart,
|
||||
launchd_install,
|
||||
@@ -2264,10 +2308,6 @@ def setup_gateway(config: dict):
|
||||
print_systemd_scope_conflict_warning()
|
||||
print()
|
||||
|
||||
if supports_systemd and has_legacy_hermes_units():
|
||||
print_legacy_unit_warning()
|
||||
print()
|
||||
|
||||
if service_running:
|
||||
if prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
||||
try:
|
||||
|
||||
@@ -515,90 +515,6 @@ def do_inspect(identifier: str, console: Optional[Console] = None) -> None:
|
||||
c.print()
|
||||
|
||||
|
||||
def browse_skills(page: int = 1, page_size: int = 20, source: str = "all") -> dict:
|
||||
"""Paginated hub browse for programmatic callers (e.g. TUI gateway).
|
||||
|
||||
Returns ``{"items": [...], "page": int, "total_pages": int, "total": int}``.
|
||||
"""
|
||||
from tools.skills_hub import GitHubAuth, create_source_router
|
||||
|
||||
page_size = max(1, min(page_size, 100))
|
||||
_TRUST_RANK = {"builtin": 3, "trusted": 2, "community": 1}
|
||||
_PER_SOURCE_LIMIT = {"official": 100, "skills-sh": 100, "well-known": 25, "github": 100, "clawhub": 50,
|
||||
"claude-marketplace": 50, "lobehub": 50}
|
||||
auth = GitHubAuth()
|
||||
sources = create_source_router(auth)
|
||||
all_results: list = []
|
||||
for src in sources:
|
||||
sid = src.source_id()
|
||||
if source != "all" and sid != source and sid != "official":
|
||||
continue
|
||||
try:
|
||||
limit = _PER_SOURCE_LIMIT.get(sid, 50)
|
||||
all_results.extend(src.search("", limit=limit))
|
||||
except Exception:
|
||||
continue
|
||||
if not all_results:
|
||||
return {"items": [], "page": 1, "total_pages": 1, "total": 0}
|
||||
seen: dict = {}
|
||||
for r in all_results:
|
||||
rank = _TRUST_RANK.get(r.trust_level, 0)
|
||||
if r.name not in seen or rank > _TRUST_RANK.get(seen[r.name].trust_level, 0):
|
||||
seen[r.name] = r
|
||||
deduped = list(seen.values())
|
||||
deduped.sort(key=lambda r: (-_TRUST_RANK.get(r.trust_level, 0), r.source != "official", r.name.lower()))
|
||||
total = len(deduped)
|
||||
total_pages = max(1, (total + page_size - 1) // page_size)
|
||||
page = max(1, min(page, total_pages))
|
||||
start = (page - 1) * page_size
|
||||
page_items = deduped[start : min(start + page_size, total)]
|
||||
return {
|
||||
"items": [{"name": r.name, "description": r.description, "source": r.source,
|
||||
"trust": r.trust_level} for r in page_items],
|
||||
"page": page,
|
||||
"total_pages": total_pages,
|
||||
"total": total,
|
||||
}
|
||||
|
||||
|
||||
def inspect_skill(identifier: str) -> Optional[dict]:
|
||||
"""Skill metadata (+ SKILL.md preview) for programmatic callers."""
|
||||
from tools.skills_hub import GitHubAuth, create_source_router
|
||||
|
||||
class _Q:
|
||||
def print(self, *a, **k):
|
||||
pass
|
||||
|
||||
c = _Q()
|
||||
auth = GitHubAuth()
|
||||
sources = create_source_router(auth)
|
||||
ident = identifier
|
||||
if "/" not in ident:
|
||||
ident = _resolve_short_name(ident, sources, c)
|
||||
if not ident:
|
||||
return None
|
||||
meta, bundle, _ = _resolve_source_meta_and_bundle(ident, sources)
|
||||
if not meta:
|
||||
return None
|
||||
out: dict = {
|
||||
"name": meta.name,
|
||||
"description": meta.description,
|
||||
"source": meta.source,
|
||||
"identifier": meta.identifier,
|
||||
"tags": list(meta.tags) if meta.tags else [],
|
||||
}
|
||||
if bundle and "SKILL.md" in bundle.files:
|
||||
content = bundle.files["SKILL.md"]
|
||||
if isinstance(content, bytes):
|
||||
content = content.decode("utf-8", errors="replace")
|
||||
lines = content.split("\n")
|
||||
preview = "\n".join(lines[:50])
|
||||
if len(lines) > 50:
|
||||
preview += f"\n\n... ({len(lines) - 50} more lines)"
|
||||
out["skill_md_preview"] = preview
|
||||
return out
|
||||
|
||||
|
||||
def do_list(source_filter: str = "all", console: Optional[Console] = None) -> None:
|
||||
"""List installed skills, distinguishing hub, builtin, and local skills."""
|
||||
from tools.skills_hub import HubLockFile, ensure_hub_dirs
|
||||
|
||||
@@ -23,7 +23,7 @@ All fields are optional. Missing values inherit from the ``default`` skin.
|
||||
banner_dim: "#B8860B" # Dim/muted text (separators, labels)
|
||||
banner_text: "#FFF8DC" # Body text (tool names, skill names)
|
||||
ui_accent: "#FFBF00" # General UI accent
|
||||
ui_label: "#DAA520" # UI labels (warm gold; teal clashed w/ default banner gold)
|
||||
ui_label: "#4dd0e1" # UI labels
|
||||
ui_ok: "#4caf50" # Success indicators
|
||||
ui_error: "#ef5350" # Error indicators
|
||||
ui_warn: "#ffa726" # Warning indicators
|
||||
@@ -163,7 +163,7 @@ _BUILTIN_SKINS: Dict[str, Dict[str, Any]] = {
|
||||
"banner_dim": "#B8860B",
|
||||
"banner_text": "#FFF8DC",
|
||||
"ui_accent": "#FFBF00",
|
||||
"ui_label": "#DAA520",
|
||||
"ui_label": "#4dd0e1",
|
||||
"ui_ok": "#4caf50",
|
||||
"ui_error": "#ef5350",
|
||||
"ui_warn": "#ffa726",
|
||||
|
||||
+64
-30
@@ -317,7 +317,7 @@ def show_status(args):
|
||||
"WeCom Callback": ("WECOM_CALLBACK_CORP_ID", None),
|
||||
"Weixin": ("WEIXIN_ACCOUNT_ID", "WEIXIN_HOME_CHANNEL"),
|
||||
"BlueBubbles": ("BLUEBUBBLES_SERVER_URL", "BLUEBUBBLES_HOME_CHANNEL"),
|
||||
"QQBot": ("QQ_APP_ID", "QQBOT_HOME_CHANNEL"),
|
||||
"QQBot": ("QQ_APP_ID", "QQ_HOME_CHANNEL"),
|
||||
}
|
||||
|
||||
for name, (token_var, home_var) in platforms.items():
|
||||
@@ -327,9 +327,6 @@ def show_status(args):
|
||||
home_channel = ""
|
||||
if home_var:
|
||||
home_channel = os.getenv(home_var, "")
|
||||
# Back-compat: QQBot home channel was renamed from QQ_HOME_CHANNEL to QQBOT_HOME_CHANNEL
|
||||
if not home_channel and home_var == "QQBOT_HOME_CHANNEL":
|
||||
home_channel = os.getenv("QQ_HOME_CHANNEL", "")
|
||||
|
||||
status = "configured" if has_token else "not configured"
|
||||
if home_channel:
|
||||
@@ -342,36 +339,73 @@ def show_status(args):
|
||||
# =========================================================================
|
||||
print()
|
||||
print(color("◆ Gateway Service", Colors.CYAN, Colors.BOLD))
|
||||
|
||||
try:
|
||||
from hermes_cli.gateway import get_gateway_runtime_snapshot, _format_gateway_pids
|
||||
|
||||
snapshot = get_gateway_runtime_snapshot()
|
||||
is_running = snapshot.running
|
||||
|
||||
if _is_termux():
|
||||
try:
|
||||
from hermes_cli.gateway import find_gateway_pids
|
||||
gateway_pids = find_gateway_pids()
|
||||
except Exception:
|
||||
gateway_pids = []
|
||||
is_running = bool(gateway_pids)
|
||||
print(f" Status: {check_mark(is_running)} {'running' if is_running else 'stopped'}")
|
||||
print(f" Manager: {snapshot.manager}")
|
||||
if snapshot.gateway_pids:
|
||||
print(f" PID(s): {_format_gateway_pids(snapshot.gateway_pids)}")
|
||||
if snapshot.has_process_service_mismatch:
|
||||
print(" Service: installed but not managing the current running gateway")
|
||||
elif _is_termux() and not snapshot.gateway_pids:
|
||||
print(" Manager: Termux / manual process")
|
||||
if gateway_pids:
|
||||
rendered = ", ".join(str(pid) for pid in gateway_pids[:3])
|
||||
if len(gateway_pids) > 3:
|
||||
rendered += ", ..."
|
||||
print(f" PID(s): {rendered}")
|
||||
else:
|
||||
print(" Start with: hermes gateway")
|
||||
print(" Note: Android may stop background jobs when Termux is suspended")
|
||||
elif snapshot.service_installed and not snapshot.service_running:
|
||||
print(" Service: installed but stopped")
|
||||
except Exception:
|
||||
if _is_termux():
|
||||
print(f" Status: {color('unknown', Colors.DIM)}")
|
||||
print(" Manager: Termux / manual process")
|
||||
elif sys.platform.startswith('linux'):
|
||||
print(f" Status: {color('unknown', Colors.DIM)}")
|
||||
print(" Manager: systemd/manual")
|
||||
elif sys.platform == 'darwin':
|
||||
print(f" Status: {color('unknown', Colors.DIM)}")
|
||||
print(" Manager: launchd")
|
||||
|
||||
elif sys.platform.startswith('linux'):
|
||||
from hermes_constants import is_container
|
||||
if is_container():
|
||||
# Docker/Podman: no systemd — check for running gateway processes
|
||||
try:
|
||||
from hermes_cli.gateway import find_gateway_pids
|
||||
gateway_pids = find_gateway_pids()
|
||||
is_active = len(gateway_pids) > 0
|
||||
except Exception:
|
||||
is_active = False
|
||||
print(f" Status: {check_mark(is_active)} {'running' if is_active else 'stopped'}")
|
||||
print(" Manager: docker (foreground)")
|
||||
else:
|
||||
print(f" Status: {color('N/A', Colors.DIM)}")
|
||||
print(" Manager: (not supported on this platform)")
|
||||
try:
|
||||
from hermes_cli.gateway import get_service_name
|
||||
_gw_svc = get_service_name()
|
||||
except Exception:
|
||||
_gw_svc = "hermes-gateway"
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["systemctl", "--user", "is-active", _gw_svc],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5
|
||||
)
|
||||
is_active = result.stdout.strip() == "active"
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||
is_active = False
|
||||
print(f" Status: {check_mark(is_active)} {'running' if is_active else 'stopped'}")
|
||||
print(" Manager: systemd (user)")
|
||||
|
||||
elif sys.platform == 'darwin':
|
||||
from hermes_cli.gateway import get_launchd_label
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["launchctl", "list", get_launchd_label()],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5
|
||||
)
|
||||
is_loaded = result.returncode == 0
|
||||
except subprocess.TimeoutExpired:
|
||||
is_loaded = False
|
||||
print(f" Status: {check_mark(is_loaded)} {'loaded' if is_loaded else 'not loaded'}")
|
||||
print(" Manager: launchd")
|
||||
else:
|
||||
print(f" Status: {color('N/A', Colors.DIM)}")
|
||||
print(" Manager: (not supported on this platform)")
|
||||
|
||||
# =========================================================================
|
||||
# Cron Jobs
|
||||
|
||||
@@ -512,7 +512,7 @@ def _get_platform_tools(
|
||||
"""Resolve which individual toolset names are enabled for a platform."""
|
||||
from toolsets import resolve_toolset
|
||||
|
||||
platform_toolsets = config.get("platform_toolsets") or {}
|
||||
platform_toolsets = config.get("platform_toolsets", {})
|
||||
toolset_names = platform_toolsets.get(platform)
|
||||
|
||||
if toolset_names is None or not isinstance(toolset_names, list):
|
||||
|
||||
+53
-208
@@ -118,164 +118,57 @@ def remove_wrapper_script():
|
||||
|
||||
|
||||
def uninstall_gateway_service():
|
||||
"""Stop and uninstall the gateway service (systemd, launchd) and kill any
|
||||
standalone gateway processes.
|
||||
|
||||
Delegates to the gateway module which handles:
|
||||
- Linux: user + system systemd services (with proper DBUS env setup)
|
||||
- macOS: launchd plists
|
||||
- All platforms: standalone ``hermes gateway run`` processes
|
||||
- Termux/Android: skips systemd (no systemd on Android), still kills standalone processes
|
||||
"""
|
||||
"""Stop and uninstall the gateway service if running."""
|
||||
import platform
|
||||
stopped_something = False
|
||||
|
||||
# 1. Kill any standalone gateway processes (all platforms, including Termux)
|
||||
try:
|
||||
from hermes_cli.gateway import kill_gateway_processes, find_gateway_pids
|
||||
pids = find_gateway_pids()
|
||||
if pids:
|
||||
killed = kill_gateway_processes()
|
||||
if killed:
|
||||
log_success(f"Killed {killed} running gateway process(es)")
|
||||
stopped_something = True
|
||||
except Exception as e:
|
||||
log_warn(f"Could not check for gateway processes: {e}")
|
||||
|
||||
system = platform.system()
|
||||
|
||||
# Termux/Android has no systemd and no launchd — nothing left to do.
|
||||
prefix = os.getenv("PREFIX", "")
|
||||
is_termux = bool(os.getenv("TERMUX_VERSION") or "com.termux/files/usr" in prefix)
|
||||
if is_termux:
|
||||
return stopped_something
|
||||
|
||||
# 2. Linux: uninstall systemd services (both user and system scopes)
|
||||
if system == "Linux":
|
||||
try:
|
||||
from hermes_cli.gateway import (
|
||||
get_systemd_unit_path,
|
||||
get_service_name,
|
||||
_systemctl_cmd,
|
||||
)
|
||||
svc_name = get_service_name()
|
||||
|
||||
for is_system in (False, True):
|
||||
unit_path = get_systemd_unit_path(system=is_system)
|
||||
if not unit_path.exists():
|
||||
continue
|
||||
|
||||
scope = "system" if is_system else "user"
|
||||
try:
|
||||
if is_system and os.geteuid() != 0:
|
||||
log_warn(f"System gateway service exists at {unit_path} "
|
||||
f"but needs sudo to remove")
|
||||
continue
|
||||
|
||||
cmd = _systemctl_cmd(is_system)
|
||||
subprocess.run(cmd + ["stop", svc_name],
|
||||
capture_output=True, check=False)
|
||||
subprocess.run(cmd + ["disable", svc_name],
|
||||
capture_output=True, check=False)
|
||||
unit_path.unlink()
|
||||
subprocess.run(cmd + ["daemon-reload"],
|
||||
capture_output=True, check=False)
|
||||
log_success(f"Removed {scope} gateway service ({unit_path})")
|
||||
stopped_something = True
|
||||
except Exception as e:
|
||||
log_warn(f"Could not remove {scope} gateway service: {e}")
|
||||
except Exception as e:
|
||||
log_warn(f"Could not check systemd gateway services: {e}")
|
||||
|
||||
# 3. macOS: uninstall launchd plist
|
||||
elif system == "Darwin":
|
||||
try:
|
||||
from hermes_cli.gateway import get_launchd_plist_path
|
||||
plist_path = get_launchd_plist_path()
|
||||
if plist_path.exists():
|
||||
subprocess.run(["launchctl", "unload", str(plist_path)],
|
||||
capture_output=True, check=False)
|
||||
plist_path.unlink()
|
||||
log_success(f"Removed macOS gateway service ({plist_path})")
|
||||
stopped_something = True
|
||||
except Exception as e:
|
||||
log_warn(f"Could not remove launchd gateway service: {e}")
|
||||
|
||||
return stopped_something
|
||||
|
||||
|
||||
def _is_default_hermes_home(hermes_home: Path) -> bool:
|
||||
"""Return True when ``hermes_home`` points at the default (non-profile) root."""
|
||||
try:
|
||||
from hermes_constants import get_default_hermes_root
|
||||
return hermes_home.resolve() == get_default_hermes_root().resolve()
|
||||
except Exception:
|
||||
|
||||
if platform.system() != "Linux":
|
||||
return False
|
||||
|
||||
|
||||
def _discover_named_profiles():
|
||||
"""Return a list of ``ProfileInfo`` for every non-default profile, or ``[]``
|
||||
if profile support is unavailable or nothing is installed beyond the
|
||||
default root."""
|
||||
prefix = os.getenv("PREFIX", "")
|
||||
if os.getenv("TERMUX_VERSION") or "com.termux/files/usr" in prefix:
|
||||
return False
|
||||
|
||||
try:
|
||||
from hermes_cli.profiles import list_profiles
|
||||
from hermes_cli.gateway import get_service_name
|
||||
svc_name = get_service_name()
|
||||
except Exception:
|
||||
return []
|
||||
svc_name = "hermes-gateway"
|
||||
|
||||
service_file = Path.home() / ".config" / "systemd" / "user" / f"{svc_name}.service"
|
||||
|
||||
if not service_file.exists():
|
||||
return False
|
||||
|
||||
try:
|
||||
return [p for p in list_profiles() if not getattr(p, "is_default", False)]
|
||||
# Stop the service
|
||||
subprocess.run(
|
||||
["systemctl", "--user", "stop", svc_name],
|
||||
capture_output=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
# Disable the service
|
||||
subprocess.run(
|
||||
["systemctl", "--user", "disable", svc_name],
|
||||
capture_output=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
# Remove service file
|
||||
service_file.unlink()
|
||||
|
||||
# Reload systemd
|
||||
subprocess.run(
|
||||
["systemctl", "--user", "daemon-reload"],
|
||||
capture_output=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
log_warn(f"Could not enumerate profiles: {e}")
|
||||
return []
|
||||
|
||||
|
||||
def _uninstall_profile(profile) -> None:
|
||||
"""Fully uninstall a single named profile: stop its gateway service,
|
||||
remove its alias wrapper, and wipe its HERMES_HOME directory.
|
||||
|
||||
We shell out to ``hermes -p <name> gateway stop|uninstall`` because
|
||||
service names, unit paths, and plist paths are all derived from the
|
||||
current HERMES_HOME and can't be easily switched in-process.
|
||||
"""
|
||||
import sys as _sys
|
||||
name = profile.name
|
||||
profile_home = profile.path
|
||||
|
||||
log_info(f"Uninstalling profile '{name}'...")
|
||||
|
||||
# 1. Stop and remove this profile's gateway service.
|
||||
# Use `python -m hermes_cli.main` so we don't depend on a `hermes`
|
||||
# wrapper that may be half-removed mid-uninstall.
|
||||
hermes_invocation = [_sys.executable, "-m", "hermes_cli.main", "--profile", name]
|
||||
for subcmd in ("stop", "uninstall"):
|
||||
try:
|
||||
subprocess.run(
|
||||
hermes_invocation + ["gateway", subcmd],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60,
|
||||
check=False,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
log_warn(f" Gateway {subcmd} timed out for '{name}'")
|
||||
except Exception as e:
|
||||
log_warn(f" Could not run gateway {subcmd} for '{name}': {e}")
|
||||
|
||||
# 2. Remove the wrapper alias script at ~/.local/bin/<name> (if any).
|
||||
alias_path = getattr(profile, "alias_path", None)
|
||||
if alias_path and alias_path.exists():
|
||||
try:
|
||||
alias_path.unlink()
|
||||
log_success(f" Removed alias {alias_path}")
|
||||
except Exception as e:
|
||||
log_warn(f" Could not remove alias {alias_path}: {e}")
|
||||
|
||||
# 3. Wipe the profile's HERMES_HOME directory.
|
||||
try:
|
||||
if profile_home.exists():
|
||||
shutil.rmtree(profile_home)
|
||||
log_success(f" Removed {profile_home}")
|
||||
except Exception as e:
|
||||
log_warn(f" Could not remove {profile_home}: {e}")
|
||||
log_warn(f"Could not fully remove gateway service: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def run_uninstall(args):
|
||||
@@ -288,13 +181,7 @@ def run_uninstall(args):
|
||||
"""
|
||||
project_root = get_project_root()
|
||||
hermes_home = get_hermes_home()
|
||||
|
||||
# Detect named profiles when uninstalling from the default root —
|
||||
# offer to clean them up too instead of leaving zombie HERMES_HOMEs
|
||||
# and systemd units behind.
|
||||
is_default_profile = _is_default_hermes_home(hermes_home)
|
||||
named_profiles = _discover_named_profiles() if is_default_profile else []
|
||||
|
||||
|
||||
print()
|
||||
print(color("┌─────────────────────────────────────────────────────────┐", Colors.MAGENTA, Colors.BOLD))
|
||||
print(color("│ ⚕ Hermes Agent Uninstaller │", Colors.MAGENTA, Colors.BOLD))
|
||||
@@ -308,13 +195,6 @@ def run_uninstall(args):
|
||||
print(f" Secrets: {hermes_home / '.env'}")
|
||||
print(f" Data: {hermes_home / 'cron/'}, {hermes_home / 'sessions/'}, {hermes_home / 'logs/'}")
|
||||
print()
|
||||
|
||||
if named_profiles:
|
||||
print(color("Other profiles detected:", Colors.CYAN, Colors.BOLD))
|
||||
for p in named_profiles:
|
||||
running = " (gateway running)" if getattr(p, "gateway_running", False) else ""
|
||||
print(f" • {p.name}{running}: {p.path}")
|
||||
print()
|
||||
|
||||
# Ask for confirmation
|
||||
print(color("Uninstall Options:", Colors.YELLOW, Colors.BOLD))
|
||||
@@ -341,40 +221,12 @@ def run_uninstall(args):
|
||||
return
|
||||
|
||||
full_uninstall = (choice == "2")
|
||||
|
||||
# When doing a full uninstall from the default profile, also offer to
|
||||
# remove any named profiles — stopping their gateway services, unlinking
|
||||
# their alias wrappers, and wiping their HERMES_HOME dirs. Otherwise
|
||||
# those leave zombie services and data behind.
|
||||
remove_profiles = False
|
||||
if full_uninstall and named_profiles:
|
||||
print()
|
||||
print(color("Other profiles will NOT be removed by default.", Colors.YELLOW))
|
||||
print(f"Found {len(named_profiles)} named profile(s): " +
|
||||
", ".join(p.name for p in named_profiles))
|
||||
print()
|
||||
try:
|
||||
resp = input(color(
|
||||
f"Also stop and remove these {len(named_profiles)} profile(s)? [y/N]: ",
|
||||
Colors.BOLD
|
||||
)).strip().lower()
|
||||
except (KeyboardInterrupt, EOFError):
|
||||
print()
|
||||
print("Cancelled.")
|
||||
return
|
||||
remove_profiles = resp in ("y", "yes")
|
||||
|
||||
|
||||
# Final confirmation
|
||||
print()
|
||||
if full_uninstall:
|
||||
print(color("⚠️ WARNING: This will permanently delete ALL Hermes data!", Colors.RED, Colors.BOLD))
|
||||
print(color(" Including: configs, API keys, sessions, scheduled jobs, logs", Colors.RED))
|
||||
if remove_profiles:
|
||||
print(color(
|
||||
f" Plus {len(named_profiles)} profile(s): " +
|
||||
", ".join(p.name for p in named_profiles),
|
||||
Colors.RED
|
||||
))
|
||||
else:
|
||||
print("This will remove the Hermes code but keep your configuration and data.")
|
||||
|
||||
@@ -395,10 +247,12 @@ def run_uninstall(args):
|
||||
print(color("Uninstalling...", Colors.CYAN, Colors.BOLD))
|
||||
print()
|
||||
|
||||
# 1. Stop and uninstall gateway service + kill standalone processes
|
||||
log_info("Checking for running gateway...")
|
||||
if not uninstall_gateway_service():
|
||||
log_info("No gateway service or processes found")
|
||||
# 1. Stop and uninstall gateway service
|
||||
log_info("Checking for gateway service...")
|
||||
if uninstall_gateway_service():
|
||||
log_success("Gateway service stopped and removed")
|
||||
else:
|
||||
log_info("No gateway service found")
|
||||
|
||||
# 2. Remove PATH entries from shell configs
|
||||
log_info("Removing PATH entries from shell configs...")
|
||||
@@ -437,17 +291,8 @@ def run_uninstall(args):
|
||||
log_warn(f"Could not fully remove {project_root}: {e}")
|
||||
log_info("You may need to manually remove it")
|
||||
|
||||
# 5. Optionally remove ~/.hermes/ data directory (and named profiles)
|
||||
# 5. Optionally remove ~/.hermes/ data directory
|
||||
if full_uninstall:
|
||||
# 5a. Stop and remove each named profile's gateway service and
|
||||
# alias wrapper. The profile HERMES_HOME dirs live under
|
||||
# ``<default>/profiles/<name>/`` and will be swept away by the
|
||||
# rmtree below, but services + alias scripts live OUTSIDE the
|
||||
# default root and have to be cleaned up explicitly.
|
||||
if remove_profiles and named_profiles:
|
||||
for prof in named_profiles:
|
||||
_uninstall_profile(prof)
|
||||
|
||||
log_info("Removing configuration and data...")
|
||||
try:
|
||||
if hermes_home.exists():
|
||||
|
||||
@@ -56,10 +56,10 @@ try:
|
||||
except ImportError:
|
||||
raise SystemExit(
|
||||
"Web UI requires fastapi and uvicorn.\n"
|
||||
f"Install with: {sys.executable} -m pip install 'fastapi' 'uvicorn[standard]'"
|
||||
"Run 'hermes web' to auto-install, or: pip install hermes-agent[web]"
|
||||
)
|
||||
|
||||
WEB_DIST = Path(os.environ["HERMES_WEB_DIST"]) if "HERMES_WEB_DIST" in os.environ else Path(__file__).parent / "web_dist"
|
||||
WEB_DIST = Path(__file__).parent / "web_dist"
|
||||
_log = logging.getLogger(__name__)
|
||||
|
||||
app = FastAPI(title="Hermes Agent", version=__version__)
|
||||
@@ -1444,8 +1444,38 @@ def _nous_poller(session_id: str) -> None:
|
||||
auth_state, min_key_ttl_seconds=300, timeout_seconds=15.0,
|
||||
force_refresh=False, force_mint=True,
|
||||
)
|
||||
from hermes_cli.auth import persist_nous_credentials
|
||||
persist_nous_credentials(full_state)
|
||||
# Save into credential pool same as auth_commands.py does
|
||||
from agent.credential_pool import (
|
||||
PooledCredential,
|
||||
load_pool,
|
||||
AUTH_TYPE_OAUTH,
|
||||
SOURCE_MANUAL,
|
||||
)
|
||||
pool = load_pool("nous")
|
||||
entry = PooledCredential.from_dict("nous", {
|
||||
**full_state,
|
||||
"label": "dashboard device_code",
|
||||
"auth_type": AUTH_TYPE_OAUTH,
|
||||
"source": f"{SOURCE_MANUAL}:dashboard_device_code",
|
||||
"base_url": full_state.get("inference_base_url"),
|
||||
})
|
||||
pool.add_entry(entry)
|
||||
# Also persist to auth store so get_nous_auth_status() sees it
|
||||
# (matches what _login_nous in auth.py does for the CLI flow).
|
||||
try:
|
||||
from hermes_cli.auth import (
|
||||
_load_auth_store, _save_provider_state, _save_auth_store,
|
||||
_auth_store_lock,
|
||||
)
|
||||
with _auth_store_lock():
|
||||
auth_store = _load_auth_store()
|
||||
_save_provider_state(auth_store, "nous", full_state)
|
||||
_save_auth_store(auth_store)
|
||||
except Exception as store_exc:
|
||||
_log.warning(
|
||||
"oauth/device: credential pool saved but auth store write failed "
|
||||
"(session=%s): %s", session_id, store_exc,
|
||||
)
|
||||
with _oauth_sessions_lock:
|
||||
sess["status"] = "approved"
|
||||
_log.info("oauth/device: nous login completed (session=%s)", session_id)
|
||||
|
||||
+1
-2
@@ -14,8 +14,7 @@ def get_hermes_home() -> Path:
|
||||
Reads HERMES_HOME env var, falls back to ~/.hermes.
|
||||
This is the single source of truth — all other copies should import this.
|
||||
"""
|
||||
val = os.environ.get("HERMES_HOME", "").strip()
|
||||
return Path(val) if val else Path.home() / ".hermes"
|
||||
return Path(os.getenv("HERMES_HOME", Path.home() / ".hermes"))
|
||||
|
||||
|
||||
def get_default_hermes_root() -> Path:
|
||||
|
||||
+2
-57
@@ -987,22 +987,6 @@ class SessionDB:
|
||||
|
||||
return sanitized.strip()
|
||||
|
||||
|
||||
@staticmethod
|
||||
def _contains_cjk(text: str) -> bool:
|
||||
"""Check if text contains CJK (Chinese, Japanese, Korean) characters."""
|
||||
for ch in text:
|
||||
cp = ord(ch)
|
||||
if (0x4E00 <= cp <= 0x9FFF or # CJK Unified Ideographs
|
||||
0x3400 <= cp <= 0x4DBF or # CJK Extension A
|
||||
0x20000 <= cp <= 0x2A6DF or # CJK Extension B
|
||||
0x3000 <= cp <= 0x303F or # CJK Symbols
|
||||
0x3040 <= cp <= 0x309F or # Hiragana
|
||||
0x30A0 <= cp <= 0x30FF or # Katakana
|
||||
0xAC00 <= cp <= 0xD7AF): # Hangul Syllables
|
||||
return True
|
||||
return False
|
||||
|
||||
def search_messages(
|
||||
self,
|
||||
query: str,
|
||||
@@ -1078,47 +1062,8 @@ class SessionDB:
|
||||
cursor = self._conn.execute(sql, params)
|
||||
except sqlite3.OperationalError:
|
||||
# FTS5 query syntax error despite sanitization — return empty
|
||||
# unless query contains CJK (fall back to LIKE below)
|
||||
if not self._contains_cjk(query):
|
||||
return []
|
||||
matches = []
|
||||
else:
|
||||
matches = [dict(row) for row in cursor.fetchall()]
|
||||
|
||||
# LIKE fallback for CJK queries: FTS5 default tokenizer splits CJK
|
||||
# characters individually, causing multi-character queries to fail.
|
||||
if not matches and self._contains_cjk(query):
|
||||
raw_query = query.strip('"').strip()
|
||||
like_where = ["m.content LIKE ?"]
|
||||
like_params: list = [f"%{raw_query}%"]
|
||||
if source_filter is not None:
|
||||
like_where.append(f"s.source IN ({','.join('?' for _ in source_filter)})")
|
||||
like_params.extend(source_filter)
|
||||
if exclude_sources is not None:
|
||||
like_where.append(f"s.source NOT IN ({','.join('?' for _ in exclude_sources)})")
|
||||
like_params.extend(exclude_sources)
|
||||
if role_filter:
|
||||
like_where.append(f"m.role IN ({','.join('?' for _ in role_filter)})")
|
||||
like_params.extend(role_filter)
|
||||
like_sql = f"""
|
||||
SELECT m.id, m.session_id, m.role,
|
||||
substr(m.content,
|
||||
max(1, instr(m.content, ?) - 40),
|
||||
120) AS snippet,
|
||||
m.content, m.timestamp, m.tool_name,
|
||||
s.source, s.model, s.started_at AS session_started
|
||||
FROM messages m
|
||||
JOIN sessions s ON s.id = m.session_id
|
||||
WHERE {' AND '.join(like_where)}
|
||||
ORDER BY m.timestamp DESC
|
||||
LIMIT ? OFFSET ?
|
||||
"""
|
||||
like_params.extend([limit, offset])
|
||||
# instr() parameter goes first in the bound list
|
||||
like_params = [raw_query] + like_params
|
||||
with self._lock:
|
||||
like_cursor = self._conn.execute(like_sql, like_params)
|
||||
matches = [dict(row) for row in like_cursor.fetchall()]
|
||||
return []
|
||||
matches = [dict(row) for row in cursor.fetchall()]
|
||||
|
||||
# Add surrounding context (1 message before + after each match).
|
||||
# Done outside the lock so we don't hold it across N sequential queries.
|
||||
|
||||
+2
-2
@@ -433,7 +433,7 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
||||
if not _MCP_SERVER_AVAILABLE:
|
||||
raise ImportError(
|
||||
"MCP server requires the 'mcp' package. "
|
||||
f"Install with: {sys.executable} -m pip install 'mcp'"
|
||||
"Install with: pip install 'hermes-agent[mcp]'"
|
||||
)
|
||||
|
||||
mcp = FastMCP(
|
||||
@@ -838,7 +838,7 @@ def run_mcp_server(verbose: bool = False) -> None:
|
||||
if not _MCP_SERVER_AVAILABLE:
|
||||
print(
|
||||
"Error: MCP server requires the 'mcp' package.\n"
|
||||
f"Install with: {sys.executable} -m pip install 'mcp'",
|
||||
"Install with: pip install 'hermes-agent[mcp]'",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
+6
-20
@@ -43,15 +43,6 @@ from dotenv import load_dotenv
|
||||
load_dotenv()
|
||||
|
||||
|
||||
def _effective_temperature_for_model(model: str) -> Optional[float]:
|
||||
"""Return a fixed temperature for models with strict sampling contracts."""
|
||||
try:
|
||||
from agent.auxiliary_client import _fixed_temperature_for_model
|
||||
except Exception:
|
||||
return None
|
||||
return _fixed_temperature_for_model(model)
|
||||
|
||||
|
||||
|
||||
|
||||
# ============================================================================
|
||||
@@ -451,17 +442,12 @@ Complete the user's task step by step."""
|
||||
|
||||
# Make API call
|
||||
try:
|
||||
api_kwargs = {
|
||||
"model": self.model,
|
||||
"messages": api_messages,
|
||||
"tools": self.tools,
|
||||
"timeout": 300.0,
|
||||
}
|
||||
fixed_temperature = _effective_temperature_for_model(self.model)
|
||||
if fixed_temperature is not None:
|
||||
api_kwargs["temperature"] = fixed_temperature
|
||||
|
||||
response = self.client.chat.completions.create(**api_kwargs)
|
||||
response = self.client.chat.completions.create(
|
||||
model=self.model,
|
||||
messages=api_messages,
|
||||
tools=self.tools,
|
||||
timeout=300.0
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"API call failed: {e}")
|
||||
break
|
||||
|
||||
+2
-2
@@ -274,9 +274,9 @@ def get_tool_definitions(
|
||||
# execute_code" even when the API key isn't configured or the toolset is
|
||||
# disabled (#560-discord).
|
||||
if "execute_code" in available_tool_names:
|
||||
from tools.code_execution_tool import SANDBOX_ALLOWED_TOOLS, build_execute_code_schema, _get_execution_mode
|
||||
from tools.code_execution_tool import SANDBOX_ALLOWED_TOOLS, build_execute_code_schema
|
||||
sandbox_enabled = SANDBOX_ALLOWED_TOOLS & available_tool_names
|
||||
dynamic_schema = build_execute_code_schema(sandbox_enabled, mode=_get_execution_mode())
|
||||
dynamic_schema = build_execute_code_schema(sandbox_enabled)
|
||||
for i, td in enumerate(filtered_tools):
|
||||
if td.get("function", {}).get("name") == "execute_code":
|
||||
filtered_tools[i] = {"type": "function", "function": dynamic_schema}
|
||||
|
||||
+1
-69
@@ -37,30 +37,7 @@ json.dump(sorted(leaf_paths(DEFAULT_CONFIG)), sys.stdout, indent=2)
|
||||
in {
|
||||
packages.configKeys = configKeys;
|
||||
|
||||
checks = {
|
||||
# Cross-platform evaluation — catches "not supported for interpreter"
|
||||
# errors (e.g. sphinx dropping python311) without needing a darwin builder.
|
||||
# Evaluation is pure and instant; it doesn't build anything.
|
||||
cross-eval = let
|
||||
targetSystems = builtins.filter
|
||||
(s: inputs.self.packages ? ${s})
|
||||
[ "x86_64-linux" "aarch64-linux" "aarch64-darwin" "x86_64-darwin" ];
|
||||
tryEvalPkg = sys:
|
||||
let pkg = inputs.self.packages.${sys}.default;
|
||||
in builtins.tryEval (builtins.seq pkg.drvPath true);
|
||||
results = map (sys: { inherit sys; result = tryEvalPkg sys; }) targetSystems;
|
||||
failures = builtins.filter (r: !r.result.success) results;
|
||||
failMsg = lib.concatMapStringsSep "\n" (r: " - ${r.sys}") failures;
|
||||
in pkgs.runCommand "hermes-cross-eval" { } (
|
||||
if failures != [] then
|
||||
builtins.throw "Package fails to evaluate on:\n${failMsg}"
|
||||
else ''
|
||||
echo "PASS: package evaluates on all ${toString (builtins.length targetSystems)} platforms"
|
||||
mkdir -p $out
|
||||
echo "ok" > $out/result
|
||||
''
|
||||
);
|
||||
} // lib.optionalAttrs pkgs.stdenv.hostPlatform.isLinux {
|
||||
checks = lib.optionalAttrs pkgs.stdenv.hostPlatform.isLinux {
|
||||
# Verify binaries exist and are executable
|
||||
package-contents = pkgs.runCommand "hermes-package-contents" { } ''
|
||||
set -e
|
||||
@@ -126,51 +103,6 @@ json.dump(sorted(leaf_paths(DEFAULT_CONFIG)), sys.stdout, indent=2)
|
||||
echo "ok" > $out/result
|
||||
'';
|
||||
|
||||
# Verify bundled TUI is present and compiled
|
||||
bundled-tui = pkgs.runCommand "hermes-bundled-tui" { } ''
|
||||
set -e
|
||||
echo "=== Checking bundled TUI ==="
|
||||
test -d ${hermes-agent}/ui-tui || (echo "FAIL: ui-tui directory missing"; exit 1)
|
||||
echo "PASS: ui-tui directory exists"
|
||||
|
||||
test -f ${hermes-agent}/ui-tui/dist/entry.js || (echo "FAIL: compiled entry.js missing"; exit 1)
|
||||
echo "PASS: compiled entry.js present"
|
||||
|
||||
test -d ${hermes-agent}/ui-tui/node_modules || (echo "FAIL: node_modules missing"; exit 1)
|
||||
echo "PASS: node_modules present"
|
||||
|
||||
grep -q "HERMES_TUI_DIR" ${hermes-agent}/bin/hermes || \
|
||||
(echo "FAIL: HERMES_TUI_DIR not in wrapper"; exit 1)
|
||||
echo "PASS: HERMES_TUI_DIR set in wrapper"
|
||||
|
||||
echo "=== All bundled TUI checks passed ==="
|
||||
mkdir -p $out
|
||||
echo "ok" > $out/result
|
||||
'';
|
||||
|
||||
# Verify HERMES_NODE is set in wrapper and points to Node 20+
|
||||
# (string-width uses the /v regex flag which requires Node 20+)
|
||||
hermes-node = pkgs.runCommand "hermes-node-version" { } ''
|
||||
set -e
|
||||
echo "=== Checking HERMES_NODE in wrapper ==="
|
||||
grep -q "HERMES_NODE" ${hermes-agent}/bin/hermes || \
|
||||
(echo "FAIL: HERMES_NODE not set in wrapper"; exit 1)
|
||||
echo "PASS: HERMES_NODE present in wrapper"
|
||||
|
||||
HERMES_NODE=$(sed -n "s/^export HERMES_NODE='\(.*\)'/\1/p" ${hermes-agent}/bin/hermes)
|
||||
test -x "$HERMES_NODE" || (echo "FAIL: HERMES_NODE=$HERMES_NODE not executable"; exit 1)
|
||||
echo "PASS: HERMES_NODE executable at $HERMES_NODE"
|
||||
|
||||
NODE_MAJOR=$("$HERMES_NODE" --version | sed 's/^v//' | cut -d. -f1)
|
||||
test "$NODE_MAJOR" -ge 20 || \
|
||||
(echo "FAIL: Node v$NODE_MAJOR < 20, TUI needs /v regex flag support"; exit 1)
|
||||
echo "PASS: Node v$NODE_MAJOR >= 20"
|
||||
|
||||
echo "=== All HERMES_NODE checks passed ==="
|
||||
mkdir -p $out
|
||||
echo "ok" > $out/result
|
||||
'';
|
||||
|
||||
# Verify HERMES_MANAGED guard works on all mutation commands
|
||||
managed-guard = pkgs.runCommand "hermes-managed-guard" { } ''
|
||||
set -e
|
||||
|
||||
+38
-15
@@ -1,26 +1,49 @@
|
||||
# nix/devShell.nix — Dev shell that delegates setup to each package
|
||||
#
|
||||
# Each package in inputsFrom exposes passthru.devShellHook — a bash snippet
|
||||
# with stamp-checked setup logic. This file collects and runs them all.
|
||||
# nix/devShell.nix — Fast dev shell with stamp-file optimization
|
||||
{ inputs, ... }: {
|
||||
perSystem = { pkgs, system, ... }:
|
||||
perSystem = { pkgs, ... }:
|
||||
let
|
||||
hermes-agent = inputs.self.packages.${system}.default;
|
||||
hermes-tui = inputs.self.packages.${system}.tui;
|
||||
packages = [ hermes-agent hermes-tui ];
|
||||
python = pkgs.python311;
|
||||
in {
|
||||
devShells.default = pkgs.mkShell {
|
||||
inputsFrom = packages;
|
||||
packages = with pkgs; [
|
||||
python312 uv nodejs_22 ripgrep git openssh ffmpeg
|
||||
python uv nodejs_20 ripgrep git openssh ffmpeg
|
||||
];
|
||||
|
||||
shellHook = let
|
||||
hooks = map (p: p.passthru.devShellHook or "") packages;
|
||||
combined = pkgs.lib.concatStringsSep "\n" (builtins.filter (h: h != "") hooks);
|
||||
in ''
|
||||
shellHook = ''
|
||||
echo "Hermes Agent dev shell"
|
||||
${combined}
|
||||
|
||||
# Composite stamp: changes when nix python or uv change
|
||||
STAMP_VALUE="${python}:${pkgs.uv}"
|
||||
STAMP_FILE=".venv/.nix-stamp"
|
||||
|
||||
# Create venv if missing
|
||||
if [ ! -d .venv ]; then
|
||||
echo "Creating Python 3.11 venv..."
|
||||
uv venv .venv --python ${python}/bin/python3
|
||||
fi
|
||||
|
||||
source .venv/bin/activate
|
||||
|
||||
# Only install if stamp is stale or missing
|
||||
if [ ! -f "$STAMP_FILE" ] || [ "$(cat "$STAMP_FILE")" != "$STAMP_VALUE" ]; then
|
||||
echo "Installing Python dependencies..."
|
||||
uv pip install -e ".[all]"
|
||||
if [ -d mini-swe-agent ]; then
|
||||
uv pip install -e ./mini-swe-agent 2>/dev/null || true
|
||||
fi
|
||||
if [ -d tinker-atropos ]; then
|
||||
uv pip install -e ./tinker-atropos 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# Install npm deps
|
||||
if [ -f package.json ] && [ ! -d node_modules ]; then
|
||||
echo "Installing npm dependencies..."
|
||||
npm install
|
||||
fi
|
||||
|
||||
echo "$STAMP_VALUE" > "$STAMP_FILE"
|
||||
fi
|
||||
|
||||
echo "Ready. Run 'hermes' to start."
|
||||
'';
|
||||
};
|
||||
|
||||
+7
-14
@@ -121,19 +121,11 @@
|
||||
# ── Provision apt packages (first boot only, cached in writable layer) ──
|
||||
# sudo: agent self-modification
|
||||
# nodejs/npm: writable node so npm i -g works (nix store copies are read-only)
|
||||
# Node 22 via NodeSource — Ubuntu 24.04 ships Node 18 which is EOL.
|
||||
# curl: needed for uv installer + NodeSource setup
|
||||
# curl: needed for uv installer
|
||||
if [ ! -f /var/lib/hermes-tools-provisioned ] && command -v apt-get >/dev/null 2>&1; then
|
||||
echo "First boot: provisioning agent tools..."
|
||||
apt-get update -qq
|
||||
apt-get install -y -qq sudo curl ca-certificates gnupg
|
||||
mkdir -p /etc/apt/keyrings
|
||||
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key \
|
||||
| gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
|
||||
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_22.x nodistro main" \
|
||||
> /etc/apt/sources.list.d/nodesource.list
|
||||
apt-get update -qq
|
||||
apt-get install -y -qq nodejs
|
||||
apt-get install -y -qq sudo nodejs npm curl
|
||||
touch /var/lib/hermes-tools-provisioned
|
||||
fi
|
||||
|
||||
@@ -148,14 +140,15 @@
|
||||
su -s /bin/sh "$TARGET_USER" -c 'curl -LsSf https://astral.sh/uv/install.sh | sh' || true
|
||||
fi
|
||||
|
||||
# Python 3.12 venv — gives the agent a writable Python with pip.
|
||||
# Python 3.11 venv — gives the agent a writable Python with pip.
|
||||
# Uses uv to install Python 3.11 (Ubuntu 24.04 ships 3.12).
|
||||
# --seed includes pip/setuptools so bare `pip install` works.
|
||||
_UV_BIN="$TARGET_HOME/.local/bin/uv"
|
||||
if [ ! -d "$TARGET_HOME/.venv" ] && [ -x "$_UV_BIN" ]; then
|
||||
su -s /bin/sh "$TARGET_USER" -c "
|
||||
export PATH=\"\$HOME/.local/bin:\$PATH\"
|
||||
uv python install 3.12
|
||||
uv venv --python 3.12 --seed \"\$HOME/.venv\"
|
||||
uv python install 3.11
|
||||
uv venv --python 3.11 --seed \"\$HOME/.venv\"
|
||||
" || true
|
||||
fi
|
||||
|
||||
@@ -178,7 +171,7 @@
|
||||
# Package and entrypoint use stable symlinks (current-package, current-entrypoint)
|
||||
# so they can update without recreation. Env vars go through $HERMES_HOME/.env.
|
||||
containerIdentity = builtins.hashString "sha256" (builtins.toJSON {
|
||||
schema = 4; # bump when identity inputs change (4: Node 18→22 via NodeSource)
|
||||
schema = 3; # bump when identity inputs change
|
||||
image = cfg.container.image;
|
||||
extraVolumes = cfg.container.extraVolumes;
|
||||
extraOptions = cfg.container.extraOptions;
|
||||
|
||||
+29
-91
@@ -1,116 +1,54 @@
|
||||
# nix/packages.nix — Hermes Agent package built with uv2nix
|
||||
{ inputs, ... }:
|
||||
{
|
||||
perSystem =
|
||||
{ pkgs, inputs', ... }:
|
||||
{ inputs, ... }: {
|
||||
perSystem = { pkgs, system, ... }:
|
||||
let
|
||||
hermesVenv = pkgs.callPackage ./python.nix {
|
||||
inherit (inputs) uv2nix pyproject-nix pyproject-build-systems;
|
||||
};
|
||||
|
||||
hermesTui = pkgs.callPackage ./tui.nix {
|
||||
npm-lockfile-fix = inputs'.npm-lockfile-fix.packages.default;
|
||||
};
|
||||
|
||||
# Import bundled skills, excluding runtime caches
|
||||
bundledSkills = pkgs.lib.cleanSourceWith {
|
||||
src = ../skills;
|
||||
filter = path: _type: !(pkgs.lib.hasInfix "/index-cache/" path);
|
||||
};
|
||||
|
||||
hermesWeb = pkgs.callPackage ./web.nix {
|
||||
npm-lockfile-fix = inputs'.npm-lockfile-fix.packages.default;
|
||||
filter = path: _type:
|
||||
!(pkgs.lib.hasInfix "/index-cache/" path);
|
||||
};
|
||||
|
||||
runtimeDeps = with pkgs; [
|
||||
nodejs_22
|
||||
ripgrep
|
||||
git
|
||||
openssh
|
||||
ffmpeg
|
||||
tirith
|
||||
nodejs_20 ripgrep git openssh ffmpeg tirith
|
||||
];
|
||||
|
||||
runtimePath = pkgs.lib.makeBinPath runtimeDeps;
|
||||
in {
|
||||
packages.default = pkgs.stdenv.mkDerivation {
|
||||
pname = "hermes-agent";
|
||||
version = (builtins.fromTOML (builtins.readFile ../pyproject.toml)).project.version;
|
||||
|
||||
# Lockfile hashes for dev shell stamps
|
||||
pyprojectHash = builtins.hashString "sha256" (builtins.readFile ../pyproject.toml);
|
||||
uvLockHash =
|
||||
if builtins.pathExists ../uv.lock then
|
||||
builtins.hashString "sha256" (builtins.readFile ../uv.lock)
|
||||
else
|
||||
"none";
|
||||
in
|
||||
{
|
||||
packages = {
|
||||
default = pkgs.stdenv.mkDerivation {
|
||||
pname = "hermes-agent";
|
||||
version = (fromTOML (builtins.readFile ../pyproject.toml)).project.version;
|
||||
dontUnpack = true;
|
||||
dontBuild = true;
|
||||
nativeBuildInputs = [ pkgs.makeWrapper ];
|
||||
|
||||
dontUnpack = true;
|
||||
dontBuild = true;
|
||||
nativeBuildInputs = [ pkgs.makeWrapper ];
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
mkdir -p $out/share/hermes-agent $out/bin
|
||||
cp -r ${bundledSkills} $out/share/hermes-agent/skills
|
||||
|
||||
mkdir -p $out/share/hermes-agent $out/bin
|
||||
cp -r ${bundledSkills} $out/share/hermes-agent/skills
|
||||
cp -r ${hermesWeb} $out/share/hermes-agent/web_dist
|
||||
${pkgs.lib.concatMapStringsSep "\n" (name: ''
|
||||
makeWrapper ${hermesVenv}/bin/${name} $out/bin/${name} \
|
||||
--suffix PATH : "${runtimePath}" \
|
||||
--set HERMES_BUNDLED_SKILLS $out/share/hermes-agent/skills
|
||||
'') [ "hermes" "hermes-agent" "hermes-acp" ]}
|
||||
|
||||
# copy pre-built TUI (same layout as dev: ui-tui/dist/ + node_modules/)
|
||||
mkdir -p $out/ui-tui
|
||||
cp -r ${hermesTui}/lib/hermes-tui/* $out/ui-tui/
|
||||
runHook postInstall
|
||||
'';
|
||||
|
||||
${pkgs.lib.concatMapStringsSep "\n"
|
||||
(name: ''
|
||||
makeWrapper ${hermesVenv}/bin/${name} $out/bin/${name} \
|
||||
--suffix PATH : "${runtimePath}" \
|
||||
--set HERMES_BUNDLED_SKILLS $out/share/hermes-agent/skills \
|
||||
--set HERMES_WEB_DIST $out/share/hermes-agent/web_dist \
|
||||
--set HERMES_TUI_DIR $out/ui-tui \
|
||||
--set HERMES_PYTHON ${hermesVenv}/bin/python3 \
|
||||
--set HERMES_NODE ${pkgs.nodejs_22}/bin/node
|
||||
'')
|
||||
[
|
||||
"hermes"
|
||||
"hermes-agent"
|
||||
"hermes-acp"
|
||||
]
|
||||
}
|
||||
|
||||
runHook postInstall
|
||||
'';
|
||||
|
||||
passthru.devShellHook = ''
|
||||
STAMP=".nix-stamps/hermes-agent"
|
||||
STAMP_VALUE="${pyprojectHash}:${uvLockHash}"
|
||||
if [ ! -f "$STAMP" ] || [ "$(cat "$STAMP")" != "$STAMP_VALUE" ]; then
|
||||
echo "hermes-agent: installing Python dependencies..."
|
||||
uv venv .venv --python ${pkgs.python312}/bin/python3 2>/dev/null || true
|
||||
source .venv/bin/activate
|
||||
uv pip install -e ".[all]"
|
||||
[ -d mini-swe-agent ] && uv pip install -e ./mini-swe-agent 2>/dev/null || true
|
||||
[ -d tinker-atropos ] && uv pip install -e ./tinker-atropos 2>/dev/null || true
|
||||
mkdir -p .nix-stamps
|
||||
echo "$STAMP_VALUE" > "$STAMP"
|
||||
else
|
||||
source .venv/bin/activate
|
||||
export HERMES_PYTHON=${hermesVenv}/bin/python3
|
||||
fi
|
||||
'';
|
||||
|
||||
meta = with pkgs.lib; {
|
||||
description = "AI agent with advanced tool-calling capabilities";
|
||||
homepage = "https://github.com/NousResearch/hermes-agent";
|
||||
mainProgram = "hermes";
|
||||
license = licenses.mit;
|
||||
platforms = platforms.unix;
|
||||
};
|
||||
meta = with pkgs.lib; {
|
||||
description = "AI agent with advanced tool-calling capabilities";
|
||||
homepage = "https://github.com/NousResearch/hermes-agent";
|
||||
mainProgram = "hermes";
|
||||
license = licenses.mit;
|
||||
platforms = platforms.unix;
|
||||
};
|
||||
|
||||
tui = hermesTui;
|
||||
web = hermesWeb;
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
+9
-26
@@ -1,6 +1,6 @@
|
||||
# nix/python.nix — uv2nix virtual environment builder
|
||||
{
|
||||
python312,
|
||||
python311,
|
||||
lib,
|
||||
callPackage,
|
||||
uv2nix,
|
||||
@@ -35,46 +35,30 @@ let
|
||||
};
|
||||
};
|
||||
|
||||
# Legacy alibabacloud packages ship only sdists with setup.py/setup.cfg
|
||||
# and no pyproject.toml, so setuptools isn't declared as a build dep.
|
||||
buildSystemOverrides = final: prev: builtins.mapAttrs
|
||||
(name: _: prev.${name}.overrideAttrs (old: {
|
||||
nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ final.setuptools ];
|
||||
}))
|
||||
(lib.genAttrs [
|
||||
"alibabacloud-credentials-api"
|
||||
"alibabacloud-endpoint-util"
|
||||
"alibabacloud-gateway-dingtalk"
|
||||
"alibabacloud-gateway-spi"
|
||||
"alibabacloud-tea"
|
||||
] (_: null));
|
||||
|
||||
pythonPackageOverrides = final: _prev:
|
||||
if isAarch64Darwin then {
|
||||
numpy = mkPrebuiltOverride final python312.pkgs.numpy { };
|
||||
numpy = mkPrebuiltOverride final python311.pkgs.numpy { };
|
||||
|
||||
pyarrow = mkPrebuiltOverride final python312.pkgs.pyarrow { };
|
||||
av = mkPrebuiltOverride final python311.pkgs.av { };
|
||||
|
||||
av = mkPrebuiltOverride final python312.pkgs.av { };
|
||||
humanfriendly = mkPrebuiltOverride final python311.pkgs.humanfriendly { };
|
||||
|
||||
humanfriendly = mkPrebuiltOverride final python312.pkgs.humanfriendly { };
|
||||
|
||||
coloredlogs = mkPrebuiltOverride final python312.pkgs.coloredlogs {
|
||||
coloredlogs = mkPrebuiltOverride final python311.pkgs.coloredlogs {
|
||||
humanfriendly = [ ];
|
||||
};
|
||||
|
||||
onnxruntime = mkPrebuiltOverride final python312.pkgs.onnxruntime {
|
||||
onnxruntime = mkPrebuiltOverride final python311.pkgs.onnxruntime {
|
||||
coloredlogs = [ ];
|
||||
numpy = [ ];
|
||||
packaging = [ ];
|
||||
};
|
||||
|
||||
ctranslate2 = mkPrebuiltOverride final python312.pkgs.ctranslate2 {
|
||||
ctranslate2 = mkPrebuiltOverride final python311.pkgs.ctranslate2 {
|
||||
numpy = [ ];
|
||||
pyyaml = [ ];
|
||||
};
|
||||
|
||||
faster-whisper = mkPrebuiltOverride final python312.pkgs.faster-whisper {
|
||||
faster-whisper = mkPrebuiltOverride final python311.pkgs.faster-whisper {
|
||||
av = [ ];
|
||||
ctranslate2 = [ ];
|
||||
huggingface-hub = [ ];
|
||||
@@ -86,12 +70,11 @@ let
|
||||
|
||||
pythonSet =
|
||||
(callPackage pyproject-nix.build.packages {
|
||||
python = python312;
|
||||
python = python311;
|
||||
}).overrideScope
|
||||
(lib.composeManyExtensions [
|
||||
pyproject-build-systems.overlays.default
|
||||
overlay
|
||||
buildSystemOverrides
|
||||
pythonPackageOverrides
|
||||
]);
|
||||
in
|
||||
|
||||
-77
@@ -1,77 +0,0 @@
|
||||
# nix/tui.nix — Hermes TUI (Ink/React) compiled with tsc and bundled
|
||||
{ pkgs, npm-lockfile-fix, ... }:
|
||||
let
|
||||
src = ../ui-tui;
|
||||
npmDeps = pkgs.fetchNpmDeps {
|
||||
inherit src;
|
||||
hash = "sha256-mG3vpgGi4ljt4X3XIf3I/5mIcm+rVTUAmx2DQ6YVA90=";
|
||||
};
|
||||
|
||||
packageJson = builtins.fromJSON (builtins.readFile (src + "/package.json"));
|
||||
version = packageJson.version;
|
||||
|
||||
npmLockHash = builtins.hashString "sha256" (builtins.readFile ../ui-tui/package-lock.json);
|
||||
in
|
||||
pkgs.buildNpmPackage {
|
||||
pname = "hermes-tui";
|
||||
inherit src npmDeps version;
|
||||
|
||||
doCheck = false;
|
||||
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
|
||||
mkdir -p $out/lib/hermes-tui
|
||||
|
||||
cp -r dist $out/lib/hermes-tui/dist
|
||||
|
||||
# runtime node_modules
|
||||
cp -r node_modules $out/lib/hermes-tui/node_modules
|
||||
|
||||
# @hermes/ink is a file: dependency, we need to copy it in fr
|
||||
rm -f $out/lib/hermes-tui/node_modules/@hermes/ink
|
||||
cp -r packages/hermes-ink $out/lib/hermes-tui/node_modules/@hermes/ink
|
||||
|
||||
# package.json needed for "type": "module" resolution
|
||||
cp package.json $out/lib/hermes-tui/
|
||||
|
||||
runHook postInstall
|
||||
'';
|
||||
|
||||
nativeBuildInputs = [
|
||||
(pkgs.writeShellScriptBin "update_tui_lockfile" ''
|
||||
set -euox pipefail
|
||||
|
||||
# get root of repo
|
||||
REPO_ROOT=$(git rev-parse --show-toplevel)
|
||||
|
||||
# cd into ui-tui and reinstall
|
||||
cd "$REPO_ROOT/ui-tui"
|
||||
rm -rf node_modules/
|
||||
npm cache clean --force
|
||||
CI=true npm install # ci env var to suppress annoying unicode install banner lag
|
||||
${pkgs.lib.getExe npm-lockfile-fix} ./package-lock.json
|
||||
|
||||
NIX_FILE="$REPO_ROOT/nix/tui.nix"
|
||||
# compute the new hash
|
||||
sed -i "s/hash = \"[^\"]*\";/hash = \"\";/" $NIX_FILE
|
||||
NIX_OUTPUT=$(nix build .#tui 2>&1 || true)
|
||||
NEW_HASH=$(echo "$NIX_OUTPUT" | grep 'got:' | awk '{print $2}')
|
||||
echo got new hash $NEW_HASH
|
||||
sed -i "s|hash = \"[^\"]*\";|hash = \"$NEW_HASH\";|" $NIX_FILE
|
||||
nix build .#tui
|
||||
echo "Updated npm hash in $NIX_FILE to $NEW_HASH"
|
||||
'')
|
||||
];
|
||||
|
||||
passthru.devShellHook = ''
|
||||
STAMP=".nix-stamps/hermes-tui"
|
||||
STAMP_VALUE="${npmLockHash}"
|
||||
if [ ! -f "$STAMP" ] || [ "$(cat "$STAMP")" != "$STAMP_VALUE" ]; then
|
||||
echo "hermes-tui: installing npm dependencies..."
|
||||
cd ui-tui && CI=true npm install --silent --no-fund --no-audit 2>/dev/null && cd ..
|
||||
mkdir -p .nix-stamps
|
||||
echo "$STAMP_VALUE" > "$STAMP"
|
||||
fi
|
||||
'';
|
||||
}
|
||||
-63
@@ -1,63 +0,0 @@
|
||||
# nix/web.nix — Hermes Web Dashboard (Vite/React) frontend build
|
||||
{ pkgs, npm-lockfile-fix, ... }:
|
||||
let
|
||||
src = ../web;
|
||||
npmDeps = pkgs.fetchNpmDeps {
|
||||
inherit src;
|
||||
hash = "sha256-Y0pOzdFG8BLjfvCLmsvqYpjxFjAQabXp1i7X9W/cCU4=";
|
||||
};
|
||||
|
||||
npmLockHash = builtins.hashString "sha256" (builtins.readFile ../web/package-lock.json);
|
||||
in
|
||||
pkgs.buildNpmPackage {
|
||||
pname = "hermes-web";
|
||||
version = "0.0.0";
|
||||
inherit src npmDeps;
|
||||
|
||||
doCheck = false;
|
||||
|
||||
buildPhase = ''
|
||||
npx tsc -b
|
||||
npx vite build --outDir dist
|
||||
'';
|
||||
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
cp -r dist $out
|
||||
runHook postInstall
|
||||
'';
|
||||
|
||||
nativeBuildInputs = [
|
||||
(pkgs.writeShellScriptBin "update_web_lockfile" ''
|
||||
set -euox pipefail
|
||||
|
||||
REPO_ROOT=$(git rev-parse --show-toplevel)
|
||||
|
||||
cd "$REPO_ROOT/web"
|
||||
rm -rf node_modules/
|
||||
npm cache clean --force
|
||||
CI=true npm install
|
||||
${pkgs.lib.getExe npm-lockfile-fix} ./package-lock.json
|
||||
|
||||
NIX_FILE="$REPO_ROOT/nix/web.nix"
|
||||
sed -i "s/hash = \"[^\"]*\";/hash = \"\";/" $NIX_FILE
|
||||
NIX_OUTPUT=$(nix build .#web 2>&1 || true)
|
||||
NEW_HASH=$(echo "$NIX_OUTPUT" | grep 'got:' | awk '{print $2}')
|
||||
echo got new hash $NEW_HASH
|
||||
sed -i "s|hash = \"[^\"]*\";|hash = \"$NEW_HASH\";|" $NIX_FILE
|
||||
nix build .#web
|
||||
echo "Updated npm hash in $NIX_FILE to $NEW_HASH"
|
||||
'')
|
||||
];
|
||||
|
||||
passthru.devShellHook = ''
|
||||
STAMP=".nix-stamps/hermes-web"
|
||||
STAMP_VALUE="${npmLockHash}"
|
||||
if [ ! -f "$STAMP" ] || [ "$(cat "$STAMP")" != "$STAMP_VALUE" ]; then
|
||||
echo "hermes-web: installing npm dependencies..."
|
||||
cd web && CI=true npm install --silent --no-fund --no-audit 2>/dev/null && cd ..
|
||||
mkdir -p .nix-stamps
|
||||
echo "$STAMP_VALUE" > "$STAMP"
|
||||
fi
|
||||
'';
|
||||
}
|
||||
@@ -145,10 +145,10 @@ Controls **how often** dialectic and context calls happen.
|
||||
| Key | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `contextCadence` | `1` | Min turns between context API calls |
|
||||
| `dialecticCadence` | `2` | Min turns between dialectic API calls. Recommended 1–5 |
|
||||
| `dialecticCadence` | `3` | Min turns between dialectic API calls |
|
||||
| `injectionFrequency` | `every-turn` | `every-turn` or `first-turn` for base context injection |
|
||||
|
||||
Higher cadence values fire the dialectic LLM less often. `dialecticCadence: 2` means the engine fires every other turn. Setting it to `1` fires every turn.
|
||||
Higher cadence values reduce API calls and cost. `dialecticCadence: 3` (default) means the dialectic engine fires at most every 3rd turn.
|
||||
|
||||
### Depth (how many)
|
||||
|
||||
@@ -180,8 +180,6 @@ If `dialecticDepthLevels` is omitted, rounds use **proportional levels** derived
|
||||
|
||||
This keeps earlier passes cheap while using full depth on the final synthesis.
|
||||
|
||||
**Depth at session start.** The session-start prewarm runs the full configured `dialecticDepth` in the background before turn 1. A single-pass prewarm on a cold peer often returns thin output — multi-pass depth runs the audit/reconcile cycle before the user ever speaks. Turn 1 consumes the prewarm result directly; if prewarm hasn't landed in time, turn 1 falls back to a synchronous call with a bounded timeout.
|
||||
|
||||
### Level (how hard)
|
||||
|
||||
Controls the **intensity** of each dialectic reasoning round.
|
||||
@@ -370,7 +368,7 @@ Config file: `$HERMES_HOME/honcho.json` (profile-local) or `~/.honcho/config.jso
|
||||
| `contextTokens` | uncapped | Max tokens for the combined base context injection (summary + representation + card). Opt-in cap — omit to leave uncapped, set to an integer to bound injection size. |
|
||||
| `injectionFrequency` | `every-turn` | `every-turn` or `first-turn` |
|
||||
| `contextCadence` | `1` | Min turns between context API calls |
|
||||
| `dialecticCadence` | `2` | Min turns between dialectic LLM calls (recommended 1–5) |
|
||||
| `dialecticCadence` | `3` | Min turns between dialectic LLM calls |
|
||||
|
||||
The `contextTokens` budget is enforced at injection time. If the session summary + representation + card exceed the budget, Honcho trims the summary first, then the representation, preserving the card. This prevents context blowup in long sessions.
|
||||
|
||||
|
||||
@@ -1,339 +0,0 @@
|
||||
---
|
||||
name: touchdesigner-mcp
|
||||
description: "Control a running TouchDesigner instance via twozero MCP — create operators, set parameters, wire connections, execute Python, build real-time visuals. 36 native tools."
|
||||
version: 1.0.0
|
||||
author: kshitijk4poor
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [TouchDesigner, MCP, twozero, creative-coding, real-time-visuals, generative-art, audio-reactive, VJ, installation, GLSL]
|
||||
related_skills: [native-mcp, ascii-video, manim-video, hermes-video]
|
||||
|
||||
---
|
||||
|
||||
# TouchDesigner Integration (twozero MCP)
|
||||
|
||||
## CRITICAL RULES
|
||||
|
||||
1. **NEVER guess parameter names.** Call `td_get_par_info` for the op type FIRST. Your training data is wrong for TD 2025.32.
|
||||
2. **If `tdAttributeError` fires, STOP.** Call `td_get_operator_info` on the failing node before continuing.
|
||||
3. **NEVER hardcode absolute paths** in script callbacks. Use `me.parent()` / `scriptOp.parent()`.
|
||||
4. **Prefer native MCP tools over td_execute_python.** Use `td_create_operator`, `td_set_operator_pars`, `td_get_errors` etc. Only fall back to `td_execute_python` for complex multi-step logic.
|
||||
5. **Call `td_get_hints` before building.** It returns patterns specific to the op type you're working with.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Hermes Agent -> MCP (Streamable HTTP) -> twozero.tox (port 40404) -> TD Python
|
||||
```
|
||||
|
||||
36 native tools. Free plugin (no payment/license — confirmed April 2026).
|
||||
Context-aware (knows selected OP, current network).
|
||||
Hub health check: `GET http://localhost:40404/mcp` returns JSON with instance PID, project name, TD version.
|
||||
|
||||
## Setup (Automated)
|
||||
|
||||
Run the setup script to handle everything:
|
||||
|
||||
```bash
|
||||
bash "${HERMES_HOME:-$HOME/.hermes}/skills/creative/touchdesigner-mcp/scripts/setup.sh"
|
||||
```
|
||||
|
||||
The script will:
|
||||
1. Check if TD is running
|
||||
2. Download twozero.tox if not already cached
|
||||
3. Add `twozero_td` MCP server to Hermes config (if missing)
|
||||
4. Test the MCP connection on port 40404
|
||||
5. Report what manual steps remain (drag .tox into TD, enable MCP toggle)
|
||||
|
||||
### Manual steps (one-time, cannot be automated)
|
||||
|
||||
1. **Drag `~/Downloads/twozero.tox` into the TD network editor** → click Install
|
||||
2. **Enable MCP:** click twozero icon → Settings → mcp → "auto start MCP" → Yes
|
||||
3. **Restart Hermes session** to pick up the new MCP server
|
||||
|
||||
After setup, verify:
|
||||
```bash
|
||||
nc -z 127.0.0.1 40404 && echo "twozero MCP: READY"
|
||||
```
|
||||
|
||||
## Environment Notes
|
||||
|
||||
- **Non-Commercial TD** caps resolution at 1280×1280. Use `outputresolution = 'custom'` and set width/height explicitly.
|
||||
- **Codecs:** `prores` (preferred on macOS) or `mjpa` as fallback. H.264/H.265/AV1 require a Commercial license.
|
||||
- Always call `td_get_par_info` before setting params — names vary by TD version (see CRITICAL RULES #1).
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 0: Discover (before building anything)
|
||||
|
||||
```
|
||||
Call td_get_par_info with op_type for each type you plan to use.
|
||||
Call td_get_hints with the topic you're building (e.g. "glsl", "audio reactive", "feedback").
|
||||
Call td_get_focus to see where the user is and what's selected.
|
||||
Call td_get_network to see what already exists.
|
||||
```
|
||||
|
||||
No temp nodes, no cleanup. This replaces the old discovery dance entirely.
|
||||
|
||||
### Step 1: Clean + Build
|
||||
|
||||
**IMPORTANT: Split cleanup and creation into SEPARATE MCP calls.** Destroying and recreating same-named nodes in one `td_execute_python` script causes "Invalid OP object" errors. See pitfalls #11b.
|
||||
|
||||
Use `td_create_operator` for each node (handles viewport positioning automatically):
|
||||
|
||||
```
|
||||
td_create_operator(type="noiseTOP", parent="/project1", name="bg", parameters={"resolutionw": 1280, "resolutionh": 720})
|
||||
td_create_operator(type="levelTOP", parent="/project1", name="brightness")
|
||||
td_create_operator(type="nullTOP", parent="/project1", name="out")
|
||||
```
|
||||
|
||||
For bulk creation or wiring, use `td_execute_python`:
|
||||
|
||||
```python
|
||||
# td_execute_python script:
|
||||
root = op('/project1')
|
||||
nodes = []
|
||||
for name, optype in [('bg', noiseTOP), ('fx', levelTOP), ('out', nullTOP)]:
|
||||
n = root.create(optype, name)
|
||||
nodes.append(n.path)
|
||||
# Wire chain
|
||||
for i in range(len(nodes)-1):
|
||||
op(nodes[i]).outputConnectors[0].connect(op(nodes[i+1]).inputConnectors[0])
|
||||
result = {'created': nodes}
|
||||
```
|
||||
|
||||
### Step 2: Set Parameters
|
||||
|
||||
Prefer the native tool (validates params, won't crash):
|
||||
|
||||
```
|
||||
td_set_operator_pars(path="/project1/bg", parameters={"roughness": 0.6, "monochrome": true})
|
||||
```
|
||||
|
||||
For expressions or modes, use `td_execute_python`:
|
||||
|
||||
```python
|
||||
op('/project1/time_driver').par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
```
|
||||
|
||||
### Step 3: Wire
|
||||
|
||||
Use `td_execute_python` — no native wire tool exists:
|
||||
|
||||
```python
|
||||
op('/project1/bg').outputConnectors[0].connect(op('/project1/fx').inputConnectors[0])
|
||||
```
|
||||
|
||||
### Step 4: Verify
|
||||
|
||||
```
|
||||
td_get_errors(path="/project1", recursive=true)
|
||||
td_get_perf()
|
||||
td_get_operator_info(path="/project1/out", detail="full")
|
||||
```
|
||||
|
||||
### Step 5: Display / Capture
|
||||
|
||||
```
|
||||
td_get_screenshot(path="/project1/out")
|
||||
```
|
||||
|
||||
Or open a window via script:
|
||||
|
||||
```python
|
||||
win = op('/project1').create(windowCOMP, 'display')
|
||||
win.par.winop = op('/project1/out').path
|
||||
win.par.winw = 1280; win.par.winh = 720
|
||||
win.par.winopen.pulse()
|
||||
```
|
||||
|
||||
## MCP Tool Quick Reference
|
||||
|
||||
**Core (use these most):**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_execute_python` | Run arbitrary Python in TD. Full API access. |
|
||||
| `td_create_operator` | Create node with params + auto-positioning |
|
||||
| `td_set_operator_pars` | Set params safely (validates, won't crash) |
|
||||
| `td_get_operator_info` | Inspect one node: connections, params, errors |
|
||||
| `td_get_operators_info` | Inspect multiple nodes in one call |
|
||||
| `td_get_network` | See network structure at a path |
|
||||
| `td_get_errors` | Find errors/warnings recursively |
|
||||
| `td_get_par_info` | Get param names for an OP type (replaces discovery) |
|
||||
| `td_get_hints` | Get patterns/tips before building |
|
||||
| `td_get_focus` | What network is open, what's selected |
|
||||
|
||||
**Read/Write:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_read_dat` | Read DAT text content |
|
||||
| `td_write_dat` | Write/patch DAT content |
|
||||
| `td_read_chop` | Read CHOP channel values |
|
||||
| `td_read_textport` | Read TD console output |
|
||||
|
||||
**Visual:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_get_screenshot` | Capture one OP viewer to file |
|
||||
| `td_get_screenshots` | Capture multiple OPs at once |
|
||||
| `td_get_screen_screenshot` | Capture actual screen via TD |
|
||||
| `td_navigate_to` | Jump network editor to an OP |
|
||||
|
||||
**Search:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_find_op` | Find ops by name/type across project |
|
||||
| `td_search` | Search code, expressions, string params |
|
||||
|
||||
**System:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_get_perf` | Performance profiling (FPS, slow ops) |
|
||||
| `td_list_instances` | List all running TD instances |
|
||||
| `td_get_docs` | In-depth docs on a TD topic |
|
||||
| `td_agents_md` | Read/write per-COMP markdown docs |
|
||||
| `td_reinit_extension` | Reload extension after code edit |
|
||||
| `td_clear_textport` | Clear console before debug session |
|
||||
|
||||
**Input Automation:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_input_execute` | Send mouse/keyboard to TD |
|
||||
| `td_input_status` | Poll input queue status |
|
||||
| `td_input_clear` | Stop input automation |
|
||||
| `td_op_screen_rect` | Get screen coords of a node |
|
||||
| `td_click_screen_point` | Click a point in a screenshot |
|
||||
|
||||
See `references/mcp-tools.md` for full parameter schemas.
|
||||
|
||||
## Key Implementation Rules
|
||||
|
||||
**GLSL time:** No `uTDCurrentTime` in GLSL TOP. Use the Values page:
|
||||
```python
|
||||
# Call td_get_par_info(op_type="glslTOP") first to confirm param names
|
||||
td_set_operator_pars(path="/project1/shader", parameters={"value0name": "uTime"})
|
||||
# Then set expression via script:
|
||||
# op('/project1/shader').par.value0.expr = "absTime.seconds"
|
||||
# In GLSL: uniform float uTime;
|
||||
```
|
||||
|
||||
Fallback: Constant TOP in `rgba32float` format (8-bit clamps to 0-1, freezing the shader).
|
||||
|
||||
**Feedback TOP:** Use `top` parameter reference, not direct input wire. "Not enough sources" resolves after first cook. "Cook dependency loop" warning is expected.
|
||||
|
||||
**Resolution:** Non-Commercial caps at 1280×1280. Use `outputresolution = 'custom'`.
|
||||
|
||||
**Large shaders:** Write GLSL to `/tmp/file.glsl`, then use `td_write_dat` or `td_execute_python` to load.
|
||||
|
||||
**Vertex/Point access (TD 2025.32):** `point.P[0]`, `point.P[1]`, `point.P[2]` — NOT `.x`, `.y`, `.z`.
|
||||
|
||||
**Extensions:** `ext0object` format is `"op('./datName').module.ClassName(me)"` in CONSTANT mode. After editing extension code with `td_write_dat`, call `td_reinit_extension`.
|
||||
|
||||
**Script callbacks:** ALWAYS use relative paths via `me.parent()` / `scriptOp.parent()`.
|
||||
|
||||
**Cleaning nodes:** Always `list(root.children)` before iterating + `child.valid` check.
|
||||
|
||||
## Recording / Exporting Video
|
||||
|
||||
```python
|
||||
# via td_execute_python:
|
||||
root = op('/project1')
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
op('/project1/out').outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'prores' # Apple ProRes — NOT license-restricted on macOS
|
||||
rec.par.record = True # start
|
||||
# rec.par.record = False # stop (call separately later)
|
||||
```
|
||||
|
||||
H.264/H.265/AV1 need Commercial license. Use `prores` on macOS or `mjpa` as fallback.
|
||||
Extract frames: `ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png`
|
||||
|
||||
**TOP.save() is useless for animation** — captures same GPU texture every time. Always use MovieFileOut.
|
||||
|
||||
### Before Recording: Checklist
|
||||
|
||||
1. **Verify FPS > 0** via `td_get_perf`. If FPS=0 the recording will be empty. See pitfalls #38-39.
|
||||
2. **Verify shader output is not black** via `td_get_screenshot`. Black output = shader error or missing input. See pitfalls #8, #40.
|
||||
3. **If recording with audio:** cue audio to start first, then delay recording by 3 frames. See pitfalls #19.
|
||||
4. **Set output path before starting record** — setting both in the same script can race.
|
||||
|
||||
## Audio-Reactive GLSL (Proven Recipe)
|
||||
|
||||
### Correct signal chain (tested April 2026)
|
||||
|
||||
```
|
||||
AudioFileIn CHOP (playmode=sequential)
|
||||
→ AudioSpectrum CHOP (FFT=512, outputmenu=setmanually, outlength=256, timeslice=ON)
|
||||
→ Math CHOP (gain=10)
|
||||
→ CHOP to TOP (dataformat=r, layout=rowscropped)
|
||||
→ GLSL TOP input 1 (spectrum texture, 256x2)
|
||||
|
||||
Constant TOP (rgba32float, time) → GLSL TOP input 0
|
||||
GLSL TOP → Null TOP → MovieFileOut
|
||||
```
|
||||
|
||||
### Critical audio-reactive rules (empirically verified)
|
||||
|
||||
1. **TimeSlice must stay ON** for AudioSpectrum. OFF = processes entire audio file → 24000+ samples → CHOP to TOP overflow.
|
||||
2. **Set Output Length manually** to 256 via `outputmenu='setmanually'` and `outlength=256`. Default outputs 22050 samples.
|
||||
3. **DO NOT use Lag CHOP for spectrum smoothing.** Lag CHOP operates in timeslice mode and expands 256 samples to 2400+, averaging all values to near-zero (~1e-06). The shader receives no usable data. This was the #1 audio sync failure in testing.
|
||||
4. **DO NOT use Filter CHOP either** — same timeslice expansion problem with spectrum data.
|
||||
5. **Smoothing belongs in the GLSL shader** if needed, via temporal lerp with a feedback texture: `mix(prevValue, newValue, 0.3)`. This gives frame-perfect sync with zero pipeline latency.
|
||||
6. **CHOP to TOP dataformat = 'r'**, layout = 'rowscropped'. Spectrum output is 256x2 (stereo). Sample at y=0.25 for first channel.
|
||||
7. **Math gain = 10** (not 5). Raw spectrum values are ~0.19 in bass range. Gain of 10 gives usable ~5.0 for the shader.
|
||||
8. **No Resample CHOP needed.** Control output size via AudioSpectrum's `outlength` param directly.
|
||||
|
||||
### GLSL spectrum sampling
|
||||
|
||||
```glsl
|
||||
// Input 0 = time (1x1 rgba32float), Input 1 = spectrum (256x2)
|
||||
float iTime = texture(sTD2DInputs[0], vec2(0.5)).r;
|
||||
|
||||
// Sample multiple points per band and average for stability:
|
||||
// NOTE: y=0.25 for first channel (stereo texture is 256x2, first row center is 0.25)
|
||||
float bass = (texture(sTD2DInputs[1], vec2(0.02, 0.25)).r +
|
||||
texture(sTD2DInputs[1], vec2(0.05, 0.25)).r) / 2.0;
|
||||
float mid = (texture(sTD2DInputs[1], vec2(0.2, 0.25)).r +
|
||||
texture(sTD2DInputs[1], vec2(0.35, 0.25)).r) / 2.0;
|
||||
float hi = (texture(sTD2DInputs[1], vec2(0.6, 0.25)).r +
|
||||
texture(sTD2DInputs[1], vec2(0.8, 0.25)).r) / 2.0;
|
||||
```
|
||||
|
||||
See `references/network-patterns.md` for complete build scripts + shader code.
|
||||
|
||||
## Operator Quick Reference
|
||||
|
||||
| Family | Color | Python class / MCP type | Suffix |
|
||||
|--------|-------|-------------|--------|
|
||||
| TOP | Purple | noiseTOP, glslTOP, compositeTOP, levelTop, blurTOP, textTOP, nullTOP | TOP |
|
||||
| CHOP | Green | audiofileinCHOP, audiospectrumCHOP, mathCHOP, lfoCHOP, constantCHOP | CHOP |
|
||||
| SOP | Blue | gridSOP, sphereSOP, transformSOP, noiseSOP | SOP |
|
||||
| DAT | White | textDAT, tableDAT, scriptDAT, webserverDAT | DAT |
|
||||
| MAT | Yellow | phongMAT, pbrMAT, glslMAT, constMAT | MAT |
|
||||
| COMP | Gray | geometryCOMP, containerCOMP, cameraCOMP, lightCOMP, windowCOMP | COMP |
|
||||
|
||||
## Security Notes
|
||||
|
||||
- MCP runs on localhost only (port 40404). No authentication — any local process can send commands.
|
||||
- `td_execute_python` has unrestricted access to the TD Python environment and filesystem as the TD process user.
|
||||
- `setup.sh` downloads twozero.tox from the official 404zero.com URL. Verify the download if concerned.
|
||||
- The skill never sends data outside localhost. All MCP communication is local.
|
||||
|
||||
## References
|
||||
|
||||
| File | What |
|
||||
|------|------|
|
||||
| `references/pitfalls.md` | Hard-won lessons from real sessions |
|
||||
| `references/operators.md` | All operator families with params and use cases |
|
||||
| `references/network-patterns.md` | Recipes: audio-reactive, generative, GLSL, instancing |
|
||||
| `references/mcp-tools.md` | Full twozero MCP tool parameter schemas |
|
||||
| `references/python-api.md` | TD Python: op(), scripting, extensions |
|
||||
| `references/troubleshooting.md` | Connection diagnostics, debugging |
|
||||
| `scripts/setup.sh` | Automated setup script |
|
||||
|
||||
---
|
||||
|
||||
> You're not writing code. You're conducting light.
|
||||
@@ -1,382 +0,0 @@
|
||||
# twozero MCP Tools Reference
|
||||
|
||||
36 tools from twozero MCP v2.774+ (April 2026).
|
||||
All tools accept an optional `target_instance` param for multi-TD-instance scenarios.
|
||||
|
||||
## Execution & Scripting
|
||||
|
||||
### td_execute_python
|
||||
|
||||
Execute Python code inside TouchDesigner and return the result. Has full access to TD Python API (op, project, app, etc). Print statements and the last expression value are captured. Best for: wiring connections (inputConnectors), setting expressions (par.X.expr/mode), querying parameter names, and batch creation scripts (5+ operators). For creating 1-4 operators, prefer td_create_operator instead.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `code` | string | yes | Python code to execute in TouchDesigner |
|
||||
|
||||
## Network & Structure
|
||||
|
||||
### td_get_network
|
||||
|
||||
Get the operator network structure in TouchDesigner (TD) at a given path. Returns compact list: name OPType flags. First line is full path of queried op. Flags: ch:N=children count, !cook=allowCooking off, bypass, private=isPrivate, blocked:reason, "comment text". depth=0 (default) = current level only. depth=1 = one level of children (indented). To explore deeper, call again on a specific COMP path. System operators (/ui, /sys) are hidden by default.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | no | Network path to inspect, e.g. '/' or '/project1' |
|
||||
| `depth` | integer | no | How many levels deep to recurse. 0=current level only (recommended), 1=include direct children of COMPs |
|
||||
| `includeSystem` | boolean | no | Include system operators (/ui, /sys). Default false. |
|
||||
| `nodeXY` | boolean | no | Include nodeX,nodeY coordinates. Default false. |
|
||||
|
||||
### td_create_operator
|
||||
|
||||
Create a new operator (node) in TouchDesigner (TD). Preferred way to create operators — handles viewport positioning, viewer flag, and docked ops automatically. For batch creation (5+ ops), you may use td_execute_python with a script instead, but then call td_get_hints('construction') first for correct parameter names and layout rules. Supports all TD operator types: TOP, CHOP, SOP, DAT, COMP, MAT. If parent is omitted, creates in the currently open network at the user's viewport position. When building a container: first create baseCOMP (no parent), then create children with parent=compPath.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `type` | string | yes | Operator type, e.g. 'textDAT', 'constantCHOP', 'noiseTOP', 'transformTOP', 'baseCOMP' |
|
||||
| `parent` | string | no | Path to the parent operator. If omitted, uses the currently open network in TD. |
|
||||
| `name` | string | no | Name for the new operator (optional, TD auto-names if omitted) |
|
||||
| `parameters` | object | no | Key-value pairs of parameters to set on the created operator |
|
||||
|
||||
### td_find_op
|
||||
|
||||
Find operators by name and/or type across the project. Returns TSV: path, OPType, flags. Flags: bypass, !cook, private, blocked:reason. Use td_search to search inside code/expressions; use td_find_op to find operators themselves.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `name` | string | no | Substring to match in operator name (case-insensitive). E.g. 'noise' finds noise1, noise2, myNoise. |
|
||||
| `type` | string | no | Substring to match in OPType (case-insensitive). E.g. 'noiseTOP', 'baseCOMP', 'CHOP'. Use exact type for precision or partial for broader matches. |
|
||||
| `root` | string | no | Root operator path to search from. Default '/project1'. |
|
||||
| `max_results` | number | no | Maximum results to return. Default 50. |
|
||||
| `max_depth` | number | no | Max recursion depth from root. Default unlimited. |
|
||||
| `detail` | `basic` / `summary` | no | Result detail level. 'basic' = name/path/type (fast). 'summary' = + connections, non-default pars, expressions. Default 'basic'. |
|
||||
|
||||
### td_search
|
||||
|
||||
Search for text across all code (DAT scripts), parameter expressions, and string parameter values in the TD project. Returns TSV: path, kind (code/expression/parameter/ref), line, text. JSON when context>0. Words are OR-matched. Use quotes for exact phrases: 'GetLogin "op('login')"'. Use count_only=true to quickly check if something is referenced without fetching full results.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `query` | string | yes | Search query. Multiple words = OR (any match). Wrap in quotes for exact phrase. Example: 'GetLogin getLogin' finds either. |
|
||||
| `root` | string | no | Root operator path to search from. Default '/project1'. |
|
||||
| `scope` | `all` / `code` / `editable` / `expressions` / `parameters` | no | What to search. 'code' = DAT scripts only (fast, ~0.05s). 'editable' = only editable code (skips inherited/ref DATs). 'expressions' = parameter expressions only. 'parameters' = string parameter values only. 'all' = everything (slow, ~1.5s due to parameter scan). Default 'all'. |
|
||||
| `case_sensitive` | boolean | no | Case-sensitive matching. Default false. |
|
||||
| `max_results` | number | no | Maximum results to return. Default 50. |
|
||||
| `context` | number | no | Lines to show before/after each code match. Saves td_read_dat calls. Default 0. |
|
||||
| `count_only` | boolean | no | Return only match count, not results. Fast existence check. |
|
||||
| `max_depth` | number | no | Max recursion depth from root. Default unlimited. |
|
||||
|
||||
### td_navigate_to
|
||||
|
||||
Navigate the TouchDesigner Network Editor viewport to show a specific operator. Opens the operator's parent network and centers the view on it. Use this to show the user where a problem is, or to navigate to an operator before modifying it.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the operator to navigate to, e.g. '/project1/noise1' |
|
||||
|
||||
## Operator Inspection
|
||||
|
||||
### td_get_operator_info
|
||||
|
||||
Get information about a specific operator (node) in TouchDesigner (TD). detail='summary': connections, non-default pars, expressions, CHOP channels (compact). detail='full': all of the above PLUS every parameter with value/default/label.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Full path to the operator, e.g. '/project1/noise1' |
|
||||
| `detail` | `summary` / `full` | no | Level of detail. 'summary' = connections, expressions, non-default pars, custom pars (pulse marked), CHOP channels. 'full' = summary + all parameters. Default 'full'. |
|
||||
|
||||
### td_get_operators_info
|
||||
|
||||
Get information about multiple operators in one call. Returns an array of operator info objects. Use instead of calling td_get_operator_info multiple times.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `paths` | array | yes | Array of full operator paths, e.g. ['/project1/null1', '/project1/null2'] |
|
||||
| `detail` | `summary` / `full` | no | Level of detail. Default 'summary'. |
|
||||
|
||||
### td_get_par_info
|
||||
|
||||
Get parameter names and details for a TouchDesigner operator type. Without specific pars: returns compact list of all parameters with their names, types, and menu options. With pars: returns full details (help text, menu values, style) for specific parameters. Use this when you need to know exact parameter names before setting them.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `op_type` | string | yes | TD operator type name, e.g. 'noiseTOP', 'blurTOP', 'lfoCHOP', 'compositeTOP' |
|
||||
| `pars` | array | no | Optional list of specific parameter names to get full details for |
|
||||
|
||||
## Parameter Setting
|
||||
|
||||
### td_set_operator_pars
|
||||
|
||||
Set parameters and flags on an operator in TouchDesigner (TD). Safer than td_execute_python for simple parameter changes. Can set values, toggle bypass/viewer, without writing Python code.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the operator |
|
||||
| `parameters` | object | no | Key-value pairs of parameters to set |
|
||||
| `bypass` | boolean | no | Set bypass state of the operator (not available on COMPs) |
|
||||
| `viewer` | boolean | no | Set viewer state of the operator |
|
||||
| `allowCooking` | boolean | no | Set cooking flag on a COMP. When False, internal network stops cooking (0 CPU). COMP-only. |
|
||||
|
||||
## Data Read/Write
|
||||
|
||||
### td_read_dat
|
||||
|
||||
Read the text content of a DAT operator in TouchDesigner (TD). Returns content with line numbers. Use to read scripts, extensions, GLSL shaders, table data.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the DAT operator |
|
||||
| `start_line` | integer | no | Start line (1-based). Omit to read from beginning. |
|
||||
| `end_line` | integer | no | End line (inclusive). Omit to read to end. |
|
||||
|
||||
### td_write_dat
|
||||
|
||||
Write or patch text content of a DAT operator in TouchDesigner (TD). Can do full replacement or StrReplace-style patching (old_text -> new_text). Use for editing scripts, extensions, shaders. Does NOT reinit extensions automatically.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the DAT operator |
|
||||
| `text` | string | no | Full replacement text. Use this OR old_text+new_text, not both. |
|
||||
| `old_text` | string | no | Text to find and replace (must be unique in the DAT) |
|
||||
| `new_text` | string | no | Replacement text |
|
||||
| `replace_all` | boolean | no | If true, replaces ALL occurrences of old_text (default: false, requires unique match) |
|
||||
|
||||
### td_read_chop
|
||||
|
||||
Read CHOP channel sample data. Returns channel values as arrays. Use when you need the actual sample values (animation curves, lookup tables, waveforms), not just the summary from td_get_operator_info.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the CHOP operator |
|
||||
| `channels` | array | no | Channel names to read. Omit to read all channels. |
|
||||
| `start` | integer | no | Start sample index (0-based). Omit to read from beginning. |
|
||||
| `end` | integer | no | End sample index (inclusive). Omit to read to end. |
|
||||
|
||||
### td_read_textport
|
||||
|
||||
Read the last N lines from the TouchDesigner (TD) log/textport (console output). Use this to see errors, warnings and print output from TD.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `lines` | integer | no | Number of recent lines to return |
|
||||
|
||||
### td_clear_textport
|
||||
|
||||
Clear the MCP textport log buffer. Use this before starting a debug session or an edit-run-check loop to keep td_read_textport output focused and minimal.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
## Visual Capture
|
||||
|
||||
### td_get_screenshot
|
||||
|
||||
Get a screenshot of an operator's viewer in TouchDesigner (TD). Saves the image to a file and returns the file path. Use your file-reading tool to view the image. Shows what the operator looks like in its viewer (TOP output, CHOP waveform graph, SOP geometry, DAT table, parameter UI, etc). Use this to visually inspect any operator, or to generate images via TD for use in your project. TWO-STEP ASYNC USAGE: Step 1 — call with 'path' to start: returns {'status': 'pending', 'requestId': '...'}. Step 2 — call with 'request_id' to retrieve: returns {'file': '/tmp/.../opname_id.jpg'}. Then read the file to see the image. If step 2 still returns pending, make one other tool call then retry.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | no | Full operator path to screenshot, e.g. '/project1/noise1'. Required for step 1. |
|
||||
| `request_id` | string | no | Request ID from step 1 to retrieve the completed screenshot. |
|
||||
| `max_size` | integer | no | Max pixel size for the longer side (default 512). Use 0 for original operator resolution (useful for pixel-accurate UI work). Higher values (e.g. 1024) for more detail. |
|
||||
| `output_path` | string | no | Optional absolute path where the image should be saved (e.g. '/Users/me/project/render.png'). If omitted, saved to /tmp/pisang_mcp/screenshots/. Use absolute paths — TD's working directory may differ from the agent's. |
|
||||
| `as_top` | boolean | no | If true, captures the operator directly as a TOP (bypasses the viewer renderer), preserving alpha/transparency. Only works for TOP operators — if the target is not a TOP, falls back to the viewer automatically. Use this when you need a clean PNG with alpha, e.g. to save a generated image for use in another project. |
|
||||
| `format` | `auto` / `jpg` / `png` | no | Image format. 'auto' (default): JPEG for viewer mode, PNG for as_top=true. 'jpg': always JPEG (smaller). 'png': always PNG (lossless). |
|
||||
|
||||
### td_get_screenshots
|
||||
|
||||
Get screenshots of multiple operators in one batch. Saves images to files and returns file paths. Use your file-reading tool to view images. TWO-STEP ASYNC USAGE: Step 1 — call with 'paths' array to start: returns {'status': 'pending', 'batchId': '...', 'total': N}. Step 2 — call with 'batch_id' to retrieve: returns {'files': [{op, file}, ...]}. Then read the files to see the images. If still processing returns {'status': 'pending', 'ready': K, 'total': N}.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `paths` | array | no | List of full operator paths to screenshot. Required for step 1. |
|
||||
| `batch_id` | string | no | Batch ID from step 1 to retrieve completed screenshots. |
|
||||
| `max_size` | integer | no | Max pixel size for longer side (default 512). Use 0 for original resolution. |
|
||||
| `as_top` | boolean | no | If true, captures TOP operators directly (preserves alpha). Non-TOP operators fall back to viewer. |
|
||||
| `output_dir` | string | no | Optional absolute path to a directory. Each screenshot saved as <opname>.jpg or .png inside it and kept on disk. |
|
||||
| `format` | `auto` / `jpg` / `png` | no | Image format. 'auto' (default): JPEG for viewer mode, PNG for as_top=true. 'jpg': always JPEG (smaller). 'png': always PNG (lossless). |
|
||||
|
||||
### td_get_screen_screenshot
|
||||
|
||||
Capture a screenshot of the actual screen via TD's screenGrabTOP. Saves the image to a file and returns the file path. Use your file-reading tool to view the image. Unlike td_get_screenshot (operator viewer), this shows what the user literally sees on their monitor — TD windows, UI panels, everything. Use when simulating mouse/keyboard input to verify what happened on screen. Workflow: td_get_screen_screenshot → read file → td_input_execute → wait idle → td_get_screen_screenshot again. TWO-STEP ASYNC: Step 1 — call without request_id: returns {'status':'pending','requestId':'...'}. Step 2 — call with request_id: returns {'file': '/tmp/.../screen_id.jpg', 'info': '...metadata...'}. Then read the file to see the image. The requestId also stays usable with td_screen_point_to_global for later coordinate lookup. crop_x/y/w/h are in ACTUAL SCREEN PIXELS (not image pixels). Crops exceeding screen bounds are auto-clamped. SMART DEFAULTS: max_size is auto when omitted — 1920 for full screen (good overview), max(crop_w,crop_h) for cropped (guarantees 1:1 scale). At 1:1 scale: screen_coord = crop_origin + image_pixel. Otherwise use the formula from metadata.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `request_id` | string | no | Request ID from step 1 to retrieve the completed screenshot. |
|
||||
| `max_size` | integer | no | Max pixel size for the longer side. Auto when omitted: 1920 for full screen, max(crop_w,crop_h) for cropped (1:1). Set explicitly to override. |
|
||||
| `crop_x` | integer | no | Left edge in screen pixels. |
|
||||
| `crop_y` | integer | no | Top edge in screen pixels (y=0 at top of screen). |
|
||||
| `crop_w` | integer | no | Width in pixels. |
|
||||
| `crop_h` | integer | no | Height in pixels. |
|
||||
| `display` | integer | no | Screen index (default 0 = primary display). |
|
||||
|
||||
## Context & Focus
|
||||
|
||||
### td_get_focus
|
||||
|
||||
Get the current user focus in TouchDesigner (TD): which network is open, selected operators, current operator, and rollover (what is under the mouse cursor). IMPORTANT: when the user says 'this operator' or 'вот этот', they mean the SELECTED/CURRENT operator, NOT the rollover. Rollover is just incidental mouse position and should be ignored for intent. Pass screenshots=true to immediately start a screenshot batch for all selected operators — response includes a 'screenshots' field with batchId; retrieve with td_get_screenshots(batch_id=...).
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `screenshots` | boolean | no | If true, start a screenshot batch for all selected operators. Retrieve with td_get_screenshots(batch_id=...). |
|
||||
| `max_size` | integer | no | Max screenshot size when screenshots=true (default 512). |
|
||||
| `as_top` | boolean | no | Passed to the screenshot batch when screenshots=true. |
|
||||
|
||||
### td_get_errors
|
||||
|
||||
Find errors and warnings in TouchDesigner (TD) operators. Checks operator errors, warnings, AND broken parameter expressions (missing channels, bad references, etc). Also includes recent script errors from the log (tracebacks), grouped and deduplicated — e.g. 1000 identical mouse-move errors shown as ×1000 with one entry. If path is given, checks that operator and its children. If no path, checks the currently open network. Use '/' for entire project. Use when user says something is broken, has errors, red nodes, горит ошибка, etc. TIP: call td_clear_textport before reproducing an error to keep log focused. TIP: combine with td_get_perf when user says 'тупит/лагает' to check both errors and performance.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | no | Path to check. If omitted, checks the current network. Use '/' to scan entire project. |
|
||||
| `recursive` | boolean | no | Check children recursively (default true) |
|
||||
| `include_log` | boolean | no | Include recent script errors from log, grouped by unique signature (default true). Use td_clear_textport before reproducing an error to keep results focused. |
|
||||
|
||||
### td_get_perf
|
||||
|
||||
Get performance data from TouchDesigner (TD). Returns TSV: header with fps/budget/memory summary, then slowest operators sorted by cook time. Columns: path, OPType, cpu/cook(ms), gpu/cook(ms), cpu/s, gpu/s, rate, flags. Use when user reports lag, low FPS, slow performance, тупит, тормозит.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | no | Path to profile. If omitted, profiles the current network. Use '/' for entire project. |
|
||||
| `top` | integer | no | Number of slowest operators to return |
|
||||
|
||||
## Documentation
|
||||
|
||||
### td_get_docs
|
||||
|
||||
Get comprehensive documentation on a TouchDesigner topic. Unlike td_get_hints (compact tips), this returns in-depth reference material. Call without arguments to see available topics with descriptions. Call with a topic name to get the full documentation.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `topic` | string | no | Topic to get docs for. Omit to list available topics. |
|
||||
|
||||
### td_get_hints
|
||||
|
||||
Get TouchDesigner tips and common patterns for a topic. Call this BEFORE creating operators or writing TD Python code to learn correct parameter names, expressions, and idiomatic approaches. Available topics: animation, noise, connections, parameters, scripting, construction, ui_analysis, panel_layout, screenshots, input_simulation, undo. IMPORTANT: always call with topic='construction' before building multi-operator setups to get correct TOP/CHOP parameter names, compositeTOP input ordering, and layout guidelines. IMPORTANT: always call with topic='input_simulation' before using td_input_execute to learn focus recovery, coordinate systems, and testing workflow.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `topic` | string | yes | Topic to get hints for. Available: 'animation', 'noise', 'connections', 'parameters', 'scripting', 'construction', 'ui_analysis', 'panel_layout', 'screenshots', 'input_simulation', 'undo', 'networking', 'all' |
|
||||
|
||||
### td_agents_md
|
||||
|
||||
Read, write, or update the agents_md documentation inside a COMP container. agents_md is a Markdown textDAT describing the container's purpose, structure, and conventions. action='read': returns content + staleness check (compares documented children vs live state). action='update': refreshes auto-generated sections (children list, connections) from live state, preserves human-written sections. action='write': sets full content, creates the DAT if missing.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the COMP container |
|
||||
| `action` | `read` / `update` / `write` | yes | read=get content+staleness, update=refresh auto sections, write=set content |
|
||||
| `content` | string | no | Markdown content (only for action='write') |
|
||||
|
||||
## Input Automation
|
||||
|
||||
### td_input_execute
|
||||
|
||||
Send a sequence of mouse/keyboard commands to TouchDesigner. Commands execute sequentially with smooth bezier movement. Returns immediately — poll td_input_status() until status='idle' before proceeding. Command types: 'focus' — bring TD to foreground. 'move' — smooth mouse move: {type,x,y,duration,easing}. 'click' — click: {type,x,y,button,hold,duration,easing}. hold=seconds to hold down. duration=smooth move before click. 'dblclick' — double click: {type,x,y,duration}. 'mousedown'/'mouseup' — {type,x,y,button}. 'key' — keystroke: {type,keys} e.g. 'ctrl+z','tab','escape','shift+f5'. Requires Accessibility permission on Mac. 'type' — human-like typing: {type,text,wpm,variance} — layout-independent Unicode, variable timing. 'wait' — pause: {type,duration}. 'scroll' — {type,x,y,dx,dy,steps} — human-like scroll: moves mouse to (x,y) first, then sends dy (vertical, +up) and dx (horizontal, +right) as multiple ticks with natural timing. steps=4 by default. Mouse commands may include coord_space='logical' (default) or coord_space='physical'. On macOS, 'physical' means actual screen pixels from td_get_screen_screenshot and is converted to CGEvent logical coords automatically. Top-level coord_space applies to commands that do not override it. on_error: 'stop' (default) clears queue on error; 'continue' skips failed command. IMPORTANT: call td_get_hints('input_simulation') before first use to learn focus recovery, coordinate systems, and testing workflow.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `commands` | array | yes | List of command dicts to execute in sequence. |
|
||||
| `coord_space` | `logical` / `physical` | no | Default coordinate space for mouse commands that do not specify their own coord_space. 'logical' uses CGEvent coords directly. 'physical' uses actual screen pixels from td_get_screen_screenshot and is auto-converted on macOS. |
|
||||
| `on_error` | `stop` / `continue` | no | What to do on error. Default 'stop'. |
|
||||
|
||||
### td_input_status
|
||||
|
||||
Get current status of the td_input command queue. Poll this after td_input_execute until status='idle'. Returns: status ('idle'/'running'), current command, queue_remaining, last error.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
### td_input_clear
|
||||
|
||||
Clear the td_input command queue and stop current execution immediately.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
### td_op_screen_rect
|
||||
|
||||
Get the screen coordinates of an operator node in the network editor. Returns {x,y,w,h,cx,cy} where cx,cy is the center for clicking. Use this to find where to click on a specific operator. Only works if the operator's parent network is currently open in a network editor pane.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Full path to the operator, e.g. '/project1/myComp/noise1' |
|
||||
|
||||
### td_click_screen_point
|
||||
|
||||
Resolve a point inside a previous td_get_screen_screenshot result and click it. Pass the screenshot request_id plus either normalized u/v or image_x/image_y. Queues a td_input click using physical screen coordinates, so it works directly with screenshot-derived points. Use duration/easing to control the cursor travel before the click.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `request_id` | string | yes | Request ID originally returned by td_get_screen_screenshot. |
|
||||
| `u` | number | no | Normalized horizontal position inside the screenshot region (0=left, 1=right). Use with v. |
|
||||
| `v` | number | no | Normalized vertical position inside the screenshot region (0=top, 1=bottom). Use with u. |
|
||||
| `image_x` | number | no | Horizontal pixel coordinate inside the returned screenshot image. Use with image_y. |
|
||||
| `image_y` | number | no | Vertical pixel coordinate inside the returned screenshot image. Use with image_x. |
|
||||
| `button` | `left` / `right` / `middle` | no | Mouse button to click. Default left. |
|
||||
| `hold` | number | no | Seconds to hold the mouse button down before releasing. |
|
||||
| `duration` | number | no | Seconds for the cursor to travel to the target before clicking. |
|
||||
| `easing` | `linear` / `ease-in` / `ease-out` / `ease-in-out` | no | Cursor movement easing for the pre-click travel. |
|
||||
| `focus` | boolean | no | If true, bring TD to the front before clicking and wait briefly for focus to settle. |
|
||||
|
||||
### td_screen_point_to_global
|
||||
|
||||
Convert a point inside a previous td_get_screen_screenshot result into absolute screen coordinates. Pass the screenshot request_id plus either normalized u/v (0..1 inside that screenshot region) or image_x/image_y in returned image pixels. Returns absolute physical screen coordinates, logical coordinates, and a ready-to-use td_input_execute payload. Metadata is kept for the most recent screen screenshots so multiple agents can resolve points later by request_id.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `request_id` | string | yes | Request ID originally returned by td_get_screen_screenshot. |
|
||||
| `u` | number | no | Normalized horizontal position inside the screenshot region (0=left, 1=right). Use with v. |
|
||||
| `v` | number | no | Normalized vertical position inside the screenshot region (0=top, 1=bottom). Use with u. |
|
||||
| `image_x` | number | no | Horizontal pixel coordinate inside the returned screenshot image. Use with image_y. |
|
||||
| `image_y` | number | no | Vertical pixel coordinate inside the returned screenshot image. Use with image_x. |
|
||||
|
||||
## System
|
||||
|
||||
### td_list_instances
|
||||
|
||||
List all running TouchDesigner (TD) instances with active MCP servers. Returns port, project name, PID, and instanceId for each instance. Call this at the start of every conversation to discover available instances and choose which one to work with. instanceId is stable for the lifetime of a TD process and is used as target_instance in all other tool calls.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
### td_project_quit
|
||||
|
||||
Save and/or close the current TouchDesigner (TD) project. Can save before closing. Reports if project has unsaved changes. To close a different instance, pass target_instance=instanceId. WARNING: this will shut down the MCP server on that instance.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `save` | boolean | no | Save the project before closing. Default true. |
|
||||
| `force` | boolean | no | Force close without save dialog. Default false. |
|
||||
|
||||
### td_reinit_extension
|
||||
|
||||
Reinitialize an extension on a COMP in TouchDesigner (TD). Call this AFTER finishing all code edits via td_write_dat to apply changes. Do NOT call after every small edit - batch your changes first.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the COMP with the extension |
|
||||
|
||||
### td_dev_log
|
||||
|
||||
Read the last N entries from the MCP dev log. Only available when Devmode is enabled. Shows request/response history.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `count` | integer | no | Number of recent log entries to return |
|
||||
|
||||
### td_clear_dev_log
|
||||
|
||||
Clear the current MCP dev log by closing the old file and starting a fresh one. Only available when Devmode is enabled.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
### td_test_session
|
||||
|
||||
Manage test sessions, bug reports, and conversation export. IMPORTANT: Do NOT proactively suggest exporting chat or submitting reports. These are tools for specific situations: - export_chat / submit_report: ONLY when the user encounters a BUG with the plugin or TouchDesigner and wants to report it, or when the user explicitly asks to export the conversation. Never suggest this at session end or as routine action. USER PHRASES → ACTIONS: 'разбор тестовых сессий' / 'analyze test sessions' → list, then pull, read meta.json → index.jsonl → calls/. 'разбор репортов' / 'analyze user reports' → list with session='user', then pull by name. 'экспортируй чат' / 'export chat' → (1) export_chat_id → marker, (2) export_chat with session=marker. 'сообщи о проблеме' / 'report bug' → export chat, review for privacy, then submit_report with summary + tags + result_op=file_path. ACTIONS: export_chat_id | export_chat | submit_report | start | note | import_chat | end | list | pull. list: default=auto-detect repo. session='user' for user_reports (dev only). pull: auto-searches both repos. Auto-detects dev vs user Hub access.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `action` | `export_chat_id` / `export_chat` / `submit_report` / `start` / `note` / `import_chat` / `end` / `list` / `pull` | yes | Action: export_chat_id / export_chat / submit_report / start / note / import_chat / end / list / pull |
|
||||
| `prompt` | string | no | (start) The test prompt/task description |
|
||||
| `tags` | array | no | (start) Tags for categorization, e.g. ['ui', 'layout'] |
|
||||
| `text` | string | no | (note) Observation text. (import_chat) Full conversation text. |
|
||||
| `outcome` | `success` / `partial` / `failure` | no | (end) Result: success / partial / failure |
|
||||
| `summary` | string | no | (end) Brief summary of what happened |
|
||||
| `result_op` | string | no | (end) Path to operator to save as result.tox |
|
||||
| `session` | string | no | (pull) Session name or substring to download |
|
||||
@@ -1,966 +0,0 @@
|
||||
# TouchDesigner Network Patterns
|
||||
|
||||
Complete network recipes for common creative coding tasks. Each pattern shows the operator chain, MCP tool calls to build it, and key parameter settings.
|
||||
|
||||
## Audio-Reactive Visuals
|
||||
|
||||
### Pattern 1: Audio Spectrum -> Noise Displacement
|
||||
|
||||
Audio drives noise parameters for organic, music-responsive textures.
|
||||
|
||||
```
|
||||
Audio File In CHOP -> Audio Spectrum CHOP -> Math CHOP (scale)
|
||||
|
|
||||
v (export to noise params)
|
||||
Noise TOP -> Level TOP -> Feedback TOP -> Composite TOP -> Null TOP (out)
|
||||
^ |
|
||||
|________________|
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="audiofileinChop", name="audio_in")
|
||||
2. td_create_operator(parent="/project1", type="audiospectrumChop", name="spectrum")
|
||||
3. td_create_operator(parent="/project1", type="mathChop", name="spectrum_scale")
|
||||
4. td_create_operator(parent="/project1", type="noiseTop", name="noise1")
|
||||
5. td_create_operator(parent="/project1", type="levelTop", name="level1")
|
||||
6. td_create_operator(parent="/project1", type="feedbackTop", name="feedback1")
|
||||
7. td_create_operator(parent="/project1", type="compositeTop", name="comp1")
|
||||
8. td_create_operator(parent="/project1", type="nullTop", name="out")
|
||||
|
||||
9. td_set_operator_pars(path="/project1/audio_in",
|
||||
properties={"file": "/path/to/music.wav", "play": true})
|
||||
10. td_set_operator_pars(path="/project1/spectrum",
|
||||
properties={"size": 512})
|
||||
11. td_set_operator_pars(path="/project1/spectrum_scale",
|
||||
properties={"gain": 2.0, "postoff": 0.0})
|
||||
12. td_set_operator_pars(path="/project1/noise1",
|
||||
properties={"type": 1, "monochrome": false, "resolutionw": 1280, "resolutionh": 720,
|
||||
"period": 4.0, "harmonics": 3, "amp": 1.0})
|
||||
13. td_set_operator_pars(path="/project1/level1",
|
||||
properties={"opacity": 0.95, "gamma1": 0.75})
|
||||
14. td_set_operator_pars(path="/project1/feedback1",
|
||||
properties={"top": "/project1/comp1"})
|
||||
15. td_set_operator_pars(path="/project1/comp1",
|
||||
properties={"operand": 0})
|
||||
|
||||
16. td_execute_python: """
|
||||
op('/project1/audio_in').outputConnectors[0].connect(op('/project1/spectrum'))
|
||||
op('/project1/spectrum').outputConnectors[0].connect(op('/project1/spectrum_scale'))
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))
|
||||
op('/project1/level1').outputConnectors[0].connect(op('/project1/comp1').inputConnectors[0])
|
||||
op('/project1/feedback1').outputConnectors[0].connect(op('/project1/comp1').inputConnectors[1])
|
||||
op('/project1/comp1').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
|
||||
17. td_execute_python: """
|
||||
# Export spectrum values to drive noise parameters
|
||||
# This makes the noise react to audio frequencies
|
||||
op('/project1/noise1').par.seed.expr = "op('/project1/spectrum_scale')['chan1']"
|
||||
op('/project1/noise1').par.period.expr = "tdu.remap(op('/project1/spectrum_scale')['chan1'].eval(), 0, 1, 1, 8)"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 2: Beat Detection -> Visual Pulses
|
||||
|
||||
Detect beats from audio and trigger visual events.
|
||||
|
||||
```
|
||||
Audio Device In CHOP -> Audio Spectrum CHOP -> Math CHOP (isolate bass)
|
||||
|
|
||||
Trigger CHOP (envelope)
|
||||
|
|
||||
[export to visual params]
|
||||
```
|
||||
|
||||
**Key parameter settings:**
|
||||
|
||||
```
|
||||
# Isolate bass frequencies (20-200 Hz)
|
||||
Math CHOP: chanop=1 (Add channels), range1low=0, range1high=10
|
||||
(first 10 FFT bins = bass frequencies with 512 FFT at 44100Hz)
|
||||
|
||||
# ADSR envelope on each beat
|
||||
Trigger CHOP: attack=0.02, peak=1.0, decay=0.3, sustain=0.0, release=0.1
|
||||
|
||||
# Export to visual: Scale, brightness, or color intensity
|
||||
td_execute_python: "op('/project1/level1').par.brightness1.expr = \"1.0 + op('/project1/trigger1')['chan1'] * 0.5\""
|
||||
```
|
||||
|
||||
### Pattern 3: Multi-Band Audio -> Multi-Layer Visuals
|
||||
|
||||
Split audio into frequency bands, drive different visual layers per band.
|
||||
|
||||
```
|
||||
Audio In -> Spectrum -> Audio Band EQ (3 bands: bass, mid, treble)
|
||||
|
|
||||
+---------+---------+
|
||||
| | |
|
||||
Bass Mids Treble
|
||||
| | |
|
||||
Noise TOP Circle TOP Text TOP
|
||||
(slow,dark) (mid,warm) (fast,bright)
|
||||
| | |
|
||||
+-----+----+----+----+
|
||||
| |
|
||||
Composite Composite
|
||||
|
|
||||
Out
|
||||
```
|
||||
|
||||
### Pattern 3b: Audio-Reactive GLSL Fractal (Proven Recipe)
|
||||
|
||||
Complete working recipe. Plays an MP3, runs FFT, feeds spectrum as a texture into a GLSL shader where inner fractal reacts to bass, outer to treble.
|
||||
|
||||
**Network:**
|
||||
```
|
||||
AudioFileIn CHOP → AudioSpectrum CHOP (FFT=512, outlength=256)
|
||||
→ Math CHOP (gain=10) → CHOP To TOP (256x2 spectrum texture, dataformat=r)
|
||||
↓
|
||||
Constant TOP (time, rgba32float) → GLSL TOP (input 0=time, input 1=spectrum) → Null → MovieFileOut
|
||||
↓
|
||||
AudioFileIn CHOP → Audio Device Out CHOP Record to .mov
|
||||
```
|
||||
|
||||
**Build via td_execute_python (one call per step for reliability):**
|
||||
|
||||
```python
|
||||
# Step 1: Audio chain
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
root = op('/project1')
|
||||
audio = root.create(audiofileinCHOP, 'audio_in')
|
||||
audio.par.file = '/path/to/music.mp3'
|
||||
audio.par.playmode = 0 # Locked to timeline
|
||||
audio.par.volume = 0.5
|
||||
|
||||
spec = root.create(audiospectrumCHOP, 'spectrum')
|
||||
audio.outputConnectors[0].connect(spec.inputConnectors[0])
|
||||
|
||||
math_n = root.create(mathCHOP, 'math_norm')
|
||||
spec.outputConnectors[0].connect(math_n.inputConnectors[0])
|
||||
math_n.par.gain = 5 # boost signal
|
||||
|
||||
resamp = root.create(resampleCHOP, 'resample_spec')
|
||||
math_n.outputConnectors[0].connect(resamp.inputConnectors[0])
|
||||
resamp.par.timeslice = True
|
||||
resamp.par.rate = 256
|
||||
|
||||
chop2top = root.create(choptoTOP, 'spectrum_tex')
|
||||
chop2top.par.chop = resamp # CHOP To TOP has NO input connectors — use par.chop reference
|
||||
|
||||
# Audio output (hear the music)
|
||||
aout = root.create(audiodeviceoutCHOP, 'audio_out')
|
||||
audio.outputConnectors[0].connect(aout.inputConnectors[0])
|
||||
result = 'audio chain ok'
|
||||
""")
|
||||
|
||||
# Step 2: Time driver (MUST be rgba32float — see pitfalls #6)
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
root = op('/project1')
|
||||
td = root.create(constantTOP, 'time_driver')
|
||||
td.par.format = 'rgba32float'
|
||||
td.par.outputresolution = 'custom'
|
||||
td.par.resolutionw = 1
|
||||
td.par.resolutionh = 1
|
||||
td.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
td.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
result = 'time ok'
|
||||
""")
|
||||
|
||||
# Step 3: GLSL shader (write to /tmp, load from file)
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
root = op('/project1')
|
||||
glsl = root.create(glslTOP, 'audio_shader')
|
||||
glsl.par.outputresolution = 'custom'
|
||||
glsl.par.resolutionw = 1280
|
||||
glsl.par.resolutionh = 720
|
||||
|
||||
sd = root.create(textDAT, 'shader_code')
|
||||
sd.text = open('/tmp/my_shader.glsl').read()
|
||||
glsl.par.pixeldat = sd
|
||||
|
||||
# Wire: input 0 = time, input 1 = spectrum texture
|
||||
op('/project1/time_driver').outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
op('/project1/spectrum_tex').outputConnectors[0].connect(glsl.inputConnectors[1])
|
||||
result = 'glsl ok'
|
||||
""")
|
||||
|
||||
# Step 4: Output + recorder
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
root = op('/project1')
|
||||
out = root.create(nullTOP, 'output')
|
||||
op('/project1/audio_shader').outputConnectors[0].connect(out.inputConnectors[0])
|
||||
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
out.outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'mjpa'
|
||||
result = 'output ok'
|
||||
""")
|
||||
```
|
||||
|
||||
**GLSL shader pattern (audio-reactive fractal):**
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
|
||||
vec3 palette(float t) {
|
||||
vec3 a = vec3(0.5); vec3 b = vec3(0.5);
|
||||
vec3 c = vec3(1.0); vec3 d = vec3(0.263, 0.416, 0.557);
|
||||
return a + b * cos(6.28318 * (c * t + d));
|
||||
}
|
||||
|
||||
void main() {
|
||||
// Input 0 = time (1x1 rgba32float constant)
|
||||
// Input 1 = audio spectrum (256x2 CHOP To TOP, stereo — sample at y=0.25 for first channel)
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
vec2 uv = (gl_FragCoord.xy * 2.0 - res) / min(res.x, res.y);
|
||||
vec2 uv0 = uv;
|
||||
vec3 finalColor = vec3(0.0);
|
||||
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r;
|
||||
float mids = texture(sTD2DInputs[1], vec2(0.25, 0.25)).r;
|
||||
|
||||
for (float i = 0.0; i < 4.0; i++) {
|
||||
uv = fract(uv * (1.4 + bass * 0.3)) - 0.5;
|
||||
float d = length(uv) * exp(-length(uv0));
|
||||
|
||||
// Sample spectrum at distance: inner=bass, outer=treble
|
||||
float freq = texture(sTD2DInputs[1], vec2(clamp(d * 0.5, 0.0, 1.0), 0.25)).r;
|
||||
|
||||
vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35);
|
||||
d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0;
|
||||
d = abs(d);
|
||||
d = pow(0.012 / d, 1.2 + freq * 0.8 + bass * 0.5);
|
||||
finalColor += col * d;
|
||||
}
|
||||
|
||||
// Tone mapping
|
||||
finalColor = finalColor / (finalColor + vec3(1.0));
|
||||
fragColor = TDOutputSwizzle(vec4(finalColor, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
**Key insights from testing:**
|
||||
- `spectrum_tex` (CHOP To TOP) produces a 256x2 texture — x position = frequency, y=0.25 for first channel
|
||||
- Sampling at `vec2(0.05, 0.0)` gets bass, `vec2(0.65, 0.0)` gets treble
|
||||
- Sampling based on pixel distance (`d * 0.5`) makes inner fractal react to bass, outer to treble
|
||||
- `bass * 0.3` in the `fract()` zoom makes the fractal breathe with kicks
|
||||
- Math CHOP gain of 5 is needed because raw spectrum values are very small
|
||||
|
||||
## Generative Art
|
||||
|
||||
### Pattern 4: Feedback Loop with Transform
|
||||
|
||||
Classic generative technique — texture evolves through recursive transformation.
|
||||
|
||||
```
|
||||
Noise TOP -> Composite TOP -> Level TOP -> Null TOP (out)
|
||||
^ |
|
||||
| v
|
||||
Transform TOP <- Feedback TOP
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="noiseTop", name="seed_noise")
|
||||
2. td_create_operator(parent="/project1", type="compositeTop", name="mix")
|
||||
3. td_create_operator(parent="/project1", type="transformTop", name="evolve")
|
||||
4. td_create_operator(parent="/project1", type="feedbackTop", name="fb")
|
||||
5. td_create_operator(parent="/project1", type="levelTop", name="color_correct")
|
||||
6. td_create_operator(parent="/project1", type="nullTop", name="out")
|
||||
|
||||
7. td_set_operator_pars(path="/project1/seed_noise",
|
||||
properties={"type": 1, "monochrome": false, "period": 2.0, "amp": 0.3,
|
||||
"resolutionw": 1280, "resolutionh": 720})
|
||||
8. td_set_operator_pars(path="/project1/mix",
|
||||
properties={"operand": 27}) # 27 = Screen blend
|
||||
9. td_set_operator_pars(path="/project1/evolve",
|
||||
properties={"sx": 1.003, "sy": 1.003, "rz": 0.5, "extend": 2}) # slight zoom + rotate, repeat edges
|
||||
10. td_set_operator_pars(path="/project1/fb",
|
||||
properties={"top": "/project1/mix"})
|
||||
11. td_set_operator_pars(path="/project1/color_correct",
|
||||
properties={"opacity": 0.98, "gamma1": 0.85})
|
||||
|
||||
12. td_execute_python: """
|
||||
op('/project1/seed_noise').outputConnectors[0].connect(op('/project1/mix').inputConnectors[0])
|
||||
op('/project1/fb').outputConnectors[0].connect(op('/project1/evolve'))
|
||||
op('/project1/evolve').outputConnectors[0].connect(op('/project1/mix').inputConnectors[1])
|
||||
op('/project1/mix').outputConnectors[0].connect(op('/project1/color_correct'))
|
||||
op('/project1/color_correct').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
```
|
||||
|
||||
**Variations:**
|
||||
- Change Transform: `rz` (rotation), `sx/sy` (zoom), `tx/ty` (drift)
|
||||
- Change Composite operand: Screen (glow), Add (bright), Multiply (dark)
|
||||
- Add HSV Adjust in the feedback loop for color evolution
|
||||
- Add Blur for dreamlike softness
|
||||
- Replace Noise with a GLSL TOP for custom seed patterns
|
||||
|
||||
### Pattern 5: Instancing (Particle-Like Systems)
|
||||
|
||||
Render thousands of copies of geometry, each with unique position/rotation/scale driven by CHOP data or DATs.
|
||||
|
||||
```
|
||||
Table DAT (instance data) -> DAT to CHOP -> Geometry COMP (instancing on) -> Render TOP
|
||||
+ Sphere SOP (template geometry)
|
||||
+ Constant MAT (material)
|
||||
+ Camera COMP
|
||||
+ Light COMP
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="tableDat", name="instance_data")
|
||||
2. td_create_operator(parent="/project1", type="geometryComp", name="geo1")
|
||||
3. td_create_operator(parent="/project1/geo1", type="sphereSop", name="sphere")
|
||||
4. td_create_operator(parent="/project1", type="constMat", name="mat1")
|
||||
5. td_create_operator(parent="/project1", type="cameraComp", name="cam1")
|
||||
6. td_create_operator(parent="/project1", type="lightComp", name="light1")
|
||||
7. td_create_operator(parent="/project1", type="renderTop", name="render1")
|
||||
|
||||
8. td_execute_python: """
|
||||
import random, math
|
||||
dat = op('/project1/instance_data')
|
||||
dat.clear()
|
||||
dat.appendRow(['tx', 'ty', 'tz', 'sx', 'sy', 'sz', 'cr', 'cg', 'cb'])
|
||||
for i in range(500):
|
||||
angle = i * 0.1
|
||||
r = 2 + i * 0.01
|
||||
dat.appendRow([
|
||||
str(math.cos(angle) * r),
|
||||
str(math.sin(angle) * r),
|
||||
str((i - 250) * 0.02),
|
||||
'0.05', '0.05', '0.05',
|
||||
str(random.random()),
|
||||
str(random.random()),
|
||||
str(random.random())
|
||||
])
|
||||
"""
|
||||
|
||||
9. td_set_operator_pars(path="/project1/geo1",
|
||||
properties={"instancing": true, "instancechop": "",
|
||||
"instancedat": "/project1/instance_data",
|
||||
"material": "/project1/mat1"})
|
||||
10. td_set_operator_pars(path="/project1/render1",
|
||||
properties={"camera": "/project1/cam1", "geometry": "/project1/geo1",
|
||||
"light": "/project1/light1",
|
||||
"resolutionw": 1280, "resolutionh": 720})
|
||||
11. td_set_operator_pars(path="/project1/cam1",
|
||||
properties={"tz": 10})
|
||||
```
|
||||
|
||||
### Pattern 6: Reaction-Diffusion (GLSL)
|
||||
|
||||
Classic Gray-Scott reaction-diffusion system running on the GPU.
|
||||
|
||||
```
|
||||
Text DAT (GLSL code) -> GLSL TOP (resolution, dat reference) -> Feedback TOP
|
||||
^ |
|
||||
|_______________________________________|
|
||||
Level TOP (out)
|
||||
```
|
||||
|
||||
**Key GLSL code (write to Text DAT via td_execute_python):**
|
||||
|
||||
```glsl
|
||||
// Gray-Scott reaction-diffusion
|
||||
uniform float feed; // 0.037
|
||||
uniform float kill; // 0.06
|
||||
uniform float dA; // 1.0
|
||||
uniform float dB; // 0.5
|
||||
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
vec2 texel = 1.0 / uTDOutputInfo.res.zw;
|
||||
|
||||
vec4 c = texture(sTD2DInputs[0], uv);
|
||||
float a = c.r;
|
||||
float b = c.g;
|
||||
|
||||
// Laplacian (9-point stencil)
|
||||
float lA = 0.0, lB = 0.0;
|
||||
for(int dx = -1; dx <= 1; dx++) {
|
||||
for(int dy = -1; dy <= 1; dy++) {
|
||||
float w = (dx == 0 && dy == 0) ? -1.0 : (abs(dx) + abs(dy) == 1 ? 0.2 : 0.05);
|
||||
vec4 s = texture(sTD2DInputs[0], uv + vec2(dx, dy) * texel);
|
||||
lA += s.r * w;
|
||||
lB += s.g * w;
|
||||
}
|
||||
}
|
||||
|
||||
float reaction = a * b * b;
|
||||
float newA = a + (dA * lA - reaction + feed * (1.0 - a));
|
||||
float newB = b + (dB * lB + reaction - (kill + feed) * b);
|
||||
|
||||
fragColor = vec4(clamp(newA, 0.0, 1.0), clamp(newB, 0.0, 1.0), 0.0, 1.0);
|
||||
}
|
||||
```
|
||||
|
||||
## Video Processing
|
||||
|
||||
### Pattern 7: Video Effects Chain
|
||||
|
||||
Apply a chain of effects to a video file.
|
||||
|
||||
```
|
||||
Movie File In TOP -> HSV Adjust TOP -> Level TOP -> Blur TOP -> Composite TOP -> Null TOP (out)
|
||||
^
|
||||
Text TOP ---+
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="moviefileinTop", name="video_in")
|
||||
2. td_create_operator(parent="/project1", type="hsvadjustTop", name="color")
|
||||
3. td_create_operator(parent="/project1", type="levelTop", name="levels")
|
||||
4. td_create_operator(parent="/project1", type="blurTop", name="blur")
|
||||
5. td_create_operator(parent="/project1", type="compositeTop", name="overlay")
|
||||
6. td_create_operator(parent="/project1", type="textTop", name="title")
|
||||
7. td_create_operator(parent="/project1", type="nullTop", name="out")
|
||||
|
||||
8. td_set_operator_pars(path="/project1/video_in",
|
||||
properties={"file": "/path/to/video.mp4", "play": true})
|
||||
9. td_set_operator_pars(path="/project1/color",
|
||||
properties={"hueoffset": 0.1, "saturationmult": 1.3})
|
||||
10. td_set_operator_pars(path="/project1/levels",
|
||||
properties={"brightness1": 1.1, "contrast": 1.2, "gamma1": 0.9})
|
||||
11. td_set_operator_pars(path="/project1/blur",
|
||||
properties={"sizex": 2, "sizey": 2})
|
||||
12. td_set_operator_pars(path="/project1/title",
|
||||
properties={"text": "My Video", "fontsizex": 48, "alignx": 1, "aligny": 1})
|
||||
|
||||
13. td_execute_python: """
|
||||
chain = ['video_in', 'color', 'levels', 'blur']
|
||||
for i in range(len(chain) - 1):
|
||||
op(f'/project1/{chain[i]}').outputConnectors[0].connect(op(f'/project1/{chain[i+1]}'))
|
||||
op('/project1/blur').outputConnectors[0].connect(op('/project1/overlay').inputConnectors[0])
|
||||
op('/project1/title').outputConnectors[0].connect(op('/project1/overlay').inputConnectors[1])
|
||||
op('/project1/overlay').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 8: Video Recording
|
||||
|
||||
Record the output to a file. **H.264/H.265 require a Commercial license** — use Motion JPEG (`mjpa`) on Non-Commercial.
|
||||
|
||||
```
|
||||
[any TOP chain] -> Null TOP -> Movie File Out TOP
|
||||
```
|
||||
|
||||
```python
|
||||
# Build via td_execute_python:
|
||||
root = op('/project1')
|
||||
|
||||
# Always put a Null TOP before the recorder
|
||||
null_out = root.op('out') # or create one
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
null_out.outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial
|
||||
|
||||
# Start recording (par.record is a toggle — .record() method may not exist)
|
||||
rec.par.record = True
|
||||
# ... let TD run for desired duration ...
|
||||
rec.par.record = False
|
||||
|
||||
# For image sequences:
|
||||
# rec.par.type = 'imagesequence'
|
||||
# rec.par.imagefiletype = 'png'
|
||||
# rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" # fileSuffix REQUIRED
|
||||
```
|
||||
|
||||
**Pitfalls:**
|
||||
- Setting `par.file` + `par.record = True` in the same script may race — use `run("...", delayFrames=2)`
|
||||
- `TOP.save()` called rapidly always captures the same frame — use MovieFileOut for animation
|
||||
- See `pitfalls.md` #25-27 for full details
|
||||
|
||||
### Pattern 8b: TD → External Pipeline (FFmpeg / Python / Post-Processing)
|
||||
|
||||
Export TD visuals for use in another tool (ffmpeg, Python, ASCII art, etc.). This is the standard workflow when you need to composite TD output with external processing (ASCII conversion, Python shader chains, ML inference, etc.).
|
||||
|
||||
**Step 1: Record to video in TD**
|
||||
|
||||
```python
|
||||
# Preferred: ProRes on macOS (lossless, Non-Commercial OK, ~55MB/s at 1280x720)
|
||||
rec.par.videocodec = 'prores'
|
||||
# Fallback for non-macOS: mjpa (Motion JPEG)
|
||||
# rec.par.videocodec = 'mjpa'
|
||||
rec.par.record = True
|
||||
# ... wait N seconds ...
|
||||
rec.par.record = False
|
||||
```
|
||||
|
||||
**Step 2: Extract frames with ffmpeg**
|
||||
|
||||
```bash
|
||||
# Extract all frames at 30fps
|
||||
ffmpeg -y -i /tmp/output.mov -vf 'fps=30' /tmp/frames/frame_%06d.png
|
||||
|
||||
# Or extract a specific duration
|
||||
ffmpeg -y -i /tmp/output.mov -t 25 -vf 'fps=30' /tmp/frames/frame_%06d.png
|
||||
|
||||
# Or extract specific frame range
|
||||
ffmpeg -y -i /tmp/output.mov -vf 'select=between(n\,0\,749)' -vsync vfr /tmp/frames/frame_%06d.png
|
||||
```
|
||||
|
||||
**Step 3: Process frames in Python**
|
||||
|
||||
```python
|
||||
from PIL import Image
|
||||
import os
|
||||
|
||||
frames_dir = '/tmp/frames'
|
||||
output_dir = '/tmp/processed'
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
for fname in sorted(os.listdir(frames_dir)):
|
||||
if not fname.endswith('.png'):
|
||||
continue
|
||||
img = Image.open(os.path.join(frames_dir, fname))
|
||||
# ... apply your processing ...
|
||||
img.save(os.path.join(output_dir, fname))
|
||||
```
|
||||
|
||||
**Step 4: Mux processed frames back with audio**
|
||||
|
||||
```bash
|
||||
# Create video from processed frames + audio with fade-out
|
||||
ffmpeg -y \
|
||||
-framerate 30 -i /tmp/processed/frame_%06d.png \
|
||||
-i /tmp/audio.mp3 \
|
||||
-c:v libx264 -pix_fmt yuv420p -crf 18 \
|
||||
-c:a aac -b:a 192k \
|
||||
-shortest \
|
||||
-af 'afade=t=out:st=23:d=2' \
|
||||
/tmp/final_output.mp4
|
||||
```
|
||||
|
||||
**Key considerations:**
|
||||
- Use ProRes for the TD recording step to avoid generation loss during compositing
|
||||
- Extract at the target output framerate (not TD's render framerate)
|
||||
- For audio-synced content, analyze the audio file separately in Python (scipy FFT) to get per-frame features (rms, spectral bands, beats) and drive compositing parameters
|
||||
- Always verify TD FPS > 0 before recording (see pitfalls #37, #38)
|
||||
|
||||
## Data Visualization
|
||||
|
||||
### Pattern 9: Table Data -> Bar Chart via Instancing
|
||||
|
||||
Visualize tabular data as a 3D bar chart.
|
||||
|
||||
```
|
||||
Table DAT (data) -> Script DAT (transform to instance format) -> DAT to CHOP
|
||||
|
|
||||
Box SOP -> Geometry COMP (instancing from CHOP) -> Render TOP -> Null TOP (out)
|
||||
+ PBR MAT
|
||||
+ Camera COMP
|
||||
+ Light COMP
|
||||
```
|
||||
|
||||
```python
|
||||
# Script DAT code to transform data to instance positions
|
||||
td_execute_python: """
|
||||
source = op('/project1/data_table')
|
||||
instance = op('/project1/instance_transform')
|
||||
instance.clear()
|
||||
instance.appendRow(['tx', 'ty', 'tz', 'sx', 'sy', 'sz', 'cr', 'cg', 'cb'])
|
||||
|
||||
for i in range(1, source.numRows):
|
||||
value = float(source[i, 'value'])
|
||||
name = source[i, 'name']
|
||||
instance.appendRow([
|
||||
str(i * 1.5), # x position (spread bars)
|
||||
str(value / 2), # y position (center bar vertically)
|
||||
'0', # z position
|
||||
'1', str(value), '1', # scale (height = data value)
|
||||
'0.2', '0.6', '1.0' # color (blue)
|
||||
])
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 9b: Audio-Reactive GLSL Fractal (Proven Recipe)
|
||||
|
||||
Audio spectrum drives a GLSL fractal shader directly via a spectrum texture input. Bass thickens inner fractal lines, mids twist rotation, highs light outer edges. **Always run discovery (SKILL.md Step 0) before using any param names from these recipes — they may differ in your TD version.**
|
||||
|
||||
```
|
||||
Audio File In CHOP → Audio Spectrum CHOP (FFT=512, outlength=256)
|
||||
→ Math CHOP (gain=10)
|
||||
→ CHOP To TOP (spectrum texture, 256x2, dataformat=r)
|
||||
↓ (input 1)
|
||||
Constant TOP (rgba32float, time) → GLSL TOP (audio-reactive shader) → Null TOP
|
||||
(input 0) ↑
|
||||
Text DAT (shader code)
|
||||
```
|
||||
|
||||
**Build via td_execute_python (complete working script):**
|
||||
|
||||
```python
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
import os
|
||||
root = op('/project1')
|
||||
|
||||
# Audio input
|
||||
audio = root.create(audiofileinCHOP, 'audio_in')
|
||||
audio.par.file = '/path/to/music.mp3'
|
||||
audio.par.playmode = 0 # Locked to timeline
|
||||
|
||||
# FFT analysis (output length manually set to 256 bins)
|
||||
spectrum = root.create(audiospectrumCHOP, 'spectrum')
|
||||
audio.outputConnectors[0].connect(spectrum.inputConnectors[0])
|
||||
spectrum.par.fftsize = '512'
|
||||
spectrum.par.outputmenu = 'setmanually'
|
||||
spectrum.par.outlength = 256
|
||||
|
||||
# THEN boost gain on the raw spectrum (NO Lag CHOP — see pitfall #34)
|
||||
math = root.create(mathCHOP, 'math_norm')
|
||||
spectrum.outputConnectors[0].connect(math.inputConnectors[0])
|
||||
math.par.gain = 10
|
||||
|
||||
# Spectrum → texture (256x2 image — stereo, sample at y=0.25 for first channel)
|
||||
# NOTE: choptoTOP has NO input connectors — use par.chop reference!
|
||||
spec_tex = root.create(choptoTOP, 'spectrum_tex')
|
||||
spec_tex.par.chop = math
|
||||
spec_tex.par.dataformat = 'r'
|
||||
spec_tex.par.layout = 'rowscropped'
|
||||
|
||||
# Time driver (rgba32float to avoid 0-1 clamping!)
|
||||
time_drv = root.create(constantTOP, 'time_driver')
|
||||
time_drv.par.format = 'rgba32float'
|
||||
time_drv.par.outputresolution = 'custom'
|
||||
time_drv.par.resolutionw = 1
|
||||
time_drv.par.resolutionh = 1
|
||||
time_drv.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
time_drv.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
|
||||
# GLSL shader
|
||||
glsl = root.create(glslTOP, 'audio_shader')
|
||||
glsl.par.outputresolution = 'custom'
|
||||
glsl.par.resolutionw = 1280; glsl.par.resolutionh = 720
|
||||
|
||||
shader_dat = root.create(textDAT, 'shader_code')
|
||||
shader_dat.text = open('/tmp/shader.glsl').read()
|
||||
glsl.par.pixeldat = shader_dat
|
||||
|
||||
# Wire: input 0=time, input 1=spectrum
|
||||
time_drv.outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
spec_tex.outputConnectors[0].connect(glsl.inputConnectors[1])
|
||||
|
||||
# Output + audio playback
|
||||
out = root.create(nullTOP, 'output')
|
||||
glsl.outputConnectors[0].connect(out.inputConnectors[0])
|
||||
audio_out = root.create(audiodeviceoutCHOP, 'audio_out')
|
||||
audio.outputConnectors[0].connect(audio_out.inputConnectors[0])
|
||||
|
||||
result = 'network built'
|
||||
""")
|
||||
```
|
||||
|
||||
**GLSL shader (reads spectrum from input 1 texture):**
|
||||
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
|
||||
vec3 palette(float t) {
|
||||
vec3 a = vec3(0.5); vec3 b = vec3(0.5);
|
||||
vec3 c = vec3(1.0); vec3 d = vec3(0.263, 0.416, 0.557);
|
||||
return a + b * cos(6.28318 * (c * t + d));
|
||||
}
|
||||
|
||||
void main() {
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
vec2 uv = (gl_FragCoord.xy * 2.0 - res) / min(res.x, res.y);
|
||||
vec2 uv0 = uv;
|
||||
vec3 finalColor = vec3(0.0);
|
||||
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r;
|
||||
float mids = texture(sTD2DInputs[1], vec2(0.25, 0.25)).r;
|
||||
float highs = texture(sTD2DInputs[1], vec2(0.65, 0.25)).r;
|
||||
|
||||
float ca = cos(t * (0.15 + mids * 0.3));
|
||||
float sa = sin(t * (0.15 + mids * 0.3));
|
||||
uv = mat2(ca, -sa, sa, ca) * uv;
|
||||
|
||||
for (float i = 0.0; i < 4.0; i++) {
|
||||
uv = fract(uv * (1.4 + bass * 0.3)) - 0.5;
|
||||
float d = length(uv) * exp(-length(uv0));
|
||||
float freq = texture(sTD2DInputs[1], vec2(clamp(d*0.5, 0.0, 1.0), 0.25)).r;
|
||||
vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35);
|
||||
d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0;
|
||||
d = abs(d);
|
||||
d = pow(0.012 / d, 1.2 + freq * 0.8 + bass * 0.5);
|
||||
finalColor += col * d;
|
||||
}
|
||||
|
||||
float glow = (0.03 + bass * 0.05) / (length(uv0) + 0.03);
|
||||
finalColor += vec3(0.4, 0.1, 0.7) * glow * (0.6 + 0.4 * sin(t * 2.5));
|
||||
|
||||
float ring = abs(length(uv0) - 0.4 - mids * 0.3);
|
||||
finalColor += vec3(0.1, 0.6, 0.8) * (0.005 / ring) * (0.2 + highs * 0.5);
|
||||
|
||||
finalColor *= smoothstep(0.0, 1.0, 1.0 - dot(uv0*0.55, uv0*0.55));
|
||||
finalColor = finalColor / (finalColor + vec3(1.0));
|
||||
|
||||
fragColor = TDOutputSwizzle(vec4(finalColor, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
**How spectrum sampling drives the visual:**
|
||||
- `texture(sTD2DInputs[1], vec2(x, 0.0)).r` — x position = frequency (0=bass, 1=treble)
|
||||
- Inner fractal iterations sample lower x → react to bass
|
||||
- Outer iterations sample higher x → react to treble
|
||||
- `bass * 0.3` on `fract()` scale → fractal zoom pulses with bass
|
||||
- `bass * 4.0` on sin frequency → line density pulses with bass
|
||||
- `mids * 0.3` on rotation speed → spiral twists faster during vocal/mid sections
|
||||
- `highs * 0.5` on ring opacity → high-frequency sparkle on outer ring
|
||||
|
||||
**Recording the output:** Use MovieFileOut TOP with `mjpa` codec (H.264 requires Commercial license). See pitfalls #25-27.
|
||||
|
||||
## GLSL Shaders
|
||||
|
||||
### Pattern 10: Custom Fragment Shader
|
||||
|
||||
Write a custom visual effect as a GLSL fragment shader.
|
||||
|
||||
```
|
||||
Text DAT (shader code) -> GLSL TOP -> Level TOP -> Null TOP (out)
|
||||
+ optional input TOPs for texture sampling
|
||||
```
|
||||
|
||||
**Common GLSL uniforms available in TouchDesigner:**
|
||||
|
||||
```glsl
|
||||
// Automatically provided by TD
|
||||
uniform vec4 uTDOutputInfo; // .res.zw = resolution
|
||||
|
||||
// NOTE: uTDCurrentTime does NOT exist in TD 099!
|
||||
// Feed time via a 1x1 Constant TOP (format=rgba32float):
|
||||
// t.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
// t.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
// Then read in GLSL:
|
||||
// vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
// float t = td.r + td.g * 1000.0;
|
||||
|
||||
// Input textures (from connected TOP inputs)
|
||||
uniform sampler2D sTD2DInputs[1]; // array of input samplers
|
||||
|
||||
// From vertex shader
|
||||
in vec3 vUV; // UV coordinates (0-1 range)
|
||||
```
|
||||
|
||||
**Example: Plasma shader (using time from input texture)**
|
||||
|
||||
```glsl
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
// Read time from Constant TOP input 0 (rgba32float format)
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
float v1 = sin(uv.x * 10.0 + t);
|
||||
float v2 = sin(uv.y * 10.0 + t * 0.7);
|
||||
float v3 = sin((uv.x + uv.y) * 10.0 + t * 1.3);
|
||||
float v4 = sin(length(uv - 0.5) * 20.0 - t * 2.0);
|
||||
|
||||
float v = (v1 + v2 + v3 + v4) * 0.25;
|
||||
|
||||
vec3 color = vec3(
|
||||
sin(v * 3.14159 + 0.0) * 0.5 + 0.5,
|
||||
sin(v * 3.14159 + 2.094) * 0.5 + 0.5,
|
||||
sin(v * 3.14159 + 4.189) * 0.5 + 0.5
|
||||
);
|
||||
|
||||
fragColor = vec4(color, 1.0);
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 11: Multi-Pass GLSL (Ping-Pong)
|
||||
|
||||
For effects needing state across frames (particles, fluid, cellular automata), use GLSL Multi TOP with multiple passes or a Feedback TOP loop.
|
||||
|
||||
```
|
||||
GLSL Multi TOP (pass 0: simulation, pass 1: rendering)
|
||||
+ Text DAT (simulation shader)
|
||||
+ Text DAT (render shader)
|
||||
-> Level TOP -> Null TOP (out)
|
||||
^
|
||||
|__ Feedback TOP (feeds simulation state back)
|
||||
```
|
||||
|
||||
## Interactive Installations
|
||||
|
||||
### Pattern 12: Mouse/Touch -> Visual Response
|
||||
|
||||
```
|
||||
Mouse In CHOP -> Math CHOP (normalize to 0-1) -> [export to visual params]
|
||||
|
||||
# Or for touch/multi-touch:
|
||||
Multi Touch In DAT -> Script CHOP (parse touches) -> [export to visual params]
|
||||
```
|
||||
|
||||
```python
|
||||
# Normalize mouse position to 0-1 range
|
||||
td_execute_python: """
|
||||
op('/project1/noise1').par.offsetx.expr = "op('/project1/mouse_norm')['tx']"
|
||||
op('/project1/noise1').par.offsety.expr = "op('/project1/mouse_norm')['ty']"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 13: OSC Control (from external software)
|
||||
|
||||
```
|
||||
OSC In CHOP (port 7000) -> Select CHOP (pick channels) -> [export to visual params]
|
||||
```
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="oscinChop", name="osc_in")
|
||||
2. td_set_operator_pars(path="/project1/osc_in", properties={"port": 7000})
|
||||
|
||||
# OSC messages like /frequency 440 will appear as channel "frequency" with value 440
|
||||
# Export to any parameter:
|
||||
3. td_execute_python: "op('/project1/noise1').par.period.expr = \"op('/project1/osc_in')['frequency']\""
|
||||
```
|
||||
|
||||
### Pattern 14: MIDI Control (DJ/VJ)
|
||||
|
||||
```
|
||||
MIDI In CHOP (device) -> Select CHOP -> [export channels to visual params]
|
||||
```
|
||||
|
||||
Common MIDI mappings:
|
||||
- CC channels (knobs/faders): continuous 0-127, map to float params
|
||||
- Note On/Off: binary triggers, map to Trigger CHOP for envelopes
|
||||
- Velocity: intensity/brightness
|
||||
|
||||
## Live Performance
|
||||
|
||||
### Pattern 15: Multi-Source VJ Setup
|
||||
|
||||
```
|
||||
Source A (generative) ----+
|
||||
Source B (video) ---------+-- Switch/Cross TOP -- Level TOP -- Window COMP (output)
|
||||
Source C (camera) --------+
|
||||
^
|
||||
MIDI/OSC control selects active source and crossfade
|
||||
```
|
||||
|
||||
```python
|
||||
# MIDI CC1 controls which source is active (0-127 -> 0-2)
|
||||
td_execute_python: """
|
||||
op('/project1/switch1').par.index.expr = "int(op('/project1/midi_in')['cc1'] / 42)"
|
||||
"""
|
||||
|
||||
# MIDI CC2 controls crossfade between current and next
|
||||
td_execute_python: """
|
||||
op('/project1/cross1').par.cross.expr = "op('/project1/midi_in')['cc2'] / 127.0"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 16: Projection Mapping
|
||||
|
||||
```
|
||||
Content TOPs ----+
|
||||
|
|
||||
Stoner TOP (UV mapping) -> Composite TOP -> Window COMP (projector output)
|
||||
or
|
||||
Kantan Mapper COMP (external .tox)
|
||||
```
|
||||
|
||||
For projection mapping, the key is:
|
||||
1. Create your visual content as standard TOPs
|
||||
2. Use Stoner TOP or a third-party mapping tool to UV-map content to physical surfaces
|
||||
3. Output via Window COMP to the projector
|
||||
|
||||
### Pattern 17: Cue System
|
||||
|
||||
```
|
||||
Table DAT (cue list: cue_number, scene_name, duration, transition_type)
|
||||
|
|
||||
Script CHOP (cue state: current_cue, progress, next_cue_trigger)
|
||||
|
|
||||
[export to Switch/Cross TOPs to transition between scenes]
|
||||
```
|
||||
|
||||
```python
|
||||
td_execute_python: """
|
||||
# Simple cue system
|
||||
cue_table = op('/project1/cue_list')
|
||||
cue_state = op('/project1/cue_state')
|
||||
|
||||
def advance_cue():
|
||||
current = int(cue_state.par.value0.val)
|
||||
next_cue = min(current + 1, cue_table.numRows - 1)
|
||||
cue_state.par.value0.val = next_cue
|
||||
|
||||
scene = cue_table[next_cue, 'scene']
|
||||
duration = float(cue_table[next_cue, 'duration'])
|
||||
|
||||
# Set crossfade target and duration
|
||||
op('/project1/cross1').par.cross.val = 0
|
||||
# Animate cross to 1.0 over duration seconds
|
||||
# (use a Timer CHOP or LFO CHOP for smooth animation)
|
||||
"""
|
||||
```
|
||||
|
||||
## Networking
|
||||
|
||||
### Pattern 18: OSC Server/Client
|
||||
|
||||
```
|
||||
# Sending OSC
|
||||
OSC Out CHOP -> (network) -> external application
|
||||
|
||||
# Receiving OSC
|
||||
(network) -> OSC In CHOP -> Select CHOP -> [use values]
|
||||
```
|
||||
|
||||
### Pattern 19: NDI Video Streaming
|
||||
|
||||
```
|
||||
# Send video over network
|
||||
[any TOP chain] -> NDI Out TOP (source name)
|
||||
|
||||
# Receive video from network
|
||||
NDI In TOP (select source) -> [process as normal TOP]
|
||||
```
|
||||
|
||||
### Pattern 20: WebSocket Communication
|
||||
|
||||
```
|
||||
WebSocket DAT -> Script DAT (parse JSON messages) -> [update visuals]
|
||||
```
|
||||
|
||||
```python
|
||||
td_execute_python: """
|
||||
ws = op('/project1/websocket1')
|
||||
ws.par.address = 'ws://localhost:8080'
|
||||
ws.par.active = True
|
||||
|
||||
# In a DAT Execute callback (Script DAT watching WebSocket DAT):
|
||||
# def onTableChange(dat):
|
||||
# import json
|
||||
# msg = json.loads(dat.text)
|
||||
# op('/project1/noise1').par.seed.val = msg.get('seed', 0)
|
||||
"""
|
||||
```
|
||||
@@ -1,239 +0,0 @@
|
||||
# TouchDesigner Operator Reference
|
||||
|
||||
## Operator Families Overview
|
||||
|
||||
TouchDesigner has 6 operator families. Each family processes a specific data type and is color-coded in the UI. Operators can only connect to others of the SAME family (with cross-family converters as the bridge).
|
||||
|
||||
## TOPs — Texture Operators (Purple)
|
||||
|
||||
2D image/texture processing on the GPU. The workhorse of visual output.
|
||||
|
||||
### Generators (create images from nothing)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Noise TOP | `noiseTop` | `type` (0-6), `monochrome`, `seed`, `period`, `harmonics`, `exponent`, `amp`, `offset`, `resolutionw/h` | Procedural noise textures — Perlin, Simplex, Sparse, etc. Foundation of generative art. |
|
||||
| Constant TOP | `constantTop` | `colorr/g/b/a`, `resolutionw/h` | Solid color. Use as background or blend input. |
|
||||
| Text TOP | `textTop` | `text`, `fontsizex`, `fontfile`, `alignx/y`, `colorr/g/b` | Render text to texture. Supports multi-line, word wrap. |
|
||||
| Ramp TOP | `rampTop` | `type` (0=horizontal, 1=vertical, 2=radial, 3=circular), `phase`, `period` | Gradient textures for masking, color mapping. |
|
||||
| Circle TOP | `circleTop` | `radiusx/y`, `centerx/y`, `width` | Circles, rings, ellipses. |
|
||||
| Rectangle TOP | `rectangleTop` | `sizex/y`, `centerx/y`, `softness` | Rectangles with optional softness. |
|
||||
| GLSL TOP | `glslTop` | `dat` (points to shader DAT), `resolutionw/h`, `outputformat`, custom uniforms | Custom fragment shaders. Most powerful TOP for custom visuals. |
|
||||
| GLSL Multi TOP | `glslmultiTop` | `dat`, `numinputs`, `numoutputs`, `numcomputepasses` | Multi-pass GLSL with compute shaders. Advanced. |
|
||||
| Render TOP | `renderTop` | `camera`, `geometry`, `lights`, `resolutionw/h` | Renders 3D scenes (SOPs + MATs + Camera/Light COMPs). |
|
||||
|
||||
### Filters (modify a single input)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Level TOP | `levelTop` | `opacity`, `brightness1/2`, `gamma1/2`, `contrast`, `invert`, `blacklevel/whitelevel` | Brightness, contrast, gamma, levels. Essential color correction. |
|
||||
| Blur TOP | `blurTop` | `sizex/y`, `type` (0=Gaussian, 1=Box, 2=Bartlett) | Gaussian/box blur. |
|
||||
| Transform TOP | `transformTop` | `tx/ty`, `sx/sy`, `rz`, `pivotx/y`, `extend` (0=Hold, 1=Zero, 2=Repeat, 3=Mirror) | Translate, scale, rotate textures. |
|
||||
| HSV Adjust TOP | `hsvadjustTop` | `hueoffset`, `saturationmult`, `valuemult` | HSV color adjustments. |
|
||||
| Lookup TOP | `lookupTop` | (input: texture + lookup table) | Color remapping via lookup table texture. |
|
||||
| Edge TOP | `edgeTop` | `type` (0=Sobel, 1=Frei-Chen) | Edge detection. |
|
||||
| Displace TOP | `displaceTop` | `scalex/y` | Pixel displacement using a second input as displacement map. |
|
||||
| Flip TOP | `flipTop` | `flipx`, `flipy`, `flop` (diagonal) | Mirror/flip textures. |
|
||||
| Crop TOP | `cropTop` | `cropleft/right/top/bottom` | Crop region of texture. |
|
||||
| Resolution TOP | `resolutionTop` | `resolutionw/h`, `outputresolution` | Resize textures. |
|
||||
| Null TOP | `nullTop` | (none significant) | Pass-through. Use for organization, referencing, feedback delay. |
|
||||
| Cache TOP | `cacheTop` | `length`, `step` | Store N frames of history. Useful for trails, time effects. |
|
||||
|
||||
### Compositors (combine multiple inputs)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Composite TOP | `compositeTop` | `operand` (0-31: Over, Add, Multiply, Screen, etc.) | Blend two textures with standard compositing modes. |
|
||||
| Over TOP | `overTop` | (simple alpha compositing) | Layer with alpha. Simpler than Composite. |
|
||||
| Add TOP | `addTop` | (additive blend) | Additive blending. Great for glow, light effects. |
|
||||
| Multiply TOP | `multiplyTop` | (multiplicative blend) | Multiply blend. Good for masking, darkening. |
|
||||
| Switch TOP | `switchTop` | `index` (0-based) | Switch between multiple inputs by index. |
|
||||
| Cross TOP | `crossTop` | `cross` (0.0-1.0) | Crossfade between two inputs. |
|
||||
|
||||
### I/O (input/output)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Movie File In TOP | `moviefileinTop` | `file`, `speed`, `trim`, `index` | Load video files, image sequences. |
|
||||
| Movie File Out TOP | `moviefileoutTop` | `file`, `type` (codec), `record` (toggle) | Record/export video files. |
|
||||
| NDI In TOP | `ndiinTop` | `sourcename` | Receive NDI video streams. |
|
||||
| NDI Out TOP | `ndioutTop` | `sourcename` | Send NDI video streams. |
|
||||
| Syphon Spout In/Out TOP | `syphonspoutinTop` / `syphonspoutoutTop` | `servername` | Inter-app texture sharing. |
|
||||
| Video Device In TOP | `videodeviceinTop` | `device` | Webcam/capture card input. |
|
||||
| Feedback TOP | `feedbackTop` | `top` (path to the TOP to feed back) | One-frame delay feedback. Essential for recursive effects. |
|
||||
|
||||
### Converters
|
||||
|
||||
| Operator | Type Name | Direction | Use |
|
||||
|----------|-----------|-----------|-----|
|
||||
| CHOP to TOP | `choptopTop` | CHOP -> TOP | Visualize channel data as texture (waveform, spectrum display). |
|
||||
| TOP to CHOP | `topchopChop` | TOP -> CHOP | Sample texture pixels as channel data. |
|
||||
|
||||
## CHOPs — Channel Operators (Green)
|
||||
|
||||
Time-varying numeric data: audio, animation curves, sensor data, control signals.
|
||||
|
||||
### Generators
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Constant CHOP | `constantChop` | `name0/value0`, `name1/value1`... | Static named channels. Control panel for parameters. |
|
||||
| LFO CHOP | `lfoChop` | `frequency`, `type` (0=Sin, 1=Tri, 2=Square, 3=Ramp, 4=Pulse), `amp`, `offset`, `phase` | Low frequency oscillator. Animation driver. |
|
||||
| Noise CHOP | `noiseChop` | `type`, `roughness`, `period`, `amp`, `seed`, `channels` | Smooth random motion. Organic animation. |
|
||||
| Pattern CHOP | `patternChop` | `type` (0=Sine, 1=Triangle, ...), `length`, `cycles` | Generate waveform patterns. |
|
||||
| Timer CHOP | `timerChop` | `length`, `play`, `cue`, `cycles` | Countdown/count-up timer with cue points. |
|
||||
| Count CHOP | `countChop` | `threshold`, `limittype`, `limitmin/max` | Event counter with wrapping/clamping. |
|
||||
|
||||
### Audio
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Audio File In CHOP | `audiofileinChop` | `file`, `volume`, `play`, `speed`, `trim` | Play audio files. |
|
||||
| Audio Device In CHOP | `audiodeviceinChop` | `device`, `channels` | Live microphone/line input. |
|
||||
| Audio Spectrum CHOP | `audiospectrumChop` | `size` (FFT size), `outputformat` (0=Power, 1=Magnitude) | FFT frequency analysis. |
|
||||
| Audio Band EQ CHOP | `audiobandeqChop` | `bands`, `gaindb` per band | Frequency band isolation. |
|
||||
| Audio Device Out CHOP | `audiodeviceoutChop` | `device` | Audio playback output. |
|
||||
|
||||
### Math/Logic
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Math CHOP | `mathChop` | `preoff`, `gain`, `postoff`, `chanop` (0=Off, 1=Add, 2=Subtract, 3=Multiply...) | Math operations on channels. The Swiss army knife. |
|
||||
| Logic CHOP | `logicChop` | `preop` (0=Off, 1=AND, 2=OR, 3=XOR, 4=NAND), `convert` | Boolean logic on channels. |
|
||||
| Filter CHOP | `filterChop` | `type` (0=Low Pass, 1=Band Pass, 2=High Pass, 3=Notch), `cutofffreq`, `filterwidth` | Smooth, dampen, filter signals. |
|
||||
| Lag CHOP | `lagChop` | `lag1/2`, `overshoot1/2` | Smooth transitions with overshoot. |
|
||||
| Limit CHOP | `limitChop` | `type` (0=Clamp, 1=Loop, 2=ZigZag), `min/max` | Clamp or wrap channel values. |
|
||||
| Speed CHOP | `speedChop` | (none significant) | Integrate values (velocity to position, acceleration to velocity). |
|
||||
| Trigger CHOP | `triggerChop` | `attack`, `peak`, `decay`, `sustain`, `release` | ADSR envelope from trigger events. |
|
||||
| Select CHOP | `selectChop` | `chop` (path), `channames` | Reference channels from another CHOP. |
|
||||
| Merge CHOP | `mergeChop` | `align` (0=Extend, 1=Trim to First, 2=Trim to Shortest) | Combine channels from multiple CHOPs. |
|
||||
| Null CHOP | `nullChop` | (none significant) | Pass-through for organization and referencing. |
|
||||
|
||||
### Input Devices
|
||||
|
||||
| Operator | Type Name | Use |
|
||||
|----------|-----------|-----|
|
||||
| Mouse In CHOP | `mouseinChop` | Mouse position, buttons, wheel. |
|
||||
| Keyboard In CHOP | `keyboardinChop` | Keyboard key states. |
|
||||
| MIDI In CHOP | `midiinChop` | MIDI note/CC input. |
|
||||
| OSC In CHOP | `oscinChop` | OSC message input (network). |
|
||||
|
||||
## SOPs — Surface Operators (Blue)
|
||||
|
||||
3D geometry: points, polygons, NURBS, meshes.
|
||||
|
||||
### Generators
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Grid SOP | `gridSop` | `rows`, `cols`, `sizex/y`, `type` (0=Polygon, 1=Mesh, 2=NURBS) | Flat grid mesh. Foundation for displacement, instancing. |
|
||||
| Sphere SOP | `sphereSop` | `type`, `rows`, `cols`, `radius` | Sphere geometry. |
|
||||
| Box SOP | `boxSop` | `sizex/y/z` | Box geometry. |
|
||||
| Torus SOP | `torusSop` | `radiusx/y`, `rows`, `cols` | Donut shape. |
|
||||
| Circle SOP | `circleSop` | `type`, `radius`, `divs` | Circle/ring geometry. |
|
||||
| Line SOP | `lineSop` | `dist`, `points` | Line segments. |
|
||||
| Text SOP | `textSop` | `text`, `fontsizex`, `fontfile`, `extrude` | 3D text geometry. |
|
||||
|
||||
### Modifiers
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Transform SOP | `transformSop` | `tx/ty/tz`, `rx/ry/rz`, `sx/sy/sz` | Transform geometry (translate, rotate, scale). |
|
||||
| Noise SOP | `noiseSop` | `type`, `amp`, `period`, `roughness` | Deform geometry with noise. |
|
||||
| Sort SOP | `sortSop` | `ptsort`, `primsort` | Reorder points/primitives. |
|
||||
| Facet SOP | `facetSop` | `unique`, `consolidate`, `computenormals` | Normals, consolidation, unique points. |
|
||||
| Merge SOP | `mergeSop` | (none significant) | Combine multiple geometry inputs. |
|
||||
| Null SOP | `nullSop` | (none significant) | Pass-through. |
|
||||
|
||||
## DATs — Data Operators (White)
|
||||
|
||||
Text, tables, scripts, network data.
|
||||
|
||||
### Core
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Table DAT | `tableDat` | (edit content directly) | Spreadsheet-like data tables. |
|
||||
| Text DAT | `textDat` | (edit content directly) | Arbitrary text content. Shader code, configs, scripts. |
|
||||
| Script DAT | `scriptDat` | `language` (0=Python, 1=C++) | Custom callbacks and DAT processing. |
|
||||
| CHOP Execute DAT | `chopexecDat` | `chop` (path to watch), callbacks | Trigger Python on CHOP value changes. |
|
||||
| DAT Execute DAT | `datexecDat` | `dat` (path to watch) | Trigger Python on DAT content changes. |
|
||||
| Panel Execute DAT | `panelexecDat` | `panel` | Trigger Python on UI panel events. |
|
||||
|
||||
### I/O
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Web DAT | `webDat` | `url`, `fetchmethod` (0=GET, 1=POST) | HTTP requests. API integration. |
|
||||
| TCP/IP DAT | `tcpipDat` | `address`, `port`, `mode` | TCP networking. |
|
||||
| OSC In DAT | `oscinDat` | `port` | Receive OSC as text messages. |
|
||||
| Serial DAT | `serialDat` | `port`, `baudrate` | Serial port communication (Arduino, etc.). |
|
||||
| File In DAT | `fileinDat` | `file` | Read text files. |
|
||||
| File Out DAT | `fileoutDat` | `file`, `write` | Write text files. |
|
||||
|
||||
### Conversions
|
||||
|
||||
| Operator | Type Name | Direction | Use |
|
||||
|----------|-----------|-----------|-----|
|
||||
| DAT to CHOP | `dattochopChop` | DAT -> CHOP | Convert table data to channels. |
|
||||
| CHOP to DAT | `choptodatDat` | CHOP -> DAT | Convert channel data to table rows. |
|
||||
| SOP to DAT | `soptodatDat` | SOP -> DAT | Extract geometry data as table. |
|
||||
|
||||
## MATs — Material Operators (Yellow)
|
||||
|
||||
Materials for 3D rendering in Render TOP / Geometry COMP.
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Phong MAT | `phongMat` | `diff_colorr/g/b`, `spec_colorr/g/b`, `shininess`, `colormap`, `normalmap` | Classic Phong shading. Simple, fast. |
|
||||
| PBR MAT | `pbrMat` | `basecolorr/g/b`, `metallic`, `roughness`, `normalmap`, `emitcolorr/g/b` | Physically-based rendering. Realistic materials. |
|
||||
| GLSL MAT | `glslMat` | `dat` (shader DAT), custom uniforms | Custom vertex + fragment shaders for 3D. |
|
||||
| Constant MAT | `constMat` | `colorr/g/b`, `colormap` | Flat unlit color/texture. No shading. |
|
||||
| Point Sprite MAT | `pointspriteMat` | `colormap`, `scale` | Render points as camera-facing sprites. Great for particles. |
|
||||
| Wireframe MAT | `wireframeMat` | `colorr/g/b`, `width` | Wireframe rendering. |
|
||||
| Depth MAT | `depthMat` | `near`, `far` | Render depth buffer as grayscale. |
|
||||
|
||||
## COMPs — Component Operators (Gray)
|
||||
|
||||
Containers, 3D scene elements, UI components.
|
||||
|
||||
### 3D Scene
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Geometry COMP | `geometryComp` | `material` (path), `instancechop` (path), `instancing` (toggle) | Renders geometry with material. Instancing host. |
|
||||
| Camera COMP | `cameraComp` | `tx/ty/tz`, `rx/ry/rz`, `fov`, `near/far` | Camera for Render TOP. |
|
||||
| Light COMP | `lightComp` | `lighttype` (0=Point, 1=Directional, 2=Spot, 3=Cone), `dimmer`, `colorr/g/b` | Lighting for 3D scenes. |
|
||||
| Ambient Light COMP | `ambientlightComp` | `dimmer`, `colorr/g/b` | Ambient lighting. |
|
||||
| Environment Light COMP | `envlightComp` | `envmap` | Image-based lighting (IBL). |
|
||||
|
||||
### Containers
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Container COMP | `containerComp` | `w`, `h`, `bgcolor1/2/3` | UI container. Holds other COMPs for panel layouts. |
|
||||
| Base COMP | `baseComp` | (none significant) | Generic container. Networks-inside-networks. |
|
||||
| Replicator COMP | `replicatorComp` | `template`, `operatorsdat` | Clone a template operator N times from a table. |
|
||||
|
||||
### Utilities
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Window COMP | `windowComp` | `winw/h`, `winoffsetx/y`, `monitor`, `borders` | Output window for display/projection. |
|
||||
| Select COMP | `selectComp` | `rowcol`, `panel` | Select and display content from elsewhere. |
|
||||
| Engine COMP | `engineComp` | `tox`, `externaltox` | Load external .tox components. Sub-process isolation. |
|
||||
|
||||
## Cross-Family Converter Summary
|
||||
|
||||
| From | To | Operator | Type Name |
|
||||
|------|-----|----------|-----------|
|
||||
| CHOP | TOP | CHOP to TOP | `choptopTop` |
|
||||
| TOP | CHOP | TOP to CHOP | `topchopChop` |
|
||||
| DAT | CHOP | DAT to CHOP | `dattochopChop` |
|
||||
| CHOP | DAT | CHOP to DAT | `choptodatDat` |
|
||||
| SOP | CHOP | SOP to CHOP | `soptochopChop` |
|
||||
| CHOP | SOP | CHOP to SOP | `choptosopSop` |
|
||||
| SOP | DAT | SOP to DAT | `soptodatDat` |
|
||||
| DAT | SOP | DAT to SOP | `dattosopSop` |
|
||||
| SOP | TOP | (use Render TOP + Geometry COMP) | — |
|
||||
| TOP | SOP | TOP to SOP | `toptosopSop` |
|
||||
@@ -1,508 +0,0 @@
|
||||
# TouchDesigner MCP — Pitfalls & Lessons Learned
|
||||
|
||||
Hard-won knowledge from real TD sessions. Read this before building anything.
|
||||
|
||||
## Parameter Names
|
||||
|
||||
### 1. NEVER hardcode parameter names — always discover
|
||||
|
||||
Parameter names change between TD versions. What works in one build may not work in another. ALWAYS use td_get_par_info to discover actual names from TD.
|
||||
|
||||
The agent's LLM training data contains WRONG parameter names. Do not trust them.
|
||||
|
||||
Known historical differences (may vary further — always verify):
|
||||
| What docs/training say | Actual in some versions | Notes |
|
||||
|---------------|---------------|-------|
|
||||
| `dat` | `pixeldat` | GLSL TOP pixel shader DAT |
|
||||
| `colora` | `alpha` | Constant TOP alpha |
|
||||
| `sizex` / `sizey` | `size` | Blur TOP (single value) |
|
||||
| `fontr/g/b/a` | `fontcolorr/g/b/a` | Text TOP font color (r/g/b) |
|
||||
| `fontcolora` | `fontalpha` | Text TOP font alpha (NOT `fontcolora`) |
|
||||
| `bgcolora` | `bgalpha` | Text TOP bg alpha |
|
||||
| `value1name` | `vec0name` | GLSL TOP uniform name |
|
||||
|
||||
### 2. twozero td_execute_python response format
|
||||
|
||||
When calling `td_execute_python` via twozero MCP, successful responses return `(ok)` followed by FPS/error summary (e.g. `[fps 60.0/60] [0 err/0 warn]`), NOT the raw Python `result` dict. If you're parsing responses programmatically, check for the `(ok)` prefix — don't pattern-match on Python variable names from the script. Use `td_get_operator_info` or separate inspection calls to read back values.
|
||||
|
||||
### 3. When using td_set_operator_pars, param names must match exactly
|
||||
|
||||
Use td_get_par_info to discover them. The MCP tool validates parameter names and returns clear errors explaining what went wrong, unlike raw Python which crashes the whole script with tdAttributeError and stops execution. Always discover before setting.
|
||||
|
||||
### 4. Use `safe_par()` pattern for cross-version compatibility
|
||||
|
||||
```python
|
||||
def safe_par(node, name, value):
|
||||
p = getattr(node.par, name, None)
|
||||
if p is not None:
|
||||
p.val = value
|
||||
return True
|
||||
return False
|
||||
```
|
||||
|
||||
### 5. `td.tdAttributeError` crashes the whole script — use defensive access
|
||||
|
||||
If you do `node.par.nonexistent = value`, TD raises `tdAttributeError` and stops the entire script. Prevention is better than catching:
|
||||
- Use `op()` instead of `opex()` — `op()` returns None on failure, `opex()` raises
|
||||
- Use `hasattr(node.par, 'name')` before accessing any parameter
|
||||
- Use `getattr(node.par, 'name', None)` with a default
|
||||
- Use the `safe_par()` pattern from pitfall #3
|
||||
|
||||
```python
|
||||
# WRONG — crashes if param doesn't exist:
|
||||
node.par.nonexistent = value
|
||||
|
||||
# CORRECT — defensive access:
|
||||
if hasattr(node.par, 'nonexistent'):
|
||||
node.par.nonexistent = value
|
||||
```
|
||||
|
||||
### 6. `outputresolution` is a string menu, not an integer
|
||||
|
||||
```
|
||||
menuNames: ['useinput','eighth','quarter','half','2x','4x','8x','fit','limit','custom','parpanel']
|
||||
```
|
||||
Always use the string form. Setting `outputresolution = 9` may silently fail.
|
||||
```python
|
||||
node.par.outputresolution = 'custom' # correct
|
||||
node.par.resolutionw = 1280; node.par.resolutionh = 720
|
||||
```
|
||||
Discover valid values: `list(node.par.outputresolution.menuNames)`
|
||||
|
||||
## GLSL Shaders
|
||||
|
||||
### 7. `uTDCurrentTime` does NOT exist in GLSL TOP
|
||||
|
||||
There is NO built-in time uniform for GLSL TOPs. GLSL MAT has `uTDGeneral.seconds` but that's NOT available in GLSL TOP context.
|
||||
|
||||
**PRIMARY — GLSL TOP Vectors/Values page:**
|
||||
```python
|
||||
gl.par.value0name = 'uTime'
|
||||
gl.par.value0.expr = "absTime.seconds"
|
||||
# In GLSL: uniform float uTime;
|
||||
```
|
||||
|
||||
**FALLBACK — Constant TOP texture (for complex time data):**
|
||||
|
||||
CRITICAL: set format to `rgba32float` — default 8-bit clamps to 0-1:
|
||||
```python
|
||||
t = root.create(constantTOP, 'time_driver')
|
||||
t.par.format = 'rgba32float'
|
||||
t.par.outputresolution = 'custom'
|
||||
t.par.resolutionw = 1; t.par.resolutionh = 1
|
||||
t.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
t.outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
```
|
||||
|
||||
### 8. GLSL compile errors are silent in the API
|
||||
|
||||
The GLSL TOP shows a yellow warning triangle in the UI but `node.errors()` may return empty string. Check `node.warnings()` too, and create an Info DAT pointed at the GLSL TOP to read the actual compiler output.
|
||||
|
||||
### 9. TD GLSL uses `vUV.st` not `gl_FragCoord` — and REQUIRES `TDOutputSwizzle()` on macOS
|
||||
|
||||
Standard GLSL patterns don't work. TD provides:
|
||||
- `vUV.st` — UV coordinates (0-1)
|
||||
- `uTDOutputInfo.res.zw` — resolution
|
||||
- `sTD2DInputs[0]` — input textures
|
||||
- `layout(location = 0) out vec4 fragColor` — output
|
||||
|
||||
CRITICAL on macOS: Always wrap output with `TDOutputSwizzle()`:
|
||||
```glsl
|
||||
fragColor = TDOutputSwizzle(color);
|
||||
```
|
||||
TD uses GLSL 4.60 (Vulkan backend). GLSL 3.30 and earlier removed.
|
||||
|
||||
### 10. Large GLSL shaders — write to temp file
|
||||
|
||||
GLSL code with special characters can corrupt JSON payloads. Write the shader to a temp file and load it in TD:
|
||||
```python
|
||||
# Agent side: write shader to /tmp/shader.glsl via write_file
|
||||
# TD side:
|
||||
sd = root.create(textDAT, 'shader_code')
|
||||
with open('/tmp/shader.glsl', 'r') as f:
|
||||
sd.text = f.read()
|
||||
```
|
||||
|
||||
## Node Management
|
||||
|
||||
### 11. Destroying nodes while iterating `root.children` causes `tdError`
|
||||
|
||||
The iterator is invalidated when a child is destroyed. Always snapshot first:
|
||||
```python
|
||||
kids = list(root.children) # snapshot
|
||||
for child in kids:
|
||||
if child.valid: # check — earlier destroys may cascade
|
||||
child.destroy()
|
||||
```
|
||||
|
||||
### 11b. Split cleanup and creation into SEPARATE td_execute_python calls
|
||||
|
||||
Creating nodes with the same names you just destroyed in the SAME script causes "Invalid OP object" errors — even with `list()` snapshot. TD's internal references can go stale within one execution context.
|
||||
|
||||
**WRONG (single call):**
|
||||
```python
|
||||
# td_execute_python:
|
||||
for c in list(root.children):
|
||||
if c.valid and c.name.startswith('promo_'):
|
||||
c.destroy()
|
||||
# ... then create promo_audio, promo_shader etc. in same script → CRASHES
|
||||
```
|
||||
|
||||
**CORRECT (two separate calls):**
|
||||
```python
|
||||
# Call 1: td_execute_python — clean only
|
||||
for c in list(root.children):
|
||||
if c.valid and c.name.startswith('promo_'):
|
||||
c.destroy()
|
||||
|
||||
# Call 2: td_execute_python — build (separate MCP call)
|
||||
audio = root.create(audiofileinCHOP, 'promo_audio')
|
||||
# ... rest of build
|
||||
```
|
||||
|
||||
### 12. Feedback TOP: use `top` parameter, NOT direct input wire
|
||||
|
||||
The feedbackTOP's `top` parameter references which TOP to delay. Do NOT also wire that TOP directly into the feedback's input — this creates a real cook dependency loop.
|
||||
|
||||
Correct setup:
|
||||
```python
|
||||
fb = root.create(feedbackTOP, 'fb_delay')
|
||||
fb.par.top = comp.path # reference only — no wire to fb input
|
||||
fb.outputConnectors[0].connect(xf) # fb output -> transform -> fade -> comp
|
||||
```
|
||||
|
||||
The "Cook dependency loop detected" warning on the transform/fade chain is expected.
|
||||
|
||||
### 13. GLSL TOP auto-creates companion nodes
|
||||
|
||||
Creating a `glslTOP` also creates `name_pixel` (Text DAT), `name_info` (Info DAT), and `name_compute` (Text DAT). These are visible in the network. Don't be alarmed by "extra" nodes.
|
||||
|
||||
### 14. The default project root is `/project1`
|
||||
|
||||
New TD files start with `/project1` as the main container. System nodes live at `/`, `/ui`, `/sys`, `/local`, `/perform`. Don't create user nodes outside `/project1`.
|
||||
|
||||
### 15. Non-Commercial license caps resolution at 1280x1280
|
||||
|
||||
Setting `resolutionw=1920` silently clamps to 1280. Always check effective resolution after creation:
|
||||
```python
|
||||
n.cook(force=True)
|
||||
actual = str(n.width) + 'x' + str(n.height)
|
||||
```
|
||||
|
||||
## Recording & Codecs
|
||||
|
||||
### 16. MovieFileOut TOP: H.264/H.265/AV1 requires Commercial license
|
||||
|
||||
In Non-Commercial TD, these codecs produce an error. Recommended alternatives:
|
||||
- `prores` — Apple ProRes, **best on macOS**, HW accelerated, NOT license-restricted. ~55MB/s at 1280x720 but lossless quality. **Use this as default on macOS.**
|
||||
- `cineform` — GoPro Cineform, supports alpha
|
||||
- `hap` — GPU-accelerated playback, large files
|
||||
- `notchlc` — GPU-accelerated, good quality
|
||||
- `mjpa` — Motion JPEG, legacy fallback (lossy, use only if ProRes unavailable)
|
||||
|
||||
For image sequences: `rec.par.type = 'imagesequence'`, `rec.par.imagefiletype = 'png'`
|
||||
|
||||
### 17. MovieFileOut `.record()` method may not exist
|
||||
|
||||
Use the toggle parameter instead:
|
||||
```python
|
||||
rec.par.record = True # start recording
|
||||
rec.par.record = False # stop recording
|
||||
```
|
||||
|
||||
When setting file path and starting recording in the same script, use delayFrames:
|
||||
```python
|
||||
rec.par.file = '/tmp/new_output.mov'
|
||||
run("op('/project1/recorder').par.record = True", delayFrames=2)
|
||||
```
|
||||
|
||||
### 18. TOP.save() captures same frame when called rapidly
|
||||
|
||||
Use MovieFileOut for real-time recording. Set `project.realTime = False` for frame-accurate output.
|
||||
|
||||
### 19. AudioFileIn CHOP: cue and recording sequence matters
|
||||
|
||||
The recording sequence must be done in exact order, or the recording will be empty, audio will start mid-file, or the file won't be written.
|
||||
|
||||
**Proven recording sequence:**
|
||||
|
||||
```python
|
||||
# Step 1: Stop any existing recording
|
||||
rec.par.record = False
|
||||
|
||||
# Step 2: Reset audio to beginning
|
||||
audio.par.play = False
|
||||
audio.par.cue = True
|
||||
audio.par.cuepoint = 0 # may need cuepointunit=0 too
|
||||
# Verify: audio.par.cue.eval() should be True
|
||||
|
||||
# Step 3: Set output file path
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
|
||||
# Step 4: Release cue + start playing + start recording (with frame delay)
|
||||
audio.par.cue = False
|
||||
audio.par.play = True
|
||||
audio.par.playmode = 2 # Sequential — plays once through
|
||||
run("op('/project1/recorder').par.record = True", delayFrames=3)
|
||||
```
|
||||
|
||||
**Why each step matters:**
|
||||
- `rec.par.record = False` first — if a previous recording is active, setting `par.file` may fail silently
|
||||
- `audio.par.cue = True` + `cuepoint = 0` — guarantees audio starts from the beginning, otherwise the spectrum may be silent for the first few seconds
|
||||
- `delayFrames=3` on the record start — setting `par.file` and `par.record = True` in the same script can race; the file path needs a frame to register before recording starts
|
||||
- `playmode = 2` (Sequential) — plays the file once. Use `playmode = 0` (Locked to Timeline) if you want TD's timeline to control position
|
||||
|
||||
## TD Python API Patterns
|
||||
|
||||
### 20. COMP extension setup: ext0object format is CRITICAL
|
||||
|
||||
`ext0object` expects a CONSTANT string (NOT expression mode):
|
||||
```python
|
||||
comp.par.ext0object = "op('./myExtensionDat').module.MyClassName(me)"
|
||||
```
|
||||
NEVER set as just the DAT name. NEVER use ParMode.EXPRESSION. ALWAYS ensure the DAT has `par.language='python'`.
|
||||
|
||||
### 21. td.Panel is NOT subscriptable — use attribute access
|
||||
|
||||
```python
|
||||
comp.panel.select # correct (attribute access, returns float)
|
||||
comp.panel['select'] # WRONG — 'td.Panel' object is not subscriptable
|
||||
```
|
||||
|
||||
### 22. ALWAYS use relative paths in script callbacks
|
||||
|
||||
In scriptTOP/CHOP/SOP/DAT callbacks, use paths relative to `scriptOp` or `me`:
|
||||
```python
|
||||
root = scriptOp.parent().parent()
|
||||
dat = root.op('pixel_data')
|
||||
```
|
||||
NEVER hardcode absolute paths like `op('/project1/myComp/child')` — they break when containers are renamed or copied.
|
||||
|
||||
### 23. keyboardinCHOP channel names have 'k' prefix
|
||||
|
||||
Channel names are `kup`, `kdown`, `kleft`, `kright`, `ka`, `kb`, etc. — NOT `up`, `down`, `a`, `b`. Always verify with:
|
||||
```python
|
||||
channels = [c.name for c in op('/project1/keyboard1').chans()]
|
||||
```
|
||||
|
||||
### 24. expressCHOP cook-only properties — false positive errors
|
||||
|
||||
`me.inputVal`, `me.chanIndex`, `me.sampleIndex` work ONLY in cook-context. Calling `par.expr0expr.eval()` from outside always raises an error — this is NOT a real operator error. Ignore these in error scans.
|
||||
|
||||
### 25. td.Vertex attributes — use index access not named attributes
|
||||
|
||||
In TD 2025.32, `td.Vertex` objects do NOT have `.x`, `.y`, `.z` attributes:
|
||||
```python
|
||||
# WRONG — crashes:
|
||||
vertex.x, vertex.y, vertex.z
|
||||
|
||||
# CORRECT — index-based:
|
||||
vertex.point.P[0], vertex.point.P[1], vertex.point.P[2]
|
||||
# Or for SOP point positions:
|
||||
pt = sop.points()[i]
|
||||
pos = pt.P # use P[0], P[1], P[2]
|
||||
```
|
||||
|
||||
## Audio
|
||||
|
||||
### 26. Audio Spectrum CHOP output is weak — boost it
|
||||
|
||||
Raw output is very small (0.001-0.05). Use built-in boost: `spectrum.par.highfrequencyboost = 3.0`
|
||||
|
||||
If still weak, add Math CHOP in Range mode: `fromrangehi=0.05, torangehi=1.0`
|
||||
|
||||
### 27. AudioSpectrum CHOP: timeslice and sample count are the #1 gotcha
|
||||
|
||||
AudioSpectrum at 44100Hz with `timeslice=False` outputs the ENTIRE audio file as samples (~24000+). CHOP-to-TOP then exceeds texture resolution max and warns/fails.
|
||||
|
||||
**Fix:** Keep `timeslice = True` (default) for real-time per-frame FFT. Set `fftsize` to control bin count (it's a STRING enum: `'256'` not `256`).
|
||||
|
||||
If the CHOP-to-TOP still gets too many samples, set `layout = 'rowscropped'` on the choptoTOP.
|
||||
|
||||
```python
|
||||
spectrum.par.fftsize = '256' # STRING, not int — enum values
|
||||
spectrum.par.timeslice = True # MUST be True for real-time audio reactivity
|
||||
spectex.par.layout = 'rowscropped' # handles oversized CHOP inputs
|
||||
```
|
||||
|
||||
**resampleCHOP has NO `numsamples` param.** It uses `rate`, `start`, `end`, `method`. Don't guess — always `td_get_par_info('resampleCHOP')` first.
|
||||
|
||||
### 28. CHOP To TOP has NO input connectors — use par.chop reference
|
||||
|
||||
```python
|
||||
spec_tex = root.create(choptoTOP, 'spectrum_tex')
|
||||
spec_tex.par.chop = resample # correct: parameter reference
|
||||
# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) # WRONG
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### 29. Always verify after building — errors are silent
|
||||
|
||||
Node errors and broken connections produce no output. Always check:
|
||||
```python
|
||||
for c in list(root.children):
|
||||
e = c.errors()
|
||||
w = c.warnings()
|
||||
if e: print(c.name, 'ERR:', e)
|
||||
if w: print(c.name, 'WARN:', w)
|
||||
```
|
||||
|
||||
### 30. Window COMP param for display target is `winop`
|
||||
|
||||
```python
|
||||
win = root.create(windowCOMP, 'display')
|
||||
win.par.winop = '/project1/logo_out'
|
||||
win.par.winw = 1280; win.par.winh = 720
|
||||
win.par.winopen.pulse()
|
||||
```
|
||||
|
||||
### 31. `sample()` returns frozen pixels in rapid calls
|
||||
|
||||
`out.sample(x, y)` returns pixels from a single cook snapshot. Compare samples with 2+ second delays, or use screencapture on the display window.
|
||||
|
||||
### 32. Audio-reactive GLSL: dual-layer sync pipeline
|
||||
|
||||
For audio-synced visuals, use BOTH layers for maximum effect:
|
||||
|
||||
**Layer 1 (TD-side, real-time):** AudioFileIn → AudioSpectrum(timeslice=True, fftsize='256') → Math(gain=5) → choptoTOP(par.chop=math, layout='rowscropped') → GLSL input. The shader samples `sTD2DInputs[1]` at different x positions for bass/mid/hi. Record the TD output with MovieFileOut.
|
||||
|
||||
**Layer 2 (Python-side, post-hoc):** scipy FFT on the SAME audio file → per-frame features (rms, bass, mid, hi, beat detection) → drive ASCII brightness, chromatic aberration, beat flashes during the render pass.
|
||||
|
||||
Both layers locked to the same audio file = visuals genuinely sync to the beat at two independent stages.
|
||||
|
||||
**Key gotcha:** AudioFileIn must be cued (`par.cue=True` → `par.cuepulse.pulse()`) then uncued (`par.cue=False`, `par.play=True`) before recording starts. Otherwise the spectrum is silent for the first few seconds.
|
||||
|
||||
### 33. twozero MCP: benchmark and prefer native tools
|
||||
|
||||
Benchmarked April 2026: twozero MCP with 36 native tools. The old curl/REST method (port 9981) had zero native tools.
|
||||
|
||||
**Always prefer native MCP tools over td_execute_python:**
|
||||
- `td_create_operator` over `root.create()` scripts (handles viewport positioning)
|
||||
- `td_set_operator_pars` over `node.par.X = Y` scripts (validates param names)
|
||||
- `td_get_par_info` over temp-node discovery dance (instant, no cleanup)
|
||||
- `td_get_errors` over manual `c.errors()` loops
|
||||
- `td_get_focus` for context awareness (no equivalent in old method)
|
||||
|
||||
Only fall back to `td_execute_python` for multi-step logic (wiring chains, conditional builds, loops).
|
||||
|
||||
### 34. twozero td_execute_python response wrapping
|
||||
|
||||
twozero wraps `td_execute_python` responses with status info: `(ok)\n\n[fps 60.0/60] [0 err/0 warn]`. Your Python `result` variable value may not appear verbatim in the response text. If you need to check results programmatically, use `print()` statements in the script — they appear in the response. Don't rely on string-matching the `result` dict.
|
||||
|
||||
### 35. Audio-reactive chain: DO NOT use Lag CHOP or Filter CHOP for spectrum smoothing
|
||||
|
||||
The Derivative docs and tutorials suggest using Lag CHOP (lag1=0.2, lag2=0.5) to smooth raw FFT output before passing to a shader. **This does NOT work with AudioSpectrum → CHOP to TOP → GLSL.**
|
||||
|
||||
What happens: Lag CHOP operates in timeslice mode. A 256-sample spectrum input gets expanded to 1600-2400 samples. The Lag averaging drives all values to near-zero (~1e-06). The CHOP to TOP produces a 2400x2 texture instead of 256x2. The shader receives effectively zero audio data.
|
||||
|
||||
**The correct chain is: Spectrum(outlength=256) → Math(gain=10) → CHOPtoTOP → GLSL.** No CHOP smoothing at all. If you need smoothing, do it in the GLSL shader via temporal lerp with a feedback texture.
|
||||
|
||||
Verified values with audio playing:
|
||||
- Without Lag CHOP: bass bins = 5.0-5.4, mid bins = 1.0-1.7 (strong, usable)
|
||||
- With Lag CHOP: ALL bins = 0.000001-0.00004 (dead, zero audio reactivity)
|
||||
|
||||
### 36. AudioSpectrum Output Length: set manually to avoid CHOP to TOP overflow
|
||||
|
||||
AudioSpectrum in Visualization mode with FFT 8192 outputs 22,050 samples by default (1 per Hz, 0–22050). CHOP to TOP cannot handle this — you get "Number of samples exceeded texture resolution max".
|
||||
|
||||
Fix: `spectrum.par.outputmenu = 'setmanually'` and `spectrum.par.outlength = 256`. This gives 256 frequency bins — plenty for visual FFT.
|
||||
|
||||
DO NOT set `timeslice = False` as a workaround — that processes the entire audio file at once and produces even more samples.
|
||||
|
||||
### 37. GLSL spectrum texture from CHOP to TOP is 256x2 not 256x1
|
||||
|
||||
AudioSpectrum outputs 2 channels (stereo: chan1, chan2). CHOP to TOP with `dataformat='r'` creates a 256x2 texture — one row per channel. Sample the first channel at `y=0.25` (center of first row), NOT `y=0.5` (boundary between rows):
|
||||
|
||||
```glsl
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r; // correct
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.5)).r; // WRONG — samples between rows
|
||||
```
|
||||
|
||||
### 38. FPS=0 doesn't mean ops aren't cooking — check play state
|
||||
|
||||
TD can show `fps:0` in `td_get_perf` while ops still cook and `TOP.save()` still produces valid screenshots. The two most common causes:
|
||||
|
||||
**a) Project is paused (playbar stopped).** TD's playbar can be toggled with spacebar. The `root` at `/` has no `.playbar` attribute (it's on the perform COMP). The easiest fix is sending a spacebar keypress via `td_input_execute`, though this tool can sometimes error. As a workaround, `TOP.save()` always works regardless of play state — use it to verify rendering is actually happening before spending time debugging FPS.
|
||||
|
||||
**b) Audio device CHOP blocking the main thread.** An `audiooutCHOP` with an active audio device can consume 300-400ms/s (2000%+ of frame budget), stalling the cook loop at FPS=0. Fix: keep the CHOP active but set `volume=0` to prevent the audio driver from blocking. Disabling it entirely (`active=False`) may also work but can prevent downstream audio processing CHOPs from cooking.
|
||||
|
||||
Diagnostic sequence when FPS=0:
|
||||
1. `td_get_perf` — check if any op has extreme CPU/s
|
||||
2. `TOP.save()` on the output — if it produces a valid image, the pipeline works, just not at real-time rate
|
||||
3. Check for blocking CHOPs (audioout, audiodevin, etc.)
|
||||
4. Toggle play state (spacebar, or check if absTime.seconds is advancing)
|
||||
|
||||
### 39. Recording while FPS=0 produces empty or near-empty files
|
||||
|
||||
This is the #1 cause of "I recorded for 30 seconds but got a 2-frame video." If TD's cook loop is stalled (FPS=0 or very low), MovieFileOut has nothing to record. Unlike `TOP.save()` which captures the last cooked frame regardless, MovieFileOut only writes frames that actually cook.
|
||||
|
||||
**Always verify FPS before starting a recording:**
|
||||
```python
|
||||
# Check via td_get_perf first
|
||||
# If FPS < 30, do NOT start recording — fix the performance issue first
|
||||
# If FPS=0, the playbar is likely paused — see pitfall #37
|
||||
```
|
||||
|
||||
Common causes of recording empty video:
|
||||
- Playbar paused (FPS=0) — see pitfall #37
|
||||
- Audio device CHOP blocking the main thread — see pitfall #37b
|
||||
- Recording started before audio was cued — audio is silent, GLSL outputs black, MovieFileOut records black frames that look empty
|
||||
- `par.file` set in the same script as `par.record = True` — see pitfall #18
|
||||
|
||||
### 40. GLSL shader produces black output — test before committing to a long render
|
||||
|
||||
New GLSL shaders can fail silently (see pitfall #7). Before recording a long take, always:
|
||||
|
||||
1. **Write a minimal test shader first** that just outputs a solid color or pass-through:
|
||||
```glsl
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
fragColor = TDOutputSwizzle(vec4(uv, 0.0, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
2. **Verify the test renders correctly** via `td_get_screenshot` on the GLSL TOP's output.
|
||||
|
||||
3. **Swap in the real shader** and screenshot again immediately. If black, the shader has a compile error or logic issue.
|
||||
|
||||
4. **Only then start recording.** A 90-second ProRes recording is ~5GB. Recording black frames wastes disk and time.
|
||||
|
||||
Common causes of black GLSL output:
|
||||
- Missing `TDOutputSwizzle()` on macOS (pitfall #8)
|
||||
- Time uniform not connected — shader uses default 0.0, fractal stays at origin
|
||||
- Spectrum texture not connected — audio values all 0.0, driving everything to black
|
||||
- Integer division where float division was expected (`1/2 = 0` not `0.5`)
|
||||
- `absTime.seconds % 1000.0` rolled over past 1000 and the modulo produces unexpected values
|
||||
|
||||
### 41. td_write_dat uses `text` parameter, NOT `content`
|
||||
|
||||
The MCP tool `td_write_dat` expects a `text` parameter for full replacement. Passing `content` returns an error: `"Provide either 'text' for full replace, or 'old_text'+'new_text' for patching"`.
|
||||
|
||||
If `td_write_dat` fails, fall back to `td_execute_python`:
|
||||
```python
|
||||
op("/project1/shader_code").text = shader_string
|
||||
```
|
||||
|
||||
### 42. td_execute_python does NOT return stdout or print() output
|
||||
|
||||
Despite what earlier versions of pitfall #33 stated, `print()` and `debug()` output from `td_execute_python` scripts does NOT appear in the MCP response. The response is always just `(ok)` + FPS/error summary. To read values back, use dedicated inspection tools (`td_get_operator_info`, `td_read_dat`, `td_read_chop`) instead of trying to print from within a script.
|
||||
|
||||
### 43. td_get_operator_info JSON is appended with `[fps X.X/X]` — breaks json.loads()
|
||||
|
||||
The response text from `td_get_operator_info` has `[fps 60.0/60]` appended after the JSON object. This causes `json.loads()` to fail with "Extra data" errors. Strip it before parsing:
|
||||
```python
|
||||
clean = response_text.rsplit('[fps', 1)[0]
|
||||
data = json.loads(clean)
|
||||
```
|
||||
|
||||
### 44. td_get_screenshot is asynchronous — returns `{"status": "pending"}`
|
||||
|
||||
Screenshots don't complete instantly. The tool returns `{"status": "pending", "requestId": "..."}` and the actual file appears later. Wait a few seconds before checking for the file. There is no callback or completion notification — poll the filesystem.
|
||||
|
||||
### 45. Recording duration is manual — no auto-stop at audio end
|
||||
|
||||
MovieFileOut records until `par.record = False` is set. If audio ends before you stop recording, the file keeps growing with repeated frames. Always stop recording promptly after the audio duration. For precision: set a timer on the agent side matching the audio length, then send `par.record = False`. Trim excess with ffmpeg as a safety net:
|
||||
```bash
|
||||
ffmpeg -i raw.mov -t 25 -c copy trimmed.mov
|
||||
```
|
||||
@@ -1,463 +0,0 @@
|
||||
# TouchDesigner Python API Reference
|
||||
|
||||
## The td Module
|
||||
|
||||
TouchDesigner's Python environment auto-imports the `td` module. All TD-specific classes, functions, and constants live here. Scripts inside TD (Script DATs, CHOP/DAT Execute callbacks, Extensions) have full access.
|
||||
|
||||
When using the MCP `execute_python_script` tool, these globals are pre-loaded:
|
||||
- `op` — shortcut for `td.op()`, finds operators by path
|
||||
- `ops` — shortcut for `td.ops()`, finds multiple operators by pattern
|
||||
- `me` — the operator running the script (via MCP this is the twozero internal executor)
|
||||
- `parent` — shortcut for `me.parent()`
|
||||
- `project` — the root project component
|
||||
- `td` — the full td module
|
||||
|
||||
## Finding Operators: op() and ops()
|
||||
|
||||
### op(path) — Find a single operator
|
||||
|
||||
```python
|
||||
# Absolute path (always works from MCP)
|
||||
node = op('/project1/noise1')
|
||||
|
||||
# Relative path (relative to current operator — only in Script DATs)
|
||||
node = op('noise1') # sibling
|
||||
node = op('../noise1') # parent's sibling
|
||||
|
||||
# Returns None if not found (does NOT raise)
|
||||
node = op('/project1/nonexistent') # None
|
||||
```
|
||||
|
||||
### ops(pattern) — Find multiple operators
|
||||
|
||||
```python
|
||||
# Glob patterns
|
||||
nodes = ops('/project1/noise*') # all nodes starting with "noise"
|
||||
nodes = ops('/project1/*') # all direct children
|
||||
nodes = ops('/project1/container1/*') # all children of container1
|
||||
|
||||
# Returns a tuple of operators (may be empty)
|
||||
for n in ops('/project1/*'):
|
||||
print(n.name, n.OPType)
|
||||
```
|
||||
|
||||
### Navigation from a node
|
||||
|
||||
```python
|
||||
node = op('/project1/noise1')
|
||||
|
||||
node.name # 'noise1'
|
||||
node.path # '/project1/noise1'
|
||||
node.OPType # 'noiseTop'
|
||||
node.type # <class 'noiseTop'>
|
||||
node.family # 'TOP'
|
||||
|
||||
# Parent / children
|
||||
node.parent() # the parent COMP
|
||||
node.parent().children # all siblings + self
|
||||
node.parent().findChildren(name='noise*') # filtered
|
||||
|
||||
# Type checking
|
||||
node.isTOP # True
|
||||
node.isCHOP # False
|
||||
node.isSOP # False
|
||||
node.isDAT # False
|
||||
node.isMAT # False
|
||||
node.isCOMP # False
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
Every operator has parameters accessed via the `.par` attribute.
|
||||
|
||||
### Reading parameters
|
||||
|
||||
```python
|
||||
node = op('/project1/noise1')
|
||||
|
||||
# Direct access
|
||||
node.par.seed.val # current evaluated value (may be an expression result)
|
||||
node.par.seed.eval() # same as .val
|
||||
node.par.seed.default # default value
|
||||
node.par.monochrome.val # boolean parameters: True/False
|
||||
|
||||
# List all parameters
|
||||
for p in node.pars():
|
||||
print(f"{p.name}: {p.val} (default: {p.default})")
|
||||
|
||||
# Filter by page (parameter group)
|
||||
for p in node.pars('Noise'): # page name
|
||||
print(f"{p.name}: {p.val}")
|
||||
```
|
||||
|
||||
### Setting parameters
|
||||
|
||||
```python
|
||||
# Direct value setting
|
||||
node.par.seed.val = 42
|
||||
node.par.monochrome.val = True
|
||||
node.par.resolutionw.val = 1920
|
||||
node.par.resolutionh.val = 1080
|
||||
|
||||
# String parameters
|
||||
op('/project1/text1').par.text.val = 'Hello World'
|
||||
|
||||
# File paths
|
||||
op('/project1/moviefilein1').par.file.val = '/path/to/video.mp4'
|
||||
|
||||
# Reference another operator (for "dat", "chop", "top" type parameters)
|
||||
op('/project1/glsl1').par.dat.val = '/project1/shader_code'
|
||||
```
|
||||
|
||||
### Parameter expressions
|
||||
|
||||
```python
|
||||
# Python expressions that evaluate dynamically
|
||||
node.par.seed.expr = "me.time.frame"
|
||||
node.par.tx.expr = "math.sin(me.time.seconds * 2)"
|
||||
|
||||
# Reference another parameter
|
||||
node.par.brightness1.expr = "op('/project1/constant1').par.value0.val"
|
||||
|
||||
# Export (one-way binding from CHOP to parameter)
|
||||
# This makes the parameter follow a CHOP channel value
|
||||
op('/project1/noise1').par.seed.val # can also be driven by exports
|
||||
```
|
||||
|
||||
### Parameter types
|
||||
|
||||
| Type | Python Type | Example |
|
||||
|------|------------|---------|
|
||||
| Float | `float` | `node.par.brightness1.val = 0.5` |
|
||||
| Int | `int` | `node.par.seed.val = 42` |
|
||||
| Toggle | `bool` | `node.par.monochrome.val = True` |
|
||||
| String | `str` | `node.par.text.val = 'hello'` |
|
||||
| Menu | `int` (index) or `str` (label) | `node.par.type.val = 'sine'` |
|
||||
| File | `str` (path) | `node.par.file.val = '/path/to/file'` |
|
||||
| OP reference | `str` (path) | `node.par.dat.val = '/project1/text1'` |
|
||||
| Color | separate r/g/b/a floats | `node.par.colorr.val = 1.0` |
|
||||
| XY/XYZ | separate x/y/z floats | `node.par.tx.val = 0.5` |
|
||||
|
||||
## Creating and Deleting Operators
|
||||
|
||||
```python
|
||||
# Create via parent component
|
||||
parent = op('/project1')
|
||||
new_node = parent.create(noiseTop) # using class reference
|
||||
new_node = parent.create(noiseTop, 'my_noise') # with custom name
|
||||
|
||||
# The MCP create_td_node tool handles this automatically:
|
||||
# create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="my_noise")
|
||||
|
||||
# Delete
|
||||
node = op('/project1/my_noise')
|
||||
node.destroy()
|
||||
|
||||
# Copy
|
||||
original = op('/project1/noise1')
|
||||
copy = parent.copy(original, name='noise1_copy')
|
||||
```
|
||||
|
||||
## Connections (Wiring Operators)
|
||||
|
||||
### Output to Input connections
|
||||
|
||||
```python
|
||||
# Connect noise1's output to level1's input
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))
|
||||
|
||||
# Connect to specific input index (for multi-input operators like Composite)
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/composite1').inputConnectors[0])
|
||||
op('/project1/text1').outputConnectors[0].connect(op('/project1/composite1').inputConnectors[1])
|
||||
|
||||
# Disconnect all outputs
|
||||
op('/project1/noise1').outputConnectors[0].disconnect()
|
||||
|
||||
# Query connections
|
||||
node = op('/project1/level1')
|
||||
inputs = node.inputs # list of connected input operators
|
||||
outputs = node.outputs # list of connected output operators
|
||||
```
|
||||
|
||||
### Connection patterns for common setups
|
||||
|
||||
```python
|
||||
# Linear chain: A -> B -> C -> D
|
||||
ops_list = [op(f'/project1/{name}') for name in ['noise1', 'level1', 'blur1', 'null1']]
|
||||
for i in range(len(ops_list) - 1):
|
||||
ops_list[i].outputConnectors[0].connect(ops_list[i+1])
|
||||
|
||||
# Fan-out: A -> B, A -> C, A -> D
|
||||
source = op('/project1/noise1')
|
||||
for target_name in ['level1', 'composite1', 'transform1']:
|
||||
source.outputConnectors[0].connect(op(f'/project1/{target_name}'))
|
||||
|
||||
# Merge: A + B + C -> Composite
|
||||
comp = op('/project1/composite1')
|
||||
for i, source_name in enumerate(['noise1', 'text1', 'ramp1']):
|
||||
op(f'/project1/{source_name}').outputConnectors[0].connect(comp.inputConnectors[i])
|
||||
```
|
||||
|
||||
## DAT Content Manipulation
|
||||
|
||||
### Text DATs
|
||||
|
||||
```python
|
||||
dat = op('/project1/text1')
|
||||
|
||||
# Read
|
||||
content = dat.text # full text as string
|
||||
|
||||
# Write
|
||||
dat.text = "new content"
|
||||
dat.text = '''multi
|
||||
line
|
||||
content'''
|
||||
|
||||
# Append
|
||||
dat.text += "\nnew line"
|
||||
```
|
||||
|
||||
### Table DATs
|
||||
|
||||
```python
|
||||
dat = op('/project1/table1')
|
||||
|
||||
# Read cell
|
||||
val = dat[0, 0] # row 0, col 0
|
||||
val = dat[0, 'name'] # row 0, column named 'name'
|
||||
val = dat['key', 1] # row named 'key', col 1
|
||||
|
||||
# Write cell
|
||||
dat[0, 0] = 'value'
|
||||
|
||||
# Read row/col
|
||||
row = dat.row(0) # list of Cell objects
|
||||
col = dat.col('name') # list of Cell objects
|
||||
|
||||
# Dimensions
|
||||
rows = dat.numRows
|
||||
cols = dat.numCols
|
||||
|
||||
# Append row
|
||||
dat.appendRow(['col1_val', 'col2_val', 'col3_val'])
|
||||
|
||||
# Clear
|
||||
dat.clear()
|
||||
|
||||
# Set entire table
|
||||
dat.clear()
|
||||
dat.appendRow(['name', 'value', 'type'])
|
||||
dat.appendRow(['frequency', '440', 'float'])
|
||||
dat.appendRow(['amplitude', '0.8', 'float'])
|
||||
```
|
||||
|
||||
## Time and Animation
|
||||
|
||||
```python
|
||||
# Global time
|
||||
td.absTime.frame # absolute frame number (never resets)
|
||||
td.absTime.seconds # absolute seconds
|
||||
|
||||
# Timeline time (affected by play/pause/loop)
|
||||
me.time.frame # current frame on timeline
|
||||
me.time.seconds # current seconds on timeline
|
||||
me.time.rate # FPS setting
|
||||
|
||||
# Timeline control (via execute_python_script)
|
||||
project.play = True
|
||||
project.play = False
|
||||
project.frameRange = (1, 300) # set timeline range
|
||||
|
||||
# Cook frame (when operator was last computed)
|
||||
node.cookFrame
|
||||
node.cookTime
|
||||
```
|
||||
|
||||
## Extensions (Custom Python Classes on Components)
|
||||
|
||||
Extensions add custom Python methods and attributes to COMPs.
|
||||
|
||||
```python
|
||||
# Create extension on a Base COMP
|
||||
base = op('/project1/myBase')
|
||||
|
||||
# The extension class is defined in a Text DAT inside the COMP
|
||||
# Typically named 'ExtClass' with the extension code:
|
||||
|
||||
extension_code = '''
|
||||
class MyExtension:
|
||||
def __init__(self, ownerComp):
|
||||
self.ownerComp = ownerComp
|
||||
self.counter = 0
|
||||
|
||||
def Reset(self):
|
||||
self.counter = 0
|
||||
|
||||
def Increment(self):
|
||||
self.counter += 1
|
||||
return self.counter
|
||||
|
||||
@property
|
||||
def Count(self):
|
||||
return self.counter
|
||||
'''
|
||||
|
||||
# Write extension code to DAT inside the COMP
|
||||
op('/project1/myBase/extClass').text = extension_code
|
||||
|
||||
# Configure the extension on the COMP
|
||||
base.par.extension1 = 'extClass' # name of the DAT
|
||||
base.par.promoteextension1 = True # promote methods to parent
|
||||
|
||||
# Call extension methods
|
||||
base.Increment() # calls MyExtension.Increment()
|
||||
count = base.Count # accesses MyExtension.Count property
|
||||
base.Reset()
|
||||
```
|
||||
|
||||
## Useful Built-in Modules
|
||||
|
||||
### tdu — TouchDesigner Utilities
|
||||
|
||||
```python
|
||||
import tdu
|
||||
|
||||
# Dependency tracking (reactive values)
|
||||
dep = tdu.Dependency(initial_value)
|
||||
dep.val = new_value # triggers dependents to recook
|
||||
|
||||
# File path utilities
|
||||
tdu.expandPath('$HOME/Desktop/output.mov')
|
||||
|
||||
# Math
|
||||
tdu.clamp(value, min, max)
|
||||
tdu.remap(value, from_min, from_max, to_min, to_max)
|
||||
```
|
||||
|
||||
### TDFunctions
|
||||
|
||||
```python
|
||||
from TDFunctions import *
|
||||
|
||||
# Commonly used utilities
|
||||
clamp(value, low, high)
|
||||
remap(value, inLow, inHigh, outLow, outHigh)
|
||||
interp(value1, value2, t) # linear interpolation
|
||||
```
|
||||
|
||||
### TDStoreTools — Persistent Storage
|
||||
|
||||
```python
|
||||
from TDStoreTools import StorageManager
|
||||
|
||||
# Store data that survives project reload
|
||||
me.store('myKey', 'myValue')
|
||||
val = me.fetch('myKey', default='fallback')
|
||||
|
||||
# Storage dict
|
||||
me.storage['key'] = value
|
||||
```
|
||||
|
||||
## Common Patterns via execute_python_script
|
||||
|
||||
### Build a complete chain
|
||||
|
||||
```python
|
||||
# Create a complete audio-reactive noise chain
|
||||
parent = op('/project1')
|
||||
|
||||
# Create operators
|
||||
audio_in = parent.create(audiofileinChop, 'audio_in')
|
||||
spectrum = parent.create(audiospectrumChop, 'spectrum')
|
||||
chop_to_top = parent.create(choptopTop, 'chop_to_top')
|
||||
noise = parent.create(noiseTop, 'noise1')
|
||||
level = parent.create(levelTop, 'level1')
|
||||
null_out = parent.create(nullTop, 'out')
|
||||
|
||||
# Wire the chain
|
||||
audio_in.outputConnectors[0].connect(spectrum)
|
||||
spectrum.outputConnectors[0].connect(chop_to_top)
|
||||
noise.outputConnectors[0].connect(level)
|
||||
level.outputConnectors[0].connect(null_out)
|
||||
|
||||
# Set parameters
|
||||
audio_in.par.file = '/path/to/music.wav'
|
||||
audio_in.par.play = True
|
||||
spectrum.par.size = 512
|
||||
noise.par.type = 1 # Sparse
|
||||
noise.par.monochrome = False
|
||||
noise.par.resolutionw = 1920
|
||||
noise.par.resolutionh = 1080
|
||||
level.par.opacity = 0.8
|
||||
level.par.gamma1 = 0.7
|
||||
```
|
||||
|
||||
### Query network state
|
||||
|
||||
```python
|
||||
# Get all TOPs in the project
|
||||
tops = [c for c in op('/project1').findChildren(type=TOP)]
|
||||
for t in tops:
|
||||
print(f"{t.path}: {t.OPType} {'ERROR' if t.errors() else 'OK'}")
|
||||
|
||||
# Find all operators with errors
|
||||
def find_errors(parent_path='/project1'):
|
||||
parent = op(parent_path)
|
||||
errors = []
|
||||
for child in parent.findChildren(depth=-1):
|
||||
if child.errors():
|
||||
errors.append((child.path, child.errors()))
|
||||
return errors
|
||||
|
||||
result = find_errors()
|
||||
```
|
||||
|
||||
### Batch parameter changes
|
||||
|
||||
```python
|
||||
# Set parameters on multiple nodes at once
|
||||
settings = {
|
||||
'/project1/noise1': {'seed': 42, 'monochrome': False, 'resolutionw': 1920},
|
||||
'/project1/level1': {'brightness1': 1.2, 'gamma1': 0.8},
|
||||
'/project1/blur1': {'sizex': 5, 'sizey': 5},
|
||||
}
|
||||
|
||||
for path, params in settings.items():
|
||||
node = op(path)
|
||||
if node:
|
||||
for key, val in params.items():
|
||||
setattr(node.par, key, val)
|
||||
```
|
||||
|
||||
## Python Version and Packages
|
||||
|
||||
TouchDesigner bundles Python 3.11+ with these pre-installed:
|
||||
- **numpy** — array operations, fast math
|
||||
- **scipy** — signal processing, FFT
|
||||
- **OpenCV** (cv2) — computer vision
|
||||
- **PIL/Pillow** — image processing
|
||||
- **requests** — HTTP client
|
||||
- **json**, **re**, **os**, **sys** — standard library
|
||||
|
||||
**IMPORTANT:** Parameter names in examples below are illustrative. Always run discovery (SKILL.md Step 0) to get actual names for your TD version. Do NOT copy param names from these examples verbatim.
|
||||
|
||||
Custom packages can be installed to TD's Python site-packages directory. See TD documentation for the exact path per platform.
|
||||
|
||||
## SOP Vertex/Point Access (TD 2025.32)
|
||||
|
||||
In TD 2025.32, `td.Vertex` does NOT have `.x`, `.y`, `.z` attributes. Use index access:
|
||||
|
||||
```python
|
||||
# WRONG — crashes in TD 2025.32:
|
||||
vertex.x, vertex.y, vertex.z
|
||||
|
||||
# CORRECT — index/attribute access:
|
||||
pt = sop.points()[i]
|
||||
pos = pt.P # Position object
|
||||
x, y, z = pos[0], pos[1], pos[2]
|
||||
|
||||
# Always introspect first:
|
||||
dir(sop.points()[0]) # see what attributes actually exist
|
||||
dir(sop.points()[0].P) # see Position object interface
|
||||
```
|
||||
@@ -1,244 +0,0 @@
|
||||
# TouchDesigner Troubleshooting (twozero MCP)
|
||||
|
||||
> See `references/pitfalls.md` for the comprehensive lessons-learned list.
|
||||
|
||||
## 1. Connection Issues
|
||||
|
||||
### Port 40404 not responding
|
||||
|
||||
Check these in order:
|
||||
|
||||
1. Is TouchDesigner running?
|
||||
```bash
|
||||
pgrep TouchDesigner
|
||||
```
|
||||
|
||||
1b. Quick hub health check (no JSON-RPC needed):
|
||||
A plain GET to the MCP URL returns instance info:
|
||||
```
|
||||
curl -s http://localhost:40404/mcp
|
||||
```
|
||||
Returns: `{"hub": true, "pid": ..., "instances": {"127.0.0.1_PID": {"project": "...", "tdVersion": "...", ...}}}`
|
||||
If this returns JSON but `instances` is empty, TD is running but twozero hasn't registered yet.
|
||||
|
||||
2. Is twozero installed in TD?
|
||||
Open TD Palette Browser > twozero should be listed. If not, install it.
|
||||
|
||||
3. Is MCP enabled in twozero settings?
|
||||
In TD, open twozero preferences and confirm MCP server is toggled ON.
|
||||
|
||||
4. Test the port directly:
|
||||
```bash
|
||||
nc -z 127.0.0.1 40404
|
||||
```
|
||||
|
||||
5. Test the MCP endpoint:
|
||||
```bash
|
||||
curl -s http://localhost:40404/mcp
|
||||
```
|
||||
Should return JSON with hub info. If it does, the server is running.
|
||||
|
||||
### Hub responds but no TD instances
|
||||
|
||||
The twozero MCP hub is running but TD hasn't registered. Causes:
|
||||
- TD project not loaded yet (still on splash screen)
|
||||
- twozero COMP not initialized in the current project
|
||||
- twozero version mismatch
|
||||
|
||||
Fix: Open/reload a TD project that contains the twozero COMP. Use td_list_instances
|
||||
to check which TD instances are registered.
|
||||
|
||||
### Multi-instance setup
|
||||
|
||||
twozero auto-assigns ports for multiple TD instances:
|
||||
- First instance: 40404
|
||||
- Second instance: 40405
|
||||
- Third instance: 40406
|
||||
- etc.
|
||||
|
||||
Use `td_list_instances` to discover all running instances and their ports.
|
||||
|
||||
## 2. MCP Tool Errors
|
||||
|
||||
### td_execute_python returns error
|
||||
|
||||
The error message from td_execute_python often contains the Python traceback.
|
||||
If it's unclear, use `td_read_textport` to see the full TD console output —
|
||||
Python exceptions are always printed there.
|
||||
|
||||
Common causes:
|
||||
- Syntax error in the script
|
||||
- Referencing a node that doesn't exist (op() returns None, then you call .par on None)
|
||||
- Using wrong parameter names (see pitfalls.md)
|
||||
|
||||
### td_set_operator_pars fails
|
||||
|
||||
Parameter name mismatch is the #1 cause. The tool validates param names and
|
||||
returns clear errors, but you must use exact names.
|
||||
|
||||
Fix: ALWAYS call `td_get_par_info` first to discover the real parameter names:
|
||||
```
|
||||
td_get_par_info(op_type='glslTOP')
|
||||
td_get_par_info(op_type='noiseTOP')
|
||||
```
|
||||
|
||||
### td_create_operator type name errors
|
||||
|
||||
Operator type names use camelCase with family suffix:
|
||||
- CORRECT: noiseTOP, glslTOP, levelTOP, compositeTOP, audiospectrumCHOP
|
||||
- WRONG: NoiseTOP, noise_top, NOISE TOP, Noise
|
||||
|
||||
### td_get_operator_info for deep inspection
|
||||
|
||||
If unsure about any aspect of an operator (params, inputs, outputs, state):
|
||||
```
|
||||
td_get_operator_info(path='/project1/noise1', detail='full')
|
||||
```
|
||||
|
||||
## 3. Parameter Discovery
|
||||
|
||||
CRITICAL: ALWAYS use td_get_par_info to discover parameter names.
|
||||
|
||||
The agent's LLM training data contains WRONG parameter names for TouchDesigner.
|
||||
Do not trust them. Known wrong names include dat vs pixeldat, colora vs alpha,
|
||||
sizex vs size, and many more. See pitfalls.md for the full list.
|
||||
|
||||
Workflow:
|
||||
1. td_get_par_info(op_type='glslTOP') — get all params for a type
|
||||
2. td_get_operator_info(path='/project1/mynode', detail='full') — get params for a specific instance
|
||||
3. Use ONLY the names returned by these tools
|
||||
|
||||
## 4. Performance
|
||||
|
||||
### Diagnosing slow performance
|
||||
|
||||
Use `td_get_perf` to see which operators are slow. Look at cook times —
|
||||
anything over 1ms per frame is worth investigating.
|
||||
|
||||
Common causes:
|
||||
- Resolution too high (especially on Non-Commercial)
|
||||
- Complex GLSL shaders
|
||||
- Too many TOP-to-CHOP or CHOP-to-TOP transfers (GPU-CPU memory copies)
|
||||
- Feedback loops without decay (values accumulate, memory grows)
|
||||
|
||||
### Non-Commercial license restrictions
|
||||
|
||||
- Resolution cap: 1280x1280. Setting resolutionw=1920 silently clamps to 1280.
|
||||
- H.264/H.265/AV1 encoding requires Commercial license. Use ProRes or Hap instead.
|
||||
- No commercial use of output.
|
||||
|
||||
Always check effective resolution after creation:
|
||||
```python
|
||||
n.cook(force=True)
|
||||
actual = str(n.width) + 'x' + str(n.height)
|
||||
```
|
||||
|
||||
## 5. Hermes Configuration
|
||||
|
||||
### Config location
|
||||
|
||||
`$HERMES_HOME/config.yaml` (defaults to `~/.hermes/config.yaml` when `HERMES_HOME` is unset)
|
||||
|
||||
### MCP entry format
|
||||
|
||||
The twozero TD entry should look like:
|
||||
```yaml
|
||||
mcpServers:
|
||||
twozero_td:
|
||||
url: http://localhost:40404/mcp
|
||||
```
|
||||
|
||||
### After config changes
|
||||
|
||||
Restart the Hermes session for changes to take effect. The MCP connection is
|
||||
established at session startup.
|
||||
|
||||
### Verifying MCP tools are available
|
||||
|
||||
After restarting, the session log should show twozero MCP tools registered.
|
||||
If tools show as registered but aren't callable, check:
|
||||
- The twozero MCP hub is still running (curl test above)
|
||||
- TD is still running with a project loaded
|
||||
- No firewall blocking localhost:40404
|
||||
|
||||
## 6. Node Creation Issues
|
||||
|
||||
### "Node type not found" error
|
||||
|
||||
Wrong type string. Use camelCase with family suffix:
|
||||
- Wrong: NoiseTop, noise_top, NOISE TOP
|
||||
- Right: noiseTOP
|
||||
|
||||
### Node created but not visible
|
||||
|
||||
Check parentPath — use absolute paths like /project1. The default project
|
||||
root is /project1. System nodes live at /, /ui, /sys, /local, /perform.
|
||||
Don't create user nodes outside /project1.
|
||||
|
||||
### Cannot create node inside a non-COMP
|
||||
|
||||
Only COMP operators (Container, Base, Geometry, etc.) can contain children.
|
||||
You cannot create nodes inside a TOP, CHOP, SOP, DAT, or MAT.
|
||||
|
||||
## 7. Wiring Issues
|
||||
|
||||
### Cross-family wiring
|
||||
|
||||
TOPs connect to TOPs, CHOPs to CHOPs, SOPs to SOPs, DATs to DATs.
|
||||
Use converter operators to bridge: choptoTOP, topToCHOP, soptoDAT, etc.
|
||||
|
||||
Note: choptoTOP has NO input connectors. Use par.chop reference instead:
|
||||
```python
|
||||
spec_tex.par.chop = resample_node # correct
|
||||
# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0])
|
||||
```
|
||||
|
||||
### Feedback loops
|
||||
|
||||
Never create A -> B -> A directly. Use a Feedback TOP:
|
||||
```python
|
||||
fb = root.create(feedbackTOP, 'fb')
|
||||
fb.par.top = comp.path # reference only, no wire to fb input
|
||||
fb.outputConnectors[0].connect(next_node)
|
||||
```
|
||||
"Cook dependency loop detected" warning on the chain is expected and correct.
|
||||
|
||||
## 8. GLSL Issues
|
||||
|
||||
### Shader compilation errors are silent
|
||||
|
||||
GLSL TOP shows a yellow warning in the UI but node.errors() may return empty.
|
||||
Check node.warnings() too. Create an Info DAT pointed at the GLSL TOP for
|
||||
full compiler output.
|
||||
|
||||
### TD GLSL specifics
|
||||
|
||||
- Uses GLSL 4.60 (Vulkan backend). GLSL 3.30 and earlier removed.
|
||||
- UV coordinates: vUV.st (not gl_FragCoord)
|
||||
- Input textures: sTD2DInputs[0]
|
||||
- Output: layout(location = 0) out vec4 fragColor
|
||||
- macOS CRITICAL: Always wrap output with TDOutputSwizzle(color)
|
||||
- No built-in time uniform. Pass time via GLSL TOP Values page or Constant TOP.
|
||||
|
||||
## 9. Recording Issues
|
||||
|
||||
### H.264/H.265/AV1 requires Commercial license
|
||||
|
||||
Use Apple ProRes on macOS (hardware accelerated, not license-restricted):
|
||||
```python
|
||||
rec.par.videocodec = 'prores' # Preferred on macOS — lossless, Non-Commercial OK
|
||||
# rec.par.videocodec = 'mjpa' # Fallback — lossy, works everywhere
|
||||
```
|
||||
|
||||
### MovieFileOut has no .record() method
|
||||
|
||||
Use the toggle parameter:
|
||||
```python
|
||||
rec.par.record = True # start
|
||||
rec.par.record = False # stop
|
||||
```
|
||||
|
||||
### All exported frames identical
|
||||
|
||||
TOP.save() captures same frame when called rapidly. Use MovieFileOut for
|
||||
real-time recording. Set project.realTime = False for frame-accurate output.
|
||||
@@ -1,115 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# setup.sh — Automated setup for twozero MCP plugin for TouchDesigner
|
||||
# Idempotent: safe to run multiple times.
|
||||
set -euo pipefail
|
||||
|
||||
GREEN='\033[0;32m'; RED='\033[0;31m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m'
|
||||
OK="${GREEN}✔${NC}"; FAIL="${RED}✘${NC}"; WARN="${YELLOW}⚠${NC}"
|
||||
|
||||
TWOZERO_URL="https://www.404zero.com/pisang/twozero.tox"
|
||||
TOX_PATH="$HOME/Downloads/twozero.tox"
|
||||
HERMES_HOME_DIR="${HERMES_HOME:-$HOME/.hermes}"
|
||||
HERMES_CFG="${HERMES_HOME_DIR}/config.yaml"
|
||||
MCP_PORT=40404
|
||||
MCP_ENDPOINT="http://localhost:${MCP_PORT}/mcp"
|
||||
|
||||
manual_steps=()
|
||||
|
||||
echo -e "\n${CYAN}═══ twozero MCP for TouchDesigner — Setup ═══${NC}\n"
|
||||
|
||||
# ── 1. Check if TouchDesigner is running ──
|
||||
# Match on process *name* (not full cmdline) to avoid self-matching shells
|
||||
# that happen to have "TouchDesigner" in their args. macOS and Linux pgrep
|
||||
# both support -x for exact name match.
|
||||
if pgrep -x TouchDesigner >/dev/null 2>&1 || pgrep -x TouchDesignerFTE >/dev/null 2>&1; then
|
||||
echo -e " ${OK} TouchDesigner is running"
|
||||
td_running=true
|
||||
else
|
||||
echo -e " ${WARN} TouchDesigner is not running"
|
||||
td_running=false
|
||||
fi
|
||||
|
||||
# ── 2. Ensure twozero.tox exists ──
|
||||
if [[ -f "$TOX_PATH" ]]; then
|
||||
echo -e " ${OK} twozero.tox already exists at ${TOX_PATH}"
|
||||
else
|
||||
echo -e " ${WARN} twozero.tox not found — downloading..."
|
||||
if curl -fSL -o "$TOX_PATH" "$TWOZERO_URL" 2>/dev/null; then
|
||||
echo -e " ${OK} Downloaded twozero.tox to ${TOX_PATH}"
|
||||
else
|
||||
echo -e " ${FAIL} Failed to download twozero.tox from ${TWOZERO_URL}"
|
||||
echo " Please download manually and place at ${TOX_PATH}"
|
||||
manual_steps+=("Download twozero.tox from ${TWOZERO_URL} to ${TOX_PATH}")
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── 3. Ensure Hermes config has twozero_td MCP entry ──
|
||||
if [[ ! -f "$HERMES_CFG" ]]; then
|
||||
echo -e " ${FAIL} Hermes config not found at ${HERMES_CFG}"
|
||||
manual_steps+=("Create ${HERMES_CFG} with twozero_td MCP server entry")
|
||||
elif grep -q 'twozero_td' "$HERMES_CFG" 2>/dev/null; then
|
||||
echo -e " ${OK} twozero_td MCP entry exists in Hermes config"
|
||||
else
|
||||
echo -e " ${WARN} Adding twozero_td MCP entry to Hermes config..."
|
||||
python3 -c "
|
||||
import yaml, sys, copy
|
||||
|
||||
cfg_path = '$HERMES_CFG'
|
||||
with open(cfg_path, 'r') as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
|
||||
if 'mcp_servers' not in cfg:
|
||||
cfg['mcp_servers'] = {}
|
||||
|
||||
if 'twozero_td' not in cfg['mcp_servers']:
|
||||
cfg['mcp_servers']['twozero_td'] = {
|
||||
'url': '${MCP_ENDPOINT}',
|
||||
'timeout': 120,
|
||||
'connect_timeout': 60
|
||||
}
|
||||
with open(cfg_path, 'w') as f:
|
||||
yaml.dump(cfg, f, default_flow_style=False, sort_keys=False)
|
||||
" 2>/dev/null && echo -e " ${OK} twozero_td MCP entry added to config" \
|
||||
|| { echo -e " ${FAIL} Could not update config (is PyYAML installed?)"; \
|
||||
manual_steps+=("Add twozero_td MCP entry to ${HERMES_CFG} manually"); }
|
||||
manual_steps+=("Restart Hermes session to pick up config change")
|
||||
fi
|
||||
|
||||
# ── 4. Test if MCP port is responding ──
|
||||
if nc -z 127.0.0.1 "$MCP_PORT" 2>/dev/null; then
|
||||
echo -e " ${OK} Port ${MCP_PORT} is open"
|
||||
|
||||
# ── 5. Verify MCP endpoint responds ──
|
||||
resp=$(curl -s --max-time 3 "$MCP_ENDPOINT" 2>/dev/null || true)
|
||||
if [[ -n "$resp" ]]; then
|
||||
echo -e " ${OK} MCP endpoint responded at ${MCP_ENDPOINT}"
|
||||
else
|
||||
echo -e " ${WARN} Port open but MCP endpoint returned empty response"
|
||||
manual_steps+=("Verify MCP is enabled in twozero settings")
|
||||
fi
|
||||
else
|
||||
echo -e " ${WARN} Port ${MCP_PORT} is not open"
|
||||
if [[ "$td_running" == true ]]; then
|
||||
manual_steps+=("In TD: drag twozero.tox into network editor → click Install")
|
||||
manual_steps+=("Enable MCP: twozero icon → Settings → mcp → 'auto start MCP' → Yes")
|
||||
else
|
||||
manual_steps+=("Launch TouchDesigner")
|
||||
manual_steps+=("Drag twozero.tox into the TD network editor and click Install")
|
||||
manual_steps+=("Enable MCP: twozero icon → Settings → mcp → 'auto start MCP' → Yes")
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Status Report ──
|
||||
echo -e "\n${CYAN}═══ Status Report ═══${NC}\n"
|
||||
|
||||
if [[ ${#manual_steps[@]} -eq 0 ]]; then
|
||||
echo -e " ${OK} ${GREEN}Fully configured! twozero MCP is ready to use.${NC}\n"
|
||||
exit 0
|
||||
else
|
||||
echo -e " ${WARN} ${YELLOW}Manual steps remaining:${NC}\n"
|
||||
for i in "${!manual_steps[@]}"; do
|
||||
echo -e " $((i+1)). ${manual_steps[$i]}"
|
||||
done
|
||||
echo ""
|
||||
exit 1
|
||||
fi
|
||||
@@ -7,7 +7,7 @@ license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [atropos, rl, environments, training, reinforcement-learning, reward-functions]
|
||||
related_skills: [axolotl, fine-tuning-with-trl, lm-evaluation-harness]
|
||||
related_skills: [axolotl, grpo-rl-training, trl-fine-tuning, lm-evaluation-harness]
|
||||
---
|
||||
|
||||
# Hermes Agent Atropos Environments
|
||||
|
||||
@@ -19,7 +19,6 @@ import json
|
||||
import logging
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from agent.memory_provider import MemoryProvider
|
||||
@@ -207,19 +206,13 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
self._turn_count = 0
|
||||
self._injection_frequency = "every-turn" # or "first-turn"
|
||||
self._context_cadence = 1 # minimum turns between context API calls
|
||||
self._dialectic_cadence = 1 # backwards-compat fallback; wizard writes 2 on new configs
|
||||
self._dialectic_cadence = 3 # minimum turns between dialectic API calls
|
||||
self._dialectic_depth = 1 # how many .chat() calls per dialectic cycle (1-3)
|
||||
self._dialectic_depth_levels: list[str] | None = None # per-pass reasoning levels
|
||||
self._reasoning_heuristic: bool = True # scale base level by query length
|
||||
self._reasoning_level_cap: str = "high" # ceiling for auto-selected level
|
||||
self._reasoning_level_cap: Optional[str] = None # "minimal", "low", "medium", "high"
|
||||
self._last_context_turn = -999
|
||||
self._last_dialectic_turn = -999
|
||||
|
||||
# Liveness + observability state
|
||||
self._prefetch_thread_started_at: float = 0.0 # monotonic ts of current thread
|
||||
self._prefetch_result_fired_at: int = -999 # turn the pending result was fired at
|
||||
self._dialectic_empty_streak: int = 0 # consecutive empty returns
|
||||
|
||||
# Port #1957: lazy session init for tools-only mode
|
||||
self._session_initialized = False
|
||||
self._lazy_init_kwargs: Optional[dict] = None
|
||||
@@ -293,6 +286,14 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
logger.debug("Honcho not configured — plugin inactive")
|
||||
return
|
||||
|
||||
# Override peer_name with gateway user_id for per-user memory scoping.
|
||||
# Only when no explicit peerName was configured — an explicit peerName
|
||||
# means the user chose their identity; a raw user_id (e.g. Telegram
|
||||
# chat ID) should not silently replace it.
|
||||
_gw_user_id = kwargs.get("user_id")
|
||||
if _gw_user_id and not cfg.peer_name:
|
||||
cfg.peer_name = _gw_user_id
|
||||
|
||||
self._config = cfg
|
||||
|
||||
# ----- B1: recall_mode from config -----
|
||||
@@ -304,16 +305,12 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
raw = cfg.raw or {}
|
||||
self._injection_frequency = raw.get("injectionFrequency", "every-turn")
|
||||
self._context_cadence = int(raw.get("contextCadence", 1))
|
||||
# Backwards-compat: unset dialecticCadence falls back to 1
|
||||
# (every turn) so existing honcho.json configs without the key
|
||||
# behave as they did before. New setups via `hermes honcho setup`
|
||||
# get dialecticCadence=2 written explicitly by the wizard.
|
||||
self._dialectic_cadence = int(raw.get("dialecticCadence", 1))
|
||||
self._dialectic_cadence = int(raw.get("dialecticCadence", 3))
|
||||
self._dialectic_depth = max(1, min(cfg.dialectic_depth, 3))
|
||||
self._dialectic_depth_levels = cfg.dialectic_depth_levels
|
||||
self._reasoning_heuristic = cfg.reasoning_heuristic
|
||||
if cfg.reasoning_level_cap in self._LEVEL_ORDER:
|
||||
self._reasoning_level_cap = cfg.reasoning_level_cap
|
||||
cap = raw.get("reasoningLevelCap")
|
||||
if cap and cap in ("minimal", "low", "medium", "high"):
|
||||
self._reasoning_level_cap = cap
|
||||
except Exception as e:
|
||||
logger.debug("Honcho cost-awareness config parse error: %s", e)
|
||||
|
||||
@@ -355,7 +352,6 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
honcho=client,
|
||||
config=cfg,
|
||||
context_tokens=cfg.context_tokens,
|
||||
runtime_user_peer_name=kwargs.get("user_id") or None,
|
||||
)
|
||||
|
||||
# ----- B3: resolve_session_name -----
|
||||
@@ -395,45 +391,14 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
except Exception as e:
|
||||
logger.debug("Honcho memory file migration skipped: %s", e)
|
||||
|
||||
# ----- B7: Pre-warming at init -----
|
||||
# Context prewarm warms peer.context() (base layer), consumed via
|
||||
# pop_context_result() in prefetch(). Dialectic prewarm runs the
|
||||
# full configured depth and writes into _prefetch_result so turn 1
|
||||
# consumes the result directly.
|
||||
# ----- B7: Pre-warming context at init -----
|
||||
if self._recall_mode in ("context", "hybrid"):
|
||||
try:
|
||||
self._manager.prefetch_context(self._session_key)
|
||||
self._manager.prefetch_dialectic(self._session_key, "What should I know about this user?")
|
||||
logger.debug("Honcho pre-warm threads started for session: %s", self._session_key)
|
||||
except Exception as e:
|
||||
logger.debug("Honcho context prewarm failed: %s", e)
|
||||
|
||||
_prewarm_query = (
|
||||
"Summarize what you know about this user. "
|
||||
"Focus on preferences, current projects, and working style."
|
||||
)
|
||||
|
||||
def _prewarm_dialectic() -> None:
|
||||
try:
|
||||
r = self._run_dialectic_depth(_prewarm_query)
|
||||
except Exception as exc:
|
||||
logger.debug("Honcho dialectic prewarm failed: %s", exc)
|
||||
self._dialectic_empty_streak += 1
|
||||
return
|
||||
if r and r.strip():
|
||||
with self._prefetch_lock:
|
||||
self._prefetch_result = r
|
||||
self._prefetch_result_fired_at = 0
|
||||
# Treat prewarm as turn 0 so cadence gating starts clean.
|
||||
self._last_dialectic_turn = 0
|
||||
self._dialectic_empty_streak = 0
|
||||
else:
|
||||
self._dialectic_empty_streak += 1
|
||||
|
||||
self._prefetch_thread_started_at = time.monotonic()
|
||||
self._prefetch_thread = threading.Thread(
|
||||
target=_prewarm_dialectic, daemon=True, name="honcho-prewarm-dialectic"
|
||||
)
|
||||
self._prefetch_thread.start()
|
||||
logger.debug("Honcho pre-warm started for session: %s", self._session_key)
|
||||
logger.debug("Honcho pre-warm failed: %s", e)
|
||||
|
||||
def _ensure_session(self) -> bool:
|
||||
"""Lazily initialize the Honcho session (for tools-only mode).
|
||||
@@ -522,8 +487,7 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
"# Honcho Memory\n"
|
||||
"Active (tools-only mode). Use honcho_profile for a quick factual snapshot, "
|
||||
"honcho_search for raw excerpts, honcho_context for raw peer context, "
|
||||
"honcho_reasoning for synthesized answers (pass reasoning_level "
|
||||
"minimal/low/medium/high/max — you pick the depth per call), "
|
||||
"honcho_reasoning for synthesized answers, "
|
||||
"honcho_conclude to save facts about the user. "
|
||||
"No automatic context injection — you must use tools to access memory."
|
||||
)
|
||||
@@ -533,8 +497,7 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
"Active (hybrid mode). Relevant context is auto-injected AND memory tools are available. "
|
||||
"Use honcho_profile for a quick factual snapshot, "
|
||||
"honcho_search for raw excerpts, honcho_context for raw peer context, "
|
||||
"honcho_reasoning for synthesized answers (pass reasoning_level "
|
||||
"minimal/low/medium/high/max — you pick the depth per call), "
|
||||
"honcho_reasoning for synthesized answers, "
|
||||
"honcho_conclude to save facts about the user."
|
||||
)
|
||||
|
||||
@@ -563,10 +526,6 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
if self._injection_frequency == "first-turn" and self._turn_count > 1:
|
||||
return ""
|
||||
|
||||
# Trivial prompts ("ok", "yes", slash commands) carry no semantic signal.
|
||||
if self._is_trivial_prompt(query):
|
||||
return ""
|
||||
|
||||
parts = []
|
||||
|
||||
# ----- Layer 1: Base context (representation + card) -----
|
||||
@@ -601,72 +560,43 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
# On the very first turn, no queue_prefetch() has run yet so the
|
||||
# dialectic result is empty. Run with a bounded timeout so a slow
|
||||
# Honcho connection doesn't block the first response indefinitely.
|
||||
# On timeout we let the thread keep running and write its result into
|
||||
# _prefetch_result under the lock, so the next turn picks it up.
|
||||
#
|
||||
# Skip if the session-start prewarm already filled _prefetch_result —
|
||||
# firing another .chat() would be duplicate work.
|
||||
with self._prefetch_lock:
|
||||
_prewarm_landed = bool(self._prefetch_result)
|
||||
if _prewarm_landed and self._last_dialectic_turn == -999:
|
||||
self._last_dialectic_turn = self._turn_count
|
||||
|
||||
# On timeout the result is skipped and queue_prefetch() will pick it
|
||||
# up at the next cadence-allowed turn.
|
||||
if self._last_dialectic_turn == -999 and query:
|
||||
_first_turn_timeout = (
|
||||
self._config.timeout if self._config and self._config.timeout else 8.0
|
||||
)
|
||||
_fired_at = self._turn_count
|
||||
_result_holder: list[str] = []
|
||||
|
||||
def _run_first_turn() -> None:
|
||||
try:
|
||||
r = self._run_dialectic_depth(query)
|
||||
_result_holder.append(self._run_dialectic_depth(query))
|
||||
except Exception as exc:
|
||||
logger.debug("Honcho first-turn dialectic failed: %s", exc)
|
||||
self._dialectic_empty_streak += 1
|
||||
return
|
||||
if r and r.strip():
|
||||
with self._prefetch_lock:
|
||||
self._prefetch_result = r
|
||||
self._prefetch_result_fired_at = _fired_at
|
||||
# Advance cadence only on a non-empty result so the next
|
||||
# turn retries when the call returned nothing.
|
||||
self._last_dialectic_turn = _fired_at
|
||||
self._dialectic_empty_streak = 0
|
||||
else:
|
||||
self._dialectic_empty_streak += 1
|
||||
|
||||
self._prefetch_thread_started_at = time.monotonic()
|
||||
self._prefetch_thread = threading.Thread(
|
||||
target=_run_first_turn, daemon=True, name="honcho-prefetch-first"
|
||||
)
|
||||
self._prefetch_thread.start()
|
||||
self._prefetch_thread.join(timeout=_first_turn_timeout)
|
||||
if self._prefetch_thread.is_alive():
|
||||
_t = threading.Thread(target=_run_first_turn, daemon=True)
|
||||
_t.start()
|
||||
_t.join(timeout=_first_turn_timeout)
|
||||
if not _t.is_alive():
|
||||
first_turn_dialectic = _result_holder[0] if _result_holder else ""
|
||||
if first_turn_dialectic and first_turn_dialectic.strip():
|
||||
with self._prefetch_lock:
|
||||
self._prefetch_result = first_turn_dialectic
|
||||
self._last_dialectic_turn = self._turn_count
|
||||
else:
|
||||
logger.debug(
|
||||
"Honcho first-turn dialectic still running after %.1fs — "
|
||||
"will surface on next turn",
|
||||
"Honcho first-turn dialectic timed out (%.1fs) — "
|
||||
"will inject at next cadence-allowed turn",
|
||||
_first_turn_timeout,
|
||||
)
|
||||
# Don't update _last_dialectic_turn: queue_prefetch() will
|
||||
# retry at the next cadence-allowed turn via the async path.
|
||||
|
||||
if self._prefetch_thread and self._prefetch_thread.is_alive():
|
||||
self._prefetch_thread.join(timeout=3.0)
|
||||
with self._prefetch_lock:
|
||||
dialectic_result = self._prefetch_result
|
||||
fired_at = self._prefetch_result_fired_at
|
||||
self._prefetch_result = ""
|
||||
self._prefetch_result_fired_at = -999
|
||||
|
||||
# Discard stale pending results: if the fire happened more than
|
||||
# cadence × multiplier turns ago (e.g. a run of trivial-prompt turns
|
||||
# passed without consumption), the content likely no longer tracks
|
||||
# the current conversational pivot.
|
||||
stale_limit = self._dialectic_cadence * self._STALE_RESULT_MULTIPLIER
|
||||
if dialectic_result and fired_at >= 0 and (self._turn_count - fired_at) > stale_limit:
|
||||
logger.debug(
|
||||
"Honcho pending dialectic discarded as stale: fired_at=%d, "
|
||||
"turn=%d, limit=%d", fired_at, self._turn_count, stale_limit,
|
||||
)
|
||||
dialectic_result = ""
|
||||
|
||||
if dialectic_result and dialectic_result.strip():
|
||||
parts.append(dialectic_result)
|
||||
@@ -711,10 +641,6 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
if self._recall_mode == "tools":
|
||||
return
|
||||
|
||||
# Trivial prompts don't warrant either a context refresh or a dialectic call.
|
||||
if self._is_trivial_prompt(query):
|
||||
return
|
||||
|
||||
# ----- Context refresh (base layer) — independent cadence -----
|
||||
if self._context_cadence <= 1 or (self._turn_count - self._last_context_turn) >= self._context_cadence:
|
||||
self._last_context_turn = self._turn_count
|
||||
@@ -724,46 +650,24 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
logger.debug("Honcho context prefetch failed: %s", e)
|
||||
|
||||
# ----- Dialectic prefetch (supplement layer) -----
|
||||
# Thread-alive guard with stale-thread recovery: a hung Honcho call
|
||||
# older than timeout × multiplier is treated as dead so it can't
|
||||
# block subsequent fires.
|
||||
if self._thread_is_live():
|
||||
logger.debug("Honcho dialectic prefetch skipped: prior thread still running")
|
||||
return
|
||||
# B5: cadence check — skip if too soon since last dialectic call
|
||||
if self._dialectic_cadence > 1:
|
||||
if (self._turn_count - self._last_dialectic_turn) < self._dialectic_cadence:
|
||||
logger.debug("Honcho dialectic prefetch skipped: cadence %d, turns since last: %d",
|
||||
self._dialectic_cadence, self._turn_count - self._last_dialectic_turn)
|
||||
return
|
||||
|
||||
# Cadence gate, widened by the empty-streak backoff so a persistently
|
||||
# silent backend doesn't retry every turn forever.
|
||||
effective = self._effective_cadence()
|
||||
if (self._turn_count - self._last_dialectic_turn) < effective:
|
||||
logger.debug(
|
||||
"Honcho dialectic prefetch skipped: effective cadence %d "
|
||||
"(base %d, empty streak %d), turns since last: %d",
|
||||
effective, self._dialectic_cadence, self._dialectic_empty_streak,
|
||||
self._turn_count - self._last_dialectic_turn,
|
||||
)
|
||||
return
|
||||
|
||||
# Cadence advances only on a non-empty result so empty returns
|
||||
# (transient API error, sparse representation) retry next turn.
|
||||
_fired_at = self._turn_count
|
||||
self._last_dialectic_turn = self._turn_count
|
||||
|
||||
def _run():
|
||||
try:
|
||||
result = self._run_dialectic_depth(query)
|
||||
if result and result.strip():
|
||||
with self._prefetch_lock:
|
||||
self._prefetch_result = result
|
||||
except Exception as e:
|
||||
logger.debug("Honcho prefetch failed: %s", e)
|
||||
self._dialectic_empty_streak += 1
|
||||
return
|
||||
if result and result.strip():
|
||||
with self._prefetch_lock:
|
||||
self._prefetch_result = result
|
||||
self._prefetch_result_fired_at = _fired_at
|
||||
self._last_dialectic_turn = _fired_at
|
||||
self._dialectic_empty_streak = 0
|
||||
else:
|
||||
self._dialectic_empty_streak += 1
|
||||
|
||||
self._prefetch_thread_started_at = time.monotonic()
|
||||
self._prefetch_thread = threading.Thread(
|
||||
target=_run, daemon=True, name="honcho-prefetch"
|
||||
)
|
||||
@@ -788,91 +692,11 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
|
||||
_LEVEL_ORDER = ("minimal", "low", "medium", "high", "max")
|
||||
|
||||
# Char-count thresholds for the query-length reasoning heuristic.
|
||||
_HEURISTIC_LENGTH_MEDIUM = 120
|
||||
_HEURISTIC_LENGTH_HIGH = 400
|
||||
|
||||
# Liveness constants. A thread older than timeout × multiplier is treated
|
||||
# as dead so a hung Honcho call can't block future retries indefinitely.
|
||||
_STALE_THREAD_MULTIPLIER = 2.0
|
||||
# Pending result whose fire-turn is older than cadence × multiplier is
|
||||
# discarded on read so we don't inject context for a stale conversational
|
||||
# pivot after a gap of trivial-prompt turns.
|
||||
_STALE_RESULT_MULTIPLIER = 2
|
||||
# Cap on the empty-streak backoff so a persistently silent backend
|
||||
# eventually settles on a ceiling instead of unbounded widening.
|
||||
_BACKOFF_MAX = 8
|
||||
|
||||
def _thread_is_live(self) -> bool:
|
||||
"""Thread-alive guard that treats threads older than the stale
|
||||
threshold as dead, so a hung Honcho request can't block new fires."""
|
||||
if not self._prefetch_thread or not self._prefetch_thread.is_alive():
|
||||
return False
|
||||
timeout = (self._config.timeout if self._config and self._config.timeout else 8.0)
|
||||
age = time.monotonic() - self._prefetch_thread_started_at
|
||||
if age > timeout * self._STALE_THREAD_MULTIPLIER:
|
||||
logger.debug(
|
||||
"Honcho prefetch thread age %.1fs exceeds stale threshold "
|
||||
"%.1fs — treating as dead", age, timeout * self._STALE_THREAD_MULTIPLIER,
|
||||
)
|
||||
return False
|
||||
return True
|
||||
|
||||
def _effective_cadence(self) -> int:
|
||||
"""Cadence plus empty-streak backoff, capped at _BACKOFF_MAX × base."""
|
||||
if self._dialectic_empty_streak <= 0:
|
||||
return self._dialectic_cadence
|
||||
widened = self._dialectic_cadence + self._dialectic_empty_streak
|
||||
ceiling = self._dialectic_cadence * self._BACKOFF_MAX
|
||||
return min(widened, ceiling)
|
||||
|
||||
def liveness_snapshot(self) -> dict:
|
||||
"""In-process snapshot of dialectic liveness state for diagnostics.
|
||||
|
||||
Returns current turn, last successful dialectic turn, pending-result
|
||||
fire turn, empty streak, effective cadence, and thread status.
|
||||
"""
|
||||
thread_age = None
|
||||
if self._prefetch_thread and self._prefetch_thread.is_alive():
|
||||
thread_age = time.monotonic() - self._prefetch_thread_started_at
|
||||
return {
|
||||
"turn_count": self._turn_count,
|
||||
"last_dialectic_turn": self._last_dialectic_turn,
|
||||
"pending_result_fired_at": self._prefetch_result_fired_at,
|
||||
"empty_streak": self._dialectic_empty_streak,
|
||||
"effective_cadence": self._effective_cadence(),
|
||||
"thread_alive": thread_age is not None,
|
||||
"thread_age_seconds": thread_age,
|
||||
}
|
||||
|
||||
def _apply_reasoning_heuristic(self, base: str, query: str) -> str:
|
||||
"""Scale `base` up by query length, clamped at reasoning_level_cap.
|
||||
|
||||
Char-count heuristic: +1 at >=120 chars, +2 at >=400.
|
||||
"""
|
||||
if not self._reasoning_heuristic or not query:
|
||||
return base
|
||||
if base not in self._LEVEL_ORDER:
|
||||
return base
|
||||
n = len(query)
|
||||
if n < self._HEURISTIC_LENGTH_MEDIUM:
|
||||
bump = 0
|
||||
elif n < self._HEURISTIC_LENGTH_HIGH:
|
||||
bump = 1
|
||||
else:
|
||||
bump = 2
|
||||
base_idx = self._LEVEL_ORDER.index(base)
|
||||
cap_idx = self._LEVEL_ORDER.index(self._reasoning_level_cap)
|
||||
return self._LEVEL_ORDER[min(base_idx + bump, cap_idx)]
|
||||
|
||||
def _resolve_pass_level(self, pass_idx: int, query: str = "") -> str:
|
||||
def _resolve_pass_level(self, pass_idx: int) -> str:
|
||||
"""Resolve reasoning level for a given pass index.
|
||||
|
||||
Precedence:
|
||||
1. dialecticDepthLevels (explicit per-pass) — wins absolutely
|
||||
2. _PROPORTIONAL_LEVELS table (depth>1 lighter-early passes)
|
||||
3. Base level = dialecticReasoningLevel, optionally scaled by the
|
||||
reasoning heuristic when the mapping falls through to 'base'
|
||||
Uses dialecticDepthLevels if configured, otherwise proportional
|
||||
defaults relative to dialecticReasoningLevel.
|
||||
"""
|
||||
if self._dialectic_depth_levels and pass_idx < len(self._dialectic_depth_levels):
|
||||
return self._dialectic_depth_levels[pass_idx]
|
||||
@@ -880,7 +704,7 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
base = (self._config.dialectic_reasoning_level if self._config else "low")
|
||||
mapping = self._PROPORTIONAL_LEVELS.get((self._dialectic_depth, pass_idx))
|
||||
if mapping is None or mapping == "base":
|
||||
return self._apply_reasoning_heuristic(base, query)
|
||||
return base
|
||||
return mapping
|
||||
|
||||
def _build_dialectic_prompt(self, pass_idx: int, prior_results: list[str], is_cold: bool) -> str:
|
||||
@@ -967,7 +791,7 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
break
|
||||
prompt = self._build_dialectic_prompt(i, results, is_cold)
|
||||
|
||||
level = self._resolve_pass_level(i, query=query)
|
||||
level = self._resolve_pass_level(i)
|
||||
logger.debug("Honcho dialectic depth %d: pass %d, level=%s, cold=%s",
|
||||
self._dialectic_depth, i, level, is_cold)
|
||||
|
||||
@@ -984,29 +808,6 @@ class HonchoMemoryProvider(MemoryProvider):
|
||||
return r
|
||||
return ""
|
||||
|
||||
# Prompts that carry no semantic signal — trivial acknowledgements, slash
|
||||
# commands, empty input. Skipping injection here saves tokens and prevents
|
||||
# stale user-model context from derailing one-word replies.
|
||||
_TRIVIAL_PROMPT_RE = re.compile(
|
||||
r'^(yes|no|ok|okay|sure|thanks|thank you|y|n|yep|nope|yeah|nah|'
|
||||
r'continue|go ahead|do it|proceed|got it|cool|nice|great|done|next|lgtm|k)$',
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _is_trivial_prompt(cls, text: str) -> bool:
|
||||
"""Return True if the prompt is too trivial to warrant context injection."""
|
||||
if not text:
|
||||
return True
|
||||
stripped = text.strip()
|
||||
if not stripped:
|
||||
return True
|
||||
if stripped.startswith("/"):
|
||||
return True
|
||||
if cls._TRIVIAL_PROMPT_RE.match(stripped):
|
||||
return True
|
||||
return False
|
||||
|
||||
def on_turn_start(self, turn_number: int, message: str, **kwargs) -> None:
|
||||
"""Track turn count for cadence and injection_frequency logic."""
|
||||
self._turn_count = turn_number
|
||||
|
||||
@@ -460,37 +460,17 @@ def cmd_setup(args) -> None:
|
||||
pass # keep current
|
||||
|
||||
# --- 7b. Dialectic cadence ---
|
||||
current_dialectic = str(hermes_host.get("dialecticCadence") or cfg.get("dialecticCadence") or "2")
|
||||
current_dialectic = str(hermes_host.get("dialecticCadence") or cfg.get("dialecticCadence") or "3")
|
||||
print("\n Dialectic cadence:")
|
||||
print(" How often Honcho rebuilds its user model (LLM call on Honcho backend).")
|
||||
print(" 1 = every turn, 2 = every other turn, 3+ = sparser.")
|
||||
print(" Recommended: 1-5.")
|
||||
print(" 1 = every turn (aggressive), 3 = every 3 turns (recommended), 5+ = sparse.")
|
||||
new_dialectic = _prompt("Dialectic cadence", default=current_dialectic)
|
||||
try:
|
||||
val = int(new_dialectic)
|
||||
if val >= 1:
|
||||
hermes_host["dialecticCadence"] = val
|
||||
except (ValueError, TypeError):
|
||||
hermes_host["dialecticCadence"] = 2
|
||||
|
||||
# --- 7c. Dialectic reasoning level ---
|
||||
current_reasoning = (
|
||||
hermes_host.get("dialecticReasoningLevel")
|
||||
or cfg.get("dialecticReasoningLevel")
|
||||
or "low"
|
||||
)
|
||||
print("\n Dialectic reasoning level:")
|
||||
print(" Depth Honcho uses when synthesizing user context on auto-injected calls.")
|
||||
print(" minimal -- quick factual lookups")
|
||||
print(" low -- straightforward questions (default)")
|
||||
print(" medium -- multi-aspect synthesis")
|
||||
print(" high -- complex behavioral patterns")
|
||||
print(" max -- thorough audit-level analysis")
|
||||
new_reasoning = _prompt("Reasoning level", default=current_reasoning)
|
||||
if new_reasoning in ("minimal", "low", "medium", "high", "max"):
|
||||
hermes_host["dialecticReasoningLevel"] = new_reasoning
|
||||
else:
|
||||
hermes_host["dialecticReasoningLevel"] = "low"
|
||||
hermes_host["dialecticCadence"] = 3
|
||||
|
||||
# --- 8. Session strategy ---
|
||||
current_strat = hermes_host.get("sessionStrategy") or cfg.get("sessionStrategy", "per-session")
|
||||
@@ -656,11 +636,8 @@ def cmd_status(args) -> None:
|
||||
print(f" Recall mode: {hcfg.recall_mode}")
|
||||
print(f" Context budget: {hcfg.context_tokens or '(uncapped)'} tokens")
|
||||
raw = getattr(hcfg, "raw", None) or {}
|
||||
dialectic_cadence = raw.get("dialecticCadence") or 1
|
||||
dialectic_cadence = raw.get("dialecticCadence") or 3
|
||||
print(f" Dialectic cad: every {dialectic_cadence} turn{'s' if dialectic_cadence != 1 else ''}")
|
||||
reasoning_cap = raw.get("reasoningLevelCap") or hcfg.reasoning_level_cap
|
||||
heuristic_on = "on" if hcfg.reasoning_heuristic else "off"
|
||||
print(f" Reasoning: base={hcfg.dialectic_reasoning_level}, cap={reasoning_cap}, heuristic={heuristic_on}")
|
||||
print(f" Observation: user(me={hcfg.user_observe_me},others={hcfg.user_observe_others}) ai(me={hcfg.ai_observe_me},others={hcfg.ai_observe_others})")
|
||||
print(f" Write freq: {hcfg.write_frequency}")
|
||||
|
||||
|
||||
@@ -251,11 +251,6 @@ class HonchoClientConfig:
|
||||
# matching dialectic_depth length. When None, uses proportional defaults
|
||||
# derived from dialectic_reasoning_level.
|
||||
dialectic_depth_levels: list[str] | None = None
|
||||
# When true, the auto-injected dialectic scales reasoning level up on
|
||||
# longer queries. See HonchoMemoryProvider for thresholds.
|
||||
reasoning_heuristic: bool = True
|
||||
# Ceiling for the heuristic-selected reasoning level.
|
||||
reasoning_level_cap: str = "high"
|
||||
# Honcho API limits — configurable for self-hosted instances
|
||||
# Max chars per message sent via add_messages() (Honcho cloud: 25000)
|
||||
message_max_chars: int = 25000
|
||||
@@ -451,16 +446,6 @@ class HonchoClientConfig:
|
||||
raw.get("dialecticDepthLevels"),
|
||||
depth=_parse_dialectic_depth(host_block.get("dialecticDepth"), raw.get("dialecticDepth")),
|
||||
),
|
||||
reasoning_heuristic=_resolve_bool(
|
||||
host_block.get("reasoningHeuristic"),
|
||||
raw.get("reasoningHeuristic"),
|
||||
default=True,
|
||||
),
|
||||
reasoning_level_cap=(
|
||||
host_block.get("reasoningLevelCap")
|
||||
or raw.get("reasoningLevelCap")
|
||||
or "high"
|
||||
),
|
||||
message_max_chars=int(
|
||||
host_block.get("messageMaxChars")
|
||||
or raw.get("messageMaxChars")
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user