Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2b4062f964 | |||
| 1dba69095d | |||
| c96eb06dc4 | |||
| f6cfff4ac6 |
@@ -423,24 +423,3 @@ IMAGE_TOOLS_DEBUG=false
|
|||||||
# TEAMS_HOME_CHANNEL= # Default channel/chat ID for cron delivery
|
# TEAMS_HOME_CHANNEL= # Default channel/chat ID for cron delivery
|
||||||
# TEAMS_HOME_CHANNEL_NAME= # Display name for the home channel
|
# TEAMS_HOME_CHANNEL_NAME= # Display name for the home channel
|
||||||
# TEAMS_PORT=3978 # Webhook listen port (Bot Framework default)
|
# TEAMS_PORT=3978 # Webhook listen port (Bot Framework default)
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# GOOGLE CHAT INTEGRATION
|
|
||||||
# =============================================================================
|
|
||||||
# Connects via Cloud Pub/Sub pull subscription (no public URL required).
|
|
||||||
# Setup walkthrough: website/docs/user-guide/messaging/google_chat.md.
|
|
||||||
# 1. Create a GCP project, enable the Google Chat API and Cloud Pub/Sub.
|
|
||||||
# 2. Create a Service Account with roles/pubsub.subscriber on the
|
|
||||||
# subscription (NOT project-wide); download the JSON key.
|
|
||||||
# 3. Configure your Chat app at console.cloud.google.com/apis/credentials
|
|
||||||
# → Google Chat API → Configuration → Cloud Pub/Sub topic.
|
|
||||||
# 4. (Optional, for native attachment delivery) Each user runs
|
|
||||||
# `/setup-files` once in their own DM after Pub/Sub is wired up.
|
|
||||||
#
|
|
||||||
# GOOGLE_CHAT_PROJECT_ID= # GCP project hosting the topic (or set GOOGLE_CLOUD_PROJECT)
|
|
||||||
# GOOGLE_CHAT_SUBSCRIPTION_NAME= # Full path: projects/<id>/subscriptions/<name>
|
|
||||||
# GOOGLE_CHAT_SERVICE_ACCOUNT_JSON= # Path to SA JSON (or set GOOGLE_APPLICATION_CREDENTIALS)
|
|
||||||
# GOOGLE_CHAT_ALLOWED_USERS= # Comma-separated emails allowed to talk to the bot
|
|
||||||
# GOOGLE_CHAT_ALLOW_ALL_USERS=false # Set true to skip the allowlist
|
|
||||||
# GOOGLE_CHAT_HOME_CHANNEL= # Default space (spaces/XXXX) for cron delivery
|
|
||||||
# GOOGLE_CHAT_HOME_CHANNEL_NAME= # Display name for the home channel
|
|
||||||
|
|||||||
@@ -16,13 +16,9 @@ on:
|
|||||||
permissions:
|
permissions:
|
||||||
contents: read
|
contents: read
|
||||||
|
|
||||||
# Top-level concurrency: do NOT cancel in-flight builds when a new push lands.
|
|
||||||
# Every commit deserves its own SHA-tagged image in the registry, and we guard
|
|
||||||
# the :latest tag in a separate job below (with its own concurrency group) so
|
|
||||||
# a slow run can't clobber :latest with older bits.
|
|
||||||
concurrency:
|
concurrency:
|
||||||
group: docker-${{ github.ref }}
|
group: docker-${{ github.ref }}
|
||||||
cancel-in-progress: false
|
cancel-in-progress: true
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
build-and-push:
|
build-and-push:
|
||||||
@@ -30,18 +26,11 @@ jobs:
|
|||||||
if: github.repository == 'NousResearch/hermes-agent'
|
if: github.repository == 'NousResearch/hermes-agent'
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
timeout-minutes: 60
|
timeout-minutes: 60
|
||||||
outputs:
|
|
||||||
pushed_sha_tag: ${{ steps.mark_pushed.outputs.pushed }}
|
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout code
|
- name: Checkout code
|
||||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||||
with:
|
with:
|
||||||
submodules: recursive
|
submodules: recursive
|
||||||
# Fetch enough history to run `git merge-base --is-ancestor` in the
|
|
||||||
# move-latest job. That job reuses this checkout via its own
|
|
||||||
# actions/checkout call, but commits reachable from main up to ~1000
|
|
||||||
# back are plenty for any realistic race window.
|
|
||||||
fetch-depth: 1000
|
|
||||||
|
|
||||||
- name: Set up QEMU
|
- name: Set up QEMU
|
||||||
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
|
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
|
||||||
@@ -65,31 +54,19 @@ jobs:
|
|||||||
|
|
||||||
- name: Test image starts
|
- name: Test image starts
|
||||||
run: |
|
run: |
|
||||||
mkdir -p /tmp/hermes-test
|
|
||||||
sudo chown -R 10000:10000 /tmp/hermes-test
|
|
||||||
# The image runs as the hermes user (UID 10000). GitHub Actions
|
# The image runs as the hermes user (UID 10000). GitHub Actions
|
||||||
# creates /tmp/hermes-test root-owned by default, which hermes
|
# creates /tmp/hermes-test root-owned by default, which hermes
|
||||||
# can't write to — chown it to match the in-container UID before
|
# can't write to — chown it to match the in-container UID before
|
||||||
# bind-mounting. Real users doing `docker run -v ~/.hermes:...`
|
# bind-mounting. Real users doing `docker run -v ~/.hermes:...`
|
||||||
# with their own UID hit the same issue and have their own
|
# with their own UID hit the same issue and have their own
|
||||||
# remediations (HERMES_UID env var, or chown locally).
|
# remediations (HERMES_UID env var, or chown locally).
|
||||||
|
mkdir -p /tmp/hermes-test
|
||||||
|
sudo chown -R 10000:10000 /tmp/hermes-test
|
||||||
docker run --rm \
|
docker run --rm \
|
||||||
-v /tmp/hermes-test:/opt/data \
|
-v /tmp/hermes-test:/opt/data \
|
||||||
--entrypoint /opt/hermes/docker/entrypoint.sh \
|
--entrypoint /opt/hermes/docker/entrypoint.sh \
|
||||||
nousresearch/hermes-agent:test --help
|
nousresearch/hermes-agent:test --help
|
||||||
|
|
||||||
- name: Test dashboard subcommand
|
|
||||||
run: |
|
|
||||||
mkdir -p /tmp/hermes-test
|
|
||||||
sudo chown -R 10000:10000 /tmp/hermes-test
|
|
||||||
# Verify the dashboard subcommand is included in the Docker image.
|
|
||||||
# This prevents regressions like #9153 where the dashboard command
|
|
||||||
# was present in source but missing from the published image.
|
|
||||||
docker run --rm \
|
|
||||||
-v /tmp/hermes-test:/opt/data \
|
|
||||||
--entrypoint /opt/hermes/docker/entrypoint.sh \
|
|
||||||
nousresearch/hermes-agent:test dashboard --help
|
|
||||||
|
|
||||||
- name: Log in to Docker Hub
|
- name: Log in to Docker Hub
|
||||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main' || github.event_name == 'release'
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main' || github.event_name == 'release'
|
||||||
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
|
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
|
||||||
@@ -97,12 +74,7 @@ jobs:
|
|||||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||||
|
|
||||||
# Always push a per-commit SHA tag on main. This is race-free because
|
- name: Push multi-arch image (main branch)
|
||||||
# every commit has a unique SHA — concurrent runs can't clobber each
|
|
||||||
# other here. We also embed the git SHA as an OCI label so the
|
|
||||||
# move-latest job (below) can read it back off the registry's `:latest`.
|
|
||||||
- name: Push multi-arch image with SHA tag (main branch)
|
|
||||||
id: push_sha
|
|
||||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||||
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
||||||
with:
|
with:
|
||||||
@@ -110,17 +82,10 @@ jobs:
|
|||||||
file: Dockerfile
|
file: Dockerfile
|
||||||
push: true
|
push: true
|
||||||
platforms: linux/amd64,linux/arm64
|
platforms: linux/amd64,linux/arm64
|
||||||
tags: nousresearch/hermes-agent:sha-${{ github.sha }}
|
tags: nousresearch/hermes-agent:latest
|
||||||
labels: |
|
|
||||||
org.opencontainers.image.revision=${{ github.sha }}
|
|
||||||
cache-from: type=gha
|
cache-from: type=gha
|
||||||
cache-to: type=gha,mode=max
|
cache-to: type=gha,mode=max
|
||||||
|
|
||||||
- name: Mark SHA tag pushed
|
|
||||||
id: mark_pushed
|
|
||||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
||||||
run: echo "pushed=true" >> "$GITHUB_OUTPUT"
|
|
||||||
|
|
||||||
- name: Push multi-arch image (release)
|
- name: Push multi-arch image (release)
|
||||||
if: github.event_name == 'release'
|
if: github.event_name == 'release'
|
||||||
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
||||||
@@ -132,119 +97,3 @@ jobs:
|
|||||||
tags: nousresearch/hermes-agent:${{ github.event.release.tag_name }}
|
tags: nousresearch/hermes-agent:${{ github.event.release.tag_name }}
|
||||||
cache-from: type=gha
|
cache-from: type=gha
|
||||||
cache-to: type=gha,mode=max
|
cache-to: type=gha,mode=max
|
||||||
|
|
||||||
# Second job: moves `:latest` to point at the SHA tag the first job pushed.
|
|
||||||
#
|
|
||||||
# Has its own concurrency group with `cancel-in-progress: true`, which
|
|
||||||
# gives us the serialization we need: if a newer push arrives while an
|
|
||||||
# older run is mid-way through this job, the older run is cancelled
|
|
||||||
# before it can clobber `:latest`. Combined with the ancestor check
|
|
||||||
# below, this means `:latest` only ever moves forward in git history.
|
|
||||||
move-latest:
|
|
||||||
if: |
|
|
||||||
github.repository == 'NousResearch/hermes-agent'
|
|
||||||
&& github.event_name == 'push'
|
|
||||||
&& github.ref == 'refs/heads/main'
|
|
||||||
&& needs.build-and-push.outputs.pushed_sha_tag == 'true'
|
|
||||||
needs: build-and-push
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
timeout-minutes: 10
|
|
||||||
concurrency:
|
|
||||||
group: docker-move-latest-${{ github.ref }}
|
|
||||||
cancel-in-progress: true
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
|
||||||
with:
|
|
||||||
fetch-depth: 1000
|
|
||||||
|
|
||||||
- name: Set up Docker Buildx
|
|
||||||
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
|
||||||
|
|
||||||
- name: Log in to Docker Hub
|
|
||||||
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
|
|
||||||
with:
|
|
||||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
||||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
||||||
|
|
||||||
# Read the git revision label off the current `:latest` manifest, then
|
|
||||||
# use `git merge-base --is-ancestor` to check whether our commit is a
|
|
||||||
# descendant of it. If `:latest` doesn't exist yet, or its label is
|
|
||||||
# missing, we treat that as "safe to publish". If another run already
|
|
||||||
# advanced `:latest` past us (or diverged), we skip and leave it alone.
|
|
||||||
- name: Decide whether to move :latest
|
|
||||||
id: latest_check
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
image=nousresearch/hermes-agent
|
|
||||||
|
|
||||||
# Pull the JSON for the linux/amd64 sub-manifest's config and extract
|
|
||||||
# the OCI revision label with jq — Go template field access can't
|
|
||||||
# handle dots in map keys, so using json+jq is the robust route.
|
|
||||||
image_json=$(
|
|
||||||
docker buildx imagetools inspect "${image}:latest" \
|
|
||||||
--format '{{ json (index .Image "linux/amd64") }}' \
|
|
||||||
2>/dev/null || true
|
|
||||||
)
|
|
||||||
|
|
||||||
if [ -z "${image_json}" ]; then
|
|
||||||
echo "No existing :latest (or inspect failed) — safe to publish."
|
|
||||||
echo "push_latest=true" >> "$GITHUB_OUTPUT"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
current_sha=$(
|
|
||||||
printf '%s' "${image_json}" \
|
|
||||||
| jq -r '.config.Labels."org.opencontainers.image.revision" // ""'
|
|
||||||
)
|
|
||||||
|
|
||||||
if [ -z "${current_sha}" ]; then
|
|
||||||
echo "Registry :latest has no revision label — safe to publish."
|
|
||||||
echo "push_latest=true" >> "$GITHUB_OUTPUT"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "Registry :latest is at ${current_sha}"
|
|
||||||
echo "This run is at ${GITHUB_SHA}"
|
|
||||||
|
|
||||||
if [ "${current_sha}" = "${GITHUB_SHA}" ]; then
|
|
||||||
echo ":latest already points at our SHA — nothing to do."
|
|
||||||
echo "push_latest=false" >> "$GITHUB_OUTPUT"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Make sure we have the :latest commit locally for merge-base.
|
|
||||||
if ! git cat-file -e "${current_sha}^{commit}" 2>/dev/null; then
|
|
||||||
git fetch --no-tags --prune origin \
|
|
||||||
"+refs/heads/main:refs/remotes/origin/main" \
|
|
||||||
|| true
|
|
||||||
fi
|
|
||||||
|
|
||||||
if ! git cat-file -e "${current_sha}^{commit}" 2>/dev/null; then
|
|
||||||
echo "Registry :latest points at an unknown commit (${current_sha}); refusing to overwrite."
|
|
||||||
echo "push_latest=false" >> "$GITHUB_OUTPUT"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Our SHA must be a descendant of the current :latest to be safe.
|
|
||||||
if git merge-base --is-ancestor "${current_sha}" "${GITHUB_SHA}"; then
|
|
||||||
echo "Our commit is a descendant of :latest — safe to advance."
|
|
||||||
echo "push_latest=true" >> "$GITHUB_OUTPUT"
|
|
||||||
else
|
|
||||||
echo "Another run advanced :latest past us (or diverged) — leaving it alone."
|
|
||||||
echo "push_latest=false" >> "$GITHUB_OUTPUT"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Retag the already-pushed SHA manifest as :latest. This is a registry-
|
|
||||||
# side operation — no rebuild, no layer re-push — so it's quick and
|
|
||||||
# atomic per-tag. The ancestor check above plus the cancel-in-progress
|
|
||||||
# concurrency on this job together guarantee we only ever move :latest
|
|
||||||
# forward in git history.
|
|
||||||
- name: Move :latest to this SHA
|
|
||||||
if: steps.latest_check.outputs.push_latest == 'true'
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
image=nousresearch/hermes-agent
|
|
||||||
docker buildx imagetools create \
|
|
||||||
--tag "${image}:latest" \
|
|
||||||
"${image}:sha-${GITHUB_SHA}"
|
|
||||||
|
|||||||
@@ -48,12 +48,16 @@ jobs:
|
|||||||
|
|
||||||
- name: Determine base ref
|
- name: Determine base ref
|
||||||
id: base
|
id: base
|
||||||
|
env:
|
||||||
|
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||||
run: |
|
run: |
|
||||||
# For PRs, diff against the merge base with the target branch.
|
# For PRs, diff against the PR's pinned parent commit
|
||||||
|
# (github.event.pull_request.base.sha — snapshot at PR open time,
|
||||||
|
# so later pushes to main don't leak into the diff).
|
||||||
# For pushes to main, diff against the previous commit on main.
|
# For pushes to main, diff against the previous commit on main.
|
||||||
if [ "${{ github.event_name }}" = "pull_request" ]; then
|
if [ "${{ github.event_name }}" = "pull_request" ]; then
|
||||||
BASE_SHA=$(git merge-base "origin/${{ github.base_ref }}" HEAD)
|
BASE_SHA="${PR_BASE_SHA}"
|
||||||
BASE_REF="origin/${{ github.base_ref }}"
|
BASE_REF="PR base (${BASE_SHA:0:7})"
|
||||||
else
|
else
|
||||||
BASE_SHA=$(git rev-parse HEAD~1 2>/dev/null || git rev-parse HEAD)
|
BASE_SHA=$(git rev-parse HEAD~1 2>/dev/null || git rev-parse HEAD)
|
||||||
BASE_REF="HEAD~1"
|
BASE_REF="HEAD~1"
|
||||||
@@ -117,6 +121,9 @@ jobs:
|
|||||||
name: lint-reports
|
name: lint-reports
|
||||||
path: .lint-reports/
|
path: .lint-reports/
|
||||||
retention-days: 14
|
retention-days: 14
|
||||||
|
# .lint-reports/ is a dotfile-prefixed directory, and upload-artifact@v4
|
||||||
|
# skips hidden files by default (breaking change from v3). Opt back in.
|
||||||
|
include-hidden-files: true
|
||||||
|
|
||||||
- name: Post / update PR comment
|
- name: Post / update PR comment
|
||||||
if: github.event_name == 'pull_request'
|
if: github.event_name == 'pull_request'
|
||||||
|
|||||||
+9
-16
@@ -106,11 +106,6 @@ hermes chat -q "Hello"
|
|||||||
### Run tests
|
### Run tests
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Preferred — matches CI (hermetic env, 4 xdist workers); see AGENTS.md
|
|
||||||
scripts/run_tests.sh
|
|
||||||
|
|
||||||
# Alternative (activate the venv first). The wrapper is still recommended
|
|
||||||
# for parity with GitHub Actions before you open a PR:
|
|
||||||
pytest tests/ -v
|
pytest tests/ -v
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -291,18 +286,16 @@ registry.register(
|
|||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Wire into a toolset (required):** Built-in tools are auto-discovered: any
|
Then add the import to `model_tools.py` in the `_modules` list:
|
||||||
`tools/*.py` file that contains a top-level `registry.register(...)` call is
|
|
||||||
imported by `discover_builtin_tools()` in `tools/registry.py` when `model_tools`
|
|
||||||
loads. There is **no** manual import list in `model_tools.py` to maintain.
|
|
||||||
|
|
||||||
You must still add the tool name to the appropriate list in `toolsets.py`
|
```python
|
||||||
(for example `_HERMES_CORE_TOOLS` or a dedicated toolset); otherwise the tool
|
_modules = [
|
||||||
registers but is never exposed to the agent. If you introduce a new toolset,
|
# ... existing modules ...
|
||||||
add it in `toolsets.py` and wire it into the relevant platform presets.
|
"tools.my_tool",
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
See `AGENTS.md` (section **Adding New Tools**) for profile-aware paths and
|
If it's a new toolset, add it to `toolsets.py` and to the relevant platform presets.
|
||||||
plugin vs core guidance.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -602,7 +595,7 @@ refactor/description # Code restructuring
|
|||||||
|
|
||||||
### Before submitting
|
### Before submitting
|
||||||
|
|
||||||
1. **Run tests**: `scripts/run_tests.sh` (recommended; same as CI) or `pytest tests/ -v` with the project venv activated
|
1. **Run tests**: `pytest tests/ -v`
|
||||||
2. **Test manually**: Run `hermes` and exercise the code path you changed
|
2. **Test manually**: Run `hermes` and exercise the code path you changed
|
||||||
3. **Check cross-platform impact**: If you touch file I/O, process management, or terminal handling, consider macOS, Linux, and WSL2
|
3. **Check cross-platform impact**: If you touch file I/O, process management, or terminal handling, consider macOS, Linux, and WSL2
|
||||||
4. **Keep PRs focused**: One logical change per PR. Don't mix a bug fix with a refactor with a new feature.
|
4. **Keep PRs focused**: One logical change per PR. Don't mix a bug fix with a refactor with a new feature.
|
||||||
|
|||||||
+1
-7
@@ -66,14 +66,8 @@ RUN cd web && npm run build && \
|
|||||||
# ---------- Permissions ----------
|
# ---------- Permissions ----------
|
||||||
# Make install dir world-readable so any HERMES_UID can read it at runtime.
|
# Make install dir world-readable so any HERMES_UID can read it at runtime.
|
||||||
# The venv needs to be traversable too.
|
# The venv needs to be traversable too.
|
||||||
# node_modules trees additionally need to be writable by the hermes user
|
|
||||||
# so the runtime `npm install` triggered by _tui_need_npm_install() in
|
|
||||||
# hermes_cli/main.py succeeds (see #18800). /opt/hermes/web is build-time
|
|
||||||
# only (HERMES_WEB_DIST points at hermes_cli/web_dist) and is intentionally
|
|
||||||
# not chowned here.
|
|
||||||
USER root
|
USER root
|
||||||
RUN chmod -R a+rX /opt/hermes && \
|
RUN chmod -R a+rX /opt/hermes
|
||||||
chown -R hermes:hermes /opt/hermes/ui-tui /opt/hermes/node_modules
|
|
||||||
# Start as root so the entrypoint can usermod/groupmod + gosu.
|
# Start as root so the entrypoint can usermod/groupmod + gosu.
|
||||||
# If HERMES_UID is unset, the entrypoint drops to the default hermes user (10000).
|
# If HERMES_UID is unset, the entrypoint drops to the default hermes user (10000).
|
||||||
|
|
||||||
|
|||||||
@@ -30,27 +30,15 @@ Use any model you want — [Nous Portal](https://portal.nousresearch.com), [Open
|
|||||||
|
|
||||||
## Quick Install
|
## Quick Install
|
||||||
|
|
||||||
### Linux, macOS, WSL2, Termux
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
|
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
|
||||||
```
|
```
|
||||||
|
|
||||||
### Windows (native, PowerShell)
|
Works on Linux, macOS, WSL2, and Android via Termux. The installer handles the platform-specific setup for you.
|
||||||
|
|
||||||
Run this in PowerShell:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
irm https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.ps1 | iex
|
|
||||||
```
|
|
||||||
|
|
||||||
The installer handles everything: uv, Python 3.11, Node.js, ripgrep, ffmpeg, **and a portable Git Bash** (MinGit, unpacked to `%LOCALAPPDATA%\hermes\git` — no admin required, completely isolated from any system Git install). Hermes uses this bundled Git Bash to run shell commands.
|
|
||||||
|
|
||||||
If you already have Git installed, the installer detects it and uses that instead. Otherwise a ~45MB MinGit download is all you need — it won't touch or interfere with any system Git.
|
|
||||||
|
|
||||||
> **Android / Termux:** The tested manual path is documented in the [Termux guide](https://hermes-agent.nousresearch.com/docs/getting-started/termux). On Termux, Hermes installs a curated `.[termux]` extra because the full `.[all]` extra currently pulls Android-incompatible voice dependencies.
|
> **Android / Termux:** The tested manual path is documented in the [Termux guide](https://hermes-agent.nousresearch.com/docs/getting-started/termux). On Termux, Hermes installs a curated `.[termux]` extra because the full `.[all]` extra currently pulls Android-incompatible voice dependencies.
|
||||||
>
|
>
|
||||||
> **Windows:** Native Windows is supported — the PowerShell one-liner above installs everything. If you'd rather use WSL2, the Linux command works there too. Native Windows install lives under `%LOCALAPPDATA%\hermes`; WSL2 installs under `~/.hermes` as on Linux. The only Hermes feature that currently needs WSL2 specifically is the browser-based dashboard chat pane (it uses a POSIX PTY — classic CLI and gateway both run natively).
|
> **Windows:** Native Windows is not supported. Please install [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install) and run the command above.
|
||||||
|
|
||||||
After installation:
|
After installation:
|
||||||
|
|
||||||
@@ -167,13 +155,13 @@ Manual path (equivalent to the above):
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||||
uv venv .venv --python 3.11
|
uv venv venv --python 3.11
|
||||||
source .venv/bin/activate
|
source venv/bin/activate
|
||||||
uv pip install -e ".[all,dev]"
|
uv pip install -e ".[all,dev]"
|
||||||
scripts/run_tests.sh
|
scripts/run_tests.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
> **RL Training (optional):** The RL/Atropos integration (`environments/`) — see [`CONTRIBUTING.md`](https://github.com/NousResearch/hermes-agent/blob/main/CONTRIBUTING.md#development-setup) for the full setup.
|
> **RL Training (optional):** The RL/Atropos integration (`environments/`) ships via the `atroposlib` and `tinker` dependencies pulled in by `.[all,dev]` — no submodule setup required.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -1,641 +0,0 @@
|
|||||||
# Hermes Agent v0.13.0 (v2026.5.7)
|
|
||||||
|
|
||||||
**Release Date:** May 7, 2026
|
|
||||||
**Since v0.12.0:** 864 commits · 588 merged PRs · 829 files changed · 128,366 insertions · 282 issues closed (13 P0, 36 P1) · 295 community contributors (including co-authors)
|
|
||||||
|
|
||||||
> The Tenacity Release — Hermes Agent now finishes what it starts. Kanban ships as a durable multi-agent board (heartbeat, reclaim, zombie detection, auto-block on incomplete exit, per-task retries, hallucination recovery). `/goal` keeps the agent locked on a target across turns (Ralph loop). Checkpoints v2 rewrites state persistence with real pruning. Gateway auto-resumes interrupted sessions after restart. Cron grows a `no_agent` watchdog mode. A security wave closes 8 P0s — redaction is now ON by default, Discord role-allowlists are guild-scoped, WhatsApp rejects strangers by default, and TOCTOU windows close across auth.json and MCP OAuth. Google Chat becomes the 20th platform. Providers become a pluggable surface. Seven i18n locales ship.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✨ Highlights
|
|
||||||
|
|
||||||
- **Multi-agent Kanban — delegate to an AI team that actually finishes** — Spin up a durable board, drop tasks on it, and let multiple Hermes workers pick them up, hand off, and close them out. Heartbeats, reclaim, zombie detection, retry budgets, and a hallucination gate keep the team honest. One install, many kanbans. ([#17805](https://github.com/NousResearch/hermes-agent/pull/17805), [#19653](https://github.com/NousResearch/hermes-agent/pull/19653), [#20232](https://github.com/NousResearch/hermes-agent/pull/20232), [#20332](https://github.com/NousResearch/hermes-agent/pull/20332), [#21330](https://github.com/NousResearch/hermes-agent/pull/21330), [#21183](https://github.com/NousResearch/hermes-agent/pull/21183), [#21214](https://github.com/NousResearch/hermes-agent/pull/21214))
|
|
||||||
|
|
||||||
- **`/goal` — the agent doesn't forget what you asked it to do** — Lock the agent onto a target and it stays on task across turns. The Ralph loop as a first-class primitive. ([#18262](https://github.com/NousResearch/hermes-agent/pull/18262), [#18275](https://github.com/NousResearch/hermes-agent/pull/18275), [#21287](https://github.com/NousResearch/hermes-agent/pull/21287))
|
|
||||||
|
|
||||||
- **Show it a video** — new `video_analyze` tool for native video understanding on Gemini and compatible multimodal models. (@alt-glitch) ([#19301](https://github.com/NousResearch/hermes-agent/pull/19301))
|
|
||||||
|
|
||||||
- **Clone a voice** — xAI Custom Voices lands as a TTS provider with voice cloning support. (@alt-glitch) ([#18776](https://github.com/NousResearch/hermes-agent/pull/18776))
|
|
||||||
|
|
||||||
- **Hermes speaks your language** — static gateway + CLI messages translate to 7 locales: Chinese, Japanese, German, Spanish, French, Ukrainian, and Turkish. Docs site gains a Chinese (zh-Hans) locale. ([#20231](https://github.com/NousResearch/hermes-agent/pull/20231), [#20329](https://github.com/NousResearch/hermes-agent/pull/20329), [#20467](https://github.com/NousResearch/hermes-agent/pull/20467), [#20474](https://github.com/NousResearch/hermes-agent/pull/20474), [#20430](https://github.com/NousResearch/hermes-agent/pull/20430), [#20431](https://github.com/NousResearch/hermes-agent/pull/20431))
|
|
||||||
|
|
||||||
- **Google Chat — the 20th messaging platform** — plus a generic platform-plugin hooks surface so third-party adapters drop in without touching core (IRC and Teams migrated). ([#21306](https://github.com/NousResearch/hermes-agent/pull/21306), [#21331](https://github.com/NousResearch/hermes-agent/pull/21331))
|
|
||||||
|
|
||||||
- **Sessions survive restarts** — gateway bounces mid-agent, `/update` restarts, source-file reloads — conversations auto-resume when the gateway comes back. ([#21192](https://github.com/NousResearch/hermes-agent/pull/21192))
|
|
||||||
|
|
||||||
- **Security wave — 8 P0 closures** — redaction ON by default, Discord role-allowlists guild-scoped (CVSS 8.1 cross-guild DM bypass closed), WhatsApp rejects strangers by default, TOCTOU windows closed across `auth.json` and MCP OAuth, browser enforces cloud-metadata SSRF floor, cron prompt-injection scans assembled skill content, `hermes debug share` redacts at upload. ([#21193](https://github.com/NousResearch/hermes-agent/pull/21193), [#21241](https://github.com/NousResearch/hermes-agent/pull/21241), [#21291](https://github.com/NousResearch/hermes-agent/pull/21291), [#21176](https://github.com/NousResearch/hermes-agent/pull/21176), [#21194](https://github.com/NousResearch/hermes-agent/pull/21194), [#21228](https://github.com/NousResearch/hermes-agent/pull/21228), [#21350](https://github.com/NousResearch/hermes-agent/pull/21350), [#19318](https://github.com/NousResearch/hermes-agent/pull/19318))
|
|
||||||
|
|
||||||
- **Checkpoints v2** — state persistence rewritten. Real pruning, disk guardrails, no more orphan shadow repos. ([#20709](https://github.com/NousResearch/hermes-agent/pull/20709))
|
|
||||||
|
|
||||||
- **The agent lints its own writes** — post-write delta lint on `write_file` + `patch`. Python, JSON, YAML, TOML. Syntax errors surface immediately instead of shipping downstream. ([#20191](https://github.com/NousResearch/hermes-agent/pull/20191))
|
|
||||||
|
|
||||||
- **`no_agent` cron mode — script-only watchdog** — cron jobs can now skip the agent entirely and just run a script. Empty stdout is silent, non-empty gets delivered verbatim. ([#19709](https://github.com/NousResearch/hermes-agent/pull/19709))
|
|
||||||
|
|
||||||
- **Platform allowlists everywhere** — `allowed_channels` / `allowed_chats` / `allowed_rooms` config across Slack, Telegram, Mattermost, Matrix, and DingTalk. ([#21251](https://github.com/NousResearch/hermes-agent/pull/21251))
|
|
||||||
|
|
||||||
- **Providers are now plugins** — `ProviderProfile` ABC + `plugins/model-providers/`. Drop in third-party providers without touching core. ([#20324](https://github.com/NousResearch/hermes-agent/pull/20324))
|
|
||||||
|
|
||||||
- **API server — long-term memory per session** — `X-Hermes-Session-Key` header gives memory providers a stable session identifier. ([#20199](https://github.com/NousResearch/hermes-agent/pull/20199))
|
|
||||||
|
|
||||||
- **MCP levels up** — SSE transport with OAuth forwarding, stale-pipe retries, image results surface as MEDIA tags instead of getting dropped, keepalive on long-lived lifecycle waits. ([#21227](https://github.com/NousResearch/hermes-agent/pull/21227), [#21323](https://github.com/NousResearch/hermes-agent/pull/21323), [#21289](https://github.com/NousResearch/hermes-agent/pull/21289), [#21328](https://github.com/NousResearch/hermes-agent/pull/21328), [#20209](https://github.com/NousResearch/hermes-agent/pull/20209))
|
|
||||||
|
|
||||||
- **Curator grows subcommands** — `hermes curator archive`, `prune`, `list-archived`. Manual `hermes curator run` is synchronous now — you see results without polling. ([#20200](https://github.com/NousResearch/hermes-agent/pull/20200), [#21236](https://github.com/NousResearch/hermes-agent/pull/21236), [#21216](https://github.com/NousResearch/hermes-agent/pull/21216))
|
|
||||||
|
|
||||||
- **ACP — `/steer` and `/queue`** — direct the in-flight agent or queue follow-ups from Zed, VS Code, or JetBrains. Plus atomic session persistence and reasoning-metadata preservation across restarts. (@HenkDz) ([#18114](https://github.com/NousResearch/hermes-agent/pull/18114), [#20279](https://github.com/NousResearch/hermes-agent/pull/20279), [#20296](https://github.com/NousResearch/hermes-agent/pull/20296), [#20433](https://github.com/NousResearch/hermes-agent/pull/20433))
|
|
||||||
|
|
||||||
- **TUI glow-up** — `/model` picker matches `hermes model` with inline auth (@austinpickett), collapsible startup banner sections (@kshitijk4poor), context-compression counter in the status bar. ([#18117](https://github.com/NousResearch/hermes-agent/pull/18117), [#20625](https://github.com/NousResearch/hermes-agent/pull/20625), [#21218](https://github.com/NousResearch/hermes-agent/pull/21218))
|
|
||||||
|
|
||||||
- **Dashboard grows up** — Plugins page (manage, enable/disable, auth status) (@austinpickett), Profiles management page (@vincez-hms-coder), sortable analytics tables, reverse-proxy support via `X-Forwarded-Prefix`, new `default-large` 18px theme. ([#18095](https://github.com/NousResearch/hermes-agent/pull/18095), [#16419](https://github.com/NousResearch/hermes-agent/pull/16419), [#18192](https://github.com/NousResearch/hermes-agent/pull/18192), [#21296](https://github.com/NousResearch/hermes-agent/pull/21296), [#20820](https://github.com/NousResearch/hermes-agent/pull/20820))
|
|
||||||
|
|
||||||
- **SearXNG + split web tools** — SearXNG ships as a native search-only backend; web tools now let you pick different backends per capability (search vs extract vs browse). (@kshitijk4poor) ([#20823](https://github.com/NousResearch/hermes-agent/pull/20823), [#20061](https://github.com/NousResearch/hermes-agent/pull/20061), [#20841](https://github.com/NousResearch/hermes-agent/pull/20841))
|
|
||||||
|
|
||||||
- **OpenRouter response caching** — explicit cache control for models that expose it. (@kshitijk4poor) ([#19132](https://github.com/NousResearch/hermes-agent/pull/19132))
|
|
||||||
|
|
||||||
- **`[[as_document]]` — skill media-routing directive** — skills can force the gateway to deliver output as a document on platforms that support it. ([#21210](https://github.com/NousResearch/hermes-agent/pull/21210))
|
|
||||||
|
|
||||||
- **`transform_llm_output` plugin hook** — new lifecycle hook that lets plugins reshape or filter LLM output before it hits the conversation. Useful for context-window reducers and content filters. ([#21235](https://github.com/NousResearch/hermes-agent/pull/21235))
|
|
||||||
|
|
||||||
- **Nous OAuth persists across profiles** — shared token store: sign in once, every profile inherits the session. ([#19712](https://github.com/NousResearch/hermes-agent/pull/19712))
|
|
||||||
|
|
||||||
- **QQBot — native approval keyboards** — feature parity with Telegram / Discord approval UX. Chunked upload, quoted attachments. ([#21342](https://github.com/NousResearch/hermes-agent/pull/21342), [#21353](https://github.com/NousResearch/hermes-agent/pull/21353))
|
|
||||||
|
|
||||||
- **6 new optional skills** — Shopify (Admin + Storefront GraphQL), here.now, shop-app personal shopping assistant, Anthropic financial-services bundle, kanban-video-orchestrator (@SHL0MS), searxng-search (@kshitijk4poor). ([#18116](https://github.com/NousResearch/hermes-agent/pull/18116), [#18170](https://github.com/NousResearch/hermes-agent/pull/18170), [#20702](https://github.com/NousResearch/hermes-agent/pull/20702), [#21180](https://github.com/NousResearch/hermes-agent/pull/21180), [#19281](https://github.com/NousResearch/hermes-agent/pull/19281), [#20841](https://github.com/NousResearch/hermes-agent/pull/20841))
|
|
||||||
|
|
||||||
- **New models** — `deepseek/deepseek-v4-pro`, `x-ai/grok-4.3`, `openrouter/owl-alpha` (free), `tencent/hy3-preview` (@Contentment003111), Arcee Trinity Large Thinking temperature + compression overrides. ([#20495](https://github.com/NousResearch/hermes-agent/pull/20495), [#20497](https://github.com/NousResearch/hermes-agent/pull/20497), [#18071](https://github.com/NousResearch/hermes-agent/pull/18071), [#21077](https://github.com/NousResearch/hermes-agent/pull/21077), [#20473](https://github.com/NousResearch/hermes-agent/pull/20473))
|
|
||||||
|
|
||||||
- **100 fresh CLI startup tips** — the random tip banner gets 100 new entries covering cron, kanban, curator, plugins, and lesser-known flags. ([#20168](https://github.com/NousResearch/hermes-agent/pull/20168))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧩 Multi-Agent Kanban (Durable)
|
|
||||||
|
|
||||||
### New — durable multi-profile collaboration board
|
|
||||||
- **`feat(kanban): durable multi-profile collaboration board`** — post-revert reimplementation, multi-profile by design ([#17805](https://github.com/NousResearch/hermes-agent/pull/17805))
|
|
||||||
- **Multi-project boards** — one install, many kanbans ([#19653](https://github.com/NousResearch/hermes-agent/pull/19653), [#19679](https://github.com/NousResearch/hermes-agent/pull/19679))
|
|
||||||
- **Share board, workspaces, and worker logs across profiles** ([#19378](https://github.com/NousResearch/hermes-agent/pull/19378))
|
|
||||||
- **Hallucination gate + recovery UX for worker-created-card claims** (closes #20017) ([#20232](https://github.com/NousResearch/hermes-agent/pull/20232))
|
|
||||||
- **Generic diagnostics engine for task distress signals** ([#20332](https://github.com/NousResearch/hermes-agent/pull/20332))
|
|
||||||
- **Per-task `max_retries` override** (supersedes #20972) ([#21330](https://github.com/NousResearch/hermes-agent/pull/21330))
|
|
||||||
- **Multiline textarea for inline-create title** (salvage of #20970) ([#21243](https://github.com/NousResearch/hermes-agent/pull/21243))
|
|
||||||
|
|
||||||
### Kanban Dashboard
|
|
||||||
- **Workspace kind + path inputs in inline create form** ([#19679](https://github.com/NousResearch/hermes-agent/pull/19679))
|
|
||||||
- **Per-platform home-channel notification toggles** ([#19864](https://github.com/NousResearch/hermes-agent/pull/19864))
|
|
||||||
- **Sharper home-channel toggle contrast + drop → running action** ([#19916](https://github.com/NousResearch/hermes-agent/pull/19916))
|
|
||||||
- Fix: reject direct status transition to 'running' via dashboard API (salvage of #19554) ([#19705](https://github.com/NousResearch/hermes-agent/pull/19705))
|
|
||||||
- Fix: dashboard board pin authoritative over server current file (#20879) ([#21230](https://github.com/NousResearch/hermes-agent/pull/21230))
|
|
||||||
- Fix: treat dashboard event-stream cancellation as normal shutdown (#20790) ([#21222](https://github.com/NousResearch/hermes-agent/pull/21222))
|
|
||||||
- Fix: filter dashboard board by selected tenant (#19817) ([#21349](https://github.com/NousResearch/hermes-agent/pull/21349))
|
|
||||||
- Fix: code/pre styling theme-immune across all themes (#21086) ([#21247](https://github.com/NousResearch/hermes-agent/pull/21247))
|
|
||||||
- Fix: reset `<code>` background inside dashboard board ([#20687](https://github.com/NousResearch/hermes-agent/pull/20687))
|
|
||||||
- Fix: preserve dashboard completion summaries + add kanban edit (salvages #20016) ([#20195](https://github.com/NousResearch/hermes-agent/pull/20195))
|
|
||||||
- Fix: avoid fragile failure-column renames (salvage #20848) (@kshitijk4poor) ([#20855](https://github.com/NousResearch/hermes-agent/pull/20855))
|
|
||||||
|
|
||||||
### Worker lifecycle + reliability
|
|
||||||
- **Heartbeat + reclaim + zombie + retry-cap fixes** (#21147, #21141, #21169, #20881) ([#21183](https://github.com/NousResearch/hermes-agent/pull/21183))
|
|
||||||
- **Auto-block workers that exit without completing + shutdown race** (#20894) ([#21214](https://github.com/NousResearch/hermes-agent/pull/21214))
|
|
||||||
- **Detect darwin zombie workers** (salvages #20023) ([#20188](https://github.com/NousResearch/hermes-agent/pull/20188))
|
|
||||||
- **Unify failure counter across spawn/timeout/crash outcomes** ([#20410](https://github.com/NousResearch/hermes-agent/pull/20410))
|
|
||||||
- **Enforce worker task-ownership on destructive tool calls** ([#19713](https://github.com/NousResearch/hermes-agent/pull/19713))
|
|
||||||
- **Drop worker identity claim from KANBAN_GUIDANCE** ([#19427](https://github.com/NousResearch/hermes-agent/pull/19427))
|
|
||||||
- Fix: skip dispatch for tasks assigned to non-profile lanes (salvages #20105, #20134) ([#20165](https://github.com/NousResearch/hermes-agent/pull/20165))
|
|
||||||
- Fix: include default profile in on-disk assignee enumeration (salvages #20123) ([#20170](https://github.com/NousResearch/hermes-agent/pull/20170))
|
|
||||||
- Fix: ignore stale current board pointers (salvages #20063) ([#20183](https://github.com/NousResearch/hermes-agent/pull/20183))
|
|
||||||
- Fix: profile discovery ignores HERMES_HOME in custom-root deployments (@jackey8616) ([#19020](https://github.com/NousResearch/hermes-agent/pull/19020))
|
|
||||||
- Fix: allow orchestrator profiles to see kanban tools via toolsets config ([#19606](https://github.com/NousResearch/hermes-agent/pull/19606))
|
|
||||||
|
|
||||||
### Batch salvages
|
|
||||||
- Tier-1 batch — metadata test, max_spawn config, run-id lifecycle guard (salvages #19522 #19556 #19829) ([#20440](https://github.com/NousResearch/hermes-agent/pull/20440))
|
|
||||||
- Tier-2 batch — doctor, started_at, parent-guard, latest_summary, selects, linked-children ([#20448](https://github.com/NousResearch/hermes-agent/pull/20448))
|
|
||||||
|
|
||||||
### Documentation
|
|
||||||
- Backfill multi-board refs in reference docs ([#19704](https://github.com/NousResearch/hermes-agent/pull/19704))
|
|
||||||
- Document `/kanban` slash command ([#19584](https://github.com/NousResearch/hermes-agent/pull/19584))
|
|
||||||
- Document recommended handoff evidence metadata (salvage #19512) ([#20415](https://github.com/NousResearch/hermes-agent/pull/20415))
|
|
||||||
- Fix orchestrator + worker skill setup instructions (@helix4u) ([#20958](https://github.com/NousResearch/hermes-agent/pull/20958), [#20960](https://github.com/NousResearch/hermes-agent/pull/20960))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 Persistent Goals, Checkpoints & Session Durability
|
|
||||||
|
|
||||||
### `/goal` — persistent cross-turn goals (Ralph loop)
|
|
||||||
- **`feat: /goal — persistent cross-turn goals`** ([#18262](https://github.com/NousResearch/hermes-agent/pull/18262))
|
|
||||||
- **Docs page — Persistent Goals (/goal)** ([#18275](https://github.com/NousResearch/hermes-agent/pull/18275))
|
|
||||||
- Fix: honor configured goal turn budget (salvage #19423) ([#21287](https://github.com/NousResearch/hermes-agent/pull/21287))
|
|
||||||
|
|
||||||
### Checkpoints v2
|
|
||||||
- **Single-store rewrite with real pruning + disk guardrails** ([#20709](https://github.com/NousResearch/hermes-agent/pull/20709))
|
|
||||||
|
|
||||||
### Session durability
|
|
||||||
- **Auto-resume interrupted sessions after gateway restart** (salvage #20888) ([#21192](https://github.com/NousResearch/hermes-agent/pull/21192))
|
|
||||||
- **Preserve pending update prompts across restarts** ([#20160](https://github.com/NousResearch/hermes-agent/pull/20160))
|
|
||||||
- **Preserve home-channel thread targets across restart notifications** (salvage #18440) ([#19271](https://github.com/NousResearch/hermes-agent/pull/19271))
|
|
||||||
- **Preserve thread routing from cached live session sources** ([#21206](https://github.com/NousResearch/hermes-agent/pull/21206))
|
|
||||||
- **Preserve assistant metadata when branching sessions** ([#18222](https://github.com/NousResearch/hermes-agent/pull/18222))
|
|
||||||
- **Preserve thread routing for /update progress and prompts** ([#18193](https://github.com/NousResearch/hermes-agent/pull/18193))
|
|
||||||
- **Preserve document type when merging queued events** ([#18215](https://github.com/NousResearch/hermes-agent/pull/18215))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🛡️ Security & Reliability
|
|
||||||
|
|
||||||
### Security hardening (8 P0 closures)
|
|
||||||
- **Enable secret redaction by default** (#17691, #20785) ([#21193](https://github.com/NousResearch/hermes-agent/pull/21193))
|
|
||||||
- **Discord — scope `DISCORD_ALLOWED_ROLES` to originating guild** (#12136, CVSS 8.1) ([#21241](https://github.com/NousResearch/hermes-agent/pull/21241))
|
|
||||||
- **WhatsApp — reject strangers by default, never respond in self-chat** (#8389) ([#21291](https://github.com/NousResearch/hermes-agent/pull/21291))
|
|
||||||
- **MCP OAuth — close TOCTOU window when saving credentials** ([#21176](https://github.com/NousResearch/hermes-agent/pull/21176))
|
|
||||||
- **`hermes_cli/auth.py` — close TOCTOU window in credential writers** ([#21194](https://github.com/NousResearch/hermes-agent/pull/21194))
|
|
||||||
- **Browser — enforce cloud-metadata SSRF floor in hybrid routing** (#16234) ([#21228](https://github.com/NousResearch/hermes-agent/pull/21228))
|
|
||||||
- **`hermes debug share` — redact log content at upload time** (@GodsBoy) ([#19318](https://github.com/NousResearch/hermes-agent/pull/19318))
|
|
||||||
- **Cron — scan assembled prompt including skill content for prompt injection** (#3968) ([#21350](https://github.com/NousResearch/hermes-agent/pull/21350))
|
|
||||||
- **Restore .env/auth.json/state.db with 0600 perms** ([#19699](https://github.com/NousResearch/hermes-agent/pull/19699))
|
|
||||||
- **SRI integrity for dashboard plugin scripts** (salvage #19389) ([#21277](https://github.com/NousResearch/hermes-agent/pull/21277))
|
|
||||||
- **Bind Meet node server to localhost, restrict token file to owner read** ([#19597](https://github.com/NousResearch/hermes-agent/pull/19597))
|
|
||||||
- **Extend sensitive-write target to cover shell RC and credential files** ([#19282](https://github.com/NousResearch/hermes-agent/pull/19282))
|
|
||||||
- **Harden YOLO mode env parsing against quoted-bool strings** ([#18214](https://github.com/NousResearch/hermes-agent/pull/18214))
|
|
||||||
- **OSV-Scanner CI + Dependabot for github-actions only** ([#20037](https://github.com/NousResearch/hermes-agent/pull/20037))
|
|
||||||
|
|
||||||
### Reliability — critical bug closures
|
|
||||||
- **CLI crash on startup — `Invalid key 'c-S-c'`** (P0, prompt_toolkit doesn't support Shift modifier) ([#19895](https://github.com/NousResearch/hermes-agent/pull/19895), [#19919](https://github.com/NousResearch/hermes-agent/pull/19919))
|
|
||||||
- **CLOSE_WAIT fd leak audit** — httpx keepalive + WhatsApp aiohttp leak + Feishu hygiene (#18451) ([#18766](https://github.com/NousResearch/hermes-agent/pull/18766))
|
|
||||||
- **Gateway creates AIAgent with empty OpenRouter API key when OPENROUTER_API_KEY is missing** (#20982) — fallback providers correctly honored
|
|
||||||
- **Background review + curator protected from overwriting bundled/hub skills** (#20273) ([#20194](https://github.com/NousResearch/hermes-agent/pull/20194))
|
|
||||||
- **TUI compression continuation — ghost sessions with incomplete metadata** (#20001)
|
|
||||||
- **`hermes mcp add` silently launches chat instead of registering MCP server** (#19785) ([#21204](https://github.com/NousResearch/hermes-agent/pull/21204))
|
|
||||||
- **Background review agent runtime propagation** — provider/model/credentials now actually inherit from parent
|
|
||||||
- **Inbound document host paths translated to container paths for Docker backend** (salvage #19048) ([#21184](https://github.com/NousResearch/hermes-agent/pull/21184))
|
|
||||||
- **Matrix gateway race between auto-redaction and message delivery with high-speed models** (#19075)
|
|
||||||
- **`/new` during active agent session never sends response on Telegram** (#18912)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📱 Messaging Platforms (Gateway)
|
|
||||||
|
|
||||||
### New platform
|
|
||||||
- **Google Chat — 20th platform** + generic `env_enablement_fn` / `cron_deliver_env_var` platform-plugin hooks (IRC + Teams migrated) ([#21306](https://github.com/NousResearch/hermes-agent/pull/21306), [#21331](https://github.com/NousResearch/hermes-agent/pull/21331))
|
|
||||||
|
|
||||||
### Cross-platform
|
|
||||||
- **`allowed_{channels,chats,rooms}` whitelist** — Slack (salvage #7401), Telegram, Mattermost, Matrix, DingTalk ([#21251](https://github.com/NousResearch/hermes-agent/pull/21251))
|
|
||||||
- **Per-platform `gateway_restart_notification` flag** ([#20892](https://github.com/NousResearch/hermes-agent/pull/20892))
|
|
||||||
- **`busy_ack_enabled` config — suppress ack messages** ([#18194](https://github.com/NousResearch/hermes-agent/pull/18194))
|
|
||||||
- **Auto-delete slash-command system notices after TTL** ([#18266](https://github.com/NousResearch/hermes-agent/pull/18266))
|
|
||||||
- **Opt-in cleanup of temporary progress bubbles** ([#21186](https://github.com/NousResearch/hermes-agent/pull/21186))
|
|
||||||
- **`[[as_document]]` directive — skill media routing** (salvage #19069) ([#21210](https://github.com/NousResearch/hermes-agent/pull/21210))
|
|
||||||
- **`hermes gateway list` — cross-profile status** (salvage #19129) ([#21225](https://github.com/NousResearch/hermes-agent/pull/21225))
|
|
||||||
- **Auto-resume interrupted sessions after restart** (salvage #20888) ([#21192](https://github.com/NousResearch/hermes-agent/pull/21192))
|
|
||||||
- **Atomic restart markers + Windows runtime-lock offset** (#17842) ([#18179](https://github.com/NousResearch/hermes-agent/pull/18179))
|
|
||||||
- Fix: `config.yaml` wins over `.env` for agent/display/timezone settings ([#18764](https://github.com/NousResearch/hermes-agent/pull/18764))
|
|
||||||
- Fix: auto-restart when source files change out from under us (#17648) ([#18409](https://github.com/NousResearch/hermes-agent/pull/18409))
|
|
||||||
- Fix: use git HEAD SHA for stale-code check, not file mtimes ([#19740](https://github.com/NousResearch/hermes-agent/pull/19740))
|
|
||||||
- Fix: shutdown + restart hygiene — drain timeout, false-fatal, success log ([#18761](https://github.com/NousResearch/hermes-agent/pull/18761))
|
|
||||||
- Fix: preserve max_turns after env reload (salvage #19183) ([#21240](https://github.com/NousResearch/hermes-agent/pull/21240))
|
|
||||||
- Fix: exclude ancestor PIDs from gateway process scan ([#19586](https://github.com/NousResearch/hermes-agent/pull/19586))
|
|
||||||
- Fix: move quick-command alias dispatch before built-ins ([#19588](https://github.com/NousResearch/hermes-agent/pull/19588))
|
|
||||||
- Fix: show other profiles in 'gateway status' to prevent confusion ([#19582](https://github.com/NousResearch/hermes-agent/pull/19582))
|
|
||||||
- Fix: include external_dirs skills in Telegram/Discord slash commands (salvage #8790) ([#18741](https://github.com/NousResearch/hermes-agent/pull/18741))
|
|
||||||
- Fix: match disabled/optional skills by frontmatter slug, not dir name ([#18753](https://github.com/NousResearch/hermes-agent/pull/18753))
|
|
||||||
- Fix: read /status token totals from SessionDB (#17158) ([#18206](https://github.com/NousResearch/hermes-agent/pull/18206))
|
|
||||||
- Fix: snapshot callback generation after agent binds it, not before ([#18219](https://github.com/NousResearch/hermes-agent/pull/18219))
|
|
||||||
- Fix: re-inject topic-bound skill after /new or /reset ([#18205](https://github.com/NousResearch/hermes-agent/pull/18205))
|
|
||||||
- Fix: isolate pending native image paths by session ([#18202](https://github.com/NousResearch/hermes-agent/pull/18202))
|
|
||||||
- Fix: clear queued reload skills notes on new/resume/branch ([#19431](https://github.com/NousResearch/hermes-agent/pull/19431))
|
|
||||||
- Fix: hide required-arg commands from Telegram menu ([#19400](https://github.com/NousResearch/hermes-agent/pull/19400))
|
|
||||||
- Fix: bridge top-level `require_mention` to Telegram config ([#19429](https://github.com/NousResearch/hermes-agent/pull/19429))
|
|
||||||
- Fix: suppress duplicate voice transcripts ([#19428](https://github.com/NousResearch/hermes-agent/pull/19428))
|
|
||||||
- Fix: show friendly error when service is not installed ([#19707](https://github.com/NousResearch/hermes-agent/pull/19707))
|
|
||||||
- Fix: read context_length from custom_providers in session info header ([#19708](https://github.com/NousResearch/hermes-agent/pull/19708))
|
|
||||||
- Fix: preserve WSL interop PATH in systemd units ([#19867](https://github.com/NousResearch/hermes-agent/pull/19867))
|
|
||||||
- Fix: handle planned service stops (salvage #19876) ([#19936](https://github.com/NousResearch/hermes-agent/pull/19936))
|
|
||||||
- Fix: keep DoH-confirmed Telegram IPs that match system DNS (salvage #17043) ([#20175](https://github.com/NousResearch/hermes-agent/pull/20175))
|
|
||||||
- Fix: load `reply_to_mode` from config.yaml for Discord + Telegram (salvage #17117) ([#20171](https://github.com/NousResearch/hermes-agent/pull/20171))
|
|
||||||
- Fix: tolerate malformed HERMES_HUMAN_DELAY_* env vars (salvage #16933) ([#20217](https://github.com/NousResearch/hermes-agent/pull/20217))
|
|
||||||
- Fix: deterministic thread eviction preserves newest entries (salvage #13639) ([#20285](https://github.com/NousResearch/hermes-agent/pull/20285))
|
|
||||||
- Fix: don't dead-end setup wizard when only system-scope unit is installed ([#20905](https://github.com/NousResearch/hermes-agent/pull/20905))
|
|
||||||
- Fix: wait for systemd restart readiness + harden Discord slash-command sync ([#20949](https://github.com/NousResearch/hermes-agent/pull/20949))
|
|
||||||
- Fix: avoid duplicated Responses history (salvage #18995) ([#21185](https://github.com/NousResearch/hermes-agent/pull/21185))
|
|
||||||
- Fix: surface bootstrap failures to stderr (salvage #21157) ([#21278](https://github.com/NousResearch/hermes-agent/pull/21278))
|
|
||||||
- Fix: log agent task failures instead of silently losing usage data (salvage #21159) ([#21274](https://github.com/NousResearch/hermes-agent/pull/21274))
|
|
||||||
- Fix: log runtime-status write failures with rate-limiting (salvage #21158) ([#21285](https://github.com/NousResearch/hermes-agent/pull/21285))
|
|
||||||
- Fix: reset-failed before every fallback restart so the gateway can't get stranded ([#21371](https://github.com/NousResearch/hermes-agent/pull/21371))
|
|
||||||
- Fix: Telegram — preserve `thread_id=1` for forum General typing indicator ([#21390](https://github.com/NousResearch/hermes-agent/pull/21390))
|
|
||||||
- Fix: batch critical fixes — session resume, /new race, HA WebSocket scheme (@kshitijk4poor) ([#19182](https://github.com/NousResearch/hermes-agent/pull/19182))
|
|
||||||
|
|
||||||
### Telegram
|
|
||||||
- **DM user-managed multi-session topics** (salvage of #19185) ([#19206](https://github.com/NousResearch/hermes-agent/pull/19206))
|
|
||||||
|
|
||||||
### Discord
|
|
||||||
- **Message deletion action** (salvage #19052) ([#21197](https://github.com/NousResearch/hermes-agent/pull/21197))
|
|
||||||
- Fix: allow `free_response_channels` to override `DISCORD_IGNORE_NO_MENTION` ([#19629](https://github.com/NousResearch/hermes-agent/pull/19629))
|
|
||||||
|
|
||||||
### Slack
|
|
||||||
- Fix: ephemeral slash-command ack, private notice delivery, format_message fixes (@kshitijk4poor) ([#18198](https://github.com/NousResearch/hermes-agent/pull/18198))
|
|
||||||
|
|
||||||
### WhatsApp
|
|
||||||
- Fix: load WhatsApp home channel from env overrides ([#18190](https://github.com/NousResearch/hermes-agent/pull/18190))
|
|
||||||
|
|
||||||
### Feishu
|
|
||||||
- **Operator-configurable bot admission and mention policy** ([#18208](https://github.com/NousResearch/hermes-agent/pull/18208))
|
|
||||||
- Fix: force text mode for markdown tables (salvage of #13723 by @WuTianyi123) ([#20275](https://github.com/NousResearch/hermes-agent/pull/20275))
|
|
||||||
|
|
||||||
### Matrix + Email
|
|
||||||
- Fix: `/sethome` on Matrix and Email now persists across restarts ([#18272](https://github.com/NousResearch/hermes-agent/pull/18272))
|
|
||||||
|
|
||||||
### Teams
|
|
||||||
- **Docs + feat: sidebar + threading with group-chat fallback** ([#20042](https://github.com/NousResearch/hermes-agent/pull/20042))
|
|
||||||
|
|
||||||
### Weixin
|
|
||||||
- Fix: deduplicate Weixin messages by content fingerprint ([#19742](https://github.com/NousResearch/hermes-agent/pull/19742))
|
|
||||||
|
|
||||||
### QQBot
|
|
||||||
- **Port SDK improvements in-tree — chunked upload, approval keyboards, quoted attachments** ([#21342](https://github.com/NousResearch/hermes-agent/pull/21342))
|
|
||||||
- **Wire native tool-approval UX via inline keyboards** ([#21353](https://github.com/NousResearch/hermes-agent/pull/21353))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🏗️ Core Agent & Architecture
|
|
||||||
|
|
||||||
### Provider & Model Support
|
|
||||||
|
|
||||||
#### Pluggable providers
|
|
||||||
- **ProviderProfile ABC + `plugins/model-providers/`** — inference providers are now a pluggable surface (salvage of #14424) ([#20324](https://github.com/NousResearch/hermes-agent/pull/20324))
|
|
||||||
- **`list_picker_providers`** — credential-filtered picker (salvage #13561) ([#20298](https://github.com/NousResearch/hermes-agent/pull/20298))
|
|
||||||
- **Remove `/provider` alias for `/model`** ([#20358](https://github.com/NousResearch/hermes-agent/pull/20358))
|
|
||||||
- **Shared Hermes dotenv loader across CLI + plugins** (salvage #13660) ([#20281](https://github.com/NousResearch/hermes-agent/pull/20281))
|
|
||||||
- **Nous OAuth persisted across profiles via shared token store** ([#19712](https://github.com/NousResearch/hermes-agent/pull/19712))
|
|
||||||
|
|
||||||
#### New models
|
|
||||||
- `deepseek/deepseek-v4-pro` added to OpenRouter + Nous Portal ([#20495](https://github.com/NousResearch/hermes-agent/pull/20495))
|
|
||||||
- `x-ai/grok-4.3` added to OpenRouter + Nous Portal ([#20497](https://github.com/NousResearch/hermes-agent/pull/20497))
|
|
||||||
- `openrouter/owl-alpha` (free tier) added to curated OpenRouter list ([#18071](https://github.com/NousResearch/hermes-agent/pull/18071))
|
|
||||||
- `tencent/hy3-preview` paid route on OpenRouter (@Contentment003111) ([#21077](https://github.com/NousResearch/hermes-agent/pull/21077))
|
|
||||||
- Arcee Trinity Large Thinking — temperature + compression overrides ([#20473](https://github.com/NousResearch/hermes-agent/pull/20473))
|
|
||||||
- Rename `x-ai/grok-4.20-beta` to `x-ai/grok-4.20` ([#19640](https://github.com/NousResearch/hermes-agent/pull/19640))
|
|
||||||
- Demote Vercel AI Gateway to bottom of provider picker ([#18112](https://github.com/NousResearch/hermes-agent/pull/18112))
|
|
||||||
|
|
||||||
#### Provider configuration
|
|
||||||
- **OpenRouter — response caching support** (@kshitijk4poor) ([#19132](https://github.com/NousResearch/hermes-agent/pull/19132))
|
|
||||||
- **`image_gen.model` from config.yaml honored** (salvage #19376) ([#21273](https://github.com/NousResearch/hermes-agent/pull/21273))
|
|
||||||
- Fix: honor runtime default model during delegate provider resolution (@johnncenae) ([#17587](https://github.com/NousResearch/hermes-agent/pull/17587))
|
|
||||||
- Fix: avoid Bedrock credential probe in provider picker (@helix4u) ([#18998](https://github.com/NousResearch/hermes-agent/pull/18998))
|
|
||||||
- Fix: drop stale env-var override of persisted provider for cron ([#19627](https://github.com/NousResearch/hermes-agent/pull/19627))
|
|
||||||
- Fix: auxiliary curator api_key/base_url into runtime resolution ([#19421](https://github.com/NousResearch/hermes-agent/pull/19421))
|
|
||||||
|
|
||||||
### Agent Loop & Conversation
|
|
||||||
- **`video_analyze` — native video understanding tool** (@alt-glitch) ([#19301](https://github.com/NousResearch/hermes-agent/pull/19301))
|
|
||||||
- **Show context compression count in status bar** (CLI + TUI) ([#21218](https://github.com/NousResearch/hermes-agent/pull/21218))
|
|
||||||
- **Isolate `get_tool_definitions` quiet_mode cache + dedup LCM injection** (#17335) ([#17889](https://github.com/NousResearch/hermes-agent/pull/17889))
|
|
||||||
- Fix: warning-first tool-call loop guardrails ([#18227](https://github.com/NousResearch/hermes-agent/pull/18227))
|
|
||||||
- Fix: break permanent empty-response loop from orphan tool-tail ([#21385](https://github.com/NousResearch/hermes-agent/pull/21385))
|
|
||||||
- Fix: propagate ContextVars to concurrent tool worker threads (salvage #16660) ([#18123](https://github.com/NousResearch/hermes-agent/pull/18123))
|
|
||||||
- Fix: surface self-improvement review summaries across CLI, TUI, and gateway ([#18073](https://github.com/NousResearch/hermes-agent/pull/18073))
|
|
||||||
- Fix: serialize concurrent `hermes_tools` RPC calls from `execute_code` ([#17894](https://github.com/NousResearch/hermes-agent/pull/17894), [#17902](https://github.com/NousResearch/hermes-agent/pull/17902))
|
|
||||||
- Fix: include system prompt + tool schemas in token estimates for compression ([#18265](https://github.com/NousResearch/hermes-agent/pull/18265))
|
|
||||||
|
|
||||||
### Compression
|
|
||||||
- Fix: skip non-string tool content in dedup pass to prevent AttributeError ([#19398](https://github.com/NousResearch/hermes-agent/pull/19398))
|
|
||||||
- Fix: reset `_summary_failure_cooldown_until` on session reset ([#19622](https://github.com/NousResearch/hermes-agent/pull/19622))
|
|
||||||
- Fix: trigger fallback on timeout errors alongside model-unavailable errors ([#19665](https://github.com/NousResearch/hermes-agent/pull/19665))
|
|
||||||
- Fix: `_prune_old_tool_results` boundary direction ([#19725](https://github.com/NousResearch/hermes-agent/pull/19725))
|
|
||||||
- Fix: soften summary prompt for content filters (salvage #19456) ([#21302](https://github.com/NousResearch/hermes-agent/pull/21302))
|
|
||||||
|
|
||||||
### Delegate
|
|
||||||
- Fix: inherit parent fallback_chain in `_build_child_agent` ([#19601](https://github.com/NousResearch/hermes-agent/pull/19601))
|
|
||||||
- Fix: guard `_load_config()` against `delegation: null` in config.yaml ([#19662](https://github.com/NousResearch/hermes-agent/pull/19662))
|
|
||||||
- Fix: inherit parent api_key when `delegation.base_url` set without `delegation.api_key` ([#19741](https://github.com/NousResearch/hermes-agent/pull/19741))
|
|
||||||
- Fix: expand composite toolsets before intersection (salvage #19455) ([#21300](https://github.com/NousResearch/hermes-agent/pull/21300))
|
|
||||||
- Fix: correct ACP docs — Claude Code CLI has no --acp flag (salvage #19058) ([#21201](https://github.com/NousResearch/hermes-agent/pull/21201))
|
|
||||||
|
|
||||||
### Session & Memory
|
|
||||||
- **Hindsight — probe API for `update_mode='append'` to dedupe across processes** (@nicoloboschi) ([#20222](https://github.com/NousResearch/hermes-agent/pull/20222))
|
|
||||||
|
|
||||||
### Curator
|
|
||||||
- **`hermes curator archive` and `prune` subcommands** ([#20200](https://github.com/NousResearch/hermes-agent/pull/20200))
|
|
||||||
- **`hermes curator list-archived`** (#20651) ([#21236](https://github.com/NousResearch/hermes-agent/pull/21236))
|
|
||||||
- **Synchronous manual `hermes curator run`** (#20555) ([#21216](https://github.com/NousResearch/hermes-agent/pull/21216))
|
|
||||||
- Fix: preserve `last_report_path` in state ([#18169](https://github.com/NousResearch/hermes-agent/pull/18169))
|
|
||||||
- Fix: rewrite cron job skill refs after consolidation ([#18253](https://github.com/NousResearch/hermes-agent/pull/18253))
|
|
||||||
- Fix: defer first run + `--dry-run` preview (#18373) ([#18389](https://github.com/NousResearch/hermes-agent/pull/18389))
|
|
||||||
- Fix: authoritative `absorbed_into` on delete + restore cron skill links on rollback (#18671) ([#18731](https://github.com/NousResearch/hermes-agent/pull/18731))
|
|
||||||
- Fix: prevent false-positive consolidation from substring matching ([#19573](https://github.com/NousResearch/hermes-agent/pull/19573))
|
|
||||||
- Fix: only mark agent-created for background-review sediment ([#19621](https://github.com/NousResearch/hermes-agent/pull/19621))
|
|
||||||
- Fix: protect hub skills by frontmatter name ([#20194](https://github.com/NousResearch/hermes-agent/pull/20194))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 Tool System
|
|
||||||
|
|
||||||
### File tools
|
|
||||||
- **Post-write delta lint on `write_file` + `patch`** — in-proc linters for Python, JSON, YAML, TOML ([#20191](https://github.com/NousResearch/hermes-agent/pull/20191))
|
|
||||||
|
|
||||||
### Cron
|
|
||||||
- **`no_agent` mode — script-only cron jobs (watchdog pattern)** ([#19709](https://github.com/NousResearch/hermes-agent/pull/19709))
|
|
||||||
- **`context_from` chaining docs** (salvage #15724) ([#20394](https://github.com/NousResearch/hermes-agent/pull/20394))
|
|
||||||
- Fix: treat non-dict origin as missing instead of crashing tick ([#19283](https://github.com/NousResearch/hermes-agent/pull/19283))
|
|
||||||
- Fix: bump skill usage when cron jobs load skills ([#19433](https://github.com/NousResearch/hermes-agent/pull/19433))
|
|
||||||
- Fix: recover null `next_run_at` jobs ([#19576](https://github.com/NousResearch/hermes-agent/pull/19576))
|
|
||||||
- Fix: skip AI call when prerun script produces no output ([#19628](https://github.com/NousResearch/hermes-agent/pull/19628))
|
|
||||||
- Fix: expand config.yaml refs during job execution ([#19872](https://github.com/NousResearch/hermes-agent/pull/19872))
|
|
||||||
- Fix: serialize `get_due_jobs` writes to prevent parallel state corruption ([#19874](https://github.com/NousResearch/hermes-agent/pull/19874))
|
|
||||||
- Fix: initialize MCP servers before constructing the cron AIAgent ([#21354](https://github.com/NousResearch/hermes-agent/pull/21354))
|
|
||||||
|
|
||||||
### MCP
|
|
||||||
- **SSE transport support** (salvage #19135) ([#21227](https://github.com/NousResearch/hermes-agent/pull/21227))
|
|
||||||
- **Forward OAuth auth + bump `sse_read_timeout` on SSE transport** ([#21323](https://github.com/NousResearch/hermes-agent/pull/21323))
|
|
||||||
- **Retry stale pipe transport failures as session-expired** ([#21289](https://github.com/NousResearch/hermes-agent/pull/21289))
|
|
||||||
- **Surface image tool results as MEDIA tags instead of dropping them** ([#21328](https://github.com/NousResearch/hermes-agent/pull/21328))
|
|
||||||
- **Periodic keepalive to `_wait_for_lifecycle_event`** (salvage #17016) ([#20209](https://github.com/NousResearch/hermes-agent/pull/20209))
|
|
||||||
- Fix: reconnect on terminated sessions ([#19380](https://github.com/NousResearch/hermes-agent/pull/19380))
|
|
||||||
- Fix: decouple AnyUrl import from mcp dependency ([#19695](https://github.com/NousResearch/hermes-agent/pull/19695))
|
|
||||||
- Fix: `mcp add --command` gets distinct argparse dest ([#21204](https://github.com/NousResearch/hermes-agent/pull/21204))
|
|
||||||
- Fix: clear stale thread interrupt before MCP discovery ([#21276](https://github.com/NousResearch/hermes-agent/pull/21276))
|
|
||||||
- Fix: report configured timeout in MCP call errors ([#21281](https://github.com/NousResearch/hermes-agent/pull/21281))
|
|
||||||
- Fix: include exception type in error messages when str(exc) is empty (salvage #19425) ([#21292](https://github.com/NousResearch/hermes-agent/pull/21292))
|
|
||||||
- Fix: re-raise CancelledError explicitly in `MCPServerTask.run` ([#21318](https://github.com/NousResearch/hermes-agent/pull/21318))
|
|
||||||
- Fix: coerce numeric tool args defensively in `mcp_serve` ([#21329](https://github.com/NousResearch/hermes-agent/pull/21329))
|
|
||||||
- Fix: gate utility stubs on server-advertised capabilities ([#21347](https://github.com/NousResearch/hermes-agent/pull/21347))
|
|
||||||
|
|
||||||
### Browser
|
|
||||||
- Fix: allow explicit CDP override without local agent-browser ([#19670](https://github.com/NousResearch/hermes-agent/pull/19670))
|
|
||||||
- Fix: inject `--no-sandbox` for root + AppArmor userns restrictions ([#19747](https://github.com/NousResearch/hermes-agent/pull/19747))
|
|
||||||
- Fix: tighten Lightpanda fallback edge cases (@kshitijk4poor) ([#20672](https://github.com/NousResearch/hermes-agent/pull/20672))
|
|
||||||
|
|
||||||
### Web tools
|
|
||||||
- **Per-capability backend selection — search/extract split** (@kshitijk4poor) ([#20061](https://github.com/NousResearch/hermes-agent/pull/20061))
|
|
||||||
- **SearXNG native search-only backend** (@kshitijk4poor) ([#20823](https://github.com/NousResearch/hermes-agent/pull/20823))
|
|
||||||
|
|
||||||
### Approval / Tool gating
|
|
||||||
- Fix: wake blocked gateway approvals on session cleanup ([#18171](https://github.com/NousResearch/hermes-agent/pull/18171))
|
|
||||||
- Fix: harden YOLO mode env parsing against quoted-bool strings ([#18214](https://github.com/NousResearch/hermes-agent/pull/18214))
|
|
||||||
- Fix: extend sensitive write target to cover shell RC and credential files ([#19282](https://github.com/NousResearch/hermes-agent/pull/19282))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔌 Plugin System
|
|
||||||
|
|
||||||
- **`transform_llm_output` plugin hook** (salvage of #20813) ([#21235](https://github.com/NousResearch/hermes-agent/pull/21235))
|
|
||||||
- **Document `env_enablement_fn` + `cron_deliver_env_var` platform-plugin hooks** ([#21331](https://github.com/NousResearch/hermes-agent/pull/21331))
|
|
||||||
- **Pluggable surfaces coverage — model-provider guide, full plugin map, opt-in fix** ([#20749](https://github.com/NousResearch/hermes-agent/pull/20749))
|
|
||||||
- **Plugin-authoring gaps — image-gen provider guide + publishing a skill tap** ([#20800](https://github.com/NousResearch/hermes-agent/pull/20800))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧩 Skills Ecosystem
|
|
||||||
|
|
||||||
### New optional skills
|
|
||||||
- **Shopify** — Admin + Storefront GraphQL optional skill ([#18116](https://github.com/NousResearch/hermes-agent/pull/18116))
|
|
||||||
- **here.now** — optional skill ([#18170](https://github.com/NousResearch/hermes-agent/pull/18170))
|
|
||||||
- **shop-app** — personal shopping assistant (optional) ([#20702](https://github.com/NousResearch/hermes-agent/pull/20702))
|
|
||||||
- **Anthropic financial-services bundle** — ported as optional finance skills ([#21180](https://github.com/NousResearch/hermes-agent/pull/21180))
|
|
||||||
- **kanban-video-orchestrator** — creative optional skill (@SHL0MS) ([#19281](https://github.com/NousResearch/hermes-agent/pull/19281))
|
|
||||||
- **searxng-search** — optional skill + Web Search + Extract docs page (@kshitijk4poor) ([#20841](https://github.com/NousResearch/hermes-agent/pull/20841), [#20844](https://github.com/NousResearch/hermes-agent/pull/20844))
|
|
||||||
|
|
||||||
### Skill UX
|
|
||||||
- **Linear skill — add Documents support + Python helper script** ([#20752](https://github.com/NousResearch/hermes-agent/pull/20752))
|
|
||||||
- **Modernize Obsidian skill to use file tools** (salvage #19332) ([#20413](https://github.com/NousResearch/hermes-agent/pull/20413))
|
|
||||||
- **Default custom tool creation to plugins** (@kshitijk4poor) ([#19755](https://github.com/NousResearch/hermes-agent/pull/19755))
|
|
||||||
- **skill_commands cache — rescan on platform scope changes** (salvage #14570 by @LeonSGP43) ([#18739](https://github.com/NousResearch/hermes-agent/pull/18739))
|
|
||||||
- **Skills — additional rescan paths in skill_commands cache** (salvage #19042) ([#21181](https://github.com/NousResearch/hermes-agent/pull/21181))
|
|
||||||
- Fix: regression tests for non-dict metadata in `extract_skill_conditions` ([#18213](https://github.com/NousResearch/hermes-agent/pull/18213))
|
|
||||||
- Docs: explain restoring bundled skills (salvage #19254) ([#20404](https://github.com/NousResearch/hermes-agent/pull/20404))
|
|
||||||
- Docs: document `hermes skills reset` subcommand (salvage #11544) ([#20395](https://github.com/NousResearch/hermes-agent/pull/20395))
|
|
||||||
- Docs: himalaya v1.2.0 `folder.aliases` syntax ([#19882](https://github.com/NousResearch/hermes-agent/pull/19882))
|
|
||||||
- Point agent at `hermes-agent` skill + docs site sync ([#20390](https://github.com/NousResearch/hermes-agent/pull/20390))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🖥️ CLI & User Experience
|
|
||||||
|
|
||||||
### CLI
|
|
||||||
- **`/new` accepts optional session name argument** (salvage of #19555) ([#19637](https://github.com/NousResearch/hermes-agent/pull/19637))
|
|
||||||
- **100 new CLI startup tips** ([#20168](https://github.com/NousResearch/hermes-agent/pull/20168))
|
|
||||||
- **`display.language` — static message translation** (zh/ja/de/es) ([#20231](https://github.com/NousResearch/hermes-agent/pull/20231))
|
|
||||||
- **French (fr) locale** (@Foolafroos) ([#20329](https://github.com/NousResearch/hermes-agent/pull/20329))
|
|
||||||
- **Ukrainian (uk) locale** ([#20467](https://github.com/NousResearch/hermes-agent/pull/20467))
|
|
||||||
- **Turkish (tr) locale** ([#20474](https://github.com/NousResearch/hermes-agent/pull/20474))
|
|
||||||
- Fix: recover classic CLI output after resize (@helix4u) ([#20444](https://github.com/NousResearch/hermes-agent/pull/20444))
|
|
||||||
- Fix: complete absolute paths as paths (@helix4u) ([#19930](https://github.com/NousResearch/hermes-agent/pull/19930))
|
|
||||||
- Fix: resolve lazy session creation regressions (#18370 fallout) (@alt-glitch) ([#20363](https://github.com/NousResearch/hermes-agent/pull/20363))
|
|
||||||
- Fix: local backend CLI always uses launch directory (@alt-glitch) ([#19334](https://github.com/NousResearch/hermes-agent/pull/19334))
|
|
||||||
- Refactor: drop dead c-S-c key binding (follow-up to #19895) ([#19919](https://github.com/NousResearch/hermes-agent/pull/19919))
|
|
||||||
|
|
||||||
### TUI (Ink)
|
|
||||||
- **`/model` picker overhaul to match `hermes model` with inline auth** (@austinpickett) ([#18117](https://github.com/NousResearch/hermes-agent/pull/18117))
|
|
||||||
- **Collapsible sections in startup banner** — skills, system prompt, MCP (@kshitijk4poor) ([#20625](https://github.com/NousResearch/hermes-agent/pull/20625))
|
|
||||||
- **Show context compression count in status bar** ([#21218](https://github.com/NousResearch/hermes-agent/pull/21218))
|
|
||||||
- Perf: reduce overlay render churn with focused selectors (@OutThisLife) ([#20393](https://github.com/NousResearch/hermes-agent/pull/20393))
|
|
||||||
- Fix: restore voice push-to-talk parity (salvage of #16189 by @Montbra) (@OutThisLife) ([#20897](https://github.com/NousResearch/hermes-agent/pull/20897))
|
|
||||||
- Fix: kanban button (@austinpickett) ([#18358](https://github.com/NousResearch/hermes-agent/pull/18358))
|
|
||||||
|
|
||||||
### Dashboard
|
|
||||||
- **Plugins page — manage, enable/disable, auth status** (@austinpickett) ([#18095](https://github.com/NousResearch/hermes-agent/pull/18095))
|
|
||||||
- **Profiles management page** (@vincez-hms-coder) ([#16419](https://github.com/NousResearch/hermes-agent/pull/16419))
|
|
||||||
- **Interactive column sorting in analytics tables** ([#18192](https://github.com/NousResearch/hermes-agent/pull/18192))
|
|
||||||
- **`default-large` built-in theme with 18px base size** ([#20820](https://github.com/NousResearch/hermes-agent/pull/20820))
|
|
||||||
- **Support serving under URL prefix via `X-Forwarded-Prefix`** (salvage #19450) ([#21296](https://github.com/NousResearch/hermes-agent/pull/21296))
|
|
||||||
- **Launch dashboard as side-process via `HERMES_DASHBOARD=1` in Docker** (@benbarclay) ([#19540](https://github.com/NousResearch/hermes-agent/pull/19540))
|
|
||||||
- Fix: dashboard theme layout shift (@AllardQuek) ([#17232](https://github.com/NousResearch/hermes-agent/pull/17232))
|
|
||||||
- Fix: gateway model picker current context (@helix4u) ([#20513](https://github.com/NousResearch/hermes-agent/pull/20513))
|
|
||||||
|
|
||||||
### Update + setup
|
|
||||||
- **`hermes update --yes/-y` to skip interactive prompts** ([#18261](https://github.com/NousResearch/hermes-agent/pull/18261))
|
|
||||||
- **Restart manual profile gateways after update** ([#18178](https://github.com/NousResearch/hermes-agent/pull/18178))
|
|
||||||
|
|
||||||
### Profiles
|
|
||||||
- **`--no-skills` flag for empty profile creation** ([#20986](https://github.com/NousResearch/hermes-agent/pull/20986))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎵 Voice, Image & Media
|
|
||||||
|
|
||||||
- **xAI Custom Voices — voice cloning** (@alt-glitch) ([#18776](https://github.com/NousResearch/hermes-agent/pull/18776))
|
|
||||||
- **Achievements — share card render on unlocked badges** ([#19657](https://github.com/NousResearch/hermes-agent/pull/19657))
|
|
||||||
- **Refresh systemd unit on gateway boot (not just start/restart)** (@alt-glitch) ([#19684](https://github.com/NousResearch/hermes-agent/pull/19684))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔗 API Server & Remote Access
|
|
||||||
|
|
||||||
- **`X-Hermes-Session-Key` header for long-term memory scoping** (closes #20060) ([#20199](https://github.com/NousResearch/hermes-agent/pull/20199))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧰 ACP Adapter (VS Code / Zed / JetBrains)
|
|
||||||
|
|
||||||
- **`/steer` and `/queue` slash commands** (@HenkDz) ([#18114](https://github.com/NousResearch/hermes-agent/pull/18114))
|
|
||||||
- Fix: translate Windows cwd for WSL sessions (salvage #18128) ([#18233](https://github.com/NousResearch/hermes-agent/pull/18233))
|
|
||||||
- Fix: run `/steer` as a regular prompt on idle sessions ([#18258](https://github.com/NousResearch/hermes-agent/pull/18258))
|
|
||||||
- Fix: route Zed thoughts to reasoning + polish tool/context rendering ([#19139](https://github.com/NousResearch/hermes-agent/pull/19139))
|
|
||||||
- Fix: atomic session persistence via `replace_messages` (salvage #13675) ([#20279](https://github.com/NousResearch/hermes-agent/pull/20279))
|
|
||||||
- Fix: preserve assistant reasoning metadata in session persistence (salvage #13575) ([#20296](https://github.com/NousResearch/hermes-agent/pull/20296))
|
|
||||||
- Docs: update VS Code setup for ACP Client extension (salvage #12495) ([#20433](https://github.com/NousResearch/hermes-agent/pull/20433))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🐳 Docker
|
|
||||||
|
|
||||||
- **Launch dashboard as side-process via `HERMES_DASHBOARD=1`** (@benbarclay) ([#19540](https://github.com/NousResearch/hermes-agent/pull/19540))
|
|
||||||
- **Refuse root gateway runs in official image** (salvage #19215) ([#21250](https://github.com/NousResearch/hermes-agent/pull/21250))
|
|
||||||
- **Chown runtime `node_modules` trees to hermes user** (salvage #19303) ([#21267](https://github.com/NousResearch/hermes-agent/pull/21267))
|
|
||||||
- Fix: exclude compose/profile runtime state from build context ([#19626](https://github.com/NousResearch/hermes-agent/pull/19626))
|
|
||||||
- CI: don't cancel overlapping builds, guard `:latest` (@ethernet8023) ([#20890](https://github.com/NousResearch/hermes-agent/pull/20890))
|
|
||||||
- Test: align Dockerfile contract tests with simplified TUI flow (salvage #19024) ([#21174](https://github.com/NousResearch/hermes-agent/pull/21174))
|
|
||||||
- Docs: connect to local inference servers (vLLM, Ollama) (salvage #12335) ([#20407](https://github.com/NousResearch/hermes-agent/pull/20407))
|
|
||||||
- Docs: document `API_SERVER_*` env vars (salvage #11758) ([#20409](https://github.com/NousResearch/hermes-agent/pull/20409))
|
|
||||||
- Docs: clarify Docker terminal backend is a single persistent container ([#20003](https://github.com/NousResearch/hermes-agent/pull/20003))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🐛 Notable Bug Fixes
|
|
||||||
|
|
||||||
### Agent
|
|
||||||
- Fix: recover lazy session creation regressions (#18370 fallout) (@alt-glitch) ([#20363](https://github.com/NousResearch/hermes-agent/pull/20363))
|
|
||||||
- Fix: propagate ContextVars to concurrent tool worker threads (salvage #16660) ([#18123](https://github.com/NousResearch/hermes-agent/pull/18123))
|
|
||||||
- Fix: warning-first tool-call loop guardrails ([#18227](https://github.com/NousResearch/hermes-agent/pull/18227))
|
|
||||||
- Fix: surface self-improvement review summaries across CLI, TUI, and gateway ([#18073](https://github.com/NousResearch/hermes-agent/pull/18073))
|
|
||||||
|
|
||||||
### Gateway streaming
|
|
||||||
- Fix: harden StreamingConfig bool and numeric coercion (@simbam99) ([#16463](https://github.com/NousResearch/hermes-agent/pull/16463))
|
|
||||||
|
|
||||||
### Model
|
|
||||||
- Fix: avoid Bedrock credential probe in provider picker (@helix4u) ([#18998](https://github.com/NousResearch/hermes-agent/pull/18998))
|
|
||||||
|
|
||||||
### Doctor
|
|
||||||
- Fix: check global agent-browser when local install not found ([#19671](https://github.com/NousResearch/hermes-agent/pull/19671))
|
|
||||||
- Test: kimi-coding-cn provider validation regression ([#19734](https://github.com/NousResearch/hermes-agent/pull/19734))
|
|
||||||
|
|
||||||
### Update
|
|
||||||
- Fix: patch `isatty` on real streams to fix xdist-flaky `--yes` tests (salvage #19026) ([#21175](https://github.com/NousResearch/hermes-agent/pull/21175))
|
|
||||||
- Fix: teach restart-mocks about the post-update survivor sweep (salvage #19031) ([#21177](https://github.com/NousResearch/hermes-agent/pull/21177))
|
|
||||||
|
|
||||||
### Auth
|
|
||||||
- Fix: acp preserve assistant reasoning metadata ([#20296](https://github.com/NousResearch/hermes-agent/pull/20296))
|
|
||||||
|
|
||||||
### Redact
|
|
||||||
- Fix: add `code_file` param to skip false-positive ENV/JSON patterns ([#19715](https://github.com/NousResearch/hermes-agent/pull/19715))
|
|
||||||
|
|
||||||
### Email
|
|
||||||
- Fix: quoted-relative file-drop paths + Date header on tool email path ([#19646](https://github.com/NousResearch/hermes-agent/pull/19646))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧪 Testing
|
|
||||||
|
|
||||||
- **ACP — accept prompt persistence kwargs in MCP E2E mocks** (@stephenschoettler) ([#18047](https://github.com/NousResearch/hermes-agent/pull/18047))
|
|
||||||
- **Toolsets — include kanban in expected post-#17805 toolset assertions** (@briandevans) ([#18122](https://github.com/NousResearch/hermes-agent/pull/18122))
|
|
||||||
- **Agent — cover max-iterations summary message sanitization** ([#19580](https://github.com/NousResearch/hermes-agent/pull/19580))
|
|
||||||
- **run_agent — `-inf` and `nan` regression coverage for `_coerce_number`** ([#19703](https://github.com/NousResearch/hermes-agent/pull/19703))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📚 Documentation
|
|
||||||
|
|
||||||
### Major docs additions
|
|
||||||
- **`llms.txt` + `llms-full.txt` — agent-friendly ingestion** ([#18276](https://github.com/NousResearch/hermes-agent/pull/18276))
|
|
||||||
- **User Stories and Use Cases collage page** ([#18282](https://github.com/NousResearch/hermes-agent/pull/18282))
|
|
||||||
- **Persistent Goals (/goal) feature page** ([#18275](https://github.com/NousResearch/hermes-agent/pull/18275))
|
|
||||||
- **Windows (WSL2) guide expansion** — filesystem, networking, services, pitfalls ([#20748](https://github.com/NousResearch/hermes-agent/pull/20748))
|
|
||||||
- **Chinese (zh-CN) README translation** (salvage #13508) ([#20431](https://github.com/NousResearch/hermes-agent/pull/20431))
|
|
||||||
- **zh-Hans Docusaurus locale** + Tool Gateway / image-gen / WSL quickstart translations (salvage #11728) ([#20430](https://github.com/NousResearch/hermes-agent/pull/20430))
|
|
||||||
- **Tool Gateway docs restructure** — lead with what it does, config moved to bottom ([#20827](https://github.com/NousResearch/hermes-agent/pull/20827))
|
|
||||||
- **Quickstart — Onchain AI Garage Hermes tutorials playlist** ([#20192](https://github.com/NousResearch/hermes-agent/pull/20192))
|
|
||||||
- **Open WebUI bootstrap script** (salvage #9566) ([#20427](https://github.com/NousResearch/hermes-agent/pull/20427))
|
|
||||||
- **Local Ollama setup guide** (salvage #5842) ([#20426](https://github.com/NousResearch/hermes-agent/pull/20426))
|
|
||||||
- **Google Gemini guide** (salvage #17450) ([#20401](https://github.com/NousResearch/hermes-agent/pull/20401))
|
|
||||||
- **Custom model aliases for /model command** ([#20475](https://github.com/NousResearch/hermes-agent/pull/20475))
|
|
||||||
- **Together/Groq/Perplexity cookbook via `custom_providers`** (salvage #15214) ([#20400](https://github.com/NousResearch/hermes-agent/pull/20400))
|
|
||||||
- **Doubao speech integration examples** (TTS + STT) (salvage #18065) ([#20418](https://github.com/NousResearch/hermes-agent/pull/20418))
|
|
||||||
- **WSL-to-Windows Chrome MCP bridge** (salvage #8313) ([#20428](https://github.com/NousResearch/hermes-agent/pull/20428))
|
|
||||||
- **Hermes skills docs sync** — slash commands + durable-systems section ([#20390](https://github.com/NousResearch/hermes-agent/pull/20390))
|
|
||||||
- **AGENTS.md — curator/cron/delegation/toolsets + fix plugin tree** ([#20226](https://github.com/NousResearch/hermes-agent/pull/20226))
|
|
||||||
- **Bedrock quickstart entry + fallback comment + deployment link** (salvage #11093) ([#20397](https://github.com/NousResearch/hermes-agent/pull/20397))
|
|
||||||
|
|
||||||
### Docs polish
|
|
||||||
- Collapse exploding skills tree to a single Skills node ([#18259](https://github.com/NousResearch/hermes-agent/pull/18259))
|
|
||||||
- Clarify `session_search` auxiliary model docs ([#19593](https://github.com/NousResearch/hermes-agent/pull/19593))
|
|
||||||
- Open WebUI Quick Setup gap fill ([#19654](https://github.com/NousResearch/hermes-agent/pull/19654))
|
|
||||||
- Default custom tool creation to plugins (@kshitijk4poor) ([#19755](https://github.com/NousResearch/hermes-agent/pull/19755))
|
|
||||||
- Clarify Telegram group chat troubleshooting (salvage #18672) ([#20416](https://github.com/NousResearch/hermes-agent/pull/20416))
|
|
||||||
- Codex OAuth auth prerequisite clarification (salvage #18688) ([#20417](https://github.com/NousResearch/hermes-agent/pull/20417))
|
|
||||||
- Discord Server Members Intent + SSRC-mapping drift + /voice join slash Choice (salvage #11350) ([#20411](https://github.com/NousResearch/hermes-agent/pull/20411))
|
|
||||||
- Document `ctx.dispatch_tool()` (salvage #10955) ([#20391](https://github.com/NousResearch/hermes-agent/pull/20391))
|
|
||||||
- Document `hermes webhook subscribe --deliver-only` (salvage #12612) ([#20392](https://github.com/NousResearch/hermes-agent/pull/20392))
|
|
||||||
- Document `hermes import` reference (salvage #14711) ([#20396](https://github.com/NousResearch/hermes-agent/pull/20396))
|
|
||||||
- Document per-provider TTS `max_text_length` caps (salvage #13825) ([#20389](https://github.com/NousResearch/hermes-agent/pull/20389))
|
|
||||||
- Clarify supported prompt customization surfaces (salvage #19987) ([#20383](https://github.com/NousResearch/hermes-agent/pull/20383))
|
|
||||||
- Correct `web_extract` summarizer timeout comment (salvage #20051) ([#20381](https://github.com/NousResearch/hermes-agent/pull/20381))
|
|
||||||
- Fix fallback provider config paths (salvage #20033) ([#20382](https://github.com/NousResearch/hermes-agent/pull/20382))
|
|
||||||
- Fix misleading RL install-extras claim (salvage #19080) ([#21213](https://github.com/NousResearch/hermes-agent/pull/21213))
|
|
||||||
- Clarify API server tool execution locality (salvage #19117) ([#21223](https://github.com/NousResearch/hermes-agent/pull/21223))
|
|
||||||
- Prefer `.venv` to match AGENTS.md and scripts/run_tests.sh (@xxxigm) ([#21334](https://github.com/NousResearch/hermes-agent/pull/21334))
|
|
||||||
- Align tool discovery + test runner with AGENTS.md (@xxxigm) ([#20791](https://github.com/NousResearch/hermes-agent/pull/20791))
|
|
||||||
- Align terminal-backend count and naming across docs and code (salvage #19044) ([#20402](https://github.com/NousResearch/hermes-agent/pull/20402))
|
|
||||||
- Refresh stale platform counts (salvage #19053) ([#20403](https://github.com/NousResearch/hermes-agent/pull/20403))
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 👥 Contributors
|
|
||||||
|
|
||||||
### Core
|
|
||||||
- **@teknium1** — salvage, triage, review, feature work, and release management
|
|
||||||
|
|
||||||
### Top Community Contributors
|
|
||||||
|
|
||||||
- **@kshitijk4poor** (21 PRs) — SearXNG native search backend, per-capability backend selection, collapsible TUI startup banner, Slack ephemeral ack + format fixes, Lightpanda fallback hardening, searxng-search optional skill + Web Search + Extract docs, default custom tool creation to plugins, kanban failure-column fix
|
|
||||||
- **@alt-glitch** (13 PRs) — video_analyze tool, xAI Custom Voices (voice cloning), local-backend CLI launch-directory fix, lazy-session creation regression recovery, systemd unit refresh on gateway boot
|
|
||||||
- **@OutThisLife** (9 PRs) — TUI perf — overlay render churn reduction, voice push-to-talk parity restoration (salvaging @Montbra)
|
|
||||||
- **@helix4u** (6 PRs) — Classic CLI output recovery after resize, absolute-path TUI completion, gateway model picker current-context fix, Bedrock credential probe avoidance, kanban docs fixes
|
|
||||||
- **@ethernet8023** (3 PRs) — Docker CI — don't cancel overlapping builds, :latest guard
|
|
||||||
- **@benbarclay** (3 PRs) — Docker — launch dashboard as side-process via HERMES_DASHBOARD=1
|
|
||||||
- **@austinpickett** (3 PRs) — Dashboard Plugins page, TUI /model picker overhaul with inline auth, kanban button fix
|
|
||||||
- **@sprmn24** (2 PRs) — Contributor (2 PRs)
|
|
||||||
- **@asheriif** (2 PRs) — Contributor (2 PRs)
|
|
||||||
- **@xxxigm** (2 PRs) — Contributing docs — .venv preference and test runner alignment with AGENTS.md
|
|
||||||
- **@stephenschoettler** (1 PR) — ACP — MCP E2E mock kwargs
|
|
||||||
- **@vincez-hms-coder** (1 PR) — Dashboard — Profiles management page
|
|
||||||
- **@cdanis** (1 PR) — Contributor
|
|
||||||
- **@briandevans** (1 PR) — Toolsets test — kanban assertions post-#17805
|
|
||||||
- **@heyitsaamir** (1 PR) — Contributor
|
|
||||||
|
|
||||||
### All Contributors
|
|
||||||
|
|
||||||
Thanks to everyone who contributed to v0.13.0 — commits, co-authored work, and salvaged PRs. 295 contributors in one week.
|
|
||||||
|
|
||||||
@0oAstro, @0xDevNinja, @0xharryriddle, @0xKingBack, @0xsir0000, @0xyg3n, @0z1-ghb, @abhinav11082001-stack,
|
|
||||||
@acc001k, @acesjohnny, @adamludwin, @adybag14-cyber, @agentlinker, @agilejava, @ai-ag2026, @AJV20,
|
|
||||||
@alanxchen85, @albert748, @AllardQuek, @alt-glitch, @altmazza0-star, @ambition0802, @amitgaur, @amroessam,
|
|
||||||
@andrewhosf, @Asce66, @asheriif, @ashermorse, @asimons81, @Aslaaen, @Asunfly, @atongrun, @austinpickett,
|
|
||||||
@banditburai, @barteqpl, @Bartok9, @Beandon13, @beardthelion, @beibi9966, @benbarclay, @binhnt92, @bjianhang,
|
|
||||||
@BlackJulySnow, @bobashopcashier, @bogerman1, @Bongulielmi, @Brecht-H, @briandevans, @brooklynnicholson,
|
|
||||||
@c3115644151, @camaragon, @CashWilliams, @CCClelo, @cdanis, @CES4751, @cg2aigc, @changchun989, @ChanlerDev,
|
|
||||||
@CharlieKerfoot, @chengoak, @chenyunbo411, @chinadbo, @CIRWEL, @cixuuz, @cmcgrabby-hue, @colorcross,
|
|
||||||
@Contentment003111, @CoreyNoDream, @counterposition, @curiouscleo, @DaniuXie, @deep-name, @dengtaoyuan450-a11y,
|
|
||||||
@discodirector, @donramon77, @dpaluy, @ee-blog, @ehz0ah, @el-analista, @elmatadorgh, @EmelyanenkoK,
|
|
||||||
@Emidomenge, @emozilla, @Es1la, @EthanGuo-coder, @etherman-os, @ethernet8023, @EvilDrag0n, @exxmen, @Fearvox,
|
|
||||||
@Feranmi10, @firefly, @flobo3, @fmercurio, @Foolafroos, @formulahendry, @franksong2702, @ggnnggez, @GinWU05,
|
|
||||||
@giwaov, @glesperance, @gnanirahulnutakki, @GodsBoy, @Gosuj, @Grey0202, @guillaumemeyer, @Gutslabs, @h0tp-ftw,
|
|
||||||
@haidao1919, @halmisen, @happy5318, @hedirman, @helix4u, @hendrixfreire, @HenkDz, @hex-clawd, @heyitsaamir,
|
|
||||||
@hharry11, @Hinotoi-agent, @holynn-q, @hrkzogw, @Hypn0sis, @Hypnus-Yuan, @ideathinklab01-source, @IMHaoyan,
|
|
||||||
@Interstellar-code, @ishardo, @jacdevos, @jackey8616, @JanCong, @jasonoutland, @jatingodnani, @JayGwod,
|
|
||||||
@jethac, @JezzaHehn, @JiaDe-Wu, @jjjojoj, @jkausel-ai, @John-tip, @johnncenae, @jrusso1020, @jslizar,
|
|
||||||
@JTroyerOvermatch, @julysir, @Junass1, @JustinUssuri, @Kailigithub, @keepcalmqqf, @kiala9, @konsisumer,
|
|
||||||
@kowenhaoai, @Krionex, @kshitijk4poor, @kyan12, @leavrcn, @leon7609, @LeonSGP43, @leprincep35700, @lhysdl,
|
|
||||||
@likejudy, @lisanhu, @liu-collab, @liuguangyong93, @liuhao1024, @LucianoSP, @luoyuctl, @luyao618, @M3RCUR2Y,
|
|
||||||
@maciekczech, @Magicray1217, @magicray1217, @MaHaoHao-ch, @malaiwah, @manateelazycat, @masonjames, @megastary,
|
|
||||||
@memosr, @MichaelWDanko, @mikeyobrien, @millerc79, @Mind-Dragon, @mioimotoai-lgtm, @misery-hl, @molvikar,
|
|
||||||
@momowind, @Montbra, @MottledShadow, @mrbob-git, @mrcharlesiv, @mrcoferland, @ms-alan, @mwnickerson,
|
|
||||||
@nazirulhafiy, @nftpoetrist, @nicoloboschi, @nightq, @nikolay-bratanov, @NikolayGusev-astra, @nocturnum91,
|
|
||||||
@noOne-list, @nouseman666, @novax635, @npmisantosh, @nudiltoys-cmyk, @olisikh, @oluwadareab12, @Oxidane-bot,
|
|
||||||
@pama0227, @pander, @pasevin, @paul-tian, @pdonizete, @perlowja, @pingchesu, @PratikRai0101, @priveperfumes,
|
|
||||||
@probepark, @QifengKuang, @quocanh261997, @qWaitCrypto, @qxxaa, @r266-tech, @rames-jusso, @revaraver,
|
|
||||||
@Ricardo-M-L, @rob-maron, @Roy-oss1, @rxdxxxx, @SandroHub013, @Sanjays2402, @Sertug17, @shashwatgokhe,
|
|
||||||
@shellybotmoyer, @SHL0MS, @SimbaKingjoe, @simbam99, @simplenamebox-ops, @socrates1024, @sonic-netizen,
|
|
||||||
@sprmn24, @steezkelly, @stephen0110, @stephenschoettler, @stevenchanin, @stevenchouai, @stormhierta,
|
|
||||||
@subtract0, @suncokret12, @swithek, @taeng0204, @TakeshiSawaguchi, @tangyuanjc, @TheEpTic, @thelumiereguy,
|
|
||||||
@Tkander1715, @tmdgusya, @Tranquil-Flow, @TruaShamu, @UgwujaGeorge, @valda, @vincez-hms-coder, @VinVC,
|
|
||||||
@vominh1919, @wabrent, @WadydX, @wanazhar, @WanderWang, @warabe1122, @web-dev0521, @WideLee, @willy-scr,
|
|
||||||
@wmagev, @WuTianyi123, @wxst, @wysie, @Wysie, @xsfX20, @xxxigm, @xyiy001, @YanzhongSu, @ygd58, @Yoimex,
|
|
||||||
@yuehei, @Yukipukii1, @yuqianma, @YX234, @zeejaytan, @zhanggttry, @zhao0112, @zng8418, @zons-zhaozhy, @Zyproth
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Full Changelog**: [v2026.4.30...v2026.5.7](https://github.com/NousResearch/hermes-agent/compare/v2026.4.30...v2026.5.7)
|
|
||||||
@@ -13,10 +13,6 @@ Usage::
|
|||||||
hermes-acp
|
hermes-acp
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
|
|
||||||
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
|
|
||||||
import hermes_bootstrap # noqa: F401
|
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import logging
|
import logging
|
||||||
import sys
|
import sys
|
||||||
|
|||||||
+2
-288
@@ -3,16 +3,13 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import base64
|
|
||||||
import contextvars
|
import contextvars
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
from collections import defaultdict, deque
|
from collections import defaultdict, deque
|
||||||
from concurrent.futures import ThreadPoolExecutor
|
from concurrent.futures import ThreadPoolExecutor
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Deque, Optional
|
from typing import Any, Deque, Optional
|
||||||
from urllib.parse import unquote, urlparse
|
|
||||||
|
|
||||||
import acp
|
import acp
|
||||||
from acp.schema import (
|
from acp.schema import (
|
||||||
@@ -21,7 +18,6 @@ from acp.schema import (
|
|||||||
AuthenticateResponse,
|
AuthenticateResponse,
|
||||||
AvailableCommand,
|
AvailableCommand,
|
||||||
AvailableCommandsUpdate,
|
AvailableCommandsUpdate,
|
||||||
BlobResourceContents,
|
|
||||||
ClientCapabilities,
|
ClientCapabilities,
|
||||||
EmbeddedResourceContentBlock,
|
EmbeddedResourceContentBlock,
|
||||||
ForkSessionResponse,
|
ForkSessionResponse,
|
||||||
@@ -50,7 +46,6 @@ from acp.schema import (
|
|||||||
SessionResumeCapabilities,
|
SessionResumeCapabilities,
|
||||||
SessionInfo,
|
SessionInfo,
|
||||||
TextContentBlock,
|
TextContentBlock,
|
||||||
TextResourceContents,
|
|
||||||
UnstructuredCommandInput,
|
UnstructuredCommandInput,
|
||||||
Usage,
|
Usage,
|
||||||
UsageUpdate,
|
UsageUpdate,
|
||||||
@@ -88,272 +83,6 @@ _executor = ThreadPoolExecutor(max_workers=4, thread_name_prefix="acp-agent")
|
|||||||
# does not expose a client-side limit, so this is a fixed cap that clients
|
# does not expose a client-side limit, so this is a fixed cap that clients
|
||||||
# paginate against using `cursor` / `next_cursor`.
|
# paginate against using `cursor` / `next_cursor`.
|
||||||
_LIST_SESSIONS_PAGE_SIZE = 50
|
_LIST_SESSIONS_PAGE_SIZE = 50
|
||||||
_MAX_ACP_RESOURCE_BYTES = 512 * 1024
|
|
||||||
_TEXT_RESOURCE_MIME_PREFIXES = ("text/",)
|
|
||||||
_TEXT_RESOURCE_MIME_TYPES = {
|
|
||||||
"application/json",
|
|
||||||
"application/javascript",
|
|
||||||
"application/typescript",
|
|
||||||
"application/xml",
|
|
||||||
"application/x-yaml",
|
|
||||||
"application/yaml",
|
|
||||||
"application/toml",
|
|
||||||
"application/sql",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _resource_display_name(uri: str, name: str | None = None, title: str | None = None) -> str:
|
|
||||||
"""Human-readable attachment name for prompt context."""
|
|
||||||
raw_name = (name or "").strip()
|
|
||||||
raw_title = (title or "").strip()
|
|
||||||
if raw_title and raw_name and raw_title != raw_name:
|
|
||||||
return f"{raw_title} ({raw_name})"
|
|
||||||
if raw_title:
|
|
||||||
return raw_title
|
|
||||||
if raw_name:
|
|
||||||
return raw_name
|
|
||||||
parsed = urlparse(uri)
|
|
||||||
candidate = parsed.path if parsed.scheme else uri
|
|
||||||
return Path(unquote(candidate)).name or uri or "resource"
|
|
||||||
|
|
||||||
|
|
||||||
def _is_text_resource(mime_type: str | None) -> bool:
|
|
||||||
mime = (mime_type or "").split(";", 1)[0].strip().lower()
|
|
||||||
if not mime:
|
|
||||||
return False
|
|
||||||
return mime.startswith(_TEXT_RESOURCE_MIME_PREFIXES) or mime in _TEXT_RESOURCE_MIME_TYPES
|
|
||||||
|
|
||||||
|
|
||||||
def _is_image_resource(mime_type: str | None) -> bool:
|
|
||||||
mime = (mime_type or "").split(";", 1)[0].strip().lower()
|
|
||||||
return mime.startswith("image/")
|
|
||||||
|
|
||||||
|
|
||||||
def _guess_image_mime_from_path(path: Path) -> str | None:
|
|
||||||
suffix = path.suffix.lower()
|
|
||||||
return {
|
|
||||||
".png": "image/png",
|
|
||||||
".jpg": "image/jpeg",
|
|
||||||
".jpeg": "image/jpeg",
|
|
||||||
".gif": "image/gif",
|
|
||||||
".webp": "image/webp",
|
|
||||||
".bmp": "image/bmp",
|
|
||||||
".svg": "image/svg+xml",
|
|
||||||
}.get(suffix)
|
|
||||||
|
|
||||||
|
|
||||||
def _image_data_url(data: bytes, mime_type: str) -> str:
|
|
||||||
return f"data:{mime_type};base64,{base64.b64encode(data).decode('ascii')}"
|
|
||||||
|
|
||||||
|
|
||||||
def _path_from_file_uri(uri: str) -> Path | None:
|
|
||||||
"""Convert local file URIs/paths from ACP clients into a readable Path.
|
|
||||||
|
|
||||||
Zed may send POSIX file URIs from Linux/WSL workspaces or Windows-ish paths
|
|
||||||
when launched through wsl.exe. Translate the common Windows drive form to
|
|
||||||
/mnt/<drive>/... so Hermes running in WSL can read it.
|
|
||||||
"""
|
|
||||||
raw = (uri or "").strip()
|
|
||||||
if not raw:
|
|
||||||
return None
|
|
||||||
|
|
||||||
parsed = urlparse(raw)
|
|
||||||
if parsed.scheme and parsed.scheme != "file":
|
|
||||||
return None
|
|
||||||
|
|
||||||
if parsed.scheme == "file":
|
|
||||||
if parsed.netloc and parsed.netloc not in {"", "localhost"}:
|
|
||||||
return None
|
|
||||||
path_text = unquote(parsed.path or "")
|
|
||||||
else:
|
|
||||||
path_text = unquote(raw)
|
|
||||||
|
|
||||||
# file:///C:/Users/... or C:\Users\...
|
|
||||||
if len(path_text) >= 3 and path_text[0] == "/" and path_text[2] == ":" and path_text[1].isalpha():
|
|
||||||
drive = path_text[1].lower()
|
|
||||||
rest = path_text[3:].lstrip("/\\").replace("\\", "/")
|
|
||||||
return Path("/mnt") / drive / rest
|
|
||||||
if len(path_text) >= 2 and path_text[1] == ":" and path_text[0].isalpha():
|
|
||||||
drive = path_text[0].lower()
|
|
||||||
rest = path_text[2:].lstrip("/\\").replace("\\", "/")
|
|
||||||
return Path("/mnt") / drive / rest
|
|
||||||
|
|
||||||
return Path(path_text)
|
|
||||||
|
|
||||||
|
|
||||||
def _decode_text_bytes(data: bytes, mime_type: str | None) -> str | None:
|
|
||||||
"""Decode resource bytes if they are probably text; return None for binary."""
|
|
||||||
if b"\x00" in data and not _is_text_resource(mime_type):
|
|
||||||
return None
|
|
||||||
for encoding in ("utf-8-sig", "utf-8", "latin-1"):
|
|
||||||
try:
|
|
||||||
return data.decode(encoding)
|
|
||||||
except UnicodeDecodeError:
|
|
||||||
continue
|
|
||||||
return data.decode("utf-8", errors="replace")
|
|
||||||
|
|
||||||
|
|
||||||
def _format_resource_text(
|
|
||||||
*,
|
|
||||||
uri: str,
|
|
||||||
body: str,
|
|
||||||
name: str | None = None,
|
|
||||||
title: str | None = None,
|
|
||||||
note: str | None = None,
|
|
||||||
) -> str:
|
|
||||||
display = _resource_display_name(uri, name=name, title=title)
|
|
||||||
header = f"[Attached file: {display}]"
|
|
||||||
if note:
|
|
||||||
header += f" ({note})"
|
|
||||||
return f"{header}\nURI: {uri}\n\n{body}"
|
|
||||||
|
|
||||||
|
|
||||||
def _resource_link_to_parts(block: ResourceContentBlock) -> list[dict[str, Any]]:
|
|
||||||
"""Convert an ACP resource_link block to OpenAI content parts.
|
|
||||||
|
|
||||||
Returns a list of {"type": "text", ...} and/or {"type": "image_url", ...}
|
|
||||||
parts. Image resources produce an image_url part with a small text header
|
|
||||||
so the model knows which attachment it is. Non-image resources return a
|
|
||||||
single text part with the inlined file body (or a binary-omit note).
|
|
||||||
"""
|
|
||||||
uri = str(getattr(block, "uri", "") or "").strip()
|
|
||||||
if not uri:
|
|
||||||
return []
|
|
||||||
|
|
||||||
name = str(getattr(block, "name", "") or "").strip() or None
|
|
||||||
title = str(getattr(block, "title", "") or "").strip() or None
|
|
||||||
mime_type = str(getattr(block, "mime_type", "") or "").strip() or None
|
|
||||||
path = _path_from_file_uri(uri)
|
|
||||||
|
|
||||||
if path is None:
|
|
||||||
return [{
|
|
||||||
"type": "text",
|
|
||||||
"text": _format_resource_text(
|
|
||||||
uri=uri,
|
|
||||||
name=name,
|
|
||||||
title=title,
|
|
||||||
body="[Resource link only; Hermes cannot read non-file ACP resource URIs directly.]",
|
|
||||||
),
|
|
||||||
}]
|
|
||||||
|
|
||||||
# Image files: emit a short text header + image_url data URL so vision
|
|
||||||
# models can see the attachment instead of a "binary omitted" note.
|
|
||||||
image_mime = mime_type if _is_image_resource(mime_type) else _guess_image_mime_from_path(path)
|
|
||||||
if image_mime and _is_image_resource(image_mime):
|
|
||||||
try:
|
|
||||||
size = path.stat().st_size
|
|
||||||
if size > _MAX_ACP_RESOURCE_BYTES:
|
|
||||||
return [{
|
|
||||||
"type": "text",
|
|
||||||
"text": _format_resource_text(
|
|
||||||
uri=uri,
|
|
||||||
name=name,
|
|
||||||
title=title,
|
|
||||||
body=f"[Image too large to inline: {size} bytes, cap={_MAX_ACP_RESOURCE_BYTES}]",
|
|
||||||
),
|
|
||||||
}]
|
|
||||||
with path.open("rb") as fh:
|
|
||||||
data = fh.read()
|
|
||||||
except OSError as exc:
|
|
||||||
logger.warning("ACP image resource read failed: %s", uri, exc_info=True)
|
|
||||||
return [{
|
|
||||||
"type": "text",
|
|
||||||
"text": _format_resource_text(
|
|
||||||
uri=uri,
|
|
||||||
name=name,
|
|
||||||
title=title,
|
|
||||||
body=f"[Could not read attached image: {exc}]",
|
|
||||||
),
|
|
||||||
}]
|
|
||||||
display = _resource_display_name(uri, name=name, title=title)
|
|
||||||
return [
|
|
||||||
{"type": "text", "text": f"[Attached image: {display}]\nURI: {uri}"},
|
|
||||||
{"type": "image_url", "image_url": {"url": _image_data_url(data, image_mime)}},
|
|
||||||
]
|
|
||||||
|
|
||||||
try:
|
|
||||||
size = path.stat().st_size
|
|
||||||
read_size = min(size, _MAX_ACP_RESOURCE_BYTES)
|
|
||||||
with path.open("rb") as fh:
|
|
||||||
data = fh.read(read_size)
|
|
||||||
text = _decode_text_bytes(data, mime_type)
|
|
||||||
if text is None:
|
|
||||||
return [{
|
|
||||||
"type": "text",
|
|
||||||
"text": _format_resource_text(
|
|
||||||
uri=uri,
|
|
||||||
name=name,
|
|
||||||
title=title,
|
|
||||||
body=f"[Binary file omitted: {size} bytes, mime={mime_type or 'unknown'}]",
|
|
||||||
),
|
|
||||||
}]
|
|
||||||
note = None
|
|
||||||
if size > _MAX_ACP_RESOURCE_BYTES:
|
|
||||||
note = f"truncated to {_MAX_ACP_RESOURCE_BYTES} of {size} bytes"
|
|
||||||
return [{
|
|
||||||
"type": "text",
|
|
||||||
"text": _format_resource_text(uri=uri, name=name, title=title, body=text, note=note),
|
|
||||||
}]
|
|
||||||
except OSError as exc:
|
|
||||||
logger.warning("ACP resource read failed: %s", uri, exc_info=True)
|
|
||||||
return [{
|
|
||||||
"type": "text",
|
|
||||||
"text": _format_resource_text(
|
|
||||||
uri=uri,
|
|
||||||
name=name,
|
|
||||||
title=title,
|
|
||||||
body=f"[Could not read attached file: {exc}]",
|
|
||||||
),
|
|
||||||
}]
|
|
||||||
|
|
||||||
|
|
||||||
def _embedded_resource_to_parts(block: EmbeddedResourceContentBlock) -> list[dict[str, Any]]:
|
|
||||||
resource = getattr(block, "resource", None)
|
|
||||||
if resource is None:
|
|
||||||
return []
|
|
||||||
|
|
||||||
uri = str(getattr(resource, "uri", "") or "").strip()
|
|
||||||
mime_type = str(getattr(resource, "mime_type", "") or "").strip() or None
|
|
||||||
|
|
||||||
if isinstance(resource, TextResourceContents):
|
|
||||||
return [{"type": "text", "text": _format_resource_text(uri=uri, body=resource.text)}]
|
|
||||||
|
|
||||||
if isinstance(resource, BlobResourceContents):
|
|
||||||
blob = resource.blob or ""
|
|
||||||
try:
|
|
||||||
data = base64.b64decode(blob, validate=True)
|
|
||||||
except Exception:
|
|
||||||
data = blob.encode("utf-8", errors="replace")
|
|
||||||
|
|
||||||
# Image blobs go through as image_url so vision models can see them.
|
|
||||||
if _is_image_resource(mime_type):
|
|
||||||
if len(data) > _MAX_ACP_RESOURCE_BYTES:
|
|
||||||
return [{
|
|
||||||
"type": "text",
|
|
||||||
"text": _format_resource_text(
|
|
||||||
uri=uri,
|
|
||||||
body=f"[Embedded image too large to inline: {len(data)} bytes, cap={_MAX_ACP_RESOURCE_BYTES}]",
|
|
||||||
),
|
|
||||||
}]
|
|
||||||
display = _resource_display_name(uri)
|
|
||||||
return [
|
|
||||||
{"type": "text", "text": f"[Attached image: {display}]" + (f"\nURI: {uri}" if uri else "")},
|
|
||||||
{"type": "image_url", "image_url": {"url": _image_data_url(data, mime_type or "image/png")}},
|
|
||||||
]
|
|
||||||
|
|
||||||
text = _decode_text_bytes(data[:_MAX_ACP_RESOURCE_BYTES], mime_type)
|
|
||||||
if text is None:
|
|
||||||
body = f"[Binary embedded file omitted: {len(data)} bytes, mime={mime_type or 'unknown'}]"
|
|
||||||
else:
|
|
||||||
body = text
|
|
||||||
if len(data) > _MAX_ACP_RESOURCE_BYTES:
|
|
||||||
body += f"\n\n[Truncated to {_MAX_ACP_RESOURCE_BYTES} of {len(data)} bytes]"
|
|
||||||
return [{"type": "text", "text": _format_resource_text(uri=uri, body=body)}]
|
|
||||||
|
|
||||||
text = getattr(resource, "text", None)
|
|
||||||
if text:
|
|
||||||
return [{"type": "text", "text": _format_resource_text(uri=uri, body=str(text))}]
|
|
||||||
return []
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_text(
|
def _extract_text(
|
||||||
@@ -415,20 +144,6 @@ def _content_blocks_to_openai_user_content(
|
|||||||
if image_part is not None:
|
if image_part is not None:
|
||||||
parts.append(image_part)
|
parts.append(image_part)
|
||||||
continue
|
continue
|
||||||
if isinstance(block, ResourceContentBlock):
|
|
||||||
resource_parts = _resource_link_to_parts(block)
|
|
||||||
for part in resource_parts:
|
|
||||||
parts.append(part)
|
|
||||||
if part.get("type") == "text":
|
|
||||||
text_parts.append(part["text"])
|
|
||||||
continue
|
|
||||||
if isinstance(block, EmbeddedResourceContentBlock):
|
|
||||||
resource_parts = _embedded_resource_to_parts(block)
|
|
||||||
for part in resource_parts:
|
|
||||||
parts.append(part)
|
|
||||||
if part.get("type") == "text":
|
|
||||||
text_parts.append(part["text"])
|
|
||||||
continue
|
|
||||||
|
|
||||||
if not parts:
|
if not parts:
|
||||||
return _extract_text(prompt)
|
return _extract_text(prompt)
|
||||||
@@ -1088,7 +803,6 @@ class HermesACPAgent(acp.Agent):
|
|||||||
|
|
||||||
user_text = _extract_text(prompt).strip()
|
user_text = _extract_text(prompt).strip()
|
||||||
user_content = _content_blocks_to_openai_user_content(prompt)
|
user_content = _content_blocks_to_openai_user_content(prompt)
|
||||||
text_only_prompt = all(isinstance(block, TextContentBlock) for block in prompt)
|
|
||||||
has_content = bool(user_text) or (
|
has_content = bool(user_text) or (
|
||||||
isinstance(user_content, list) and bool(user_content)
|
isinstance(user_content, list) and bool(user_content)
|
||||||
)
|
)
|
||||||
@@ -1107,7 +821,7 @@ class HermesACPAgent(acp.Agent):
|
|||||||
# silently append to state.queued_prompts and respond with
|
# silently append to state.queued_prompts and respond with
|
||||||
# "No active turn — queued for the next turn", which looks like
|
# "No active turn — queued for the next turn", which looks like
|
||||||
# /queue even though the user never typed /queue.
|
# /queue even though the user never typed /queue.
|
||||||
if text_only_prompt and isinstance(user_content, str) and user_text.startswith("/steer"):
|
if isinstance(user_content, str) and user_text.startswith("/steer"):
|
||||||
steer_text = user_text.split(maxsplit=1)[1].strip() if len(user_text.split(maxsplit=1)) > 1 else ""
|
steer_text = user_text.split(maxsplit=1)[1].strip() if len(user_text.split(maxsplit=1)) > 1 else ""
|
||||||
interrupted_prompt = ""
|
interrupted_prompt = ""
|
||||||
rewrite_idle = False
|
rewrite_idle = False
|
||||||
@@ -1132,7 +846,7 @@ class HermesACPAgent(acp.Agent):
|
|||||||
# Slash commands are text-only; if the client included images/resources,
|
# Slash commands are text-only; if the client included images/resources,
|
||||||
# send the whole multimodal prompt to the agent instead of treating it as
|
# send the whole multimodal prompt to the agent instead of treating it as
|
||||||
# an ACP command.
|
# an ACP command.
|
||||||
if text_only_prompt and isinstance(user_content, str) and user_text.startswith("/"):
|
if isinstance(user_content, str) and user_text.startswith("/"):
|
||||||
response_text = self._handle_slash_command(user_text, state)
|
response_text = self._handle_slash_command(user_text, state)
|
||||||
if response_text is not None:
|
if response_text is not None:
|
||||||
if self._conn:
|
if self._conn:
|
||||||
|
|||||||
+32
-35
@@ -231,30 +231,33 @@ def _supports_fast_mode(model: str) -> bool:
|
|||||||
return any(v in model for v in _FAST_MODE_SUPPORTED_SUBSTRINGS)
|
return any(v in model for v in _FAST_MODE_SUPPORTED_SUBSTRINGS)
|
||||||
|
|
||||||
|
|
||||||
# Beta headers for enhanced features that are safe on ordinary/native Anthropic
|
# Beta headers for enhanced features (sent with ALL auth types).
|
||||||
# requests. As of Opus 4.7 (2026-04-16), these are GA on Claude 4.6+ — the
|
# As of Opus 4.7 (2026-04-16), the first two are GA on Claude 4.6+ — the
|
||||||
# beta headers are still accepted (harmless no-op) but not required. Kept
|
# beta headers are still accepted (harmless no-op) but not required. Kept
|
||||||
# here so older Claude (4.5, 4.1) + compatible endpoints that still gate on
|
# here so older Claude (4.5, 4.1) + third-party Anthropic-compat endpoints
|
||||||
# the headers continue to get the enhanced features.
|
# that still gate on the headers continue to get the enhanced features.
|
||||||
#
|
#
|
||||||
# Do NOT include ``context-1m-2025-08-07`` here. Anthropic returns HTTP 400
|
# ``context-1m-2025-08-07`` unlocks the 1M context window on Claude Opus 4.6/4.7
|
||||||
# ("long context beta is not yet available for this subscription") for
|
# and Sonnet 4.6 when served via AWS Bedrock or Azure AI Foundry. 1M is GA on
|
||||||
# accounts without the long-context beta, which breaks normal short auxiliary
|
# native Anthropic (api.anthropic.com) for Opus 4.6+, but Bedrock/Azure still
|
||||||
# calls like title generation/session summarization.
|
# gate it behind this beta header as of 2026-04 — without it Bedrock caps Opus
|
||||||
|
# at 200K even though model_metadata.py advertises 1M. The header is a harmless
|
||||||
|
# no-op on endpoints where 1M is GA.
|
||||||
#
|
#
|
||||||
# ``context-1m-2025-08-07`` is still required to unlock the 1M context window
|
# Migration guide: remove these if you no longer support ≤4.5 models or once
|
||||||
# on Claude Opus 4.6/4.7 and Sonnet 4.6 when served via AWS Bedrock or Azure
|
# Bedrock/Azure promote 1M to GA.
|
||||||
# AI Foundry. Add it only for those endpoint-specific paths below.
|
|
||||||
_COMMON_BETAS = [
|
_COMMON_BETAS = [
|
||||||
"interleaved-thinking-2025-05-14",
|
"interleaved-thinking-2025-05-14",
|
||||||
"fine-grained-tool-streaming-2025-05-14",
|
"fine-grained-tool-streaming-2025-05-14",
|
||||||
|
"context-1m-2025-08-07",
|
||||||
]
|
]
|
||||||
# MiniMax's Anthropic-compatible endpoints fail tool-use requests when
|
# MiniMax's Anthropic-compatible endpoints fail tool-use requests when
|
||||||
# the fine-grained tool streaming beta is present. Omit it so tool calls
|
# the fine-grained tool streaming beta is present. Omit it so tool calls
|
||||||
# fall back to the provider's default response path.
|
# fall back to the provider's default response path.
|
||||||
_TOOL_STREAMING_BETA = "fine-grained-tool-streaming-2025-05-14"
|
_TOOL_STREAMING_BETA = "fine-grained-tool-streaming-2025-05-14"
|
||||||
# 1M context beta. Native Anthropic does not get this by default because some
|
# 1M context beta — see comment on _COMMON_BETAS above. Stripped for
|
||||||
# subscriptions reject it, but Bedrock/Azure still need it for 1M context.
|
# Bearer-auth (MiniMax) endpoints since they host their own models and
|
||||||
|
# unknown Anthropic beta headers risk request rejection.
|
||||||
_CONTEXT_1M_BETA = "context-1m-2025-08-07"
|
_CONTEXT_1M_BETA = "context-1m-2025-08-07"
|
||||||
|
|
||||||
# Fast mode beta — enables the ``speed: "fast"`` request parameter for
|
# Fast mode beta — enables the ``speed: "fast"`` request parameter for
|
||||||
@@ -473,14 +476,6 @@ def _requires_bearer_auth(base_url: str | None) -> bool:
|
|||||||
return normalized.startswith(("https://api.minimax.io/anthropic", "https://api.minimaxi.com/anthropic"))
|
return normalized.startswith(("https://api.minimax.io/anthropic", "https://api.minimaxi.com/anthropic"))
|
||||||
|
|
||||||
|
|
||||||
def _base_url_needs_context_1m_beta(base_url: str | None) -> bool:
|
|
||||||
"""Return True for endpoints that still gate 1M context behind a beta."""
|
|
||||||
normalized = _normalize_base_url_text(base_url).lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
return "azure.com" in normalized
|
|
||||||
|
|
||||||
|
|
||||||
def _common_betas_for_base_url(
|
def _common_betas_for_base_url(
|
||||||
base_url: str | None,
|
base_url: str | None,
|
||||||
*,
|
*,
|
||||||
@@ -490,25 +485,27 @@ def _common_betas_for_base_url(
|
|||||||
|
|
||||||
MiniMax's Anthropic-compatible endpoints (Bearer-auth) reject requests
|
MiniMax's Anthropic-compatible endpoints (Bearer-auth) reject requests
|
||||||
that include Anthropic's ``fine-grained-tool-streaming`` beta — every
|
that include Anthropic's ``fine-grained-tool-streaming`` beta — every
|
||||||
tool-use message triggers a connection error.
|
tool-use message triggers a connection error. Strip that beta for
|
||||||
|
Bearer-auth endpoints while keeping all other betas intact.
|
||||||
|
|
||||||
The ``context-1m-2025-08-07`` beta is not sent to native Anthropic by
|
The ``context-1m-2025-08-07`` beta is also stripped for Bearer-auth
|
||||||
default because some subscriptions reject it. Add it only for endpoint
|
endpoints — MiniMax hosts its own models, not Claude, so the header is
|
||||||
families that still require it for 1M context, currently Azure AI Foundry.
|
irrelevant at best and risks request rejection at worst.
|
||||||
Bedrock uses its own client helper below and opts in explicitly.
|
|
||||||
|
|
||||||
``drop_context_1m_beta=True`` strips the 1M-context beta from any path that
|
``drop_context_1m_beta=True`` additionally strips the 1M-context beta on
|
||||||
would otherwise include it after a subscription/endpoint rejects the beta.
|
otherwise-unrelated endpoints. The OAuth retry path flips this flag after
|
||||||
|
a subscription rejects the beta with
|
||||||
|
"The long context beta is not yet available for this subscription" so
|
||||||
|
subsequent requests in the same session don't repeat the probe. See the
|
||||||
|
reactive recovery loop in ``run_agent.py`` and issue-comment history on
|
||||||
|
PR #17680 for the full rationale.
|
||||||
"""
|
"""
|
||||||
betas = list(_COMMON_BETAS)
|
|
||||||
if _base_url_needs_context_1m_beta(base_url) and not drop_context_1m_beta:
|
|
||||||
betas.append(_CONTEXT_1M_BETA)
|
|
||||||
if _requires_bearer_auth(base_url):
|
if _requires_bearer_auth(base_url):
|
||||||
_stripped = {_TOOL_STREAMING_BETA, _CONTEXT_1M_BETA}
|
_stripped = {_TOOL_STREAMING_BETA, _CONTEXT_1M_BETA}
|
||||||
return [b for b in betas if b not in _stripped]
|
return [b for b in _COMMON_BETAS if b not in _stripped]
|
||||||
if drop_context_1m_beta:
|
if drop_context_1m_beta:
|
||||||
return [b for b in betas if b != _CONTEXT_1M_BETA]
|
return [b for b in _COMMON_BETAS if b != _CONTEXT_1M_BETA]
|
||||||
return betas
|
return _COMMON_BETAS
|
||||||
|
|
||||||
|
|
||||||
def build_anthropic_client(
|
def build_anthropic_client(
|
||||||
@@ -645,7 +642,7 @@ def build_anthropic_bedrock_client(region: str):
|
|||||||
return _anthropic_sdk.AnthropicBedrock(
|
return _anthropic_sdk.AnthropicBedrock(
|
||||||
aws_region=region,
|
aws_region=region,
|
||||||
timeout=Timeout(timeout=900.0, connect=10.0),
|
timeout=Timeout(timeout=900.0, connect=10.0),
|
||||||
default_headers={"anthropic-beta": ",".join([*_COMMON_BETAS, _CONTEXT_1M_BETA])},
|
default_headers={"anthropic-beta": ",".join(_COMMON_BETAS)},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+6
-127
@@ -455,12 +455,6 @@ def _to_openai_base_url(base_url: str) -> str:
|
|||||||
"""
|
"""
|
||||||
url = str(base_url or "").strip().rstrip("/")
|
url = str(base_url or "").strip().rstrip("/")
|
||||||
if url.endswith("/anthropic"):
|
if url.endswith("/anthropic"):
|
||||||
# ZAI (open.bigmodel.cn) uses /api/anthropic for Anthropic wire
|
|
||||||
# but /api/paas/v4 for OpenAI wire — the generic /v1 rewrite is wrong.
|
|
||||||
if "open.bigmodel.cn" in url or "bigmodel" in url:
|
|
||||||
rewritten = url[: -len("/anthropic")] + "/paas/v4"
|
|
||||||
logger.debug("Auxiliary client: rewrote ZAI base URL %s → %s", url, rewritten)
|
|
||||||
return rewritten
|
|
||||||
rewritten = url[: -len("/anthropic")] + "/v1"
|
rewritten = url[: -len("/anthropic")] + "/v1"
|
||||||
logger.debug("Auxiliary client: rewrote base URL %s → %s", url, rewritten)
|
logger.debug("Auxiliary client: rewrote base URL %s → %s", url, rewritten)
|
||||||
return rewritten
|
return rewritten
|
||||||
@@ -602,14 +596,6 @@ class _CodexCompletionsAdapter:
|
|||||||
"store": False,
|
"store": False,
|
||||||
}
|
}
|
||||||
|
|
||||||
# Preserve the chat.completions timeout contract. This adapter is used
|
|
||||||
# by auxiliary calls such as context compression; if the timeout is not
|
|
||||||
# forwarded and enforced, a Codex Responses stream can sit behind a
|
|
||||||
# dead-looking CLI until the user force-interrupts the whole session.
|
|
||||||
timeout = kwargs.get("timeout")
|
|
||||||
if timeout is not None:
|
|
||||||
resp_kwargs["timeout"] = timeout
|
|
||||||
|
|
||||||
# Note: the Codex endpoint (chatgpt.com/backend-api/codex) does NOT
|
# Note: the Codex endpoint (chatgpt.com/backend-api/codex) does NOT
|
||||||
# support max_output_tokens or temperature — omit to avoid 400 errors.
|
# support max_output_tokens or temperature — omit to avoid 400 errors.
|
||||||
|
|
||||||
@@ -667,37 +653,6 @@ class _CodexCompletionsAdapter:
|
|||||||
text_parts: List[str] = []
|
text_parts: List[str] = []
|
||||||
tool_calls_raw: List[Any] = []
|
tool_calls_raw: List[Any] = []
|
||||||
usage = None
|
usage = None
|
||||||
total_timeout = timeout if isinstance(timeout, (int, float)) and timeout > 0 else None
|
|
||||||
deadline = time.monotonic() + float(total_timeout) if total_timeout else None
|
|
||||||
timed_out = threading.Event()
|
|
||||||
timeout_timer: Optional[threading.Timer] = None
|
|
||||||
|
|
||||||
def _timeout_message() -> str:
|
|
||||||
return f"Codex auxiliary Responses stream exceeded {float(total_timeout):.1f}s total timeout"
|
|
||||||
|
|
||||||
def _close_client_on_timeout() -> None:
|
|
||||||
timed_out.set()
|
|
||||||
close = getattr(self._client, "close", None)
|
|
||||||
if callable(close):
|
|
||||||
try:
|
|
||||||
close()
|
|
||||||
except Exception:
|
|
||||||
logger.debug("Codex auxiliary: client close during timeout failed", exc_info=True)
|
|
||||||
|
|
||||||
def _check_cancelled() -> None:
|
|
||||||
if deadline is not None and time.monotonic() >= deadline:
|
|
||||||
timed_out.set()
|
|
||||||
raise TimeoutError(_timeout_message())
|
|
||||||
try:
|
|
||||||
from tools.interrupt import is_interrupted
|
|
||||||
if is_interrupted():
|
|
||||||
raise InterruptedError("Codex auxiliary Responses stream interrupted")
|
|
||||||
except InterruptedError:
|
|
||||||
raise
|
|
||||||
except Exception:
|
|
||||||
# Interrupt state is a best-effort UX hook; never make it a
|
|
||||||
# new failure mode for auxiliary calls.
|
|
||||||
pass
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Collect output items and text deltas during streaming —
|
# Collect output items and text deltas during streaming —
|
||||||
@@ -706,14 +661,8 @@ class _CodexCompletionsAdapter:
|
|||||||
collected_output_items: List[Any] = []
|
collected_output_items: List[Any] = []
|
||||||
collected_text_deltas: List[str] = []
|
collected_text_deltas: List[str] = []
|
||||||
has_function_calls = False
|
has_function_calls = False
|
||||||
if total_timeout:
|
|
||||||
timeout_timer = threading.Timer(float(total_timeout), _close_client_on_timeout)
|
|
||||||
timeout_timer.daemon = True
|
|
||||||
timeout_timer.start()
|
|
||||||
_check_cancelled()
|
|
||||||
with self._client.responses.stream(**resp_kwargs) as stream:
|
with self._client.responses.stream(**resp_kwargs) as stream:
|
||||||
for _event in stream:
|
for _event in stream:
|
||||||
_check_cancelled()
|
|
||||||
_etype = getattr(_event, "type", "")
|
_etype = getattr(_event, "type", "")
|
||||||
if _etype == "response.output_item.done":
|
if _etype == "response.output_item.done":
|
||||||
_done = getattr(_event, "item", None)
|
_done = getattr(_event, "item", None)
|
||||||
@@ -725,7 +674,6 @@ class _CodexCompletionsAdapter:
|
|||||||
collected_text_deltas.append(_delta)
|
collected_text_deltas.append(_delta)
|
||||||
elif "function_call" in _etype:
|
elif "function_call" in _etype:
|
||||||
has_function_calls = True
|
has_function_calls = True
|
||||||
_check_cancelled()
|
|
||||||
final = stream.get_final_response()
|
final = stream.get_final_response()
|
||||||
|
|
||||||
# Backfill empty output from collected stream events
|
# Backfill empty output from collected stream events
|
||||||
@@ -785,13 +733,8 @@ class _CodexCompletionsAdapter:
|
|||||||
total_tokens=getattr(resp_usage, "total_tokens", 0),
|
total_tokens=getattr(resp_usage, "total_tokens", 0),
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
if timed_out.is_set():
|
|
||||||
raise TimeoutError(_timeout_message()) from exc
|
|
||||||
logger.debug("Codex auxiliary Responses API call failed: %s", exc)
|
logger.debug("Codex auxiliary Responses API call failed: %s", exc)
|
||||||
raise
|
raise
|
||||||
finally:
|
|
||||||
if timeout_timer is not None:
|
|
||||||
timeout_timer.cancel()
|
|
||||||
|
|
||||||
content = "".join(text_parts).strip() or None
|
content = "".join(text_parts).strip() or None
|
||||||
|
|
||||||
@@ -885,14 +828,7 @@ class _AnthropicCompletionsAdapter:
|
|||||||
model = kwargs.get("model", self._model)
|
model = kwargs.get("model", self._model)
|
||||||
tools = kwargs.get("tools")
|
tools = kwargs.get("tools")
|
||||||
tool_choice = kwargs.get("tool_choice")
|
tool_choice = kwargs.get("tool_choice")
|
||||||
# ZAI's Anthropic-compatible endpoint rejects max_tokens on vision
|
max_tokens = kwargs.get("max_tokens") or kwargs.get("max_completion_tokens") or 2000
|
||||||
# models (glm-4v-flash etc.) with error code 1210. When the caller
|
|
||||||
# signals this by setting _skip_zai_max_tokens in kwargs, omit it.
|
|
||||||
_skip_mt = kwargs.pop("_skip_zai_max_tokens", False)
|
|
||||||
if _skip_mt:
|
|
||||||
max_tokens = None
|
|
||||||
else:
|
|
||||||
max_tokens = kwargs.get("max_tokens") or kwargs.get("max_completion_tokens") or 2000
|
|
||||||
temperature = kwargs.get("temperature")
|
temperature = kwargs.get("temperature")
|
||||||
|
|
||||||
normalized_tool_choice = None
|
normalized_tool_choice = None
|
||||||
@@ -2899,33 +2835,6 @@ def resolve_vision_provider_client(
|
|||||||
)
|
)
|
||||||
return _finalize(requested, sync_client, default_model)
|
return _finalize(requested, sync_client, default_model)
|
||||||
|
|
||||||
# ZAI vision models must use the OpenAI-compatible endpoint, not the
|
|
||||||
# Anthropic-compatible one (which may be the main-runtime default).
|
|
||||||
# The Anthropic wire rejects max_tokens on multimodal calls (error 1210),
|
|
||||||
# while the OpenAI wire handles it correctly.
|
|
||||||
if requested == "zai" and not resolved_base_url:
|
|
||||||
zai_openai_urls = [
|
|
||||||
"https://open.bigmodel.cn/api/paas/v4",
|
|
||||||
"https://api.z.ai/api/paas/v4",
|
|
||||||
]
|
|
||||||
for _zai_url in zai_openai_urls:
|
|
||||||
client, final_model = _get_cached_client(
|
|
||||||
requested, resolved_model, async_mode,
|
|
||||||
base_url=_zai_url,
|
|
||||||
api_key=resolved_api_key or None,
|
|
||||||
api_mode="chat_completions",
|
|
||||||
is_vision=True,
|
|
||||||
)
|
|
||||||
if client is not None:
|
|
||||||
return _finalize(requested, client, final_model)
|
|
||||||
# Fallback: try without explicit base_url (old behavior)
|
|
||||||
client, final_model = _get_cached_client(requested, resolved_model, async_mode,
|
|
||||||
api_mode=resolved_api_mode,
|
|
||||||
is_vision=True)
|
|
||||||
if client is None:
|
|
||||||
return requested, None, None
|
|
||||||
return requested, client, final_model
|
|
||||||
|
|
||||||
client, final_model = _get_cached_client(requested, resolved_model, async_mode,
|
client, final_model = _get_cached_client(requested, resolved_model, async_mode,
|
||||||
api_mode=resolved_api_mode,
|
api_mode=resolved_api_mode,
|
||||||
is_vision=True)
|
is_vision=True)
|
||||||
@@ -2953,11 +2862,10 @@ def auxiliary_max_tokens_param(value: int) -> dict:
|
|||||||
"""
|
"""
|
||||||
custom_base = _current_custom_base_url()
|
custom_base = _current_custom_base_url()
|
||||||
or_key = os.getenv("OPENROUTER_API_KEY")
|
or_key = os.getenv("OPENROUTER_API_KEY")
|
||||||
# Use max_completion_tokens for direct OpenAI-compatible providers that reject
|
# Only use max_completion_tokens for direct OpenAI custom endpoints
|
||||||
# max_tokens on newer GPT-4o/o-series/GPT-5-style models.
|
|
||||||
if (not or_key
|
if (not or_key
|
||||||
and _read_nous_auth() is None
|
and _read_nous_auth() is None
|
||||||
and base_url_hostname(custom_base) in {"api.openai.com", "api.githubcopilot.com"}):
|
and base_url_hostname(custom_base) == "api.openai.com"):
|
||||||
return {"max_completion_tokens": value}
|
return {"max_completion_tokens": value}
|
||||||
return {"max_tokens": value}
|
return {"max_tokens": value}
|
||||||
|
|
||||||
@@ -3485,16 +3393,7 @@ def _build_call_kwargs(
|
|||||||
if max_tokens is not None:
|
if max_tokens is not None:
|
||||||
# Codex adapter handles max_tokens internally; OpenRouter/Nous use max_tokens.
|
# Codex adapter handles max_tokens internally; OpenRouter/Nous use max_tokens.
|
||||||
# Direct OpenAI api.openai.com with newer models needs max_completion_tokens.
|
# Direct OpenAI api.openai.com with newer models needs max_completion_tokens.
|
||||||
# ZAI vision models (glm-4v-flash, glm-4v-plus, etc.) reject max_tokens with
|
if provider == "custom":
|
||||||
# error code 1210 ("API 调用参数有误") on multimodal requests — skip it.
|
|
||||||
_model_lower = (model or "").lower()
|
|
||||||
_skip_max_tokens = (
|
|
||||||
provider == "zai"
|
|
||||||
and ("4v" in _model_lower or "5v" in _model_lower or "-v" in _model_lower)
|
|
||||||
)
|
|
||||||
if _skip_max_tokens:
|
|
||||||
pass # ZAI vision models do not accept max_tokens
|
|
||||||
elif provider == "custom":
|
|
||||||
custom_base = base_url or _current_custom_base_url()
|
custom_base = base_url or _current_custom_base_url()
|
||||||
if base_url_hostname(custom_base) == "api.openai.com":
|
if base_url_hostname(custom_base) == "api.openai.com":
|
||||||
kwargs["max_completion_tokens"] = max_tokens
|
kwargs["max_completion_tokens"] = max_tokens
|
||||||
@@ -3725,23 +3624,13 @@ def call_llm(
|
|||||||
kwargs = retry_kwargs
|
kwargs = retry_kwargs
|
||||||
|
|
||||||
err_str = str(first_err)
|
err_str = str(first_err)
|
||||||
# ZAI vision models (glm-4v-flash etc.) return error code 1210
|
|
||||||
# ("API 调用参数有误") when max_tokens is passed on multimodal
|
|
||||||
# calls. The error message does NOT contain "max_tokens" so the
|
|
||||||
# generic retry below never fires. Detect the ZAI-specific error
|
|
||||||
# and strip max_tokens before retrying.
|
|
||||||
_is_zai_param_error = (
|
|
||||||
"1210" in err_str
|
|
||||||
and "bigmodel" in str(getattr(client, "base_url", ""))
|
|
||||||
)
|
|
||||||
if max_tokens is not None and (
|
if max_tokens is not None and (
|
||||||
"max_tokens" in err_str
|
"max_tokens" in err_str
|
||||||
or "unsupported_parameter" in err_str
|
or "unsupported_parameter" in err_str
|
||||||
or _is_unsupported_parameter_error(first_err, "max_tokens")
|
or _is_unsupported_parameter_error(first_err, "max_tokens")
|
||||||
or _is_zai_param_error
|
|
||||||
):
|
):
|
||||||
kwargs.pop("max_tokens", None)
|
kwargs.pop("max_tokens", None)
|
||||||
kwargs.pop("max_completion_tokens", None)
|
kwargs["max_completion_tokens"] = max_tokens
|
||||||
try:
|
try:
|
||||||
return _validate_llm_response(
|
return _validate_llm_response(
|
||||||
client.chat.completions.create(**kwargs), task)
|
client.chat.completions.create(**kwargs), task)
|
||||||
@@ -4041,23 +3930,13 @@ async def async_call_llm(
|
|||||||
kwargs = retry_kwargs
|
kwargs = retry_kwargs
|
||||||
|
|
||||||
err_str = str(first_err)
|
err_str = str(first_err)
|
||||||
# ZAI vision models (glm-4v-flash etc.) return error code 1210
|
|
||||||
# ("API 调用参数有误") when max_tokens is passed on multimodal
|
|
||||||
# calls. The error message does NOT contain "max_tokens" so the
|
|
||||||
# generic retry below never fires. Detect the ZAI-specific error
|
|
||||||
# and strip max_tokens before retrying.
|
|
||||||
_is_zai_param_error = (
|
|
||||||
"1210" in err_str
|
|
||||||
and "bigmodel" in str(getattr(client, "base_url", ""))
|
|
||||||
)
|
|
||||||
if max_tokens is not None and (
|
if max_tokens is not None and (
|
||||||
"max_tokens" in err_str
|
"max_tokens" in err_str
|
||||||
or "unsupported_parameter" in err_str
|
or "unsupported_parameter" in err_str
|
||||||
or _is_unsupported_parameter_error(first_err, "max_tokens")
|
or _is_unsupported_parameter_error(first_err, "max_tokens")
|
||||||
or _is_zai_param_error
|
|
||||||
):
|
):
|
||||||
kwargs.pop("max_tokens", None)
|
kwargs.pop("max_tokens", None)
|
||||||
kwargs.pop("max_completion_tokens", None)
|
kwargs["max_completion_tokens"] = max_tokens
|
||||||
try:
|
try:
|
||||||
return _validate_llm_response(
|
return _validate_llm_response(
|
||||||
await client.chat.completions.create(**kwargs), task)
|
await client.chat.completions.create(**kwargs), task)
|
||||||
|
|||||||
@@ -631,18 +631,11 @@ def normalize_converse_response(response: Dict) -> SimpleNamespace:
|
|||||||
stop_reason = response.get("stopReason", "end_turn")
|
stop_reason = response.get("stopReason", "end_turn")
|
||||||
|
|
||||||
text_parts = []
|
text_parts = []
|
||||||
reasoning_parts = []
|
|
||||||
tool_calls = []
|
tool_calls = []
|
||||||
|
|
||||||
for block in content_blocks:
|
for block in content_blocks:
|
||||||
if "text" in block:
|
if "text" in block:
|
||||||
text_parts.append(block["text"])
|
text_parts.append(block["text"])
|
||||||
elif "reasoningContent" in block:
|
|
||||||
reasoning = block["reasoningContent"]
|
|
||||||
if isinstance(reasoning, dict):
|
|
||||||
thinking_text = reasoning.get("text", "")
|
|
||||||
if thinking_text:
|
|
||||||
reasoning_parts.append(str(thinking_text))
|
|
||||||
elif "toolUse" in block:
|
elif "toolUse" in block:
|
||||||
tu = block["toolUse"]
|
tu = block["toolUse"]
|
||||||
tool_calls.append(SimpleNamespace(
|
tool_calls.append(SimpleNamespace(
|
||||||
@@ -659,7 +652,6 @@ def normalize_converse_response(response: Dict) -> SimpleNamespace:
|
|||||||
role="assistant",
|
role="assistant",
|
||||||
content="\n".join(text_parts) if text_parts else None,
|
content="\n".join(text_parts) if text_parts else None,
|
||||||
tool_calls=tool_calls if tool_calls else None,
|
tool_calls=tool_calls if tool_calls else None,
|
||||||
reasoning_content="\n\n".join(reasoning_parts) if reasoning_parts else None,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# Build usage stats
|
# Build usage stats
|
||||||
@@ -740,7 +732,6 @@ def stream_converse_with_callbacks(
|
|||||||
``normalize_converse_response()``.
|
``normalize_converse_response()``.
|
||||||
"""
|
"""
|
||||||
text_parts: List[str] = []
|
text_parts: List[str] = []
|
||||||
reasoning_parts: List[str] = []
|
|
||||||
tool_calls: List[SimpleNamespace] = []
|
tool_calls: List[SimpleNamespace] = []
|
||||||
current_tool: Optional[Dict] = None
|
current_tool: Optional[Dict] = None
|
||||||
current_text_buffer: List[str] = []
|
current_text_buffer: List[str] = []
|
||||||
@@ -786,10 +777,8 @@ def stream_converse_with_callbacks(
|
|||||||
reasoning = delta["reasoningContent"]
|
reasoning = delta["reasoningContent"]
|
||||||
if isinstance(reasoning, dict):
|
if isinstance(reasoning, dict):
|
||||||
thinking_text = reasoning.get("text", "")
|
thinking_text = reasoning.get("text", "")
|
||||||
if thinking_text:
|
if thinking_text and on_reasoning_delta:
|
||||||
reasoning_parts.append(str(thinking_text))
|
on_reasoning_delta(thinking_text)
|
||||||
if on_reasoning_delta:
|
|
||||||
on_reasoning_delta(thinking_text)
|
|
||||||
|
|
||||||
elif "contentBlockStop" in event:
|
elif "contentBlockStop" in event:
|
||||||
if current_tool is not None:
|
if current_tool is not None:
|
||||||
@@ -828,7 +817,6 @@ def stream_converse_with_callbacks(
|
|||||||
role="assistant",
|
role="assistant",
|
||||||
content="\n".join(text_parts) if text_parts else None,
|
content="\n".join(text_parts) if text_parts else None,
|
||||||
tool_calls=tool_calls if tool_calls else None,
|
tool_calls=tool_calls if tool_calls else None,
|
||||||
reasoning_content="\n\n".join(reasoning_parts) if reasoning_parts else None,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
usage = SimpleNamespace(
|
usage = SimpleNamespace(
|
||||||
|
|||||||
+12
-10
@@ -6,7 +6,8 @@ protecting head and tail context.
|
|||||||
|
|
||||||
Improvements over v2:
|
Improvements over v2:
|
||||||
- Structured summary template with Resolved/Pending question tracking
|
- Structured summary template with Resolved/Pending question tracking
|
||||||
- Filter-safe summarizer preamble that treats prior turns as source material
|
- Summarizer preamble: "Do not respond to any questions" (from OpenCode)
|
||||||
|
- Handoff framing: "different assistant" (from Codex) to create separation
|
||||||
- "Remaining Work" replaces "Next Steps" to avoid reading as active instructions
|
- "Remaining Work" replaces "Next Steps" to avoid reading as active instructions
|
||||||
- Clear separator when summary merges into tail message
|
- Clear separator when summary merges into tail message
|
||||||
- Iterative summary updates (preserves info across multiple compactions)
|
- Iterative summary updates (preserves info across multiple compactions)
|
||||||
@@ -754,14 +755,15 @@ class ContextCompressor(ContextEngine):
|
|||||||
content_to_summarize = self._serialize_for_summary(turns_to_summarize)
|
content_to_summarize = self._serialize_for_summary(turns_to_summarize)
|
||||||
|
|
||||||
# Preamble shared by both first-compaction and iterative-update prompts.
|
# Preamble shared by both first-compaction and iterative-update prompts.
|
||||||
# Keep the wording deliberately plain: Azure/OpenAI-compatible content
|
# Inspired by OpenCode's "do not respond to any questions" instruction
|
||||||
# filters have flagged stronger "injection" / "do not respond" framing.
|
# and Codex's "another language model" framing.
|
||||||
_summarizer_preamble = (
|
_summarizer_preamble = (
|
||||||
"You are a summarization agent creating a context checkpoint. "
|
"You are a summarization agent creating a context checkpoint. "
|
||||||
"Treat the conversation turns below as source material for a "
|
"Your output will be injected as reference material for a DIFFERENT "
|
||||||
"compact record of prior work. "
|
"assistant that continues the conversation. "
|
||||||
"Produce only the structured summary; do not add a greeting, "
|
"Do NOT respond to any questions or requests in the conversation — "
|
||||||
"preamble, or prefix. "
|
"only output the structured summary. "
|
||||||
|
"Do NOT include any preamble, greeting, or prefix. "
|
||||||
"Write the summary in the same language the user was using in the "
|
"Write the summary in the same language the user was using in the "
|
||||||
"conversation — do not translate or switch to English. "
|
"conversation — do not translate or switch to English. "
|
||||||
"NEVER include API keys, tokens, passwords, secrets, credentials, "
|
"NEVER include API keys, tokens, passwords, secrets, credentials, "
|
||||||
@@ -775,7 +777,7 @@ class ContextCompressor(ContextEngine):
|
|||||||
[THE SINGLE MOST IMPORTANT FIELD. Copy the user's most recent request or
|
[THE SINGLE MOST IMPORTANT FIELD. Copy the user's most recent request or
|
||||||
task assignment verbatim — the exact words they used. If multiple tasks
|
task assignment verbatim — the exact words they used. If multiple tasks
|
||||||
were requested and only some are done, list only the ones NOT yet completed.
|
were requested and only some are done, list only the ones NOT yet completed.
|
||||||
Continuation should pick up exactly here. Example:
|
The next assistant must pick up exactly here. Example:
|
||||||
"User asked: 'Now refactor the auth module to use JWT instead of sessions'"
|
"User asked: 'Now refactor the auth module to use JWT instead of sessions'"
|
||||||
If no outstanding task exists, write "None."]
|
If no outstanding task exists, write "None."]
|
||||||
|
|
||||||
@@ -812,7 +814,7 @@ Be specific with file paths, commands, line numbers, and results.]
|
|||||||
[Important technical decisions and WHY they were made]
|
[Important technical decisions and WHY they were made]
|
||||||
|
|
||||||
## Resolved Questions
|
## Resolved Questions
|
||||||
[Questions the user asked that were ALREADY answered — include the answer so it is not repeated]
|
[Questions the user asked that were ALREADY answered — include the answer so the next assistant does not re-answer them]
|
||||||
|
|
||||||
## Pending User Asks
|
## Pending User Asks
|
||||||
[Questions or requests from the user that have NOT yet been answered or fulfilled. If none, write "None."]
|
[Questions or requests from the user that have NOT yet been answered or fulfilled. If none, write "None."]
|
||||||
@@ -849,7 +851,7 @@ Update the summary using this exact structure. PRESERVE all existing information
|
|||||||
# First compaction: summarize from scratch
|
# First compaction: summarize from scratch
|
||||||
prompt = f"""{_summarizer_preamble}
|
prompt = f"""{_summarizer_preamble}
|
||||||
|
|
||||||
Create a structured checkpoint summary for the conversation after earlier turns are compacted. The summary should preserve enough detail for continuity without re-reading the original turns.
|
Create a structured handoff summary for a different assistant that will continue this conversation after earlier turns are compacted. The next assistant should be able to understand what happened without re-reading the original turns.
|
||||||
|
|
||||||
TURNS TO SUMMARIZE:
|
TURNS TO SUMMARIZE:
|
||||||
{content_to_summarize}
|
{content_to_summarize}
|
||||||
|
|||||||
@@ -477,8 +477,8 @@ class CopilotACPClient:
|
|||||||
proc.stdin.write(json.dumps(payload) + "\n")
|
proc.stdin.write(json.dumps(payload) + "\n")
|
||||||
proc.stdin.flush()
|
proc.stdin.flush()
|
||||||
|
|
||||||
deadline = time.monotonic() + timeout_seconds
|
deadline = time.time() + timeout_seconds
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
if proc.poll() is not None:
|
if proc.poll() is not None:
|
||||||
break
|
break
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -68,10 +68,8 @@ SUPPORTED_POOL_STRATEGIES = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
# Cooldown before retrying an exhausted credential.
|
# Cooldown before retrying an exhausted credential.
|
||||||
# Transient 401 auth failures cool down briefly so single-key setups can recover.
|
# 429 (rate-limited) and 402 (billing/quota) both cool down after 1 hour.
|
||||||
# 429 (rate-limited), 402 (billing/quota), and other failures cool down after 1 hour.
|
|
||||||
# Provider-supplied reset_at timestamps override these defaults.
|
# Provider-supplied reset_at timestamps override these defaults.
|
||||||
EXHAUSTED_TTL_401_SECONDS = 5 * 60 # 5 minutes
|
|
||||||
EXHAUSTED_TTL_429_SECONDS = 60 * 60 # 1 hour
|
EXHAUSTED_TTL_429_SECONDS = 60 * 60 # 1 hour
|
||||||
EXHAUSTED_TTL_DEFAULT_SECONDS = 60 * 60 # 1 hour
|
EXHAUSTED_TTL_DEFAULT_SECONDS = 60 * 60 # 1 hour
|
||||||
|
|
||||||
@@ -192,8 +190,6 @@ def _is_manual_source(source: str) -> bool:
|
|||||||
|
|
||||||
def _exhausted_ttl(error_code: Optional[int]) -> int:
|
def _exhausted_ttl(error_code: Optional[int]) -> int:
|
||||||
"""Return cooldown seconds based on the HTTP status that caused exhaustion."""
|
"""Return cooldown seconds based on the HTTP status that caused exhaustion."""
|
||||||
if error_code == 401:
|
|
||||||
return EXHAUSTED_TTL_401_SECONDS
|
|
||||||
if error_code == 429:
|
if error_code == 429:
|
||||||
return EXHAUSTED_TTL_429_SECONDS
|
return EXHAUSTED_TTL_429_SECONDS
|
||||||
return EXHAUSTED_TTL_DEFAULT_SECONDS
|
return EXHAUSTED_TTL_DEFAULT_SECONDS
|
||||||
@@ -309,29 +305,14 @@ def _iter_custom_providers(config: Optional[dict] = None):
|
|||||||
yield _normalize_custom_pool_name(name), entry
|
yield _normalize_custom_pool_name(name), entry
|
||||||
|
|
||||||
|
|
||||||
def get_custom_provider_pool_key(base_url: str, provider_name: Optional[str] = None) -> Optional[str]:
|
def get_custom_provider_pool_key(base_url: str) -> Optional[str]:
|
||||||
"""Look up the custom_providers list in config.yaml and return 'custom:<name>' for a matching base_url.
|
"""Look up the custom_providers list in config.yaml and return 'custom:<name>' for a matching base_url.
|
||||||
|
|
||||||
When provider_name is given, prefer matching by name first (solving the case where
|
|
||||||
multiple custom providers share the same base_url but have different API keys).
|
|
||||||
Falls back to base_url matching when no name match is found.
|
|
||||||
|
|
||||||
Returns None if no match is found.
|
Returns None if no match is found.
|
||||||
"""
|
"""
|
||||||
if not base_url:
|
if not base_url:
|
||||||
return None
|
return None
|
||||||
normalized_url = base_url.strip().rstrip("/")
|
normalized_url = base_url.strip().rstrip("/")
|
||||||
|
|
||||||
# When a provider name is given, try to match by name first.
|
|
||||||
# This fixes the P1 bug where two custom providers sharing the same
|
|
||||||
# base_url always resolve to the first one's credentials.
|
|
||||||
if provider_name:
|
|
||||||
normalized_name = _normalize_custom_pool_name(provider_name)
|
|
||||||
for norm_name, entry in _iter_custom_providers():
|
|
||||||
if norm_name == normalized_name:
|
|
||||||
return f"{CUSTOM_POOL_PREFIX}{norm_name}"
|
|
||||||
|
|
||||||
# Fall back to base_url matching (original behavior)
|
|
||||||
for norm_name, entry in _iter_custom_providers():
|
for norm_name, entry in _iter_custom_providers():
|
||||||
entry_url = str(entry.get("base_url") or "").strip().rstrip("/")
|
entry_url = str(entry.get("base_url") or "").strip().rstrip("/")
|
||||||
if entry_url and entry_url == normalized_url:
|
if entry_url and entry_url == normalized_url:
|
||||||
|
|||||||
+1
-1
@@ -1607,7 +1607,7 @@ def _run_llm_review(prompt: str) -> Dict[str, Any]:
|
|||||||
# terminal. The background-thread runner also hides it; this
|
# terminal. The background-thread runner also hides it; this
|
||||||
# belt-and-suspenders path matters when a caller invokes
|
# belt-and-suspenders path matters when a caller invokes
|
||||||
# run_curator_review(synchronous=True) from the CLI.
|
# run_curator_review(synchronous=True) from the CLI.
|
||||||
with open(os.devnull, "w", encoding="utf-8") as _devnull, \
|
with open(os.devnull, "w") as _devnull, \
|
||||||
contextlib.redirect_stdout(_devnull), \
|
contextlib.redirect_stdout(_devnull), \
|
||||||
contextlib.redirect_stderr(_devnull):
|
contextlib.redirect_stderr(_devnull):
|
||||||
conv_result = review_agent.run_conversation(user_message=prompt)
|
conv_result = review_agent.run_conversation(user_message=prompt)
|
||||||
|
|||||||
+2
-4
@@ -852,15 +852,13 @@ def get_cute_tool_message(
|
|||||||
s = str(s)
|
s = str(s)
|
||||||
if _tool_preview_max_len == 0:
|
if _tool_preview_max_len == 0:
|
||||||
return s # no limit
|
return s # no limit
|
||||||
limit = _tool_preview_max_len
|
return (s[:n-3] + "...") if len(s) > n else s
|
||||||
return (s[:limit-3] + "...") if len(s) > limit else s
|
|
||||||
|
|
||||||
def _path(p, n=35):
|
def _path(p, n=35):
|
||||||
p = str(p)
|
p = str(p)
|
||||||
if _tool_preview_max_len == 0:
|
if _tool_preview_max_len == 0:
|
||||||
return p # no limit
|
return p # no limit
|
||||||
limit = _tool_preview_max_len
|
return ("..." + p[-(n-3):]) if len(p) > n else p
|
||||||
return ("..." + p[-(limit-3):]) if len(p) > limit else p
|
|
||||||
|
|
||||||
def _wrap(line: str) -> str:
|
def _wrap(line: str) -> str:
|
||||||
"""Apply skin tool prefix and failure suffix."""
|
"""Apply skin tool prefix and failure suffix."""
|
||||||
|
|||||||
+13
-78
@@ -144,51 +144,7 @@ def decide_image_input_mode(
|
|||||||
# it fires, which is cheaper than permanent quality loss.
|
# it fires, which is cheaper than permanent quality loss.
|
||||||
|
|
||||||
|
|
||||||
def _sniff_mime_from_bytes(raw: bytes) -> Optional[str]:
|
def _guess_mime(path: Path) -> str:
|
||||||
"""Detect image MIME from magic bytes. Returns None if unrecognised.
|
|
||||||
|
|
||||||
Filename-based detection (``mimetypes.guess_type``) is unreliable when
|
|
||||||
upstream platforms lie about content-type. Discord, for example, can
|
|
||||||
serve a PNG with ``content_type=image/webp`` for proxied/animated
|
|
||||||
stickers, custom emoji previews, or images uploaded via certain bots.
|
|
||||||
Anthropic strictly validates that declared media_type matches the
|
|
||||||
actual bytes and returns HTTP 400 on mismatch, so we sniff to be safe.
|
|
||||||
"""
|
|
||||||
if not raw:
|
|
||||||
return None
|
|
||||||
# PNG: 89 50 4E 47 0D 0A 1A 0A
|
|
||||||
if raw.startswith(b"\x89PNG\r\n\x1a\n"):
|
|
||||||
return "image/png"
|
|
||||||
# JPEG: FF D8 FF
|
|
||||||
if raw.startswith(b"\xff\xd8\xff"):
|
|
||||||
return "image/jpeg"
|
|
||||||
# GIF87a / GIF89a
|
|
||||||
if raw[:6] in (b"GIF87a", b"GIF89a"):
|
|
||||||
return "image/gif"
|
|
||||||
# WEBP: "RIFF" .... "WEBP"
|
|
||||||
if len(raw) >= 12 and raw[:4] == b"RIFF" and raw[8:12] == b"WEBP":
|
|
||||||
return "image/webp"
|
|
||||||
# BMP: "BM"
|
|
||||||
if raw.startswith(b"BM"):
|
|
||||||
return "image/bmp"
|
|
||||||
# HEIC/HEIF: ftypheic / ftypheix / ftypmif1 / ftypmsf1 etc.
|
|
||||||
if len(raw) >= 12 and raw[4:8] == b"ftyp" and raw[8:12] in (
|
|
||||||
b"heic", b"heix", b"hevc", b"hevx", b"mif1", b"msf1", b"heim", b"heis",
|
|
||||||
):
|
|
||||||
return "image/heic"
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _guess_mime(path: Path, raw: Optional[bytes] = None) -> str:
|
|
||||||
"""Return image MIME type for *path*.
|
|
||||||
|
|
||||||
If *raw* bytes are provided, magic-byte sniffing wins (authoritative).
|
|
||||||
Otherwise we fall back to ``mimetypes`` then suffix-based defaults.
|
|
||||||
"""
|
|
||||||
if raw is not None:
|
|
||||||
sniffed = _sniff_mime_from_bytes(raw)
|
|
||||||
if sniffed:
|
|
||||||
return sniffed
|
|
||||||
mime, _ = mimetypes.guess_type(str(path))
|
mime, _ = mimetypes.guess_type(str(path))
|
||||||
if mime and mime.startswith("image/"):
|
if mime and mime.startswith("image/"):
|
||||||
return mime
|
return mime
|
||||||
@@ -222,7 +178,7 @@ def _file_to_data_url(path: Path) -> Optional[str]:
|
|||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.warning("image_routing: failed to read %s — %s", path, exc)
|
logger.warning("image_routing: failed to read %s — %s", path, exc)
|
||||||
return None
|
return None
|
||||||
mime = _guess_mime(path, raw=raw)
|
mime = _guess_mime(path)
|
||||||
b64 = base64.b64encode(raw).decode("ascii")
|
b64 = base64.b64encode(raw).decode("ascii")
|
||||||
return f"data:{mime};base64,{b64}"
|
return f"data:{mime};base64,{b64}"
|
||||||
|
|
||||||
@@ -234,30 +190,24 @@ def build_native_content_parts(
|
|||||||
"""Build an OpenAI-style ``content`` list for a user turn.
|
"""Build an OpenAI-style ``content`` list for a user turn.
|
||||||
|
|
||||||
Shape:
|
Shape:
|
||||||
[{"type": "text", "text": "...\\n\\n[Image attached at: /local/path]"},
|
[{"type": "text", "text": "..."},
|
||||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}},
|
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}},
|
||||||
...]
|
...]
|
||||||
|
|
||||||
The local path of each successfully attached image is appended to the
|
|
||||||
text part as ``[Image attached at: <path>]``. The model still sees the
|
|
||||||
pixels via the ``image_url`` part (full native vision); the path note
|
|
||||||
just gives it a string handle so MCP/skill tools that take an image
|
|
||||||
path or URL argument can be invoked on the same image without an
|
|
||||||
extra round-trip. This parallels the text-mode hint produced by
|
|
||||||
``Runner._enrich_message_with_vision`` (``vision_analyze using image_url:
|
|
||||||
<path>``) so behaviour is consistent across both image input modes.
|
|
||||||
|
|
||||||
Images are attached at their native size. If a provider rejects the
|
Images are attached at their native size. If a provider rejects the
|
||||||
request because an image is too large (e.g. Anthropic's 5 MB per-image
|
request because an image is too large (e.g. Anthropic's 5 MB per-image
|
||||||
ceiling), the agent's retry loop transparently shrinks and retries
|
ceiling), the agent's retry loop transparently shrinks and retries
|
||||||
once — see ``run_agent._try_shrink_image_parts_in_messages``.
|
once — see ``run_agent._try_shrink_image_parts_in_messages``.
|
||||||
|
|
||||||
Returns (content_parts, skipped_paths). Skipped paths are files that
|
Returns (content_parts, skipped_paths). Skipped paths are files that
|
||||||
couldn't be read from disk and are NOT advertised in the path hints.
|
couldn't be read from disk.
|
||||||
"""
|
"""
|
||||||
|
parts: List[Dict[str, Any]] = []
|
||||||
skipped: List[str] = []
|
skipped: List[str] = []
|
||||||
image_parts: List[Dict[str, Any]] = []
|
|
||||||
attached_paths: List[str] = []
|
text = (user_text or "").strip()
|
||||||
|
if text:
|
||||||
|
parts.append({"type": "text", "text": text})
|
||||||
|
|
||||||
for raw_path in image_paths:
|
for raw_path in image_paths:
|
||||||
p = Path(raw_path)
|
p = Path(raw_path)
|
||||||
@@ -268,30 +218,15 @@ def build_native_content_parts(
|
|||||||
if not data_url:
|
if not data_url:
|
||||||
skipped.append(str(raw_path))
|
skipped.append(str(raw_path))
|
||||||
continue
|
continue
|
||||||
image_parts.append({
|
parts.append({
|
||||||
"type": "image_url",
|
"type": "image_url",
|
||||||
"image_url": {"url": data_url},
|
"image_url": {"url": data_url},
|
||||||
})
|
})
|
||||||
attached_paths.append(str(raw_path))
|
|
||||||
|
|
||||||
text = (user_text or "").strip()
|
# If the text was empty, add a neutral prompt so the turn isn't just images.
|
||||||
|
if not text and any(p.get("type") == "image_url" for p in parts):
|
||||||
|
parts.insert(0, {"type": "text", "text": "What do you see in this image?"})
|
||||||
|
|
||||||
# If at least one image attached, build a single text part that combines
|
|
||||||
# the user's caption (or a neutral default) with one path hint per image.
|
|
||||||
if attached_paths:
|
|
||||||
base_text = text or "What do you see in this image?"
|
|
||||||
path_hints = "\n".join(
|
|
||||||
f"[Image attached at: {p}]" for p in attached_paths
|
|
||||||
)
|
|
||||||
combined_text = f"{base_text}\n\n{path_hints}"
|
|
||||||
parts: List[Dict[str, Any]] = [{"type": "text", "text": combined_text}]
|
|
||||||
parts.extend(image_parts)
|
|
||||||
return parts, skipped
|
|
||||||
|
|
||||||
# No images successfully attached — fall back to plain text-only behaviour.
|
|
||||||
parts = []
|
|
||||||
if text:
|
|
||||||
parts.append({"type": "text", "text": text})
|
|
||||||
return parts, skipped
|
return parts, skipped
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -754,7 +754,7 @@ def _load_context_cache() -> Dict[str, int]:
|
|||||||
if not path.exists():
|
if not path.exists():
|
||||||
return {}
|
return {}
|
||||||
try:
|
try:
|
||||||
with open(path, encoding="utf-8") as f:
|
with open(path) as f:
|
||||||
data = yaml.safe_load(f) or {}
|
data = yaml.safe_load(f) or {}
|
||||||
return data.get("context_lengths", {})
|
return data.get("context_lengths", {})
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -776,7 +776,7 @@ def save_context_length(model: str, base_url: str, length: int) -> None:
|
|||||||
path = _get_context_cache_path()
|
path = _get_context_cache_path()
|
||||||
try:
|
try:
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
with open(path, "w", encoding="utf-8") as f:
|
with open(path, "w") as f:
|
||||||
yaml.dump({"context_lengths": cache}, f, default_flow_style=False)
|
yaml.dump({"context_lengths": cache}, f, default_flow_style=False)
|
||||||
logger.info("Cached context length %s -> %s tokens", key, f"{length:,}")
|
logger.info("Cached context length %s -> %s tokens", key, f"{length:,}")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -800,7 +800,7 @@ def _invalidate_cached_context_length(model: str, base_url: str) -> None:
|
|||||||
path = _get_context_cache_path()
|
path = _get_context_cache_path()
|
||||||
try:
|
try:
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
with open(path, "w", encoding="utf-8") as f:
|
with open(path, "w") as f:
|
||||||
yaml.dump({"context_lengths": cache}, f, default_flow_style=False)
|
yaml.dump({"context_lengths": cache}, f, default_flow_style=False)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Failed to invalidate context length cache entry %s: %s", key, e)
|
logger.debug("Failed to invalidate context length cache entry %s: %s", key, e)
|
||||||
|
|||||||
+5
-9
@@ -381,18 +381,14 @@ def get_model_capabilities(provider: str, model: str) -> Optional[ModelCapabilit
|
|||||||
|
|
||||||
# Extract capability flags (default to False if missing)
|
# Extract capability flags (default to False if missing)
|
||||||
supports_tools = bool(entry.get("tool_call", False))
|
supports_tools = bool(entry.get("tool_call", False))
|
||||||
# Vision: prefer explicit `modalities.input` when models.dev provides it.
|
# Vision: check both the `attachment` flag and `modalities.input` for "image".
|
||||||
# The older `attachment` flag can be stale or too broad for image routing;
|
# Some models (e.g. gemma-4) list image in input modalities but not attachment.
|
||||||
# fall back to it only when the input modalities are absent/invalid.
|
|
||||||
input_mods = entry.get("modalities", {})
|
input_mods = entry.get("modalities", {})
|
||||||
if isinstance(input_mods, dict):
|
if isinstance(input_mods, dict):
|
||||||
input_mods = input_mods.get("input")
|
input_mods = input_mods.get("input", [])
|
||||||
else:
|
else:
|
||||||
input_mods = None
|
input_mods = []
|
||||||
if isinstance(input_mods, list):
|
supports_vision = bool(entry.get("attachment", False)) or "image" in input_mods
|
||||||
supports_vision = "image" in input_mods
|
|
||||||
else:
|
|
||||||
supports_vision = bool(entry.get("attachment", False))
|
|
||||||
supports_reasoning = bool(entry.get("reasoning", False))
|
supports_reasoning = bool(entry.get("reasoning", False))
|
||||||
|
|
||||||
# Extract limits
|
# Extract limits
|
||||||
|
|||||||
@@ -144,7 +144,7 @@ def nous_rate_limit_remaining() -> Optional[float]:
|
|||||||
"""
|
"""
|
||||||
path = _state_path()
|
path = _state_path()
|
||||||
try:
|
try:
|
||||||
with open(path, encoding="utf-8") as f:
|
with open(path) as f:
|
||||||
state = json.load(f)
|
state = json.load(f)
|
||||||
reset_at = state.get("reset_at", 0)
|
reset_at = state.get("reset_at", 0)
|
||||||
remaining = reset_at - time.time()
|
remaining = reset_at - time.time()
|
||||||
|
|||||||
+6
-9
@@ -56,15 +56,12 @@ _SENSITIVE_BODY_KEYS = frozenset({
|
|||||||
})
|
})
|
||||||
|
|
||||||
# Snapshot at import time so runtime env mutations (e.g. LLM-generated
|
# Snapshot at import time so runtime env mutations (e.g. LLM-generated
|
||||||
# `export HERMES_REDACT_SECRETS=false`) cannot disable redaction
|
# `export HERMES_REDACT_SECRETS=true`) cannot enable/disable redaction
|
||||||
# mid-session. ON by default — secure default per issue #17691. Users who
|
# mid-session. OFF by default — user must opt in via
|
||||||
# need raw credential values in tool output (e.g. working on the redactor
|
# `security.redact_secrets: true` in config.yaml (bridged to this env var
|
||||||
# itself) can opt out via `security.redact_secrets: false` in config.yaml
|
# in hermes_cli/main.py and gateway/run.py) or `HERMES_REDACT_SECRETS=true`
|
||||||
# (bridged to this env var in hermes_cli/main.py, gateway/run.py, and
|
# in ~/.hermes/.env.
|
||||||
# cli.py) or `HERMES_REDACT_SECRETS=false` in ~/.hermes/.env. An opt-out
|
_REDACT_ENABLED = os.getenv("HERMES_REDACT_SECRETS", "").lower() in ("1", "true", "yes", "on")
|
||||||
# warning is logged at gateway and CLI startup so operators see the
|
|
||||||
# downgrade — see `_log_redaction_status()` in gateway/run.py and cli.py.
|
|
||||||
_REDACT_ENABLED = os.getenv("HERMES_REDACT_SECRETS", "true").lower() in ("1", "true", "yes", "on")
|
|
||||||
|
|
||||||
# Known API key prefixes -- match the prefix + contiguous token chars
|
# Known API key prefixes -- match the prefix + contiguous token chars
|
||||||
_PREFIX_PATTERNS = [
|
_PREFIX_PATTERNS = [
|
||||||
|
|||||||
@@ -617,7 +617,7 @@ def _locked_update_approvals() -> Iterator[Dict[str, Any]]:
|
|||||||
save_allowlist(data)
|
save_allowlist(data)
|
||||||
return
|
return
|
||||||
|
|
||||||
with open(lock_path, "a+", encoding="utf-8") as lock_fh:
|
with open(lock_path, "a+") as lock_fh:
|
||||||
fcntl.flock(lock_fh.fileno(), fcntl.LOCK_EX)
|
fcntl.flock(lock_fh.fileno(), fcntl.LOCK_EX)
|
||||||
try:
|
try:
|
||||||
data = load_allowlist()
|
data = load_allowlist()
|
||||||
|
|||||||
+14
-159
@@ -1,6 +1,5 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import re
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
@@ -83,121 +82,6 @@ _UTC_NOW = lambda: datetime.now(timezone.utc)
|
|||||||
# Official docs snapshot entries. Models whose published pricing and cache
|
# Official docs snapshot entries. Models whose published pricing and cache
|
||||||
# semantics are stable enough to encode exactly.
|
# semantics are stable enough to encode exactly.
|
||||||
_OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
_OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||||
# ── Anthropic Claude 4.7 ─────────────────────────────────────────────
|
|
||||||
# Opus 4.5/4.6/4.7 share $5/$25 pricing (new tokenizer, up to 35% more
|
|
||||||
# tokens for the same text).
|
|
||||||
# Source: https://platform.claude.com/docs/en/about-claude/pricing
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-opus-4-7",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("5.00"),
|
|
||||||
output_cost_per_million=Decimal("25.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.50"),
|
|
||||||
cache_write_cost_per_million=Decimal("6.25"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-opus-4-7-20250507",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("5.00"),
|
|
||||||
output_cost_per_million=Decimal("25.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.50"),
|
|
||||||
cache_write_cost_per_million=Decimal("6.25"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
# ── Anthropic Claude 4.6 ─────────────────────────────────────────────
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-opus-4-6",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("5.00"),
|
|
||||||
output_cost_per_million=Decimal("25.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.50"),
|
|
||||||
cache_write_cost_per_million=Decimal("6.25"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-opus-4-6-20250414",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("5.00"),
|
|
||||||
output_cost_per_million=Decimal("25.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.50"),
|
|
||||||
cache_write_cost_per_million=Decimal("6.25"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-sonnet-4-6",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("3.00"),
|
|
||||||
output_cost_per_million=Decimal("15.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.30"),
|
|
||||||
cache_write_cost_per_million=Decimal("3.75"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-sonnet-4-6-20250414",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("3.00"),
|
|
||||||
output_cost_per_million=Decimal("15.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.30"),
|
|
||||||
cache_write_cost_per_million=Decimal("3.75"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
# ── Anthropic Claude 4.5 ─────────────────────────────────────────────
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-opus-4-5",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("5.00"),
|
|
||||||
output_cost_per_million=Decimal("25.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.50"),
|
|
||||||
cache_write_cost_per_million=Decimal("6.25"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-sonnet-4-5",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("3.00"),
|
|
||||||
output_cost_per_million=Decimal("15.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.30"),
|
|
||||||
cache_write_cost_per_million=Decimal("3.75"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"anthropic",
|
|
||||||
"claude-haiku-4-5",
|
|
||||||
): PricingEntry(
|
|
||||||
input_cost_per_million=Decimal("1.00"),
|
|
||||||
output_cost_per_million=Decimal("5.00"),
|
|
||||||
cache_read_cost_per_million=Decimal("0.10"),
|
|
||||||
cache_write_cost_per_million=Decimal("1.25"),
|
|
||||||
source="official_docs_snapshot",
|
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
|
||||||
),
|
|
||||||
# ── Anthropic Claude 4 / 4.1 ─────────────────────────────────────────
|
|
||||||
(
|
(
|
||||||
"anthropic",
|
"anthropic",
|
||||||
"claude-opus-4-20250514",
|
"claude-opus-4-20250514",
|
||||||
@@ -207,8 +91,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
|||||||
cache_read_cost_per_million=Decimal("1.50"),
|
cache_read_cost_per_million=Decimal("1.50"),
|
||||||
cache_write_cost_per_million=Decimal("18.75"),
|
cache_write_cost_per_million=Decimal("18.75"),
|
||||||
source="official_docs_snapshot",
|
source="official_docs_snapshot",
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
pricing_version="anthropic-prompt-caching-2026-03-16",
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
"anthropic",
|
"anthropic",
|
||||||
@@ -219,8 +103,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
|||||||
cache_read_cost_per_million=Decimal("0.30"),
|
cache_read_cost_per_million=Decimal("0.30"),
|
||||||
cache_write_cost_per_million=Decimal("3.75"),
|
cache_write_cost_per_million=Decimal("3.75"),
|
||||||
source="official_docs_snapshot",
|
source="official_docs_snapshot",
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
pricing_version="anthropic-prompt-caching-2026-03-16",
|
||||||
),
|
),
|
||||||
# OpenAI
|
# OpenAI
|
||||||
(
|
(
|
||||||
@@ -300,7 +184,7 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
|||||||
source_url="https://openai.com/api/pricing/",
|
source_url="https://openai.com/api/pricing/",
|
||||||
pricing_version="openai-pricing-2026-03-16",
|
pricing_version="openai-pricing-2026-03-16",
|
||||||
),
|
),
|
||||||
# ── Anthropic older models (pre-4.5 generation) ────────────────────────
|
# Anthropic older models (pre-4.6 generation)
|
||||||
(
|
(
|
||||||
"anthropic",
|
"anthropic",
|
||||||
"claude-3-5-sonnet-20241022",
|
"claude-3-5-sonnet-20241022",
|
||||||
@@ -310,8 +194,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
|||||||
cache_read_cost_per_million=Decimal("0.30"),
|
cache_read_cost_per_million=Decimal("0.30"),
|
||||||
cache_write_cost_per_million=Decimal("3.75"),
|
cache_write_cost_per_million=Decimal("3.75"),
|
||||||
source="official_docs_snapshot",
|
source="official_docs_snapshot",
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
pricing_version="anthropic-pricing-2026-03-16",
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
"anthropic",
|
"anthropic",
|
||||||
@@ -322,8 +206,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
|||||||
cache_read_cost_per_million=Decimal("0.08"),
|
cache_read_cost_per_million=Decimal("0.08"),
|
||||||
cache_write_cost_per_million=Decimal("1.00"),
|
cache_write_cost_per_million=Decimal("1.00"),
|
||||||
source="official_docs_snapshot",
|
source="official_docs_snapshot",
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
pricing_version="anthropic-pricing-2026-03-16",
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
"anthropic",
|
"anthropic",
|
||||||
@@ -334,8 +218,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
|||||||
cache_read_cost_per_million=Decimal("1.50"),
|
cache_read_cost_per_million=Decimal("1.50"),
|
||||||
cache_write_cost_per_million=Decimal("18.75"),
|
cache_write_cost_per_million=Decimal("18.75"),
|
||||||
source="official_docs_snapshot",
|
source="official_docs_snapshot",
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
pricing_version="anthropic-pricing-2026-03-16",
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
"anthropic",
|
"anthropic",
|
||||||
@@ -346,8 +230,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
|||||||
cache_read_cost_per_million=Decimal("0.03"),
|
cache_read_cost_per_million=Decimal("0.03"),
|
||||||
cache_write_cost_per_million=Decimal("0.30"),
|
cache_write_cost_per_million=Decimal("0.30"),
|
||||||
source="official_docs_snapshot",
|
source="official_docs_snapshot",
|
||||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||||
pricing_version="anthropic-pricing-2026-05",
|
pricing_version="anthropic-pricing-2026-03-16",
|
||||||
),
|
),
|
||||||
# DeepSeek
|
# DeepSeek
|
||||||
(
|
(
|
||||||
@@ -542,37 +426,8 @@ def resolve_billing_route(
|
|||||||
return BillingRoute(provider=provider_name or "unknown", model=model.split("/")[-1] if model else "", base_url=base_url or "", billing_mode="unknown")
|
return BillingRoute(provider=provider_name or "unknown", model=model.split("/")[-1] if model else "", base_url=base_url or "", billing_mode="unknown")
|
||||||
|
|
||||||
|
|
||||||
def _normalize_anthropic_model_name(model: str) -> str:
|
|
||||||
"""Normalize Anthropic model name variants to canonical form.
|
|
||||||
|
|
||||||
Handles:
|
|
||||||
- Dot notation: claude-opus-4.7 → claude-opus-4-7
|
|
||||||
- Short aliases: claude-opus-4.7 → claude-opus-4-7
|
|
||||||
- Strips anthropic/ prefix if present
|
|
||||||
"""
|
|
||||||
name = model.lower().strip()
|
|
||||||
if name.startswith("anthropic/"):
|
|
||||||
name = name[len("anthropic/"):]
|
|
||||||
# Normalize dots to dashes in version numbers (e.g. 4.7 → 4-7, 4.6 → 4-6)
|
|
||||||
# But preserve the rest of the name structure
|
|
||||||
name = re.sub(r"(\d+)\.(\d+)", r"\1-\2", name)
|
|
||||||
return name
|
|
||||||
|
|
||||||
|
|
||||||
def _lookup_official_docs_pricing(route: BillingRoute) -> Optional[PricingEntry]:
|
def _lookup_official_docs_pricing(route: BillingRoute) -> Optional[PricingEntry]:
|
||||||
model = route.model.lower()
|
return _OFFICIAL_DOCS_PRICING.get((route.provider, route.model.lower()))
|
||||||
# Direct lookup first
|
|
||||||
entry = _OFFICIAL_DOCS_PRICING.get((route.provider, model))
|
|
||||||
if entry:
|
|
||||||
return entry
|
|
||||||
# Try normalized name for Anthropic (handles dot-notation like opus-4.7)
|
|
||||||
if route.provider == "anthropic":
|
|
||||||
normalized = _normalize_anthropic_model_name(model)
|
|
||||||
if normalized != model:
|
|
||||||
entry = _OFFICIAL_DOCS_PRICING.get((route.provider, normalized))
|
|
||||||
if entry:
|
|
||||||
return entry
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _openrouter_pricing_entry(route: BillingRoute) -> Optional[PricingEntry]:
|
def _openrouter_pricing_entry(route: BillingRoute) -> Optional[PricingEntry]:
|
||||||
|
|||||||
@@ -20,10 +20,6 @@ Usage:
|
|||||||
python batch_runner.py --dataset_file=data.jsonl --batch_size=10 --run_name=my_run --distribution=image_gen
|
python batch_runner.py --dataset_file=data.jsonl --batch_size=10 --run_name=my_run --distribution=image_gen
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
|
|
||||||
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
|
|
||||||
import hermes_bootstrap # noqa: F401
|
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
|
|||||||
+1
-19
@@ -601,7 +601,7 @@ agent:
|
|||||||
# - A preset like "hermes-cli" or "hermes-telegram" (curated tool set)
|
# - A preset like "hermes-cli" or "hermes-telegram" (curated tool set)
|
||||||
# - A list of individual toolsets to compose your own (see list below)
|
# - A list of individual toolsets to compose your own (see list below)
|
||||||
#
|
#
|
||||||
# Supported platform keys: cli, telegram, discord, whatsapp, slack, qqbot, teams, google_chat
|
# Supported platform keys: cli, telegram, discord, whatsapp, slack, qqbot, teams
|
||||||
#
|
#
|
||||||
# Examples:
|
# Examples:
|
||||||
#
|
#
|
||||||
@@ -632,7 +632,6 @@ agent:
|
|||||||
# homeassistant: hermes-homeassistant (same as telegram)
|
# homeassistant: hermes-homeassistant (same as telegram)
|
||||||
# qqbot: hermes-qqbot (same as telegram)
|
# qqbot: hermes-qqbot (same as telegram)
|
||||||
# teams: hermes-teams (same as telegram)
|
# teams: hermes-teams (same as telegram)
|
||||||
# google_chat: hermes-google_chat (same as telegram)
|
|
||||||
#
|
#
|
||||||
platform_toolsets:
|
platform_toolsets:
|
||||||
cli: [hermes-cli]
|
cli: [hermes-cli]
|
||||||
@@ -645,7 +644,6 @@ platform_toolsets:
|
|||||||
qqbot: [hermes-qqbot]
|
qqbot: [hermes-qqbot]
|
||||||
yuanbao: [hermes-yuanbao]
|
yuanbao: [hermes-yuanbao]
|
||||||
teams: [hermes-teams]
|
teams: [hermes-teams]
|
||||||
google_chat: [hermes-google_chat]
|
|
||||||
|
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
# Gateway Platform Settings
|
# Gateway Platform Settings
|
||||||
@@ -877,22 +875,6 @@ display:
|
|||||||
# Toggle at runtime with /verbose in the CLI
|
# Toggle at runtime with /verbose in the CLI
|
||||||
tool_progress: all
|
tool_progress: all
|
||||||
|
|
||||||
# Auto-cleanup of temporary progress bubbles after the final response lands.
|
|
||||||
# On platforms that support message deletion (currently Telegram), this
|
|
||||||
# removes the tool-progress bubble, "⏳ Still working..." notices, and
|
|
||||||
# context-pressure status messages once the final reply has been delivered —
|
|
||||||
# keeping long-running turns visible live, then tidy afterward. Failed runs
|
|
||||||
# leave the bubbles in place as breadcrumbs. Off by default.
|
|
||||||
# Per-platform override: display.platforms.telegram.cleanup_progress
|
|
||||||
# true: Delete tracked progress/status bubbles on successful turn
|
|
||||||
# false: Leave everything in place (default)
|
|
||||||
# Example:
|
|
||||||
# display:
|
|
||||||
# platforms:
|
|
||||||
# telegram:
|
|
||||||
# cleanup_progress: true
|
|
||||||
cleanup_progress: false
|
|
||||||
|
|
||||||
# Gateway-only natural mid-turn assistant updates.
|
# Gateway-only natural mid-turn assistant updates.
|
||||||
# When true, completed assistant status messages are sent as separate chat
|
# When true, completed assistant status messages are sent as separate chat
|
||||||
# messages. This is independent of tool_progress and gateway streaming.
|
# messages. This is independent of tool_progress and gateway streaming.
|
||||||
|
|||||||
@@ -9,13 +9,10 @@ Usage:
|
|||||||
python cli.py # Start interactive mode with all tools
|
python cli.py # Start interactive mode with all tools
|
||||||
python cli.py --toolsets web,terminal # Start with specific toolsets
|
python cli.py --toolsets web,terminal # Start with specific toolsets
|
||||||
python cli.py --skills hermes-agent-dev,github-auth
|
python cli.py --skills hermes-agent-dev,github-auth
|
||||||
|
python cli.py -q "your question" # Single query mode
|
||||||
python cli.py --list-tools # List available tools and exit
|
python cli.py --list-tools # List available tools and exit
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
|
|
||||||
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
|
|
||||||
import hermes_bootstrap # noqa: F401
|
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
@@ -731,43 +728,8 @@ def _run_cleanup():
|
|||||||
_active_worktree: Optional[Dict[str, str]] = None
|
_active_worktree: Optional[Dict[str, str]] = None
|
||||||
|
|
||||||
|
|
||||||
def _normalize_git_bash_path(p: Optional[str]) -> Optional[str]:
|
|
||||||
"""Translate a Git Bash-style path (``/c/Users/...``) to the native
|
|
||||||
Windows form (``C:\\Users\\...``) that Python's ``subprocess.Popen``
|
|
||||||
and ``pathlib.Path`` accept.
|
|
||||||
|
|
||||||
No-op on non-Windows and for paths that already look native. Git on
|
|
||||||
native Windows normally emits forward-slash Windows paths
|
|
||||||
(``C:/Users/...``) which both bash and Python handle, but certain
|
|
||||||
configurations (Git Bash shells, MSYS2, WSL-mounted repos) surface
|
|
||||||
``/c/...`` or ``/cygdrive/c/...`` variants.
|
|
||||||
"""
|
|
||||||
if not p:
|
|
||||||
return p
|
|
||||||
if sys.platform != "win32":
|
|
||||||
return p
|
|
||||||
import re as _re
|
|
||||||
# /c/Users/... or /C/Users/...
|
|
||||||
m = _re.match(r"^/([a-zA-Z])/(.*)$", p)
|
|
||||||
if m:
|
|
||||||
drive, rest = m.group(1), m.group(2)
|
|
||||||
return f"{drive.upper()}:\\{rest.replace('/', chr(92))}"
|
|
||||||
# /cygdrive/c/... or /mnt/c/...
|
|
||||||
m = _re.match(r"^/(?:cygdrive|mnt)/([a-zA-Z])/(.*)$", p)
|
|
||||||
if m:
|
|
||||||
drive, rest = m.group(1), m.group(2)
|
|
||||||
return f"{drive.upper()}:\\{rest.replace('/', chr(92))}"
|
|
||||||
return p
|
|
||||||
|
|
||||||
|
|
||||||
def _git_repo_root() -> Optional[str]:
|
def _git_repo_root() -> Optional[str]:
|
||||||
"""Return the git repo root for CWD, or None if not in a repo.
|
"""Return the git repo root for CWD, or None if not in a repo."""
|
||||||
|
|
||||||
Runs through :func:`_normalize_git_bash_path` so callers can pass
|
|
||||||
the result directly to ``Path``/``subprocess.Popen(cwd=...)`` on
|
|
||||||
Windows without hitting ``C:\\c\\Users\\...`` style resolution
|
|
||||||
mistakes.
|
|
||||||
"""
|
|
||||||
import subprocess
|
import subprocess
|
||||||
try:
|
try:
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
@@ -775,7 +737,7 @@ def _git_repo_root() -> Optional[str]:
|
|||||||
capture_output=True, text=True, timeout=5,
|
capture_output=True, text=True, timeout=5,
|
||||||
)
|
)
|
||||||
if result.returncode == 0:
|
if result.returncode == 0:
|
||||||
return _normalize_git_bash_path(result.stdout.strip())
|
return result.stdout.strip()
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
return None
|
return None
|
||||||
@@ -819,7 +781,7 @@ def _setup_worktree(repo_root: str = None) -> Optional[Dict[str, str]]:
|
|||||||
try:
|
try:
|
||||||
existing = gitignore.read_text() if gitignore.exists() else ""
|
existing = gitignore.read_text() if gitignore.exists() else ""
|
||||||
if _ignore_entry not in existing.splitlines():
|
if _ignore_entry not in existing.splitlines():
|
||||||
with open(gitignore, "a", encoding="utf-8") as f:
|
with open(gitignore, "a") as f:
|
||||||
if existing and not existing.endswith("\n"):
|
if existing and not existing.endswith("\n"):
|
||||||
f.write("\n")
|
f.write("\n")
|
||||||
f.write(f"{_ignore_entry}\n")
|
f.write(f"{_ignore_entry}\n")
|
||||||
@@ -870,39 +832,10 @@ def _setup_worktree(repo_root: str = None) -> Optional[Dict[str, str]]:
|
|||||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||||
shutil.copy2(str(src), str(dst))
|
shutil.copy2(str(src), str(dst))
|
||||||
elif src.is_dir():
|
elif src.is_dir():
|
||||||
# Symlink directories (faster, saves disk). On Windows,
|
# Symlink directories (faster, saves disk)
|
||||||
# symlink creation requires Developer Mode or elevation,
|
|
||||||
# and fails with OSError otherwise — fall back to a
|
|
||||||
# recursive copy so the worktree is still usable. The
|
|
||||||
# copy is slower and uses disk, but it doesn't require
|
|
||||||
# admin and matches the Linux/macOS symlink outcome
|
|
||||||
# functionally.
|
|
||||||
if not dst.exists():
|
if not dst.exists():
|
||||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||||
try:
|
os.symlink(str(src_resolved), str(dst))
|
||||||
os.symlink(str(src_resolved), str(dst))
|
|
||||||
except (OSError, NotImplementedError) as _sym_err:
|
|
||||||
if sys.platform == "win32":
|
|
||||||
logger.info(
|
|
||||||
".worktreeinclude: symlink failed (%s) — "
|
|
||||||
"falling back to copytree on Windows.",
|
|
||||||
_sym_err,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
shutil.copytree(
|
|
||||||
str(src_resolved),
|
|
||||||
str(dst),
|
|
||||||
symlinks=True,
|
|
||||||
dirs_exist_ok=False,
|
|
||||||
)
|
|
||||||
except Exception as _copy_err:
|
|
||||||
logger.warning(
|
|
||||||
".worktreeinclude: copy fallback "
|
|
||||||
"also failed for %s -> %s: %s",
|
|
||||||
src, dst, _copy_err,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
raise
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Error copying .worktreeinclude entries: %s", e)
|
logger.debug("Error copying .worktreeinclude entries: %s", e)
|
||||||
|
|
||||||
@@ -1475,13 +1408,7 @@ def _cprint(text: str):
|
|||||||
|
|
||||||
import asyncio as _asyncio
|
import asyncio as _asyncio
|
||||||
try:
|
try:
|
||||||
# Use get_running_loop() instead of get_event_loop() to avoid the
|
current_loop = _asyncio.get_event_loop_policy().get_event_loop()
|
||||||
# DeprecationWarning / RuntimeWarning emitted by Python 3.10+ when
|
|
||||||
# get_event_loop() is called from a thread that has no current event
|
|
||||||
# loop set (e.g. the process_loop background thread). Fixes #19285.
|
|
||||||
current_loop = _asyncio.get_running_loop()
|
|
||||||
except RuntimeError:
|
|
||||||
current_loop = None
|
|
||||||
except Exception:
|
except Exception:
|
||||||
current_loop = None
|
current_loop = None
|
||||||
# Same thread as the app's loop → safe to print directly.
|
# Same thread as the app's loop → safe to print directly.
|
||||||
@@ -2147,7 +2074,7 @@ def save_config_value(key_path: str, value: any) -> bool:
|
|||||||
|
|
||||||
# Load existing config
|
# Load existing config
|
||||||
if config_path.exists():
|
if config_path.exists():
|
||||||
with open(config_path, 'r', encoding="utf-8") as f:
|
with open(config_path, 'r') as f:
|
||||||
config = yaml.safe_load(f) or {}
|
config = yaml.safe_load(f) or {}
|
||||||
else:
|
else:
|
||||||
config = {}
|
config = {}
|
||||||
@@ -2644,15 +2571,6 @@ class HermesCLI:
|
|||||||
return "class:status-bar-warn"
|
return "class:status-bar-warn"
|
||||||
return "class:status-bar-good"
|
return "class:status-bar-good"
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _compression_count_style(count: int) -> str:
|
|
||||||
"""Return a style class reflecting context compression pressure."""
|
|
||||||
if count >= 10:
|
|
||||||
return "class:status-bar-bad"
|
|
||||||
if count >= 5:
|
|
||||||
return "class:status-bar-warn"
|
|
||||||
return "class:status-bar-dim"
|
|
||||||
|
|
||||||
def _build_context_bar(self, percent_used: Optional[int], width: int = 10) -> str:
|
def _build_context_bar(self, percent_used: Optional[int], width: int = 10) -> str:
|
||||||
safe_percent = max(0, min(100, percent_used or 0))
|
safe_percent = max(0, min(100, percent_used or 0))
|
||||||
filled = round((safe_percent / 100) * width)
|
filled = round((safe_percent / 100) * width)
|
||||||
@@ -2936,9 +2854,6 @@ class HermesCLI:
|
|||||||
return self._trim_status_bar_text(text, width)
|
return self._trim_status_bar_text(text, width)
|
||||||
if width < 76:
|
if width < 76:
|
||||||
parts = [f"⚕ {snapshot['model_short']}", percent_label]
|
parts = [f"⚕ {snapshot['model_short']}", percent_label]
|
||||||
compressions = snapshot.get("compressions", 0)
|
|
||||||
if compressions:
|
|
||||||
parts.append(f"🗜️ {compressions}")
|
|
||||||
parts.append(duration_label)
|
parts.append(duration_label)
|
||||||
return self._trim_status_bar_text(" · ".join(parts), width)
|
return self._trim_status_bar_text(" · ".join(parts), width)
|
||||||
|
|
||||||
@@ -2949,10 +2864,7 @@ class HermesCLI:
|
|||||||
else:
|
else:
|
||||||
context_label = "ctx --"
|
context_label = "ctx --"
|
||||||
|
|
||||||
compressions = snapshot.get("compressions", 0)
|
|
||||||
parts = [f"⚕ {snapshot['model_short']}", context_label, percent_label]
|
parts = [f"⚕ {snapshot['model_short']}", context_label, percent_label]
|
||||||
if compressions:
|
|
||||||
parts.append(f"🗜️ {compressions}")
|
|
||||||
parts.append(duration_label)
|
parts.append(duration_label)
|
||||||
prompt_elapsed = snapshot.get("prompt_elapsed")
|
prompt_elapsed = snapshot.get("prompt_elapsed")
|
||||||
if prompt_elapsed:
|
if prompt_elapsed:
|
||||||
@@ -2986,21 +2898,15 @@ class HermesCLI:
|
|||||||
percent = snapshot["context_percent"]
|
percent = snapshot["context_percent"]
|
||||||
percent_label = f"{percent}%" if percent is not None else "--"
|
percent_label = f"{percent}%" if percent is not None else "--"
|
||||||
if width < 76:
|
if width < 76:
|
||||||
compressions = snapshot.get("compressions", 0)
|
|
||||||
frags = [
|
frags = [
|
||||||
("class:status-bar", " ⚕ "),
|
("class:status-bar", " ⚕ "),
|
||||||
("class:status-bar-strong", snapshot["model_short"]),
|
("class:status-bar-strong", snapshot["model_short"]),
|
||||||
("class:status-bar-dim", " · "),
|
("class:status-bar-dim", " · "),
|
||||||
(self._status_bar_context_style(percent), percent_label),
|
(self._status_bar_context_style(percent), percent_label),
|
||||||
]
|
|
||||||
if compressions:
|
|
||||||
frags.append(("class:status-bar-dim", " · "))
|
|
||||||
frags.append((self._compression_count_style(compressions), f"🗜️ {compressions}"))
|
|
||||||
frags.extend([
|
|
||||||
("class:status-bar-dim", " · "),
|
("class:status-bar-dim", " · "),
|
||||||
("class:status-bar-dim", duration_label),
|
("class:status-bar-dim", duration_label),
|
||||||
("class:status-bar", " "),
|
("class:status-bar", " "),
|
||||||
])
|
]
|
||||||
else:
|
else:
|
||||||
if snapshot["context_length"]:
|
if snapshot["context_length"]:
|
||||||
ctx_total = _format_context_length(snapshot["context_length"])
|
ctx_total = _format_context_length(snapshot["context_length"])
|
||||||
@@ -3010,7 +2916,6 @@ class HermesCLI:
|
|||||||
context_label = "ctx --"
|
context_label = "ctx --"
|
||||||
|
|
||||||
bar_style = self._status_bar_context_style(percent)
|
bar_style = self._status_bar_context_style(percent)
|
||||||
compressions = snapshot.get("compressions", 0)
|
|
||||||
frags = [
|
frags = [
|
||||||
("class:status-bar", " ⚕ "),
|
("class:status-bar", " ⚕ "),
|
||||||
("class:status-bar-strong", snapshot["model_short"]),
|
("class:status-bar-strong", snapshot["model_short"]),
|
||||||
@@ -3020,14 +2925,9 @@ class HermesCLI:
|
|||||||
(bar_style, self._build_context_bar(percent)),
|
(bar_style, self._build_context_bar(percent)),
|
||||||
("class:status-bar-dim", " "),
|
("class:status-bar-dim", " "),
|
||||||
(bar_style, percent_label),
|
(bar_style, percent_label),
|
||||||
]
|
|
||||||
if compressions:
|
|
||||||
frags.append(("class:status-bar-dim", " │ "))
|
|
||||||
frags.append((self._compression_count_style(compressions), f"🗜️ {compressions}"))
|
|
||||||
frags.extend([
|
|
||||||
("class:status-bar-dim", " │ "),
|
("class:status-bar-dim", " │ "),
|
||||||
("class:status-bar-dim", duration_label),
|
("class:status-bar-dim", duration_label),
|
||||||
])
|
]
|
||||||
# Position 7: per-prompt elapsed timer (live or frozen)
|
# Position 7: per-prompt elapsed timer (live or frozen)
|
||||||
prompt_elapsed = snapshot.get("prompt_elapsed")
|
prompt_elapsed = snapshot.get("prompt_elapsed")
|
||||||
if prompt_elapsed:
|
if prompt_elapsed:
|
||||||
@@ -8058,7 +7958,6 @@ class HermesCLI:
|
|||||||
output_tokens = getattr(agent, "session_output_tokens", 0) or 0
|
output_tokens = getattr(agent, "session_output_tokens", 0) or 0
|
||||||
cache_read_tokens = getattr(agent, "session_cache_read_tokens", 0) or 0
|
cache_read_tokens = getattr(agent, "session_cache_read_tokens", 0) or 0
|
||||||
cache_write_tokens = getattr(agent, "session_cache_write_tokens", 0) or 0
|
cache_write_tokens = getattr(agent, "session_cache_write_tokens", 0) or 0
|
||||||
reasoning_tokens = getattr(agent, "session_reasoning_tokens", 0) or 0
|
|
||||||
prompt = agent.session_prompt_tokens
|
prompt = agent.session_prompt_tokens
|
||||||
completion = agent.session_completion_tokens
|
completion = agent.session_completion_tokens
|
||||||
total = agent.session_total_tokens
|
total = agent.session_total_tokens
|
||||||
@@ -8090,8 +7989,6 @@ class HermesCLI:
|
|||||||
print(f" Cache read tokens: {cache_read_tokens:>10,}")
|
print(f" Cache read tokens: {cache_read_tokens:>10,}")
|
||||||
print(f" Cache write tokens: {cache_write_tokens:>10,}")
|
print(f" Cache write tokens: {cache_write_tokens:>10,}")
|
||||||
print(f" Output tokens: {output_tokens:>10,}")
|
print(f" Output tokens: {output_tokens:>10,}")
|
||||||
if reasoning_tokens:
|
|
||||||
print(f" ↳ Reasoning (subset): {reasoning_tokens:>10,}")
|
|
||||||
print(f" Prompt tokens (total): {prompt:>10,}")
|
print(f" Prompt tokens (total): {prompt:>10,}")
|
||||||
print(f" Completion tokens: {completion:>10,}")
|
print(f" Completion tokens: {completion:>10,}")
|
||||||
print(f" Total tokens: {total:>10,}")
|
print(f" Total tokens: {total:>10,}")
|
||||||
@@ -9773,7 +9670,7 @@ class HermesCLI:
|
|||||||
# Debug: log to file (stdout may be devnull from redirect_stdout)
|
# Debug: log to file (stdout may be devnull from redirect_stdout)
|
||||||
try:
|
try:
|
||||||
_dbg = _hermes_home / "interrupt_debug.log"
|
_dbg = _hermes_home / "interrupt_debug.log"
|
||||||
with open(_dbg, "a", encoding="utf-8") as _f:
|
with open(_dbg, "a") as _f:
|
||||||
_f.write(f"{time.strftime('%H:%M:%S')} interrupt fired: msg={str(interrupt_msg)[:60]!r}, "
|
_f.write(f"{time.strftime('%H:%M:%S')} interrupt fired: msg={str(interrupt_msg)[:60]!r}, "
|
||||||
f"children={len(self.agent._active_children)}, "
|
f"children={len(self.agent._active_children)}, "
|
||||||
f"parent._interrupt={self.agent._interrupt_requested}\n")
|
f"parent._interrupt={self.agent._interrupt_requested}\n")
|
||||||
@@ -10316,24 +10213,6 @@ class HermesCLI:
|
|||||||
_welcome_text = "Welcome to Hermes Agent! Type your message or /help for commands."
|
_welcome_text = "Welcome to Hermes Agent! Type your message or /help for commands."
|
||||||
_welcome_color = "#FFF8DC"
|
_welcome_color = "#FFF8DC"
|
||||||
self._console_print(f"[{_welcome_color}]{_welcome_text}[/]")
|
self._console_print(f"[{_welcome_color}]{_welcome_text}[/]")
|
||||||
|
|
||||||
# Redaction opt-out warning (#17691): ON by default, loud when off.
|
|
||||||
# The redactor snapshots its state at import time so any toggle now
|
|
||||||
# won't affect the running process — we just want the operator to
|
|
||||||
# see that they're running without the safety net.
|
|
||||||
try:
|
|
||||||
_redact_raw = os.getenv("HERMES_REDACT_SECRETS", "true")
|
|
||||||
if _redact_raw.lower() not in ("1", "true", "yes", "on"):
|
|
||||||
self._console_print(
|
|
||||||
"[bold red]⚠ Secret redaction is DISABLED[/] "
|
|
||||||
f"(HERMES_REDACT_SECRETS={_redact_raw}). "
|
|
||||||
"API keys and tokens may appear verbatim in chat output, "
|
|
||||||
"session JSONs, and logs. Set "
|
|
||||||
"[cyan]security.redact_secrets: true[/] in config.yaml "
|
|
||||||
"to re-enable."
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
# First-time OpenClaw-residue banner — fires once if ~/.openclaw/ exists
|
# First-time OpenClaw-residue banner — fires once if ~/.openclaw/ exists
|
||||||
# after an OpenClaw→Hermes migration (especially migrations done by
|
# after an OpenClaw→Hermes migration (especially migrations done by
|
||||||
# OpenClaw's own tool, which doesn't archive the source directory).
|
# OpenClaw's own tool, which doesn't archive the source directory).
|
||||||
@@ -10605,7 +10484,7 @@ class HermesCLI:
|
|||||||
# Debug: log to file when message enters interrupt queue
|
# Debug: log to file when message enters interrupt queue
|
||||||
try:
|
try:
|
||||||
_dbg = _hermes_home / "interrupt_debug.log"
|
_dbg = _hermes_home / "interrupt_debug.log"
|
||||||
with open(_dbg, "a", encoding="utf-8") as _f:
|
with open(_dbg, "a") as _f:
|
||||||
_f.write(f"{time.strftime('%H:%M:%S')} ENTER: queued interrupt msg={str(payload)[:60]!r}, "
|
_f.write(f"{time.strftime('%H:%M:%S')} ENTER: queued interrupt msg={str(payload)[:60]!r}, "
|
||||||
f"agent_running={self._agent_running}\n")
|
f"agent_running={self._agent_running}\n")
|
||||||
except Exception:
|
except Exception:
|
||||||
@@ -12266,12 +12145,8 @@ class HermesCLI:
|
|||||||
# Set the custom handler on prompt_toolkit's event loop
|
# Set the custom handler on prompt_toolkit's event loop
|
||||||
try:
|
try:
|
||||||
import asyncio as _aio
|
import asyncio as _aio
|
||||||
# Use get_running_loop() to avoid DeprecationWarning on
|
_loop = _aio.get_event_loop()
|
||||||
# Python 3.10+ when called outside an async context.
|
|
||||||
_loop = _aio.get_running_loop()
|
|
||||||
_loop.set_exception_handler(_suppress_closed_loop_errors)
|
_loop.set_exception_handler(_suppress_closed_loop_errors)
|
||||||
except RuntimeError:
|
|
||||||
pass # No running loop -- nothing to patch
|
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
app.run()
|
app.run()
|
||||||
@@ -12409,15 +12284,6 @@ def main(
|
|||||||
"""
|
"""
|
||||||
global _active_worktree
|
global _active_worktree
|
||||||
|
|
||||||
# Force UTF-8 stdio on Windows before any banner/print() runs — the
|
|
||||||
# Rich console prints Unicode box-drawing characters that would
|
|
||||||
# UnicodeEncodeError on cp1252. No-op on Linux/macOS.
|
|
||||||
try:
|
|
||||||
from hermes_cli.stdio import configure_windows_stdio
|
|
||||||
configure_windows_stdio()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
# Signal to terminal_tool that we're in interactive mode
|
# Signal to terminal_tool that we're in interactive mode
|
||||||
# This enables interactive sudo password prompts with timeout
|
# This enables interactive sudo password prompts with timeout
|
||||||
os.environ["HERMES_INTERACTIVE"] = "1"
|
os.environ["HERMES_INTERACTIVE"] = "1"
|
||||||
@@ -12615,18 +12481,7 @@ def main(
|
|||||||
):
|
):
|
||||||
cli.session_id = cli.agent.session_id
|
cli.session_id = cli.agent.session_id
|
||||||
response = result.get("final_response", "") if isinstance(result, dict) else str(result)
|
response = result.get("final_response", "") if isinstance(result, dict) else str(result)
|
||||||
# Surface backend errors that produced no visible output
|
if response:
|
||||||
# (e.g. invalid model slug → provider 4xx). Mirrors the
|
|
||||||
# interactive CLI path. Write to stderr so piped stdout
|
|
||||||
# stays clean for automation wrappers.
|
|
||||||
if (
|
|
||||||
not response
|
|
||||||
and isinstance(result, dict)
|
|
||||||
and result.get("error")
|
|
||||||
and (result.get("failed") or result.get("partial"))
|
|
||||||
):
|
|
||||||
print(f"Error: {result['error']}", file=sys.stderr)
|
|
||||||
elif response:
|
|
||||||
print(response)
|
print(response)
|
||||||
# Session ID goes to stderr so piped stdout is clean.
|
# Session ID goes to stderr so piped stdout is clean.
|
||||||
print(f"\nsession_id: {cli.session_id}", file=sys.stderr)
|
print(f"\nsession_id: {cli.session_id}", file=sys.stderr)
|
||||||
|
|||||||
+10
-171
@@ -14,7 +14,6 @@ import contextvars
|
|||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import shutil
|
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
@@ -42,19 +41,6 @@ from hermes_time import now as _hermes_now
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
class CronPromptInjectionBlocked(Exception):
|
|
||||||
"""Raised by _build_job_prompt when the fully-assembled prompt trips the
|
|
||||||
injection scanner. Caught in run_job so the operator sees a clean
|
|
||||||
"job blocked" delivery instead of the scheduler crashing.
|
|
||||||
|
|
||||||
Assembled-prompt scanning (including loaded skill content) plugs the
|
|
||||||
gap from #3968: create-time scanning only covers the user-supplied
|
|
||||||
prompt field; skill content loaded at runtime was never scanned, so a
|
|
||||||
malicious skill could carry an injection payload that reached the
|
|
||||||
non-interactive (auto-approve) cron agent.
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_cron_enabled_toolsets(job: dict, cfg: dict) -> list[str] | None:
|
def _resolve_cron_enabled_toolsets(job: dict, cfg: dict) -> list[str] | None:
|
||||||
"""Resolve the toolset list for a cron job.
|
"""Resolve the toolset list for a cron job.
|
||||||
|
|
||||||
@@ -166,54 +152,9 @@ def _resolve_origin(job: dict) -> Optional[dict]:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def _plugin_cron_env_var(platform_name: str) -> str:
|
|
||||||
"""Return the cron home-channel env var registered by a plugin platform.
|
|
||||||
|
|
||||||
Falls through the platform registry so plugins that set
|
|
||||||
``cron_deliver_env_var`` on their ``PlatformEntry`` get cron delivery
|
|
||||||
support without editing this module.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
from hermes_cli.plugins import discover_plugins
|
|
||||||
discover_plugins() # idempotent
|
|
||||||
from gateway.platform_registry import platform_registry
|
|
||||||
entry = platform_registry.get(platform_name.lower())
|
|
||||||
if entry and entry.cron_deliver_env_var:
|
|
||||||
return entry.cron_deliver_env_var
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return ""
|
|
||||||
|
|
||||||
|
|
||||||
def _is_known_delivery_platform(platform_name: str) -> bool:
|
|
||||||
"""Whether ``platform_name`` is a valid cron delivery target.
|
|
||||||
|
|
||||||
Hardcoded built-ins in ``_KNOWN_DELIVERY_PLATFORMS`` are checked first;
|
|
||||||
plugin platforms registered via ``PlatformEntry`` are accepted if they
|
|
||||||
provide a ``cron_deliver_env_var``.
|
|
||||||
"""
|
|
||||||
name = platform_name.lower()
|
|
||||||
if name in _KNOWN_DELIVERY_PLATFORMS:
|
|
||||||
return True
|
|
||||||
return bool(_plugin_cron_env_var(name))
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_home_env_var(platform_name: str) -> str:
|
|
||||||
"""Return the env var name for a platform's cron home channel.
|
|
||||||
|
|
||||||
Built-in platforms are in ``_HOME_TARGET_ENV_VARS``; plugin platforms are
|
|
||||||
resolved from the platform registry.
|
|
||||||
"""
|
|
||||||
name = platform_name.lower()
|
|
||||||
env_var = _HOME_TARGET_ENV_VARS.get(name)
|
|
||||||
if env_var:
|
|
||||||
return env_var
|
|
||||||
return _plugin_cron_env_var(name)
|
|
||||||
|
|
||||||
|
|
||||||
def _get_home_target_chat_id(platform_name: str) -> str:
|
def _get_home_target_chat_id(platform_name: str) -> str:
|
||||||
"""Return the configured home target chat/room ID for a delivery platform."""
|
"""Return the configured home target chat/room ID for a delivery platform."""
|
||||||
env_var = _resolve_home_env_var(platform_name)
|
env_var = _HOME_TARGET_ENV_VARS.get(platform_name.lower())
|
||||||
if not env_var:
|
if not env_var:
|
||||||
return ""
|
return ""
|
||||||
value = os.getenv(env_var, "")
|
value = os.getenv(env_var, "")
|
||||||
@@ -226,7 +167,7 @@ def _get_home_target_chat_id(platform_name: str) -> str:
|
|||||||
|
|
||||||
def _get_home_target_thread_id(platform_name: str) -> Optional[str]:
|
def _get_home_target_thread_id(platform_name: str) -> Optional[str]:
|
||||||
"""Return the optional thread/topic ID for a platform home target."""
|
"""Return the optional thread/topic ID for a platform home target."""
|
||||||
env_var = _resolve_home_env_var(platform_name)
|
env_var = _HOME_TARGET_ENV_VARS.get(platform_name.lower())
|
||||||
if not env_var:
|
if not env_var:
|
||||||
return None
|
return None
|
||||||
value = os.getenv(f"{env_var}_THREAD_ID", "").strip()
|
value = os.getenv(f"{env_var}_THREAD_ID", "").strip()
|
||||||
@@ -237,24 +178,6 @@ def _get_home_target_thread_id(platform_name: str) -> Optional[str]:
|
|||||||
return value or None
|
return value or None
|
||||||
|
|
||||||
|
|
||||||
def _iter_home_target_platforms():
|
|
||||||
"""Iterate built-in + plugin platform names that expose a home channel.
|
|
||||||
|
|
||||||
Used by the ``deliver=origin`` fallback when the job has no origin.
|
|
||||||
"""
|
|
||||||
for name in _HOME_TARGET_ENV_VARS:
|
|
||||||
yield name
|
|
||||||
try:
|
|
||||||
from hermes_cli.plugins import discover_plugins
|
|
||||||
discover_plugins() # idempotent
|
|
||||||
from gateway.platform_registry import platform_registry
|
|
||||||
for entry in platform_registry.plugin_entries():
|
|
||||||
if entry.cron_deliver_env_var and entry.name not in _HOME_TARGET_ENV_VARS:
|
|
||||||
yield entry.name
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[dict]:
|
def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[dict]:
|
||||||
"""Resolve one concrete auto-delivery target for a cron job."""
|
"""Resolve one concrete auto-delivery target for a cron job."""
|
||||||
|
|
||||||
@@ -272,7 +195,7 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
|||||||
}
|
}
|
||||||
# Origin missing (e.g. job created via API/script) — try each
|
# Origin missing (e.g. job created via API/script) — try each
|
||||||
# platform's home channel as a fallback instead of silently dropping.
|
# platform's home channel as a fallback instead of silently dropping.
|
||||||
for platform_name in _iter_home_target_platforms():
|
for platform_name in _HOME_TARGET_ENV_VARS:
|
||||||
chat_id = _get_home_target_chat_id(platform_name)
|
chat_id = _get_home_target_chat_id(platform_name)
|
||||||
if chat_id:
|
if chat_id:
|
||||||
logger.info(
|
logger.info(
|
||||||
@@ -328,7 +251,7 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
|||||||
"thread_id": origin.get("thread_id"),
|
"thread_id": origin.get("thread_id"),
|
||||||
}
|
}
|
||||||
|
|
||||||
if not _is_known_delivery_platform(platform_name):
|
if platform_name.lower() not in _KNOWN_DELIVERY_PLATFORMS:
|
||||||
return None
|
return None
|
||||||
chat_id = _get_home_target_chat_id(platform_name)
|
chat_id = _get_home_target_chat_id(platform_name)
|
||||||
if not chat_id:
|
if not chat_id:
|
||||||
@@ -715,21 +638,7 @@ def _run_job_script(script_path: str) -> tuple[bool, str]:
|
|||||||
# choice explicit here keeps the allowed surface small and auditable.
|
# choice explicit here keeps the allowed surface small and auditable.
|
||||||
suffix = path.suffix.lower()
|
suffix = path.suffix.lower()
|
||||||
if suffix in (".sh", ".bash"):
|
if suffix in (".sh", ".bash"):
|
||||||
# Resolve bash dynamically so Windows (Git Bash) and Linux/macOS
|
argv = ["/bin/bash", str(path)]
|
||||||
# all work. On native Windows without Git for Windows installed
|
|
||||||
# shutil.which returns None — fall back to a clear error rather
|
|
||||||
# than a FileNotFoundError with a confusing "[WinError 2]"
|
|
||||||
# traceback.
|
|
||||||
_bash = shutil.which("bash") or (
|
|
||||||
"/bin/bash" if os.path.isfile("/bin/bash") else None
|
|
||||||
)
|
|
||||||
if _bash is None:
|
|
||||||
return False, (
|
|
||||||
f"Cannot run .sh/.bash script {path.name!r}: bash not found on PATH. "
|
|
||||||
"On Windows, install Git for Windows (which ships Git Bash) "
|
|
||||||
"or rewrite the script as Python (.py)."
|
|
||||||
)
|
|
||||||
argv = [_bash, str(path)]
|
|
||||||
else:
|
else:
|
||||||
argv = [sys.executable, str(path)]
|
argv = [sys.executable, str(path)]
|
||||||
|
|
||||||
@@ -896,7 +805,7 @@ def _build_job_prompt(job: dict, prerun_script: Optional[tuple] = None) -> str:
|
|||||||
|
|
||||||
skill_names = [str(name).strip() for name in skills if str(name).strip()]
|
skill_names = [str(name).strip() for name in skills if str(name).strip()]
|
||||||
if not skill_names:
|
if not skill_names:
|
||||||
return _scan_assembled_cron_prompt(prompt, job)
|
return prompt
|
||||||
|
|
||||||
from tools.skills_tool import skill_view
|
from tools.skills_tool import skill_view
|
||||||
from tools.skill_usage import bump_use
|
from tools.skill_usage import bump_use
|
||||||
@@ -939,32 +848,7 @@ def _build_job_prompt(job: dict, prerun_script: Optional[tuple] = None) -> str:
|
|||||||
|
|
||||||
if prompt:
|
if prompt:
|
||||||
parts.extend(["", f"The user has provided the following instruction alongside the skill invocation: {prompt}"])
|
parts.extend(["", f"The user has provided the following instruction alongside the skill invocation: {prompt}"])
|
||||||
return _scan_assembled_cron_prompt("\n".join(parts), job)
|
return "\n".join(parts)
|
||||||
|
|
||||||
|
|
||||||
def _scan_assembled_cron_prompt(assembled: str, job: dict) -> str:
|
|
||||||
"""Scan the fully-assembled cron prompt (including skill content) for
|
|
||||||
injection patterns. Raises ``CronPromptInjectionBlocked`` when a match
|
|
||||||
fires so ``run_job`` can surface a clear refusal to the operator.
|
|
||||||
|
|
||||||
Plugs the #3968 gap: ``_scan_cron_prompt`` runs on the user-supplied
|
|
||||||
prompt at create/update, but skill content is loaded from disk at
|
|
||||||
runtime and was never scanned. Since cron runs non-interactively
|
|
||||||
(auto-approves tool calls), a malicious skill carrying an injection
|
|
||||||
payload bypassed every gate.
|
|
||||||
"""
|
|
||||||
from tools.cronjob_tools import _scan_cron_prompt
|
|
||||||
|
|
||||||
scan_error = _scan_cron_prompt(assembled)
|
|
||||||
if scan_error:
|
|
||||||
job_label = job.get("name") or job.get("id") or "<unknown>"
|
|
||||||
logger.warning(
|
|
||||||
"Cron job '%s': assembled prompt blocked by injection scanner — %s",
|
|
||||||
job_label,
|
|
||||||
scan_error,
|
|
||||||
)
|
|
||||||
raise CronPromptInjectionBlocked(scan_error)
|
|
||||||
return assembled
|
|
||||||
|
|
||||||
|
|
||||||
def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||||
@@ -1119,31 +1003,7 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
|||||||
)
|
)
|
||||||
return True, silent_doc, SILENT_MARKER, None
|
return True, silent_doc, SILENT_MARKER, None
|
||||||
|
|
||||||
try:
|
prompt = _build_job_prompt(job, prerun_script=prerun_script)
|
||||||
prompt = _build_job_prompt(job, prerun_script=prerun_script)
|
|
||||||
except CronPromptInjectionBlocked as block_exc:
|
|
||||||
# Assembled prompt (user prompt + loaded skill content) tripped the
|
|
||||||
# injection scanner. Refuse to run the agent this tick and surface
|
|
||||||
# a clear failure to the operator so they see WHY the scheduled job
|
|
||||||
# didn't run and can audit the offending skill.
|
|
||||||
logger.warning(
|
|
||||||
"Job '%s' (ID: %s): blocked by prompt-injection scanner — %s",
|
|
||||||
job_name, job_id, block_exc,
|
|
||||||
)
|
|
||||||
blocked_doc = (
|
|
||||||
f"# Cron Job: {job_name}\n\n"
|
|
||||||
f"**Job ID:** {job_id}\n"
|
|
||||||
f"**Run Time:** {_hermes_now().strftime('%Y-%m-%d %H:%M:%S')}\n"
|
|
||||||
f"**Status:** BLOCKED\n\n"
|
|
||||||
"The assembled prompt (user prompt + loaded skill content) tripped "
|
|
||||||
"the cron injection scanner and the agent was NOT run.\n\n"
|
|
||||||
f"**Scanner result:** {block_exc}\n\n"
|
|
||||||
"Audit the skill(s) attached to this job for prompt-injection "
|
|
||||||
"payloads or invisible-unicode markers. If the skill is legitimate "
|
|
||||||
"and the match is a false positive, rephrase the content to avoid "
|
|
||||||
"the threat pattern (`tools/cronjob_tools.py::_CRON_THREAT_PATTERNS`)."
|
|
||||||
)
|
|
||||||
return False, blocked_doc, "", str(block_exc)
|
|
||||||
if prompt is None:
|
if prompt is None:
|
||||||
logger.info("Job '%s': script produced no output, skipping AI call.", job_name)
|
logger.info("Job '%s': script produced no output, skipping AI call.", job_name)
|
||||||
return True, "", SILENT_MARKER, None
|
return True, "", SILENT_MARKER, None
|
||||||
@@ -1228,7 +1088,7 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
|||||||
import yaml
|
import yaml
|
||||||
_cfg_path = str(_get_hermes_home() / "config.yaml")
|
_cfg_path = str(_get_hermes_home() / "config.yaml")
|
||||||
if os.path.exists(_cfg_path):
|
if os.path.exists(_cfg_path):
|
||||||
with open(_cfg_path, encoding="utf-8") as _f:
|
with open(_cfg_path) as _f:
|
||||||
_cfg = yaml.safe_load(_f) or {}
|
_cfg = yaml.safe_load(_f) or {}
|
||||||
_cfg = _expand_env_vars(_cfg)
|
_cfg = _expand_env_vars(_cfg)
|
||||||
_model_cfg = _cfg.get("model", {})
|
_model_cfg = _cfg.get("model", {})
|
||||||
@@ -1338,27 +1198,6 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Job '%s': failed to load credential pool for %s: %s", job_id, runtime_provider, e)
|
logger.debug("Job '%s': failed to load credential pool for %s: %s", job_id, runtime_provider, e)
|
||||||
|
|
||||||
# Initialize MCP servers so configured mcp_servers are available to
|
|
||||||
# the agent's tool registry before AIAgent is constructed. Without
|
|
||||||
# this, cron jobs never saw any MCP tools — only the gateway / CLI
|
|
||||||
# paths called discover_mcp_tools() at startup. Idempotent: subsequent
|
|
||||||
# ticks short-circuit on already-connected servers inside
|
|
||||||
# register_mcp_servers(). Non-fatal on failure: a broken MCP server
|
|
||||||
# shouldn't kill an otherwise-working cron job. See #4219.
|
|
||||||
try:
|
|
||||||
from tools.mcp_tool import discover_mcp_tools
|
|
||||||
_mcp_tools = discover_mcp_tools()
|
|
||||||
if _mcp_tools:
|
|
||||||
logger.info(
|
|
||||||
"Job '%s': %d MCP tool(s) available",
|
|
||||||
job_id, len(_mcp_tools),
|
|
||||||
)
|
|
||||||
except Exception as _mcp_exc:
|
|
||||||
logger.warning(
|
|
||||||
"Job '%s': MCP initialization failed (non-fatal): %s",
|
|
||||||
job_id, _mcp_exc,
|
|
||||||
)
|
|
||||||
|
|
||||||
agent = AIAgent(
|
agent = AIAgent(
|
||||||
model=model,
|
model=model,
|
||||||
api_key=runtime.get("api_key"),
|
api_key=runtime.get("api_key"),
|
||||||
@@ -1611,7 +1450,7 @@ def tick(verbose: bool = True, adapters=None, loop=None) -> int:
|
|||||||
# Cross-platform file locking: fcntl on Unix, msvcrt on Windows
|
# Cross-platform file locking: fcntl on Unix, msvcrt on Windows
|
||||||
lock_fd = None
|
lock_fd = None
|
||||||
try:
|
try:
|
||||||
lock_fd = open(lock_file, "w", encoding="utf-8")
|
lock_fd = open(lock_file, "w")
|
||||||
if fcntl:
|
if fcntl:
|
||||||
fcntl.flock(lock_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
fcntl.flock(lock_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||||
elif msvcrt:
|
elif msvcrt:
|
||||||
|
|||||||
@@ -14,9 +14,6 @@
|
|||||||
# keys; exposing it on LAN without auth is unsafe. If you want remote
|
# keys; exposing it on LAN without auth is unsafe. If you want remote
|
||||||
# access, use an SSH tunnel or put it behind a reverse proxy that
|
# access, use an SSH tunnel or put it behind a reverse proxy that
|
||||||
# adds authentication — do NOT pass --insecure --host 0.0.0.0.
|
# adds authentication — do NOT pass --insecure --host 0.0.0.0.
|
||||||
# - If you override entrypoint, keep /opt/hermes/docker/entrypoint.sh in
|
|
||||||
# the command chain. It drops root to the hermes user before gateway
|
|
||||||
# files such as gateway.lock are created.
|
|
||||||
# - The gateway's API server is off unless you uncomment API_SERVER_KEY
|
# - The gateway's API server is off unless you uncomment API_SERVER_KEY
|
||||||
# and API_SERVER_HOST. See docs/user-guide/api-server.md before doing
|
# and API_SERVER_HOST. See docs/user-guide/api-server.md before doing
|
||||||
# this on an internet-facing host.
|
# this on an internet-facing host.
|
||||||
@@ -44,15 +41,6 @@ services:
|
|||||||
# - TEAMS_TENANT_ID=${TEAMS_TENANT_ID}
|
# - TEAMS_TENANT_ID=${TEAMS_TENANT_ID}
|
||||||
# - TEAMS_ALLOWED_USERS=${TEAMS_ALLOWED_USERS}
|
# - TEAMS_ALLOWED_USERS=${TEAMS_ALLOWED_USERS}
|
||||||
# - TEAMS_PORT=${TEAMS_PORT:-3978}
|
# - TEAMS_PORT=${TEAMS_PORT:-3978}
|
||||||
# Google Chat — uncomment and fill in to enable the Google Chat gateway.
|
|
||||||
# See website/docs/user-guide/messaging/google_chat.md for the full setup.
|
|
||||||
# The SA JSON path must point to a file mounted into the container —
|
|
||||||
# add a volume entry above (e.g. ``- ~/.hermes/google-chat-sa.json:/secrets/google-chat-sa.json:ro``)
|
|
||||||
# then set GOOGLE_CHAT_SERVICE_ACCOUNT_JSON to that mount path.
|
|
||||||
# - GOOGLE_CHAT_PROJECT_ID=${GOOGLE_CHAT_PROJECT_ID}
|
|
||||||
# - GOOGLE_CHAT_SUBSCRIPTION_NAME=${GOOGLE_CHAT_SUBSCRIPTION_NAME}
|
|
||||||
# - GOOGLE_CHAT_SERVICE_ACCOUNT_JSON=${GOOGLE_CHAT_SERVICE_ACCOUNT_JSON}
|
|
||||||
# - GOOGLE_CHAT_ALLOWED_USERS=${GOOGLE_CHAT_ALLOWED_USERS}
|
|
||||||
command: ["gateway", "run"]
|
command: ["gateway", "run"]
|
||||||
|
|
||||||
dashboard:
|
dashboard:
|
||||||
|
|||||||
@@ -365,7 +365,7 @@ class TerminalBench2EvalEnv(HermesAgentBaseEnv):
|
|||||||
os.makedirs(log_dir, exist_ok=True)
|
os.makedirs(log_dir, exist_ok=True)
|
||||||
run_ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
run_ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||||
self._streaming_path = os.path.join(log_dir, f"samples_{run_ts}.jsonl")
|
self._streaming_path = os.path.join(log_dir, f"samples_{run_ts}.jsonl")
|
||||||
self._streaming_file = open(self._streaming_path, "w", encoding="utf-8")
|
self._streaming_file = open(self._streaming_path, "w")
|
||||||
self._streaming_lock = __import__("threading").Lock()
|
self._streaming_lock = __import__("threading").Lock()
|
||||||
print(f" Streaming results to: {self._streaming_path}")
|
print(f" Streaming results to: {self._streaming_path}")
|
||||||
|
|
||||||
|
|||||||
@@ -422,7 +422,7 @@ class YCBenchEvalEnv(HermesAgentBaseEnv):
|
|||||||
os.makedirs(log_dir, exist_ok=True)
|
os.makedirs(log_dir, exist_ok=True)
|
||||||
run_ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
run_ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||||
self._streaming_path = os.path.join(log_dir, f"samples_{run_ts}.jsonl")
|
self._streaming_path = os.path.join(log_dir, f"samples_{run_ts}.jsonl")
|
||||||
self._streaming_file = open(self._streaming_path, "w", encoding="utf-8")
|
self._streaming_file = open(self._streaming_path, "w")
|
||||||
self._streaming_lock = threading.Lock()
|
self._streaming_lock = threading.Lock()
|
||||||
|
|
||||||
print(f"\nYC-Bench eval matrix: {len(self.all_eval_items)} runs")
|
print(f"\nYC-Bench eval matrix: {len(self.all_eval_items)} runs")
|
||||||
|
|||||||
+5
-82
@@ -809,12 +809,6 @@ def load_gateway_config() -> GatewayConfig:
|
|||||||
os.environ["SLACK_FREE_RESPONSE_CHANNELS"] = str(frc)
|
os.environ["SLACK_FREE_RESPONSE_CHANNELS"] = str(frc)
|
||||||
if "reactions" in slack_cfg and not os.getenv("SLACK_REACTIONS"):
|
if "reactions" in slack_cfg and not os.getenv("SLACK_REACTIONS"):
|
||||||
os.environ["SLACK_REACTIONS"] = str(slack_cfg["reactions"]).lower()
|
os.environ["SLACK_REACTIONS"] = str(slack_cfg["reactions"]).lower()
|
||||||
# allowed_channels: if set, bot ONLY responds in these channels (whitelist)
|
|
||||||
ac = slack_cfg.get("allowed_channels")
|
|
||||||
if ac is not None and not os.getenv("SLACK_ALLOWED_CHANNELS"):
|
|
||||||
if isinstance(ac, list):
|
|
||||||
ac = ",".join(str(v) for v in ac)
|
|
||||||
os.environ["SLACK_ALLOWED_CHANNELS"] = str(ac)
|
|
||||||
|
|
||||||
# Discord settings → env vars (env vars take precedence)
|
# Discord settings → env vars (env vars take precedence)
|
||||||
discord_cfg = yaml_cfg.get("discord", {})
|
discord_cfg = yaml_cfg.get("discord", {})
|
||||||
@@ -899,12 +893,6 @@ def load_gateway_config() -> GatewayConfig:
|
|||||||
if isinstance(frc, list):
|
if isinstance(frc, list):
|
||||||
frc = ",".join(str(v) for v in frc)
|
frc = ",".join(str(v) for v in frc)
|
||||||
os.environ["TELEGRAM_FREE_RESPONSE_CHATS"] = str(frc)
|
os.environ["TELEGRAM_FREE_RESPONSE_CHATS"] = str(frc)
|
||||||
# allowed_chats: if set, bot ONLY responds in these group chats (whitelist)
|
|
||||||
ac = telegram_cfg.get("allowed_chats")
|
|
||||||
if ac is not None and not os.getenv("TELEGRAM_ALLOWED_CHATS"):
|
|
||||||
if isinstance(ac, list):
|
|
||||||
ac = ",".join(str(v) for v in ac)
|
|
||||||
os.environ["TELEGRAM_ALLOWED_CHATS"] = str(ac)
|
|
||||||
ignored_threads = telegram_cfg.get("ignored_threads")
|
ignored_threads = telegram_cfg.get("ignored_threads")
|
||||||
if ignored_threads is not None and not os.getenv("TELEGRAM_IGNORED_THREADS"):
|
if ignored_threads is not None and not os.getenv("TELEGRAM_IGNORED_THREADS"):
|
||||||
if isinstance(ignored_threads, list):
|
if isinstance(ignored_threads, list):
|
||||||
@@ -988,35 +976,12 @@ def load_gateway_config() -> GatewayConfig:
|
|||||||
if isinstance(frc, list):
|
if isinstance(frc, list):
|
||||||
frc = ",".join(str(v) for v in frc)
|
frc = ",".join(str(v) for v in frc)
|
||||||
os.environ["DINGTALK_FREE_RESPONSE_CHATS"] = str(frc)
|
os.environ["DINGTALK_FREE_RESPONSE_CHATS"] = str(frc)
|
||||||
# allowed_chats: if set, bot ONLY responds in these group chats (whitelist)
|
|
||||||
ac = dingtalk_cfg.get("allowed_chats")
|
|
||||||
if ac is not None and not os.getenv("DINGTALK_ALLOWED_CHATS"):
|
|
||||||
if isinstance(ac, list):
|
|
||||||
ac = ",".join(str(v) for v in ac)
|
|
||||||
os.environ["DINGTALK_ALLOWED_CHATS"] = str(ac)
|
|
||||||
allowed = dingtalk_cfg.get("allowed_users")
|
allowed = dingtalk_cfg.get("allowed_users")
|
||||||
if allowed is not None and not os.getenv("DINGTALK_ALLOWED_USERS"):
|
if allowed is not None and not os.getenv("DINGTALK_ALLOWED_USERS"):
|
||||||
if isinstance(allowed, list):
|
if isinstance(allowed, list):
|
||||||
allowed = ",".join(str(v) for v in allowed)
|
allowed = ",".join(str(v) for v in allowed)
|
||||||
os.environ["DINGTALK_ALLOWED_USERS"] = str(allowed)
|
os.environ["DINGTALK_ALLOWED_USERS"] = str(allowed)
|
||||||
|
|
||||||
# Mattermost settings → env vars (env vars take precedence)
|
|
||||||
mattermost_cfg = yaml_cfg.get("mattermost", {})
|
|
||||||
if isinstance(mattermost_cfg, dict):
|
|
||||||
if "require_mention" in mattermost_cfg and not os.getenv("MATTERMOST_REQUIRE_MENTION"):
|
|
||||||
os.environ["MATTERMOST_REQUIRE_MENTION"] = str(mattermost_cfg["require_mention"]).lower()
|
|
||||||
frc = mattermost_cfg.get("free_response_channels")
|
|
||||||
if frc is not None and not os.getenv("MATTERMOST_FREE_RESPONSE_CHANNELS"):
|
|
||||||
if isinstance(frc, list):
|
|
||||||
frc = ",".join(str(v) for v in frc)
|
|
||||||
os.environ["MATTERMOST_FREE_RESPONSE_CHANNELS"] = str(frc)
|
|
||||||
# allowed_channels: if set, bot ONLY responds in these channels (whitelist)
|
|
||||||
ac = mattermost_cfg.get("allowed_channels")
|
|
||||||
if ac is not None and not os.getenv("MATTERMOST_ALLOWED_CHANNELS"):
|
|
||||||
if isinstance(ac, list):
|
|
||||||
ac = ",".join(str(v) for v in ac)
|
|
||||||
os.environ["MATTERMOST_ALLOWED_CHANNELS"] = str(ac)
|
|
||||||
|
|
||||||
# Matrix settings → env vars (env vars take precedence)
|
# Matrix settings → env vars (env vars take precedence)
|
||||||
matrix_cfg = yaml_cfg.get("matrix", {})
|
matrix_cfg = yaml_cfg.get("matrix", {})
|
||||||
if isinstance(matrix_cfg, dict):
|
if isinstance(matrix_cfg, dict):
|
||||||
@@ -1027,12 +992,6 @@ def load_gateway_config() -> GatewayConfig:
|
|||||||
if isinstance(frc, list):
|
if isinstance(frc, list):
|
||||||
frc = ",".join(str(v) for v in frc)
|
frc = ",".join(str(v) for v in frc)
|
||||||
os.environ["MATRIX_FREE_RESPONSE_ROOMS"] = str(frc)
|
os.environ["MATRIX_FREE_RESPONSE_ROOMS"] = str(frc)
|
||||||
# allowed_rooms: if set, bot ONLY responds in these rooms (whitelist)
|
|
||||||
ar = matrix_cfg.get("allowed_rooms")
|
|
||||||
if ar is not None and not os.getenv("MATRIX_ALLOWED_ROOMS"):
|
|
||||||
if isinstance(ar, list):
|
|
||||||
ar = ",".join(str(v) for v in ar)
|
|
||||||
os.environ["MATRIX_ALLOWED_ROOMS"] = str(ar)
|
|
||||||
if "auto_thread" in matrix_cfg and not os.getenv("MATRIX_AUTO_THREAD"):
|
if "auto_thread" in matrix_cfg and not os.getenv("MATRIX_AUTO_THREAD"):
|
||||||
os.environ["MATRIX_AUTO_THREAD"] = str(matrix_cfg["auto_thread"]).lower()
|
os.environ["MATRIX_AUTO_THREAD"] = str(matrix_cfg["auto_thread"]).lower()
|
||||||
if "dm_mention_threads" in matrix_cfg and not os.getenv("MATRIX_DM_MENTION_THREADS"):
|
if "dm_mention_threads" in matrix_cfg and not os.getenv("MATRIX_DM_MENTION_THREADS"):
|
||||||
@@ -1193,17 +1152,10 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
|||||||
|
|
||||||
# WhatsApp (typically uses different auth mechanism)
|
# WhatsApp (typically uses different auth mechanism)
|
||||||
whatsapp_enabled = os.getenv("WHATSAPP_ENABLED", "").lower() in ("true", "1", "yes")
|
whatsapp_enabled = os.getenv("WHATSAPP_ENABLED", "").lower() in ("true", "1", "yes")
|
||||||
whatsapp_disabled_explicitly = os.getenv("WHATSAPP_ENABLED", "").lower() in ("false", "0", "no")
|
if whatsapp_enabled:
|
||||||
if Platform.WHATSAPP in config.platforms:
|
if Platform.WHATSAPP not in config.platforms:
|
||||||
# YAML config exists — respect explicit disable
|
config.platforms[Platform.WHATSAPP] = PlatformConfig()
|
||||||
wa_cfg = config.platforms[Platform.WHATSAPP]
|
config.platforms[Platform.WHATSAPP].enabled = True
|
||||||
if whatsapp_disabled_explicitly:
|
|
||||||
wa_cfg.enabled = False
|
|
||||||
elif whatsapp_enabled:
|
|
||||||
wa_cfg.enabled = True
|
|
||||||
# else: keep whatever the YAML set
|
|
||||||
elif whatsapp_enabled:
|
|
||||||
config.platforms[Platform.WHATSAPP] = PlatformConfig(enabled=True)
|
|
||||||
whatsapp_home = os.getenv("WHATSAPP_HOME_CHANNEL")
|
whatsapp_home = os.getenv("WHATSAPP_HOME_CHANNEL")
|
||||||
if whatsapp_home and Platform.WHATSAPP in config.platforms:
|
if whatsapp_home and Platform.WHATSAPP in config.platforms:
|
||||||
config.platforms[Platform.WHATSAPP].home_channel = HomeChannel(
|
config.platforms[Platform.WHATSAPP].home_channel = HomeChannel(
|
||||||
@@ -1664,10 +1616,7 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
|||||||
# Registry-driven enable for plugin platforms. Built-ins have explicit
|
# Registry-driven enable for plugin platforms. Built-ins have explicit
|
||||||
# blocks above; plugins expose check_fn() which is the single source of
|
# blocks above; plugins expose check_fn() which is the single source of
|
||||||
# truth for "are my env vars set?". When it returns True, ensure the
|
# truth for "are my env vars set?". When it returns True, ensure the
|
||||||
# platform is enabled so start() will create its adapter. Plugins that
|
# platform is enabled so start() will create its adapter.
|
||||||
# need to seed ``PlatformConfig.extra`` from env vars (e.g. Google Chat's
|
|
||||||
# project_id / subscription_name) can supply ``env_enablement_fn`` on
|
|
||||||
# their PlatformEntry — called here BEFORE adapter construction.
|
|
||||||
try:
|
try:
|
||||||
from hermes_cli.plugins import discover_plugins
|
from hermes_cli.plugins import discover_plugins
|
||||||
discover_plugins() # idempotent
|
discover_plugins() # idempotent
|
||||||
@@ -1683,31 +1632,5 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
|||||||
if platform not in config.platforms:
|
if platform not in config.platforms:
|
||||||
config.platforms[platform] = PlatformConfig()
|
config.platforms[platform] = PlatformConfig()
|
||||||
config.platforms[platform].enabled = True
|
config.platforms[platform].enabled = True
|
||||||
# Seed extras from env if the plugin opted in.
|
|
||||||
if entry.env_enablement_fn is not None:
|
|
||||||
try:
|
|
||||||
seed = entry.env_enablement_fn()
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug(
|
|
||||||
"env_enablement_fn for %s raised: %s", entry.name, e
|
|
||||||
)
|
|
||||||
seed = None
|
|
||||||
if isinstance(seed, dict) and seed:
|
|
||||||
# Extract the home_channel dict (if provided) so we wire it
|
|
||||||
# up as a proper HomeChannel dataclass. Everything else is
|
|
||||||
# merged into ``extra``.
|
|
||||||
home = seed.pop("home_channel", None)
|
|
||||||
config.platforms[platform].extra.update(seed)
|
|
||||||
if isinstance(home, dict) and home.get("chat_id"):
|
|
||||||
config.platforms[platform].home_channel = HomeChannel(
|
|
||||||
platform=platform,
|
|
||||||
chat_id=str(home["chat_id"]),
|
|
||||||
name=str(home.get("name") or "Home"),
|
|
||||||
thread_id=(
|
|
||||||
str(home["thread_id"])
|
|
||||||
if home.get("thread_id")
|
|
||||||
else None
|
|
||||||
),
|
|
||||||
)
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Plugin platform enable pass failed: %s", e)
|
logger.debug("Plugin platform enable pass failed: %s", e)
|
||||||
|
|||||||
@@ -35,12 +35,6 @@ _GLOBAL_DEFAULTS: dict[str, Any] = {
|
|||||||
"show_reasoning": False,
|
"show_reasoning": False,
|
||||||
"tool_preview_length": 0,
|
"tool_preview_length": 0,
|
||||||
"streaming": None, # None = follow top-level streaming config
|
"streaming": None, # None = follow top-level streaming config
|
||||||
# When true, delete tool-progress / "Still working..." / status bubbles
|
|
||||||
# after the final response lands on platforms that support message
|
|
||||||
# deletion (e.g. Telegram). Off by default — progress is still shown
|
|
||||||
# live, just cleaned up after success so the chat doesn't fill up with
|
|
||||||
# stale breadcrumbs. Failed runs leave bubbles in place as breadcrumbs.
|
|
||||||
"cleanup_progress": False,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -194,10 +188,6 @@ def _normalise(setting: str, value: Any) -> Any:
|
|||||||
if isinstance(value, str):
|
if isinstance(value, str):
|
||||||
return value.lower() in ("true", "1", "yes", "on")
|
return value.lower() in ("true", "1", "yes", "on")
|
||||||
return bool(value)
|
return bool(value)
|
||||||
if setting == "cleanup_progress":
|
|
||||||
if isinstance(value, str):
|
|
||||||
return value.lower() in ("true", "1", "yes", "on")
|
|
||||||
return bool(value)
|
|
||||||
if setting == "tool_preview_length":
|
if setting == "tool_preview_length":
|
||||||
try:
|
try:
|
||||||
return int(value)
|
return int(value)
|
||||||
|
|||||||
+1
-12
@@ -195,23 +195,12 @@ class PairingStore:
|
|||||||
"""
|
"""
|
||||||
Approve a pairing code. Adds the user to the approved list.
|
Approve a pairing code. Adds the user to the approved list.
|
||||||
|
|
||||||
Returns {user_id, user_name} on success, None if code is
|
Returns {user_id, user_name} on success, None if code is invalid/expired.
|
||||||
invalid/expired OR the platform is currently locked out after
|
|
||||||
``MAX_FAILED_ATTEMPTS`` failed approvals (#10195). Callers can
|
|
||||||
disambiguate with ``_is_locked_out(platform)``.
|
|
||||||
"""
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._cleanup_expired(platform)
|
self._cleanup_expired(platform)
|
||||||
code = code.upper().strip()
|
code = code.upper().strip()
|
||||||
|
|
||||||
# Lockout check — must run before the pending lookup so a
|
|
||||||
# valid code (e.g. one already sitting in pending) cannot be
|
|
||||||
# accepted once the lockout fires. Without this, the lockout
|
|
||||||
# only blocks `generate_code`, not `approve_code` — nullifying
|
|
||||||
# the brute-force protection for any code already issued.
|
|
||||||
if self._is_locked_out(platform):
|
|
||||||
return None
|
|
||||||
|
|
||||||
pending = self._load_json(self._pending_path(platform))
|
pending = self._load_json(self._pending_path(platform))
|
||||||
if code not in pending:
|
if code not in pending:
|
||||||
self._record_failed_attempt(platform)
|
self._record_failed_attempt(platform)
|
||||||
|
|||||||
@@ -110,21 +110,6 @@ class PlatformEntry:
|
|||||||
# Do not use markdown."). Empty string = no hint.
|
# Do not use markdown."). Empty string = no hint.
|
||||||
platform_hint: str = ""
|
platform_hint: str = ""
|
||||||
|
|
||||||
# ── Env-driven auto-configuration ──
|
|
||||||
# Optional: read env vars, return a dict of ``PlatformConfig.extra`` fields
|
|
||||||
# to seed when the platform is auto-enabled. Called during
|
|
||||||
# ``_apply_env_overrides`` BEFORE the adapter is constructed, so
|
|
||||||
# ``gateway status`` etc. can reflect env-only configuration without
|
|
||||||
# instantiating the adapter. Return ``None`` (or an empty dict) to skip.
|
|
||||||
# Signature: () -> Optional[dict[str, Any]]
|
|
||||||
env_enablement_fn: Optional[Callable[[], Optional[dict]]] = None
|
|
||||||
|
|
||||||
# Optional: home-channel env var name for cron/notification delivery
|
|
||||||
# (e.g. ``"IRC_HOME_CHANNEL"``). When set, ``cron.scheduler`` treats this
|
|
||||||
# platform as a valid ``deliver=<name>`` target and reads the env var to
|
|
||||||
# resolve the default chat/room ID. Empty = no cron home-channel support.
|
|
||||||
cron_deliver_env_var: str = ""
|
|
||||||
|
|
||||||
|
|
||||||
class PlatformRegistry:
|
class PlatformRegistry:
|
||||||
"""Central registry of platform adapters.
|
"""Central registry of platform adapters.
|
||||||
|
|||||||
@@ -4,34 +4,18 @@ There are two ways to add a platform to the Hermes gateway:
|
|||||||
|
|
||||||
## Plugin Path (Recommended for Community/Third-Party)
|
## Plugin Path (Recommended for Community/Third-Party)
|
||||||
|
|
||||||
Create a plugin directory in `~/.hermes/plugins/` (or under `plugins/platforms/`
|
Create a plugin directory in `~/.hermes/plugins/` with a `PLUGIN.yaml` and
|
||||||
for bundled plugins) with a `plugin.yaml` and `adapter.py`. The adapter
|
`adapter.py`. The adapter inherits from `BasePlatformAdapter` and registers
|
||||||
inherits from `BasePlatformAdapter` and registers via
|
via `ctx.register_platform()` in the `register(ctx)` entry point. This
|
||||||
`ctx.register_platform()` in the `register(ctx)` entry point. This requires
|
requires **zero changes to core Hermes code**.
|
||||||
**zero changes to core Hermes code**.
|
|
||||||
|
|
||||||
The plugin system automatically handles: adapter creation, config parsing,
|
The plugin system automatically handles: adapter creation, config parsing,
|
||||||
user authorization, cron delivery, send_message routing, system prompt hints,
|
user authorization, cron delivery, send_message routing, system prompt hints,
|
||||||
status display, gateway setup, and more.
|
status display, gateway setup, and more.
|
||||||
|
|
||||||
**Three optional hooks cover the edges most adapters need:**
|
See `plugins/platforms/irc/` for a complete reference implementation, and
|
||||||
|
|
||||||
- `env_enablement_fn: () -> Optional[dict]` — seeds `PlatformConfig.extra`
|
|
||||||
(and an optional `home_channel` dict) from env vars BEFORE the adapter is
|
|
||||||
constructed. Without this, env-only setups don't surface in
|
|
||||||
`hermes gateway status` or `get_connected_platforms()` until the SDK
|
|
||||||
instantiates.
|
|
||||||
- `cron_deliver_env_var: str` — name of the `*_HOME_CHANNEL` env var. When
|
|
||||||
set, `deliver=<name>` cron jobs route to this var without editing
|
|
||||||
`cron/scheduler.py`'s hardcoded sets.
|
|
||||||
- `plugin.yaml` `requires_env` / `optional_env` rich-dict entries —
|
|
||||||
auto-populate `OPTIONAL_ENV_VARS` in `hermes_cli/config.py` so the setup
|
|
||||||
wizard surfaces proper descriptions, prompts, password flags, and URLs.
|
|
||||||
|
|
||||||
See `plugins/platforms/irc/`, `plugins/platforms/teams/`, and
|
|
||||||
`plugins/platforms/google_chat/` for complete working examples, and
|
|
||||||
`website/docs/developer-guide/adding-platform-adapters.md` for the full
|
`website/docs/developer-guide/adding-platform-adapters.md` for the full
|
||||||
plugin guide with code examples and hook documentation.
|
plugin guide with code examples.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -917,16 +917,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
|||||||
"type": "bearer",
|
"type": "bearer",
|
||||||
"required": bool(self._api_key),
|
"required": bool(self._api_key),
|
||||||
},
|
},
|
||||||
"runtime": {
|
|
||||||
"mode": "server_agent",
|
|
||||||
"tool_execution": "server",
|
|
||||||
"split_runtime": False,
|
|
||||||
"description": (
|
|
||||||
"The API server creates a server-side Hermes AIAgent; "
|
|
||||||
"tools execute on the API-server host unless a future "
|
|
||||||
"explicit split-runtime mode is enabled."
|
|
||||||
),
|
|
||||||
},
|
|
||||||
"features": {
|
"features": {
|
||||||
"chat_completions": True,
|
"chat_completions": True,
|
||||||
"chat_completions_streaming": True,
|
"chat_completions_streaming": True,
|
||||||
@@ -1326,8 +1316,8 @@ class APIServerAdapter(BasePlatformAdapter):
|
|||||||
try:
|
try:
|
||||||
result, agent_usage = await agent_task
|
result, agent_usage = await agent_task
|
||||||
usage = agent_usage or usage
|
usage = agent_usage or usage
|
||||||
except Exception as exc:
|
except Exception:
|
||||||
logger.warning("Agent task %s failed, usage data lost: %s", completion_id, exc)
|
pass
|
||||||
|
|
||||||
# Finish chunk
|
# Finish chunk
|
||||||
finish_chunk = {
|
finish_chunk = {
|
||||||
@@ -1898,12 +1888,12 @@ class APIServerAdapter(BasePlatformAdapter):
|
|||||||
"output_tokens": usage.get("output_tokens", 0),
|
"output_tokens": usage.get("output_tokens", 0),
|
||||||
"total_tokens": usage.get("total_tokens", 0),
|
"total_tokens": usage.get("total_tokens", 0),
|
||||||
}
|
}
|
||||||
full_history = self._build_response_conversation_history(
|
full_history = list(conversation_history)
|
||||||
conversation_history,
|
full_history.append({"role": "user", "content": user_message})
|
||||||
user_message,
|
if isinstance(result, dict) and result.get("messages"):
|
||||||
result,
|
full_history.extend(result["messages"])
|
||||||
final_response_text,
|
else:
|
||||||
)
|
full_history.append({"role": "assistant", "content": final_response_text})
|
||||||
_persist_response_snapshot(
|
_persist_response_snapshot(
|
||||||
completed_env,
|
completed_env,
|
||||||
conversation_history_snapshot=full_history,
|
conversation_history_snapshot=full_history,
|
||||||
@@ -2202,22 +2192,17 @@ class APIServerAdapter(BasePlatformAdapter):
|
|||||||
|
|
||||||
# Build the full conversation history for storage
|
# Build the full conversation history for storage
|
||||||
# (includes tool calls from the agent run)
|
# (includes tool calls from the agent run)
|
||||||
full_history = self._build_response_conversation_history(
|
full_history = list(conversation_history)
|
||||||
conversation_history,
|
full_history.append({"role": "user", "content": user_message})
|
||||||
user_message,
|
# Add agent's internal messages if available
|
||||||
result,
|
agent_messages = result.get("messages", [])
|
||||||
final_response,
|
if agent_messages:
|
||||||
)
|
full_history.extend(agent_messages)
|
||||||
|
else:
|
||||||
|
full_history.append({"role": "assistant", "content": final_response})
|
||||||
|
|
||||||
# Build output items from the current turn only. AIAgent returns a
|
# Build output items (includes tool calls + final message)
|
||||||
# full transcript in result["messages"], while older/mocked paths may
|
output_items = self._extract_output_items(result)
|
||||||
# return only the current turn suffix.
|
|
||||||
output_start_index = self._response_messages_turn_start_index(
|
|
||||||
conversation_history,
|
|
||||||
user_message,
|
|
||||||
result,
|
|
||||||
)
|
|
||||||
output_items = self._extract_output_items(result, start_index=output_start_index)
|
|
||||||
|
|
||||||
response_data = {
|
response_data = {
|
||||||
"id": response_id,
|
"id": response_id,
|
||||||
@@ -2509,70 +2494,17 @@ class APIServerAdapter(BasePlatformAdapter):
|
|||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _build_response_conversation_history(
|
def _extract_output_items(result: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||||
conversation_history: List[Dict[str, Any]],
|
|
||||||
user_message: Any,
|
|
||||||
result: Dict[str, Any],
|
|
||||||
final_response: Any,
|
|
||||||
) -> List[Dict[str, Any]]:
|
|
||||||
"""Build the stored Responses transcript without duplicating history."""
|
|
||||||
prior = list(conversation_history)
|
|
||||||
current_user = {"role": "user", "content": user_message}
|
|
||||||
agent_messages = result.get("messages") if isinstance(result, dict) else None
|
|
||||||
|
|
||||||
if isinstance(agent_messages, list) and agent_messages:
|
|
||||||
turn_start = APIServerAdapter._response_messages_turn_start_index(
|
|
||||||
conversation_history,
|
|
||||||
user_message,
|
|
||||||
result,
|
|
||||||
)
|
|
||||||
if turn_start:
|
|
||||||
return list(agent_messages)
|
|
||||||
|
|
||||||
full_history = prior
|
|
||||||
full_history.append(current_user)
|
|
||||||
full_history.extend(agent_messages)
|
|
||||||
return full_history
|
|
||||||
|
|
||||||
full_history = prior
|
|
||||||
full_history.append(current_user)
|
|
||||||
full_history.append({"role": "assistant", "content": final_response})
|
|
||||||
return full_history
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _response_messages_turn_start_index(
|
|
||||||
conversation_history: List[Dict[str, Any]],
|
|
||||||
user_message: Any,
|
|
||||||
result: Dict[str, Any],
|
|
||||||
) -> int:
|
|
||||||
"""Detect transcript-shaped result["messages"] and return turn start."""
|
|
||||||
agent_messages = result.get("messages") if isinstance(result, dict) else None
|
|
||||||
if not isinstance(agent_messages, list) or not agent_messages:
|
|
||||||
return 0
|
|
||||||
|
|
||||||
prior = list(conversation_history)
|
|
||||||
current_user = {"role": "user", "content": user_message}
|
|
||||||
expected_prefix = prior + [current_user]
|
|
||||||
if agent_messages[:len(expected_prefix)] == expected_prefix:
|
|
||||||
return len(expected_prefix)
|
|
||||||
if prior and agent_messages[:len(prior)] == prior:
|
|
||||||
return len(prior)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _extract_output_items(result: Dict[str, Any], start_index: int = 0) -> List[Dict[str, Any]]:
|
|
||||||
"""
|
"""
|
||||||
Build the output item array from the agent's messages.
|
Build the full output item array from the agent's messages.
|
||||||
|
|
||||||
Walks *result["messages"]* starting at *start_index* and emits:
|
Walks *result["messages"]* and emits:
|
||||||
- ``function_call`` items for each tool_call on assistant messages
|
- ``function_call`` items for each tool_call on assistant messages
|
||||||
- ``function_call_output`` items for each tool-role message
|
- ``function_call_output`` items for each tool-role message
|
||||||
- a final ``message`` item with the assistant's text reply
|
- a final ``message`` item with the assistant's text reply
|
||||||
"""
|
"""
|
||||||
items: List[Dict[str, Any]] = []
|
items: List[Dict[str, Any]] = []
|
||||||
messages = result.get("messages", [])
|
messages = result.get("messages", [])
|
||||||
if start_index > 0:
|
|
||||||
messages = messages[start_index:]
|
|
||||||
|
|
||||||
for msg in messages:
|
for msg in messages:
|
||||||
role = msg.get("role")
|
role = msg.get("role")
|
||||||
|
|||||||
+26
-114
@@ -1304,52 +1304,37 @@ class BasePlatformAdapter(ABC):
|
|||||||
self._fatal_error_code = None
|
self._fatal_error_code = None
|
||||||
self._fatal_error_message = None
|
self._fatal_error_message = None
|
||||||
self._fatal_error_retryable = True
|
self._fatal_error_retryable = True
|
||||||
self._write_runtime_status_safe("connected", platform_state="connected", error_code=None, error_message=None)
|
try:
|
||||||
|
from gateway.status import write_runtime_status
|
||||||
|
write_runtime_status(platform=self.platform.value, platform_state="connected", error_code=None, error_message=None)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
def _mark_disconnected(self) -> None:
|
def _mark_disconnected(self) -> None:
|
||||||
self._running = False
|
self._running = False
|
||||||
if self.has_fatal_error:
|
if self.has_fatal_error:
|
||||||
return
|
return
|
||||||
self._write_runtime_status_safe("disconnected", platform_state="disconnected", error_code=None, error_message=None)
|
try:
|
||||||
|
from gateway.status import write_runtime_status
|
||||||
|
write_runtime_status(platform=self.platform.value, platform_state="disconnected", error_code=None, error_message=None)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
def _set_fatal_error(self, code: str, message: str, *, retryable: bool) -> None:
|
def _set_fatal_error(self, code: str, message: str, *, retryable: bool) -> None:
|
||||||
self._running = False
|
self._running = False
|
||||||
self._fatal_error_code = code
|
self._fatal_error_code = code
|
||||||
self._fatal_error_message = message
|
self._fatal_error_message = message
|
||||||
self._fatal_error_retryable = retryable
|
self._fatal_error_retryable = retryable
|
||||||
self._write_runtime_status_safe("fatal", platform_state="fatal", error_code=code, error_message=message)
|
|
||||||
|
|
||||||
def _write_runtime_status_safe(self, context: str, **kwargs) -> None:
|
|
||||||
"""Write runtime status; log first failure per context at warning, rest at debug.
|
|
||||||
|
|
||||||
Status writes can fail on permissions, ENOSPC, missing status dir, etc.
|
|
||||||
A persistently failing status dir used to be silent (``except: pass``).
|
|
||||||
Logging every failure would spam the log on reconnect loops, so this
|
|
||||||
surfaces the first failure per (platform, context) at warning level and
|
|
||||||
downgrades subsequent failures to debug.
|
|
||||||
"""
|
|
||||||
try:
|
try:
|
||||||
from gateway.status import write_runtime_status
|
from gateway.status import write_runtime_status
|
||||||
write_runtime_status(platform=self.platform.value, **kwargs)
|
write_runtime_status(
|
||||||
except Exception as exc:
|
platform=self.platform.value,
|
||||||
# Use getattr so object.__new__(...) test harnesses that skip __init__
|
platform_state="fatal",
|
||||||
# don't blow up on attribute access.
|
error_code=code,
|
||||||
logged = getattr(self, "_status_write_logged", None)
|
error_message=message,
|
||||||
if logged is None:
|
)
|
||||||
logged = set()
|
except Exception:
|
||||||
try:
|
pass
|
||||||
self._status_write_logged = logged
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
key = (self.platform.value, context)
|
|
||||||
if key not in logged:
|
|
||||||
logger.warning(
|
|
||||||
"Failed to write runtime status (%s) for %s: %s (further failures at debug level)",
|
|
||||||
context, self.platform.value, exc,
|
|
||||||
)
|
|
||||||
logged.add(key)
|
|
||||||
else:
|
|
||||||
logger.debug("Failed to write runtime status (%s) for %s: %s", context, self.platform.value, exc)
|
|
||||||
|
|
||||||
async def _notify_fatal_error(self) -> None:
|
async def _notify_fatal_error(self) -> None:
|
||||||
handler = self._fatal_error_handler
|
handler = self._fatal_error_handler
|
||||||
@@ -1889,38 +1874,23 @@ class BasePlatformAdapter(ABC):
|
|||||||
def extract_media(content: str) -> Tuple[List[Tuple[str, bool]], str]:
|
def extract_media(content: str) -> Tuple[List[Tuple[str, bool]], str]:
|
||||||
"""
|
"""
|
||||||
Extract MEDIA:<path> tags and [[audio_as_voice]] directives from response text.
|
Extract MEDIA:<path> tags and [[audio_as_voice]] directives from response text.
|
||||||
|
|
||||||
The TTS tool returns responses like:
|
The TTS tool returns responses like:
|
||||||
[[audio_as_voice]]
|
[[audio_as_voice]]
|
||||||
MEDIA:/path/to/audio.ogg
|
MEDIA:/path/to/audio.ogg
|
||||||
|
|
||||||
Skills that produce large/lossless images (e.g. info-graph, where a
|
|
||||||
rendered JPG is 1-2 MB but Telegram's sendPhoto recompresses to
|
|
||||||
~200 KB at 1280px) can use ``[[as_document]]`` to request unmodified
|
|
||||||
delivery via sendDocument instead of sendPhoto/sendMediaGroup. The
|
|
||||||
directive is detected at the dispatch sites (which have access to the
|
|
||||||
original response); this method just strips it so it never leaks into
|
|
||||||
user-visible text. Per-file granularity is intentionally not exposed —
|
|
||||||
when an agent emits ``[[as_document]]`` once, every image path in the
|
|
||||||
same response is delivered as a document, mirroring the all-or-nothing
|
|
||||||
scope of ``[[audio_as_voice]]``.
|
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
content: The response text to scan.
|
content: The response text to scan.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Tuple of (list of (path, is_voice) pairs, cleaned content with tags removed).
|
Tuple of (list of (path, is_voice) pairs, cleaned content with tags removed).
|
||||||
"""
|
"""
|
||||||
media = []
|
media = []
|
||||||
cleaned = content
|
cleaned = content
|
||||||
|
|
||||||
# Check for [[audio_as_voice]] directive
|
# Check for [[audio_as_voice]] directive
|
||||||
has_voice_tag = "[[audio_as_voice]]" in content
|
has_voice_tag = "[[audio_as_voice]]" in content
|
||||||
cleaned = cleaned.replace("[[audio_as_voice]]", "")
|
cleaned = cleaned.replace("[[audio_as_voice]]", "")
|
||||||
# Strip [[as_document]] directive — callers inspect the original
|
|
||||||
# ``content`` for it (so they can still react to it); here we just
|
|
||||||
# keep it out of the user-visible cleaned text.
|
|
||||||
cleaned = cleaned.replace("[[as_document]]", "")
|
|
||||||
|
|
||||||
# Extract MEDIA:<path> tags, allowing optional whitespace after the colon
|
# Extract MEDIA:<path> tags, allowing optional whitespace after the colon
|
||||||
# and quoted/backticked paths for LLM-formatted outputs.
|
# and quoted/backticked paths for LLM-formatted outputs.
|
||||||
@@ -2126,52 +2096,9 @@ class BasePlatformAdapter(ABC):
|
|||||||
|
|
||||||
``generation`` lets callers tie the callback to a specific gateway run
|
``generation`` lets callers tie the callback to a specific gateway run
|
||||||
generation so stale runs cannot clear callbacks owned by a fresher run.
|
generation so stale runs cannot clear callbacks owned by a fresher run.
|
||||||
|
|
||||||
If a callback for the same ``session_key`` (and generation, when set)
|
|
||||||
is already registered, the new callback is chained — both fire, in
|
|
||||||
registration order, with per-callback exception isolation. This lets
|
|
||||||
independent features (background-review release + temporary-bubble
|
|
||||||
cleanup) coexist without clobbering each other. Stale-generation
|
|
||||||
callers never overwrite a fresher generation's slot.
|
|
||||||
"""
|
"""
|
||||||
if not session_key or not callable(callback):
|
if not session_key or not callable(callback):
|
||||||
return
|
return
|
||||||
|
|
||||||
existing = self._post_delivery_callbacks.get(session_key)
|
|
||||||
if existing is not None:
|
|
||||||
if isinstance(existing, tuple) and len(existing) == 2:
|
|
||||||
existing_gen, existing_cb = existing
|
|
||||||
else:
|
|
||||||
existing_gen, existing_cb = None, existing
|
|
||||||
# Stale-generation registrations never overwrite a fresher slot.
|
|
||||||
if (
|
|
||||||
existing_gen is not None
|
|
||||||
and generation is not None
|
|
||||||
and int(generation) < int(existing_gen)
|
|
||||||
):
|
|
||||||
return
|
|
||||||
# Same-or-newer generation: chain with the existing callback so
|
|
||||||
# both fire in registration order.
|
|
||||||
if callable(existing_cb) and (
|
|
||||||
existing_gen is None
|
|
||||||
or generation is None
|
|
||||||
or int(existing_gen) == int(generation)
|
|
||||||
):
|
|
||||||
_prev = existing_cb
|
|
||||||
_new = callback
|
|
||||||
|
|
||||||
def _chained() -> None:
|
|
||||||
try:
|
|
||||||
_prev()
|
|
||||||
except Exception:
|
|
||||||
logger.debug("Post-delivery callback failed", exc_info=True)
|
|
||||||
try:
|
|
||||||
_new()
|
|
||||||
except Exception:
|
|
||||||
logger.debug("Post-delivery callback failed", exc_info=True)
|
|
||||||
|
|
||||||
callback = _chained
|
|
||||||
|
|
||||||
if generation is None:
|
if generation is None:
|
||||||
self._post_delivery_callbacks[session_key] = callback
|
self._post_delivery_callbacks[session_key] = callback
|
||||||
else:
|
else:
|
||||||
@@ -2845,21 +2772,13 @@ class BasePlatformAdapter(ABC):
|
|||||||
if not response:
|
if not response:
|
||||||
logger.debug("[%s] Handler returned empty/None response for %s", self.name, event.source.chat_id)
|
logger.debug("[%s] Handler returned empty/None response for %s", self.name, event.source.chat_id)
|
||||||
if response:
|
if response:
|
||||||
# Capture [[as_document]] before extract_media strips it, so the
|
|
||||||
# dispatch partition below can route image-extension files
|
|
||||||
# through send_document instead of send_multiple_images. Used
|
|
||||||
# by skills that produce large/lossless images (e.g. info-graph)
|
|
||||||
# where Telegram's sendPhoto recompression destroys legibility.
|
|
||||||
force_document_attachments = "[[as_document]]" in response
|
|
||||||
|
|
||||||
# Extract MEDIA:<path> tags (from TTS tool) before other processing
|
# Extract MEDIA:<path> tags (from TTS tool) before other processing
|
||||||
media_files, response = self.extract_media(response)
|
media_files, response = self.extract_media(response)
|
||||||
|
|
||||||
# Extract image URLs and send them as native platform attachments
|
# Extract image URLs and send them as native platform attachments
|
||||||
images, text_content = self.extract_images(response)
|
images, text_content = self.extract_images(response)
|
||||||
# Strip any remaining internal directives from message body (fixes #1561)
|
# Strip any remaining internal directives from message body (fixes #1561)
|
||||||
text_content = text_content.replace("[[audio_as_voice]]", "").strip()
|
text_content = text_content.replace("[[audio_as_voice]]", "").strip()
|
||||||
text_content = text_content.replace("[[as_document]]", "").strip()
|
|
||||||
text_content = re.sub(r"MEDIA:\s*\S+", "", text_content).strip()
|
text_content = re.sub(r"MEDIA:\s*\S+", "", text_content).strip()
|
||||||
if images:
|
if images:
|
||||||
logger.info("[%s] extract_images found %d image(s) in response (%d chars)", self.name, len(images), len(response))
|
logger.info("[%s] extract_images found %d image(s) in response (%d chars)", self.name, len(images), len(response))
|
||||||
@@ -2961,26 +2880,19 @@ class BasePlatformAdapter(ABC):
|
|||||||
_IMAGE_EXTS = {'.jpg', '.jpeg', '.png', '.webp', '.gif'}
|
_IMAGE_EXTS = {'.jpg', '.jpeg', '.png', '.webp', '.gif'}
|
||||||
|
|
||||||
# Partition images out of media_files + local_files so they
|
# Partition images out of media_files + local_files so they
|
||||||
# can be sent as a single batch (Signal RPC). When
|
# can be sent as a single batch (Signal RPC)
|
||||||
# ``[[as_document]]`` was set on the original response, image
|
|
||||||
# files skip the photo path and route to send_document below
|
|
||||||
# so they're delivered with original bytes (no Telegram
|
|
||||||
# sendPhoto recompression).
|
|
||||||
from urllib.parse import quote as _quote
|
from urllib.parse import quote as _quote
|
||||||
_image_paths: list = []
|
_image_paths: list = []
|
||||||
_non_image_media: list = []
|
_non_image_media: list = []
|
||||||
for media_path, is_voice in media_files:
|
for media_path, is_voice in media_files:
|
||||||
_ext = Path(media_path).suffix.lower()
|
_ext = Path(media_path).suffix.lower()
|
||||||
if (_ext in _IMAGE_EXTS
|
if _ext in _IMAGE_EXTS and not is_voice:
|
||||||
and not is_voice
|
|
||||||
and not force_document_attachments):
|
|
||||||
_image_paths.append(media_path)
|
_image_paths.append(media_path)
|
||||||
else:
|
else:
|
||||||
_non_image_media.append((media_path, is_voice))
|
_non_image_media.append((media_path, is_voice))
|
||||||
_non_image_local: list = []
|
_non_image_local: list = []
|
||||||
for file_path in local_files:
|
for file_path in local_files:
|
||||||
if (Path(file_path).suffix.lower() in _IMAGE_EXTS
|
if Path(file_path).suffix.lower() in _IMAGE_EXTS:
|
||||||
and not force_document_attachments):
|
|
||||||
_image_paths.append(file_path)
|
_image_paths.append(file_path)
|
||||||
else:
|
else:
|
||||||
_non_image_local.append(file_path)
|
_non_image_local.append(file_path)
|
||||||
|
|||||||
@@ -365,20 +365,6 @@ class DingTalkAdapter(BasePlatformAdapter):
|
|||||||
return {str(part).strip() for part in raw if str(part).strip()}
|
return {str(part).strip() for part in raw if str(part).strip()}
|
||||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
||||||
|
|
||||||
def _dingtalk_allowed_chats(self) -> Set[str]:
|
|
||||||
"""Return the whitelist of group chat IDs the bot will respond in.
|
|
||||||
|
|
||||||
When non-empty, group messages from chats NOT in this set are silently
|
|
||||||
ignored — even if the bot is @mentioned. DMs are never filtered.
|
|
||||||
Empty set means no restriction (fully backward compatible).
|
|
||||||
"""
|
|
||||||
raw = self.config.extra.get("allowed_chats") if self.config.extra else None
|
|
||||||
if raw is None:
|
|
||||||
raw = os.getenv("DINGTALK_ALLOWED_CHATS", "")
|
|
||||||
if isinstance(raw, list):
|
|
||||||
return {str(part).strip() for part in raw if str(part).strip()}
|
|
||||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
|
||||||
|
|
||||||
def _compile_mention_patterns(self) -> List[re.Pattern]:
|
def _compile_mention_patterns(self) -> List[re.Pattern]:
|
||||||
"""Compile optional regex wake-word patterns for group triggers."""
|
"""Compile optional regex wake-word patterns for group triggers."""
|
||||||
patterns = self.config.extra.get("mention_patterns") if self.config.extra else None
|
patterns = self.config.extra.get("mention_patterns") if self.config.extra else None
|
||||||
@@ -457,21 +443,13 @@ class DingTalkAdapter(BasePlatformAdapter):
|
|||||||
|
|
||||||
DMs remain unrestricted (subject to ``allowed_users`` which is enforced
|
DMs remain unrestricted (subject to ``allowed_users`` which is enforced
|
||||||
earlier). Group messages are accepted when:
|
earlier). Group messages are accepted when:
|
||||||
- the chat passes the ``allowed_chats`` whitelist (when set)
|
|
||||||
- the chat is explicitly allowlisted in ``free_response_chats``
|
- the chat is explicitly allowlisted in ``free_response_chats``
|
||||||
- ``require_mention`` is disabled
|
- ``require_mention`` is disabled
|
||||||
- the bot is @mentioned (``is_in_at_list``)
|
- the bot is @mentioned (``is_in_at_list``)
|
||||||
- the text matches a configured regex wake-word pattern
|
- the text matches a configured regex wake-word pattern
|
||||||
|
|
||||||
When ``allowed_chats`` is non-empty, it acts as a hard gate — messages
|
|
||||||
from any group chat not in the list are ignored regardless of the
|
|
||||||
other rules.
|
|
||||||
"""
|
"""
|
||||||
if not is_group:
|
if not is_group:
|
||||||
return True
|
return True
|
||||||
allowed = self._dingtalk_allowed_chats()
|
|
||||||
if allowed and chat_id and chat_id not in allowed:
|
|
||||||
return False
|
|
||||||
if chat_id and chat_id in self._dingtalk_free_response_chats():
|
if chat_id and chat_id in self._dingtalk_free_response_chats():
|
||||||
return True
|
return True
|
||||||
if not self._dingtalk_require_mention():
|
if not self._dingtalk_require_mention():
|
||||||
|
|||||||
+42
-348
@@ -10,8 +10,6 @@ Uses discord.py library for:
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import struct
|
import struct
|
||||||
@@ -26,10 +24,6 @@ logger = logging.getLogger(__name__)
|
|||||||
|
|
||||||
VALID_THREAD_AUTO_ARCHIVE_MINUTES = {60, 1440, 4320, 10080}
|
VALID_THREAD_AUTO_ARCHIVE_MINUTES = {60, 1440, 4320, 10080}
|
||||||
_DISCORD_COMMAND_SYNC_POLICIES = {"safe", "bulk", "off"}
|
_DISCORD_COMMAND_SYNC_POLICIES = {"safe", "bulk", "off"}
|
||||||
_DISCORD_COMMAND_SYNC_STATE_SUBDIR = "gateway"
|
|
||||||
_DISCORD_COMMAND_SYNC_STATE_FILENAME = "discord_command_sync_state.json"
|
|
||||||
_DISCORD_COMMAND_SYNC_MUTATION_INTERVAL_SECONDS = 4.5
|
|
||||||
_DISCORD_COMMAND_SYNC_MAX_RATE_LIMIT_SLEEP_SECONDS = 30.0
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
import discord
|
import discord
|
||||||
@@ -51,7 +45,6 @@ from gateway.config import Platform, PlatformConfig
|
|||||||
import re
|
import re
|
||||||
|
|
||||||
from gateway.platforms.helpers import MessageDeduplicator, ThreadParticipationTracker
|
from gateway.platforms.helpers import MessageDeduplicator, ThreadParticipationTracker
|
||||||
from utils import atomic_json_write
|
|
||||||
from gateway.platforms.base import (
|
from gateway.platforms.base import (
|
||||||
BasePlatformAdapter,
|
BasePlatformAdapter,
|
||||||
MessageEvent,
|
MessageEvent,
|
||||||
@@ -477,34 +470,6 @@ class VoiceReceiver:
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
def _read_dm_role_auth_guild() -> Optional[int]:
|
|
||||||
"""Return the guild ID opted-in for DM role-based auth, or None.
|
|
||||||
|
|
||||||
Reads ``discord.dm_role_auth_guild`` from config.yaml. This is
|
|
||||||
deliberately a config.yaml-only setting (not an env var): per repo
|
|
||||||
policy, ``~/.hermes/.env`` is for secrets only, and this is a
|
|
||||||
behavioral setting. Guild IDs aren't secrets.
|
|
||||||
|
|
||||||
Accepts ints or numeric strings in the config. Anything else
|
|
||||||
(empty, malformed, None) returns None, which keeps the secure
|
|
||||||
default (DM role-auth disabled).
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
from hermes_cli.config import read_raw_config
|
|
||||||
cfg = read_raw_config() or {}
|
|
||||||
discord_cfg = cfg.get("discord", {}) or {}
|
|
||||||
raw = discord_cfg.get("dm_role_auth_guild")
|
|
||||||
except Exception:
|
|
||||||
return None
|
|
||||||
if raw is None or raw == "":
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
guild_id = int(raw)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return None
|
|
||||||
return guild_id if guild_id > 0 else None
|
|
||||||
|
|
||||||
|
|
||||||
class DiscordAdapter(BasePlatformAdapter):
|
class DiscordAdapter(BasePlatformAdapter):
|
||||||
"""
|
"""
|
||||||
Discord bot adapter.
|
Discord bot adapter.
|
||||||
@@ -729,17 +694,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
# human-user allowlist below (bots aren't in it).
|
# human-user allowlist below (bots aren't in it).
|
||||||
else:
|
else:
|
||||||
# Non-bot: enforce the configured user/role allowlists.
|
# Non-bot: enforce the configured user/role allowlists.
|
||||||
# Pass guild + is_dm so role checks are scoped to the
|
if not self._is_allowed_user(str(message.author.id), message.author):
|
||||||
# originating guild (prevents cross-guild DM bypass, see
|
|
||||||
# _is_allowed_user docstring).
|
|
||||||
_msg_guild = getattr(message, "guild", None)
|
|
||||||
_is_dm = isinstance(message.channel, discord.DMChannel) or _msg_guild is None
|
|
||||||
if not self._is_allowed_user(
|
|
||||||
str(message.author.id),
|
|
||||||
message.author,
|
|
||||||
guild=_msg_guild,
|
|
||||||
is_dm=_is_dm,
|
|
||||||
):
|
|
||||||
return
|
return
|
||||||
|
|
||||||
# Multi-agent filtering: if the message mentions specific bots
|
# Multi-agent filtering: if the message mentions specific bots
|
||||||
@@ -870,167 +825,6 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
|
|
||||||
logger.info("[%s] Disconnected", self.name)
|
logger.info("[%s] Disconnected", self.name)
|
||||||
|
|
||||||
def _command_sync_state_path(self) -> _Path:
|
|
||||||
from hermes_constants import get_hermes_home
|
|
||||||
|
|
||||||
directory = get_hermes_home() / _DISCORD_COMMAND_SYNC_STATE_SUBDIR
|
|
||||||
try:
|
|
||||||
directory.mkdir(parents=True, exist_ok=True)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return directory / _DISCORD_COMMAND_SYNC_STATE_FILENAME
|
|
||||||
|
|
||||||
def _read_command_sync_state(self) -> dict:
|
|
||||||
try:
|
|
||||||
path = self._command_sync_state_path()
|
|
||||||
if not path.exists():
|
|
||||||
return {}
|
|
||||||
data = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
except Exception:
|
|
||||||
return {}
|
|
||||||
return data if isinstance(data, dict) else {}
|
|
||||||
|
|
||||||
def _write_command_sync_state(self, state: dict) -> None:
|
|
||||||
atomic_json_write(
|
|
||||||
self._command_sync_state_path(),
|
|
||||||
state,
|
|
||||||
indent=None,
|
|
||||||
separators=(",", ":"),
|
|
||||||
)
|
|
||||||
|
|
||||||
def _command_sync_state_key(self, app_id: Any) -> str:
|
|
||||||
return str(app_id or "unknown")
|
|
||||||
|
|
||||||
def _desired_command_sync_fingerprint(self) -> str:
|
|
||||||
tree = self._client.tree if self._client else None
|
|
||||||
desired = []
|
|
||||||
if tree is not None:
|
|
||||||
desired = [
|
|
||||||
self._canonicalize_app_command_payload(command.to_dict(tree))
|
|
||||||
for command in tree.get_commands()
|
|
||||||
]
|
|
||||||
desired.sort(key=lambda item: (item.get("type", 1), item.get("name", "")))
|
|
||||||
payload = json.dumps(desired, sort_keys=True, separators=(",", ":"))
|
|
||||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
|
||||||
|
|
||||||
def _command_sync_skip_reason(self, app_id: Any, fingerprint: str) -> Optional[str]:
|
|
||||||
entry = self._read_command_sync_state().get(self._command_sync_state_key(app_id))
|
|
||||||
if not isinstance(entry, dict):
|
|
||||||
return None
|
|
||||||
now = time.time()
|
|
||||||
retry_after_until = float(entry.get("retry_after_until") or 0)
|
|
||||||
if retry_after_until > now:
|
|
||||||
remaining = max(1, int(retry_after_until - now))
|
|
||||||
return f"Discord asked us to wait before syncing slash commands; retry in {remaining}s"
|
|
||||||
if entry.get("fingerprint") == fingerprint and entry.get("last_success_at"):
|
|
||||||
return "same slash-command fingerprint already synced"
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _record_command_sync_attempt(self, app_id: Any, fingerprint: str) -> None:
|
|
||||||
state = self._read_command_sync_state()
|
|
||||||
state[self._command_sync_state_key(app_id)] = {
|
|
||||||
**(
|
|
||||||
state.get(self._command_sync_state_key(app_id))
|
|
||||||
if isinstance(state.get(self._command_sync_state_key(app_id)), dict)
|
|
||||||
else {}
|
|
||||||
),
|
|
||||||
"fingerprint": fingerprint,
|
|
||||||
"last_attempt_at": time.time(),
|
|
||||||
}
|
|
||||||
self._write_command_sync_state(state)
|
|
||||||
|
|
||||||
def _record_command_sync_rate_limit(self, app_id: Any, fingerprint: str, retry_after: float) -> None:
|
|
||||||
retry_after = max(1.0, float(retry_after))
|
|
||||||
state = self._read_command_sync_state()
|
|
||||||
state[self._command_sync_state_key(app_id)] = {
|
|
||||||
**(
|
|
||||||
state.get(self._command_sync_state_key(app_id))
|
|
||||||
if isinstance(state.get(self._command_sync_state_key(app_id)), dict)
|
|
||||||
else {}
|
|
||||||
),
|
|
||||||
"fingerprint": fingerprint,
|
|
||||||
"last_attempt_at": time.time(),
|
|
||||||
"retry_after_until": time.time() + retry_after,
|
|
||||||
"retry_after": retry_after,
|
|
||||||
}
|
|
||||||
self._write_command_sync_state(state)
|
|
||||||
|
|
||||||
def _record_command_sync_success(self, app_id: Any, fingerprint: str, summary: dict) -> None:
|
|
||||||
state = self._read_command_sync_state()
|
|
||||||
state[self._command_sync_state_key(app_id)] = {
|
|
||||||
"fingerprint": fingerprint,
|
|
||||||
"last_attempt_at": time.time(),
|
|
||||||
"last_success_at": time.time(),
|
|
||||||
"summary": summary,
|
|
||||||
}
|
|
||||||
self._write_command_sync_state(state)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _extract_discord_retry_after(exc: BaseException) -> Optional[float]:
|
|
||||||
value = getattr(exc, "retry_after", None)
|
|
||||||
if value is not None:
|
|
||||||
try:
|
|
||||||
return max(1.0, float(value))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return None
|
|
||||||
response = getattr(exc, "response", None)
|
|
||||||
headers = getattr(response, "headers", None)
|
|
||||||
if headers:
|
|
||||||
for key in ("Retry-After", "X-RateLimit-Reset-After"):
|
|
||||||
try:
|
|
||||||
raw = headers.get(key)
|
|
||||||
except Exception:
|
|
||||||
raw = None
|
|
||||||
if raw is None:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
return max(1.0, float(raw))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
continue
|
|
||||||
return None
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _is_discord_rate_limit(exc: BaseException) -> bool:
|
|
||||||
"""True only for exceptions that look like Discord 429 rate limits.
|
|
||||||
|
|
||||||
Narrower than ``hasattr(exc, 'retry_after')``: discord.py's own
|
|
||||||
``RateLimited`` exception and any HTTPException with status 429
|
|
||||||
qualify. This prevents suppressing unrelated failures that happen
|
|
||||||
to expose a ``retry_after`` attribute."""
|
|
||||||
# discord.py emits RateLimited / HTTPException subclasses for 429s.
|
|
||||||
# Guard with isinstance-of-class so a mocked ``discord`` module
|
|
||||||
# (where attrs are MagicMocks, not types) doesn't trip isinstance.
|
|
||||||
if DISCORD_AVAILABLE and discord is not None:
|
|
||||||
for attr_name in ("RateLimited", "HTTPException"):
|
|
||||||
cls = getattr(discord, attr_name, None)
|
|
||||||
if not isinstance(cls, type):
|
|
||||||
continue
|
|
||||||
if isinstance(exc, cls):
|
|
||||||
if attr_name == "RateLimited":
|
|
||||||
return True
|
|
||||||
status = getattr(exc, "status", None)
|
|
||||||
if status == 429:
|
|
||||||
return True
|
|
||||||
# Fallback duck-type: something named like a rate-limit with a
|
|
||||||
# numeric retry_after. Covers mocked clients in tests and exotic
|
|
||||||
# transports, without swallowing arbitrary exceptions.
|
|
||||||
name = type(exc).__name__.lower()
|
|
||||||
if ("ratelimit" in name or "rate_limit" in name) and getattr(exc, "retry_after", None) is not None:
|
|
||||||
return True
|
|
||||||
response = getattr(exc, "response", None)
|
|
||||||
status = getattr(response, "status", None) or getattr(response, "status_code", None)
|
|
||||||
if status == 429:
|
|
||||||
return True
|
|
||||||
return False
|
|
||||||
|
|
||||||
def _command_sync_mutation_interval_seconds(self) -> float:
|
|
||||||
return _DISCORD_COMMAND_SYNC_MUTATION_INTERVAL_SECONDS
|
|
||||||
|
|
||||||
async def _sleep_between_command_sync_mutations(self) -> None:
|
|
||||||
interval = self._command_sync_mutation_interval_seconds()
|
|
||||||
if interval > 0:
|
|
||||||
await asyncio.sleep(interval)
|
|
||||||
|
|
||||||
async def _run_post_connect_initialization(self) -> None:
|
async def _run_post_connect_initialization(self) -> None:
|
||||||
"""Finish non-critical startup work after Discord is connected."""
|
"""Finish non-critical startup work after Discord is connected."""
|
||||||
if not self._client:
|
if not self._client:
|
||||||
@@ -1046,46 +840,14 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
logger.info("[%s] Synced %d slash command(s) via bulk tree sync", self.name, len(synced))
|
logger.info("[%s] Synced %d slash command(s) via bulk tree sync", self.name, len(synced))
|
||||||
return
|
return
|
||||||
|
|
||||||
app_id = getattr(self._client, "application_id", None) or getattr(getattr(self._client, "user", None), "id", None)
|
# Discord's per-app command-management bucket is ~5 writes / 20 s,
|
||||||
fingerprint = self._desired_command_sync_fingerprint()
|
# so a mass-prune-plus-upsert reconcile (e.g. 77 orphans + 30
|
||||||
skip_reason = self._command_sync_skip_reason(app_id, fingerprint)
|
# desired = 107 writes) takes several minutes of forced waits.
|
||||||
if skip_reason:
|
# A flat 30 s budget blew up reliably under bucket pressure and
|
||||||
logger.info("[%s] Skipping Discord slash command sync: %s", self.name, skip_reason)
|
# left slash commands broken for ~60 min until the bucket fully
|
||||||
return
|
# recovered. Use a wide ceiling; the cap still guards against a
|
||||||
self._record_command_sync_attempt(app_id, fingerprint)
|
# true hang. (#16713)
|
||||||
|
summary = await asyncio.wait_for(self._safe_sync_slash_commands(), timeout=600)
|
||||||
http = getattr(self._client, "http", None)
|
|
||||||
has_ratelimit_timeout = http is not None and hasattr(http, "max_ratelimit_timeout")
|
|
||||||
previous_ratelimit_timeout = getattr(http, "max_ratelimit_timeout", None) if has_ratelimit_timeout else None
|
|
||||||
if has_ratelimit_timeout:
|
|
||||||
http.max_ratelimit_timeout = _DISCORD_COMMAND_SYNC_MAX_RATE_LIMIT_SLEEP_SECONDS
|
|
||||||
|
|
||||||
try:
|
|
||||||
# Discord's per-app command-management bucket is small, and
|
|
||||||
# discord.py can otherwise sit inside one long retry sleep
|
|
||||||
# before surfacing the 429. Keep the whole sync bounded and
|
|
||||||
# persist Discord's retry-after when it refuses the batch.
|
|
||||||
summary = await asyncio.wait_for(self._safe_sync_slash_commands(), timeout=600)
|
|
||||||
except Exception as e:
|
|
||||||
if not self._is_discord_rate_limit(e):
|
|
||||||
raise
|
|
||||||
retry_after = self._extract_discord_retry_after(e)
|
|
||||||
if retry_after is None:
|
|
||||||
# Rate-limited but no retry-after signal — back off for a
|
|
||||||
# conservative default so we don't slam the bucket again.
|
|
||||||
retry_after = _DISCORD_COMMAND_SYNC_MAX_RATE_LIMIT_SLEEP_SECONDS
|
|
||||||
self._record_command_sync_rate_limit(app_id, fingerprint, retry_after)
|
|
||||||
logger.warning(
|
|
||||||
"[%s] Discord rate-limited slash command sync; retrying after %.0fs",
|
|
||||||
self.name,
|
|
||||||
retry_after,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
finally:
|
|
||||||
if has_ratelimit_timeout:
|
|
||||||
http.max_ratelimit_timeout = previous_ratelimit_timeout
|
|
||||||
|
|
||||||
self._record_command_sync_success(app_id, fingerprint, summary)
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"[%s] Safely reconciled %d slash command(s): unchanged=%d updated=%d recreated=%d created=%d deleted=%d",
|
"[%s] Safely reconciled %d slash command(s): unchanged=%d updated=%d recreated=%d created=%d deleted=%d",
|
||||||
self.name,
|
self.name,
|
||||||
@@ -1247,20 +1009,11 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
created = 0
|
created = 0
|
||||||
deleted = 0
|
deleted = 0
|
||||||
http = self._client.http
|
http = self._client.http
|
||||||
mutation_count = 0
|
|
||||||
|
|
||||||
async def mutate(call, *args):
|
|
||||||
nonlocal mutation_count
|
|
||||||
if mutation_count:
|
|
||||||
await self._sleep_between_command_sync_mutations()
|
|
||||||
result = await call(*args)
|
|
||||||
mutation_count += 1
|
|
||||||
return result
|
|
||||||
|
|
||||||
for key, desired in desired_by_key.items():
|
for key, desired in desired_by_key.items():
|
||||||
current = existing_by_key.pop(key, None)
|
current = existing_by_key.pop(key, None)
|
||||||
if current is None:
|
if current is None:
|
||||||
await mutate(http.upsert_global_command, app_id, desired)
|
await http.upsert_global_command(app_id, desired)
|
||||||
created += 1
|
created += 1
|
||||||
continue
|
continue
|
||||||
|
|
||||||
@@ -1272,16 +1025,16 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
continue
|
continue
|
||||||
|
|
||||||
if self._patchable_app_command_payload(current_existing_payload) == self._patchable_app_command_payload(desired):
|
if self._patchable_app_command_payload(current_existing_payload) == self._patchable_app_command_payload(desired):
|
||||||
await mutate(http.delete_global_command, app_id, current.id)
|
await http.delete_global_command(app_id, current.id)
|
||||||
await mutate(http.upsert_global_command, app_id, desired)
|
await http.upsert_global_command(app_id, desired)
|
||||||
recreated += 1
|
recreated += 1
|
||||||
continue
|
continue
|
||||||
|
|
||||||
await mutate(http.edit_global_command, app_id, current.id, desired)
|
await http.edit_global_command(app_id, current.id, desired)
|
||||||
updated += 1
|
updated += 1
|
||||||
|
|
||||||
for current in existing_by_key.values():
|
for current in existing_by_key.values():
|
||||||
await mutate(http.delete_global_command, app_id, current.id)
|
await http.delete_global_command(app_id, current.id)
|
||||||
deleted += 1
|
deleted += 1
|
||||||
|
|
||||||
return {
|
return {
|
||||||
@@ -2101,16 +1854,8 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
completed = receiver.check_silence()
|
completed = receiver.check_silence()
|
||||||
# Voice inputs always originate from a specific guild
|
|
||||||
# (guild_id is in scope). Pass it so role checks are
|
|
||||||
# guild-scoped and not cross-guild.
|
|
||||||
_vc_guild = self._client.get_guild(guild_id) if self._client is not None else None
|
|
||||||
for user_id, pcm_data in completed:
|
for user_id, pcm_data in completed:
|
||||||
if not self._is_allowed_user(
|
if not self._is_allowed_user(str(user_id)):
|
||||||
str(user_id),
|
|
||||||
guild=_vc_guild,
|
|
||||||
is_dm=False,
|
|
||||||
):
|
|
||||||
continue
|
continue
|
||||||
await self._process_voice_input(guild_id, user_id, pcm_data)
|
await self._process_voice_input(guild_id, user_id, pcm_data)
|
||||||
except asyncio.CancelledError:
|
except asyncio.CancelledError:
|
||||||
@@ -2153,32 +1898,13 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
except OSError:
|
except OSError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def _is_allowed_user(
|
def _is_allowed_user(self, user_id: str, author=None) -> bool:
|
||||||
self,
|
|
||||||
user_id: str,
|
|
||||||
author=None,
|
|
||||||
*,
|
|
||||||
guild=None,
|
|
||||||
is_dm: bool = False,
|
|
||||||
) -> bool:
|
|
||||||
"""Check if user is allowed via DISCORD_ALLOWED_USERS or DISCORD_ALLOWED_ROLES.
|
"""Check if user is allowed via DISCORD_ALLOWED_USERS or DISCORD_ALLOWED_ROLES.
|
||||||
|
|
||||||
Uses OR semantics: if the user matches EITHER allowlist, they're allowed.
|
Uses OR semantics: if the user matches EITHER allowlist, they're allowed.
|
||||||
If both allowlists are empty, everyone is allowed (backwards compatible).
|
If both allowlists are empty, everyone is allowed (backwards compatible).
|
||||||
|
When author is a Member, checks .roles directly; otherwise falls back
|
||||||
Role checks are **scoped to the guild the message originated from**.
|
to scanning the bot's mutual guilds for a Member record.
|
||||||
For DMs (no guild context), role-based auth is disabled by default and
|
|
||||||
only user-ID allowlist applies. Set ``discord.dm_role_auth_guild``
|
|
||||||
in config.yaml to a specific guild ID to opt-in: role membership in
|
|
||||||
that one guild will authorize DMs. This prevents cross-guild
|
|
||||||
privilege escalation where a user with the configured role in any
|
|
||||||
shared public server could DM the bot and pass the allowlist.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
user_id: Author ID as a string.
|
|
||||||
author: Optional Member/User object for in-guild role lookup.
|
|
||||||
guild: The guild the message arrived in (None for DMs).
|
|
||||||
is_dm: True if the message came from a DM channel.
|
|
||||||
"""
|
"""
|
||||||
# ``getattr`` fallbacks here guard against test fixtures that build
|
# ``getattr`` fallbacks here guard against test fixtures that build
|
||||||
# an adapter via ``object.__new__(DiscordAdapter)`` and skip __init__
|
# an adapter via ``object.__new__(DiscordAdapter)`` and skip __init__
|
||||||
@@ -2189,54 +1915,31 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
has_roles = bool(allowed_roles)
|
has_roles = bool(allowed_roles)
|
||||||
if not has_users and not has_roles:
|
if not has_users and not has_roles:
|
||||||
return True
|
return True
|
||||||
# Check user ID allowlist (works for both DMs and guild messages)
|
# Check user ID allowlist
|
||||||
if has_users and user_id in allowed_users:
|
if has_users and user_id in allowed_users:
|
||||||
return True
|
return True
|
||||||
# Role allowlist is only consulted when configured.
|
# Check role allowlist
|
||||||
if not has_roles:
|
if has_roles:
|
||||||
return False
|
# Try direct role check from Member object
|
||||||
|
direct_roles = getattr(author, "roles", None) if author is not None else None
|
||||||
# DM path: roles require explicit opt-in via
|
if direct_roles:
|
||||||
# ``discord.dm_role_auth_guild`` in config.yaml. Without this, a
|
if any(getattr(r, "id", None) in allowed_roles for r in direct_roles):
|
||||||
# user with the configured role in ANY mutual guild could DM the
|
return True
|
||||||
# bot and bypass the allowlist (cross-guild leakage).
|
# Fallback: scan mutual guilds for member's roles
|
||||||
if is_dm or guild is None:
|
if self._client is not None:
|
||||||
dm_guild_id = _read_dm_role_auth_guild()
|
try:
|
||||||
if dm_guild_id is None:
|
uid_int = int(user_id)
|
||||||
return False
|
except (TypeError, ValueError):
|
||||||
if self._client is None:
|
uid_int = None
|
||||||
return False
|
if uid_int is not None:
|
||||||
dm_guild = self._client.get_guild(dm_guild_id)
|
for guild in self._client.guilds:
|
||||||
if dm_guild is None:
|
m = guild.get_member(uid_int)
|
||||||
return False
|
if m is None:
|
||||||
try:
|
continue
|
||||||
uid_int = int(user_id)
|
m_roles = getattr(m, "roles", None) or []
|
||||||
except (TypeError, ValueError):
|
if any(getattr(r, "id", None) in allowed_roles for r in m_roles):
|
||||||
return False
|
return True
|
||||||
m = dm_guild.get_member(uid_int)
|
return False
|
||||||
if m is None:
|
|
||||||
return False
|
|
||||||
m_roles = getattr(m, "roles", None) or []
|
|
||||||
return any(getattr(r, "id", None) in allowed_roles for r in m_roles)
|
|
||||||
|
|
||||||
# Guild path: role check is scoped to THIS guild only.
|
|
||||||
# 1) Prefer the direct Member object passed in (correct guild by construction).
|
|
||||||
direct_roles = getattr(author, "roles", None) if author is not None else None
|
|
||||||
author_guild = getattr(author, "guild", None)
|
|
||||||
if direct_roles and (author_guild is None or author_guild.id == guild.id):
|
|
||||||
if any(getattr(r, "id", None) in allowed_roles for r in direct_roles):
|
|
||||||
return True
|
|
||||||
# 2) Fallback: resolve the Member in the message's guild only — NEVER
|
|
||||||
# scan other mutual guilds (that is the cross-guild bypass bug).
|
|
||||||
try:
|
|
||||||
uid_int = int(user_id)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return False
|
|
||||||
m = guild.get_member(uid_int)
|
|
||||||
if m is None:
|
|
||||||
return False
|
|
||||||
m_roles = getattr(m, "roles", None) or []
|
|
||||||
return any(getattr(r, "id", None) in allowed_roles for r in m_roles)
|
|
||||||
|
|
||||||
# ── Slash command authorization ─────────────────────────────────────
|
# ── Slash command authorization ─────────────────────────────────────
|
||||||
# Slash commands (``_run_simple_slash`` and ``_handle_thread_create_slash``)
|
# Slash commands (``_run_simple_slash`` and ``_handle_thread_create_slash``)
|
||||||
@@ -2333,16 +2036,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
|||||||
return (True, None)
|
return (True, None)
|
||||||
|
|
||||||
user_id = str(user.id)
|
user_id = str(user.id)
|
||||||
# Pass guild + is_dm so role check is scoped to the originating
|
if not self._is_allowed_user(user_id, author=user):
|
||||||
# guild and cross-guild DM bypass (#12136) can't land via the
|
|
||||||
# slash surface either.
|
|
||||||
interaction_guild = getattr(interaction, "guild", None)
|
|
||||||
if not self._is_allowed_user(
|
|
||||||
user_id,
|
|
||||||
author=user,
|
|
||||||
guild=interaction_guild,
|
|
||||||
is_dm=in_dm,
|
|
||||||
):
|
|
||||||
return (
|
return (
|
||||||
False,
|
False,
|
||||||
"user not in DISCORD_ALLOWED_USERS / DISCORD_ALLOWED_ROLES",
|
"user not in DISCORD_ALLOWED_USERS / DISCORD_ALLOWED_ROLES",
|
||||||
|
|||||||
@@ -4591,12 +4591,12 @@ def _poll_registration(
|
|||||||
Returns dict with app_id, app_secret, domain, open_id on success.
|
Returns dict with app_id, app_secret, domain, open_id on success.
|
||||||
Returns None on failure.
|
Returns None on failure.
|
||||||
"""
|
"""
|
||||||
deadline = time.monotonic() + expire_in
|
deadline = time.time() + expire_in
|
||||||
current_domain = domain
|
current_domain = domain
|
||||||
domain_switched = False
|
domain_switched = False
|
||||||
poll_count = 0
|
poll_count = 0
|
||||||
|
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
base_url = _accounts_base_url(current_domain)
|
base_url = _accounts_base_url(current_domain)
|
||||||
try:
|
try:
|
||||||
res = _post_registration(base_url, {
|
res = _post_registration(base_url, {
|
||||||
|
|||||||
+12
-87
@@ -17,8 +17,7 @@ Environment variables:
|
|||||||
MATRIX_REACTIONS Set "false" to disable processing lifecycle reactions
|
MATRIX_REACTIONS Set "false" to disable processing lifecycle reactions
|
||||||
(eyes/checkmark/cross). Default: true
|
(eyes/checkmark/cross). Default: true
|
||||||
MATRIX_REQUIRE_MENTION Require @mention in rooms (default: true)
|
MATRIX_REQUIRE_MENTION Require @mention in rooms (default: true)
|
||||||
MATRIX_FREE_RESPONSE_ROOMS Comma-separated room IDs exempt from mention requirement (alias of matrix.free_response_rooms)
|
MATRIX_FREE_RESPONSE_ROOMS Comma-separated room IDs exempt from mention requirement
|
||||||
MATRIX_ALLOWED_ROOMS Comma-separated room IDs; if set, bot ONLY responds in these rooms (whitelist, DMs exempt; alias of matrix.allowed_rooms)
|
|
||||||
MATRIX_AUTO_THREAD Auto-create threads for room messages (default: true)
|
MATRIX_AUTO_THREAD Auto-create threads for room messages (default: true)
|
||||||
MATRIX_DM_AUTO_THREAD Auto-create threads for DM messages (default: false)
|
MATRIX_DM_AUTO_THREAD Auto-create threads for DM messages (default: false)
|
||||||
MATRIX_RECOVERY_KEY Recovery key for cross-signing verification after device key rotation
|
MATRIX_RECOVERY_KEY Recovery key for cross-signing verification after device key rotation
|
||||||
@@ -344,29 +343,10 @@ class MatrixAdapter(BasePlatformAdapter):
|
|||||||
self._require_mention: bool = os.getenv(
|
self._require_mention: bool = os.getenv(
|
||||||
"MATRIX_REQUIRE_MENTION", "true"
|
"MATRIX_REQUIRE_MENTION", "true"
|
||||||
).lower() not in ("false", "0", "no")
|
).lower() not in ("false", "0", "no")
|
||||||
free_rooms_raw = config.extra.get("free_response_rooms")
|
free_rooms_raw = os.getenv("MATRIX_FREE_RESPONSE_ROOMS", "")
|
||||||
if free_rooms_raw is None:
|
self._free_rooms: Set[str] = {
|
||||||
free_rooms_raw = os.getenv("MATRIX_FREE_RESPONSE_ROOMS", "")
|
r.strip() for r in free_rooms_raw.split(",") if r.strip()
|
||||||
if isinstance(free_rooms_raw, list):
|
}
|
||||||
self._free_rooms: Set[str] = {
|
|
||||||
str(r).strip() for r in free_rooms_raw if str(r).strip()
|
|
||||||
}
|
|
||||||
else:
|
|
||||||
self._free_rooms: Set[str] = {
|
|
||||||
r.strip() for r in str(free_rooms_raw).split(",") if r.strip()
|
|
||||||
}
|
|
||||||
# If non-empty, bot ONLY responds in these rooms (whitelist); DMs exempt.
|
|
||||||
allowed_rooms_raw = config.extra.get("allowed_rooms")
|
|
||||||
if allowed_rooms_raw is None:
|
|
||||||
allowed_rooms_raw = os.getenv("MATRIX_ALLOWED_ROOMS", "")
|
|
||||||
if isinstance(allowed_rooms_raw, list):
|
|
||||||
self._allowed_rooms: Set[str] = {
|
|
||||||
str(r).strip() for r in allowed_rooms_raw if str(r).strip()
|
|
||||||
}
|
|
||||||
else:
|
|
||||||
self._allowed_rooms: Set[str] = {
|
|
||||||
r.strip() for r in str(allowed_rooms_raw).split(",") if r.strip()
|
|
||||||
}
|
|
||||||
self._auto_thread: bool = os.getenv("MATRIX_AUTO_THREAD", "true").lower() in (
|
self._auto_thread: bool = os.getenv("MATRIX_AUTO_THREAD", "true").lower() in (
|
||||||
"true",
|
"true",
|
||||||
"1",
|
"1",
|
||||||
@@ -384,12 +364,6 @@ class MatrixAdapter(BasePlatformAdapter):
|
|||||||
"MATRIX_REACTIONS", "true"
|
"MATRIX_REACTIONS", "true"
|
||||||
).lower() not in ("false", "0", "no")
|
).lower() not in ("false", "0", "no")
|
||||||
self._pending_reactions: dict[tuple[str, str], str] = {}
|
self._pending_reactions: dict[tuple[str, str], str] = {}
|
||||||
# Delay before redacting reactions so Matrix homeservers have time to
|
|
||||||
# deliver the final message event without tripping "missing event"
|
|
||||||
# errors in some clients. 5s is empirically safe; not user-tunable —
|
|
||||||
# if that changes, add a config.yaml entry rather than an env var.
|
|
||||||
self._reaction_redaction_delay_seconds = 5.0
|
|
||||||
self._reaction_redaction_tasks: Set[asyncio.Task] = set()
|
|
||||||
|
|
||||||
# Proxy support — resolve once at init, reuse for all HTTP traffic.
|
# Proxy support — resolve once at init, reuse for all HTTP traffic.
|
||||||
self._proxy_url: str | None = resolve_proxy_url(platform_env_var="MATRIX_PROXY")
|
self._proxy_url: str | None = resolve_proxy_url(platform_env_var="MATRIX_PROXY")
|
||||||
@@ -877,14 +851,6 @@ class MatrixAdapter(BasePlatformAdapter):
|
|||||||
except (asyncio.CancelledError, Exception):
|
except (asyncio.CancelledError, Exception):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
redaction_tasks = list(self._reaction_redaction_tasks)
|
|
||||||
for task in redaction_tasks:
|
|
||||||
if not task.done():
|
|
||||||
task.cancel()
|
|
||||||
if redaction_tasks:
|
|
||||||
await asyncio.gather(*redaction_tasks, return_exceptions=True)
|
|
||||||
self._reaction_redaction_tasks.clear()
|
|
||||||
|
|
||||||
# Close the SQLite crypto store database.
|
# Close the SQLite crypto store database.
|
||||||
if hasattr(self, "_crypto_db") and self._crypto_db:
|
if hasattr(self, "_crypto_db") and self._crypto_db:
|
||||||
try:
|
try:
|
||||||
@@ -1593,18 +1559,6 @@ class MatrixAdapter(BasePlatformAdapter):
|
|||||||
|
|
||||||
# Require-mention gating.
|
# Require-mention gating.
|
||||||
if not is_dm:
|
if not is_dm:
|
||||||
# allowed_rooms check (whitelist — must pass before other gating).
|
|
||||||
# When set, messages from rooms NOT in this whitelist are silently
|
|
||||||
# ignored, even if @mentioned. DMs are already excluded above.
|
|
||||||
if self._allowed_rooms and room_id not in self._allowed_rooms:
|
|
||||||
logger.debug(
|
|
||||||
"Matrix: ignoring message %s in %s — room not in "
|
|
||||||
"MATRIX_ALLOWED_ROOMS whitelist",
|
|
||||||
event_id,
|
|
||||||
room_id,
|
|
||||||
)
|
|
||||||
return None
|
|
||||||
|
|
||||||
is_free_room = room_id in self._free_rooms
|
is_free_room = room_id in self._free_rooms
|
||||||
in_bot_thread = bool(thread_id and thread_id in self._threads)
|
in_bot_thread = bool(thread_id and thread_id in self._threads)
|
||||||
if self._require_mention and not is_free_room and not in_bot_thread:
|
if self._require_mention and not is_free_room and not in_bot_thread:
|
||||||
@@ -1975,35 +1929,6 @@ class MatrixAdapter(BasePlatformAdapter):
|
|||||||
"""Remove a reaction by redacting its event."""
|
"""Remove a reaction by redacting its event."""
|
||||||
return await self.redact_message(room_id, reaction_event_id, reason)
|
return await self.redact_message(room_id, reaction_event_id, reason)
|
||||||
|
|
||||||
def _schedule_reaction_redaction(
|
|
||||||
self,
|
|
||||||
room_id: str,
|
|
||||||
reaction_event_id: str,
|
|
||||||
reason: str = "",
|
|
||||||
) -> None:
|
|
||||||
"""Redact a reaction after a short delay so message delivery settles."""
|
|
||||||
|
|
||||||
async def _redact_later() -> None:
|
|
||||||
try:
|
|
||||||
if self._reaction_redaction_delay_seconds:
|
|
||||||
await asyncio.sleep(self._reaction_redaction_delay_seconds)
|
|
||||||
if not await self._redact_reaction(room_id, reaction_event_id, reason):
|
|
||||||
logger.debug(
|
|
||||||
"Matrix: failed to redact reaction %s", reaction_event_id
|
|
||||||
)
|
|
||||||
except asyncio.CancelledError:
|
|
||||||
raise
|
|
||||||
except Exception as exc:
|
|
||||||
logger.debug(
|
|
||||||
"Matrix: delayed reaction redaction failed for %s: %s",
|
|
||||||
reaction_event_id,
|
|
||||||
exc,
|
|
||||||
)
|
|
||||||
|
|
||||||
task = asyncio.create_task(_redact_later())
|
|
||||||
self._reaction_redaction_tasks.add(task)
|
|
||||||
task.add_done_callback(self._reaction_redaction_tasks.discard)
|
|
||||||
|
|
||||||
async def on_processing_start(self, event: MessageEvent) -> None:
|
async def on_processing_start(self, event: MessageEvent) -> None:
|
||||||
"""Add eyes reaction when the agent starts processing a message."""
|
"""Add eyes reaction when the agent starts processing a message."""
|
||||||
if not self._reactions_enabled:
|
if not self._reactions_enabled:
|
||||||
@@ -2032,11 +1957,8 @@ class MatrixAdapter(BasePlatformAdapter):
|
|||||||
reaction_key = (room_id, msg_id)
|
reaction_key = (room_id, msg_id)
|
||||||
if reaction_key in self._pending_reactions:
|
if reaction_key in self._pending_reactions:
|
||||||
eyes_event_id = self._pending_reactions.pop(reaction_key)
|
eyes_event_id = self._pending_reactions.pop(reaction_key)
|
||||||
self._schedule_reaction_redaction(
|
if not await self._redact_reaction(room_id, eyes_event_id):
|
||||||
room_id,
|
logger.debug("Matrix: failed to redact eyes reaction %s", eyes_event_id)
|
||||||
eyes_event_id,
|
|
||||||
"processing complete",
|
|
||||||
)
|
|
||||||
await self._send_reaction(
|
await self._send_reaction(
|
||||||
room_id,
|
room_id,
|
||||||
msg_id,
|
msg_id,
|
||||||
@@ -2115,8 +2037,11 @@ class MatrixAdapter(BasePlatformAdapter):
|
|||||||
) -> None:
|
) -> None:
|
||||||
"""Redact the bot's seed ✅/❎ reactions, leaving only the user's reaction."""
|
"""Redact the bot's seed ✅/❎ reactions, leaving only the user's reaction."""
|
||||||
for emoji, evt_id in prompt.bot_reaction_events.items():
|
for emoji, evt_id in prompt.bot_reaction_events.items():
|
||||||
self._schedule_reaction_redaction(room_id, evt_id, "approval resolved")
|
try:
|
||||||
logger.debug("Matrix: scheduled bot reaction redaction %s (%s)", emoji, evt_id)
|
await self.redact_message(room_id, evt_id, "approval resolved")
|
||||||
|
logger.debug("Matrix: redacted bot reaction %s (%s)", emoji, evt_id)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.debug("Matrix: failed to redact bot reaction %s: %s", emoji, exc)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Text message aggregation (handles Matrix client-side splits)
|
# Text message aggregation (handles Matrix client-side splits)
|
||||||
|
|||||||
@@ -706,30 +706,10 @@ class MattermostAdapter(BasePlatformAdapter):
|
|||||||
message_text = post.get("message", "")
|
message_text = post.get("message", "")
|
||||||
|
|
||||||
# Mention-gating for non-DM channels.
|
# Mention-gating for non-DM channels.
|
||||||
# Config (config.yaml `mattermost.*` with env-var fallback):
|
# Config (env vars):
|
||||||
# require_mention / MATTERMOST_REQUIRE_MENTION: Require @mention in channels (default: true)
|
# MATTERMOST_REQUIRE_MENTION: Require @mention in channels (default: true)
|
||||||
# free_response_channels / MATTERMOST_FREE_RESPONSE_CHANNELS: Channel IDs where bot responds without mention
|
# MATTERMOST_FREE_RESPONSE_CHANNELS: Channel IDs where bot responds without mention
|
||||||
# allowed_channels / MATTERMOST_ALLOWED_CHANNELS: If set, bot ONLY responds in these channels (whitelist)
|
|
||||||
if channel_type_raw != "D":
|
if channel_type_raw != "D":
|
||||||
# allowed_channels check (whitelist — must pass before other gating).
|
|
||||||
# When set, messages from channels NOT in this list are silently
|
|
||||||
# ignored, even if @mentioned. DMs are already excluded above.
|
|
||||||
allowed_raw = self.config.extra.get("allowed_channels") if self.config.extra else None
|
|
||||||
if allowed_raw is None:
|
|
||||||
allowed_raw = os.getenv("MATTERMOST_ALLOWED_CHANNELS", "")
|
|
||||||
if isinstance(allowed_raw, list):
|
|
||||||
allowed_channels = {str(c).strip() for c in allowed_raw if str(c).strip()}
|
|
||||||
else:
|
|
||||||
allowed_channels = {
|
|
||||||
c.strip() for c in str(allowed_raw).split(",") if c.strip()
|
|
||||||
}
|
|
||||||
if allowed_channels and channel_id not in allowed_channels:
|
|
||||||
logger.debug(
|
|
||||||
"Mattermost: ignoring message in non-allowed channel: %s",
|
|
||||||
channel_id,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
require_mention = os.getenv(
|
require_mention = os.getenv(
|
||||||
"MATTERMOST_REQUIRE_MENTION", "true"
|
"MATTERMOST_REQUIRE_MENTION", "true"
|
||||||
).lower() not in ("false", "0", "no")
|
).lower() not in ("false", "0", "no")
|
||||||
|
|||||||
@@ -34,27 +34,6 @@ from .crypto import decrypt_secret, generate_bind_key # noqa: F401
|
|||||||
# -- Utils -----------------------------------------------------------------
|
# -- Utils -----------------------------------------------------------------
|
||||||
from .utils import build_user_agent, get_api_headers, coerce_list # noqa: F401
|
from .utils import build_user_agent, get_api_headers, coerce_list # noqa: F401
|
||||||
|
|
||||||
# -- Chunked upload --------------------------------------------------------
|
|
||||||
from .chunked_upload import ( # noqa: F401
|
|
||||||
ChunkedUploader,
|
|
||||||
UploadDailyLimitExceededError,
|
|
||||||
UploadFileTooLargeError,
|
|
||||||
)
|
|
||||||
|
|
||||||
# -- Inline keyboards ------------------------------------------------------
|
|
||||||
from .keyboards import ( # noqa: F401
|
|
||||||
ApprovalRequest,
|
|
||||||
ApprovalSender,
|
|
||||||
InlineKeyboard,
|
|
||||||
InteractionEvent,
|
|
||||||
build_approval_keyboard,
|
|
||||||
build_approval_text,
|
|
||||||
build_update_prompt_keyboard,
|
|
||||||
parse_approval_button_data,
|
|
||||||
parse_interaction_event,
|
|
||||||
parse_update_prompt_button_data,
|
|
||||||
)
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
# adapter
|
# adapter
|
||||||
"QQAdapter",
|
"QQAdapter",
|
||||||
@@ -73,19 +52,4 @@ __all__ = [
|
|||||||
"build_user_agent",
|
"build_user_agent",
|
||||||
"get_api_headers",
|
"get_api_headers",
|
||||||
"coerce_list",
|
"coerce_list",
|
||||||
# chunked upload
|
|
||||||
"ChunkedUploader",
|
|
||||||
"UploadDailyLimitExceededError",
|
|
||||||
"UploadFileTooLargeError",
|
|
||||||
# keyboards
|
|
||||||
"ApprovalRequest",
|
|
||||||
"ApprovalSender",
|
|
||||||
"InlineKeyboard",
|
|
||||||
"InteractionEvent",
|
|
||||||
"build_approval_keyboard",
|
|
||||||
"build_approval_text",
|
|
||||||
"build_update_prompt_keyboard",
|
|
||||||
"parse_approval_button_data",
|
|
||||||
"parse_interaction_event",
|
|
||||||
"parse_update_prompt_button_data",
|
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -41,7 +41,7 @@ import time
|
|||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Awaitable, Callable, Dict, List, Optional, Tuple
|
from typing import Any, Dict, List, Optional, Tuple
|
||||||
from urllib.parse import urlparse
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -119,22 +119,6 @@ from gateway.platforms.qqbot.utils import (
|
|||||||
coerce_list as _coerce_list_impl,
|
coerce_list as _coerce_list_impl,
|
||||||
build_user_agent,
|
build_user_agent,
|
||||||
)
|
)
|
||||||
from gateway.platforms.qqbot.chunked_upload import (
|
|
||||||
ChunkedUploader,
|
|
||||||
UploadDailyLimitExceededError,
|
|
||||||
UploadFileTooLargeError,
|
|
||||||
)
|
|
||||||
from gateway.platforms.qqbot.keyboards import (
|
|
||||||
ApprovalRequest,
|
|
||||||
ApprovalSender,
|
|
||||||
InlineKeyboard,
|
|
||||||
InteractionEvent,
|
|
||||||
build_approval_keyboard,
|
|
||||||
build_update_prompt_keyboard,
|
|
||||||
parse_approval_button_data,
|
|
||||||
parse_interaction_event,
|
|
||||||
parse_update_prompt_button_data,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def check_qq_requirements() -> bool:
|
def check_qq_requirements() -> bool:
|
||||||
@@ -224,22 +208,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
# Upload cache: content_hash -> {file_info, file_uuid, expires_at}
|
# Upload cache: content_hash -> {file_info, file_uuid, expires_at}
|
||||||
self._upload_cache: Dict[str, Dict[str, Any]] = {}
|
self._upload_cache: Dict[str, Dict[str, Any]] = {}
|
||||||
|
|
||||||
# Inline-keyboard interaction routing. The callback (if set) is invoked
|
|
||||||
# for every INTERACTION_CREATE event after the adapter has already
|
|
||||||
# ACKed it. Callers (gateway wiring for approvals / update prompts)
|
|
||||||
# register via set_interaction_callback().
|
|
||||||
self._interaction_callback: Optional[
|
|
||||||
Callable[[InteractionEvent], Awaitable[None]]
|
|
||||||
] = None
|
|
||||||
|
|
||||||
# Default interaction dispatcher: routes approval-button clicks to
|
|
||||||
# tools.approval.resolve_gateway_approval() and update-prompt clicks
|
|
||||||
# to ~/.hermes/.update_response. Set here so the cross-adapter gateway
|
|
||||||
# contract (send_exec_approval / send_update_prompt) works out of the
|
|
||||||
# box; callers can override with set_interaction_callback(None) or
|
|
||||||
# register a custom handler.
|
|
||||||
self._interaction_callback = self._default_interaction_dispatch
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Properties
|
# Properties
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -791,8 +759,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
"GUILD_AT_MESSAGE_CREATE",
|
"GUILD_AT_MESSAGE_CREATE",
|
||||||
):
|
):
|
||||||
asyncio.create_task(self._on_message(t, d))
|
asyncio.create_task(self._on_message(t, d))
|
||||||
elif t == "INTERACTION_CREATE":
|
|
||||||
self._create_task(self._on_interaction(d))
|
|
||||||
else:
|
else:
|
||||||
logger.debug("[%s] Unhandled dispatch: %s", self._log_tag, t)
|
logger.debug("[%s] Unhandled dispatch: %s", self._log_tag, t)
|
||||||
return
|
return
|
||||||
@@ -866,206 +832,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
elif event_type == "DIRECT_MESSAGE_CREATE":
|
elif event_type == "DIRECT_MESSAGE_CREATE":
|
||||||
await self._handle_dm_message(d, msg_id, content, author, timestamp)
|
await self._handle_dm_message(d, msg_id, content, author, timestamp)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
# Inline-keyboard interactions (INTERACTION_CREATE)
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
|
|
||||||
def set_interaction_callback(
|
|
||||||
self,
|
|
||||||
callback: Optional[Callable[[InteractionEvent], Awaitable[None]]],
|
|
||||||
) -> None:
|
|
||||||
"""Register (or clear) the interaction callback.
|
|
||||||
|
|
||||||
Invoked once per ``INTERACTION_CREATE`` event *after* the adapter has
|
|
||||||
ACKed the interaction. The callback is responsible for routing the
|
|
||||||
button click to the right subsystem (approval resolver, update-prompt
|
|
||||||
resolver, etc.) based on the ``button_data`` payload.
|
|
||||||
"""
|
|
||||||
self._interaction_callback = callback
|
|
||||||
|
|
||||||
async def _on_interaction(self, d: Any) -> None:
|
|
||||||
"""Handle an ``INTERACTION_CREATE`` event.
|
|
||||||
|
|
||||||
Responsibilities:
|
|
||||||
|
|
||||||
1. Parse the raw payload into an :class:`InteractionEvent`.
|
|
||||||
2. ACK the interaction (``PUT /interactions/{id}``) so the client
|
|
||||||
stops showing a loading indicator on the button.
|
|
||||||
3. Dispatch to the registered interaction callback, if any.
|
|
||||||
"""
|
|
||||||
if not isinstance(d, dict):
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
event = parse_interaction_event(d)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.warning(
|
|
||||||
"[%s] Failed to parse INTERACTION_CREATE: %s", self._log_tag, exc
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
if not event.id:
|
|
||||||
logger.warning(
|
|
||||||
"[%s] INTERACTION_CREATE missing id, skipping ACK", self._log_tag
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
# ACK the interaction promptly — per the QQ docs the client will show
|
|
||||||
# an error icon on the button if we don't respond quickly.
|
|
||||||
try:
|
|
||||||
await self._acknowledge_interaction(event.id)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.warning(
|
|
||||||
"[%s] Failed to ACK interaction %s: %s",
|
|
||||||
self._log_tag, event.id, exc,
|
|
||||||
)
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"[%s] Interaction: scene=%s button_data=%r operator=%s",
|
|
||||||
self._log_tag, event.scene, event.button_data, event.operator_openid,
|
|
||||||
)
|
|
||||||
|
|
||||||
callback = self._interaction_callback
|
|
||||||
if callback is None:
|
|
||||||
logger.debug(
|
|
||||||
"[%s] No interaction callback registered; dropping button "
|
|
||||||
"click %r",
|
|
||||||
self._log_tag, event.button_data,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
await callback(event)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.error(
|
|
||||||
"[%s] Interaction callback raised: %s",
|
|
||||||
self._log_tag, exc, exc_info=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
async def _acknowledge_interaction(
|
|
||||||
self,
|
|
||||||
interaction_id: str,
|
|
||||||
code: int = 0,
|
|
||||||
) -> None:
|
|
||||||
"""ACK a button interaction via ``PUT /interactions/{id}``.
|
|
||||||
|
|
||||||
:param interaction_id: The ``id`` field from the
|
|
||||||
``INTERACTION_CREATE`` event.
|
|
||||||
:param code: Response code (``0`` = success).
|
|
||||||
"""
|
|
||||||
if not self._http_client:
|
|
||||||
raise RuntimeError("HTTP client not initialized — not connected?")
|
|
||||||
token = await self._ensure_token()
|
|
||||||
headers = {
|
|
||||||
"Authorization": f"QQBot {token}",
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
"User-Agent": build_user_agent(),
|
|
||||||
}
|
|
||||||
resp = await self._http_client.put(
|
|
||||||
f"{API_BASE}/interactions/{interaction_id}",
|
|
||||||
headers=headers,
|
|
||||||
json={"code": code},
|
|
||||||
timeout=DEFAULT_API_TIMEOUT,
|
|
||||||
)
|
|
||||||
if resp.status_code >= 400:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"Interaction ACK failed [{resp.status_code}]: "
|
|
||||||
f"{resp.text[:200]}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Mapping from QQ keyboard button decisions → the ``choice`` vocabulary
|
|
||||||
# accepted by ``tools.approval.resolve_gateway_approval``. QQ's 3-button
|
|
||||||
# layout (mobile-space constraint) collapses "session" and "always" into
|
|
||||||
# a single "always" button; users wanting session-only approval can fall
|
|
||||||
# back to the ``/approve session`` text command.
|
|
||||||
_APPROVAL_BUTTON_TO_CHOICE = {
|
|
||||||
"allow-once": "once",
|
|
||||||
"allow-always": "always",
|
|
||||||
"deny": "deny",
|
|
||||||
}
|
|
||||||
|
|
||||||
async def _default_interaction_dispatch(
|
|
||||||
self,
|
|
||||||
event: InteractionEvent,
|
|
||||||
) -> None:
|
|
||||||
"""Route ``INTERACTION_CREATE`` button clicks to the right subsystem.
|
|
||||||
|
|
||||||
- ``approve:<session_key>:<decision>`` →
|
|
||||||
:func:`tools.approval.resolve_gateway_approval`
|
|
||||||
(unblocks the agent thread waiting on a dangerous-command approval).
|
|
||||||
- ``update_prompt:<answer>`` →
|
|
||||||
writes the answer to ``~/.hermes/.update_response`` for the
|
|
||||||
detached ``hermes update --gateway`` process to consume.
|
|
||||||
- Anything else is logged at DEBUG and ignored.
|
|
||||||
|
|
||||||
Installed as the adapter's default interaction callback in
|
|
||||||
``__init__``. Callers can replace via
|
|
||||||
:meth:`set_interaction_callback` to route clicks elsewhere (or pass
|
|
||||||
``None`` to drop them entirely).
|
|
||||||
"""
|
|
||||||
button_data = event.button_data
|
|
||||||
if not button_data:
|
|
||||||
return
|
|
||||||
|
|
||||||
approval = parse_approval_button_data(button_data)
|
|
||||||
if approval is not None:
|
|
||||||
session_key, decision = approval
|
|
||||||
choice = self._APPROVAL_BUTTON_TO_CHOICE.get(decision)
|
|
||||||
if choice is None:
|
|
||||||
logger.warning(
|
|
||||||
"[%s] Unknown approval decision %r (session=%s)",
|
|
||||||
self._log_tag, decision, session_key,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
# Import lazily to keep the adapter importable in tests that
|
|
||||||
# don't exercise the approval subsystem.
|
|
||||||
from tools.approval import resolve_gateway_approval
|
|
||||||
count = resolve_gateway_approval(session_key, choice)
|
|
||||||
logger.info(
|
|
||||||
"[%s] Button resolved %d approval(s) for session %s "
|
|
||||||
"(choice=%s, operator=%s)",
|
|
||||||
self._log_tag, count, session_key, choice,
|
|
||||||
event.operator_openid,
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.error(
|
|
||||||
"[%s] resolve_gateway_approval failed for session %s: %s",
|
|
||||||
self._log_tag, session_key, exc,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
update_answer = parse_update_prompt_button_data(button_data)
|
|
||||||
if update_answer is not None:
|
|
||||||
self._write_update_response(update_answer, event.operator_openid)
|
|
||||||
return
|
|
||||||
|
|
||||||
logger.debug(
|
|
||||||
"[%s] Unrecognised button_data %r from interaction %s",
|
|
||||||
self._log_tag, button_data, event.id,
|
|
||||||
)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _write_update_response(answer: str, operator: str = "") -> None:
|
|
||||||
"""Atomically write the update-prompt answer to ``.update_response``.
|
|
||||||
|
|
||||||
Mirrors the Discord / Telegram / Feishu adapters: the detached
|
|
||||||
``hermes update --gateway`` watcher polls this file for a ``y``/``n``
|
|
||||||
response to its interactive prompts (stash-restore, config migration).
|
|
||||||
Writes via ``tmp + rename`` so a partial write can't fool the reader.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
from hermes_constants import get_hermes_home
|
|
||||||
home = get_hermes_home()
|
|
||||||
response_path = home / ".update_response"
|
|
||||||
tmp = response_path.with_suffix(".tmp")
|
|
||||||
tmp.write_text(answer)
|
|
||||||
tmp.replace(response_path)
|
|
||||||
logger.info(
|
|
||||||
"QQ update prompt answered %r by %s",
|
|
||||||
answer, operator or "(unknown)",
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.error("Failed to write update response: %s", exc)
|
|
||||||
|
|
||||||
async def _handle_c2c_message(
|
async def _handle_c2c_message(
|
||||||
self,
|
self,
|
||||||
d: Dict[str, Any],
|
d: Dict[str, Any],
|
||||||
@@ -1134,13 +900,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
len(voice_transcripts),
|
len(voice_transcripts),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Merge any quoted-message context (message_type=103 → msg_elements[0]).
|
|
||||||
quoted = await self._process_quoted_context(d)
|
|
||||||
text = self._merge_quote_into(text, quoted["quote_block"])
|
|
||||||
if quoted["image_urls"]:
|
|
||||||
image_urls = image_urls + quoted["image_urls"]
|
|
||||||
image_media_types = image_media_types + quoted["image_media_types"]
|
|
||||||
|
|
||||||
if not text.strip() and not image_urls:
|
if not text.strip() and not image_urls:
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -1199,13 +958,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
else attachment_info
|
else attachment_info
|
||||||
)
|
)
|
||||||
|
|
||||||
# Merge any quoted-message context (message_type=103 → msg_elements[0]).
|
|
||||||
quoted = await self._process_quoted_context(d)
|
|
||||||
text = self._merge_quote_into(text, quoted["quote_block"])
|
|
||||||
if quoted["image_urls"]:
|
|
||||||
image_urls = image_urls + quoted["image_urls"]
|
|
||||||
image_media_types = image_media_types + quoted["image_media_types"]
|
|
||||||
|
|
||||||
if not text.strip() and not image_urls:
|
if not text.strip() and not image_urls:
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -1273,13 +1025,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
else attachment_info
|
else attachment_info
|
||||||
)
|
)
|
||||||
|
|
||||||
# Merge any quoted-message context (message_type=103 → msg_elements[0]).
|
|
||||||
quoted = await self._process_quoted_context(d)
|
|
||||||
text = self._merge_quote_into(text, quoted["quote_block"])
|
|
||||||
if quoted["image_urls"]:
|
|
||||||
image_urls = image_urls + quoted["image_urls"]
|
|
||||||
image_media_types = image_media_types + quoted["image_media_types"]
|
|
||||||
|
|
||||||
if not text.strip() and not image_urls:
|
if not text.strip() and not image_urls:
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -1344,13 +1089,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
else attachment_info
|
else attachment_info
|
||||||
)
|
)
|
||||||
|
|
||||||
# Merge any quoted-message context (message_type=103 → msg_elements[0]).
|
|
||||||
quoted = await self._process_quoted_context(d)
|
|
||||||
text = self._merge_quote_into(text, quoted["quote_block"])
|
|
||||||
if quoted["image_urls"]:
|
|
||||||
image_urls = image_urls + quoted["image_urls"]
|
|
||||||
image_media_types = image_media_types + quoted["image_media_types"]
|
|
||||||
|
|
||||||
if not text.strip() and not image_urls:
|
if not text.strip() and not image_urls:
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -1371,113 +1109,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
)
|
)
|
||||||
await self.handle_message(event)
|
await self.handle_message(event)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
# Quoted-message handling
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
|
|
||||||
async def _process_quoted_context(
|
|
||||||
self,
|
|
||||||
d: Dict[str, Any],
|
|
||||||
) -> Dict[str, Any]:
|
|
||||||
"""Process the quoted message a user is replying to.
|
|
||||||
|
|
||||||
When a user replies while quoting another message, the platform sets
|
|
||||||
``message_type = 103`` and pushes the referenced message's content and
|
|
||||||
attachments inside ``msg_elements[0]``. The old adapter ignored
|
|
||||||
``msg_elements`` entirely, so:
|
|
||||||
|
|
||||||
- Quoted text was surfaced only when the user typed something of
|
|
||||||
their own — bare quote-replies showed nothing.
|
|
||||||
- Quoted attachments (images, voice, files) were never downloaded
|
|
||||||
or described.
|
|
||||||
- Quoted voice messages specifically produced no transcript, so the
|
|
||||||
LLM had no way to see what the user was referring to.
|
|
||||||
|
|
||||||
This method parses ``msg_elements`` and runs the quoted attachments
|
|
||||||
through the same :meth:`_process_attachments` pipeline as the main
|
|
||||||
message body, so quoted voice messages get STT transcripts and
|
|
||||||
quoted images are cached identically.
|
|
||||||
|
|
||||||
:param d: Raw inbound message dict (from the WS dispatch payload).
|
|
||||||
:returns: Dict with keys:
|
|
||||||
|
|
||||||
- ``quote_block``: string to prepend to the user's text body
|
|
||||||
(empty when there's nothing quoted).
|
|
||||||
- ``image_urls``: list of cached quoted-image paths.
|
|
||||||
- ``image_media_types``: parallel list of image MIME types.
|
|
||||||
"""
|
|
||||||
empty = {
|
|
||||||
"quote_block": "",
|
|
||||||
"image_urls": [],
|
|
||||||
"image_media_types": [],
|
|
||||||
}
|
|
||||||
# Short-circuit: only message_type 103 indicates a quote.
|
|
||||||
try:
|
|
||||||
if int(d.get("message_type", 0) or 0) != 103:
|
|
||||||
return empty
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return empty
|
|
||||||
|
|
||||||
elements = d.get("msg_elements")
|
|
||||||
if not isinstance(elements, list) or not elements:
|
|
||||||
return empty
|
|
||||||
|
|
||||||
# msg_elements[0] carries the referenced message. Additional elements
|
|
||||||
# (if any) are very rare in practice; we concatenate their text and
|
|
||||||
# union their attachments for completeness.
|
|
||||||
quoted_text_parts: List[str] = []
|
|
||||||
all_attachments: List[Dict[str, Any]] = []
|
|
||||||
for elem in elements:
|
|
||||||
if not isinstance(elem, dict):
|
|
||||||
continue
|
|
||||||
etext = str(elem.get("content", "")).strip()
|
|
||||||
if etext:
|
|
||||||
quoted_text_parts.append(etext)
|
|
||||||
eatts = elem.get("attachments")
|
|
||||||
if isinstance(eatts, list):
|
|
||||||
for a in eatts:
|
|
||||||
if isinstance(a, dict):
|
|
||||||
all_attachments.append(a)
|
|
||||||
|
|
||||||
att_result = await self._process_attachments(all_attachments)
|
|
||||||
quoted_voice = att_result.get("voice_transcripts") or []
|
|
||||||
quoted_info = att_result.get("attachment_info") or ""
|
|
||||||
quoted_images = att_result.get("image_urls") or []
|
|
||||||
quoted_image_types = att_result.get("image_media_types") or []
|
|
||||||
|
|
||||||
lines: List[str] = []
|
|
||||||
if quoted_text_parts:
|
|
||||||
lines.append(" ".join(quoted_text_parts))
|
|
||||||
for t in quoted_voice:
|
|
||||||
lines.append(t)
|
|
||||||
if quoted_info:
|
|
||||||
lines.append(quoted_info)
|
|
||||||
|
|
||||||
if not lines and not quoted_images:
|
|
||||||
return empty
|
|
||||||
|
|
||||||
if lines:
|
|
||||||
quote_block = "[Quoted message]:\n" + "\n".join(lines)
|
|
||||||
else:
|
|
||||||
# Images-only quote: give the LLM at least a marker so it knows
|
|
||||||
# context was referenced.
|
|
||||||
quote_block = "[Quoted message]: (image)"
|
|
||||||
|
|
||||||
return {
|
|
||||||
"quote_block": quote_block,
|
|
||||||
"image_urls": quoted_images,
|
|
||||||
"image_media_types": quoted_image_types,
|
|
||||||
}
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _merge_quote_into(text: str, quote_block: str) -> str:
|
|
||||||
"""Prepend ``quote_block`` to *text*, separated by a blank line."""
|
|
||||||
if not quote_block:
|
|
||||||
return text
|
|
||||||
if text.strip():
|
|
||||||
return f"{quote_block}\n\n{text}".strip()
|
|
||||||
return quote_block
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Attachment processing
|
# Attachment processing
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -2361,44 +1992,26 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
return SendResult(success=False, error=error_msg, retryable=retryable)
|
return SendResult(success=False, error=error_msg, retryable=retryable)
|
||||||
|
|
||||||
async def _send_c2c_text(
|
async def _send_c2c_text(
|
||||||
self,
|
self, openid: str, content: str, reply_to: Optional[str] = None
|
||||||
openid: str,
|
|
||||||
content: str,
|
|
||||||
reply_to: Optional[str] = None,
|
|
||||||
keyboard: Optional[InlineKeyboard] = None,
|
|
||||||
) -> SendResult:
|
) -> SendResult:
|
||||||
"""Send text to a C2C user via REST API.
|
"""Send text to a C2C user via REST API."""
|
||||||
|
|
||||||
:param keyboard: Optional inline keyboard attached to the message.
|
|
||||||
"""
|
|
||||||
self._next_msg_seq(reply_to or openid)
|
self._next_msg_seq(reply_to or openid)
|
||||||
body = self._build_text_body(content, reply_to)
|
body = self._build_text_body(content, reply_to)
|
||||||
if reply_to:
|
if reply_to:
|
||||||
body["msg_id"] = reply_to
|
body["msg_id"] = reply_to
|
||||||
if keyboard is not None:
|
|
||||||
body["keyboard"] = keyboard.to_dict()
|
|
||||||
|
|
||||||
data = await self._api_request("POST", f"/v2/users/{openid}/messages", body)
|
data = await self._api_request("POST", f"/v2/users/{openid}/messages", body)
|
||||||
msg_id = str(data.get("id", uuid.uuid4().hex[:12]))
|
msg_id = str(data.get("id", uuid.uuid4().hex[:12]))
|
||||||
return SendResult(success=True, message_id=msg_id, raw_response=data)
|
return SendResult(success=True, message_id=msg_id, raw_response=data)
|
||||||
|
|
||||||
async def _send_group_text(
|
async def _send_group_text(
|
||||||
self,
|
self, group_openid: str, content: str, reply_to: Optional[str] = None
|
||||||
group_openid: str,
|
|
||||||
content: str,
|
|
||||||
reply_to: Optional[str] = None,
|
|
||||||
keyboard: Optional[InlineKeyboard] = None,
|
|
||||||
) -> SendResult:
|
) -> SendResult:
|
||||||
"""Send text to a group via REST API.
|
"""Send text to a group via REST API."""
|
||||||
|
|
||||||
:param keyboard: Optional inline keyboard attached to the message.
|
|
||||||
"""
|
|
||||||
self._next_msg_seq(reply_to or group_openid)
|
self._next_msg_seq(reply_to or group_openid)
|
||||||
body = self._build_text_body(content, reply_to)
|
body = self._build_text_body(content, reply_to)
|
||||||
if reply_to:
|
if reply_to:
|
||||||
body["msg_id"] = reply_to
|
body["msg_id"] = reply_to
|
||||||
if keyboard is not None:
|
|
||||||
body["keyboard"] = keyboard.to_dict()
|
|
||||||
|
|
||||||
data = await self._api_request(
|
data = await self._api_request(
|
||||||
"POST", f"/v2/groups/{group_openid}/messages", body
|
"POST", f"/v2/groups/{group_openid}/messages", body
|
||||||
@@ -2418,156 +2031,6 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
msg_id = str(data.get("id", uuid.uuid4().hex[:12]))
|
msg_id = str(data.get("id", uuid.uuid4().hex[:12]))
|
||||||
return SendResult(success=True, message_id=msg_id, raw_response=data)
|
return SendResult(success=True, message_id=msg_id, raw_response=data)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
# Inline-keyboard outbound helpers (approval / update-prompt flows)
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
|
|
||||||
async def send_with_keyboard(
|
|
||||||
self,
|
|
||||||
chat_id: str,
|
|
||||||
content: str,
|
|
||||||
keyboard: InlineKeyboard,
|
|
||||||
reply_to: Optional[str] = None,
|
|
||||||
) -> SendResult:
|
|
||||||
"""Send a single text message with an inline keyboard attached.
|
|
||||||
|
|
||||||
Unlike :meth:`send`, this does NOT split long content into chunks —
|
|
||||||
a keyboard message has exactly one interactive surface, and splitting
|
|
||||||
would orphan the buttons from the first chunk. Callers should keep
|
|
||||||
approval/update-prompt bodies short.
|
|
||||||
|
|
||||||
Guild (channel) chats don't support inline keyboards; returns a
|
|
||||||
non-retryable failure for those.
|
|
||||||
"""
|
|
||||||
if not self.is_connected:
|
|
||||||
if not await self._wait_for_reconnection():
|
|
||||||
return SendResult(
|
|
||||||
success=False, error="Not connected", retryable=True
|
|
||||||
)
|
|
||||||
|
|
||||||
chat_type = self._guess_chat_type(chat_id)
|
|
||||||
formatted = self.format_message(content)
|
|
||||||
truncated = formatted[: self.MAX_MESSAGE_LENGTH]
|
|
||||||
try:
|
|
||||||
if chat_type == "c2c":
|
|
||||||
return await self._send_c2c_text(
|
|
||||||
chat_id, truncated, reply_to, keyboard=keyboard,
|
|
||||||
)
|
|
||||||
if chat_type == "group":
|
|
||||||
return await self._send_group_text(
|
|
||||||
chat_id, truncated, reply_to, keyboard=keyboard,
|
|
||||||
)
|
|
||||||
return SendResult(
|
|
||||||
success=False,
|
|
||||||
error=(
|
|
||||||
f"Inline keyboards not supported for chat_type "
|
|
||||||
f"{chat_type!r}"
|
|
||||||
),
|
|
||||||
retryable=False,
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.error(
|
|
||||||
"[%s] send_with_keyboard failed: %s", self._log_tag, exc
|
|
||||||
)
|
|
||||||
return SendResult(success=False, error=str(exc))
|
|
||||||
|
|
||||||
async def send_approval_request(
|
|
||||||
self,
|
|
||||||
chat_id: str,
|
|
||||||
req: ApprovalRequest,
|
|
||||||
reply_to: Optional[str] = None,
|
|
||||||
) -> SendResult:
|
|
||||||
"""Send a 3-button approval request (``allow-once / allow-always / deny``).
|
|
||||||
|
|
||||||
The rendered text comes from :func:`build_approval_text`; callers can
|
|
||||||
override by passing a custom :class:`ApprovalRequest`.
|
|
||||||
|
|
||||||
Users click the button → ``INTERACTION_CREATE`` fires → the adapter's
|
|
||||||
registered :meth:`set_interaction_callback` handler decodes
|
|
||||||
``button_data`` via :func:`parse_approval_button_data`.
|
|
||||||
"""
|
|
||||||
from gateway.platforms.qqbot.keyboards import build_approval_text
|
|
||||||
return await self.send_with_keyboard(
|
|
||||||
chat_id,
|
|
||||||
build_approval_text(req),
|
|
||||||
build_approval_keyboard(req.session_key),
|
|
||||||
reply_to=reply_to,
|
|
||||||
)
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
# Cross-adapter gateway contract — send_exec_approval + send_update_prompt
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
#
|
|
||||||
# These mirror the signatures that gateway/run.py detects on the adapter
|
|
||||||
# class (e.g. type(adapter).send_exec_approval, type(adapter).send_update_prompt)
|
|
||||||
# for button-based approval / update-confirm UX. Discord, Telegram, Slack,
|
|
||||||
# Matrix, and Feishu already implement the same contract.
|
|
||||||
|
|
||||||
async def send_exec_approval(
|
|
||||||
self,
|
|
||||||
chat_id: str,
|
|
||||||
command: str,
|
|
||||||
session_key: str,
|
|
||||||
description: str = "dangerous command",
|
|
||||||
metadata: Optional[Dict[str, Any]] = None,
|
|
||||||
) -> SendResult:
|
|
||||||
"""Send a button-based exec-approval prompt for a dangerous command.
|
|
||||||
|
|
||||||
Called by ``gateway/run.py``'s ``_approval_notify_sync`` when the
|
|
||||||
agent is blocked waiting for approval. Button clicks resolve via
|
|
||||||
:func:`tools.approval.resolve_gateway_approval` — dispatched by the
|
|
||||||
adapter's interaction callback (:meth:`_default_interaction_dispatch`).
|
|
||||||
"""
|
|
||||||
del metadata # QQ doesn't have thread_id / DM targeting overrides.
|
|
||||||
|
|
||||||
# Use the reply-to message for passive-message context when we have one.
|
|
||||||
# QQ requires a msg_id on outbound messages to a user we've never
|
|
||||||
# seen; the last inbound msg_id is the natural choice.
|
|
||||||
msg_id = self._last_msg_id.get(chat_id)
|
|
||||||
|
|
||||||
req = ApprovalRequest(
|
|
||||||
session_key=session_key,
|
|
||||||
title=f"Execute this command?",
|
|
||||||
description=description,
|
|
||||||
command_preview=command,
|
|
||||||
timeout_sec=self._APPROVAL_TIMEOUT_SECONDS,
|
|
||||||
)
|
|
||||||
return await self.send_approval_request(
|
|
||||||
chat_id, req, reply_to=msg_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
_APPROVAL_TIMEOUT_SECONDS = 300 # matches gateway's default gateway_timeout
|
|
||||||
|
|
||||||
async def send_update_prompt(
|
|
||||||
self,
|
|
||||||
chat_id: str,
|
|
||||||
prompt: str,
|
|
||||||
default: str = "",
|
|
||||||
session_key: str = "",
|
|
||||||
metadata: Optional[Dict[str, Any]] = None,
|
|
||||||
) -> SendResult:
|
|
||||||
"""Send a Yes/No update-confirmation prompt with inline buttons.
|
|
||||||
|
|
||||||
Matches the cross-adapter contract used by
|
|
||||||
``gateway/run.py``'s ``hermes update --gateway`` watcher. Button
|
|
||||||
clicks surface as ``INTERACTION_CREATE`` with
|
|
||||||
``button_data = 'update_prompt:y'`` or ``'update_prompt:n'``;
|
|
||||||
the adapter's interaction callback writes the answer to
|
|
||||||
``~/.hermes/.update_response`` so the detached update process
|
|
||||||
can read it.
|
|
||||||
"""
|
|
||||||
del session_key, metadata # present for contract parity only.
|
|
||||||
|
|
||||||
default_hint = f" (default: {default})" if default else ""
|
|
||||||
content = f"⚕ **Update Needs Your Input**\n\n{prompt}{default_hint}"
|
|
||||||
msg_id = self._last_msg_id.get(chat_id)
|
|
||||||
return await self.send_with_keyboard(
|
|
||||||
chat_id,
|
|
||||||
content,
|
|
||||||
build_update_prompt_keyboard(),
|
|
||||||
reply_to=msg_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
def _build_text_body(
|
def _build_text_body(
|
||||||
self, content: str, reply_to: Optional[str] = None
|
self, content: str, reply_to: Optional[str] = None
|
||||||
) -> Dict[str, Any]:
|
) -> Dict[str, Any]:
|
||||||
@@ -2697,62 +2160,42 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
reply_to: Optional[str] = None,
|
reply_to: Optional[str] = None,
|
||||||
file_name: Optional[str] = None,
|
file_name: Optional[str] = None,
|
||||||
) -> SendResult:
|
) -> SendResult:
|
||||||
"""Upload media and send as a native message.
|
"""Upload media and send as a native message."""
|
||||||
|
|
||||||
Upload strategy:
|
|
||||||
|
|
||||||
- **HTTP(S) URLs** → single ``POST /v2/{users|groups}/{id}/files``
|
|
||||||
with ``url=...``. The QQ platform fetches the URL directly; fastest
|
|
||||||
path when the source is already hosted.
|
|
||||||
- **Local files** → three-step chunked upload (prepare / PUT parts /
|
|
||||||
complete). Handles files up to the platform's ~100 MB per-file
|
|
||||||
limit without the ~10 MB inline-base64 cap of the old adapter.
|
|
||||||
"""
|
|
||||||
if not self.is_connected:
|
if not self.is_connected:
|
||||||
if not await self._wait_for_reconnection():
|
if not await self._wait_for_reconnection():
|
||||||
return SendResult(success=False, error="Not connected", retryable=True)
|
return SendResult(success=False, error="Not connected", retryable=True)
|
||||||
|
|
||||||
chat_type = self._guess_chat_type(chat_id)
|
try:
|
||||||
if chat_type == "guild":
|
# Resolve media source
|
||||||
# Guild channels don't support native media upload in the same way.
|
data, content_type, resolved_name = await self._load_media(
|
||||||
return SendResult(
|
media_source, file_name
|
||||||
success=False,
|
|
||||||
error="Guild media send not supported via this path",
|
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
# Route
|
||||||
if self._is_url(media_source):
|
chat_type = self._guess_chat_type(chat_id)
|
||||||
# URL upload — let the platform fetch it directly.
|
|
||||||
resolved_name = (
|
if chat_type == "guild":
|
||||||
file_name
|
# Guild channels don't support native media upload in the same way
|
||||||
or Path(urlparse(media_source).path).name
|
# Send as URL fallback
|
||||||
or "media"
|
return SendResult(
|
||||||
)
|
success=False, error="Guild media send not supported via this path"
|
||||||
upload = await self._upload_media(
|
|
||||||
chat_type,
|
|
||||||
chat_id,
|
|
||||||
file_type,
|
|
||||||
url=media_source,
|
|
||||||
srv_send_msg=False,
|
|
||||||
file_name=resolved_name if file_type == MEDIA_TYPE_FILE else None,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
# Local file — chunked upload (prepare / PUT parts / complete).
|
|
||||||
resolved_name, upload = await self._upload_local_file(
|
|
||||||
chat_type,
|
|
||||||
chat_id,
|
|
||||||
media_source,
|
|
||||||
file_type,
|
|
||||||
file_name,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
file_info = upload.get("file_info") or (
|
# Upload
|
||||||
upload.get("data", {}) or {}
|
upload = await self._upload_media(
|
||||||
).get("file_info")
|
chat_type,
|
||||||
|
chat_id,
|
||||||
|
file_type,
|
||||||
|
file_data=data if not self._is_url(media_source) else None,
|
||||||
|
url=media_source if self._is_url(media_source) else None,
|
||||||
|
srv_send_msg=False,
|
||||||
|
file_name=resolved_name if file_type == MEDIA_TYPE_FILE else None,
|
||||||
|
)
|
||||||
|
|
||||||
|
file_info = upload.get("file_info")
|
||||||
if not file_info:
|
if not file_info:
|
||||||
return SendResult(
|
return SendResult(
|
||||||
success=False,
|
success=False, error=f"Upload returned no file_info: {upload}"
|
||||||
error=f"Upload returned no file_info: {upload}",
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# Send media message
|
# Send media message
|
||||||
@@ -2781,86 +2224,10 @@ class QQAdapter(BasePlatformAdapter):
|
|||||||
message_id=str(send_data.get("id", uuid.uuid4().hex[:12])),
|
message_id=str(send_data.get("id", uuid.uuid4().hex[:12])),
|
||||||
raw_response=send_data,
|
raw_response=send_data,
|
||||||
)
|
)
|
||||||
except UploadDailyLimitExceededError as exc:
|
|
||||||
# Non-retryable: daily quota hit. Give the caller actionable text
|
|
||||||
# so the model can compose a helpful reply.
|
|
||||||
logger.warning(
|
|
||||||
"[%s] Daily upload limit exceeded for %s (%s)",
|
|
||||||
self._log_tag, exc.file_name, exc.file_size_human,
|
|
||||||
)
|
|
||||||
return SendResult(
|
|
||||||
success=False,
|
|
||||||
error=(
|
|
||||||
f"QQ daily upload limit exceeded for {exc.file_name!r} "
|
|
||||||
f"({exc.file_size_human}). Retry tomorrow."
|
|
||||||
),
|
|
||||||
retryable=False,
|
|
||||||
)
|
|
||||||
except UploadFileTooLargeError as exc:
|
|
||||||
logger.warning(
|
|
||||||
"[%s] File too large: %s (%s, platform limit %s)",
|
|
||||||
self._log_tag, exc.file_name, exc.file_size_human, exc.limit_human,
|
|
||||||
)
|
|
||||||
return SendResult(
|
|
||||||
success=False,
|
|
||||||
error=(
|
|
||||||
f"{exc.file_name!r} ({exc.file_size_human}) exceeds the "
|
|
||||||
f"QQ per-file upload limit ({exc.limit_human})."
|
|
||||||
),
|
|
||||||
retryable=False,
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error("[%s] Media send failed: %s", self._log_tag, exc)
|
logger.error("[%s] Media send failed: %s", self._log_tag, exc)
|
||||||
return SendResult(success=False, error=str(exc))
|
return SendResult(success=False, error=str(exc))
|
||||||
|
|
||||||
async def _upload_local_file(
|
|
||||||
self,
|
|
||||||
chat_type: str,
|
|
||||||
chat_id: str,
|
|
||||||
media_source: str,
|
|
||||||
file_type: int,
|
|
||||||
file_name: Optional[str],
|
|
||||||
) -> Tuple[str, Dict[str, Any]]:
|
|
||||||
"""Chunked-upload a local file and return ``(resolved_name, complete_response)``.
|
|
||||||
|
|
||||||
The returned ``complete_response`` contains the ``file_info`` token
|
|
||||||
that goes into the subsequent RichMedia message body.
|
|
||||||
|
|
||||||
:raises UploadDailyLimitExceededError: On biz_code 40093002.
|
|
||||||
:raises UploadFileTooLargeError: When the file exceeds the platform limit.
|
|
||||||
:raises FileNotFoundError: If the path does not exist.
|
|
||||||
:raises ValueError: If the path looks like a placeholder (``<path>``).
|
|
||||||
:raises RuntimeError: If the HTTP client is not initialized.
|
|
||||||
"""
|
|
||||||
if not self._http_client:
|
|
||||||
raise RuntimeError("HTTP client not initialized — not connected?")
|
|
||||||
|
|
||||||
local_path = Path(media_source).expanduser()
|
|
||||||
if not local_path.is_absolute():
|
|
||||||
local_path = (Path.cwd() / local_path).resolve()
|
|
||||||
|
|
||||||
if not local_path.exists() or not local_path.is_file():
|
|
||||||
if media_source.startswith("<") or len(media_source) < 3:
|
|
||||||
raise ValueError(
|
|
||||||
f"Invalid media source (looks like a placeholder): {media_source!r}"
|
|
||||||
)
|
|
||||||
raise FileNotFoundError(f"Media file not found: {local_path}")
|
|
||||||
|
|
||||||
resolved_name = file_name or local_path.name
|
|
||||||
uploader = ChunkedUploader(
|
|
||||||
api_request=self._api_request,
|
|
||||||
http_put=self._http_client.put,
|
|
||||||
log_tag=self._log_tag,
|
|
||||||
)
|
|
||||||
complete = await uploader.upload(
|
|
||||||
chat_type=chat_type,
|
|
||||||
target_id=chat_id,
|
|
||||||
file_path=str(local_path),
|
|
||||||
file_type=file_type,
|
|
||||||
file_name=resolved_name,
|
|
||||||
)
|
|
||||||
return resolved_name, complete
|
|
||||||
|
|
||||||
async def _load_media(
|
async def _load_media(
|
||||||
self, source: str, file_name: Optional[str] = None
|
self, source: str, file_name: Optional[str] = None
|
||||||
) -> Tuple[str, str, str]:
|
) -> Tuple[str, str, str]:
|
||||||
|
|||||||
@@ -1,603 +0,0 @@
|
|||||||
"""QQ Bot chunked upload flow.
|
|
||||||
|
|
||||||
The QQ v2 API caps inline base64 uploads (``file_data`` / ``url``) at ~10 MB.
|
|
||||||
For files between 10 MB and ~100 MB we have to use the three-step chunked
|
|
||||||
upload flow::
|
|
||||||
|
|
||||||
1. POST /v2/{users|groups}/{id}/upload_prepare
|
|
||||||
→ returns upload_id, block_size, and an array of pre-signed COS part URLs.
|
|
||||||
2. For each part:
|
|
||||||
PUT the part bytes to its pre-signed COS URL,
|
|
||||||
then POST /v2/{users|groups}/{id}/upload_part_finish to acknowledge.
|
|
||||||
3. POST /v2/{users|groups}/{id}/files with {"upload_id": ...}
|
|
||||||
→ returns the ``file_info`` token the caller uses in a RichMedia
|
|
||||||
message.
|
|
||||||
|
|
||||||
Error-code semantics (from the QQ Bot v2 API spec):
|
|
||||||
|
|
||||||
- ``40093001`` — ``upload_part_finish`` retryable. Retry until the server-provided
|
|
||||||
``retry_timeout`` elapses (or a local cap).
|
|
||||||
- ``40093002`` — daily cumulative upload quota exceeded. Not retryable; surface
|
|
||||||
as :class:`UploadDailyLimitExceededError` so the caller can build a
|
|
||||||
user-friendly reply.
|
|
||||||
|
|
||||||
Exceptions:
|
|
||||||
|
|
||||||
- :class:`UploadDailyLimitExceededError` — daily quota hit (non-retryable).
|
|
||||||
- :class:`UploadFileTooLargeError` — file exceeds the platform per-file limit.
|
|
||||||
- :class:`RuntimeError` — generic upload failure (network, part PUT, complete).
|
|
||||||
|
|
||||||
Ported from WideLee's qqbot-agent-sdk v1.2.2 (``media_loader.py::ChunkedUploader``)
|
|
||||||
so the heavy-upload path stays in-tree. Authorship preserved via Co-authored-by.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import functools
|
|
||||||
import hashlib
|
|
||||||
import logging
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Awaitable, Callable, Dict, List, Optional
|
|
||||||
|
|
||||||
from gateway.platforms.qqbot.constants import FILE_UPLOAD_TIMEOUT
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Error codes ──────────────────────────────────────────────────────
|
|
||||||
_BIZ_CODE_DAILY_LIMIT = 40093002 # upload_prepare: daily cumulative limit
|
|
||||||
_BIZ_CODE_PART_RETRYABLE = 40093001 # upload_part_finish: transient
|
|
||||||
|
|
||||||
# ── Part upload tuning ───────────────────────────────────────────────
|
|
||||||
_DEFAULT_CONCURRENT_PARTS = 1
|
|
||||||
_MAX_CONCURRENT_PARTS = 10
|
|
||||||
|
|
||||||
_PART_UPLOAD_TIMEOUT = 300.0 # 5 minutes per COS PUT
|
|
||||||
_PART_UPLOAD_MAX_RETRIES = 2
|
|
||||||
_PART_FINISH_RETRY_INTERVAL = 1.0
|
|
||||||
_PART_FINISH_DEFAULT_TIMEOUT = 120.0
|
|
||||||
_PART_FINISH_MAX_TIMEOUT = 600.0
|
|
||||||
|
|
||||||
_COMPLETE_UPLOAD_MAX_RETRIES = 2
|
|
||||||
_COMPLETE_UPLOAD_BASE_DELAY = 2.0
|
|
||||||
|
|
||||||
# First 10,002,432 bytes used for the ``md5_10m`` hash (per QQ API spec).
|
|
||||||
_MD5_10M_SIZE = 10_002_432
|
|
||||||
|
|
||||||
|
|
||||||
# ── Exceptions ───────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
class UploadDailyLimitExceededError(Exception):
|
|
||||||
"""Raised when ``upload_prepare`` returns biz_code 40093002.
|
|
||||||
|
|
||||||
The daily cumulative upload quota for this bot has been reached. Callers
|
|
||||||
should surface :attr:`file_name` + :attr:`file_size_human` so the model
|
|
||||||
can compose a helpful reply.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, file_name: str, file_size: int, message: str = "") -> None:
|
|
||||||
self.file_name = file_name
|
|
||||||
self.file_size = file_size
|
|
||||||
super().__init__(
|
|
||||||
message or f"Daily upload limit exceeded for {file_name!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
@property
|
|
||||||
def file_size_human(self) -> str:
|
|
||||||
return format_size(self.file_size)
|
|
||||||
|
|
||||||
|
|
||||||
class UploadFileTooLargeError(Exception):
|
|
||||||
"""Raised when a file exceeds the platform per-file size limit."""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
file_name: str,
|
|
||||||
file_size: int,
|
|
||||||
limit_bytes: int = 0,
|
|
||||||
message: str = "",
|
|
||||||
) -> None:
|
|
||||||
self.file_name = file_name
|
|
||||||
self.file_size = file_size
|
|
||||||
self.limit_bytes = limit_bytes
|
|
||||||
limit_str = f" ({format_size(limit_bytes)})" if limit_bytes else ""
|
|
||||||
super().__init__(
|
|
||||||
message
|
|
||||||
or (
|
|
||||||
f"File {file_name!r} ({format_size(file_size)}) "
|
|
||||||
f"exceeds platform limit{limit_str}"
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
@property
|
|
||||||
def file_size_human(self) -> str:
|
|
||||||
return format_size(self.file_size)
|
|
||||||
|
|
||||||
@property
|
|
||||||
def limit_human(self) -> str:
|
|
||||||
return format_size(self.limit_bytes) if self.limit_bytes else "unknown"
|
|
||||||
|
|
||||||
|
|
||||||
# ── Progress tracking ────────────────────────────────────────────────
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _UploadProgress:
|
|
||||||
total_parts: int = 0
|
|
||||||
total_bytes: int = 0
|
|
||||||
completed_parts: int = 0
|
|
||||||
uploaded_bytes: int = 0
|
|
||||||
|
|
||||||
|
|
||||||
# ── Prepare-response shape ───────────────────────────────────────────
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _PreparePart:
|
|
||||||
index: int
|
|
||||||
presigned_url: str
|
|
||||||
block_size: int = 0
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _PrepareResult:
|
|
||||||
upload_id: str
|
|
||||||
block_size: int
|
|
||||||
parts: List[_PreparePart]
|
|
||||||
concurrency: int = _DEFAULT_CONCURRENT_PARTS
|
|
||||||
retry_timeout: float = 0.0
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_prepare_response(raw: Dict[str, Any]) -> _PrepareResult:
|
|
||||||
"""Parse the upload_prepare API response into a normalized shape.
|
|
||||||
|
|
||||||
The API may return the response directly or wrapped in ``data``.
|
|
||||||
"""
|
|
||||||
src = raw.get("data") if isinstance(raw.get("data"), dict) else raw
|
|
||||||
upload_id = str(src.get("upload_id", ""))
|
|
||||||
if not upload_id:
|
|
||||||
raise ValueError(
|
|
||||||
f"upload_prepare response missing upload_id: {str(raw)[:200]}"
|
|
||||||
)
|
|
||||||
block_size = int(src.get("block_size", 0))
|
|
||||||
raw_parts = src.get("parts") or src.get("part_list") or []
|
|
||||||
if not isinstance(raw_parts, list) or not raw_parts:
|
|
||||||
raise ValueError(
|
|
||||||
f"upload_prepare response missing parts: {str(raw)[:200]}"
|
|
||||||
)
|
|
||||||
parts: List[_PreparePart] = []
|
|
||||||
for p in raw_parts:
|
|
||||||
if not isinstance(p, dict):
|
|
||||||
continue
|
|
||||||
parts.append(
|
|
||||||
_PreparePart(
|
|
||||||
index=int(p.get("part_index") or p.get("index") or 0),
|
|
||||||
presigned_url=str(
|
|
||||||
p.get("presigned_url") or p.get("url") or ""
|
|
||||||
),
|
|
||||||
block_size=int(p.get("block_size", 0)),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return _PrepareResult(
|
|
||||||
upload_id=upload_id,
|
|
||||||
block_size=block_size,
|
|
||||||
parts=parts,
|
|
||||||
concurrency=int(src.get("concurrency", _DEFAULT_CONCURRENT_PARTS)) or _DEFAULT_CONCURRENT_PARTS,
|
|
||||||
retry_timeout=float(src.get("retry_timeout", 0.0) or 0.0),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Chunked upload driver ────────────────────────────────────────────
|
|
||||||
|
|
||||||
ApiRequestFn = Callable[..., Awaitable[Dict[str, Any]]]
|
|
||||||
"""Signature of the adapter's ``_api_request`` callable.
|
|
||||||
|
|
||||||
We pass the bound method in rather than importing the adapter, to avoid
|
|
||||||
circular imports and keep this module testable in isolation.
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
class ChunkedUploader:
|
|
||||||
"""Run the prepare → PUT parts → complete sequence.
|
|
||||||
|
|
||||||
:param api_request: Bound ``_api_request(method, path, body=..., timeout=...)``
|
|
||||||
coroutine from the adapter. Must raise ``RuntimeError`` with the biz_code
|
|
||||||
embedded in the message on API errors.
|
|
||||||
:param http_put: Coroutine ``(url, data, headers, timeout) -> response`` for
|
|
||||||
COS part uploads. Typically wraps ``httpx.AsyncClient.put``.
|
|
||||||
:param log_tag: Log prefix.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
api_request: ApiRequestFn,
|
|
||||||
http_put: Callable[..., Awaitable[Any]],
|
|
||||||
log_tag: str = "QQBot",
|
|
||||||
) -> None:
|
|
||||||
self._api_request = api_request
|
|
||||||
self._http_put = http_put
|
|
||||||
self._log_tag = log_tag
|
|
||||||
|
|
||||||
async def upload(
|
|
||||||
self,
|
|
||||||
chat_type: str,
|
|
||||||
target_id: str,
|
|
||||||
file_path: str,
|
|
||||||
file_type: int,
|
|
||||||
file_name: str,
|
|
||||||
) -> Dict[str, Any]:
|
|
||||||
"""Run the full chunked upload and return the ``complete_upload`` response.
|
|
||||||
|
|
||||||
:param chat_type: ``'c2c'`` or ``'group'``.
|
|
||||||
:param target_id: User or group openid.
|
|
||||||
:param file_path: Absolute path to a local file.
|
|
||||||
:param file_type: ``MEDIA_TYPE_*`` constant.
|
|
||||||
:param file_name: Original filename (for upload_prepare).
|
|
||||||
:returns: The raw response dict from ``complete_upload`` — contains
|
|
||||||
``file_info`` that the caller uses in a RichMedia message body.
|
|
||||||
:raises UploadDailyLimitExceededError: On biz_code 40093002.
|
|
||||||
:raises UploadFileTooLargeError: When the file exceeds the platform limit.
|
|
||||||
:raises RuntimeError: On other API or I/O failures.
|
|
||||||
"""
|
|
||||||
if chat_type not in ("c2c", "group"):
|
|
||||||
raise ValueError(
|
|
||||||
f"ChunkedUploader: unsupported chat_type {chat_type!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
path = Path(file_path)
|
|
||||||
file_size = path.stat().st_size
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"[%s] Chunked upload start: file=%s size=%s type=%d",
|
|
||||||
self._log_tag, file_name, format_size(file_size), file_type,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Step 1: compute hashes (blocking I/O → executor).
|
|
||||||
hashes = await asyncio.get_running_loop().run_in_executor(
|
|
||||||
None, _compute_file_hashes, file_path, file_size
|
|
||||||
)
|
|
||||||
|
|
||||||
# Step 2: upload_prepare.
|
|
||||||
prepare = await self._prepare(
|
|
||||||
chat_type, target_id, file_type, file_name, file_size, hashes
|
|
||||||
)
|
|
||||||
max_concurrent = min(prepare.concurrency, _MAX_CONCURRENT_PARTS)
|
|
||||||
retry_timeout = min(
|
|
||||||
prepare.retry_timeout if prepare.retry_timeout > 0 else _PART_FINISH_DEFAULT_TIMEOUT,
|
|
||||||
_PART_FINISH_MAX_TIMEOUT,
|
|
||||||
)
|
|
||||||
logger.info(
|
|
||||||
"[%s] Prepared: upload_id=%s block_size=%s parts=%d concurrency=%d",
|
|
||||||
self._log_tag, prepare.upload_id, format_size(prepare.block_size),
|
|
||||||
len(prepare.parts), max_concurrent,
|
|
||||||
)
|
|
||||||
|
|
||||||
progress = _UploadProgress(
|
|
||||||
total_parts=len(prepare.parts),
|
|
||||||
total_bytes=file_size,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Step 3: PUT each part + notify.
|
|
||||||
tasks: List[Callable[[], Awaitable[None]]] = [
|
|
||||||
functools.partial(
|
|
||||||
self._upload_one_part,
|
|
||||||
chat_type=chat_type,
|
|
||||||
target_id=target_id,
|
|
||||||
file_path=file_path,
|
|
||||||
file_size=file_size,
|
|
||||||
upload_id=prepare.upload_id,
|
|
||||||
rsp_block_size=prepare.block_size,
|
|
||||||
part=part,
|
|
||||||
retry_timeout=retry_timeout,
|
|
||||||
progress=progress,
|
|
||||||
)
|
|
||||||
for part in prepare.parts
|
|
||||||
]
|
|
||||||
await _run_with_concurrency(tasks, max_concurrent)
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"[%s] All %d parts uploaded, completing…",
|
|
||||||
self._log_tag, len(prepare.parts),
|
|
||||||
)
|
|
||||||
|
|
||||||
# Step 4: complete_upload (retry on transient errors).
|
|
||||||
return await self._complete(chat_type, target_id, prepare.upload_id)
|
|
||||||
|
|
||||||
# ──────────────────────────────────────────────────────────────────
|
|
||||||
# Step 1 — upload_prepare
|
|
||||||
# ──────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
async def _prepare(
|
|
||||||
self,
|
|
||||||
chat_type: str,
|
|
||||||
target_id: str,
|
|
||||||
file_type: int,
|
|
||||||
file_name: str,
|
|
||||||
file_size: int,
|
|
||||||
hashes: Dict[str, str],
|
|
||||||
) -> _PrepareResult:
|
|
||||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
|
||||||
path = f"{base}/{target_id}/upload_prepare"
|
|
||||||
body = {
|
|
||||||
"file_type": file_type,
|
|
||||||
"file_name": file_name,
|
|
||||||
"file_size": file_size,
|
|
||||||
"md5": hashes["md5"],
|
|
||||||
"sha1": hashes["sha1"],
|
|
||||||
"md5_10m": hashes["md5_10m"],
|
|
||||||
}
|
|
||||||
try:
|
|
||||||
raw = await self._api_request(
|
|
||||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
|
||||||
)
|
|
||||||
except RuntimeError as exc:
|
|
||||||
err_msg = str(exc)
|
|
||||||
if f"{_BIZ_CODE_DAILY_LIMIT}" in err_msg:
|
|
||||||
raise UploadDailyLimitExceededError(
|
|
||||||
file_name, file_size, err_msg
|
|
||||||
) from exc
|
|
||||||
raise
|
|
||||||
return _parse_prepare_response(raw)
|
|
||||||
|
|
||||||
# ──────────────────────────────────────────────────────────────────
|
|
||||||
# Step 2 — PUT one part + part_finish
|
|
||||||
# ──────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
async def _upload_one_part(
|
|
||||||
self,
|
|
||||||
chat_type: str,
|
|
||||||
target_id: str,
|
|
||||||
file_path: str,
|
|
||||||
file_size: int,
|
|
||||||
upload_id: str,
|
|
||||||
rsp_block_size: int,
|
|
||||||
part: _PreparePart,
|
|
||||||
retry_timeout: float,
|
|
||||||
progress: _UploadProgress,
|
|
||||||
) -> None:
|
|
||||||
"""PUT one part to COS, then call ``upload_part_finish``."""
|
|
||||||
part_index = part.index
|
|
||||||
# Per-part block_size wins; fall back to the response-level value.
|
|
||||||
actual_block_size = part.block_size if part.block_size > 0 else rsp_block_size
|
|
||||||
offset = (part_index - 1) * rsp_block_size
|
|
||||||
length = min(actual_block_size, file_size - offset)
|
|
||||||
|
|
||||||
# Read this slice of the file (blocking → executor).
|
|
||||||
data = await asyncio.get_running_loop().run_in_executor(
|
|
||||||
None, _read_file_chunk, file_path, offset, length
|
|
||||||
)
|
|
||||||
md5_hex = hashlib.md5(data).hexdigest()
|
|
||||||
|
|
||||||
logger.debug(
|
|
||||||
"[%s] Part %d/%d: uploading %s (offset=%d md5=%s)",
|
|
||||||
self._log_tag, part_index, progress.total_parts,
|
|
||||||
format_size(length), offset, md5_hex,
|
|
||||||
)
|
|
||||||
|
|
||||||
await self._put_to_presigned_url(
|
|
||||||
part.presigned_url, data, part_index, progress.total_parts
|
|
||||||
)
|
|
||||||
await self._part_finish_with_retry(
|
|
||||||
chat_type, target_id, upload_id,
|
|
||||||
part_index, length, md5_hex, retry_timeout,
|
|
||||||
)
|
|
||||||
|
|
||||||
progress.completed_parts += 1
|
|
||||||
progress.uploaded_bytes += length
|
|
||||||
logger.debug(
|
|
||||||
"[%s] Part %d/%d done (%d/%d total)",
|
|
||||||
self._log_tag, part_index, progress.total_parts,
|
|
||||||
progress.completed_parts, progress.total_parts,
|
|
||||||
)
|
|
||||||
|
|
||||||
async def _put_to_presigned_url(
|
|
||||||
self,
|
|
||||||
url: str,
|
|
||||||
data: bytes,
|
|
||||||
part_index: int,
|
|
||||||
total_parts: int,
|
|
||||||
) -> None:
|
|
||||||
"""PUT part data to a pre-signed COS URL with retry."""
|
|
||||||
last_exc: Optional[Exception] = None
|
|
||||||
for attempt in range(_PART_UPLOAD_MAX_RETRIES + 1):
|
|
||||||
try:
|
|
||||||
resp = await asyncio.wait_for(
|
|
||||||
self._http_put(
|
|
||||||
url,
|
|
||||||
data=data,
|
|
||||||
headers={"Content-Length": str(len(data))},
|
|
||||||
),
|
|
||||||
timeout=_PART_UPLOAD_TIMEOUT,
|
|
||||||
)
|
|
||||||
# Caller's http_put is expected to return an httpx-like response.
|
|
||||||
status = getattr(resp, "status_code", 0)
|
|
||||||
if 200 <= status < 300:
|
|
||||||
logger.debug(
|
|
||||||
"[%s] PUT part %d/%d: %d OK",
|
|
||||||
self._log_tag, part_index, total_parts, status,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
body_preview = ""
|
|
||||||
try:
|
|
||||||
body_preview = getattr(resp, "text", "")[:200]
|
|
||||||
except Exception: # pragma: no cover — defensive
|
|
||||||
pass
|
|
||||||
raise RuntimeError(
|
|
||||||
f"COS PUT returned {status}: {body_preview}"
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
last_exc = exc
|
|
||||||
if attempt < _PART_UPLOAD_MAX_RETRIES:
|
|
||||||
delay = 1.0 * (2 ** attempt)
|
|
||||||
logger.warning(
|
|
||||||
"[%s] PUT part %d/%d attempt %d failed, retry in %.1fs: %s",
|
|
||||||
self._log_tag, part_index, total_parts,
|
|
||||||
attempt + 1, delay, exc,
|
|
||||||
)
|
|
||||||
await asyncio.sleep(delay)
|
|
||||||
raise RuntimeError(
|
|
||||||
f"Part {part_index}/{total_parts} upload failed after "
|
|
||||||
f"{_PART_UPLOAD_MAX_RETRIES + 1} attempts: {last_exc}"
|
|
||||||
)
|
|
||||||
|
|
||||||
async def _part_finish_with_retry(
|
|
||||||
self,
|
|
||||||
chat_type: str,
|
|
||||||
target_id: str,
|
|
||||||
upload_id: str,
|
|
||||||
part_index: int,
|
|
||||||
block_size: int,
|
|
||||||
md5: str,
|
|
||||||
retry_timeout: float,
|
|
||||||
) -> None:
|
|
||||||
"""Call ``upload_part_finish``, retrying on biz_code 40093001."""
|
|
||||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
|
||||||
path = f"{base}/{target_id}/upload_part_finish"
|
|
||||||
body = {
|
|
||||||
"upload_id": upload_id,
|
|
||||||
"part_index": part_index,
|
|
||||||
"block_size": block_size,
|
|
||||||
"md5": md5,
|
|
||||||
}
|
|
||||||
|
|
||||||
loop = asyncio.get_running_loop()
|
|
||||||
start = loop.time()
|
|
||||||
attempt = 0
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
await self._api_request(
|
|
||||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
|
||||||
)
|
|
||||||
return
|
|
||||||
except RuntimeError as exc:
|
|
||||||
err_msg = str(exc)
|
|
||||||
if f"{_BIZ_CODE_PART_RETRYABLE}" not in err_msg:
|
|
||||||
raise
|
|
||||||
elapsed = loop.time() - start
|
|
||||||
if elapsed >= retry_timeout:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"upload_part_finish persistent retry timed out "
|
|
||||||
f"after {retry_timeout:.0f}s ({attempt} retries): {exc}"
|
|
||||||
) from exc
|
|
||||||
attempt += 1
|
|
||||||
logger.debug(
|
|
||||||
"[%s] part_finish retryable error, attempt %d, "
|
|
||||||
"elapsed=%.1fs: %s",
|
|
||||||
self._log_tag, attempt, elapsed, exc,
|
|
||||||
)
|
|
||||||
await asyncio.sleep(_PART_FINISH_RETRY_INTERVAL)
|
|
||||||
|
|
||||||
# ──────────────────────────────────────────────────────────────────
|
|
||||||
# Step 3 — complete_upload
|
|
||||||
# ──────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
async def _complete(
|
|
||||||
self,
|
|
||||||
chat_type: str,
|
|
||||||
target_id: str,
|
|
||||||
upload_id: str,
|
|
||||||
) -> Dict[str, Any]:
|
|
||||||
"""Call ``complete_upload`` with retry.
|
|
||||||
|
|
||||||
This reuses the ``/files`` endpoint (same as the simple URL-based upload)
|
|
||||||
but signals the chunked-completion path by sending only ``upload_id``.
|
|
||||||
"""
|
|
||||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
|
||||||
path = f"{base}/{target_id}/files"
|
|
||||||
body = {"upload_id": upload_id}
|
|
||||||
|
|
||||||
last_exc: Optional[Exception] = None
|
|
||||||
for attempt in range(_COMPLETE_UPLOAD_MAX_RETRIES + 1):
|
|
||||||
try:
|
|
||||||
return await self._api_request(
|
|
||||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
last_exc = exc
|
|
||||||
if attempt < _COMPLETE_UPLOAD_MAX_RETRIES:
|
|
||||||
delay = _COMPLETE_UPLOAD_BASE_DELAY * (2 ** attempt)
|
|
||||||
logger.warning(
|
|
||||||
"[%s] complete_upload attempt %d failed, "
|
|
||||||
"retry in %.1fs: %s",
|
|
||||||
self._log_tag, attempt + 1, delay, exc,
|
|
||||||
)
|
|
||||||
await asyncio.sleep(delay)
|
|
||||||
raise RuntimeError(
|
|
||||||
f"complete_upload failed after "
|
|
||||||
f"{_COMPLETE_UPLOAD_MAX_RETRIES + 1} attempts: {last_exc}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Helpers (module-level for testability) ───────────────────────────
|
|
||||||
|
|
||||||
def format_size(size_bytes: int) -> str:
|
|
||||||
"""Return a human-readable file size string (e.g. ``'12.3 MB'``)."""
|
|
||||||
size = float(size_bytes)
|
|
||||||
for unit in ("B", "KB", "MB", "GB"):
|
|
||||||
if size < 1024.0:
|
|
||||||
return f"{size:.1f} {unit}"
|
|
||||||
size /= 1024.0
|
|
||||||
return f"{size:.1f} TB"
|
|
||||||
|
|
||||||
|
|
||||||
def _read_file_chunk(file_path: str, offset: int, length: int) -> bytes:
|
|
||||||
"""Read *length* bytes from *file_path* starting at *offset*.
|
|
||||||
|
|
||||||
:raises IOError: If fewer bytes were read than expected (truncated file).
|
|
||||||
"""
|
|
||||||
with open(file_path, "rb") as fh:
|
|
||||||
fh.seek(offset)
|
|
||||||
data = fh.read(length)
|
|
||||||
if len(data) != length:
|
|
||||||
raise IOError(
|
|
||||||
f"Short read from {file_path}: expected {length} bytes at "
|
|
||||||
f"offset {offset}, got {len(data)} (file may be truncated)"
|
|
||||||
)
|
|
||||||
return data
|
|
||||||
|
|
||||||
|
|
||||||
def _compute_file_hashes(file_path: str, file_size: int) -> Dict[str, str]:
|
|
||||||
"""Compute md5, sha1, and md5_10m in a single pass."""
|
|
||||||
md5 = hashlib.md5()
|
|
||||||
sha1 = hashlib.sha1()
|
|
||||||
md5_10m = hashlib.md5()
|
|
||||||
|
|
||||||
need_10m = file_size > _MD5_10M_SIZE
|
|
||||||
bytes_read = 0
|
|
||||||
|
|
||||||
with open(file_path, "rb") as fh:
|
|
||||||
while True:
|
|
||||||
chunk = fh.read(65536)
|
|
||||||
if not chunk:
|
|
||||||
break
|
|
||||||
md5.update(chunk)
|
|
||||||
sha1.update(chunk)
|
|
||||||
if need_10m:
|
|
||||||
remaining = _MD5_10M_SIZE - bytes_read
|
|
||||||
if remaining > 0:
|
|
||||||
md5_10m.update(chunk[:remaining])
|
|
||||||
bytes_read += len(chunk)
|
|
||||||
|
|
||||||
full_md5 = md5.hexdigest()
|
|
||||||
return {
|
|
||||||
"md5": full_md5,
|
|
||||||
"sha1": sha1.hexdigest(),
|
|
||||||
# For small files the "10m" hash is just the full md5.
|
|
||||||
"md5_10m": md5_10m.hexdigest() if need_10m else full_md5,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
async def _run_with_concurrency(
|
|
||||||
tasks: List[Callable[[], Awaitable[None]]],
|
|
||||||
concurrency: int,
|
|
||||||
) -> None:
|
|
||||||
"""Run a list of thunks with a bounded number in flight at once."""
|
|
||||||
if concurrency < 1:
|
|
||||||
concurrency = 1
|
|
||||||
sem = asyncio.Semaphore(concurrency)
|
|
||||||
|
|
||||||
async def _wrap(thunk: Callable[[], Awaitable[None]]) -> None:
|
|
||||||
async with sem:
|
|
||||||
await thunk()
|
|
||||||
|
|
||||||
await asyncio.gather(*(_wrap(t) for t in tasks))
|
|
||||||
@@ -1,473 +0,0 @@
|
|||||||
"""QQ Bot inline keyboards + approval / update-prompt senders.
|
|
||||||
|
|
||||||
QQ Bot v2 supports attaching inline keyboards to outbound messages. When a
|
|
||||||
user clicks a button, the platform dispatches an ``INTERACTION_CREATE``
|
|
||||||
gateway event containing the button's ``data`` payload. The bot must ACK the
|
|
||||||
interaction promptly via ``PUT /interactions/{id}`` or the user sees an
|
|
||||||
error indicator on the button.
|
|
||||||
|
|
||||||
This module provides:
|
|
||||||
|
|
||||||
- :class:`InlineKeyboard` + button dataclasses — serialized into the
|
|
||||||
``keyboard`` field of the outbound message body.
|
|
||||||
- :func:`build_approval_keyboard` — 3-button ✅ once / ⭐ always / ❌ deny
|
|
||||||
keyboard for tool-approval flows.
|
|
||||||
- :func:`build_update_prompt_keyboard` — Yes/No keyboard for update confirms.
|
|
||||||
- :func:`parse_approval_button_data` / :func:`parse_update_prompt_button_data`
|
|
||||||
— decode the ``button_data`` payload from ``INTERACTION_CREATE``.
|
|
||||||
- :class:`ApprovalRequest` + :class:`ApprovalSender` — high-level helper that
|
|
||||||
builds an approval message with keyboard and posts it to a c2c / group chat.
|
|
||||||
|
|
||||||
``button_data`` formats::
|
|
||||||
|
|
||||||
approve:<session_key>:<decision> # decision = allow-once|allow-always|deny
|
|
||||||
update_prompt:<answer> # answer = y|n
|
|
||||||
|
|
||||||
Ported from WideLee's qqbot-agent-sdk v1.2.2 (``approval.py`` + ``dto.py``
|
|
||||||
keyboard types). Authorship preserved via Co-authored-by.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import logging
|
|
||||||
import re
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from typing import Any, Awaitable, Callable, Dict, List, Optional
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
# ── button_data prefixes + patterns ──────────────────────────────────
|
|
||||||
|
|
||||||
APPROVAL_BUTTON_PREFIX = "approve:"
|
|
||||||
UPDATE_PROMPT_PREFIX = "update_prompt:"
|
|
||||||
|
|
||||||
# Pattern: approve:<session_key>:<decision>
|
|
||||||
# session_key may itself contain colons (e.g. agent:main:qqbot:c2c:OPENID),
|
|
||||||
# so the session_key group is greedy but trails the decision.
|
|
||||||
_APPROVAL_DATA_RE = re.compile(
|
|
||||||
r"^approve:(.+):(allow-once|allow-always|deny)$"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Pattern: update_prompt:y | update_prompt:n
|
|
||||||
_UPDATE_PROMPT_RE = re.compile(r"^update_prompt:(y|n)$")
|
|
||||||
|
|
||||||
|
|
||||||
# ── Keyboard dataclasses ─────────────────────────────────────────────
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class KeyboardButtonPermission:
|
|
||||||
"""Button permission metadata. ``type=2`` means all users can click."""
|
|
||||||
type: int = 2
|
|
||||||
|
|
||||||
def to_dict(self) -> Dict[str, Any]:
|
|
||||||
return {"type": self.type}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class KeyboardButtonAction:
|
|
||||||
"""What happens when the button is clicked.
|
|
||||||
|
|
||||||
:param type: ``1`` (Callback — triggers ``INTERACTION_CREATE``) or
|
|
||||||
``2`` (Link — opens a URL).
|
|
||||||
:param data: Payload delivered in ``data.resolved.button_data`` when
|
|
||||||
``type=1``.
|
|
||||||
:param permission: :class:`KeyboardButtonPermission`.
|
|
||||||
:param click_limit: Max clicks per user (``1`` = single-use).
|
|
||||||
"""
|
|
||||||
type: int
|
|
||||||
data: str
|
|
||||||
permission: KeyboardButtonPermission = field(
|
|
||||||
default_factory=KeyboardButtonPermission
|
|
||||||
)
|
|
||||||
click_limit: int = 1
|
|
||||||
|
|
||||||
def to_dict(self) -> Dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"type": self.type,
|
|
||||||
"data": self.data,
|
|
||||||
"permission": self.permission.to_dict(),
|
|
||||||
"click_limit": self.click_limit,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class KeyboardButtonRenderData:
|
|
||||||
"""Visual rendering of a button.
|
|
||||||
|
|
||||||
:param label: Pre-click label.
|
|
||||||
:param visited_label: Post-click label (button stays greyed in place).
|
|
||||||
:param style: ``0`` = grey, ``1`` = blue.
|
|
||||||
"""
|
|
||||||
label: str
|
|
||||||
visited_label: str
|
|
||||||
style: int = 1
|
|
||||||
|
|
||||||
def to_dict(self) -> Dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"label": self.label,
|
|
||||||
"visited_label": self.visited_label,
|
|
||||||
"style": self.style,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class KeyboardButton:
|
|
||||||
"""One button in a keyboard.
|
|
||||||
|
|
||||||
:param group_id: Buttons sharing a ``group_id`` are mutually exclusive —
|
|
||||||
clicking one greys the rest.
|
|
||||||
"""
|
|
||||||
id: str
|
|
||||||
render_data: KeyboardButtonRenderData
|
|
||||||
action: KeyboardButtonAction
|
|
||||||
group_id: str = "default"
|
|
||||||
|
|
||||||
def to_dict(self) -> Dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"id": self.id,
|
|
||||||
"render_data": self.render_data.to_dict(),
|
|
||||||
"action": self.action.to_dict(),
|
|
||||||
"group_id": self.group_id,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class KeyboardRow:
|
|
||||||
buttons: List[KeyboardButton] = field(default_factory=list)
|
|
||||||
|
|
||||||
def to_dict(self) -> Dict[str, Any]:
|
|
||||||
return {"buttons": [b.to_dict() for b in self.buttons]}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class KeyboardContent:
|
|
||||||
rows: List[KeyboardRow] = field(default_factory=list)
|
|
||||||
|
|
||||||
def to_dict(self) -> Dict[str, Any]:
|
|
||||||
return {"rows": [r.to_dict() for r in self.rows]}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class InlineKeyboard:
|
|
||||||
"""Top-level keyboard payload — goes into ``MessageToCreate.keyboard``."""
|
|
||||||
content: KeyboardContent = field(default_factory=KeyboardContent)
|
|
||||||
|
|
||||||
def to_dict(self) -> Dict[str, Any]:
|
|
||||||
return {"content": self.content.to_dict()}
|
|
||||||
|
|
||||||
|
|
||||||
# ── INTERACTION_CREATE parsing ───────────────────────────────────────
|
|
||||||
|
|
||||||
def parse_approval_button_data(button_data: str) -> Optional[tuple[str, str]]:
|
|
||||||
"""Parse approval ``button_data`` into ``(session_key, decision)``.
|
|
||||||
|
|
||||||
:param button_data: Raw ``data.resolved.button_data`` from
|
|
||||||
``INTERACTION_CREATE``.
|
|
||||||
:returns: ``(session_key, decision)`` or ``None`` if not an approval button.
|
|
||||||
"""
|
|
||||||
m = _APPROVAL_DATA_RE.match(button_data or "")
|
|
||||||
if not m:
|
|
||||||
return None
|
|
||||||
return m.group(1), m.group(2)
|
|
||||||
|
|
||||||
|
|
||||||
def parse_update_prompt_button_data(button_data: str) -> Optional[str]:
|
|
||||||
"""Parse update-prompt ``button_data`` into ``'y'`` or ``'n'``."""
|
|
||||||
m = _UPDATE_PROMPT_RE.match(button_data or "")
|
|
||||||
if not m:
|
|
||||||
return None
|
|
||||||
return m.group(1)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Keyboard builders ────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _make_callback_button(
|
|
||||||
btn_id: str,
|
|
||||||
label: str,
|
|
||||||
visited_label: str,
|
|
||||||
data: str,
|
|
||||||
style: int,
|
|
||||||
group_id: str,
|
|
||||||
) -> KeyboardButton:
|
|
||||||
return KeyboardButton(
|
|
||||||
id=btn_id,
|
|
||||||
render_data=KeyboardButtonRenderData(
|
|
||||||
label=label,
|
|
||||||
visited_label=visited_label,
|
|
||||||
style=style,
|
|
||||||
),
|
|
||||||
action=KeyboardButtonAction(type=1, data=data),
|
|
||||||
group_id=group_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def build_approval_keyboard(session_key: str) -> InlineKeyboard:
|
|
||||||
"""Build the 3-button approval keyboard.
|
|
||||||
|
|
||||||
Layout: ``[✅ 允许一次] [⭐ 始终允许] [❌ 拒绝]`` — all three share
|
|
||||||
``group_id='approval'`` so clicking one greys out the rest.
|
|
||||||
|
|
||||||
:param session_key: Embedded into ``button_data`` so the decision
|
|
||||||
routes back to the right pending approval.
|
|
||||||
"""
|
|
||||||
return InlineKeyboard(
|
|
||||||
content=KeyboardContent(
|
|
||||||
rows=[
|
|
||||||
KeyboardRow(buttons=[
|
|
||||||
_make_callback_button(
|
|
||||||
btn_id="allow",
|
|
||||||
label="✅ 允许一次",
|
|
||||||
visited_label="已允许",
|
|
||||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:allow-once",
|
|
||||||
style=1,
|
|
||||||
group_id="approval",
|
|
||||||
),
|
|
||||||
_make_callback_button(
|
|
||||||
btn_id="always",
|
|
||||||
label="⭐ 始终允许",
|
|
||||||
visited_label="已始终允许",
|
|
||||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:allow-always",
|
|
||||||
style=1,
|
|
||||||
group_id="approval",
|
|
||||||
),
|
|
||||||
_make_callback_button(
|
|
||||||
btn_id="deny",
|
|
||||||
label="❌ 拒绝",
|
|
||||||
visited_label="已拒绝",
|
|
||||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:deny",
|
|
||||||
style=0,
|
|
||||||
group_id="approval",
|
|
||||||
),
|
|
||||||
]),
|
|
||||||
]
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def build_update_prompt_keyboard() -> InlineKeyboard:
|
|
||||||
"""Build a Yes/No keyboard for update confirmation prompts."""
|
|
||||||
return InlineKeyboard(
|
|
||||||
content=KeyboardContent(
|
|
||||||
rows=[
|
|
||||||
KeyboardRow(buttons=[
|
|
||||||
_make_callback_button(
|
|
||||||
btn_id="yes",
|
|
||||||
label="✓ 确认",
|
|
||||||
visited_label="已确认",
|
|
||||||
data=f"{UPDATE_PROMPT_PREFIX}y",
|
|
||||||
style=1,
|
|
||||||
group_id="update_prompt",
|
|
||||||
),
|
|
||||||
_make_callback_button(
|
|
||||||
btn_id="no",
|
|
||||||
label="✗ 取消",
|
|
||||||
visited_label="已取消",
|
|
||||||
data=f"{UPDATE_PROMPT_PREFIX}n",
|
|
||||||
style=0,
|
|
||||||
group_id="update_prompt",
|
|
||||||
),
|
|
||||||
]),
|
|
||||||
]
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ── ApprovalRequest + text builder ───────────────────────────────────
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class ApprovalRequest:
|
|
||||||
"""Structured approval-request display data.
|
|
||||||
|
|
||||||
:param session_key: Routes the decision back to the waiting caller.
|
|
||||||
:param title: Short title at the top.
|
|
||||||
:param description: Optional longer description.
|
|
||||||
:param command_preview: Command text (exec approvals).
|
|
||||||
:param cwd: Working directory (exec approvals).
|
|
||||||
:param tool_name: Tool name (plugin approvals).
|
|
||||||
:param severity: ``'critical' | 'info' | ''``.
|
|
||||||
:param timeout_sec: Seconds until the approval expires.
|
|
||||||
"""
|
|
||||||
session_key: str
|
|
||||||
title: str
|
|
||||||
description: str = ""
|
|
||||||
command_preview: str = ""
|
|
||||||
cwd: str = ""
|
|
||||||
tool_name: str = ""
|
|
||||||
severity: str = ""
|
|
||||||
timeout_sec: int = 120
|
|
||||||
|
|
||||||
|
|
||||||
def build_approval_text(req: ApprovalRequest) -> str:
|
|
||||||
"""Render an :class:`ApprovalRequest` into the message body (markdown)."""
|
|
||||||
if req.command_preview or req.cwd:
|
|
||||||
return _build_exec_text(req)
|
|
||||||
return _build_plugin_text(req)
|
|
||||||
|
|
||||||
|
|
||||||
def _build_exec_text(req: ApprovalRequest) -> str:
|
|
||||||
lines: List[str] = ["🔐 **命令执行审批**", ""]
|
|
||||||
if req.command_preview:
|
|
||||||
preview = req.command_preview[:300]
|
|
||||||
lines.append(f"```\n{preview}\n```")
|
|
||||||
if req.cwd:
|
|
||||||
lines.append(f"📁 目录: {req.cwd}")
|
|
||||||
if req.title and req.title != req.command_preview:
|
|
||||||
lines.append(f"📋 {req.title}")
|
|
||||||
if req.description:
|
|
||||||
lines.append(f"📝 {req.description}")
|
|
||||||
lines.append("")
|
|
||||||
lines.append(f"⏱️ 超时: {req.timeout_sec} 秒")
|
|
||||||
return "\n".join(lines)
|
|
||||||
|
|
||||||
|
|
||||||
def _build_plugin_text(req: ApprovalRequest) -> str:
|
|
||||||
icon = (
|
|
||||||
"🔴" if req.severity == "critical"
|
|
||||||
else "🔵" if req.severity == "info"
|
|
||||||
else "🟡"
|
|
||||||
)
|
|
||||||
lines: List[str] = [f"{icon} **审批请求**", ""]
|
|
||||||
lines.append(f"📋 {req.title}")
|
|
||||||
if req.description:
|
|
||||||
lines.append(f"📝 {req.description}")
|
|
||||||
if req.tool_name:
|
|
||||||
lines.append(f"🔧 工具: {req.tool_name}")
|
|
||||||
lines.append("")
|
|
||||||
lines.append(f"⏱️ 超时: {req.timeout_sec} 秒")
|
|
||||||
return "\n".join(lines)
|
|
||||||
|
|
||||||
|
|
||||||
# ── ApprovalSender ───────────────────────────────────────────────────
|
|
||||||
|
|
||||||
PostMessageFn = Callable[..., Awaitable[Dict[str, Any]]]
|
|
||||||
"""Signature of an async POST to ``/v2/{users|groups}/{id}/messages``.
|
|
||||||
|
|
||||||
Implementations accept a body dict and return the raw API response.
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
class ApprovalSender:
|
|
||||||
"""Send an approval-request message with an inline keyboard.
|
|
||||||
|
|
||||||
Decoupled from the adapter via callables so it can be unit-tested in
|
|
||||||
isolation. Pass the adapter's ``_send_message_with_keyboard`` helper
|
|
||||||
(or any equivalent) as ``post_message``.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
post_c2c: PostMessageFn,
|
|
||||||
post_group: PostMessageFn,
|
|
||||||
log_tag: str = "QQBot",
|
|
||||||
) -> None:
|
|
||||||
self._post_c2c = post_c2c
|
|
||||||
self._post_group = post_group
|
|
||||||
self._log_tag = log_tag
|
|
||||||
|
|
||||||
async def send(
|
|
||||||
self,
|
|
||||||
chat_type: str,
|
|
||||||
chat_id: str,
|
|
||||||
req: ApprovalRequest,
|
|
||||||
msg_id: Optional[str] = None,
|
|
||||||
) -> bool:
|
|
||||||
"""Send an approval message to *chat_id*.
|
|
||||||
|
|
||||||
:param chat_type: ``'c2c'`` or ``'group'``.
|
|
||||||
:param chat_id: User openid or group openid.
|
|
||||||
:param req: :class:`ApprovalRequest`.
|
|
||||||
:param msg_id: Reply-to message id (required for passive messages).
|
|
||||||
:returns: ``True`` on success, ``False`` on failure.
|
|
||||||
"""
|
|
||||||
text = build_approval_text(req)
|
|
||||||
keyboard = build_approval_keyboard(req.session_key)
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"[%s] Sending approval request to %s:%s (session=%.20s…)",
|
|
||||||
self._log_tag, chat_type, chat_id, req.session_key,
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
if chat_type == "c2c":
|
|
||||||
await self._post_c2c(chat_id, text, msg_id, keyboard)
|
|
||||||
elif chat_type == "group":
|
|
||||||
await self._post_group(chat_id, text, msg_id, keyboard)
|
|
||||||
else:
|
|
||||||
logger.warning(
|
|
||||||
"[%s] Approval: unsupported chat_type %r",
|
|
||||||
self._log_tag, chat_type,
|
|
||||||
)
|
|
||||||
return False
|
|
||||||
logger.info(
|
|
||||||
"[%s] Approval message sent to %s:%s",
|
|
||||||
self._log_tag, chat_type, chat_id,
|
|
||||||
)
|
|
||||||
return True
|
|
||||||
except Exception as exc:
|
|
||||||
logger.error(
|
|
||||||
"[%s] Failed to send approval message to %s:%s: %s",
|
|
||||||
self._log_tag, chat_type, chat_id, exc,
|
|
||||||
)
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
# ── INTERACTION_CREATE event shape ───────────────────────────────────
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class InteractionEvent:
|
|
||||||
"""Parsed ``INTERACTION_CREATE`` event payload.
|
|
||||||
|
|
||||||
See https://bot.q.qq.com/wiki/develop/api-v2/dev-prepare/interface-framework/event-emit.html
|
|
||||||
"""
|
|
||||||
id: str = ""
|
|
||||||
"""Interaction event id — required for the ``PUT /interactions/{id}`` ACK."""
|
|
||||||
|
|
||||||
type: int = 0
|
|
||||||
"""Event type code (``11`` = message button)."""
|
|
||||||
|
|
||||||
chat_type: int = 0
|
|
||||||
"""``0`` = guild, ``1`` = group, ``2`` = c2c."""
|
|
||||||
|
|
||||||
scene: str = ""
|
|
||||||
"""``'guild'`` | ``'group'`` | ``'c2c'`` — human-readable scene."""
|
|
||||||
|
|
||||||
group_openid: str = ""
|
|
||||||
group_member_openid: str = ""
|
|
||||||
user_openid: str = ""
|
|
||||||
channel_id: str = ""
|
|
||||||
guild_id: str = ""
|
|
||||||
|
|
||||||
button_data: str = ""
|
|
||||||
button_id: str = ""
|
|
||||||
resolver_user_id: str = ""
|
|
||||||
|
|
||||||
@property
|
|
||||||
def operator_openid(self) -> str:
|
|
||||||
"""Best available operator openid (group → member; c2c → user)."""
|
|
||||||
return (
|
|
||||||
self.group_member_openid
|
|
||||||
or self.user_openid
|
|
||||||
or self.resolver_user_id
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def parse_interaction_event(raw: Dict[str, Any]) -> InteractionEvent:
|
|
||||||
"""Parse a raw ``INTERACTION_CREATE`` dispatch payload (``d``)."""
|
|
||||||
data_raw = raw.get("data") or {}
|
|
||||||
resolved = data_raw.get("resolved") or {}
|
|
||||||
scene_code = int(raw.get("chat_type", 0) or 0)
|
|
||||||
scene = {0: "guild", 1: "group", 2: "c2c"}.get(scene_code, "")
|
|
||||||
return InteractionEvent(
|
|
||||||
id=str(raw.get("id", "")),
|
|
||||||
type=int(data_raw.get("type", 0) or 0),
|
|
||||||
chat_type=scene_code,
|
|
||||||
scene=scene,
|
|
||||||
group_openid=str(raw.get("group_openid", "")),
|
|
||||||
group_member_openid=str(raw.get("group_member_openid", "")),
|
|
||||||
user_openid=str(raw.get("user_openid", "")),
|
|
||||||
channel_id=str(raw.get("channel_id", "")),
|
|
||||||
guild_id=str(raw.get("guild_id", "")),
|
|
||||||
button_data=str(resolved.get("button_data", "")),
|
|
||||||
button_id=str(resolved.get("button_id", "")),
|
|
||||||
resolver_user_id=str(resolved.get("user_id", "")),
|
|
||||||
)
|
|
||||||
@@ -1887,12 +1887,6 @@ class SlackAdapter(BasePlatformAdapter):
|
|||||||
is_thread_reply = bool(event_thread_ts and event_thread_ts != ts)
|
is_thread_reply = bool(event_thread_ts and event_thread_ts != ts)
|
||||||
|
|
||||||
if not is_dm and bot_uid:
|
if not is_dm and bot_uid:
|
||||||
# Check allowed channels — if set, only respond in these channels (whitelist)
|
|
||||||
allowed_channels = self._slack_allowed_channels()
|
|
||||||
if allowed_channels and channel_id not in allowed_channels:
|
|
||||||
logger.debug("[Slack] Ignoring message in non-allowed channel: %s", channel_id)
|
|
||||||
return
|
|
||||||
|
|
||||||
if channel_id in self._slack_free_response_channels():
|
if channel_id in self._slack_free_response_channels():
|
||||||
pass # Free-response channel — always process
|
pass # Free-response channel — always process
|
||||||
elif not self._slack_require_mention():
|
elif not self._slack_require_mention():
|
||||||
@@ -2930,19 +2924,3 @@ class SlackAdapter(BasePlatformAdapter):
|
|||||||
if s:
|
if s:
|
||||||
return {part.strip() for part in s.split(",") if part.strip()}
|
return {part.strip() for part in s.split(",") if part.strip()}
|
||||||
return set()
|
return set()
|
||||||
|
|
||||||
def _slack_allowed_channels(self) -> set:
|
|
||||||
"""Return the whitelist of channel IDs the bot will respond in.
|
|
||||||
|
|
||||||
When non-empty, messages from channels NOT in this set are silently
|
|
||||||
ignored — even if the bot is @mentioned. DMs are never filtered.
|
|
||||||
Empty set means no restriction (fully backward compatible).
|
|
||||||
"""
|
|
||||||
raw = self.config.extra.get("allowed_channels")
|
|
||||||
if raw is None:
|
|
||||||
raw = os.getenv("SLACK_ALLOWED_CHANNELS", "")
|
|
||||||
if isinstance(raw, list):
|
|
||||||
return {str(part).strip() for part in raw if str(part).strip()}
|
|
||||||
if isinstance(raw, str) and raw.strip():
|
|
||||||
return {part.strip() for part in raw.split(",") if part.strip()}
|
|
||||||
return set()
|
|
||||||
|
|||||||
+21
-104
@@ -86,22 +86,6 @@ from gateway.platforms.telegram_network import (
|
|||||||
)
|
)
|
||||||
from utils import atomic_replace
|
from utils import atomic_replace
|
||||||
|
|
||||||
_TELEGRAM_IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".gif"}
|
|
||||||
_TELEGRAM_IMAGE_MIME_TO_EXT = {
|
|
||||||
"image/png": ".png",
|
|
||||||
"image/jpeg": ".jpg",
|
|
||||||
"image/jpg": ".jpg",
|
|
||||||
"image/webp": ".webp",
|
|
||||||
"image/gif": ".gif",
|
|
||||||
}
|
|
||||||
_TELEGRAM_IMAGE_EXT_TO_MIME = {
|
|
||||||
".png": "image/png",
|
|
||||||
".jpg": "image/jpeg",
|
|
||||||
".jpeg": "image/jpeg",
|
|
||||||
".webp": "image/webp",
|
|
||||||
".gif": "image/gif",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def check_telegram_requirements() -> bool:
|
def check_telegram_requirements() -> bool:
|
||||||
"""Check if Telegram dependencies are available."""
|
"""Check if Telegram dependencies are available."""
|
||||||
@@ -369,14 +353,10 @@ class TelegramAdapter(BasePlatformAdapter):
|
|||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _message_thread_id_for_typing(cls, thread_id: Optional[str]) -> Optional[int]:
|
def _message_thread_id_for_typing(cls, thread_id: Optional[str]) -> Optional[int]:
|
||||||
# Asymmetric with _message_thread_id_for_send on purpose. Telegram's
|
# Mirrors _message_thread_id_for_send: the General forum topic (thread id
|
||||||
# sendMessage and sendChatAction treat thread id "1" (the forum General
|
# "1") is represented as "no thread id" on the wire. User-created topics
|
||||||
# topic) differently: sends reject message_thread_id=1 and must omit it,
|
# keep their real id so typing stays scoped to that topic.
|
||||||
# but sendChatAction needs message_thread_id=1 to place the typing
|
if not thread_id or str(thread_id) == cls._GENERAL_TOPIC_THREAD_ID:
|
||||||
# bubble in the General topic (omitting it hides the bubble entirely
|
|
||||||
# from the client's view of that topic). Preserve the real id here —
|
|
||||||
# sends still map "1" → None via _message_thread_id_for_send.
|
|
||||||
if not thread_id:
|
|
||||||
return None
|
return None
|
||||||
return int(thread_id)
|
return int(thread_id)
|
||||||
|
|
||||||
@@ -744,7 +724,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
|||||||
return
|
return
|
||||||
|
|
||||||
import yaml as _yaml
|
import yaml as _yaml
|
||||||
with open(config_path, "r", encoding="utf-8") as f:
|
with open(config_path, "r") as f:
|
||||||
config = _yaml.safe_load(f) or {}
|
config = _yaml.safe_load(f) or {}
|
||||||
|
|
||||||
# Navigate to platforms.telegram.extra.dm_topics
|
# Navigate to platforms.telegram.extra.dm_topics
|
||||||
@@ -2775,20 +2755,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
|||||||
return {str(part).strip() for part in raw if str(part).strip()}
|
return {str(part).strip() for part in raw if str(part).strip()}
|
||||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
||||||
|
|
||||||
def _telegram_allowed_chats(self) -> set[str]:
|
|
||||||
"""Return the whitelist of group/supergroup chat IDs the bot will respond in.
|
|
||||||
|
|
||||||
When non-empty, group messages from chats NOT in this set are silently
|
|
||||||
ignored — even if the bot is @mentioned. DMs are never filtered.
|
|
||||||
Empty set means no restriction (fully backward compatible).
|
|
||||||
"""
|
|
||||||
raw = self.config.extra.get("allowed_chats")
|
|
||||||
if raw is None:
|
|
||||||
raw = os.getenv("TELEGRAM_ALLOWED_CHATS", "")
|
|
||||||
if isinstance(raw, list):
|
|
||||||
return {str(part).strip() for part in raw if str(part).strip()}
|
|
||||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
|
||||||
|
|
||||||
def _telegram_ignored_threads(self) -> set[int]:
|
def _telegram_ignored_threads(self) -> set[int]:
|
||||||
raw = self.config.extra.get("ignored_threads")
|
raw = self.config.extra.get("ignored_threads")
|
||||||
if raw is None:
|
if raw is None:
|
||||||
@@ -2937,16 +2903,13 @@ class TelegramAdapter(BasePlatformAdapter):
|
|||||||
"""Apply Telegram group trigger rules.
|
"""Apply Telegram group trigger rules.
|
||||||
|
|
||||||
DMs remain unrestricted. Group/supergroup messages are accepted when:
|
DMs remain unrestricted. Group/supergroup messages are accepted when:
|
||||||
- the chat passes the ``allowed_chats`` whitelist (when set)
|
|
||||||
- the chat is explicitly allowlisted in ``free_response_chats``
|
- the chat is explicitly allowlisted in ``free_response_chats``
|
||||||
- ``require_mention`` is disabled
|
- ``require_mention`` is disabled
|
||||||
- the message replies to the bot
|
- the message replies to the bot
|
||||||
- the bot is @mentioned
|
- the bot is @mentioned
|
||||||
- the text/caption matches a configured regex wake-word pattern
|
- the text/caption matches a configured regex wake-word pattern
|
||||||
|
|
||||||
When ``allowed_chats`` is non-empty, it acts as a hard gate — messages
|
When ``require_mention`` is enabled, slash commands are not given
|
||||||
from any chat not in the list are ignored regardless of the other
|
|
||||||
rules. When ``require_mention`` is enabled, slash commands are not given
|
|
||||||
special treatment — they must pass the same mention/reply checks
|
special treatment — they must pass the same mention/reply checks
|
||||||
as any other group message. Users can still trigger commands via
|
as any other group message. Users can still trigger commands via
|
||||||
the Telegram bot menu (``/command@botname``) or by explicitly
|
the Telegram bot menu (``/command@botname``) or by explicitly
|
||||||
@@ -2955,14 +2918,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
|||||||
"""
|
"""
|
||||||
if not self._is_group_chat(message):
|
if not self._is_group_chat(message):
|
||||||
return True
|
return True
|
||||||
# allowed_chats check (whitelist — must pass before other gating).
|
|
||||||
# When set, group messages from chats NOT in this whitelist are
|
|
||||||
# silently ignored, even if @mentioned. DMs are already excluded above.
|
|
||||||
allowed = self._telegram_allowed_chats()
|
|
||||||
if allowed:
|
|
||||||
chat_id_str = str(getattr(getattr(message, "chat", None), "id", ""))
|
|
||||||
if chat_id_str not in allowed:
|
|
||||||
return False
|
|
||||||
thread_id = getattr(message, "message_thread_id", None)
|
thread_id = getattr(message, "message_thread_id", None)
|
||||||
if thread_id is not None:
|
if thread_id is not None:
|
||||||
try:
|
try:
|
||||||
@@ -3284,59 +3239,10 @@ class TelegramAdapter(BasePlatformAdapter):
|
|||||||
_, ext = os.path.splitext(original_filename)
|
_, ext = os.path.splitext(original_filename)
|
||||||
ext = ext.lower()
|
ext = ext.lower()
|
||||||
|
|
||||||
# Normalize mime_type for robust comparisons (some clients send
|
|
||||||
# uppercase like "IMAGE/PNG").
|
|
||||||
doc_mime = (doc.mime_type or "").lower()
|
|
||||||
|
|
||||||
# If no extension from filename, reverse-lookup from MIME type
|
# If no extension from filename, reverse-lookup from MIME type
|
||||||
if not ext and doc_mime:
|
if not ext and doc.mime_type:
|
||||||
ext = _TELEGRAM_IMAGE_MIME_TO_EXT.get(doc_mime, "")
|
mime_to_ext = {v: k for k, v in SUPPORTED_DOCUMENT_TYPES.items()}
|
||||||
if not ext:
|
ext = mime_to_ext.get(doc.mime_type, "")
|
||||||
mime_to_ext = {v: k for k, v in SUPPORTED_DOCUMENT_TYPES.items()}
|
|
||||||
ext = mime_to_ext.get(doc_mime, "")
|
|
||||||
|
|
||||||
# Check file size early so image documents cannot bypass the
|
|
||||||
# document size limit by taking the image path.
|
|
||||||
MAX_DOC_BYTES = 20 * 1024 * 1024
|
|
||||||
if not doc.file_size or doc.file_size > MAX_DOC_BYTES:
|
|
||||||
event.text = (
|
|
||||||
"The document is too large or its size could not be verified. "
|
|
||||||
"Maximum: 20 MB."
|
|
||||||
)
|
|
||||||
logger.info("[Telegram] Document too large: %s bytes", doc.file_size)
|
|
||||||
await self.handle_message(event)
|
|
||||||
return
|
|
||||||
|
|
||||||
# Telegram may deliver screenshots/photos as documents. If the
|
|
||||||
# payload is actually an image, route it through the image cache
|
|
||||||
# and batching path instead of rejecting it as a document.
|
|
||||||
if ext in _TELEGRAM_IMAGE_EXTENSIONS or doc_mime.startswith("image/"):
|
|
||||||
file_obj = await doc.get_file()
|
|
||||||
image_bytes = await file_obj.download_as_bytearray()
|
|
||||||
image_ext = ext if ext in _TELEGRAM_IMAGE_EXTENSIONS else _TELEGRAM_IMAGE_MIME_TO_EXT.get(doc_mime, ".jpg")
|
|
||||||
try:
|
|
||||||
cached_path = cache_image_from_bytes(bytes(image_bytes), ext=image_ext)
|
|
||||||
except ValueError as e:
|
|
||||||
logger.warning("[Telegram] Failed to cache image document: %s", e, exc_info=True)
|
|
||||||
event.text = (
|
|
||||||
f"Image document '{original_filename or doc_mime or ext or 'unknown'}' "
|
|
||||||
"could not be read as an image."
|
|
||||||
)
|
|
||||||
await self.handle_message(event)
|
|
||||||
return
|
|
||||||
|
|
||||||
event.message_type = MessageType.PHOTO
|
|
||||||
event.media_urls = [cached_path]
|
|
||||||
event.media_types = [doc_mime if doc_mime.startswith("image/") else _TELEGRAM_IMAGE_EXT_TO_MIME.get(image_ext, "image/jpeg")]
|
|
||||||
logger.info("[Telegram] Cached user image-document at %s", cached_path)
|
|
||||||
|
|
||||||
media_group_id = getattr(msg, "media_group_id", None)
|
|
||||||
if media_group_id:
|
|
||||||
await self._queue_media_group_event(str(media_group_id), event)
|
|
||||||
else:
|
|
||||||
batch_key = self._photo_batch_key(event, msg)
|
|
||||||
self._enqueue_photo_event(batch_key, event)
|
|
||||||
return
|
|
||||||
|
|
||||||
if not ext and doc.mime_type:
|
if not ext and doc.mime_type:
|
||||||
video_mime_to_ext = {v: k for k, v in SUPPORTED_VIDEO_TYPES.items()}
|
video_mime_to_ext = {v: k for k, v in SUPPORTED_VIDEO_TYPES.items()}
|
||||||
@@ -3364,6 +3270,17 @@ class TelegramAdapter(BasePlatformAdapter):
|
|||||||
await self.handle_message(event)
|
await self.handle_message(event)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
# Check file size (Telegram Bot API limit: 20 MB)
|
||||||
|
MAX_DOC_BYTES = 20 * 1024 * 1024
|
||||||
|
if not doc.file_size or doc.file_size > MAX_DOC_BYTES:
|
||||||
|
event.text = (
|
||||||
|
"The document is too large or its size could not be verified. "
|
||||||
|
"Maximum: 20 MB."
|
||||||
|
)
|
||||||
|
logger.info("[Telegram] Document too large: %s bytes", doc.file_size)
|
||||||
|
await self.handle_message(event)
|
||||||
|
return
|
||||||
|
|
||||||
# Download and cache
|
# Download and cache
|
||||||
file_obj = await doc.get_file()
|
file_obj = await doc.get_file()
|
||||||
doc_bytes = await file_obj.download_as_bytearray()
|
doc_bytes = await file_obj.download_as_bytearray()
|
||||||
@@ -3516,7 +3433,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
|||||||
return
|
return
|
||||||
|
|
||||||
import yaml as _yaml
|
import yaml as _yaml
|
||||||
with open(config_path, "r", encoding="utf-8") as f:
|
with open(config_path, "r") as f:
|
||||||
config = _yaml.safe_load(f) or {}
|
config = _yaml.safe_load(f) or {}
|
||||||
|
|
||||||
dm_topics = (
|
dm_topics = (
|
||||||
|
|||||||
@@ -59,29 +59,6 @@ DEFAULT_PORT = 8644
|
|||||||
_INSECURE_NO_AUTH = "INSECURE_NO_AUTH"
|
_INSECURE_NO_AUTH = "INSECURE_NO_AUTH"
|
||||||
_DYNAMIC_ROUTES_FILENAME = "webhook_subscriptions.json"
|
_DYNAMIC_ROUTES_FILENAME = "webhook_subscriptions.json"
|
||||||
|
|
||||||
# Hostnames/IP literals that only serve connections originating on the same
|
|
||||||
# machine. Anything else is treated as a public bind for safety-rail purposes.
|
|
||||||
_LOOPBACK_HOSTS = frozenset({
|
|
||||||
"127.0.0.1",
|
|
||||||
"localhost",
|
|
||||||
"::1",
|
|
||||||
"ip6-localhost",
|
|
||||||
"ip6-loopback",
|
|
||||||
})
|
|
||||||
|
|
||||||
|
|
||||||
def _is_loopback_host(host: str) -> bool:
|
|
||||||
"""True when `host` binds only to the local machine.
|
|
||||||
|
|
||||||
Covers IPv4 loopback, the standard `localhost` alias, IPv6 loopback in
|
|
||||||
both bracketed and bare form, and the common Debian-style aliases. Any
|
|
||||||
falsy value (empty string, None) is conservatively treated as non-loopback
|
|
||||||
because an unset host usually means the platform-default public bind.
|
|
||||||
"""
|
|
||||||
if not host:
|
|
||||||
return False
|
|
||||||
return host.strip().lower() in _LOOPBACK_HOSTS
|
|
||||||
|
|
||||||
|
|
||||||
def check_webhook_requirements() -> bool:
|
def check_webhook_requirements() -> bool:
|
||||||
"""Check if webhook adapter dependencies are available."""
|
"""Check if webhook adapter dependencies are available."""
|
||||||
@@ -149,17 +126,6 @@ class WebhookAdapter(BasePlatformAdapter):
|
|||||||
f"For testing without auth, set secret to '{_INSECURE_NO_AUTH}'."
|
f"For testing without auth, set secret to '{_INSECURE_NO_AUTH}'."
|
||||||
)
|
)
|
||||||
|
|
||||||
# Safety rail: refuse to start if INSECURE_NO_AUTH is combined with a
|
|
||||||
# non-loopback bind. The escape hatch is for local testing only;
|
|
||||||
# serving an unauthenticated route on a public interface is a
|
|
||||||
# deployment-grade footgun we'd rather crash early than ship.
|
|
||||||
if secret == _INSECURE_NO_AUTH and not _is_loopback_host(self._host):
|
|
||||||
raise ValueError(
|
|
||||||
f"[webhook] Route '{name}' uses INSECURE_NO_AUTH secret "
|
|
||||||
f"but is bound to non-loopback host '{self._host}'. "
|
|
||||||
f"INSECURE_NO_AUTH is for local testing only. "
|
|
||||||
f"Refusing to start to prevent accidental exposure."
|
|
||||||
)
|
|
||||||
# deliver_only routes bypass the agent — the POST body becomes a
|
# deliver_only routes bypass the agent — the POST body becomes a
|
||||||
# direct push notification via the configured delivery target.
|
# direct push notification via the configured delivery target.
|
||||||
# Validate up-front so misconfiguration surfaces at startup rather
|
# Validate up-front so misconfiguration surfaces at startup rather
|
||||||
|
|||||||
@@ -37,7 +37,6 @@ import logging
|
|||||||
import mimetypes
|
import mimetypes
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import time
|
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -1563,11 +1562,12 @@ def qr_scan_for_bot_info(
|
|||||||
print(" Fetching configuration results...", end="", flush=True)
|
print(" Fetching configuration results...", end="", flush=True)
|
||||||
|
|
||||||
# ── Step 3: Poll for result ──
|
# ── Step 3: Poll for result ──
|
||||||
deadline = time.monotonic() + timeout_seconds
|
import time
|
||||||
|
deadline = time.time() + timeout_seconds
|
||||||
query_url = f"{_QR_QUERY_URL}?scode={urllib.parse.quote(scode)}"
|
query_url = f"{_QR_QUERY_URL}?scode={urllib.parse.quote(scode)}"
|
||||||
poll_count = 0
|
poll_count = 0
|
||||||
|
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
try:
|
try:
|
||||||
req = urllib.request.Request(query_url, headers={"User-Agent": "HermesAgent/1.0"})
|
req = urllib.request.Request(query_url, headers={"User-Agent": "HermesAgent/1.0"})
|
||||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||||
|
|||||||
+22
-81
@@ -23,7 +23,6 @@ import re
|
|||||||
import secrets
|
import secrets
|
||||||
import struct
|
import struct
|
||||||
import tempfile
|
import tempfile
|
||||||
import textwrap
|
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
@@ -33,8 +32,6 @@ from urllib.parse import quote, urlparse
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
WEIXIN_COPY_LINE_WIDTH = 120
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
import aiohttp
|
import aiohttp
|
||||||
|
|
||||||
@@ -551,21 +548,17 @@ async def _upload_ciphertext(
|
|||||||
Accepts either a constructed CDN URL (from upload_param) or a direct
|
Accepts either a constructed CDN URL (from upload_param) or a direct
|
||||||
upload_full_url — both use POST with the raw ciphertext as the body.
|
upload_full_url — both use POST with the raw ciphertext as the body.
|
||||||
"""
|
"""
|
||||||
# Use asyncio.wait_for() instead of aiohttp ClientTimeout to avoid
|
timeout = aiohttp.ClientTimeout(total=120)
|
||||||
# "Timeout context manager should be used inside a task" errors when
|
async with session.post(upload_url, data=ciphertext, headers={"Content-Type": "application/octet-stream"}, timeout=timeout) as response:
|
||||||
# invoked via asyncio.run_coroutine_threadsafe() from cron jobs.
|
if response.status == 200:
|
||||||
async def _do_upload() -> str:
|
encrypted_param = response.headers.get("x-encrypted-param")
|
||||||
async with session.post(upload_url, data=ciphertext, headers={"Content-Type": "application/octet-stream"}) as response:
|
if encrypted_param:
|
||||||
if response.status == 200:
|
await response.read()
|
||||||
encrypted_param = response.headers.get("x-encrypted-param")
|
return encrypted_param
|
||||||
if encrypted_param:
|
|
||||||
await response.read()
|
|
||||||
return encrypted_param
|
|
||||||
raw = await response.text()
|
|
||||||
raise RuntimeError(f"CDN upload missing x-encrypted-param header: {raw[:200]}")
|
|
||||||
raw = await response.text()
|
raw = await response.text()
|
||||||
raise RuntimeError(f"CDN upload HTTP {response.status}: {raw[:200]}")
|
raise RuntimeError(f"CDN upload missing x-encrypted-param header: {raw[:200]}")
|
||||||
return await asyncio.wait_for(_do_upload(), timeout=120)
|
raw = await response.text()
|
||||||
|
raise RuntimeError(f"CDN upload HTTP {response.status}: {raw[:200]}")
|
||||||
|
|
||||||
|
|
||||||
async def _download_bytes(
|
async def _download_bytes(
|
||||||
@@ -574,13 +567,10 @@ async def _download_bytes(
|
|||||||
url: str,
|
url: str,
|
||||||
timeout_seconds: float = 60.0,
|
timeout_seconds: float = 60.0,
|
||||||
) -> bytes:
|
) -> bytes:
|
||||||
# Use asyncio.wait_for() instead of aiohttp ClientTimeout to avoid
|
timeout = aiohttp.ClientTimeout(total=timeout_seconds)
|
||||||
# "Timeout context manager should be used inside a task" errors.
|
async with session.get(url, timeout=timeout) as response:
|
||||||
async def _do_download() -> bytes:
|
response.raise_for_status()
|
||||||
async with session.get(url) as response:
|
return await response.read()
|
||||||
response.raise_for_status()
|
|
||||||
return await response.read()
|
|
||||||
return await asyncio.wait_for(_do_download(), timeout=timeout_seconds)
|
|
||||||
|
|
||||||
|
|
||||||
_WEIXIN_CDN_ALLOWLIST: frozenset[str] = frozenset(
|
_WEIXIN_CDN_ALLOWLIST: frozenset[str] = frozenset(
|
||||||
@@ -734,46 +724,6 @@ def _normalize_markdown_blocks(content: str) -> str:
|
|||||||
return "\n".join(result).strip()
|
return "\n".join(result).strip()
|
||||||
|
|
||||||
|
|
||||||
def _wrap_copy_friendly_lines_for_weixin(content: str) -> str:
|
|
||||||
"""Wrap long display lines that are hard to copy in WeChat clients."""
|
|
||||||
if not content:
|
|
||||||
return content
|
|
||||||
|
|
||||||
wrapped: List[str] = []
|
|
||||||
in_code_block = False
|
|
||||||
|
|
||||||
for raw_line in content.splitlines():
|
|
||||||
line = raw_line.rstrip()
|
|
||||||
stripped = line.strip()
|
|
||||||
|
|
||||||
if _FENCE_RE.match(stripped):
|
|
||||||
in_code_block = not in_code_block
|
|
||||||
wrapped.append(line)
|
|
||||||
continue
|
|
||||||
|
|
||||||
if (
|
|
||||||
in_code_block
|
|
||||||
or len(line) <= WEIXIN_COPY_LINE_WIDTH
|
|
||||||
or not stripped
|
|
||||||
or stripped.startswith("|")
|
|
||||||
or _TABLE_RULE_RE.match(stripped)
|
|
||||||
):
|
|
||||||
wrapped.append(line)
|
|
||||||
continue
|
|
||||||
|
|
||||||
wrapped_lines = textwrap.wrap(
|
|
||||||
line,
|
|
||||||
width=WEIXIN_COPY_LINE_WIDTH,
|
|
||||||
break_long_words=False,
|
|
||||||
break_on_hyphens=False,
|
|
||||||
replace_whitespace=False,
|
|
||||||
drop_whitespace=True,
|
|
||||||
)
|
|
||||||
wrapped.extend(wrapped_lines or [line])
|
|
||||||
|
|
||||||
return "\n".join(wrapped).strip()
|
|
||||||
|
|
||||||
|
|
||||||
def _split_markdown_blocks(content: str) -> List[str]:
|
def _split_markdown_blocks(content: str) -> List[str]:
|
||||||
if not content:
|
if not content:
|
||||||
return []
|
return []
|
||||||
@@ -1087,11 +1037,11 @@ async def qr_login(
|
|||||||
except Exception as _qr_exc:
|
except Exception as _qr_exc:
|
||||||
print(f"(终端二维码渲染失败: {_qr_exc},请直接打开上面的二维码链接)")
|
print(f"(终端二维码渲染失败: {_qr_exc},请直接打开上面的二维码链接)")
|
||||||
|
|
||||||
deadline = time.monotonic() + timeout_seconds
|
deadline = time.time() + timeout_seconds
|
||||||
current_base_url = ILINK_BASE_URL
|
current_base_url = ILINK_BASE_URL
|
||||||
refresh_count = 0
|
refresh_count = 0
|
||||||
|
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
try:
|
try:
|
||||||
status_resp = await _api_get(
|
status_resp = await _api_get(
|
||||||
session,
|
session,
|
||||||
@@ -1266,12 +1216,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
|||||||
logger.debug("[%s] Token lock unavailable (non-fatal): %s", self.name, exc)
|
logger.debug("[%s] Token lock unavailable (non-fatal): %s", self.name, exc)
|
||||||
|
|
||||||
self._poll_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
self._poll_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
||||||
# Disable aiohttp's built-in ClientTimeout (total=None) to prevent
|
self._send_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
||||||
# "Timeout context manager should be used inside a task" errors when
|
|
||||||
# send() is invoked via asyncio.run_coroutine_threadsafe() from cron.
|
|
||||||
# Timeout is managed externally via asyncio.wait_for() in _api_post/_api_get.
|
|
||||||
_no_aiohttp_timeout = aiohttp.ClientTimeout(total=None, connect=None, sock_connect=None, sock_read=None)
|
|
||||||
self._send_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector(), timeout=_no_aiohttp_timeout)
|
|
||||||
self._token_store.restore(self._account_id)
|
self._token_store.restore(self._account_id)
|
||||||
self._poll_task = asyncio.create_task(self._poll_loop(), name="weixin-poll")
|
self._poll_task = asyncio.create_task(self._poll_loop(), name="weixin-poll")
|
||||||
self._mark_connected()
|
self._mark_connected()
|
||||||
@@ -1879,14 +1824,10 @@ class WeixinAdapter(BasePlatformAdapter):
|
|||||||
raise ValueError(f"Blocked unsafe URL (SSRF protection): {url}")
|
raise ValueError(f"Blocked unsafe URL (SSRF protection): {url}")
|
||||||
|
|
||||||
assert self._send_session is not None
|
assert self._send_session is not None
|
||||||
# Use asyncio.wait_for() instead of aiohttp ClientTimeout to avoid
|
async with self._send_session.get(url, timeout=aiohttp.ClientTimeout(total=30)) as response:
|
||||||
# "Timeout context manager should be used inside a task" errors.
|
response.raise_for_status()
|
||||||
async def _do_fetch():
|
data = await response.read()
|
||||||
async with self._send_session.get(url) as response:
|
suffix = Path(url.split("?", 1)[0]).suffix or ".bin"
|
||||||
response.raise_for_status()
|
|
||||||
return await response.read()
|
|
||||||
data = await asyncio.wait_for(_do_fetch(), timeout=30)
|
|
||||||
suffix = Path(url.split("?", 1)[0]).suffix or ".bin"
|
|
||||||
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as handle:
|
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as handle:
|
||||||
handle.write(data)
|
handle.write(data)
|
||||||
return handle.name
|
return handle.name
|
||||||
@@ -2065,7 +2006,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
|||||||
def format_message(self, content: Optional[str]) -> str:
|
def format_message(self, content: Optional[str]) -> str:
|
||||||
if content is None:
|
if content is None:
|
||||||
return ""
|
return ""
|
||||||
return _wrap_copy_friendly_lines_for_weixin(_normalize_markdown_blocks(content))
|
return _normalize_markdown_blocks(content)
|
||||||
|
|
||||||
|
|
||||||
async def send_weixin_direct(
|
async def send_weixin_direct(
|
||||||
|
|||||||
+13
-110
@@ -21,8 +21,6 @@ import logging
|
|||||||
import os
|
import os
|
||||||
import platform
|
import platform
|
||||||
import re
|
import re
|
||||||
import shutil
|
|
||||||
import signal
|
|
||||||
import subprocess
|
import subprocess
|
||||||
|
|
||||||
_IS_WINDOWS = platform.system() == "Windows"
|
_IS_WINDOWS = platform.system() == "Windows"
|
||||||
@@ -56,77 +54,19 @@ def _kill_port_process(port: int) -> None:
|
|||||||
except subprocess.SubprocessError:
|
except subprocess.SubprocessError:
|
||||||
pass
|
pass
|
||||||
else:
|
else:
|
||||||
# Try fuser first (Linux), fall back to lsof (macOS / WSL2)
|
result = subprocess.run(
|
||||||
killed = False
|
["fuser", f"{port}/tcp"],
|
||||||
try:
|
capture_output=True, timeout=5,
|
||||||
result = subprocess.run(
|
)
|
||||||
["fuser", f"{port}/tcp"],
|
if result.returncode == 0:
|
||||||
|
subprocess.run(
|
||||||
|
["fuser", "-k", f"{port}/tcp"],
|
||||||
capture_output=True, timeout=5,
|
capture_output=True, timeout=5,
|
||||||
)
|
)
|
||||||
if result.returncode == 0:
|
|
||||||
subprocess.run(
|
|
||||||
["fuser", "-k", f"{port}/tcp"],
|
|
||||||
capture_output=True, timeout=5,
|
|
||||||
)
|
|
||||||
killed = True
|
|
||||||
except FileNotFoundError:
|
|
||||||
pass # fuser not installed
|
|
||||||
|
|
||||||
if not killed:
|
|
||||||
try:
|
|
||||||
result = subprocess.run(
|
|
||||||
["lsof", "-ti", f":{port}"],
|
|
||||||
capture_output=True, text=True, timeout=5,
|
|
||||||
)
|
|
||||||
for pid_str in result.stdout.strip().splitlines():
|
|
||||||
try:
|
|
||||||
os.kill(int(pid_str), signal.SIGTERM)
|
|
||||||
except (ValueError, ProcessLookupError, PermissionError):
|
|
||||||
pass
|
|
||||||
except FileNotFoundError:
|
|
||||||
pass # lsof not installed either
|
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
def _kill_stale_bridge_by_pidfile(session_path: Path) -> None:
|
|
||||||
"""Kill a bridge process recorded in a PID file from a previous run.
|
|
||||||
|
|
||||||
The bridge writes ``bridge.pid`` into the session directory when it
|
|
||||||
starts. If the gateway crashed without a clean shutdown the old bridge
|
|
||||||
process becomes orphaned — this helper finds and kills it.
|
|
||||||
"""
|
|
||||||
pid_file = session_path / "bridge.pid"
|
|
||||||
if not pid_file.exists():
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
pid = int(pid_file.read_text().strip())
|
|
||||||
except (ValueError, OSError, TypeError):
|
|
||||||
try:
|
|
||||||
pid_file.unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
os.kill(pid, 0) # check existence
|
|
||||||
os.kill(pid, signal.SIGTERM)
|
|
||||||
logger.info("[whatsapp] Killed stale bridge PID %d from pidfile", pid)
|
|
||||||
except (ProcessLookupError, PermissionError, OSError):
|
|
||||||
pass
|
|
||||||
try:
|
|
||||||
pid_file.unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _write_bridge_pidfile(session_path: Path, pid: int) -> None:
|
|
||||||
"""Write the bridge PID to a file for later cleanup."""
|
|
||||||
try:
|
|
||||||
(session_path / "bridge.pid").write_text(str(pid))
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _terminate_bridge_process(proc, *, force: bool = False) -> None:
|
def _terminate_bridge_process(proc, *, force: bool = False) -> None:
|
||||||
"""Terminate the bridge process using process-tree semantics where possible."""
|
"""Terminate the bridge process using process-tree semantics where possible."""
|
||||||
if _IS_WINDOWS:
|
if _IS_WINDOWS:
|
||||||
@@ -178,15 +118,10 @@ def check_whatsapp_requirements() -> bool:
|
|||||||
|
|
||||||
WhatsApp requires a Node.js bridge for most implementations.
|
WhatsApp requires a Node.js bridge for most implementations.
|
||||||
"""
|
"""
|
||||||
# Check for Node.js. Resolve via shutil.which so we respect PATHEXT
|
# Check for Node.js
|
||||||
# (node.exe vs node) and get a meaningful "not installed" signal
|
|
||||||
# instead of spawning a cmd flash on Windows.
|
|
||||||
_node = shutil.which("node")
|
|
||||||
if not _node:
|
|
||||||
return False
|
|
||||||
try:
|
try:
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
[_node, "--version"],
|
["node", "--version"],
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
text=True,
|
text=True,
|
||||||
timeout=5
|
timeout=5
|
||||||
@@ -223,7 +158,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
# WhatsApp message limits — practical UX limit, not protocol max.
|
# WhatsApp message limits — practical UX limit, not protocol max.
|
||||||
# WhatsApp allows ~65K but long messages are unreadable on mobile.
|
# WhatsApp allows ~65K but long messages are unreadable on mobile.
|
||||||
MAX_MESSAGE_LENGTH = 4096
|
MAX_MESSAGE_LENGTH = 4096
|
||||||
DEFAULT_REPLY_PREFIX = "⚕ *Hermes Agent*\n────────────\n"
|
|
||||||
|
|
||||||
# Default bridge location relative to the hermes-agent install
|
# Default bridge location relative to the hermes-agent install
|
||||||
_DEFAULT_BRIDGE_DIR = Path(__file__).resolve().parents[2] / "scripts" / "whatsapp-bridge"
|
_DEFAULT_BRIDGE_DIR = Path(__file__).resolve().parents[2] / "scripts" / "whatsapp-bridge"
|
||||||
@@ -259,25 +193,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
# notification before the normal "✓ whatsapp disconnected" fires.
|
# notification before the normal "✓ whatsapp disconnected" fires.
|
||||||
self._shutting_down: bool = False
|
self._shutting_down: bool = False
|
||||||
|
|
||||||
def _effective_reply_prefix(self) -> str:
|
|
||||||
"""Return the prefix the Node bridge will add in self-chat mode."""
|
|
||||||
whatsapp_mode = os.getenv("WHATSAPP_MODE", "self-chat")
|
|
||||||
if whatsapp_mode != "self-chat":
|
|
||||||
return ""
|
|
||||||
if self._reply_prefix is not None:
|
|
||||||
return self._reply_prefix.replace("\\n", "\n")
|
|
||||||
env_prefix = os.getenv("WHATSAPP_REPLY_PREFIX")
|
|
||||||
if env_prefix is not None:
|
|
||||||
return env_prefix.replace("\\n", "\n")
|
|
||||||
return self.DEFAULT_REPLY_PREFIX
|
|
||||||
|
|
||||||
def _outgoing_chunk_limit(self) -> int:
|
|
||||||
"""Reserve room for the bridge-side prefix so final WhatsApp text fits."""
|
|
||||||
prefix_len = len(self._effective_reply_prefix())
|
|
||||||
# Keep enough space for truncate_message's pagination indicator and
|
|
||||||
# code-fence repair even if a user configures a very long prefix.
|
|
||||||
return max(1024, self.MAX_MESSAGE_LENGTH - prefix_len)
|
|
||||||
|
|
||||||
def _whatsapp_require_mention(self) -> bool:
|
def _whatsapp_require_mention(self) -> bool:
|
||||||
configured = self.config.extra.get("require_mention")
|
configured = self.config.extra.get("require_mention")
|
||||||
if configured is not None:
|
if configured is not None:
|
||||||
@@ -470,13 +385,9 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
bridge_dir = bridge_path.parent
|
bridge_dir = bridge_path.parent
|
||||||
if not (bridge_dir / "node_modules").exists():
|
if not (bridge_dir / "node_modules").exists():
|
||||||
print(f"[{self.name}] Installing WhatsApp bridge dependencies...")
|
print(f"[{self.name}] Installing WhatsApp bridge dependencies...")
|
||||||
# Resolve npm path so Windows can execute the .cmd shim.
|
|
||||||
# shutil.which honours PATHEXT; on POSIX it returns the
|
|
||||||
# plain executable path.
|
|
||||||
_npm_bin = shutil.which("npm") or "npm"
|
|
||||||
try:
|
try:
|
||||||
install_result = subprocess.run(
|
install_result = subprocess.run(
|
||||||
[_npm_bin, "install", "--silent"],
|
["npm", "install", "--silent"],
|
||||||
cwd=str(bridge_dir),
|
cwd=str(bridge_dir),
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
text=True,
|
text=True,
|
||||||
@@ -517,7 +428,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
pass # Bridge not running, start a new one
|
pass # Bridge not running, start a new one
|
||||||
|
|
||||||
# Kill any orphaned bridge from a previous gateway run
|
# Kill any orphaned bridge from a previous gateway run
|
||||||
_kill_stale_bridge_by_pidfile(self._session_path)
|
|
||||||
_kill_port_process(self._bridge_port)
|
_kill_port_process(self._bridge_port)
|
||||||
await asyncio.sleep(1)
|
await asyncio.sleep(1)
|
||||||
|
|
||||||
@@ -526,7 +436,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
# messages are preserved for troubleshooting.
|
# messages are preserved for troubleshooting.
|
||||||
whatsapp_mode = os.getenv("WHATSAPP_MODE", "self-chat")
|
whatsapp_mode = os.getenv("WHATSAPP_MODE", "self-chat")
|
||||||
self._bridge_log = self._session_path.parent / "bridge.log"
|
self._bridge_log = self._session_path.parent / "bridge.log"
|
||||||
bridge_log_fh = open(self._bridge_log, "a", encoding="utf-8")
|
bridge_log_fh = open(self._bridge_log, "a")
|
||||||
self._bridge_log_fh = bridge_log_fh
|
self._bridge_log_fh = bridge_log_fh
|
||||||
|
|
||||||
# Build bridge subprocess environment.
|
# Build bridge subprocess environment.
|
||||||
@@ -549,7 +459,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
preexec_fn=None if _IS_WINDOWS else os.setsid,
|
preexec_fn=None if _IS_WINDOWS else os.setsid,
|
||||||
env=bridge_env,
|
env=bridge_env,
|
||||||
)
|
)
|
||||||
_write_bridge_pidfile(self._session_path, self._bridge_process.pid)
|
|
||||||
|
|
||||||
# Wait for the bridge to connect to WhatsApp.
|
# Wait for the bridge to connect to WhatsApp.
|
||||||
# Phase 1: wait for the HTTP server to come up (up to 15s).
|
# Phase 1: wait for the HTTP server to come up (up to 15s).
|
||||||
@@ -700,12 +609,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
# Bridge was not started by us, don't kill it
|
# Bridge was not started by us, don't kill it
|
||||||
print(f"[{self.name}] Disconnecting (external bridge left running)")
|
print(f"[{self.name}] Disconnecting (external bridge left running)")
|
||||||
|
|
||||||
# Clean up PID file
|
|
||||||
try:
|
|
||||||
(self._session_path / "bridge.pid").unlink(missing_ok=True)
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
# Cancel the poll task explicitly
|
# Cancel the poll task explicitly
|
||||||
if self._poll_task and not self._poll_task.done():
|
if self._poll_task and not self._poll_task.done():
|
||||||
self._poll_task.cancel()
|
self._poll_task.cancel()
|
||||||
@@ -810,7 +713,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
|
|
||||||
# Format and chunk the message
|
# Format and chunk the message
|
||||||
formatted = self.format_message(content)
|
formatted = self.format_message(content)
|
||||||
chunks = self.truncate_message(formatted, self._outgoing_chunk_limit())
|
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
|
||||||
|
|
||||||
last_message_id = None
|
last_message_id = None
|
||||||
for chunk in chunks:
|
for chunk in chunks:
|
||||||
@@ -1170,7 +1073,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
|||||||
if file_size > MAX_TEXT_INJECT_BYTES:
|
if file_size > MAX_TEXT_INJECT_BYTES:
|
||||||
print(f"[{self.name}] Skipping text injection for {doc_path} ({file_size} bytes > {MAX_TEXT_INJECT_BYTES})", flush=True)
|
print(f"[{self.name}] Skipping text injection for {doc_path} ({file_size} bytes > {MAX_TEXT_INJECT_BYTES})", flush=True)
|
||||||
continue
|
continue
|
||||||
content = Path(doc_path).read_text(encoding="utf-8", errors="replace")
|
content = Path(doc_path).read_text(errors="replace")
|
||||||
fname = Path(doc_path).name
|
fname = Path(doc_path).name
|
||||||
# Remove the doc_<hex>_ prefix for display
|
# Remove the doc_<hex>_ prefix for display
|
||||||
display_name = fname
|
display_name = fname
|
||||||
|
|||||||
+68
-538
@@ -13,10 +13,6 @@ Usage:
|
|||||||
python cli.py --gateway
|
python cli.py --gateway
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
|
|
||||||
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
|
|
||||||
import hermes_bootstrap # noqa: F401
|
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import dataclasses
|
import dataclasses
|
||||||
import inspect
|
import inspect
|
||||||
@@ -262,18 +258,13 @@ def _ensure_ssl_certs() -> None:
|
|||||||
return
|
return
|
||||||
|
|
||||||
def _home_target_env_var(platform_name: str) -> str:
|
def _home_target_env_var(platform_name: str) -> str:
|
||||||
"""Return the configured home-target env var for a platform.
|
"""Return the configured home-target env var for a platform."""
|
||||||
|
from cron.scheduler import _HOME_TARGET_ENV_VARS
|
||||||
|
|
||||||
Consults built-in ``_HOME_TARGET_ENV_VARS`` first, then the plugin
|
return _HOME_TARGET_ENV_VARS.get(
|
||||||
registry via ``cron.scheduler._resolve_home_env_var``, then falls back
|
platform_name.lower(),
|
||||||
to ``<PLATFORM>_HOME_CHANNEL`` for unknown names.
|
f"{platform_name.upper()}_HOME_CHANNEL",
|
||||||
"""
|
)
|
||||||
from cron.scheduler import _resolve_home_env_var
|
|
||||||
|
|
||||||
resolved = _resolve_home_env_var(platform_name)
|
|
||||||
if resolved:
|
|
||||||
return resolved
|
|
||||||
return f"{platform_name.upper()}_HOME_CHANNEL"
|
|
||||||
|
|
||||||
|
|
||||||
def _home_thread_env_var(platform_name: str) -> str:
|
def _home_thread_env_var(platform_name: str) -> str:
|
||||||
@@ -308,36 +299,6 @@ _env_path = _hermes_home / '.env'
|
|||||||
load_hermes_dotenv(hermes_home=_hermes_home, project_env=Path(__file__).resolve().parents[1] / '.env')
|
load_hermes_dotenv(hermes_home=_hermes_home, project_env=Path(__file__).resolve().parents[1] / '.env')
|
||||||
|
|
||||||
|
|
||||||
def _reload_runtime_env_preserving_config_authority() -> None:
|
|
||||||
"""Reload .env for fresh credentials without letting stale .env override config.
|
|
||||||
|
|
||||||
Gateway processes are long-lived, so per-turn code reloads ~/.hermes/.env to
|
|
||||||
pick up rotated API keys. config.yaml remains authoritative for agent budget
|
|
||||||
settings such as agent.max_turns; otherwise a stale HERMES_MAX_ITERATIONS in
|
|
||||||
.env can replace the startup bridge on later turns.
|
|
||||||
"""
|
|
||||||
load_hermes_dotenv(
|
|
||||||
hermes_home=_hermes_home,
|
|
||||||
project_env=Path(__file__).resolve().parents[1] / '.env',
|
|
||||||
)
|
|
||||||
|
|
||||||
config_path = _hermes_home / 'config.yaml'
|
|
||||||
if not config_path.exists():
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
import yaml as _yaml
|
|
||||||
with open(config_path, encoding="utf-8") as f:
|
|
||||||
cfg = _yaml.safe_load(f) or {}
|
|
||||||
from hermes_cli.config import _expand_env_vars
|
|
||||||
cfg = _expand_env_vars(cfg)
|
|
||||||
except Exception:
|
|
||||||
return
|
|
||||||
|
|
||||||
agent_cfg = cfg.get("agent", {})
|
|
||||||
if isinstance(agent_cfg, dict) and "max_turns" in agent_cfg:
|
|
||||||
os.environ["HERMES_MAX_ITERATIONS"] = str(agent_cfg["max_turns"])
|
|
||||||
|
|
||||||
|
|
||||||
_DOCKER_VOLUME_SPEC_RE = re.compile(r"^(?P<host>.+):(?P<container>/[^:]+?)(?::(?P<options>[^:]+))?$")
|
_DOCKER_VOLUME_SPEC_RE = re.compile(r"^(?P<host>.+):(?P<container>/[^:]+?)(?::(?P<options>[^:]+))?$")
|
||||||
_DOCKER_MEDIA_OUTPUT_CONTAINER_PATHS = {"/output", "/outputs"}
|
_DOCKER_MEDIA_OUTPUT_CONTAINER_PATHS = {"/output", "/outputs"}
|
||||||
|
|
||||||
@@ -507,22 +468,22 @@ try:
|
|||||||
_network_cfg = (_cfg if '_cfg' in dir() else {}).get("network", {})
|
_network_cfg = (_cfg if '_cfg' in dir() else {}).get("network", {})
|
||||||
if isinstance(_network_cfg, dict) and _network_cfg.get("force_ipv4"):
|
if isinstance(_network_cfg, dict) and _network_cfg.get("force_ipv4"):
|
||||||
apply_ipv4_preference(force=True)
|
apply_ipv4_preference(force=True)
|
||||||
except Exception as _bootstrap_exc:
|
except Exception:
|
||||||
print(f" Warning: IPv4 preference application failed: {_bootstrap_exc}", file=sys.stderr)
|
pass
|
||||||
|
|
||||||
# Validate config structure early — log warnings so gateway operators see problems
|
# Validate config structure early — log warnings so gateway operators see problems
|
||||||
try:
|
try:
|
||||||
from hermes_cli.config import print_config_warnings
|
from hermes_cli.config import print_config_warnings
|
||||||
print_config_warnings()
|
print_config_warnings()
|
||||||
except Exception as _bootstrap_exc:
|
except Exception:
|
||||||
print(f" Warning: config validation failed: {_bootstrap_exc}", file=sys.stderr)
|
pass
|
||||||
|
|
||||||
# Warn if user has deprecated MESSAGING_CWD / TERMINAL_CWD in .env
|
# Warn if user has deprecated MESSAGING_CWD / TERMINAL_CWD in .env
|
||||||
try:
|
try:
|
||||||
from hermes_cli.config import warn_deprecated_cwd_env_vars
|
from hermes_cli.config import warn_deprecated_cwd_env_vars
|
||||||
warn_deprecated_cwd_env_vars()
|
warn_deprecated_cwd_env_vars()
|
||||||
except Exception as _bootstrap_exc:
|
except Exception:
|
||||||
print(f" Warning: deprecation check failed: {_bootstrap_exc}", file=sys.stderr)
|
pass
|
||||||
|
|
||||||
# Gateway runs in quiet mode - suppress debug output and use cwd directly (no temp dirs)
|
# Gateway runs in quiet mode - suppress debug output and use cwd directly (no temp dirs)
|
||||||
os.environ["HERMES_QUIET"] = "1"
|
os.environ["HERMES_QUIET"] = "1"
|
||||||
@@ -652,11 +613,7 @@ def _try_resolve_fallback_provider() -> dict | None:
|
|||||||
explicit_base_url=entry.get("base_url"),
|
explicit_base_url=entry.get("base_url"),
|
||||||
explicit_api_key=entry.get("api_key"),
|
explicit_api_key=entry.get("api_key"),
|
||||||
)
|
)
|
||||||
logger.info(
|
logger.info("Fallback provider resolved: %s", runtime.get("provider"))
|
||||||
"Fallback provider resolved: %s model=%s",
|
|
||||||
runtime.get("provider"),
|
|
||||||
entry.get("model"),
|
|
||||||
)
|
|
||||||
return {
|
return {
|
||||||
"api_key": runtime.get("api_key"),
|
"api_key": runtime.get("api_key"),
|
||||||
"base_url": runtime.get("base_url"),
|
"base_url": runtime.get("base_url"),
|
||||||
@@ -665,7 +622,6 @@ def _try_resolve_fallback_provider() -> dict | None:
|
|||||||
"command": runtime.get("command"),
|
"command": runtime.get("command"),
|
||||||
"args": list(runtime.get("args") or []),
|
"args": list(runtime.get("args") or []),
|
||||||
"credential_pool": runtime.get("credential_pool"),
|
"credential_pool": runtime.get("credential_pool"),
|
||||||
"model": entry.get("model"),
|
|
||||||
}
|
}
|
||||||
except Exception as fb_exc:
|
except Exception as fb_exc:
|
||||||
logger.debug("Fallback entry %s failed: %s", entry.get("provider"), fb_exc)
|
logger.debug("Fallback entry %s failed: %s", entry.get("provider"), fb_exc)
|
||||||
@@ -1029,26 +985,6 @@ def _normalize_empty_agent_response(
|
|||||||
return response
|
return response
|
||||||
|
|
||||||
|
|
||||||
def _should_clear_resume_pending_after_turn(agent_result: dict) -> bool:
|
|
||||||
"""Return True only when a gateway turn really completed successfully.
|
|
||||||
|
|
||||||
Restart recovery uses ``resume_pending`` as a durable marker for sessions
|
|
||||||
interrupted during gateway drain. A soft interrupt can still bubble out as
|
|
||||||
a syntactically normal agent result with an empty final response; clearing
|
|
||||||
the marker in that case loses the recovery signal and startup auto-resume
|
|
||||||
has nothing to schedule.
|
|
||||||
"""
|
|
||||||
if not isinstance(agent_result, dict):
|
|
||||||
return False
|
|
||||||
if agent_result.get("interrupted"):
|
|
||||||
return False
|
|
||||||
if agent_result.get("failed") or agent_result.get("partial") or agent_result.get("error"):
|
|
||||||
return False
|
|
||||||
if agent_result.get("completed") is False:
|
|
||||||
return False
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
class GatewayRunner:
|
class GatewayRunner:
|
||||||
"""
|
"""
|
||||||
Main gateway controller.
|
Main gateway controller.
|
||||||
@@ -1130,13 +1066,6 @@ class GatewayRunner:
|
|||||||
self._pending_native_image_paths_by_session: Dict[str, List[str]] = {}
|
self._pending_native_image_paths_by_session: Dict[str, List[str]] = {}
|
||||||
self._busy_ack_ts: Dict[str, float] = {} # last busy-ack timestamp per session (debounce)
|
self._busy_ack_ts: Dict[str, float] = {} # last busy-ack timestamp per session (debounce)
|
||||||
self._session_run_generation: Dict[str, int] = {}
|
self._session_run_generation: Dict[str, int] = {}
|
||||||
# LRU cache of live SessionSources keyed by session_key. Used by
|
|
||||||
# fallback routing paths (shutdown notifications, synthetic
|
|
||||||
# background-process events) when the persisted origin is missing
|
|
||||||
# and _parse_session_key can't recover thread_id. Capped so it
|
|
||||||
# cannot grow unbounded over a long-running gateway lifetime.
|
|
||||||
self._session_sources: "OrderedDict[str, SessionSource]" = OrderedDict()
|
|
||||||
self._session_sources_max = 512
|
|
||||||
|
|
||||||
# Cache AIAgent instances per session to preserve prompt caching.
|
# Cache AIAgent instances per session to preserve prompt caching.
|
||||||
# Without this, a new AIAgent is created per message, rebuilding the
|
# Without this, a new AIAgent is created per message, rebuilding the
|
||||||
@@ -1675,14 +1604,6 @@ class GatewayRunner:
|
|||||||
)
|
)
|
||||||
|
|
||||||
runtime_kwargs = _resolve_runtime_agent_kwargs()
|
runtime_kwargs = _resolve_runtime_agent_kwargs()
|
||||||
runtime_model = runtime_kwargs.pop("model", None)
|
|
||||||
if runtime_model:
|
|
||||||
logger.info(
|
|
||||||
"Runtime provider supplied explicit model override: %s -> %s",
|
|
||||||
model,
|
|
||||||
runtime_model,
|
|
||||||
)
|
|
||||||
model = runtime_model
|
|
||||||
if override and resolved_session_key:
|
if override and resolved_session_key:
|
||||||
model, runtime_kwargs = self._apply_session_model_override(
|
model, runtime_kwargs = self._apply_session_model_override(
|
||||||
resolved_session_key, model, runtime_kwargs
|
resolved_session_key, model, runtime_kwargs
|
||||||
@@ -2510,9 +2431,6 @@ class GatewayRunner:
|
|||||||
e,
|
e,
|
||||||
)
|
)
|
||||||
|
|
||||||
if source is None:
|
|
||||||
source = self._get_cached_session_source(session_key)
|
|
||||||
|
|
||||||
if source is not None:
|
if source is not None:
|
||||||
platform_str = source.platform.value
|
platform_str = source.platform.value
|
||||||
chat_id = str(source.chat_id)
|
chat_id = str(source.chat_id)
|
||||||
@@ -2573,12 +2491,7 @@ class GatewayRunner:
|
|||||||
platform_str, chat_id, e,
|
platform_str, chat_id, e,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Snapshot adapters up front: adapter.send() can hit a fatal error
|
for platform, adapter in self.adapters.items():
|
||||||
# path that pops the adapter from self.adapters (see _handle_fatal
|
|
||||||
# elsewhere), which would otherwise trigger
|
|
||||||
# ``RuntimeError: dictionary changed size during iteration`` —
|
|
||||||
# observed in a user report during gateway shutdown.
|
|
||||||
for platform, adapter in list(self.adapters.items()):
|
|
||||||
home = self.config.get_home_channel(platform)
|
home = self.config.get_home_channel(platform)
|
||||||
if not home or not home.chat_id:
|
if not home or not home.chat_id:
|
||||||
continue
|
continue
|
||||||
@@ -2788,48 +2701,6 @@ class GatewayRunner:
|
|||||||
return
|
return
|
||||||
|
|
||||||
current_pid = os.getpid()
|
current_pid = os.getpid()
|
||||||
|
|
||||||
# On Windows there's no bash/setsid chain — spawn a tiny Python
|
|
||||||
# watcher directly via sys.executable instead. The watcher polls
|
|
||||||
# current_pid, waits for our exit, then runs `hermes gateway
|
|
||||||
# restart` with detach flags so the respawn survives the CLI
|
|
||||||
# that triggered the /restart command closing its console.
|
|
||||||
if sys.platform == "win32":
|
|
||||||
import textwrap
|
|
||||||
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
|
|
||||||
|
|
||||||
cmd_argv = [*hermes_cmd, "gateway", "restart"]
|
|
||||||
watcher = textwrap.dedent(
|
|
||||||
"""
|
|
||||||
import os, subprocess, sys, time
|
|
||||||
pid = int(sys.argv[1])
|
|
||||||
cmd = sys.argv[2:]
|
|
||||||
deadline = time.monotonic() + 120
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
try:
|
|
||||||
os.kill(pid, 0)
|
|
||||||
except (ProcessLookupError, PermissionError, OSError):
|
|
||||||
break
|
|
||||||
time.sleep(0.2)
|
|
||||||
_CREATE_NEW_PROCESS_GROUP = 0x00000200
|
|
||||||
_DETACHED_PROCESS = 0x00000008
|
|
||||||
_CREATE_NO_WINDOW = 0x08000000
|
|
||||||
subprocess.Popen(
|
|
||||||
cmd,
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
stderr=subprocess.DEVNULL,
|
|
||||||
creationflags=_CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW,
|
|
||||||
)
|
|
||||||
"""
|
|
||||||
).strip()
|
|
||||||
subprocess.Popen(
|
|
||||||
[sys.executable, "-c", watcher, str(current_pid), *cmd_argv],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
stderr=subprocess.DEVNULL,
|
|
||||||
**windows_detach_popen_kwargs(),
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
cmd = " ".join(shlex.quote(part) for part in hermes_cmd)
|
cmd = " ".join(shlex.quote(part) for part in hermes_cmd)
|
||||||
shell_cmd = (
|
shell_cmd = (
|
||||||
f"while kill -0 {current_pid} 2>/dev/null; do sleep 0.2; done; "
|
f"while kill -0 {current_pid} 2>/dev/null; do sleep 0.2; done; "
|
||||||
@@ -2868,83 +2739,6 @@ class GatewayRunner:
|
|||||||
task.add_done_callback(self._background_tasks.discard)
|
task.add_done_callback(self._background_tasks.discard)
|
||||||
return True
|
return True
|
||||||
|
|
||||||
# Drain-timeout reasons set by _stop_impl() when a still-running turn is
|
|
||||||
# force-interrupted; "restart_interrupted" is set by
|
|
||||||
# SessionStore.suspend_recently_active() on crash recovery (no
|
|
||||||
# .clean_shutdown marker). All three mean "the agent was mid-turn and
|
|
||||||
# we killed it" — eligible for startup auto-resume.
|
|
||||||
_AUTO_RESUME_REASONS = frozenset(
|
|
||||||
{"restart_timeout", "shutdown_timeout", "restart_interrupted"}
|
|
||||||
)
|
|
||||||
|
|
||||||
def _schedule_resume_pending_sessions(self) -> int:
|
|
||||||
"""Auto-continue fresh restart-interrupted sessions after startup.
|
|
||||||
|
|
||||||
``resume_pending`` already preserves the transcript AND the existing
|
|
||||||
``_is_resume_pending`` branch in ``_handle_message_with_agent``
|
|
||||||
injects a reason-aware recovery system note on the next turn. This
|
|
||||||
method closes the UX gap by synthesizing that next turn once
|
|
||||||
adapters are back online — the event text is empty so the existing
|
|
||||||
injection path owns the wording and we never double up.
|
|
||||||
|
|
||||||
Adapters that are not yet ready (adapter missing from
|
|
||||||
``self.adapters``) are skipped silently; their sessions stay
|
|
||||||
``resume_pending`` and will auto-resume on the next real user
|
|
||||||
message, or on the next gateway startup.
|
|
||||||
"""
|
|
||||||
window = _auto_continue_freshness_window()
|
|
||||||
try:
|
|
||||||
with self.session_store._lock: # noqa: SLF001 — snapshot under lock
|
|
||||||
self.session_store._ensure_loaded_locked() # noqa: SLF001
|
|
||||||
candidates = [
|
|
||||||
entry for entry in self.session_store._entries.values() # noqa: SLF001
|
|
||||||
if entry.resume_pending
|
|
||||||
and not entry.suspended
|
|
||||||
and entry.origin is not None
|
|
||||||
and entry.resume_reason in self._AUTO_RESUME_REASONS
|
|
||||||
]
|
|
||||||
except Exception as exc:
|
|
||||||
logger.warning("Failed to enumerate resume-pending sessions: %s", exc)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
now = datetime.now()
|
|
||||||
scheduled = 0
|
|
||||||
for entry in candidates:
|
|
||||||
marker = entry.last_resume_marked_at or entry.updated_at
|
|
||||||
if marker is not None and (now - marker).total_seconds() > window:
|
|
||||||
continue
|
|
||||||
|
|
||||||
source = entry.origin
|
|
||||||
adapter = self.adapters.get(source.platform)
|
|
||||||
if adapter is None:
|
|
||||||
logger.debug(
|
|
||||||
"Skipping auto-resume for %s: adapter not ready for %s",
|
|
||||||
entry.session_key,
|
|
||||||
getattr(source.platform, "value", source.platform),
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Empty-text internal event — the _is_resume_pending branch in
|
|
||||||
# _handle_message_with_agent prepends the proper reason-aware
|
|
||||||
# system note before the turn runs.
|
|
||||||
event = MessageEvent(
|
|
||||||
text="",
|
|
||||||
message_type=MessageType.TEXT,
|
|
||||||
source=source,
|
|
||||||
internal=True,
|
|
||||||
)
|
|
||||||
task = asyncio.create_task(adapter.handle_message(event))
|
|
||||||
self._background_tasks.add(task)
|
|
||||||
task.add_done_callback(self._background_tasks.discard)
|
|
||||||
scheduled += 1
|
|
||||||
|
|
||||||
if scheduled:
|
|
||||||
logger.info(
|
|
||||||
"Scheduled auto-resume for %d restart-interrupted session(s)",
|
|
||||||
scheduled,
|
|
||||||
)
|
|
||||||
return scheduled
|
|
||||||
|
|
||||||
async def start(self) -> bool:
|
async def start(self) -> bool:
|
||||||
"""
|
"""
|
||||||
Start the gateway and all configured platform adapters.
|
Start the gateway and all configured platform adapters.
|
||||||
@@ -2969,29 +2763,6 @@ class GatewayRunner:
|
|||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
# Redaction status: ON by default (#17691). Surface a prominent
|
|
||||||
# warning if an operator has explicitly opted out so they don't
|
|
||||||
# forget the downgrade is active — the redactor snapshots its
|
|
||||||
# state at import time, so this log line is the source of truth
|
|
||||||
# for this process's lifetime.
|
|
||||||
try:
|
|
||||||
_redact_raw = os.getenv("HERMES_REDACT_SECRETS", "true")
|
|
||||||
_redact_on = _redact_raw.lower() in ("1", "true", "yes", "on")
|
|
||||||
if _redact_on:
|
|
||||||
logger.info(
|
|
||||||
"Secret redaction: ENABLED (tool output, logs, and chat "
|
|
||||||
"responses are scrubbed before delivery)"
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
logger.warning(
|
|
||||||
"Secret redaction: DISABLED (HERMES_REDACT_SECRETS=%s). "
|
|
||||||
"API keys and tokens may appear verbatim in chat output, "
|
|
||||||
"session JSONs, and logs. Set security.redact_secrets: true "
|
|
||||||
"in config.yaml to re-enable.",
|
|
||||||
_redact_raw,
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
try:
|
try:
|
||||||
from hermes_cli.profiles import get_active_profile_name
|
from hermes_cli.profiles import get_active_profile_name
|
||||||
_profile = get_active_profile_name()
|
_profile = get_active_profile_name()
|
||||||
@@ -3356,12 +3127,6 @@ class GatewayRunner:
|
|||||||
skip_targets=skip_home_targets,
|
skip_targets=skip_home_targets,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Automatically continue fresh sessions that were interrupted by the
|
|
||||||
# previous gateway restart/shutdown. The resume_pending flag is cleared
|
|
||||||
# by the normal successful-turn path, so a failed auto-resume remains
|
|
||||||
# visible for manual recovery on the next user message.
|
|
||||||
self._schedule_resume_pending_sessions()
|
|
||||||
|
|
||||||
# Drain any recovered process watchers (from crash recovery checkpoint)
|
# Drain any recovered process watchers (from crash recovery checkpoint)
|
||||||
try:
|
try:
|
||||||
from tools.process_registry import process_registry
|
from tools.process_registry import process_registry
|
||||||
@@ -3880,24 +3645,6 @@ class GatewayRunner:
|
|||||||
if max_spawn is not None:
|
if max_spawn is not None:
|
||||||
logger.info(f"kanban dispatcher: max_spawn={max_spawn}")
|
logger.info(f"kanban dispatcher: max_spawn={max_spawn}")
|
||||||
|
|
||||||
raw_failure_limit = kanban_cfg.get("failure_limit", _kb.DEFAULT_FAILURE_LIMIT)
|
|
||||||
try:
|
|
||||||
failure_limit = int(raw_failure_limit)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
logger.warning(
|
|
||||||
"kanban dispatcher: invalid kanban.failure_limit=%r; using default %d",
|
|
||||||
raw_failure_limit,
|
|
||||||
_kb.DEFAULT_FAILURE_LIMIT,
|
|
||||||
)
|
|
||||||
failure_limit = _kb.DEFAULT_FAILURE_LIMIT
|
|
||||||
if failure_limit < 1:
|
|
||||||
logger.warning(
|
|
||||||
"kanban dispatcher: kanban.failure_limit=%r is below 1; using default %d",
|
|
||||||
raw_failure_limit,
|
|
||||||
_kb.DEFAULT_FAILURE_LIMIT,
|
|
||||||
)
|
|
||||||
failure_limit = _kb.DEFAULT_FAILURE_LIMIT
|
|
||||||
|
|
||||||
# Initial delay so the gateway finishes wiring adapters before the
|
# Initial delay so the gateway finishes wiring adapters before the
|
||||||
# dispatcher spawns workers (those workers may hit gateway notify
|
# dispatcher spawns workers (those workers may hit gateway notify
|
||||||
# subscriptions etc.). Matches the notifier watcher's delay.
|
# subscriptions etc.). Matches the notifier watcher's delay.
|
||||||
@@ -3926,12 +3673,7 @@ class GatewayRunner:
|
|||||||
_kb.init_db(board=slug) # idempotent, handles first-run
|
_kb.init_db(board=slug) # idempotent, handles first-run
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
return _kb.dispatch_once(
|
return _kb.dispatch_once(conn, board=slug, max_spawn=max_spawn)
|
||||||
conn,
|
|
||||||
board=slug,
|
|
||||||
max_spawn=max_spawn,
|
|
||||||
failure_limit=failure_limit,
|
|
||||||
)
|
|
||||||
except Exception:
|
except Exception:
|
||||||
logger.exception("kanban dispatcher: tick failed on board %s", slug)
|
logger.exception("kanban dispatcher: tick failed on board %s", slug)
|
||||||
return None
|
return None
|
||||||
@@ -6015,7 +5757,6 @@ class GatewayRunner:
|
|||||||
|
|
||||||
if event.media_urls and event.message_type == MessageType.DOCUMENT:
|
if event.media_urls and event.message_type == MessageType.DOCUMENT:
|
||||||
import mimetypes as _mimetypes
|
import mimetypes as _mimetypes
|
||||||
from tools.credential_files import to_agent_visible_cache_path
|
|
||||||
|
|
||||||
_TEXT_EXTENSIONS = {".txt", ".md", ".csv", ".log", ".json", ".xml", ".yaml", ".yml", ".toml", ".ini", ".cfg"}
|
_TEXT_EXTENSIONS = {".txt", ".md", ".csv", ".log", ".json", ".xml", ".yaml", ".yml", ".toml", ".ini", ".cfg"}
|
||||||
for i, path in enumerate(event.media_urls):
|
for i, path in enumerate(event.media_urls):
|
||||||
@@ -6036,21 +5777,16 @@ class GatewayRunner:
|
|||||||
display_name = parts[2] if len(parts) >= 3 else basename
|
display_name = parts[2] if len(parts) >= 3 else basename
|
||||||
display_name = re.sub(r'[^\w.\- ]', '_', display_name)
|
display_name = re.sub(r'[^\w.\- ]', '_', display_name)
|
||||||
|
|
||||||
# Translate host cache path to in-container path if running under Docker backend.
|
|
||||||
# This ensures the agent receives a path it can open inside its sandbox, as the
|
|
||||||
# cache directories are auto-mounted at /root/.hermes/cache/* by get_cache_directory_mounts().
|
|
||||||
agent_path = to_agent_visible_cache_path(path)
|
|
||||||
|
|
||||||
if mtype.startswith("text/"):
|
if mtype.startswith("text/"):
|
||||||
context_note = (
|
context_note = (
|
||||||
f"[The user sent a text document: '{display_name}'. "
|
f"[The user sent a text document: '{display_name}'. "
|
||||||
f"Its content has been included below. "
|
f"Its content has been included below. "
|
||||||
f"The file is also saved at: {agent_path}]"
|
f"The file is also saved at: {path}]"
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
context_note = (
|
context_note = (
|
||||||
f"[The user sent a document: '{display_name}'. "
|
f"[The user sent a document: '{display_name}'. "
|
||||||
f"The file is saved at: {agent_path}. "
|
f"The file is saved at: {path}. "
|
||||||
f"Ask the user what they'd like you to do with it.]"
|
f"Ask the user what they'd like you to do with it.]"
|
||||||
)
|
)
|
||||||
message_text = f"{context_note}\n\n{message_text}"
|
message_text = f"{context_note}\n\n{message_text}"
|
||||||
@@ -6115,41 +5851,6 @@ class GatewayRunner:
|
|||||||
return []
|
return []
|
||||||
return list(pending_native.pop(session_key, []) or [])
|
return list(pending_native.pop(session_key, []) or [])
|
||||||
|
|
||||||
def _cache_session_source(self, session_key: str, source) -> None:
|
|
||||||
if not session_key or source is None:
|
|
||||||
return
|
|
||||||
cached_sources = getattr(self, "_session_sources", None)
|
|
||||||
if cached_sources is None:
|
|
||||||
cached_sources = OrderedDict()
|
|
||||||
self._session_sources = cached_sources
|
|
||||||
try:
|
|
||||||
cached_sources[session_key] = dataclasses.replace(source)
|
|
||||||
except Exception:
|
|
||||||
logger.debug("Failed to cache live session source for %s", session_key, exc_info=True)
|
|
||||||
return
|
|
||||||
# LRU: mark as most-recently-used and trim to max size.
|
|
||||||
try:
|
|
||||||
cached_sources.move_to_end(session_key)
|
|
||||||
max_size = getattr(self, "_session_sources_max", 512)
|
|
||||||
while len(cached_sources) > max_size:
|
|
||||||
cached_sources.popitem(last=False)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _get_cached_session_source(self, session_key: str):
|
|
||||||
if not session_key:
|
|
||||||
return None
|
|
||||||
cached_sources = getattr(self, "_session_sources", None)
|
|
||||||
if not cached_sources:
|
|
||||||
return None
|
|
||||||
source = cached_sources.get(session_key)
|
|
||||||
if source is not None:
|
|
||||||
try:
|
|
||||||
cached_sources.move_to_end(session_key)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return source
|
|
||||||
|
|
||||||
async def _handle_message_with_agent(self, event, source, _quick_key: str, run_generation: int):
|
async def _handle_message_with_agent(self, event, source, _quick_key: str, run_generation: int):
|
||||||
"""Inner handler that runs under the _running_agents sentinel guard."""
|
"""Inner handler that runs under the _running_agents sentinel guard."""
|
||||||
_msg_start_time = time.time()
|
_msg_start_time = time.time()
|
||||||
@@ -6164,7 +5865,6 @@ class GatewayRunner:
|
|||||||
# Get or create session
|
# Get or create session
|
||||||
session_entry = self.session_store.get_or_create_session(source)
|
session_entry = self.session_store.get_or_create_session(source)
|
||||||
session_key = session_entry.session_key
|
session_key = session_entry.session_key
|
||||||
self._cache_session_source(session_key, source)
|
|
||||||
if self._is_telegram_topic_lane(source):
|
if self._is_telegram_topic_lane(source):
|
||||||
try:
|
try:
|
||||||
binding = self._session_db.get_telegram_topic_binding(
|
binding = self._session_db.get_telegram_topic_binding(
|
||||||
@@ -6801,7 +6501,7 @@ class GatewayRunner:
|
|||||||
# shutdown) — the turn ran to completion, so recovery
|
# shutdown) — the turn ran to completion, so recovery
|
||||||
# succeeded and subsequent messages should no longer receive
|
# succeeded and subsequent messages should no longer receive
|
||||||
# the restart-interruption system note.
|
# the restart-interruption system note.
|
||||||
if session_key and _should_clear_resume_pending_after_turn(agent_result):
|
if session_key:
|
||||||
self._clear_restart_failure_count(session_key)
|
self._clear_restart_failure_count(session_key)
|
||||||
try:
|
try:
|
||||||
self.session_store.clear_resume_pending(session_key)
|
self.session_store.clear_resume_pending(session_key)
|
||||||
@@ -8382,27 +8082,6 @@ class GatewayRunner:
|
|||||||
# ────────────────────────────────────────────────────────────────
|
# ────────────────────────────────────────────────────────────────
|
||||||
# /goal — persistent cross-turn goals (Ralph-style loop)
|
# /goal — persistent cross-turn goals (Ralph-style loop)
|
||||||
# ────────────────────────────────────────────────────────────────
|
# ────────────────────────────────────────────────────────────────
|
||||||
def _goal_max_turns_from_config(self) -> int:
|
|
||||||
"""Resolve the configured /goal turn budget for gateway sessions.
|
|
||||||
|
|
||||||
GatewayRunner.config is a GatewayConfig dataclass, not the full
|
|
||||||
user config mapping. Top-level config blocks such as ``goals`` are
|
|
||||||
therefore only available through hermes_cli.config.load_config().
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
goals_cfg = (
|
|
||||||
(self.config or {}).get("goals", {})
|
|
||||||
if isinstance(self.config, dict)
|
|
||||||
else getattr(self.config, "goals", {}) or {}
|
|
||||||
)
|
|
||||||
if not goals_cfg:
|
|
||||||
from hermes_cli.config import load_config
|
|
||||||
|
|
||||||
goals_cfg = (load_config() or {}).get("goals") or {}
|
|
||||||
return int(goals_cfg.get("max_turns", 20) or 20)
|
|
||||||
except Exception:
|
|
||||||
return 20
|
|
||||||
|
|
||||||
def _get_goal_manager_for_event(self, event: "MessageEvent"):
|
def _get_goal_manager_for_event(self, event: "MessageEvent"):
|
||||||
"""Return a GoalManager bound to the session for this gateway event.
|
"""Return a GoalManager bound to the session for this gateway event.
|
||||||
|
|
||||||
@@ -8422,7 +8101,15 @@ class GatewayRunner:
|
|||||||
sid = getattr(session_entry, "session_id", None) or ""
|
sid = getattr(session_entry, "session_id", None) or ""
|
||||||
if not sid:
|
if not sid:
|
||||||
return None, None
|
return None, None
|
||||||
max_turns = self._goal_max_turns_from_config()
|
try:
|
||||||
|
goals_cfg = (
|
||||||
|
(self.config or {}).get("goals", {})
|
||||||
|
if isinstance(self.config, dict)
|
||||||
|
else getattr(self.config, "goals", {}) or {}
|
||||||
|
)
|
||||||
|
max_turns = int(goals_cfg.get("max_turns", 20) or 20)
|
||||||
|
except Exception:
|
||||||
|
max_turns = 20
|
||||||
return GoalManager(session_id=sid, default_max_turns=max_turns), session_entry
|
return GoalManager(session_id=sid, default_max_turns=max_turns), session_entry
|
||||||
|
|
||||||
async def _handle_goal_command(self, event: "MessageEvent") -> str:
|
async def _handle_goal_command(self, event: "MessageEvent") -> str:
|
||||||
@@ -8522,7 +8209,15 @@ class GatewayRunner:
|
|||||||
if not sid:
|
if not sid:
|
||||||
return
|
return
|
||||||
|
|
||||||
max_turns = self._goal_max_turns_from_config()
|
try:
|
||||||
|
goals_cfg = (
|
||||||
|
(self.config or {}).get("goals", {})
|
||||||
|
if isinstance(self.config, dict)
|
||||||
|
else getattr(self.config, "goals", {}) or {}
|
||||||
|
)
|
||||||
|
max_turns = int(goals_cfg.get("max_turns", 20) or 20)
|
||||||
|
except Exception:
|
||||||
|
max_turns = 20
|
||||||
|
|
||||||
mgr = GoalManager(session_id=sid, default_max_turns=max_turns)
|
mgr = GoalManager(session_id=sid, default_max_turns=max_turns)
|
||||||
if not mgr.is_active():
|
if not mgr.is_active():
|
||||||
@@ -9065,12 +8760,6 @@ class GatewayRunner:
|
|||||||
from urllib.parse import quote as _quote
|
from urllib.parse import quote as _quote
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Capture [[as_document]] before extract_media strips it, so the
|
|
||||||
# dispatch partition below can route image-extension files
|
|
||||||
# through send_document (preserving bytes) instead of
|
|
||||||
# send_multiple_images (Telegram sendPhoto recompresses to ~1280px).
|
|
||||||
force_document_attachments = "[[as_document]]" in response
|
|
||||||
|
|
||||||
media_files, _ = adapter.extract_media(response)
|
media_files, _ = adapter.extract_media(response)
|
||||||
_, cleaned = adapter.extract_images(response)
|
_, cleaned = adapter.extract_images(response)
|
||||||
local_files, _ = adapter.extract_local_files(cleaned)
|
local_files, _ = adapter.extract_local_files(cleaned)
|
||||||
@@ -9083,24 +8772,19 @@ class GatewayRunner:
|
|||||||
_IMAGE_EXTS = {'.jpg', '.jpeg', '.png', '.webp', '.gif'}
|
_IMAGE_EXTS = {'.jpg', '.jpeg', '.png', '.webp', '.gif'}
|
||||||
|
|
||||||
# Partition out images so they can be sent as a single batch
|
# Partition out images so they can be sent as a single batch
|
||||||
# (e.g. Signal's multi-attachment RPC). When [[as_document]] was
|
# (e.g. Signal's multi-attachment RPC)
|
||||||
# set, image-extension files skip the photo path and route to
|
|
||||||
# send_document below — preserving original bytes.
|
|
||||||
image_paths: list = []
|
image_paths: list = []
|
||||||
non_image_media: list = []
|
non_image_media: list = []
|
||||||
for media_path, is_voice in media_files:
|
for media_path, is_voice in media_files:
|
||||||
ext = Path(media_path).suffix.lower()
|
ext = Path(media_path).suffix.lower()
|
||||||
if (ext in _IMAGE_EXTS
|
if ext in _IMAGE_EXTS and not is_voice:
|
||||||
and not is_voice
|
|
||||||
and not force_document_attachments):
|
|
||||||
image_paths.append(media_path)
|
image_paths.append(media_path)
|
||||||
else:
|
else:
|
||||||
non_image_media.append((media_path, is_voice))
|
non_image_media.append((media_path, is_voice))
|
||||||
|
|
||||||
non_image_local: list = []
|
non_image_local: list = []
|
||||||
for file_path in local_files:
|
for file_path in local_files:
|
||||||
if (Path(file_path).suffix.lower() in _IMAGE_EXTS
|
if Path(file_path).suffix.lower() in _IMAGE_EXTS:
|
||||||
and not force_document_attachments):
|
|
||||||
image_paths.append(file_path)
|
image_paths.append(file_path)
|
||||||
else:
|
else:
|
||||||
non_image_local.append(file_path)
|
non_image_local.append(file_path)
|
||||||
@@ -11351,78 +11035,30 @@ class GatewayRunner:
|
|||||||
# where systemd-run --user fails due to missing D-Bus session).
|
# where systemd-run --user fails due to missing D-Bus session).
|
||||||
# PYTHONUNBUFFERED ensures output is flushed line-by-line so the
|
# PYTHONUNBUFFERED ensures output is flushed line-by-line so the
|
||||||
# gateway can stream it to the messenger in near-real-time.
|
# gateway can stream it to the messenger in near-real-time.
|
||||||
# Spawn `hermes update --gateway` detached so it survives gateway restart.
|
hermes_cmd_str = " ".join(shlex.quote(part) for part in hermes_cmd)
|
||||||
# --gateway enables file-based IPC for interactive prompts (stash
|
update_cmd = (
|
||||||
# restore, config migration) so the gateway can forward them to the
|
f"PYTHONUNBUFFERED=1 {hermes_cmd_str} update --gateway"
|
||||||
# user instead of silently skipping them.
|
f" > {shlex.quote(str(output_path))} 2>&1; "
|
||||||
# Use setsid for portable session detach (works under system services
|
f"status=$?; printf '%s' \"$status\" > {shlex.quote(str(exit_code_path))}"
|
||||||
# where systemd-run --user fails due to missing D-Bus session).
|
)
|
||||||
# PYTHONUNBUFFERED ensures output is flushed line-by-line so the
|
|
||||||
# gateway can stream it to the messenger in near-real-time.
|
|
||||||
#
|
|
||||||
# Windows: no bash/setsid chain. Run `hermes update --gateway`
|
|
||||||
# directly via sys.executable; redirect stdout/stderr to the same
|
|
||||||
# output files via Popen file handles; write the exit code in a
|
|
||||||
# follow-up write. A tiny Python watcher would be cleaner but
|
|
||||||
# we're already inside gateway/run.py's update path which is async,
|
|
||||||
# so the simplest correct thing is: launch an inline Python helper
|
|
||||||
# that runs the command and writes both outputs.
|
|
||||||
try:
|
try:
|
||||||
if sys.platform == "win32":
|
setsid_bin = shutil.which("setsid")
|
||||||
import textwrap
|
if setsid_bin:
|
||||||
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
|
# Preferred: setsid creates a new session, fully detached
|
||||||
|
|
||||||
# hermes_cmd is a list of argv parts we can pass directly
|
|
||||||
# (no shell-quoting needed).
|
|
||||||
helper = textwrap.dedent(
|
|
||||||
"""
|
|
||||||
import os, subprocess, sys
|
|
||||||
output_path = sys.argv[1]
|
|
||||||
exit_code_path = sys.argv[2]
|
|
||||||
cmd = sys.argv[3:]
|
|
||||||
env = dict(os.environ)
|
|
||||||
env["PYTHONUNBUFFERED"] = "1"
|
|
||||||
with open(output_path, "wb") as f:
|
|
||||||
proc = subprocess.Popen(cmd, stdout=f, stderr=subprocess.STDOUT, env=env)
|
|
||||||
rc = proc.wait()
|
|
||||||
with open(exit_code_path, "w") as f:
|
|
||||||
f.write(str(rc))
|
|
||||||
"""
|
|
||||||
).strip()
|
|
||||||
subprocess.Popen(
|
subprocess.Popen(
|
||||||
[
|
[setsid_bin, "bash", "-c", update_cmd],
|
||||||
sys.executable, "-c", helper,
|
|
||||||
str(output_path), str(exit_code_path),
|
|
||||||
*hermes_cmd, "update", "--gateway",
|
|
||||||
],
|
|
||||||
stdout=subprocess.DEVNULL,
|
stdout=subprocess.DEVNULL,
|
||||||
stderr=subprocess.DEVNULL,
|
stderr=subprocess.DEVNULL,
|
||||||
**windows_detach_popen_kwargs(),
|
start_new_session=True,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
hermes_cmd_str = " ".join(shlex.quote(part) for part in hermes_cmd)
|
# Fallback: start_new_session=True calls os.setsid() in child
|
||||||
update_cmd = (
|
subprocess.Popen(
|
||||||
f"PYTHONUNBUFFERED=1 {hermes_cmd_str} update --gateway"
|
["bash", "-c", update_cmd],
|
||||||
f" > {shlex.quote(str(output_path))} 2>&1; "
|
stdout=subprocess.DEVNULL,
|
||||||
f"status=$?; printf '%s' \"$status\" > {shlex.quote(str(exit_code_path))}"
|
stderr=subprocess.DEVNULL,
|
||||||
|
start_new_session=True,
|
||||||
)
|
)
|
||||||
setsid_bin = shutil.which("setsid")
|
|
||||||
if setsid_bin:
|
|
||||||
# Preferred: setsid creates a new session, fully detached
|
|
||||||
subprocess.Popen(
|
|
||||||
[setsid_bin, "bash", "-c", update_cmd],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
stderr=subprocess.DEVNULL,
|
|
||||||
start_new_session=True,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
# Fallback: start_new_session=True calls os.setsid() in child
|
|
||||||
subprocess.Popen(
|
|
||||||
["bash", "-c", update_cmd],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
stderr=subprocess.DEVNULL,
|
|
||||||
start_new_session=True,
|
|
||||||
)
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
pending_path.unlink(missing_ok=True)
|
pending_path.unlink(missing_ok=True)
|
||||||
exit_code_path.unlink(missing_ok=True)
|
exit_code_path.unlink(missing_ok=True)
|
||||||
@@ -12103,10 +11739,6 @@ class GatewayRunner:
|
|||||||
exc,
|
exc,
|
||||||
)
|
)
|
||||||
|
|
||||||
cached_source = self._get_cached_session_source(session_key)
|
|
||||||
if cached_source is not None:
|
|
||||||
return cached_source
|
|
||||||
|
|
||||||
_parsed = _parse_session_key(session_key)
|
_parsed = _parse_session_key(session_key)
|
||||||
if _parsed:
|
if _parsed:
|
||||||
derived_platform = _parsed["platform"]
|
derived_platform = _parsed["platform"]
|
||||||
@@ -12350,7 +11982,6 @@ class GatewayRunner:
|
|||||||
# Add more here as new baked-at-construction config settings are added.
|
# Add more here as new baked-at-construction config settings are added.
|
||||||
_CACHE_BUSTING_CONFIG_KEYS: tuple = (
|
_CACHE_BUSTING_CONFIG_KEYS: tuple = (
|
||||||
("model", "context_length"),
|
("model", "context_length"),
|
||||||
("model", "max_tokens"),
|
|
||||||
("compression", "enabled"),
|
("compression", "enabled"),
|
||||||
("compression", "threshold"),
|
("compression", "threshold"),
|
||||||
("compression", "target_ratio"),
|
("compression", "target_ratio"),
|
||||||
@@ -13208,24 +12839,6 @@ class GatewayRunner:
|
|||||||
last_tool = [None] # Mutable container for tracking in closure
|
last_tool = [None] # Mutable container for tracking in closure
|
||||||
last_progress_msg = [None] # Track last message for dedup
|
last_progress_msg = [None] # Track last message for dedup
|
||||||
repeat_count = [0] # How many times the same message repeated
|
repeat_count = [0] # How many times the same message repeated
|
||||||
|
|
||||||
# Auto-cleanup of temporary progress bubbles (Telegram + any adapter
|
|
||||||
# that implements ``delete_message``). When enabled via
|
|
||||||
# ``display.platforms.<platform>.cleanup_progress: true``, message IDs
|
|
||||||
# from the tool-progress / "Still working..." / status-callback bubbles
|
|
||||||
# are collected here and deleted after the final response lands.
|
|
||||||
# Failed runs skip cleanup so the bubbles remain as breadcrumbs.
|
|
||||||
_cleanup_progress = bool(
|
|
||||||
resolve_display_setting(user_config, platform_key, "cleanup_progress")
|
|
||||||
)
|
|
||||||
_cleanup_adapter = self.adapters.get(source.platform) if _cleanup_progress else None
|
|
||||||
if _cleanup_adapter is not None and (
|
|
||||||
type(_cleanup_adapter).delete_message is BasePlatformAdapter.delete_message
|
|
||||||
):
|
|
||||||
# Adapter doesn't support deletion — silently disable.
|
|
||||||
_cleanup_progress = False
|
|
||||||
_cleanup_adapter = None
|
|
||||||
_cleanup_msg_ids: List[str] = []
|
|
||||||
# First-touch onboarding latch: fires at most once per run, even if
|
# First-touch onboarding latch: fires at most once per run, even if
|
||||||
# several tools exceed the threshold.
|
# several tools exceed the threshold.
|
||||||
long_tool_hint_fired = [False]
|
long_tool_hint_fired = [False]
|
||||||
@@ -13474,18 +13087,12 @@ class GatewayRunner:
|
|||||||
adapter.name,
|
adapter.name,
|
||||||
)
|
)
|
||||||
can_edit = False
|
can_edit = False
|
||||||
_flood_result = await adapter.send(
|
await adapter.send(
|
||||||
chat_id=source.chat_id,
|
chat_id=source.chat_id,
|
||||||
content=msg,
|
content=msg,
|
||||||
reply_to=_progress_reply_to,
|
reply_to=_progress_reply_to,
|
||||||
metadata=_progress_metadata,
|
metadata=_progress_metadata,
|
||||||
)
|
)
|
||||||
if (
|
|
||||||
_cleanup_progress
|
|
||||||
and getattr(_flood_result, "success", False)
|
|
||||||
and getattr(_flood_result, "message_id", None)
|
|
||||||
):
|
|
||||||
_cleanup_msg_ids.append(str(_flood_result.message_id))
|
|
||||||
else:
|
else:
|
||||||
if can_edit:
|
if can_edit:
|
||||||
# First tool: send all accumulated text as new message
|
# First tool: send all accumulated text as new message
|
||||||
@@ -13506,8 +13113,6 @@ class GatewayRunner:
|
|||||||
)
|
)
|
||||||
if result.success and result.message_id:
|
if result.success and result.message_id:
|
||||||
progress_msg_id = result.message_id
|
progress_msg_id = result.message_id
|
||||||
if _cleanup_progress:
|
|
||||||
_cleanup_msg_ids.append(str(result.message_id))
|
|
||||||
|
|
||||||
_last_edit_ts = time.monotonic()
|
_last_edit_ts = time.monotonic()
|
||||||
|
|
||||||
@@ -13621,7 +13226,7 @@ class GatewayRunner:
|
|||||||
if not _status_adapter or not _run_still_current():
|
if not _status_adapter or not _run_still_current():
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
_fut = asyncio.run_coroutine_threadsafe(
|
asyncio.run_coroutine_threadsafe(
|
||||||
_status_adapter.send(
|
_status_adapter.send(
|
||||||
_status_chat_id,
|
_status_chat_id,
|
||||||
message,
|
message,
|
||||||
@@ -13629,16 +13234,6 @@ class GatewayRunner:
|
|||||||
),
|
),
|
||||||
_loop_for_step,
|
_loop_for_step,
|
||||||
)
|
)
|
||||||
if _cleanup_progress:
|
|
||||||
def _track_status_id(fut) -> None:
|
|
||||||
try:
|
|
||||||
res = fut.result()
|
|
||||||
except Exception:
|
|
||||||
return
|
|
||||||
mid = getattr(res, "message_id", None)
|
|
||||||
if getattr(res, "success", False) and mid:
|
|
||||||
_cleanup_msg_ids.append(str(mid))
|
|
||||||
_fut.add_done_callback(_track_status_id)
|
|
||||||
except Exception as _e:
|
except Exception as _e:
|
||||||
logger.debug("status_callback error (%s): %s", event_type, _e)
|
logger.debug("status_callback error (%s): %s", event_type, _e)
|
||||||
|
|
||||||
@@ -13672,9 +13267,13 @@ class GatewayRunner:
|
|||||||
combined_ephemeral = (combined_ephemeral + "\n\n" + self._ephemeral_system_prompt).strip()
|
combined_ephemeral = (combined_ephemeral + "\n\n" + self._ephemeral_system_prompt).strip()
|
||||||
|
|
||||||
# Re-read .env and config for fresh credentials (gateway is long-lived,
|
# Re-read .env and config for fresh credentials (gateway is long-lived,
|
||||||
# keys may change without restart). Keep config.yaml authoritative for
|
# keys may change without restart).
|
||||||
# runtime budget settings bridged into env vars.
|
try:
|
||||||
_reload_runtime_env_preserving_config_authority()
|
load_dotenv(_env_path, override=True, encoding="utf-8")
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
load_dotenv(_env_path, override=True, encoding="latin-1")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
try:
|
try:
|
||||||
model, runtime_kwargs = self._resolve_session_agent_runtime(
|
model, runtime_kwargs = self._resolve_session_agent_runtime(
|
||||||
@@ -14242,11 +13841,6 @@ class GatewayRunner:
|
|||||||
"messages": result.get("messages", []),
|
"messages": result.get("messages", []),
|
||||||
"api_calls": result.get("api_calls", 0),
|
"api_calls": result.get("api_calls", 0),
|
||||||
"failed": result.get("failed", False),
|
"failed": result.get("failed", False),
|
||||||
"partial": result.get("partial", False),
|
|
||||||
"completed": result.get("completed"),
|
|
||||||
"interrupted": result.get("interrupted", False),
|
|
||||||
"interrupt_message": result.get("interrupt_message"),
|
|
||||||
"error": result.get("error"),
|
|
||||||
"compression_exhausted": result.get("compression_exhausted", False),
|
"compression_exhausted": result.get("compression_exhausted", False),
|
||||||
"tools": tools_holder[0] or [],
|
"tools": tools_holder[0] or [],
|
||||||
"history_offset": len(agent_history),
|
"history_offset": len(agent_history),
|
||||||
@@ -14362,11 +13956,6 @@ class GatewayRunner:
|
|||||||
"last_reasoning": result.get("last_reasoning"),
|
"last_reasoning": result.get("last_reasoning"),
|
||||||
"messages": result_holder[0].get("messages", []) if result_holder[0] else [],
|
"messages": result_holder[0].get("messages", []) if result_holder[0] else [],
|
||||||
"api_calls": result_holder[0].get("api_calls", 0) if result_holder[0] else 0,
|
"api_calls": result_holder[0].get("api_calls", 0) if result_holder[0] else 0,
|
||||||
"completed": result_holder[0].get("completed") if result_holder[0] else None,
|
|
||||||
"interrupted": result_holder[0].get("interrupted", False) if result_holder[0] else False,
|
|
||||||
"partial": result_holder[0].get("partial", False) if result_holder[0] else False,
|
|
||||||
"error": result_holder[0].get("error") if result_holder[0] else None,
|
|
||||||
"interrupt_message": result_holder[0].get("interrupt_message") if result_holder[0] else None,
|
|
||||||
"tools": tools_holder[0] or [],
|
"tools": tools_holder[0] or [],
|
||||||
"history_offset": _effective_history_offset,
|
"history_offset": _effective_history_offset,
|
||||||
"last_prompt_tokens": _last_prompt_toks,
|
"last_prompt_tokens": _last_prompt_toks,
|
||||||
@@ -14505,17 +14094,11 @@ class GatewayRunner:
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
try:
|
try:
|
||||||
_notify_res = await _notify_adapter.send(
|
await _notify_adapter.send(
|
||||||
source.chat_id,
|
source.chat_id,
|
||||||
f"⏳ Still working... ({_elapsed_mins} min elapsed{_status_detail})",
|
f"⏳ Still working... ({_elapsed_mins} min elapsed{_status_detail})",
|
||||||
metadata=_status_thread_metadata,
|
metadata=_status_thread_metadata,
|
||||||
)
|
)
|
||||||
if (
|
|
||||||
_cleanup_progress
|
|
||||||
and getattr(_notify_res, "success", False)
|
|
||||||
and getattr(_notify_res, "message_id", None)
|
|
||||||
):
|
|
||||||
_cleanup_msg_ids.append(str(_notify_res.message_id))
|
|
||||||
except Exception as _ne:
|
except Exception as _ne:
|
||||||
logger.debug("Long-running notification error: %s", _ne)
|
logger.debug("Long-running notification error: %s", _ne)
|
||||||
|
|
||||||
@@ -14989,49 +14572,7 @@ class GatewayRunner:
|
|||||||
_previewed,
|
_previewed,
|
||||||
)
|
)
|
||||||
response["already_sent"] = True
|
response["already_sent"] = True
|
||||||
|
|
||||||
# Schedule deletion of tracked temporary progress bubbles after the
|
|
||||||
# final response lands. Failed runs skip this so bubbles remain as
|
|
||||||
# breadcrumbs for the user to see what work happened. Only fires on
|
|
||||||
# adapters that support ``delete_message`` (see init above); failures
|
|
||||||
# are swallowed — deletion is best-effort.
|
|
||||||
if (
|
|
||||||
_cleanup_progress
|
|
||||||
and _cleanup_adapter is not None
|
|
||||||
and _cleanup_msg_ids
|
|
||||||
and session_key
|
|
||||||
and isinstance(response, dict)
|
|
||||||
and not response.get("failed")
|
|
||||||
and hasattr(_cleanup_adapter, "register_post_delivery_callback")
|
|
||||||
):
|
|
||||||
_ids_snapshot = list(_cleanup_msg_ids)
|
|
||||||
_chat_id_snapshot = source.chat_id
|
|
||||||
_adapter_snapshot = _cleanup_adapter
|
|
||||||
_loop_snapshot = asyncio.get_running_loop()
|
|
||||||
|
|
||||||
def _cleanup_temp_bubbles() -> None:
|
|
||||||
async def _delete_all() -> None:
|
|
||||||
for _mid in _ids_snapshot:
|
|
||||||
try:
|
|
||||||
await _adapter_snapshot.delete_message(
|
|
||||||
_chat_id_snapshot, _mid
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
try:
|
|
||||||
asyncio.run_coroutine_threadsafe(_delete_all(), _loop_snapshot)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
try:
|
|
||||||
_cleanup_adapter.register_post_delivery_callback(
|
|
||||||
session_key,
|
|
||||||
_cleanup_temp_bubbles,
|
|
||||||
generation=run_generation,
|
|
||||||
)
|
|
||||||
except Exception as _rpe:
|
|
||||||
logger.debug("Post-delivery cleanup registration failed: %s", _rpe)
|
|
||||||
|
|
||||||
return response
|
return response
|
||||||
|
|
||||||
|
|
||||||
@@ -15194,10 +14735,7 @@ async def start_gateway(config: Optional[GatewayConfig] = None, replace: bool =
|
|||||||
try:
|
try:
|
||||||
os.kill(existing_pid, 0)
|
os.kill(existing_pid, 0)
|
||||||
time.sleep(0.5)
|
time.sleep(0.5)
|
||||||
except (ProcessLookupError, PermissionError, OSError):
|
except (ProcessLookupError, PermissionError):
|
||||||
# OSError covers Windows' WinError 87 "invalid parameter"
|
|
||||||
# for an already-gone PID — without this the probe loop
|
|
||||||
# busy-spins for the full 10s on every --replace start.
|
|
||||||
break # Process is gone
|
break # Process is gone
|
||||||
else:
|
else:
|
||||||
# Still alive after 10s — force kill
|
# Still alive after 10s — force kill
|
||||||
@@ -15482,14 +15020,6 @@ async def start_gateway(config: Optional[GatewayConfig] = None, replace: bool =
|
|||||||
|
|
||||||
def main():
|
def main():
|
||||||
"""CLI entry point for the gateway."""
|
"""CLI entry point for the gateway."""
|
||||||
# Force UTF-8 stdio on Windows — gateway logs and startup banner would
|
|
||||||
# otherwise UnicodeEncodeError on cp1252 consoles. No-op on POSIX.
|
|
||||||
try:
|
|
||||||
from hermes_cli.stdio import configure_windows_stdio
|
|
||||||
configure_windows_stdio()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
|
|
||||||
parser = argparse.ArgumentParser(description="Hermes Gateway - Multi-platform messaging")
|
parser = argparse.ArgumentParser(description="Hermes Gateway - Multi-platform messaging")
|
||||||
|
|||||||
+3
-3
@@ -113,7 +113,7 @@ def _get_process_start_time(pid: int) -> Optional[int]:
|
|||||||
stat_path = Path(f"/proc/{pid}/stat")
|
stat_path = Path(f"/proc/{pid}/stat")
|
||||||
try:
|
try:
|
||||||
# Field 22 in /proc/<pid>/stat is process start time (clock ticks).
|
# Field 22 in /proc/<pid>/stat is process start time (clock ticks).
|
||||||
return int(stat_path.read_text(encoding="utf-8").split()[21])
|
return int(stat_path.read_text().split()[21])
|
||||||
except (FileNotFoundError, IndexError, PermissionError, ValueError, OSError):
|
except (FileNotFoundError, IndexError, PermissionError, ValueError, OSError):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@@ -197,7 +197,7 @@ def _read_json_file(path: Path) -> Optional[dict[str, Any]]:
|
|||||||
if not path.exists():
|
if not path.exists():
|
||||||
return None
|
return None
|
||||||
try:
|
try:
|
||||||
raw = path.read_text(encoding="utf-8").strip()
|
raw = path.read_text().strip()
|
||||||
except OSError:
|
except OSError:
|
||||||
return None
|
return None
|
||||||
if not raw:
|
if not raw:
|
||||||
@@ -523,7 +523,7 @@ def acquire_scoped_lock(scope: str, identity: str, metadata: Optional[dict[str,
|
|||||||
try:
|
try:
|
||||||
_proc_status = Path(f"/proc/{existing_pid}/status")
|
_proc_status = Path(f"/proc/{existing_pid}/status")
|
||||||
if _proc_status.exists():
|
if _proc_status.exists():
|
||||||
for _line in _proc_status.read_text(encoding="utf-8").splitlines():
|
for _line in _proc_status.read_text().splitlines():
|
||||||
if _line.startswith("State:"):
|
if _line.startswith("State:"):
|
||||||
_state = _line.split()[1]
|
_state = _line.split()[1]
|
||||||
if _state in ("T", "t"): # stopped or tracing stop
|
if _state in ("T", "t"): # stopped or tracing stop
|
||||||
|
|||||||
@@ -1,129 +0,0 @@
|
|||||||
"""Windows UTF-8 bootstrap for Hermes entry points.
|
|
||||||
|
|
||||||
Python on Windows has two long-standing text-encoding footguns:
|
|
||||||
|
|
||||||
1. ``sys.stdout`` / ``sys.stderr`` are bound to the console code page
|
|
||||||
(``cp1252`` on US-locale installs), so ``print("café")`` crashes with
|
|
||||||
``UnicodeEncodeError: 'charmap' codec can't encode character``.
|
|
||||||
|
|
||||||
2. Child processes spawned via ``subprocess`` don't know to use UTF-8
|
|
||||||
unless ``PYTHONUTF8`` and/or ``PYTHONIOENCODING`` are set in their
|
|
||||||
environment — so any Python subprocess (the execute_code sandbox,
|
|
||||||
delegation children, linter subprocesses, etc.) inherits the same
|
|
||||||
cp1252 defaults and hits the same UnicodeEncodeError.
|
|
||||||
|
|
||||||
This module fixes both on Windows *only* — POSIX is untouched. It
|
|
||||||
should be imported at the very top of every Hermes entry point
|
|
||||||
(``hermes``, ``hermes-agent``, ``hermes-acp``, ``python -m gateway.run``,
|
|
||||||
``batch_runner.py``, ``cron/scheduler.py``) before any other imports
|
|
||||||
that might do file I/O or print to stdout.
|
|
||||||
|
|
||||||
What this module does on Windows:
|
|
||||||
|
|
||||||
- Sets ``os.environ["PYTHONUTF8"] = "1"`` (PEP 540 UTF-8 mode) so
|
|
||||||
every child process we spawn uses UTF-8 for ``open()`` and stdio.
|
|
||||||
- Sets ``os.environ["PYTHONIOENCODING"] = "utf-8"`` for belt-and-
|
|
||||||
suspenders — some tools read this instead of / in addition to
|
|
||||||
``PYTHONUTF8``.
|
|
||||||
- Reconfigures ``sys.stdout`` / ``sys.stderr`` to UTF-8 in the current
|
|
||||||
process, using the ``reconfigure()`` API (Python 3.7+). This fixes
|
|
||||||
``print("café")`` in the parent without a re-exec.
|
|
||||||
|
|
||||||
What this module does NOT do:
|
|
||||||
|
|
||||||
- It does not re-exec Python with ``-X utf8``, so ``open()`` calls in
|
|
||||||
the *current* process still default to locale encoding. Those need
|
|
||||||
an explicit ``encoding="utf-8"`` at the call site (lint rule
|
|
||||||
``PLW1514`` / ``PYI058``). Ruff is the right tool for that sweep.
|
|
||||||
|
|
||||||
What this module does on POSIX:
|
|
||||||
|
|
||||||
- Nothing. POSIX systems are already UTF-8 by default in 99% of cases,
|
|
||||||
and we don't want to touch ``LANG``/``LC_*`` behavior that users may
|
|
||||||
have configured intentionally. If someone hits a C/POSIX locale on
|
|
||||||
Linux, they can export ``PYTHONUTF8=1`` themselves — we won't override.
|
|
||||||
|
|
||||||
Idempotent: safe to call multiple times. ``_bootstrap_once`` guards
|
|
||||||
against double-reconfigure.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
|
|
||||||
_IS_WINDOWS = sys.platform == "win32"
|
|
||||||
_bootstrap_applied = False
|
|
||||||
|
|
||||||
|
|
||||||
def apply_windows_utf8_bootstrap() -> bool:
|
|
||||||
"""Apply the Windows UTF-8 bootstrap if we're on Windows.
|
|
||||||
|
|
||||||
Returns True if bootstrap was applied (i.e. we're on Windows and
|
|
||||||
haven't already done this), False otherwise. The return value is
|
|
||||||
advisory — callers normally don't need it, but tests may want to
|
|
||||||
assert the path was taken.
|
|
||||||
|
|
||||||
Idempotent: subsequent calls after the first are a no-op.
|
|
||||||
"""
|
|
||||||
global _bootstrap_applied
|
|
||||||
|
|
||||||
if not _IS_WINDOWS:
|
|
||||||
return False
|
|
||||||
if _bootstrap_applied:
|
|
||||||
return False
|
|
||||||
|
|
||||||
# 1. Child processes inherit these and run in UTF-8 mode.
|
|
||||||
# We use setdefault() rather than overwriting so the user can
|
|
||||||
# explicitly opt out by setting PYTHONUTF8=0 in their environment
|
|
||||||
# (or PYTHONIOENCODING=something-else) if they really want to.
|
|
||||||
os.environ.setdefault("PYTHONUTF8", "1")
|
|
||||||
os.environ.setdefault("PYTHONIOENCODING", "utf-8")
|
|
||||||
|
|
||||||
# 2. Reconfigure the current process's stdio to UTF-8. Needed
|
|
||||||
# because os.environ changes don't retroactively rebind sys.stdout
|
|
||||||
# — those were bound at interpreter startup based on the console
|
|
||||||
# code page. ``reconfigure`` is a TextIOWrapper method since 3.7.
|
|
||||||
#
|
|
||||||
# errors="replace" means that if we ever *read* something from
|
|
||||||
# stdin that isn't UTF-8 (unlikely but possible with piped input
|
|
||||||
# from legacy tools), we'll get U+FFFD replacement chars rather
|
|
||||||
# than a crash. Output is pure UTF-8.
|
|
||||||
for stream_name in ("stdout", "stderr"):
|
|
||||||
stream = getattr(sys, stream_name, None)
|
|
||||||
if stream is None:
|
|
||||||
continue
|
|
||||||
reconfigure = getattr(stream, "reconfigure", None)
|
|
||||||
if reconfigure is None:
|
|
||||||
# Not a TextIOWrapper (could be redirected to a BytesIO in
|
|
||||||
# tests, or a non-standard stream in some embedded cases).
|
|
||||||
# Skip silently — the env-var fix is still in effect for
|
|
||||||
# child processes, which is the bigger win.
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
reconfigure(encoding="utf-8", errors="replace")
|
|
||||||
except (OSError, ValueError):
|
|
||||||
# Already closed, or someone replaced it with something
|
|
||||||
# non-reconfigurable. Non-fatal.
|
|
||||||
pass
|
|
||||||
|
|
||||||
# stdin is reconfigured separately with errors="replace" too — input
|
|
||||||
# from a legacy pipe shouldn't crash the process.
|
|
||||||
stdin = getattr(sys, "stdin", None)
|
|
||||||
if stdin is not None:
|
|
||||||
reconfigure = getattr(stdin, "reconfigure", None)
|
|
||||||
if reconfigure is not None:
|
|
||||||
try:
|
|
||||||
reconfigure(encoding="utf-8", errors="replace")
|
|
||||||
except (OSError, ValueError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
_bootstrap_applied = True
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
# Apply on import — entry points just need ``import hermes_bootstrap``
|
|
||||||
# (or ``from hermes_bootstrap import apply_windows_utf8_bootstrap``) at
|
|
||||||
# the very top of their module, before importing anything else. The
|
|
||||||
# import side effect does the right thing.
|
|
||||||
apply_windows_utf8_bootstrap()
|
|
||||||
@@ -14,8 +14,8 @@ Provides subcommands for:
|
|||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
__version__ = "0.13.0"
|
__version__ = "0.12.0"
|
||||||
__release_date__ = "2026.5.7"
|
__release_date__ = "2026.4.30"
|
||||||
|
|
||||||
|
|
||||||
def _ensure_utf8():
|
def _ensure_utf8():
|
||||||
|
|||||||
@@ -70,9 +70,6 @@ Examples:
|
|||||||
hermes logs --since 1h Lines from the last hour
|
hermes logs --since 1h Lines from the last hour
|
||||||
hermes debug share Upload debug report for support
|
hermes debug share Upload debug report for support
|
||||||
hermes update Update to latest version
|
hermes update Update to latest version
|
||||||
hermes dashboard Start web UI dashboard (port 9119)
|
|
||||||
hermes dashboard --stop Stop running dashboard processes
|
|
||||||
hermes dashboard --status List running dashboard processes
|
|
||||||
|
|
||||||
For more help on a command:
|
For more help on a command:
|
||||||
hermes <command> --help
|
hermes <command> --help
|
||||||
|
|||||||
@@ -1,175 +0,0 @@
|
|||||||
"""Windows subprocess compatibility helpers.
|
|
||||||
|
|
||||||
Hermes is developed on Linux / macOS and tested natively on Windows too.
|
|
||||||
Several common subprocess patterns break silently-or-loudly on Windows:
|
|
||||||
|
|
||||||
* ``["npm", "install", ...]`` — on Windows ``npm`` is ``npm.cmd``, a batch
|
|
||||||
shim. ``subprocess.Popen(["npm", ...])`` fails with WinError 193
|
|
||||||
("not a valid Win32 application") because CreateProcessW can't run a
|
|
||||||
``.cmd`` file without ``shell=True`` or PATHEXT resolution.
|
|
||||||
|
|
||||||
* ``start_new_session=True`` — on POSIX, this maps to ``os.setsid()`` and
|
|
||||||
actually detaches the child. On Windows it's silently ignored; the
|
|
||||||
Windows equivalent is ``CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS``
|
|
||||||
creationflags, which Python only applies when you pass them explicitly.
|
|
||||||
|
|
||||||
* Console-window flashes — every ``subprocess.Popen`` of a ``.exe`` on
|
|
||||||
Windows spawns a cmd window briefly unless ``CREATE_NO_WINDOW`` is
|
|
||||||
passed. Cosmetic but jarring for background daemons.
|
|
||||||
|
|
||||||
This module centralizes the platform-branching logic so the rest of the
|
|
||||||
codebase doesn't sprinkle ``if sys.platform == "win32":`` everywhere.
|
|
||||||
|
|
||||||
**All helpers are no-ops on non-Windows** — calling them in Linux/macOS
|
|
||||||
code paths is safe by design. That's the "do no damage on POSIX"
|
|
||||||
guarantee.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from typing import Optional, Sequence
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"IS_WINDOWS",
|
|
||||||
"resolve_node_command",
|
|
||||||
"windows_detach_flags",
|
|
||||||
"windows_hide_flags",
|
|
||||||
"windows_detach_popen_kwargs",
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
IS_WINDOWS = sys.platform == "win32"
|
|
||||||
|
|
||||||
|
|
||||||
# -----------------------------------------------------------------------------
|
|
||||||
# Node ecosystem launcher resolution
|
|
||||||
# -----------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_node_command(name: str, argv: Sequence[str]) -> list[str]:
|
|
||||||
"""Resolve a Node-ecosystem command name to an absolute-path argv.
|
|
||||||
|
|
||||||
On Windows, commands like ``npm``, ``npx``, ``yarn``, ``pnpm``,
|
|
||||||
``playwright``, ``prettier`` ship as ``.cmd`` files (batch shims).
|
|
||||||
``subprocess.Popen(["npm", "install"])`` fails with WinError 193
|
|
||||||
because CreateProcessW doesn't execute batch files directly.
|
|
||||||
|
|
||||||
``shutil.which(name)`` *does* resolve ``.cmd`` via PATHEXT and returns
|
|
||||||
the fully-qualified path — which CreateProcessW accepts because the
|
|
||||||
extension tells Windows to route through ``cmd.exe /c``.
|
|
||||||
|
|
||||||
On POSIX ``shutil.which`` also returns a fully-qualified path when
|
|
||||||
found. That's a small change from bare-name resolution (the OS does
|
|
||||||
its own PATH search) but functionally identical and has the side
|
|
||||||
benefit of making the argv reproducible in logs.
|
|
||||||
|
|
||||||
Behavior when the command is not on PATH:
|
|
||||||
- On Windows: return the bare name — caller can still try with
|
|
||||||
``shell=True`` as a last resort, OR the subsequent Popen will
|
|
||||||
raise FileNotFoundError with a readable error we want to surface.
|
|
||||||
- On POSIX: same. Bare ``npm`` on a Linux box without npm installed
|
|
||||||
fails the same way it did before this function existed.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
name: The command name to resolve (``npm``, ``npx``, ``node`` …).
|
|
||||||
argv: The remaining arguments. Must NOT include ``name`` itself —
|
|
||||||
this function builds the full argv list.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
A list suitable for passing to subprocess.Popen/run/call.
|
|
||||||
"""
|
|
||||||
resolved = shutil.which(name)
|
|
||||||
if resolved:
|
|
||||||
return [resolved, *argv]
|
|
||||||
return [name, *argv]
|
|
||||||
|
|
||||||
|
|
||||||
# -----------------------------------------------------------------------------
|
|
||||||
# Detached / hidden process creation
|
|
||||||
# -----------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# Win32 CreationFlags — defined here rather than imported from subprocess
|
|
||||||
# because CREATE_NO_WINDOW and DETACHED_PROCESS aren't guaranteed to be
|
|
||||||
# present on stdlib subprocess on older Pythons or non-Windows builds.
|
|
||||||
_CREATE_NEW_PROCESS_GROUP = 0x00000200
|
|
||||||
_DETACHED_PROCESS = 0x00000008
|
|
||||||
_CREATE_NO_WINDOW = 0x08000000
|
|
||||||
|
|
||||||
|
|
||||||
def windows_detach_flags() -> int:
|
|
||||||
"""Return Win32 creationflags that detach a child from the parent
|
|
||||||
console and process group. 0 on non-Windows.
|
|
||||||
|
|
||||||
Pair with ``start_new_session=False`` (default) when calling
|
|
||||||
subprocess.Popen — on POSIX use ``start_new_session=True`` instead,
|
|
||||||
which maps to ``os.setsid()`` in the child.
|
|
||||||
|
|
||||||
Rationale:
|
|
||||||
- ``CREATE_NEW_PROCESS_GROUP`` — child has its own process group so
|
|
||||||
Ctrl+C in the parent console doesn't propagate.
|
|
||||||
- ``DETACHED_PROCESS`` — child has no console at all. Necessary for
|
|
||||||
background daemons (gateway watchers, update respawners) because
|
|
||||||
without it, closing the console kills the child.
|
|
||||||
- ``CREATE_NO_WINDOW`` — suppress the brief cmd flash that would
|
|
||||||
otherwise appear when launching a console app. Redundant with
|
|
||||||
DETACHED_PROCESS but explicit for clarity.
|
|
||||||
"""
|
|
||||||
if not IS_WINDOWS:
|
|
||||||
return 0
|
|
||||||
return _CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW
|
|
||||||
|
|
||||||
|
|
||||||
def windows_hide_flags() -> int:
|
|
||||||
"""Return Win32 creationflags that merely hide the child's console
|
|
||||||
window without detaching the child. 0 on non-Windows.
|
|
||||||
|
|
||||||
Use for short-lived console apps spawned as part of a larger
|
|
||||||
operation (``taskkill``, ``where``, version probes) where we want no
|
|
||||||
flash but also want to collect stdout/exit code synchronously.
|
|
||||||
|
|
||||||
The key difference from :func:`windows_detach_flags`: NO
|
|
||||||
``DETACHED_PROCESS`` — the child still inherits stdio handles so
|
|
||||||
``capture_output=True`` works. ``DETACHED_PROCESS`` would sever
|
|
||||||
stdio and break stdout capture.
|
|
||||||
"""
|
|
||||||
if not IS_WINDOWS:
|
|
||||||
return 0
|
|
||||||
return _CREATE_NO_WINDOW
|
|
||||||
|
|
||||||
|
|
||||||
def windows_detach_popen_kwargs() -> dict:
|
|
||||||
"""Return a dict of Popen kwargs that detach a child on Windows and
|
|
||||||
fall back to the POSIX equivalent (``start_new_session=True``) on
|
|
||||||
Linux/macOS.
|
|
||||||
|
|
||||||
Usage pattern:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
subprocess.Popen(
|
|
||||||
argv,
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
stderr=subprocess.DEVNULL,
|
|
||||||
stdin=subprocess.DEVNULL,
|
|
||||||
close_fds=True,
|
|
||||||
**windows_detach_popen_kwargs(),
|
|
||||||
)
|
|
||||||
|
|
||||||
This replaces the unsafe-on-Windows pattern:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
subprocess.Popen(..., start_new_session=True)
|
|
||||||
|
|
||||||
which silently fails to detach on Windows (the flag is accepted but
|
|
||||||
has no effect — the child stays attached to the parent's console
|
|
||||||
and dies when the console closes).
|
|
||||||
"""
|
|
||||||
if IS_WINDOWS:
|
|
||||||
return {"creationflags": windows_detach_flags()}
|
|
||||||
return {"start_new_session": True}
|
|
||||||
+191
-375
@@ -853,48 +853,36 @@ def _auth_lock_path() -> Path:
|
|||||||
|
|
||||||
_auth_lock_holder = threading.local()
|
_auth_lock_holder = threading.local()
|
||||||
|
|
||||||
|
|
||||||
@contextmanager
|
@contextmanager
|
||||||
def _file_lock(
|
def _auth_store_lock(timeout_seconds: float = AUTH_LOCK_TIMEOUT_SECONDS):
|
||||||
lock_path: Path,
|
"""Cross-process advisory lock for auth.json reads+writes. Reentrant."""
|
||||||
holder: threading.local,
|
# Reentrant: if this thread already holds the lock, just yield.
|
||||||
timeout_seconds: float,
|
if getattr(_auth_lock_holder, "depth", 0) > 0:
|
||||||
timeout_message: str,
|
_auth_lock_holder.depth += 1
|
||||||
):
|
|
||||||
"""Cross-process advisory flock helper.
|
|
||||||
|
|
||||||
Reentrant per-thread via ``holder.depth``. Falls back to a depth-only
|
|
||||||
guard when neither ``fcntl`` nor ``msvcrt`` is available (rare).
|
|
||||||
Callers supply their own ``threading.local`` so independent locks
|
|
||||||
(e.g. profile auth.json vs shared Nous store) don't share reentrancy
|
|
||||||
state — that would let one lock's reentrant acquisition silently skip
|
|
||||||
the other's kernel-level flock.
|
|
||||||
"""
|
|
||||||
if getattr(holder, "depth", 0) > 0:
|
|
||||||
holder.depth += 1
|
|
||||||
try:
|
try:
|
||||||
yield
|
yield
|
||||||
finally:
|
finally:
|
||||||
holder.depth -= 1
|
_auth_lock_holder.depth -= 1
|
||||||
return
|
return
|
||||||
|
|
||||||
|
lock_path = _auth_lock_path()
|
||||||
lock_path.parent.mkdir(parents=True, exist_ok=True)
|
lock_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
if fcntl is None and msvcrt is None:
|
if fcntl is None and msvcrt is None:
|
||||||
holder.depth = 1
|
_auth_lock_holder.depth = 1
|
||||||
try:
|
try:
|
||||||
yield
|
yield
|
||||||
finally:
|
finally:
|
||||||
holder.depth = 0
|
_auth_lock_holder.depth = 0
|
||||||
return
|
return
|
||||||
|
|
||||||
# On Windows, msvcrt.locking needs the file to have content and the
|
# On Windows, msvcrt.locking needs the file to have content and the
|
||||||
# file pointer at position 0. Ensure the lock file has at least 1 byte.
|
# file pointer at position 0. Ensure the lock file has at least 1 byte.
|
||||||
if msvcrt and (not lock_path.exists() or lock_path.stat().st_size == 0):
|
if msvcrt and (not lock_path.exists() or lock_path.stat().st_size == 0):
|
||||||
lock_path.write_text(" ", encoding="utf-8")
|
lock_path.write_text(" ", encoding="utf-8")
|
||||||
|
|
||||||
with lock_path.open("r+" if msvcrt else "a+") as lock_file:
|
with lock_path.open("r+" if msvcrt else "a+") as lock_file:
|
||||||
deadline = time.monotonic() + max(1.0, timeout_seconds)
|
deadline = time.time() + max(1.0, timeout_seconds)
|
||||||
while True:
|
while True:
|
||||||
try:
|
try:
|
||||||
if fcntl:
|
if fcntl:
|
||||||
@@ -904,15 +892,15 @@ def _file_lock(
|
|||||||
msvcrt.locking(lock_file.fileno(), msvcrt.LK_NBLCK, 1)
|
msvcrt.locking(lock_file.fileno(), msvcrt.LK_NBLCK, 1)
|
||||||
break
|
break
|
||||||
except (BlockingIOError, OSError, PermissionError):
|
except (BlockingIOError, OSError, PermissionError):
|
||||||
if time.monotonic() >= deadline:
|
if time.time() >= deadline:
|
||||||
raise TimeoutError(timeout_message)
|
raise TimeoutError("Timed out waiting for auth store lock")
|
||||||
time.sleep(0.05)
|
time.sleep(0.05)
|
||||||
|
|
||||||
holder.depth = 1
|
_auth_lock_holder.depth = 1
|
||||||
try:
|
try:
|
||||||
yield
|
yield
|
||||||
finally:
|
finally:
|
||||||
holder.depth = 0
|
_auth_lock_holder.depth = 0
|
||||||
if fcntl:
|
if fcntl:
|
||||||
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
|
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
|
||||||
elif msvcrt:
|
elif msvcrt:
|
||||||
@@ -923,25 +911,6 @@ def _file_lock(
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
@contextmanager
|
|
||||||
def _auth_store_lock(timeout_seconds: float = AUTH_LOCK_TIMEOUT_SECONDS):
|
|
||||||
"""Cross-process advisory lock for auth.json reads+writes. Reentrant.
|
|
||||||
|
|
||||||
Lock ordering invariant: when this lock is held together with
|
|
||||||
``_nous_shared_store_lock``, acquire ``_auth_store_lock`` FIRST
|
|
||||||
(outer) and the shared Nous lock SECOND (inner). All runtime
|
|
||||||
refresh paths follow this order; violating it risks deadlock
|
|
||||||
against a concurrent import on the shared store.
|
|
||||||
"""
|
|
||||||
with _file_lock(
|
|
||||||
_auth_lock_path(),
|
|
||||||
_auth_lock_holder,
|
|
||||||
timeout_seconds,
|
|
||||||
"Timed out waiting for auth store lock",
|
|
||||||
):
|
|
||||||
yield
|
|
||||||
|
|
||||||
|
|
||||||
def _load_auth_store(auth_file: Optional[Path] = None) -> Dict[str, Any]:
|
def _load_auth_store(auth_file: Optional[Path] = None) -> Dict[str, Any]:
|
||||||
auth_file = auth_file or _auth_file_path()
|
auth_file = auth_file or _auth_file_path()
|
||||||
if not auth_file.exists():
|
if not auth_file.exists():
|
||||||
@@ -985,27 +954,12 @@ def _load_auth_store(auth_file: Optional[Path] = None) -> Dict[str, Any]:
|
|||||||
def _save_auth_store(auth_store: Dict[str, Any]) -> Path:
|
def _save_auth_store(auth_store: Dict[str, Any]) -> Path:
|
||||||
auth_file = _auth_file_path()
|
auth_file = _auth_file_path()
|
||||||
auth_file.parent.mkdir(parents=True, exist_ok=True)
|
auth_file.parent.mkdir(parents=True, exist_ok=True)
|
||||||
# Tighten parent dir to 0o700 so siblings can't traverse to creds.
|
|
||||||
# No-op on Windows (POSIX mode bits not enforced); ignore failures.
|
|
||||||
try:
|
|
||||||
os.chmod(auth_file.parent, 0o700)
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
auth_store["version"] = AUTH_STORE_VERSION
|
auth_store["version"] = AUTH_STORE_VERSION
|
||||||
auth_store["updated_at"] = datetime.now(timezone.utc).isoformat()
|
auth_store["updated_at"] = datetime.now(timezone.utc).isoformat()
|
||||||
payload = json.dumps(auth_store, indent=2) + "\n"
|
payload = json.dumps(auth_store, indent=2) + "\n"
|
||||||
tmp_path = auth_file.with_name(f"{auth_file.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
|
tmp_path = auth_file.with_name(f"{auth_file.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
|
||||||
try:
|
try:
|
||||||
# Create with 0o600 atomically via os.open(O_EXCL) + fdopen to close
|
with tmp_path.open("w", encoding="utf-8") as handle:
|
||||||
# the TOCTOU window where default umask (often 0o644) briefly exposed
|
|
||||||
# OAuth tokens to other local users between open() and chmod().
|
|
||||||
# Mirrors agent/google_oauth.py (#19673) and tools/mcp_oauth.py (#21148).
|
|
||||||
fd = os.open(
|
|
||||||
str(tmp_path),
|
|
||||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
|
|
||||||
stat.S_IRUSR | stat.S_IWUSR,
|
|
||||||
)
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as handle:
|
|
||||||
handle.write(payload)
|
handle.write(payload)
|
||||||
handle.flush()
|
handle.flush()
|
||||||
os.fsync(handle.fileno())
|
os.fsync(handle.fileno())
|
||||||
@@ -1569,33 +1523,10 @@ def _read_qwen_cli_tokens() -> Dict[str, Any]:
|
|||||||
def _save_qwen_cli_tokens(tokens: Dict[str, Any]) -> Path:
|
def _save_qwen_cli_tokens(tokens: Dict[str, Any]) -> Path:
|
||||||
auth_path = _qwen_cli_auth_path()
|
auth_path = _qwen_cli_auth_path()
|
||||||
auth_path.parent.mkdir(parents=True, exist_ok=True)
|
auth_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
try:
|
tmp_path = auth_path.with_suffix(".tmp")
|
||||||
os.chmod(auth_path.parent, 0o700)
|
tmp_path.write_text(json.dumps(tokens, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||||
except OSError:
|
os.chmod(tmp_path, stat.S_IRUSR | stat.S_IWUSR)
|
||||||
pass
|
tmp_path.replace(auth_path)
|
||||||
# Per-process random temp suffix avoids collisions between concurrent
|
|
||||||
# writers and stale leftovers from a crashed prior write.
|
|
||||||
tmp_path = auth_path.with_name(f"{auth_path.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
|
|
||||||
# Create with 0o600 atomically via os.open(O_EXCL) — closes the TOCTOU
|
|
||||||
# window where write_text() + post-write chmod briefly exposed tokens
|
|
||||||
# at process umask (typically 0o644). See #19673, #21148.
|
|
||||||
fd = os.open(
|
|
||||||
str(tmp_path),
|
|
||||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
|
|
||||||
stat.S_IRUSR | stat.S_IWUSR,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
|
||||||
fh.write(json.dumps(tokens, indent=2, sort_keys=True) + "\n")
|
|
||||||
fh.flush()
|
|
||||||
os.fsync(fh.fileno())
|
|
||||||
atomic_replace(tmp_path, auth_path)
|
|
||||||
finally:
|
|
||||||
try:
|
|
||||||
if tmp_path.exists():
|
|
||||||
tmp_path.unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
return auth_path
|
return auth_path
|
||||||
|
|
||||||
|
|
||||||
@@ -2012,9 +1943,9 @@ def _spotify_wait_for_callback(
|
|||||||
|
|
||||||
thread = threading.Thread(target=server.serve_forever, kwargs={"poll_interval": 0.1}, daemon=True)
|
thread = threading.Thread(target=server.serve_forever, kwargs={"poll_interval": 0.1}, daemon=True)
|
||||||
thread.start()
|
thread.start()
|
||||||
deadline = time.monotonic() + max(5.0, timeout_seconds)
|
deadline = time.time() + max(5.0, timeout_seconds)
|
||||||
try:
|
try:
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
if result["code"] or result["error"]:
|
if result["code"] or result["error"]:
|
||||||
return result
|
return result
|
||||||
time.sleep(0.1)
|
time.sleep(0.1)
|
||||||
@@ -2777,10 +2708,10 @@ def _poll_for_token(
|
|||||||
poll_interval: int,
|
poll_interval: int,
|
||||||
) -> Dict[str, Any]:
|
) -> Dict[str, Any]:
|
||||||
"""Poll the token endpoint until the user approves or the code expires."""
|
"""Poll the token endpoint until the user approves or the code expires."""
|
||||||
deadline = time.monotonic() + max(1, expires_in)
|
deadline = time.time() + max(1, expires_in)
|
||||||
current_interval = max(1, min(poll_interval, DEVICE_AUTH_POLL_INTERVAL_CAP_SECONDS))
|
current_interval = max(1, min(poll_interval, DEVICE_AUTH_POLL_INTERVAL_CAP_SECONDS))
|
||||||
|
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
response = client.post(
|
response = client.post(
|
||||||
f"{portal_base_url}/api/oauth/token",
|
f"{portal_base_url}/api/oauth/token",
|
||||||
data={
|
data={
|
||||||
@@ -2838,7 +2769,6 @@ def _poll_for_token(
|
|||||||
# -----------------------------------------------------------------------------
|
# -----------------------------------------------------------------------------
|
||||||
|
|
||||||
NOUS_SHARED_STORE_FILENAME = "nous_auth.json"
|
NOUS_SHARED_STORE_FILENAME = "nous_auth.json"
|
||||||
_nous_shared_lock_holder = threading.local()
|
|
||||||
|
|
||||||
|
|
||||||
def _nous_shared_auth_dir() -> Path:
|
def _nous_shared_auth_dir() -> Path:
|
||||||
@@ -2878,69 +2808,6 @@ def _nous_shared_store_path() -> Path:
|
|||||||
return path
|
return path
|
||||||
|
|
||||||
|
|
||||||
@contextmanager
|
|
||||||
def _nous_shared_store_lock(timeout_seconds: float = AUTH_LOCK_TIMEOUT_SECONDS):
|
|
||||||
"""Cross-profile lock for the shared Nous OAuth store.
|
|
||||||
|
|
||||||
Lock ordering invariant: if both this and ``_auth_store_lock`` need
|
|
||||||
to be held, acquire ``_auth_store_lock`` FIRST. All runtime refresh
|
|
||||||
paths follow this order. The one exception is
|
|
||||||
``_try_import_shared_nous_state``, which holds this lock alone for
|
|
||||||
the entire refresh+mint cycle so concurrent imports on sibling
|
|
||||||
profiles can't race on the single-use shared refresh token; that
|
|
||||||
helper must NOT be called with ``_auth_store_lock`` already held.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
lock_path = _nous_shared_store_path().with_suffix(".lock")
|
|
||||||
except RuntimeError:
|
|
||||||
# No HERMES_HOME yet (pre-setup): fall through without locking.
|
|
||||||
yield
|
|
||||||
return
|
|
||||||
|
|
||||||
with _file_lock(
|
|
||||||
lock_path,
|
|
||||||
_nous_shared_lock_holder,
|
|
||||||
timeout_seconds,
|
|
||||||
"Timed out waiting for shared Nous auth lock",
|
|
||||||
):
|
|
||||||
yield
|
|
||||||
|
|
||||||
|
|
||||||
def _merge_shared_nous_oauth_state(state: Dict[str, Any]) -> bool:
|
|
||||||
"""Copy fresher shared OAuth tokens into a profile-local Nous state."""
|
|
||||||
shared = _read_shared_nous_state()
|
|
||||||
if not shared:
|
|
||||||
return False
|
|
||||||
|
|
||||||
shared_refresh = shared.get("refresh_token")
|
|
||||||
if not isinstance(shared_refresh, str) or not shared_refresh.strip():
|
|
||||||
return False
|
|
||||||
|
|
||||||
local_refresh = state.get("refresh_token")
|
|
||||||
shared_access_exp = _parse_iso_timestamp(shared.get("expires_at")) or 0.0
|
|
||||||
local_access_exp = _parse_iso_timestamp(state.get("expires_at")) or 0.0
|
|
||||||
refresh_changed = shared_refresh.strip() != str(local_refresh or "").strip()
|
|
||||||
fresher_access = shared_access_exp > local_access_exp
|
|
||||||
if not refresh_changed and not fresher_access:
|
|
||||||
return False
|
|
||||||
|
|
||||||
for key in (
|
|
||||||
"access_token",
|
|
||||||
"refresh_token",
|
|
||||||
"token_type",
|
|
||||||
"scope",
|
|
||||||
"client_id",
|
|
||||||
"portal_base_url",
|
|
||||||
"inference_base_url",
|
|
||||||
"obtained_at",
|
|
||||||
"expires_at",
|
|
||||||
):
|
|
||||||
value = shared.get(key)
|
|
||||||
if value not in (None, ""):
|
|
||||||
state[key] = value
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def _write_shared_nous_state(state: Dict[str, Any]) -> None:
|
def _write_shared_nous_state(state: Dict[str, Any]) -> None:
|
||||||
"""Persist a minimal copy of the Nous OAuth state to the shared store.
|
"""Persist a minimal copy of the Nous OAuth state to the shared store.
|
||||||
|
|
||||||
@@ -2973,34 +2840,15 @@ def _write_shared_nous_state(state: Dict[str, Any]) -> None:
|
|||||||
"updated_at": datetime.now(timezone.utc).isoformat(),
|
"updated_at": datetime.now(timezone.utc).isoformat(),
|
||||||
}
|
}
|
||||||
try:
|
try:
|
||||||
with _nous_shared_store_lock():
|
path = _nous_shared_store_path()
|
||||||
path = _nous_shared_store_path()
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||||
try:
|
tmp.write_text(json.dumps(shared, indent=2, sort_keys=True))
|
||||||
os.chmod(path.parent, 0o700)
|
try:
|
||||||
except OSError:
|
os.chmod(tmp, 0o600)
|
||||||
pass
|
except OSError:
|
||||||
tmp = path.with_name(f"{path.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
|
pass
|
||||||
# Create with 0o600 atomically via os.open(O_EXCL) — closes the TOCTOU
|
os.replace(tmp, path)
|
||||||
# window where write_text() + post-write chmod briefly exposed Nous
|
|
||||||
# refresh_token at process umask. See #19673, #21148.
|
|
||||||
fd = os.open(
|
|
||||||
str(tmp),
|
|
||||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
|
|
||||||
stat.S_IRUSR | stat.S_IWUSR,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
|
||||||
fh.write(json.dumps(shared, indent=2, sort_keys=True))
|
|
||||||
fh.flush()
|
|
||||||
os.fsync(fh.fileno())
|
|
||||||
os.replace(tmp, path)
|
|
||||||
finally:
|
|
||||||
try:
|
|
||||||
if tmp.exists():
|
|
||||||
tmp.unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
_oauth_trace(
|
_oauth_trace(
|
||||||
"nous_shared_store_written",
|
"nous_shared_store_written",
|
||||||
path=str(path),
|
path=str(path),
|
||||||
@@ -3057,38 +2905,36 @@ def _try_import_shared_nous_state(
|
|||||||
etc.) — caller should then fall through to the normal device-code
|
etc.) — caller should then fall through to the normal device-code
|
||||||
flow.
|
flow.
|
||||||
"""
|
"""
|
||||||
|
shared = _read_shared_nous_state()
|
||||||
|
if not shared:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Build a full state dict so refresh_nous_oauth_from_state has every
|
||||||
|
# field it needs. force_refresh=True gets us a fresh access_token
|
||||||
|
# for this profile; force_mint=True gets us a fresh agent_key.
|
||||||
|
state: Dict[str, Any] = {
|
||||||
|
"access_token": shared.get("access_token"),
|
||||||
|
"refresh_token": shared.get("refresh_token"),
|
||||||
|
"client_id": shared.get("client_id") or DEFAULT_NOUS_CLIENT_ID,
|
||||||
|
"portal_base_url": shared.get("portal_base_url") or DEFAULT_NOUS_PORTAL_URL,
|
||||||
|
"inference_base_url": shared.get("inference_base_url") or DEFAULT_NOUS_INFERENCE_URL,
|
||||||
|
"token_type": shared.get("token_type") or "Bearer",
|
||||||
|
"scope": shared.get("scope") or DEFAULT_NOUS_SCOPE,
|
||||||
|
"obtained_at": shared.get("obtained_at"),
|
||||||
|
"expires_at": shared.get("expires_at"),
|
||||||
|
"agent_key": None,
|
||||||
|
"agent_key_expires_at": None,
|
||||||
|
"tls": {"insecure": False, "ca_bundle": None},
|
||||||
|
}
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with _nous_shared_store_lock(timeout_seconds=max(timeout_seconds + 5.0, AUTH_LOCK_TIMEOUT_SECONDS)):
|
refreshed = refresh_nous_oauth_from_state(
|
||||||
shared = _read_shared_nous_state()
|
state,
|
||||||
if not shared:
|
min_key_ttl_seconds=min_key_ttl_seconds,
|
||||||
return None
|
timeout_seconds=timeout_seconds,
|
||||||
|
force_refresh=True,
|
||||||
# Build a full state dict so refresh_nous_oauth_from_state has every
|
force_mint=True,
|
||||||
# field it needs. force_refresh=True gets us a fresh access_token
|
)
|
||||||
# for this profile; force_mint=True gets us a fresh agent_key.
|
|
||||||
state: Dict[str, Any] = {
|
|
||||||
"access_token": shared.get("access_token"),
|
|
||||||
"refresh_token": shared.get("refresh_token"),
|
|
||||||
"client_id": shared.get("client_id") or DEFAULT_NOUS_CLIENT_ID,
|
|
||||||
"portal_base_url": shared.get("portal_base_url") or DEFAULT_NOUS_PORTAL_URL,
|
|
||||||
"inference_base_url": shared.get("inference_base_url") or DEFAULT_NOUS_INFERENCE_URL,
|
|
||||||
"token_type": shared.get("token_type") or "Bearer",
|
|
||||||
"scope": shared.get("scope") or DEFAULT_NOUS_SCOPE,
|
|
||||||
"obtained_at": shared.get("obtained_at"),
|
|
||||||
"expires_at": shared.get("expires_at"),
|
|
||||||
"agent_key": None,
|
|
||||||
"agent_key_expires_at": None,
|
|
||||||
"tls": {"insecure": False, "ca_bundle": None},
|
|
||||||
}
|
|
||||||
|
|
||||||
refreshed = refresh_nous_oauth_from_state(
|
|
||||||
state,
|
|
||||||
min_key_ttl_seconds=min_key_ttl_seconds,
|
|
||||||
timeout_seconds=timeout_seconds,
|
|
||||||
force_refresh=True,
|
|
||||||
force_mint=True,
|
|
||||||
)
|
|
||||||
_write_shared_nous_state(refreshed)
|
|
||||||
except AuthError as exc:
|
except AuthError as exc:
|
||||||
_oauth_trace(
|
_oauth_trace(
|
||||||
"nous_shared_import_failed",
|
"nous_shared_import_failed",
|
||||||
@@ -3290,65 +3136,59 @@ def resolve_nous_access_token(
|
|||||||
client_id = str(state.get("client_id") or DEFAULT_NOUS_CLIENT_ID)
|
client_id = str(state.get("client_id") or DEFAULT_NOUS_CLIENT_ID)
|
||||||
verify = _resolve_verify(insecure=insecure, ca_bundle=ca_bundle, auth_state=state)
|
verify = _resolve_verify(insecure=insecure, ca_bundle=ca_bundle, auth_state=state)
|
||||||
|
|
||||||
with _nous_shared_store_lock(timeout_seconds=max(timeout_seconds + 5.0, AUTH_LOCK_TIMEOUT_SECONDS)):
|
access_token = state.get("access_token")
|
||||||
merged_shared = _merge_shared_nous_oauth_state(state)
|
refresh_token = state.get("refresh_token")
|
||||||
access_token = state.get("access_token")
|
if not isinstance(access_token, str) or not access_token:
|
||||||
refresh_token = state.get("refresh_token")
|
raise AuthError(
|
||||||
if not isinstance(access_token, str) or not access_token:
|
"No access token found for Nous Portal login.",
|
||||||
raise AuthError(
|
provider="nous",
|
||||||
"No access token found for Nous Portal login.",
|
relogin_required=True,
|
||||||
provider="nous",
|
)
|
||||||
relogin_required=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
if not _is_expiring(state.get("expires_at"), refresh_skew_seconds):
|
if not _is_expiring(state.get("expires_at"), refresh_skew_seconds):
|
||||||
if merged_shared:
|
return access_token
|
||||||
_save_provider_state(auth_store, "nous", state)
|
|
||||||
_save_auth_store(auth_store)
|
|
||||||
return access_token
|
|
||||||
|
|
||||||
if not isinstance(refresh_token, str) or not refresh_token:
|
if not isinstance(refresh_token, str) or not refresh_token:
|
||||||
raise AuthError(
|
raise AuthError(
|
||||||
"Session expired and no refresh token is available.",
|
"Session expired and no refresh token is available.",
|
||||||
provider="nous",
|
provider="nous",
|
||||||
relogin_required=True,
|
relogin_required=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
timeout = httpx.Timeout(timeout_seconds if timeout_seconds else 15.0)
|
timeout = httpx.Timeout(timeout_seconds if timeout_seconds else 15.0)
|
||||||
with httpx.Client(
|
with httpx.Client(
|
||||||
timeout=timeout,
|
timeout=timeout,
|
||||||
headers={"Accept": "application/json"},
|
headers={"Accept": "application/json"},
|
||||||
verify=verify,
|
verify=verify,
|
||||||
) as client:
|
) as client:
|
||||||
refreshed = _refresh_access_token(
|
refreshed = _refresh_access_token(
|
||||||
client=client,
|
client=client,
|
||||||
portal_base_url=portal_base_url,
|
portal_base_url=portal_base_url,
|
||||||
client_id=client_id,
|
client_id=client_id,
|
||||||
refresh_token=refresh_token,
|
refresh_token=refresh_token,
|
||||||
)
|
)
|
||||||
|
|
||||||
now = datetime.now(timezone.utc)
|
now = datetime.now(timezone.utc)
|
||||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||||
state["access_token"] = refreshed["access_token"]
|
state["access_token"] = refreshed["access_token"]
|
||||||
state["refresh_token"] = refreshed.get("refresh_token") or refresh_token
|
state["refresh_token"] = refreshed.get("refresh_token") or refresh_token
|
||||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||||
state["obtained_at"] = now.isoformat()
|
state["obtained_at"] = now.isoformat()
|
||||||
state["expires_in"] = access_ttl
|
state["expires_in"] = access_ttl
|
||||||
state["expires_at"] = datetime.fromtimestamp(
|
state["expires_at"] = datetime.fromtimestamp(
|
||||||
now.timestamp() + access_ttl,
|
now.timestamp() + access_ttl,
|
||||||
tz=timezone.utc,
|
tz=timezone.utc,
|
||||||
).isoformat()
|
).isoformat()
|
||||||
state["portal_base_url"] = portal_base_url
|
state["portal_base_url"] = portal_base_url
|
||||||
state["client_id"] = client_id
|
state["client_id"] = client_id
|
||||||
state["tls"] = {
|
state["tls"] = {
|
||||||
"insecure": verify is False,
|
"insecure": verify is False,
|
||||||
"ca_bundle": verify if isinstance(verify, str) else None,
|
"ca_bundle": verify if isinstance(verify, str) else None,
|
||||||
}
|
}
|
||||||
_save_provider_state(auth_store, "nous", state)
|
_save_provider_state(auth_store, "nous", state)
|
||||||
_save_auth_store(auth_store)
|
_save_auth_store(auth_store)
|
||||||
_write_shared_nous_state(state)
|
return state["access_token"]
|
||||||
return state["access_token"]
|
|
||||||
|
|
||||||
|
|
||||||
def refresh_nous_oauth_pure(
|
def refresh_nous_oauth_pure(
|
||||||
@@ -3616,53 +3456,46 @@ def resolve_nous_runtime_credentials(
|
|||||||
|
|
||||||
# Step 1: refresh access token if expiring
|
# Step 1: refresh access token if expiring
|
||||||
if _is_expiring(state.get("expires_at"), ACCESS_TOKEN_REFRESH_SKEW_SECONDS):
|
if _is_expiring(state.get("expires_at"), ACCESS_TOKEN_REFRESH_SKEW_SECONDS):
|
||||||
with _nous_shared_store_lock(timeout_seconds=max(timeout_seconds + 5.0, AUTH_LOCK_TIMEOUT_SECONDS)):
|
if not isinstance(refresh_token, str) or not refresh_token:
|
||||||
if _merge_shared_nous_oauth_state(state):
|
raise AuthError("Session expired and no refresh token is available.",
|
||||||
access_token = state.get("access_token")
|
provider="nous", relogin_required=True)
|
||||||
refresh_token = state.get("refresh_token")
|
|
||||||
_persist_state("post_shared_merge_access_expiring")
|
|
||||||
|
|
||||||
if _is_expiring(state.get("expires_at"), ACCESS_TOKEN_REFRESH_SKEW_SECONDS):
|
_oauth_trace(
|
||||||
if not isinstance(refresh_token, str) or not refresh_token:
|
"refresh_start",
|
||||||
raise AuthError("Session expired and no refresh token is available.",
|
sequence_id=sequence_id,
|
||||||
provider="nous", relogin_required=True)
|
reason="access_expiring",
|
||||||
|
refresh_token_fp=_token_fingerprint(refresh_token),
|
||||||
_oauth_trace(
|
)
|
||||||
"refresh_start",
|
refreshed = _refresh_access_token(
|
||||||
sequence_id=sequence_id,
|
client=client, portal_base_url=portal_base_url,
|
||||||
reason="access_expiring",
|
client_id=client_id, refresh_token=refresh_token,
|
||||||
refresh_token_fp=_token_fingerprint(refresh_token),
|
)
|
||||||
)
|
now = datetime.now(timezone.utc)
|
||||||
refreshed = _refresh_access_token(
|
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||||
client=client, portal_base_url=portal_base_url,
|
previous_refresh_token = refresh_token
|
||||||
client_id=client_id, refresh_token=refresh_token,
|
state["access_token"] = refreshed["access_token"]
|
||||||
)
|
state["refresh_token"] = refreshed.get("refresh_token") or refresh_token
|
||||||
now = datetime.now(timezone.utc)
|
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||||
previous_refresh_token = refresh_token
|
refreshed_url = _optional_base_url(refreshed.get("inference_base_url"))
|
||||||
state["access_token"] = refreshed["access_token"]
|
if refreshed_url:
|
||||||
state["refresh_token"] = refreshed.get("refresh_token") or refresh_token
|
inference_base_url = refreshed_url
|
||||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
state["obtained_at"] = now.isoformat()
|
||||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
state["expires_in"] = access_ttl
|
||||||
refreshed_url = _optional_base_url(refreshed.get("inference_base_url"))
|
state["expires_at"] = datetime.fromtimestamp(
|
||||||
if refreshed_url:
|
now.timestamp() + access_ttl, tz=timezone.utc
|
||||||
inference_base_url = refreshed_url
|
).isoformat()
|
||||||
state["obtained_at"] = now.isoformat()
|
access_token = state["access_token"]
|
||||||
state["expires_in"] = access_ttl
|
refresh_token = state["refresh_token"]
|
||||||
state["expires_at"] = datetime.fromtimestamp(
|
_oauth_trace(
|
||||||
now.timestamp() + access_ttl, tz=timezone.utc
|
"refresh_success",
|
||||||
).isoformat()
|
sequence_id=sequence_id,
|
||||||
access_token = state["access_token"]
|
reason="access_expiring",
|
||||||
refresh_token = state["refresh_token"]
|
previous_refresh_token_fp=_token_fingerprint(previous_refresh_token),
|
||||||
_oauth_trace(
|
new_refresh_token_fp=_token_fingerprint(refresh_token),
|
||||||
"refresh_success",
|
)
|
||||||
sequence_id=sequence_id,
|
# Persist immediately so downstream mint failures cannot drop rotated refresh tokens.
|
||||||
reason="access_expiring",
|
_persist_state("post_refresh_access_expiring")
|
||||||
previous_refresh_token_fp=_token_fingerprint(previous_refresh_token),
|
|
||||||
new_refresh_token_fp=_token_fingerprint(refresh_token),
|
|
||||||
)
|
|
||||||
# Persist immediately so downstream mint failures cannot drop rotated refresh tokens.
|
|
||||||
_persist_state("post_refresh_access_expiring")
|
|
||||||
|
|
||||||
# Step 2: mint agent key if missing/expiring
|
# Step 2: mint agent key if missing/expiring
|
||||||
used_cached_key = False
|
used_cached_key = False
|
||||||
@@ -3695,47 +3528,41 @@ def resolve_nous_runtime_credentials(
|
|||||||
and isinstance(latest_refresh_token, str)
|
and isinstance(latest_refresh_token, str)
|
||||||
and latest_refresh_token
|
and latest_refresh_token
|
||||||
):
|
):
|
||||||
with _nous_shared_store_lock(timeout_seconds=max(timeout_seconds + 5.0, AUTH_LOCK_TIMEOUT_SECONDS)):
|
_oauth_trace(
|
||||||
if _merge_shared_nous_oauth_state(state):
|
"refresh_start",
|
||||||
access_token = state.get("access_token")
|
sequence_id=sequence_id,
|
||||||
latest_refresh_token = state.get("refresh_token")
|
reason="mint_retry_after_invalid_token",
|
||||||
_persist_state("post_shared_merge_mint_retry")
|
refresh_token_fp=_token_fingerprint(latest_refresh_token),
|
||||||
else:
|
)
|
||||||
_oauth_trace(
|
refreshed = _refresh_access_token(
|
||||||
"refresh_start",
|
client=client, portal_base_url=portal_base_url,
|
||||||
sequence_id=sequence_id,
|
client_id=client_id, refresh_token=latest_refresh_token,
|
||||||
reason="mint_retry_after_invalid_token",
|
)
|
||||||
refresh_token_fp=_token_fingerprint(latest_refresh_token),
|
now = datetime.now(timezone.utc)
|
||||||
)
|
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||||
refreshed = _refresh_access_token(
|
state["access_token"] = refreshed["access_token"]
|
||||||
client=client, portal_base_url=portal_base_url,
|
state["refresh_token"] = refreshed.get("refresh_token") or latest_refresh_token
|
||||||
client_id=client_id, refresh_token=latest_refresh_token,
|
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||||
)
|
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||||
now = datetime.now(timezone.utc)
|
refreshed_url = _optional_base_url(refreshed.get("inference_base_url"))
|
||||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
if refreshed_url:
|
||||||
state["access_token"] = refreshed["access_token"]
|
inference_base_url = refreshed_url
|
||||||
state["refresh_token"] = refreshed.get("refresh_token") or latest_refresh_token
|
state["obtained_at"] = now.isoformat()
|
||||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
state["expires_in"] = access_ttl
|
||||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
state["expires_at"] = datetime.fromtimestamp(
|
||||||
refreshed_url = _optional_base_url(refreshed.get("inference_base_url"))
|
now.timestamp() + access_ttl, tz=timezone.utc
|
||||||
if refreshed_url:
|
).isoformat()
|
||||||
inference_base_url = refreshed_url
|
access_token = state["access_token"]
|
||||||
state["obtained_at"] = now.isoformat()
|
refresh_token = state["refresh_token"]
|
||||||
state["expires_in"] = access_ttl
|
_oauth_trace(
|
||||||
state["expires_at"] = datetime.fromtimestamp(
|
"refresh_success",
|
||||||
now.timestamp() + access_ttl, tz=timezone.utc
|
sequence_id=sequence_id,
|
||||||
).isoformat()
|
reason="mint_retry_after_invalid_token",
|
||||||
access_token = state["access_token"]
|
previous_refresh_token_fp=_token_fingerprint(latest_refresh_token),
|
||||||
refresh_token = state["refresh_token"]
|
new_refresh_token_fp=_token_fingerprint(refresh_token),
|
||||||
_oauth_trace(
|
)
|
||||||
"refresh_success",
|
# Persist retry refresh immediately for crash safety and cross-process visibility.
|
||||||
sequence_id=sequence_id,
|
_persist_state("post_refresh_mint_retry")
|
||||||
reason="mint_retry_after_invalid_token",
|
|
||||||
previous_refresh_token_fp=_token_fingerprint(latest_refresh_token),
|
|
||||||
new_refresh_token_fp=_token_fingerprint(refresh_token),
|
|
||||||
)
|
|
||||||
# Persist retry refresh immediately for crash safety and cross-process visibility.
|
|
||||||
_persist_state("post_refresh_mint_retry")
|
|
||||||
|
|
||||||
mint_payload = _mint_agent_key(
|
mint_payload = _mint_agent_key(
|
||||||
client=client, portal_base_url=portal_base_url,
|
client=client, portal_base_url=portal_base_url,
|
||||||
@@ -4231,14 +4058,6 @@ def _config_provider_matches(provider_id: Optional[str]) -> bool:
|
|||||||
return _get_config_provider() == provider_id.strip().lower()
|
return _get_config_provider() == provider_id.strip().lower()
|
||||||
|
|
||||||
|
|
||||||
def _should_reset_config_provider_on_logout(provider_id: Optional[str]) -> bool:
|
|
||||||
"""Return True when logout should reset the model provider config."""
|
|
||||||
if not provider_id:
|
|
||||||
return False
|
|
||||||
normalized = provider_id.strip().lower()
|
|
||||||
return normalized in PROVIDER_REGISTRY and _config_provider_matches(normalized)
|
|
||||||
|
|
||||||
|
|
||||||
def _logout_default_provider_from_config() -> Optional[str]:
|
def _logout_default_provider_from_config() -> Optional[str]:
|
||||||
"""Fallback logout target when auth.json has no active provider.
|
"""Fallback logout target when auth.json has no active provider.
|
||||||
|
|
||||||
@@ -4397,7 +4216,7 @@ def _prompt_model_selection(
|
|||||||
clear_screen=False,
|
clear_screen=False,
|
||||||
title=effective_title,
|
title=effective_title,
|
||||||
)
|
)
|
||||||
idx = menu.show()
|
idx: int | None = menu.show() # ty:ignore[invalid-assignment] - TerminalMenu.show() is always `int | None` when multi_select is False / not provided.
|
||||||
from hermes_cli.curses_ui import flush_stdin
|
from hermes_cli.curses_ui import flush_stdin
|
||||||
flush_stdin()
|
flush_stdin()
|
||||||
if idx is None:
|
if idx is None:
|
||||||
@@ -5324,18 +5143,15 @@ def logout_command(args) -> None:
|
|||||||
print("No provider is currently logged in.")
|
print("No provider is currently logged in.")
|
||||||
return
|
return
|
||||||
|
|
||||||
should_reset_config = _should_reset_config_provider_on_logout(target)
|
config_matches = _config_provider_matches(target)
|
||||||
provider_name = get_auth_provider_display_name(target)
|
provider_name = get_auth_provider_display_name(target)
|
||||||
|
|
||||||
if clear_provider_auth(target) or should_reset_config:
|
if clear_provider_auth(target) or config_matches:
|
||||||
if should_reset_config:
|
_reset_config_provider()
|
||||||
_reset_config_provider()
|
|
||||||
print(f"Logged out of {provider_name}.")
|
print(f"Logged out of {provider_name}.")
|
||||||
if should_reset_config and os.getenv("OPENROUTER_API_KEY"):
|
if os.getenv("OPENROUTER_API_KEY"):
|
||||||
print("Hermes will use OpenRouter for inference.")
|
print("Hermes will use OpenRouter for inference.")
|
||||||
elif should_reset_config:
|
|
||||||
print("Run `hermes model` or configure an API key to use Hermes.")
|
|
||||||
else:
|
else:
|
||||||
print("Model provider configuration was unchanged.")
|
print("Run `hermes model` or configure an API key to use Hermes.")
|
||||||
else:
|
else:
|
||||||
print(f"No auth state found for {provider_name}.")
|
print(f"No auth state found for {provider_name}.")
|
||||||
|
|||||||
@@ -573,7 +573,7 @@ def create_quick_snapshot(
|
|||||||
"total_size": sum(manifest.values()),
|
"total_size": sum(manifest.values()),
|
||||||
"files": manifest,
|
"files": manifest,
|
||||||
}
|
}
|
||||||
with open(snap_dir / "manifest.json", "w", encoding="utf-8") as f:
|
with open(snap_dir / "manifest.json", "w") as f:
|
||||||
json.dump(meta, f, indent=2)
|
json.dump(meta, f, indent=2)
|
||||||
|
|
||||||
# Auto-prune
|
# Auto-prune
|
||||||
@@ -599,7 +599,7 @@ def list_quick_snapshots(
|
|||||||
manifest_path = d / "manifest.json"
|
manifest_path = d / "manifest.json"
|
||||||
if manifest_path.exists():
|
if manifest_path.exists():
|
||||||
try:
|
try:
|
||||||
with open(manifest_path, encoding="utf-8") as f:
|
with open(manifest_path) as f:
|
||||||
results.append(json.load(f))
|
results.append(json.load(f))
|
||||||
except (json.JSONDecodeError, OSError):
|
except (json.JSONDecodeError, OSError):
|
||||||
results.append({"id": d.name, "file_count": 0, "total_size": 0})
|
results.append({"id": d.name, "file_count": 0, "total_size": 0})
|
||||||
@@ -629,7 +629,7 @@ def restore_quick_snapshot(
|
|||||||
if not manifest_path.exists():
|
if not manifest_path.exists():
|
||||||
return False
|
return False
|
||||||
|
|
||||||
with open(manifest_path, encoding="utf-8") as f:
|
with open(manifest_path) as f:
|
||||||
meta = json.load(f)
|
meta = json.load(f)
|
||||||
|
|
||||||
restored = 0
|
restored = 0
|
||||||
|
|||||||
@@ -157,9 +157,9 @@ COMMAND_REGISTRY: list[CommandDef] = [
|
|||||||
CommandDef("cron", "Manage scheduled tasks", "Tools & Skills",
|
CommandDef("cron", "Manage scheduled tasks", "Tools & Skills",
|
||||||
cli_only=True, args_hint="[subcommand]",
|
cli_only=True, args_hint="[subcommand]",
|
||||||
subcommands=("list", "add", "create", "edit", "pause", "resume", "run", "remove")),
|
subcommands=("list", "add", "create", "edit", "pause", "resume", "run", "remove")),
|
||||||
CommandDef("curator", "Background skill maintenance (status, run, pin, archive, list-archived)",
|
CommandDef("curator", "Background skill maintenance (status, run, pin, archive)",
|
||||||
"Tools & Skills", args_hint="[subcommand]",
|
"Tools & Skills", args_hint="[subcommand]",
|
||||||
subcommands=("status", "run", "pause", "resume", "pin", "unpin", "restore", "list-archived")),
|
subcommands=("status", "run", "pause", "resume", "pin", "unpin", "restore")),
|
||||||
CommandDef("kanban", "Multi-profile collaboration board (tasks, links, comments)",
|
CommandDef("kanban", "Multi-profile collaboration board (tasks, links, comments)",
|
||||||
"Tools & Skills", args_hint="[subcommand]",
|
"Tools & Skills", args_hint="[subcommand]",
|
||||||
subcommands=("list", "ls", "show", "create", "assign", "link", "unlink",
|
subcommands=("list", "ls", "show", "create", "assign", "link", "unlink",
|
||||||
|
|||||||
+19
-169
@@ -212,7 +212,7 @@ def get_container_exec_info() -> Optional[dict]:
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
info = {}
|
info = {}
|
||||||
with open(container_mode_file, "r", encoding="utf-8") as f:
|
with open(container_mode_file, "r") as f:
|
||||||
for line in f:
|
for line in f:
|
||||||
line = line.strip()
|
line = line.strip()
|
||||||
if "=" in line and not line.startswith("#"):
|
if "=" in line and not line.startswith("#"):
|
||||||
@@ -297,7 +297,7 @@ def _is_container() -> bool:
|
|||||||
return True
|
return True
|
||||||
# LXC / cgroup-based detection
|
# LXC / cgroup-based detection
|
||||||
try:
|
try:
|
||||||
with open("/proc/1/cgroup", "r", encoding="utf-8") as f:
|
with open("/proc/1/cgroup", "r") as f:
|
||||||
cgroup_content = f.read()
|
cgroup_content = f.read()
|
||||||
if "docker" in cgroup_content or "lxc" in cgroup_content or "kubepods" in cgroup_content:
|
if "docker" in cgroup_content or "lxc" in cgroup_content or "kubepods" in cgroup_content:
|
||||||
return True
|
return True
|
||||||
@@ -780,19 +780,6 @@ DEFAULT_CONFIG = {
|
|||||||
"timeout": 30,
|
"timeout": 30,
|
||||||
"extra_body": {},
|
"extra_body": {},
|
||||||
},
|
},
|
||||||
# Triage specifier — flesh out a rough one-liner in the Kanban
|
|
||||||
# Triage column into a concrete spec, then promote it to ``todo``.
|
|
||||||
# Invoked by ``hermes kanban specify`` (single id or --all). Set a
|
|
||||||
# cheap, capable model here (gemini-flash works well); the main
|
|
||||||
# model is overkill for short spec expansion.
|
|
||||||
"triage_specifier": {
|
|
||||||
"provider": "auto",
|
|
||||||
"model": "",
|
|
||||||
"base_url": "",
|
|
||||||
"api_key": "",
|
|
||||||
"timeout": 120,
|
|
||||||
"extra_body": {},
|
|
||||||
},
|
|
||||||
# Curator — skill-usage review fork. Timeout is generous because the
|
# Curator — skill-usage review fork. Timeout is generous because the
|
||||||
# review pass can take several minutes on reasoning models (umbrella
|
# review pass can take several minutes on reasoning models (umbrella
|
||||||
# building over hundreds of candidate skills). "auto" = use main chat
|
# building over hundreds of candidate skills). "auto" = use main chat
|
||||||
@@ -1113,14 +1100,6 @@ DEFAULT_CONFIG = {
|
|||||||
# Empty string means use server-local time.
|
# Empty string means use server-local time.
|
||||||
"timezone": "",
|
"timezone": "",
|
||||||
|
|
||||||
# Slack platform settings (gateway mode)
|
|
||||||
"slack": {
|
|
||||||
"require_mention": True, # Require @mention to respond in channels
|
|
||||||
"free_response_channels": "", # Comma-separated channel IDs where bot responds without mention
|
|
||||||
"allowed_channels": "", # If set, bot ONLY responds in these channel IDs (whitelist)
|
|
||||||
"channel_prompts": {}, # Per-channel ephemeral system prompts
|
|
||||||
},
|
|
||||||
|
|
||||||
# Discord platform settings (gateway mode)
|
# Discord platform settings (gateway mode)
|
||||||
"discord": {
|
"discord": {
|
||||||
"require_mention": True, # Require @mention to respond in server channels
|
"require_mention": True, # Require @mention to respond in server channels
|
||||||
@@ -1129,12 +1108,6 @@ DEFAULT_CONFIG = {
|
|||||||
"auto_thread": True, # Auto-create threads on @mention in channels (like Slack)
|
"auto_thread": True, # Auto-create threads on @mention in channels (like Slack)
|
||||||
"reactions": True, # Add 👀/✅/❌ reactions to messages during processing
|
"reactions": True, # Add 👀/✅/❌ reactions to messages during processing
|
||||||
"channel_prompts": {}, # Per-channel ephemeral system prompts (forum parents apply to child threads)
|
"channel_prompts": {}, # Per-channel ephemeral system prompts (forum parents apply to child threads)
|
||||||
# Opt-in DM role-based auth (#12136). By default, DISCORD_ALLOWED_ROLES
|
|
||||||
# authorizes only guild messages in the role's own guild — DMs require
|
|
||||||
# DISCORD_ALLOWED_USERS. Set dm_role_auth_guild to a guild ID to also
|
|
||||||
# authorize DMs from members of that one trusted guild holding the
|
|
||||||
# allowed role. Unset / empty / 0 = secure default (DM role-auth off).
|
|
||||||
"dm_role_auth_guild": "",
|
|
||||||
# discord / discord_admin tools: restrict which actions the agent may call.
|
# discord / discord_admin tools: restrict which actions the agent may call.
|
||||||
# Default (empty) = all actions allowed (subject to bot privileged intents).
|
# Default (empty) = all actions allowed (subject to bot privileged intents).
|
||||||
# Accepts comma-separated string ("list_guilds,list_channels,fetch_messages")
|
# Accepts comma-separated string ("list_guilds,list_channels,fetch_messages")
|
||||||
@@ -1157,24 +1130,18 @@ DEFAULT_CONFIG = {
|
|||||||
"telegram": {
|
"telegram": {
|
||||||
"reactions": False, # Add 👀/✅/❌ reactions to messages during processing
|
"reactions": False, # Add 👀/✅/❌ reactions to messages during processing
|
||||||
"channel_prompts": {}, # Per-chat/topic ephemeral system prompts (topics inherit from parent group)
|
"channel_prompts": {}, # Per-chat/topic ephemeral system prompts (topics inherit from parent group)
|
||||||
"allowed_chats": "", # If set, bot ONLY responds in these group/supergroup chat IDs (whitelist)
|
},
|
||||||
|
|
||||||
|
# Slack platform settings (gateway mode)
|
||||||
|
"slack": {
|
||||||
|
"channel_prompts": {}, # Per-channel ephemeral system prompts
|
||||||
},
|
},
|
||||||
|
|
||||||
# Mattermost platform settings (gateway mode)
|
# Mattermost platform settings (gateway mode)
|
||||||
"mattermost": {
|
"mattermost": {
|
||||||
"require_mention": True, # Require @mention to respond in channels
|
|
||||||
"free_response_channels": "", # Comma-separated channel IDs where bot responds without mention
|
|
||||||
"allowed_channels": "", # If set, bot ONLY responds in these channel IDs (whitelist)
|
|
||||||
"channel_prompts": {}, # Per-channel ephemeral system prompts
|
"channel_prompts": {}, # Per-channel ephemeral system prompts
|
||||||
},
|
},
|
||||||
|
|
||||||
# Matrix platform settings (gateway mode)
|
|
||||||
"matrix": {
|
|
||||||
"require_mention": True, # Require @mention to respond in rooms
|
|
||||||
"free_response_rooms": "", # Comma-separated room IDs where bot responds without mention
|
|
||||||
"allowed_rooms": "", # If set, bot ONLY responds in these room IDs (whitelist)
|
|
||||||
},
|
|
||||||
|
|
||||||
# Approval mode for dangerous commands:
|
# Approval mode for dangerous commands:
|
||||||
# manual — always prompt the user (default)
|
# manual — always prompt the user (default)
|
||||||
# smart — use auxiliary LLM to auto-approve low-risk commands, prompt for high-risk
|
# smart — use auxiliary LLM to auto-approve low-risk commands, prompt for high-risk
|
||||||
@@ -1224,7 +1191,7 @@ DEFAULT_CONFIG = {
|
|||||||
# Pre-exec security scanning via tirith
|
# Pre-exec security scanning via tirith
|
||||||
"security": {
|
"security": {
|
||||||
"allow_private_urls": False, # Allow requests to private/internal IPs (for OpenWrt, proxies, VPNs)
|
"allow_private_urls": False, # Allow requests to private/internal IPs (for OpenWrt, proxies, VPNs)
|
||||||
"redact_secrets": True,
|
"redact_secrets": False,
|
||||||
"tirith_enabled": True,
|
"tirith_enabled": True,
|
||||||
"tirith_path": "tirith",
|
"tirith_path": "tirith",
|
||||||
"tirith_timeout": 5,
|
"tirith_timeout": 5,
|
||||||
@@ -1263,10 +1230,6 @@ DEFAULT_CONFIG = {
|
|||||||
# Seconds between dispatcher ticks (idle or not). Lower = snappier
|
# Seconds between dispatcher ticks (idle or not). Lower = snappier
|
||||||
# pickup of newly-ready tasks; higher = less SQL pressure.
|
# pickup of newly-ready tasks; higher = less SQL pressure.
|
||||||
"dispatch_interval_seconds": 60,
|
"dispatch_interval_seconds": 60,
|
||||||
# Auto-block after this many consecutive non-success attempts for the
|
|
||||||
# same task/profile (spawn_failed, timed_out, or crashed). Reassignment
|
|
||||||
# resets the streak for the new profile.
|
|
||||||
"failure_limit": 2,
|
|
||||||
},
|
},
|
||||||
|
|
||||||
# execute_code settings — controls the tool used for programmatic tool calls.
|
# execute_code settings — controls the tool used for programmatic tool calls.
|
||||||
@@ -1877,14 +1840,6 @@ OPTIONAL_ENV_VARS = {
|
|||||||
"password": False,
|
"password": False,
|
||||||
"category": "tool",
|
"category": "tool",
|
||||||
},
|
},
|
||||||
"BRAVE_SEARCH_API_KEY": {
|
|
||||||
"description": "Brave Search API subscription token (free tier: 2,000 queries/mo)",
|
|
||||||
"prompt": "Brave Search subscription token",
|
|
||||||
"url": "https://brave.com/search/api/",
|
|
||||||
"tools": ["web_search"],
|
|
||||||
"password": True,
|
|
||||||
"category": "tool",
|
|
||||||
},
|
|
||||||
"BROWSERBASE_API_KEY": {
|
"BROWSERBASE_API_KEY": {
|
||||||
"description": "Browserbase API key for cloud browser (optional — local browser works without this)",
|
"description": "Browserbase API key for cloud browser (optional — local browser works without this)",
|
||||||
"prompt": "Browserbase API key",
|
"prompt": "Browserbase API key",
|
||||||
@@ -3452,7 +3407,7 @@ def migrate_config(interactive: bool = True, quiet: bool = False) -> Dict[str, A
|
|||||||
if not manifest_file.exists():
|
if not manifest_file.exists():
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
with open(manifest_file, encoding="utf-8") as _mf:
|
with open(manifest_file) as _mf:
|
||||||
manifest = yaml.safe_load(_mf) or {}
|
manifest = yaml.safe_load(_mf) or {}
|
||||||
except Exception:
|
except Exception:
|
||||||
manifest = {}
|
manifest = {}
|
||||||
@@ -4019,10 +3974,10 @@ def load_config() -> Dict[str, Any]:
|
|||||||
|
|
||||||
_SECURITY_COMMENT = """
|
_SECURITY_COMMENT = """
|
||||||
# ── Security ──────────────────────────────────────────────────────────
|
# ── Security ──────────────────────────────────────────────────────────
|
||||||
# Secret redaction is ON by default — strings that look like API keys,
|
# Secret redaction is OFF by default — tool output (terminal stdout,
|
||||||
# tokens, and passwords are masked in tool output, logs, and chat
|
# read_file results, web content) passes through unmodified. Set
|
||||||
# responses before the model or user ever sees them. Set redact_secrets
|
# redact_secrets to true to mask strings that look like API keys, tokens,
|
||||||
# to false to disable (e.g. when developing the redactor itself).
|
# and passwords before they enter the model context and logs.
|
||||||
# tirith pre-exec scanning is enabled by default when the tirith binary
|
# tirith pre-exec scanning is enabled by default when the tirith binary
|
||||||
# is available. Configure via security.tirith_* keys or env vars
|
# is available. Configure via security.tirith_* keys or env vars
|
||||||
# (TIRITH_ENABLED, TIRITH_BIN, TIRITH_TIMEOUT, TIRITH_FAIL_OPEN).
|
# (TIRITH_ENABLED, TIRITH_BIN, TIRITH_TIMEOUT, TIRITH_FAIL_OPEN).
|
||||||
@@ -4062,8 +4017,8 @@ _FALLBACK_COMMENT = """
|
|||||||
|
|
||||||
_COMMENTED_SECTIONS = """
|
_COMMENTED_SECTIONS = """
|
||||||
# ── Security ──────────────────────────────────────────────────────────
|
# ── Security ──────────────────────────────────────────────────────────
|
||||||
# Secret redaction is ON by default. Set to false to pass tool output,
|
# Secret redaction is OFF by default. Set to true to mask strings that
|
||||||
# logs, and chat responses through unmodified (e.g. for redactor dev).
|
# look like API keys, tokens, and passwords in tool output and logs.
|
||||||
#
|
#
|
||||||
# security:
|
# security:
|
||||||
# redact_secrets: true
|
# redact_secrets: true
|
||||||
@@ -4696,19 +4651,11 @@ def edit_config():
|
|||||||
|
|
||||||
# Find editor
|
# Find editor
|
||||||
editor = os.getenv('EDITOR') or os.getenv('VISUAL')
|
editor = os.getenv('EDITOR') or os.getenv('VISUAL')
|
||||||
|
|
||||||
if not editor:
|
if not editor:
|
||||||
# Try common editors — order is platform-aware so Windows users
|
# Try common editors
|
||||||
# land on a working editor (notepad) even without Git Bash or nano
|
for cmd in ['nano', 'vim', 'vi', 'code', 'notepad']:
|
||||||
# installed. On POSIX, prefer nano/vim over code/notepad because
|
import shutil
|
||||||
# it's more likely to be present on headless / server systems.
|
|
||||||
import shutil
|
|
||||||
import sys as _sys
|
|
||||||
if _sys.platform == "win32":
|
|
||||||
candidates = ['notepad', 'code', 'vim', 'vi', 'nano']
|
|
||||||
else:
|
|
||||||
candidates = ['nano', 'vim', 'vi', 'code', 'notepad']
|
|
||||||
for cmd in candidates:
|
|
||||||
if shutil.which(cmd):
|
if shutil.which(cmd):
|
||||||
editor = cmd
|
editor = cmd
|
||||||
break
|
break
|
||||||
@@ -4990,100 +4937,3 @@ def _inject_profile_env_vars() -> None:
|
|||||||
|
|
||||||
# Eagerly inject so that OPTIONAL_ENV_VARS is fully populated at import time.
|
# Eagerly inject so that OPTIONAL_ENV_VARS is fully populated at import time.
|
||||||
_inject_profile_env_vars()
|
_inject_profile_env_vars()
|
||||||
|
|
||||||
|
|
||||||
# ── Platform-plugin env var injection ────────────────────────────────────────
|
|
||||||
# Bundled platform plugins under ``plugins/platforms/*/plugin.yaml`` declare
|
|
||||||
# their required env vars via ``requires_env``. This mirror of
|
|
||||||
# ``_inject_profile_env_vars`` surfaces them in ``hermes config`` UI so users
|
|
||||||
# can configure Teams / IRC / Google Chat without the core repo ever needing
|
|
||||||
# to know they exist.
|
|
||||||
#
|
|
||||||
# Each ``requires_env`` entry may be a bare string (name only) or a dict:
|
|
||||||
#
|
|
||||||
# requires_env:
|
|
||||||
# - TEAMS_CLIENT_ID # minimal
|
|
||||||
# - name: TEAMS_CLIENT_SECRET # rich
|
|
||||||
# description: "Teams bot client secret"
|
|
||||||
# url: "https://portal.azure.com/"
|
|
||||||
# password: true
|
|
||||||
# prompt: "Teams client secret"
|
|
||||||
#
|
|
||||||
# An optional ``optional_env`` block surfaces non-required vars the same way
|
|
||||||
# (e.g. allowlist, home channel).
|
|
||||||
|
|
||||||
_platform_plugin_env_vars_injected = False
|
|
||||||
|
|
||||||
|
|
||||||
def _inject_platform_plugin_env_vars() -> None:
|
|
||||||
"""Populate OPTIONAL_ENV_VARS from bundled platform plugin manifests.
|
|
||||||
|
|
||||||
Called once at module load time. Idempotent — repeated calls are no-ops.
|
|
||||||
Failures are swallowed so a malformed plugin.yaml can't break CLI import.
|
|
||||||
"""
|
|
||||||
global _platform_plugin_env_vars_injected
|
|
||||||
if _platform_plugin_env_vars_injected:
|
|
||||||
return
|
|
||||||
_platform_plugin_env_vars_injected = True
|
|
||||||
try:
|
|
||||||
import yaml # type: ignore
|
|
||||||
|
|
||||||
# Resolve the bundled plugins dir from this file's location so the
|
|
||||||
# injector works regardless of CWD.
|
|
||||||
repo_root = Path(__file__).resolve().parents[1]
|
|
||||||
platforms_dir = repo_root / "plugins" / "platforms"
|
|
||||||
if not platforms_dir.is_dir():
|
|
||||||
return
|
|
||||||
for child in platforms_dir.iterdir():
|
|
||||||
if not child.is_dir():
|
|
||||||
continue
|
|
||||||
manifest_path = child / "plugin.yaml"
|
|
||||||
if not manifest_path.exists():
|
|
||||||
manifest_path = child / "plugin.yml"
|
|
||||||
if not manifest_path.exists():
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
with open(manifest_path, "r", encoding="utf-8") as f:
|
|
||||||
manifest = yaml.safe_load(f) or {}
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
label = manifest.get("label") or manifest.get("name") or child.name
|
|
||||||
# Merge required + optional env var declarations.
|
|
||||||
entries = list(manifest.get("requires_env") or [])
|
|
||||||
entries.extend(manifest.get("optional_env") or [])
|
|
||||||
for entry in entries:
|
|
||||||
if isinstance(entry, str):
|
|
||||||
name = entry
|
|
||||||
meta: dict = {}
|
|
||||||
elif isinstance(entry, dict) and entry.get("name"):
|
|
||||||
name = entry["name"]
|
|
||||||
meta = entry
|
|
||||||
else:
|
|
||||||
continue
|
|
||||||
if name in OPTIONAL_ENV_VARS:
|
|
||||||
continue # hardcoded entry wins (back-compat)
|
|
||||||
# Heuristic: anything named *TOKEN, *SECRET, *KEY, *PASSWORD
|
|
||||||
# is a password field unless explicitly overridden.
|
|
||||||
name_upper = name.upper()
|
|
||||||
is_secret = bool(meta.get("password") or meta.get("secret"))
|
|
||||||
if not is_secret and not meta.get("password") is False:
|
|
||||||
is_secret = any(
|
|
||||||
name_upper.endswith(suf)
|
|
||||||
for suf in ("_TOKEN", "_SECRET", "_KEY", "_PASSWORD", "_JSON")
|
|
||||||
)
|
|
||||||
OPTIONAL_ENV_VARS[name] = {
|
|
||||||
"description": (
|
|
||||||
meta.get("description")
|
|
||||||
or f"{label} configuration"
|
|
||||||
),
|
|
||||||
"prompt": meta.get("prompt") or name,
|
|
||||||
"url": meta.get("url") or None,
|
|
||||||
"password": is_secret,
|
|
||||||
"category": meta.get("category") or "messaging",
|
|
||||||
}
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
# Eagerly inject so that platform plugin env vars show up in the setup wizard.
|
|
||||||
_inject_platform_plugin_env_vars()
|
|
||||||
|
|||||||
@@ -212,9 +212,9 @@ def copilot_device_code_login(
|
|||||||
print(" Waiting for authorization...", end="", flush=True)
|
print(" Waiting for authorization...", end="", flush=True)
|
||||||
|
|
||||||
# Step 3: Poll for completion
|
# Step 3: Poll for completion
|
||||||
deadline = time.monotonic() + timeout_seconds
|
deadline = time.time() + timeout_seconds
|
||||||
|
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
time.sleep(interval + _DEVICE_CODE_POLL_SAFETY_MARGIN)
|
time.sleep(interval + _DEVICE_CODE_POLL_SAFETY_MARGIN)
|
||||||
|
|
||||||
poll_data = urllib.parse.urlencode({
|
poll_data = urllib.parse.urlencode({
|
||||||
|
|||||||
+8
-37
@@ -12,7 +12,6 @@ from __future__ import annotations
|
|||||||
import argparse
|
import argparse
|
||||||
import sys
|
import sys
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
|
|
||||||
|
|
||||||
@@ -58,8 +57,7 @@ def _cmd_status(args) -> int:
|
|||||||
print(f" last summary: {summary}")
|
print(f" last summary: {summary}")
|
||||||
_report = state.get("last_report_path")
|
_report = state.get("last_report_path")
|
||||||
if _report:
|
if _report:
|
||||||
suffix = "" if Path(_report).exists() else " (missing)"
|
print(f" last report: {_report}")
|
||||||
print(f" last report: {_report}{suffix}")
|
|
||||||
_ih = curator.get_interval_hours()
|
_ih = curator.get_interval_hours()
|
||||||
_interval_label = (
|
_interval_label = (
|
||||||
f"{_ih // 24}d" if _ih % 24 == 0 and _ih >= 24
|
f"{_ih // 24}d" if _ih % 24 == 0 and _ih >= 24
|
||||||
@@ -163,8 +161,6 @@ def _cmd_run(args) -> int:
|
|||||||
return 1
|
return 1
|
||||||
|
|
||||||
dry = bool(getattr(args, "dry_run", False))
|
dry = bool(getattr(args, "dry_run", False))
|
||||||
background = bool(getattr(args, "background", False))
|
|
||||||
synchronous = bool(getattr(args, "synchronous", False)) or not background
|
|
||||||
if dry:
|
if dry:
|
||||||
print("curator: running DRY-RUN (report only, no mutations)...")
|
print("curator: running DRY-RUN (report only, no mutations)...")
|
||||||
else:
|
else:
|
||||||
@@ -175,7 +171,7 @@ def _cmd_run(args) -> int:
|
|||||||
|
|
||||||
result = curator.run_curator_review(
|
result = curator.run_curator_review(
|
||||||
on_summary=_on_summary,
|
on_summary=_on_summary,
|
||||||
synchronous=synchronous,
|
synchronous=bool(args.synchronous),
|
||||||
dry_run=dry,
|
dry_run=dry,
|
||||||
)
|
)
|
||||||
auto = result.get("auto_transitions", {})
|
auto = result.get("auto_transitions", {})
|
||||||
@@ -192,19 +188,13 @@ def _cmd_run(args) -> int:
|
|||||||
f"archived={auto.get('archived', 0)} "
|
f"archived={auto.get('archived', 0)} "
|
||||||
f"reactivated={auto.get('reactivated', 0)}"
|
f"reactivated={auto.get('reactivated', 0)}"
|
||||||
)
|
)
|
||||||
if not synchronous:
|
if not args.synchronous:
|
||||||
print("llm pass running in background — check `hermes curator status` later")
|
print("llm pass running in background — check `hermes curator status` later")
|
||||||
if dry:
|
if dry:
|
||||||
if synchronous:
|
print(
|
||||||
print(
|
"dry-run: no changes applied. When the report lands, read it with "
|
||||||
"dry-run: no changes applied. Read the report with "
|
"`hermes curator status` and run `hermes curator run` (no flag) to apply."
|
||||||
"`hermes curator status` and run `hermes curator run` (no flag) to apply."
|
)
|
||||||
)
|
|
||||||
else:
|
|
||||||
print(
|
|
||||||
"dry-run: no changes applied. When the report lands, read it with "
|
|
||||||
"`hermes curator status` and run `hermes curator run` (no flag) to apply."
|
|
||||||
)
|
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
@@ -452,18 +442,6 @@ def _cmd_rollback(args) -> int:
|
|||||||
return 1
|
return 1
|
||||||
|
|
||||||
|
|
||||||
def _cmd_list_archived(args) -> int:
|
|
||||||
"""List archived (recoverable) skills."""
|
|
||||||
from tools import skill_usage
|
|
||||||
names = skill_usage.list_archived_skill_names()
|
|
||||||
if not names:
|
|
||||||
print("curator: no archived skills")
|
|
||||||
return 0
|
|
||||||
for name in names:
|
|
||||||
print(name)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# argparse wiring (called from hermes_cli.main)
|
# argparse wiring (called from hermes_cli.main)
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -483,11 +461,7 @@ def register_cli(parent: argparse.ArgumentParser) -> None:
|
|||||||
p_run = subs.add_parser("run", help="Trigger a curator review now")
|
p_run = subs.add_parser("run", help="Trigger a curator review now")
|
||||||
p_run.add_argument(
|
p_run.add_argument(
|
||||||
"--sync", "--synchronous", dest="synchronous", action="store_true",
|
"--sync", "--synchronous", dest="synchronous", action="store_true",
|
||||||
help="Wait for the LLM review pass to finish (default for manual runs)",
|
help="Wait for the LLM review pass to finish (default: background thread)",
|
||||||
)
|
|
||||||
p_run.add_argument(
|
|
||||||
"--background", dest="background", action="store_true",
|
|
||||||
help="Start the LLM review pass in a background thread and return immediately",
|
|
||||||
)
|
)
|
||||||
p_run.add_argument(
|
p_run.add_argument(
|
||||||
"--dry-run", dest="dry_run", action="store_true",
|
"--dry-run", dest="dry_run", action="store_true",
|
||||||
@@ -514,9 +488,6 @@ def register_cli(parent: argparse.ArgumentParser) -> None:
|
|||||||
p_restore.add_argument("skill", help="Skill name")
|
p_restore.add_argument("skill", help="Skill name")
|
||||||
p_restore.set_defaults(func=_cmd_restore)
|
p_restore.set_defaults(func=_cmd_restore)
|
||||||
|
|
||||||
subs.add_parser("list-archived", help="List archived skills") \
|
|
||||||
.set_defaults(func=_cmd_list_archived)
|
|
||||||
|
|
||||||
p_archive = subs.add_parser(
|
p_archive = subs.add_parser(
|
||||||
"archive",
|
"archive",
|
||||||
help="Manually archive a skill (move to .archive/, excluded from prompt)",
|
help="Manually archive a skill (move to .archive/, excluded from prompt)",
|
||||||
|
|||||||
+4
-31
@@ -91,15 +91,6 @@ def _termux_browser_setup_steps(node_installed: bool) -> list[str]:
|
|||||||
return steps
|
return steps
|
||||||
|
|
||||||
|
|
||||||
def _termux_install_all_fallback_notes() -> list[str]:
|
|
||||||
return [
|
|
||||||
"Termux install profile: use .[termux-all] for broad compatibility (installer default on Termux).",
|
|
||||||
"Matrix E2EE extra is excluded on Termux (python-olm currently fails to build).",
|
|
||||||
"Local faster-whisper extra is excluded on Termux (ctranslate2/av build path unavailable).",
|
|
||||||
"STT fallback: use Groq Whisper (set GROQ_API_KEY) or OpenAI Whisper (set VOICE_TOOLS_OPENAI_KEY).",
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def _has_provider_env_config(content: str) -> bool:
|
def _has_provider_env_config(content: str) -> bool:
|
||||||
"""Return True when ~/.hermes/.env contains provider auth/base URL settings."""
|
"""Return True when ~/.hermes/.env contains provider auth/base URL settings."""
|
||||||
return any(key in content for key in _PROVIDER_ENV_HINTS)
|
return any(key in content for key in _PROVIDER_ENV_HINTS)
|
||||||
@@ -598,7 +589,7 @@ def run_doctor(args):
|
|||||||
# Detect stale root-level model keys (known bug source — PR #4329)
|
# Detect stale root-level model keys (known bug source — PR #4329)
|
||||||
try:
|
try:
|
||||||
import yaml
|
import yaml
|
||||||
with open(config_path, encoding="utf-8") as f:
|
with open(config_path) as f:
|
||||||
raw_config = yaml.safe_load(f) or {}
|
raw_config = yaml.safe_load(f) or {}
|
||||||
stale_root_keys = [k for k in ("provider", "base_url") if k in raw_config and isinstance(raw_config[k], str)]
|
stale_root_keys = [k for k in ("provider", "base_url") if k in raw_config and isinstance(raw_config[k], str)]
|
||||||
if stale_root_keys:
|
if stale_root_keys:
|
||||||
@@ -1059,8 +1050,7 @@ def run_doctor(args):
|
|||||||
check_warn("Node.js not found", "(optional, needed for browser tools)")
|
check_warn("Node.js not found", "(optional, needed for browser tools)")
|
||||||
|
|
||||||
# npm audit for all Node.js packages
|
# npm audit for all Node.js packages
|
||||||
_npm_bin = _safe_which("npm")
|
if _safe_which("npm"):
|
||||||
if _npm_bin:
|
|
||||||
npm_dirs = [
|
npm_dirs = [
|
||||||
(PROJECT_ROOT, "Browser tools (agent-browser)"),
|
(PROJECT_ROOT, "Browser tools (agent-browser)"),
|
||||||
(PROJECT_ROOT / "scripts" / "whatsapp-bridge", "WhatsApp bridge"),
|
(PROJECT_ROOT / "scripts" / "whatsapp-bridge", "WhatsApp bridge"),
|
||||||
@@ -1069,10 +1059,8 @@ def run_doctor(args):
|
|||||||
if not (npm_dir / "node_modules").exists():
|
if not (npm_dir / "node_modules").exists():
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
# Use resolved absolute path so Windows can execute
|
|
||||||
# npm.cmd (CreateProcessW can't run bare .cmd names).
|
|
||||||
audit_result = subprocess.run(
|
audit_result = subprocess.run(
|
||||||
[_npm_bin, "audit", "--json"],
|
["npm", "audit", "--json"],
|
||||||
cwd=str(npm_dir),
|
cwd=str(npm_dir),
|
||||||
capture_output=True, text=True, timeout=30,
|
capture_output=True, text=True, timeout=30,
|
||||||
)
|
)
|
||||||
@@ -1096,11 +1084,6 @@ def run_doctor(args):
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
if _is_termux():
|
|
||||||
check_info("Termux compatibility fallbacks:")
|
|
||||||
for note in _termux_install_all_fallback_notes():
|
|
||||||
check_info(note)
|
|
||||||
|
|
||||||
# =========================================================================
|
# =========================================================================
|
||||||
# Check: API connectivity
|
# Check: API connectivity
|
||||||
# =========================================================================
|
# =========================================================================
|
||||||
@@ -1242,16 +1225,6 @@ def run_doctor(args):
|
|||||||
headers=_headers,
|
headers=_headers,
|
||||||
timeout=10,
|
timeout=10,
|
||||||
)
|
)
|
||||||
if (
|
|
||||||
_pname == "Alibaba/DashScope"
|
|
||||||
and not _base
|
|
||||||
and _resp.status_code == 401
|
|
||||||
):
|
|
||||||
_resp = httpx.get(
|
|
||||||
"https://dashscope.aliyuncs.com/compatible-mode/v1/models",
|
|
||||||
headers=_headers,
|
|
||||||
timeout=10,
|
|
||||||
)
|
|
||||||
if _resp.status_code == 200:
|
if _resp.status_code == 200:
|
||||||
print(f"\r {color('✓', Colors.GREEN)} {_label} ")
|
print(f"\r {color('✓', Colors.GREEN)} {_label} ")
|
||||||
elif _resp.status_code == 401:
|
elif _resp.status_code == 401:
|
||||||
@@ -1399,7 +1372,7 @@ def run_doctor(args):
|
|||||||
import yaml as _yaml
|
import yaml as _yaml
|
||||||
_mem_cfg_path = HERMES_HOME / "config.yaml"
|
_mem_cfg_path = HERMES_HOME / "config.yaml"
|
||||||
if _mem_cfg_path.exists():
|
if _mem_cfg_path.exists():
|
||||||
with open(_mem_cfg_path, encoding="utf-8") as _f:
|
with open(_mem_cfg_path) as _f:
|
||||||
_raw_cfg = _yaml.safe_load(_f) or {}
|
_raw_cfg = _yaml.safe_load(_f) or {}
|
||||||
_active_memory_provider = (_raw_cfg.get("memory") or {}).get("provider", "")
|
_active_memory_provider = (_raw_cfg.get("memory") or {}).get("provider", "")
|
||||||
except Exception:
|
except Exception:
|
||||||
|
|||||||
+47
-369
@@ -232,10 +232,6 @@ def _graceful_restart_via_sigusr1(pid: int, drain_timeout: float) -> bool:
|
|||||||
# Process still exists but we can't signal it. Treat as alive
|
# Process still exists but we can't signal it. Treat as alive
|
||||||
# so the caller falls back.
|
# so the caller falls back.
|
||||||
pass
|
pass
|
||||||
except OSError:
|
|
||||||
# Windows raises OSError (WinError 87 "invalid parameter") for
|
|
||||||
# a gone PID — treat the same as ProcessLookupError.
|
|
||||||
return True
|
|
||||||
_time.sleep(0.5)
|
_time.sleep(0.5)
|
||||||
# Drain didn't finish in time.
|
# Drain didn't finish in time.
|
||||||
return False
|
return False
|
||||||
@@ -445,25 +441,6 @@ def launch_detached_profile_gateway_restart(profile: str, old_pid: int) -> bool:
|
|||||||
if old_pid <= 0:
|
if old_pid <= 0:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
# The watcher is a tiny Python subprocess that polls the old PID and
|
|
||||||
# respawns the gateway once it's gone. Both legs of the chain need
|
|
||||||
# platform-appropriate detach semantics:
|
|
||||||
#
|
|
||||||
# POSIX — ``start_new_session=True`` (os.setsid in the child) detaches
|
|
||||||
# from the parent's process group so Ctrl+C in the CLI doesn't
|
|
||||||
# propagate and the watcher/gateway survive the CLI exiting.
|
|
||||||
#
|
|
||||||
# Windows — ``start_new_session`` is silently accepted but does NOT
|
|
||||||
# detach. The watcher stays attached to the CLI's console and dies
|
|
||||||
# when the user closes the terminal, leaving ``hermes update`` users
|
|
||||||
# with no running gateway until they re-invoke ``hermes gateway``
|
|
||||||
# manually. The Win32 equivalent is the ``CREATE_NEW_PROCESS_GROUP |
|
|
||||||
# DETACHED_PROCESS | CREATE_NO_WINDOW`` creationflags bundle.
|
|
||||||
#
|
|
||||||
# ``windows_detach_popen_kwargs()`` returns the right kwargs for the
|
|
||||||
# host platform and is a no-op on POSIX (just ``start_new_session=True``).
|
|
||||||
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
|
|
||||||
|
|
||||||
watcher = textwrap.dedent(
|
watcher = textwrap.dedent(
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
@@ -481,39 +458,22 @@ def launch_detached_profile_gateway_restart(profile: str, old_pid: int) -> bool:
|
|||||||
break
|
break
|
||||||
except PermissionError:
|
except PermissionError:
|
||||||
pass
|
pass
|
||||||
except OSError:
|
|
||||||
# Windows: gone PID raises OSError (WinError 87).
|
|
||||||
break
|
|
||||||
time.sleep(0.2)
|
time.sleep(0.2)
|
||||||
|
subprocess.Popen(
|
||||||
# Platform-appropriate detach for the respawned gateway. On POSIX
|
cmd,
|
||||||
# start_new_session=True maps to os.setsid; on Windows we need
|
stdout=subprocess.DEVNULL,
|
||||||
# explicit creationflags because start_new_session is a no-op there.
|
stderr=subprocess.DEVNULL,
|
||||||
_popen_kwargs = {
|
start_new_session=True,
|
||||||
"stdout": subprocess.DEVNULL,
|
)
|
||||||
"stderr": subprocess.DEVNULL,
|
|
||||||
}
|
|
||||||
if sys.platform == "win32":
|
|
||||||
_CREATE_NEW_PROCESS_GROUP = 0x00000200
|
|
||||||
_DETACHED_PROCESS = 0x00000008
|
|
||||||
_CREATE_NO_WINDOW = 0x08000000
|
|
||||||
_popen_kwargs["creationflags"] = (
|
|
||||||
_CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
_popen_kwargs["start_new_session"] = True
|
|
||||||
subprocess.Popen(cmd, **_popen_kwargs)
|
|
||||||
"""
|
"""
|
||||||
).strip()
|
).strip()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Same platform-aware detach for the watcher process itself — so
|
|
||||||
# closing the user's terminal doesn't kill the watcher.
|
|
||||||
subprocess.Popen(
|
subprocess.Popen(
|
||||||
[sys.executable, "-c", watcher, str(old_pid), *_gateway_run_args_for_profile(profile)],
|
[sys.executable, "-c", watcher, str(old_pid), *_gateway_run_args_for_profile(profile)],
|
||||||
stdout=subprocess.DEVNULL,
|
stdout=subprocess.DEVNULL,
|
||||||
stderr=subprocess.DEVNULL,
|
stderr=subprocess.DEVNULL,
|
||||||
**windows_detach_popen_kwargs(),
|
start_new_session=True,
|
||||||
)
|
)
|
||||||
except OSError:
|
except OSError:
|
||||||
return False
|
return False
|
||||||
@@ -545,7 +505,6 @@ def _read_systemd_unit_properties(
|
|||||||
"SubState",
|
"SubState",
|
||||||
"Result",
|
"Result",
|
||||||
"ExecMainStatus",
|
"ExecMainStatus",
|
||||||
"MainPID",
|
|
||||||
),
|
),
|
||||||
) -> dict[str, str]:
|
) -> dict[str, str]:
|
||||||
"""Return selected ``systemctl show`` properties for the gateway unit."""
|
"""Return selected ``systemctl show`` properties for the gateway unit."""
|
||||||
@@ -579,41 +538,6 @@ def _read_systemd_unit_properties(
|
|||||||
return parsed
|
return parsed
|
||||||
|
|
||||||
|
|
||||||
def _systemd_main_pid_from_props(props: dict[str, str]) -> int | None:
|
|
||||||
try:
|
|
||||||
pid = int(props.get("MainPID", "0") or "0")
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return None
|
|
||||||
return pid if pid > 0 else None
|
|
||||||
|
|
||||||
|
|
||||||
def _systemd_main_pid(system: bool = False) -> int | None:
|
|
||||||
return _systemd_main_pid_from_props(_read_systemd_unit_properties(system=system))
|
|
||||||
|
|
||||||
|
|
||||||
def _read_gateway_runtime_status() -> dict | None:
|
|
||||||
try:
|
|
||||||
from gateway.status import read_runtime_status
|
|
||||||
|
|
||||||
state = read_runtime_status()
|
|
||||||
except Exception:
|
|
||||||
return None
|
|
||||||
return state if isinstance(state, dict) else None
|
|
||||||
|
|
||||||
|
|
||||||
def _gateway_runtime_status_for_pid(pid: int | None) -> dict | None:
|
|
||||||
if not pid:
|
|
||||||
return None
|
|
||||||
state = _read_gateway_runtime_status()
|
|
||||||
if not state:
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
state_pid = int(state.get("pid", 0) or 0)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return None
|
|
||||||
return state if state_pid == pid else None
|
|
||||||
|
|
||||||
|
|
||||||
def _wait_for_systemd_service_restart(
|
def _wait_for_systemd_service_restart(
|
||||||
*,
|
*,
|
||||||
system: bool = False,
|
system: bool = False,
|
||||||
@@ -625,10 +549,9 @@ def _wait_for_systemd_service_restart(
|
|||||||
|
|
||||||
svc = get_service_name()
|
svc = get_service_name()
|
||||||
scope_label = _service_scope_label(system).capitalize()
|
scope_label = _service_scope_label(system).capitalize()
|
||||||
deadline = time.monotonic() + timeout
|
deadline = time.time() + timeout
|
||||||
printed_runtime_wait = False
|
|
||||||
|
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
props = _read_systemd_unit_properties(system=system)
|
props = _read_systemd_unit_properties(system=system)
|
||||||
active_state = props.get("ActiveState", "")
|
active_state = props.get("ActiveState", "")
|
||||||
sub_state = props.get("SubState", "")
|
sub_state = props.get("SubState", "")
|
||||||
@@ -639,32 +562,19 @@ def _wait_for_systemd_service_restart(
|
|||||||
new_pid = get_running_pid()
|
new_pid = get_running_pid()
|
||||||
except Exception:
|
except Exception:
|
||||||
new_pid = None
|
new_pid = None
|
||||||
if not new_pid:
|
|
||||||
new_pid = _systemd_main_pid_from_props(props)
|
|
||||||
|
|
||||||
if active_state == "active":
|
if active_state == "active":
|
||||||
if new_pid and (previous_pid is None or new_pid != previous_pid):
|
if new_pid and (previous_pid is None or new_pid != previous_pid):
|
||||||
runtime_state = _gateway_runtime_status_for_pid(new_pid)
|
print(f"✓ {scope_label} service restarted (PID {new_pid})")
|
||||||
gateway_state = (runtime_state or {}).get("gateway_state")
|
return True
|
||||||
if gateway_state == "running":
|
if previous_pid is None:
|
||||||
print(f"✓ {scope_label} service restarted (PID {new_pid})")
|
print(f"✓ {scope_label} service restarted")
|
||||||
return True
|
return True
|
||||||
if gateway_state == "startup_failed":
|
|
||||||
reason = (runtime_state or {}).get("exit_reason") or "startup failed"
|
|
||||||
print(f"⚠ {scope_label} service process restarted (PID {new_pid}), but gateway startup failed: {reason}")
|
|
||||||
return False
|
|
||||||
if not printed_runtime_wait:
|
|
||||||
print(f"⏳ {scope_label} service process started (PID {new_pid}); waiting for gateway runtime...")
|
|
||||||
printed_runtime_wait = True
|
|
||||||
|
|
||||||
if active_state == "activating" and sub_state == "auto-restart":
|
if active_state == "activating" and sub_state == "auto-restart":
|
||||||
time.sleep(1)
|
time.sleep(1)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
if _systemd_unit_is_start_limited(props):
|
|
||||||
_print_systemd_start_limit_wait(system=system)
|
|
||||||
return False
|
|
||||||
|
|
||||||
time.sleep(2)
|
time.sleep(2)
|
||||||
|
|
||||||
print(
|
print(
|
||||||
@@ -675,46 +585,6 @@ def _wait_for_systemd_service_restart(
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _systemd_unit_is_start_limited(props: dict[str, str]) -> bool:
|
|
||||||
result = props.get("Result", "").lower()
|
|
||||||
sub_state = props.get("SubState", "").lower()
|
|
||||||
return result == "start-limit-hit" or sub_state == "start-limit-hit"
|
|
||||||
|
|
||||||
|
|
||||||
def _systemd_error_indicates_start_limit(exc: subprocess.CalledProcessError) -> bool:
|
|
||||||
parts: list[str] = []
|
|
||||||
for attr in ("stderr", "stdout", "output"):
|
|
||||||
value = getattr(exc, attr, None)
|
|
||||||
if not value:
|
|
||||||
continue
|
|
||||||
if isinstance(value, bytes):
|
|
||||||
value = value.decode(errors="replace")
|
|
||||||
parts.append(str(value))
|
|
||||||
text = "\n".join(parts).lower()
|
|
||||||
return (
|
|
||||||
"start-limit-hit" in text
|
|
||||||
or "start request repeated too quickly" in text
|
|
||||||
or "start-limit" in text
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _systemd_service_is_start_limited(system: bool = False) -> bool:
|
|
||||||
return _systemd_unit_is_start_limited(_read_systemd_unit_properties(system=system))
|
|
||||||
|
|
||||||
|
|
||||||
def _print_systemd_start_limit_wait(system: bool = False) -> None:
|
|
||||||
svc = get_service_name()
|
|
||||||
scope_label = _service_scope_label(system).capitalize()
|
|
||||||
scope_flag = " --system" if system else ""
|
|
||||||
systemctl_prefix = "systemctl " if system else "systemctl --user "
|
|
||||||
journal_prefix = "journalctl " if system else "journalctl --user "
|
|
||||||
print(f"⏳ {scope_label} service is temporarily rate-limited by systemd.")
|
|
||||||
print(" systemd is refusing another immediate start after repeated exits.")
|
|
||||||
print(f" Wait for the start-limit window to expire, then run: {'sudo ' if system else ''}hermes gateway restart{scope_flag}")
|
|
||||||
print(f" Or clear the failed state manually: {systemctl_prefix}reset-failed {svc}")
|
|
||||||
print(f" Check logs: {journal_prefix}-u {svc} -l --since '5 min ago'")
|
|
||||||
|
|
||||||
|
|
||||||
def _recover_pending_systemd_restart(system: bool = False, previous_pid: int | None = None) -> bool:
|
def _recover_pending_systemd_restart(system: bool = False, previous_pid: int | None = None) -> bool:
|
||||||
"""Recover a planned service restart that is stuck in systemd state."""
|
"""Recover a planned service restart that is stuck in systemd state."""
|
||||||
props = _read_systemd_unit_properties(system=system)
|
props = _read_systemd_unit_properties(system=system)
|
||||||
@@ -870,46 +740,6 @@ def _print_other_profiles_gateway_status() -> None:
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
def _gateway_list() -> None:
|
|
||||||
"""List all profiles and their gateway running status.
|
|
||||||
|
|
||||||
Provides a single-command overview of every known profile and whether
|
|
||||||
its gateway is currently running, so multi-profile users don't have to
|
|
||||||
check each profile individually.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
from hermes_cli.profiles import list_profiles, get_active_profile_name
|
|
||||||
except Exception:
|
|
||||||
print("Unable to list profiles.")
|
|
||||||
return
|
|
||||||
|
|
||||||
profiles = list_profiles()
|
|
||||||
if not profiles:
|
|
||||||
print("No profiles found.")
|
|
||||||
return
|
|
||||||
|
|
||||||
current = get_active_profile_name()
|
|
||||||
|
|
||||||
print("Gateways:")
|
|
||||||
for prof in profiles:
|
|
||||||
marker = "✓" if prof.gateway_running else "✗"
|
|
||||||
label = prof.name
|
|
||||||
if prof.name == current:
|
|
||||||
label += " (current)"
|
|
||||||
parts = [f" {marker} {label:<24s}"]
|
|
||||||
if prof.gateway_running:
|
|
||||||
try:
|
|
||||||
from gateway.status import get_running_pid
|
|
||||||
pid = get_running_pid(prof.path / "gateway.pid", cleanup_stale=False)
|
|
||||||
if pid:
|
|
||||||
parts.append(f"PID {pid}")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
parts.append("not running")
|
|
||||||
print(" — ".join(parts))
|
|
||||||
|
|
||||||
|
|
||||||
def kill_gateway_processes(force: bool = False, exclude_pids: set | None = None,
|
def kill_gateway_processes(force: bool = False, exclude_pids: set | None = None,
|
||||||
all_profiles: bool = False) -> int:
|
all_profiles: bool = False) -> int:
|
||||||
"""Kill any running gateway processes. Returns count killed.
|
"""Kill any running gateway processes. Returns count killed.
|
||||||
@@ -975,8 +805,7 @@ def stop_profile_gateway() -> bool:
|
|||||||
try:
|
try:
|
||||||
os.kill(pid, 0)
|
os.kill(pid, 0)
|
||||||
_time.sleep(0.5)
|
_time.sleep(0.5)
|
||||||
except (ProcessLookupError, PermissionError, OSError):
|
except (ProcessLookupError, PermissionError):
|
||||||
# OSError covers Windows' WinError 87 for gone PIDs.
|
|
||||||
break
|
break
|
||||||
|
|
||||||
if get_running_pid() is None:
|
if get_running_pid() is None:
|
||||||
@@ -1138,27 +967,6 @@ class UserSystemdUnavailableError(RuntimeError):
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
class SystemScopeRequiresRootError(RuntimeError):
|
|
||||||
"""Raised when a system-scope gateway operation is attempted as non-root.
|
|
||||||
|
|
||||||
System-scope units live in ``/etc/systemd/system/`` and require root for
|
|
||||||
install / uninstall / start / stop / restart via ``systemctl``. The
|
|
||||||
previous behavior was ``sys.exit(1)`` which blew past the wizard's
|
|
||||||
``except Exception`` guards and dumped the user at a bare shell prompt
|
|
||||||
with no guidance. Raising a typed exception lets callers that can
|
|
||||||
recover (the setup wizard) print actionable remediation instead, while
|
|
||||||
``gateway_command`` still exits 1 with the same message for the direct
|
|
||||||
CLI path.
|
|
||||||
|
|
||||||
``args[0]`` carries the user-facing message, ``args[1]`` the action name.
|
|
||||||
``str(e)`` returns only the message (not the tuple repr) so format
|
|
||||||
strings like ``f"Failed: {e}"`` render cleanly.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __str__(self) -> str:
|
|
||||||
return self.args[0] if self.args else ""
|
|
||||||
|
|
||||||
|
|
||||||
def _user_dbus_socket_path() -> Path:
|
def _user_dbus_socket_path() -> Path:
|
||||||
"""Return the expected per-user D-Bus socket path (regardless of existence)."""
|
"""Return the expected per-user D-Bus socket path (regardless of existence)."""
|
||||||
xdg = os.environ.get("XDG_RUNTIME_DIR") or f"/run/user/{os.getuid()}"
|
xdg = os.environ.get("XDG_RUNTIME_DIR") or f"/run/user/{os.getuid()}"
|
||||||
@@ -1574,10 +1382,8 @@ def print_systemd_scope_conflict_warning() -> None:
|
|||||||
|
|
||||||
def _require_root_for_system_service(action: str) -> None:
|
def _require_root_for_system_service(action: str) -> None:
|
||||||
if os.geteuid() != 0:
|
if os.geteuid() != 0:
|
||||||
raise SystemScopeRequiresRootError(
|
print(f"System gateway {action} requires root. Re-run with sudo.")
|
||||||
f"System gateway {action} requires root. Re-run with sudo.",
|
sys.exit(1)
|
||||||
action,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _system_service_identity(run_as_user: str | None = None) -> tuple[str, str, str]:
|
def _system_service_identity(run_as_user: str | None = None) -> tuple[str, str, str]:
|
||||||
@@ -2124,47 +1930,6 @@ def _select_systemd_scope(system: bool = False) -> bool:
|
|||||||
return get_systemd_unit_path(system=True).exists() and not get_systemd_unit_path(system=False).exists()
|
return get_systemd_unit_path(system=True).exists() and not get_systemd_unit_path(system=False).exists()
|
||||||
|
|
||||||
|
|
||||||
def _system_scope_wizard_would_need_root(system: bool = False) -> bool:
|
|
||||||
"""True when the setup wizard is about to trigger a system-scope operation
|
|
||||||
as a non-root user.
|
|
||||||
|
|
||||||
Replicates the decision ``_select_systemd_scope`` makes inside
|
|
||||||
``systemd_start`` / ``systemd_restart`` / ``systemd_stop`` so the wizard
|
|
||||||
can detect the dead-end BEFORE prompting, rather than letting
|
|
||||||
``SystemScopeRequiresRootError`` propagate out and leave the user
|
|
||||||
staring at a bare shell.
|
|
||||||
"""
|
|
||||||
if os.geteuid() == 0:
|
|
||||||
return False
|
|
||||||
return _select_systemd_scope(system=system)
|
|
||||||
|
|
||||||
|
|
||||||
def _print_system_scope_remediation(action: str) -> None:
|
|
||||||
"""Print actionable remediation when the wizard skips a system-scope
|
|
||||||
prompt because the user isn't root. Keeps the wizard flowing instead of
|
|
||||||
aborting.
|
|
||||||
"""
|
|
||||||
svc = get_service_name()
|
|
||||||
print_warning(
|
|
||||||
f"Gateway is installed as a system-wide service — "
|
|
||||||
f"{action} requires root."
|
|
||||||
)
|
|
||||||
print_info(" Options:")
|
|
||||||
print_info(f" 1. {action.capitalize()} it this time:")
|
|
||||||
if action == "start":
|
|
||||||
print_info(f" sudo systemctl start {svc}")
|
|
||||||
elif action == "stop":
|
|
||||||
print_info(f" sudo systemctl stop {svc}")
|
|
||||||
elif action == "restart":
|
|
||||||
print_info(f" sudo systemctl restart {svc}")
|
|
||||||
else:
|
|
||||||
print_info(f" sudo systemctl {action} {svc}")
|
|
||||||
print_info(" 2. Switch to a per-user service (recommended for personal use):")
|
|
||||||
print_info(" sudo hermes gateway uninstall --system")
|
|
||||||
print_info(" hermes gateway install")
|
|
||||||
print_info(" hermes gateway start")
|
|
||||||
|
|
||||||
|
|
||||||
def _get_restart_drain_timeout() -> float:
|
def _get_restart_drain_timeout() -> float:
|
||||||
"""Return the configured gateway restart drain timeout in seconds."""
|
"""Return the configured gateway restart drain timeout in seconds."""
|
||||||
raw = os.getenv("HERMES_RESTART_DRAIN_TIMEOUT", "").strip()
|
raw = os.getenv("HERMES_RESTART_DRAIN_TIMEOUT", "").strip()
|
||||||
@@ -2306,52 +2071,41 @@ def systemd_restart(system: bool = False):
|
|||||||
refresh_systemd_unit_if_needed(system=system)
|
refresh_systemd_unit_if_needed(system=system)
|
||||||
from gateway.status import get_running_pid
|
from gateway.status import get_running_pid
|
||||||
|
|
||||||
pid = get_running_pid() or _systemd_main_pid(system=system)
|
pid = get_running_pid()
|
||||||
if pid is not None:
|
if pid is not None and _request_gateway_self_restart(pid):
|
||||||
|
import time
|
||||||
scope_label = _service_scope_label(system).capitalize()
|
scope_label = _service_scope_label(system).capitalize()
|
||||||
svc = get_service_name()
|
svc = get_service_name()
|
||||||
drain_timeout = _get_restart_drain_timeout()
|
|
||||||
|
|
||||||
print(f"⏳ {scope_label} service restarting gracefully (PID {pid})...")
|
# Phase 1: wait for old process to exit (drain + shutdown)
|
||||||
if _graceful_restart_via_sigusr1(pid, drain_timeout + 5):
|
print(f"⏳ {scope_label} service draining active work...")
|
||||||
# The gateway exits with code 75 for a planned service restart.
|
deadline = time.time() + 90
|
||||||
# RestartSec can otherwise delay the relaunch even though the
|
while time.time() < deadline:
|
||||||
# operator asked for an immediate restart, so kick the unit once
|
try:
|
||||||
# the old PID has exited and then wait for the replacement PID.
|
os.kill(pid, 0)
|
||||||
_run_systemctl(
|
time.sleep(1)
|
||||||
["reset-failed", svc],
|
except (ProcessLookupError, PermissionError):
|
||||||
system=system,
|
break # old process is gone
|
||||||
check=False,
|
else:
|
||||||
timeout=30,
|
print(f"⚠ Old process (PID {pid}) still alive after 90s")
|
||||||
)
|
|
||||||
_run_systemctl(
|
|
||||||
["restart", svc],
|
|
||||||
system=system,
|
|
||||||
check=False,
|
|
||||||
timeout=90,
|
|
||||||
)
|
|
||||||
if _wait_for_systemd_service_restart(system=system, previous_pid=pid):
|
|
||||||
return
|
|
||||||
if _systemd_service_is_start_limited(system=system):
|
|
||||||
return
|
|
||||||
|
|
||||||
print(
|
# The gateway exits with code 75 for a planned service restart.
|
||||||
f"⚠ Graceful restart did not complete within {int(drain_timeout + 5)}s; "
|
# systemd can sit in the RestartSec window or even wedge itself into a
|
||||||
"forcing a service restart..."
|
# failed/rate-limited state if the operator asks for another restart in
|
||||||
)
|
# the middle of that handoff. Clear any stale failed state and kick the
|
||||||
|
# unit immediately so `hermes gateway restart` behaves idempotently.
|
||||||
_run_systemctl(
|
_run_systemctl(
|
||||||
["reset-failed", svc],
|
["reset-failed", svc],
|
||||||
system=system,
|
system=system,
|
||||||
check=False,
|
check=False,
|
||||||
timeout=30,
|
timeout=30,
|
||||||
)
|
)
|
||||||
try:
|
_run_systemctl(
|
||||||
_run_systemctl(["restart", svc], system=system, check=True, timeout=90)
|
["start", svc],
|
||||||
except subprocess.CalledProcessError as exc:
|
system=system,
|
||||||
if _systemd_error_indicates_start_limit(exc) or _systemd_service_is_start_limited(system=system):
|
check=False,
|
||||||
_print_systemd_start_limit_wait(system=system)
|
timeout=90,
|
||||||
return
|
)
|
||||||
raise
|
|
||||||
_wait_for_systemd_service_restart(system=system, previous_pid=pid)
|
_wait_for_systemd_service_restart(system=system, previous_pid=pid)
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -2364,14 +2118,8 @@ def systemd_restart(system: bool = False):
|
|||||||
check=False,
|
check=False,
|
||||||
timeout=30,
|
timeout=30,
|
||||||
)
|
)
|
||||||
try:
|
_run_systemctl(["reload-or-restart", get_service_name()], system=system, check=True, timeout=90)
|
||||||
_run_systemctl(["restart", get_service_name()], system=system, check=True, timeout=90)
|
print(f"✓ {_service_scope_label(system).capitalize()} service restarted")
|
||||||
except subprocess.CalledProcessError as exc:
|
|
||||||
if _systemd_error_indicates_start_limit(exc) or _systemd_service_is_start_limited(system=system):
|
|
||||||
_print_systemd_start_limit_wait(system=system)
|
|
||||||
return
|
|
||||||
raise
|
|
||||||
_wait_for_systemd_service_restart(system=system, previous_pid=pid)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -2443,10 +2191,6 @@ def systemd_status(deep: bool = False, system: bool = False, full: bool = False)
|
|||||||
result_code = unit_props.get("Result", "")
|
result_code = unit_props.get("Result", "")
|
||||||
if active_state == "activating" and sub_state == "auto-restart":
|
if active_state == "activating" and sub_state == "auto-restart":
|
||||||
print(" ⏳ Restart pending: systemd is waiting to relaunch the gateway")
|
print(" ⏳ Restart pending: systemd is waiting to relaunch the gateway")
|
||||||
elif _systemd_unit_is_start_limited(unit_props):
|
|
||||||
print(" ⏳ Restart pending: systemd is temporarily rate-limiting starts")
|
|
||||||
print(f" Run after the start-limit window expires: {'sudo ' if system else ''}hermes gateway restart{scope_flag}")
|
|
||||||
print(f" Or clear it manually: systemctl {'--user ' if not system else ''}reset-failed {get_service_name()}")
|
|
||||||
elif active_state == "failed" and exec_main_status == str(GATEWAY_SERVICE_RESTART_EXIT_CODE):
|
elif active_state == "failed" and exec_main_status == str(GATEWAY_SERVICE_RESTART_EXIT_CODE):
|
||||||
print(" ⚠ Planned restart is stuck in systemd failed state (exit 75)")
|
print(" ⚠ Planned restart is stuck in systemd failed state (exit 75)")
|
||||||
print(f" Run: systemctl {'--user ' if not system else ''}reset-failed {get_service_name()} && {'sudo ' if system else ''}hermes gateway start{scope_flag}")
|
print(f" Run: systemctl {'--user ' if not system else ''}reset-failed {get_service_name()} && {'sudo ' if system else ''}hermes gateway start{scope_flag}")
|
||||||
@@ -2811,42 +2555,6 @@ def launchd_status(deep: bool = False):
|
|||||||
# Gateway Runner
|
# Gateway Runner
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
|
|
||||||
def _truthy_env(value: str | None) -> bool:
|
|
||||||
return str(value or "").strip().lower() in {"1", "true", "yes", "on"}
|
|
||||||
|
|
||||||
|
|
||||||
def _is_official_docker_checkout() -> bool:
|
|
||||||
return (
|
|
||||||
str(PROJECT_ROOT) == "/opt/hermes"
|
|
||||||
and (PROJECT_ROOT / "docker" / "entrypoint.sh").is_file()
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _guard_official_docker_root_gateway() -> None:
|
|
||||||
"""Refuse gateway startup when the official Docker privilege drop was bypassed."""
|
|
||||||
if not hasattr(os, "geteuid") or os.geteuid() != 0:
|
|
||||||
return
|
|
||||||
if _truthy_env(os.getenv("HERMES_ALLOW_ROOT_GATEWAY")):
|
|
||||||
return
|
|
||||||
if not _is_official_docker_checkout():
|
|
||||||
return
|
|
||||||
|
|
||||||
print_error(
|
|
||||||
"Refusing to run the Hermes gateway as root inside the official Docker image."
|
|
||||||
)
|
|
||||||
print(
|
|
||||||
" The image entrypoint normally drops privileges to the 'hermes' user. "
|
|
||||||
"If you override entrypoint in Docker Compose, include "
|
|
||||||
"/opt/hermes/docker/entrypoint.sh before the Hermes command."
|
|
||||||
)
|
|
||||||
print(
|
|
||||||
" Running the gateway as root can leave root-owned files in "
|
|
||||||
"$HERMES_HOME and break later non-root dashboard/gateway runs."
|
|
||||||
)
|
|
||||||
print(" Set HERMES_ALLOW_ROOT_GATEWAY=1 only if you intentionally accept this risk.")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
def run_gateway(verbose: int = 0, quiet: bool = False, replace: bool = False):
|
def run_gateway(verbose: int = 0, quiet: bool = False, replace: bool = False):
|
||||||
"""Run the gateway in foreground.
|
"""Run the gateway in foreground.
|
||||||
|
|
||||||
@@ -2857,7 +2565,6 @@ def run_gateway(verbose: int = 0, quiet: bool = False, replace: bool = False):
|
|||||||
This prevents systemd restart loops when the old process
|
This prevents systemd restart loops when the old process
|
||||||
hasn't fully exited yet.
|
hasn't fully exited yet.
|
||||||
"""
|
"""
|
||||||
_guard_official_docker_root_gateway()
|
|
||||||
sys.path.insert(0, str(PROJECT_ROOT))
|
sys.path.insert(0, str(PROJECT_ROOT))
|
||||||
|
|
||||||
# Refresh the systemd unit definition on every boot so that restart
|
# Refresh the systemd unit definition on every boot so that restart
|
||||||
@@ -4408,9 +4115,7 @@ def gateway_setup():
|
|||||||
print_success("Gateway service is installed and running.")
|
print_success("Gateway service is installed and running.")
|
||||||
elif service_installed:
|
elif service_installed:
|
||||||
print_warning("Gateway service is installed but not running.")
|
print_warning("Gateway service is installed but not running.")
|
||||||
if supports_systemd_services() and _system_scope_wizard_would_need_root():
|
if prompt_yes_no(" Start it now?", True):
|
||||||
_print_system_scope_remediation("start")
|
|
||||||
elif prompt_yes_no(" Start it now?", True):
|
|
||||||
try:
|
try:
|
||||||
if supports_systemd_services():
|
if supports_systemd_services():
|
||||||
systemd_start()
|
systemd_start()
|
||||||
@@ -4420,12 +4125,6 @@ def gateway_setup():
|
|||||||
print_error(" Failed to start — user systemd not reachable:")
|
print_error(" Failed to start — user systemd not reachable:")
|
||||||
for line in str(e).splitlines():
|
for line in str(e).splitlines():
|
||||||
print(f" {line}")
|
print(f" {line}")
|
||||||
except SystemScopeRequiresRootError as e:
|
|
||||||
# Defense in depth: the pre-check above should have caught
|
|
||||||
# this, but handle the race/edge case gracefully instead of
|
|
||||||
# letting the exception escape the wizard.
|
|
||||||
print_error(f" Failed to start: {e}")
|
|
||||||
_print_system_scope_remediation("start")
|
|
||||||
except subprocess.CalledProcessError as e:
|
except subprocess.CalledProcessError as e:
|
||||||
print_error(f" Failed to start: {e}")
|
print_error(f" Failed to start: {e}")
|
||||||
else:
|
else:
|
||||||
@@ -4475,9 +4174,7 @@ def gateway_setup():
|
|||||||
service_running = _is_service_running()
|
service_running = _is_service_running()
|
||||||
|
|
||||||
if service_running:
|
if service_running:
|
||||||
if supports_systemd_services() and _system_scope_wizard_would_need_root():
|
if prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
||||||
_print_system_scope_remediation("restart")
|
|
||||||
elif prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
|
||||||
try:
|
try:
|
||||||
if supports_systemd_services():
|
if supports_systemd_services():
|
||||||
systemd_restart()
|
systemd_restart()
|
||||||
@@ -4490,15 +4187,10 @@ def gateway_setup():
|
|||||||
print_error(" Restart failed — user systemd not reachable:")
|
print_error(" Restart failed — user systemd not reachable:")
|
||||||
for line in str(e).splitlines():
|
for line in str(e).splitlines():
|
||||||
print(f" {line}")
|
print(f" {line}")
|
||||||
except SystemScopeRequiresRootError as e:
|
|
||||||
print_error(f" Restart failed: {e}")
|
|
||||||
_print_system_scope_remediation("restart")
|
|
||||||
except subprocess.CalledProcessError as e:
|
except subprocess.CalledProcessError as e:
|
||||||
print_error(f" Restart failed: {e}")
|
print_error(f" Restart failed: {e}")
|
||||||
elif service_installed:
|
elif service_installed:
|
||||||
if supports_systemd_services() and _system_scope_wizard_would_need_root():
|
if prompt_yes_no(" Start the gateway service?", True):
|
||||||
_print_system_scope_remediation("start")
|
|
||||||
elif prompt_yes_no(" Start the gateway service?", True):
|
|
||||||
try:
|
try:
|
||||||
if supports_systemd_services():
|
if supports_systemd_services():
|
||||||
systemd_start()
|
systemd_start()
|
||||||
@@ -4508,9 +4200,6 @@ def gateway_setup():
|
|||||||
print_error(" Start failed — user systemd not reachable:")
|
print_error(" Start failed — user systemd not reachable:")
|
||||||
for line in str(e).splitlines():
|
for line in str(e).splitlines():
|
||||||
print(f" {line}")
|
print(f" {line}")
|
||||||
except SystemScopeRequiresRootError as e:
|
|
||||||
print_error(f" Start failed: {e}")
|
|
||||||
_print_system_scope_remediation("start")
|
|
||||||
except subprocess.CalledProcessError as e:
|
except subprocess.CalledProcessError as e:
|
||||||
print_error(f" Start failed: {e}")
|
print_error(f" Start failed: {e}")
|
||||||
else:
|
else:
|
||||||
@@ -4584,14 +4273,6 @@ def gateway_command(args):
|
|||||||
for line in str(e).splitlines():
|
for line in str(e).splitlines():
|
||||||
print(f" {line}")
|
print(f" {line}")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
except SystemScopeRequiresRootError as e:
|
|
||||||
# The direct ``hermes gateway install|uninstall|start|stop|restart``
|
|
||||||
# path lands here when the user typed a system-scope action without
|
|
||||||
# sudo. Same exit code as before — just gives the wizard a way to
|
|
||||||
# intercept the same condition with friendlier guidance before the
|
|
||||||
# error is raised.
|
|
||||||
print(str(e))
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
def _gateway_command_inner(args):
|
def _gateway_command_inner(args):
|
||||||
@@ -4916,9 +4597,6 @@ def _gateway_command_inner(args):
|
|||||||
# Show other profiles' gateway status for multi-profile awareness
|
# Show other profiles' gateway status for multi-profile awareness
|
||||||
_print_other_profiles_gateway_status()
|
_print_other_profiles_gateway_status()
|
||||||
|
|
||||||
elif subcmd == "list":
|
|
||||||
_gateway_list()
|
|
||||||
|
|
||||||
elif subcmd == "migrate-legacy":
|
elif subcmd == "migrate-legacy":
|
||||||
# Stop, disable, and remove legacy Hermes gateway unit files from
|
# Stop, disable, and remove legacy Hermes gateway unit files from
|
||||||
# pre-rename installs (e.g. hermes.service). Profile units and
|
# pre-rename installs (e.g. hermes.service). Profile units and
|
||||||
|
|||||||
+1
-1
@@ -205,7 +205,7 @@ def _cmd_test(args) -> None:
|
|||||||
|
|
||||||
if getattr(args, "payload_file", None):
|
if getattr(args, "payload_file", None):
|
||||||
try:
|
try:
|
||||||
custom = json.loads(Path(args.payload_file).read_text(encoding="utf-8"))
|
custom = json.loads(Path(args.payload_file).read_text())
|
||||||
if isinstance(custom, dict):
|
if isinstance(custom, dict):
|
||||||
payload.update(custom)
|
payload.update(custom)
|
||||||
else:
|
else:
|
||||||
|
|||||||
+2
-150
@@ -70,7 +70,6 @@ def _task_to_dict(t: kb.Task) -> dict[str, Any]:
|
|||||||
"completed_at": t.completed_at,
|
"completed_at": t.completed_at,
|
||||||
"result": t.result,
|
"result": t.result,
|
||||||
"skills": list(t.skills) if t.skills else [],
|
"skills": list(t.skills) if t.skills else [],
|
||||||
"max_retries": t.max_retries,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -285,15 +284,6 @@ def build_parser(parent_subparsers: argparse._SubParsersAction) -> argparse.Argu
|
|||||||
"(repeatable). Appended to the built-in "
|
"(repeatable). Appended to the built-in "
|
||||||
"kanban-worker skill. Example: "
|
"kanban-worker skill. Example: "
|
||||||
"--skill translation --skill github-code-review")
|
"--skill translation --skill github-code-review")
|
||||||
p_create.add_argument("--max-retries", type=int, default=None,
|
|
||||||
metavar="N",
|
|
||||||
help="Per-task override for the consecutive-failure "
|
|
||||||
"circuit breaker. Trip on the Nth failure — "
|
|
||||||
"e.g. --max-retries 1 blocks on the first "
|
|
||||||
"failure (no retries), --max-retries 3 allows "
|
|
||||||
"two retries. Omit to use the dispatcher's "
|
|
||||||
"kanban.failure_limit config "
|
|
||||||
f"(default {kb.DEFAULT_FAILURE_LIMIT}).")
|
|
||||||
p_create.add_argument("--json", action="store_true", help="Emit JSON output")
|
p_create.add_argument("--json", action="store_true", help="Emit JSON output")
|
||||||
|
|
||||||
# --- list ---
|
# --- list ---
|
||||||
@@ -453,8 +443,8 @@ def build_parser(parent_subparsers: argparse._SubParsersAction) -> argparse.Argu
|
|||||||
help="Cap number of spawns this pass")
|
help="Cap number of spawns this pass")
|
||||||
p_disp.add_argument("--failure-limit", type=int,
|
p_disp.add_argument("--failure-limit", type=int,
|
||||||
default=kb.DEFAULT_SPAWN_FAILURE_LIMIT,
|
default=kb.DEFAULT_SPAWN_FAILURE_LIMIT,
|
||||||
help=f"Auto-block a task after this many consecutive non-success attempts "
|
help=f"Auto-block a task after this many consecutive spawn failures "
|
||||||
f"(spawn_failed, timed_out, or crashed; default: {kb.DEFAULT_SPAWN_FAILURE_LIMIT})")
|
f"(default: {kb.DEFAULT_SPAWN_FAILURE_LIMIT})")
|
||||||
p_disp.add_argument("--json", action="store_true")
|
p_disp.add_argument("--json", action="store_true")
|
||||||
|
|
||||||
# --- daemon (deprecated) ---
|
# --- daemon (deprecated) ---
|
||||||
@@ -570,42 +560,6 @@ def build_parser(parent_subparsers: argparse._SubParsersAction) -> argparse.Argu
|
|||||||
)
|
)
|
||||||
p_ctx.add_argument("task_id")
|
p_ctx.add_argument("task_id")
|
||||||
|
|
||||||
# --- specify --- (triage → todo via auxiliary LLM)
|
|
||||||
p_specify = sub.add_parser(
|
|
||||||
"specify",
|
|
||||||
help="Flesh out a triage-column task into a concrete spec "
|
|
||||||
"(title + body) and promote it to todo. Uses the auxiliary "
|
|
||||||
"LLM configured under auxiliary.triage_specifier.",
|
|
||||||
)
|
|
||||||
p_specify.add_argument(
|
|
||||||
"task_id",
|
|
||||||
nargs="?",
|
|
||||||
default=None,
|
|
||||||
help="Task id to specify (required unless --all is given)",
|
|
||||||
)
|
|
||||||
p_specify.add_argument(
|
|
||||||
"--all",
|
|
||||||
dest="all_triage",
|
|
||||||
action="store_true",
|
|
||||||
help="Specify every task currently in the triage column",
|
|
||||||
)
|
|
||||||
p_specify.add_argument(
|
|
||||||
"--tenant",
|
|
||||||
default=None,
|
|
||||||
help="When used with --all, restrict the sweep to this tenant",
|
|
||||||
)
|
|
||||||
p_specify.add_argument(
|
|
||||||
"--author",
|
|
||||||
default=None,
|
|
||||||
help="Author name recorded on the audit comment "
|
|
||||||
"(default: $HERMES_PROFILE or 'specifier')",
|
|
||||||
)
|
|
||||||
p_specify.add_argument(
|
|
||||||
"--json",
|
|
||||||
action="store_true",
|
|
||||||
help="Emit one JSON object per task on stdout",
|
|
||||||
)
|
|
||||||
|
|
||||||
# --- gc ---
|
# --- gc ---
|
||||||
p_gc = sub.add_parser(
|
p_gc = sub.add_parser(
|
||||||
"gc", help="Garbage-collect archived-task workspaces, old events, and old logs",
|
"gc", help="Garbage-collect archived-task workspaces, old events, and old logs",
|
||||||
@@ -720,7 +674,6 @@ def kanban_command(args: argparse.Namespace) -> int:
|
|||||||
"notify-list": _cmd_notify_list,
|
"notify-list": _cmd_notify_list,
|
||||||
"notify-unsubscribe": _cmd_notify_unsubscribe,
|
"notify-unsubscribe": _cmd_notify_unsubscribe,
|
||||||
"context": _cmd_context,
|
"context": _cmd_context,
|
||||||
"specify": _cmd_specify,
|
|
||||||
"gc": _cmd_gc,
|
"gc": _cmd_gc,
|
||||||
}
|
}
|
||||||
handler = handlers.get(action)
|
handler = handlers.get(action)
|
||||||
@@ -1029,14 +982,6 @@ def _cmd_create(args: argparse.Namespace) -> int:
|
|||||||
except ValueError as exc:
|
except ValueError as exc:
|
||||||
print(f"kanban: --max-runtime: {exc}", file=sys.stderr)
|
print(f"kanban: --max-runtime: {exc}", file=sys.stderr)
|
||||||
return 2
|
return 2
|
||||||
max_retries = getattr(args, "max_retries", None)
|
|
||||||
if max_retries is not None and max_retries < 1:
|
|
||||||
print(
|
|
||||||
f"kanban: --max-retries must be >= 1 (got {max_retries}); "
|
|
||||||
"use 1 to trip on the first failure.",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
return 2
|
|
||||||
with kb.connect() as conn:
|
with kb.connect() as conn:
|
||||||
task_id = kb.create_task(
|
task_id = kb.create_task(
|
||||||
conn,
|
conn,
|
||||||
@@ -1053,7 +998,6 @@ def _cmd_create(args: argparse.Namespace) -> int:
|
|||||||
idempotency_key=getattr(args, "idempotency_key", None),
|
idempotency_key=getattr(args, "idempotency_key", None),
|
||||||
max_runtime_seconds=max_runtime,
|
max_runtime_seconds=max_runtime,
|
||||||
skills=getattr(args, "skills", None) or None,
|
skills=getattr(args, "skills", None) or None,
|
||||||
max_retries=max_retries,
|
|
||||||
)
|
)
|
||||||
task = kb.get_task(conn, task_id)
|
task = kb.get_task(conn, task_id)
|
||||||
if getattr(args, "json", False):
|
if getattr(args, "json", False):
|
||||||
@@ -1181,23 +1125,6 @@ def _cmd_show(args: argparse.Namespace) -> int:
|
|||||||
(f" @ {task.workspace_path}" if task.workspace_path else ""))
|
(f" @ {task.workspace_path}" if task.workspace_path else ""))
|
||||||
if task.skills:
|
if task.skills:
|
||||||
print(f" skills: {', '.join(task.skills)}")
|
print(f" skills: {', '.join(task.skills)}")
|
||||||
# Effective retry threshold. Show the per-task override if set,
|
|
||||||
# otherwise the dispatcher's resolved value from config (or the
|
|
||||||
# default if config doesn't set it either). Helps operators see
|
|
||||||
# why a task auto-blocked earlier/later than they expected.
|
|
||||||
if task.max_retries is not None:
|
|
||||||
print(f" max-retries: {task.max_retries} (task)")
|
|
||||||
else:
|
|
||||||
try:
|
|
||||||
from hermes_cli.config import load_config
|
|
||||||
cfg = load_config()
|
|
||||||
cfg_val = (cfg.get("kanban", {}) or {}).get("failure_limit")
|
|
||||||
except Exception:
|
|
||||||
cfg_val = None
|
|
||||||
if cfg_val is not None and int(cfg_val) != kb.DEFAULT_FAILURE_LIMIT:
|
|
||||||
print(f" max-retries: {int(cfg_val)} (config kanban.failure_limit)")
|
|
||||||
else:
|
|
||||||
print(f" max-retries: {kb.DEFAULT_FAILURE_LIMIT} (default)")
|
|
||||||
print(f" created: {_fmt_ts(task.created_at)} by {task.created_by or '-'}")
|
print(f" created: {_fmt_ts(task.created_at)} by {task.created_by or '-'}")
|
||||||
|
|
||||||
# Diagnostics section — surface active distress signals at the top
|
# Diagnostics section — surface active distress signals at the top
|
||||||
@@ -1730,7 +1657,6 @@ def _cmd_daemon(args: argparse.Namespace) -> int:
|
|||||||
" kanban:\n"
|
" kanban:\n"
|
||||||
" dispatch_in_gateway: true # default\n"
|
" dispatch_in_gateway: true # default\n"
|
||||||
" dispatch_interval_seconds: 60\n"
|
" dispatch_interval_seconds: 60\n"
|
||||||
" failure_limit: 2 # consecutive non-success attempts before auto-block\n"
|
|
||||||
"\n"
|
"\n"
|
||||||
"Running both the gateway AND this standalone daemon will\n"
|
"Running both the gateway AND this standalone daemon will\n"
|
||||||
"race for claims. If you truly need the old standalone\n"
|
"race for claims. If you truly need the old standalone\n"
|
||||||
@@ -2017,80 +1943,6 @@ def _cmd_context(args: argparse.Namespace) -> int:
|
|||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def _cmd_specify(args: argparse.Namespace) -> int:
|
|
||||||
"""Flesh out a triage task (or all of them) via auxiliary LLM,
|
|
||||||
then promote to todo. Thin wrapper over ``kanban_specify``."""
|
|
||||||
from hermes_cli import kanban_specify as spec
|
|
||||||
|
|
||||||
all_flag = bool(getattr(args, "all_triage", False))
|
|
||||||
tenant = getattr(args, "tenant", None)
|
|
||||||
author = getattr(args, "author", None) or _profile_author()
|
|
||||||
want_json = bool(getattr(args, "json", False))
|
|
||||||
|
|
||||||
if args.task_id and all_flag:
|
|
||||||
print(
|
|
||||||
"kanban: pass either a task id OR --all, not both",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
return 2
|
|
||||||
|
|
||||||
if all_flag:
|
|
||||||
ids = spec.list_triage_ids(tenant=tenant)
|
|
||||||
if not ids:
|
|
||||||
msg = (
|
|
||||||
"No triage tasks"
|
|
||||||
+ (f" for tenant {tenant!r}" if tenant else "")
|
|
||||||
+ "."
|
|
||||||
)
|
|
||||||
if want_json:
|
|
||||||
print(json.dumps({"specified": 0, "total": 0}))
|
|
||||||
else:
|
|
||||||
print(msg)
|
|
||||||
return 0
|
|
||||||
elif args.task_id:
|
|
||||||
ids = [args.task_id]
|
|
||||||
else:
|
|
||||||
print(
|
|
||||||
"kanban: specify requires a task id or --all",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
return 2
|
|
||||||
|
|
||||||
ok_count = 0
|
|
||||||
fail_count = 0
|
|
||||||
for tid in ids:
|
|
||||||
outcome = spec.specify_task(tid, author=author)
|
|
||||||
if outcome.ok:
|
|
||||||
ok_count += 1
|
|
||||||
else:
|
|
||||||
fail_count += 1
|
|
||||||
if want_json:
|
|
||||||
print(json.dumps({
|
|
||||||
"task_id": outcome.task_id,
|
|
||||||
"ok": outcome.ok,
|
|
||||||
"reason": outcome.reason,
|
|
||||||
"new_title": outcome.new_title,
|
|
||||||
}))
|
|
||||||
else:
|
|
||||||
if outcome.ok:
|
|
||||||
title_suffix = (
|
|
||||||
f" — retitled: {outcome.new_title!r}"
|
|
||||||
if outcome.new_title
|
|
||||||
else ""
|
|
||||||
)
|
|
||||||
print(f"Specified {outcome.task_id} → todo{title_suffix}")
|
|
||||||
else:
|
|
||||||
print(
|
|
||||||
f"kanban: specify {outcome.task_id}: {outcome.reason}",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
if not all_flag:
|
|
||||||
return 0 if ok_count == 1 else 1
|
|
||||||
# --all: succeed if at least one promotion landed; exit 1 only when
|
|
||||||
# every candidate failed (honest signal for scripts).
|
|
||||||
return 0 if (ok_count > 0 or not ids) else 1
|
|
||||||
|
|
||||||
|
|
||||||
def _cmd_gc(args: argparse.Namespace) -> int:
|
def _cmd_gc(args: argparse.Namespace) -> int:
|
||||||
"""Remove scratch workspaces of archived tasks, prune old events, and
|
"""Remove scratch workspaces of archived tasks, prune old events, and
|
||||||
delete old worker logs."""
|
delete old worker logs."""
|
||||||
|
|||||||
+60
-456
@@ -595,14 +595,6 @@ class Task:
|
|||||||
# JSON array of skill names. None = use only the defaults; empty
|
# JSON array of skill names. None = use only the defaults; empty
|
||||||
# list = explicitly no extra skills.
|
# list = explicitly no extra skills.
|
||||||
skills: Optional[list] = None
|
skills: Optional[list] = None
|
||||||
# Per-task override for the consecutive-failure circuit breaker.
|
|
||||||
# The value is the failure count at which the breaker trips — e.g.
|
|
||||||
# ``max_retries=1`` blocks on the first failure (zero retries),
|
|
||||||
# ``max_retries=3`` blocks on the third (two retries allowed).
|
|
||||||
# ``None`` (the common case) falls through to the dispatcher-level
|
|
||||||
# ``kanban.failure_limit`` config, and then to ``DEFAULT_FAILURE_LIMIT``.
|
|
||||||
# Name matches the ``--max-retries`` CLI flag on ``kanban create``.
|
|
||||||
max_retries: Optional[int] = None
|
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_row(cls, row: sqlite3.Row) -> "Task":
|
def from_row(cls, row: sqlite3.Row) -> "Task":
|
||||||
@@ -664,9 +656,6 @@ class Task:
|
|||||||
row["current_step_key"] if "current_step_key" in keys else None
|
row["current_step_key"] if "current_step_key" in keys else None
|
||||||
),
|
),
|
||||||
skills=skills_value,
|
skills=skills_value,
|
||||||
max_retries=(
|
|
||||||
row["max_retries"] if "max_retries" in keys else None
|
|
||||||
),
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -787,13 +776,7 @@ CREATE TABLE IF NOT EXISTS tasks (
|
|||||||
-- Force-loaded skills for the worker on this task, stored as JSON.
|
-- Force-loaded skills for the worker on this task, stored as JSON.
|
||||||
-- Appended to the dispatcher's built-in `--skills kanban-worker`.
|
-- Appended to the dispatcher's built-in `--skills kanban-worker`.
|
||||||
-- NULL or empty array = no extras.
|
-- NULL or empty array = no extras.
|
||||||
skills TEXT,
|
skills TEXT
|
||||||
-- Per-task override for the consecutive-failure circuit breaker.
|
|
||||||
-- The value is the failure count at which the breaker trips — e.g.
|
|
||||||
-- ``max_retries=1`` blocks on the first failure. NULL (the common
|
|
||||||
-- case) falls through to the dispatcher-level ``kanban.failure_limit``
|
|
||||||
-- config and then ``DEFAULT_FAILURE_LIMIT``.
|
|
||||||
max_retries INTEGER
|
|
||||||
);
|
);
|
||||||
|
|
||||||
CREATE TABLE IF NOT EXISTS task_links (
|
CREATE TABLE IF NOT EXISTS task_links (
|
||||||
@@ -1025,14 +1008,6 @@ def _migrate_add_optional_columns(conn: sqlite3.Connection) -> None:
|
|||||||
# for existing rows.
|
# for existing rows.
|
||||||
conn.execute("ALTER TABLE tasks ADD COLUMN skills TEXT")
|
conn.execute("ALTER TABLE tasks ADD COLUMN skills TEXT")
|
||||||
|
|
||||||
if "max_retries" not in cols:
|
|
||||||
# Per-task override for the consecutive-failure circuit breaker.
|
|
||||||
# NULL = fall through to the dispatcher-level ``kanban.failure_limit``
|
|
||||||
# config, then ``DEFAULT_FAILURE_LIMIT``. Existing rows get NULL,
|
|
||||||
# which is the correct default (they keep the global behaviour
|
|
||||||
# they were getting before the column existed).
|
|
||||||
conn.execute("ALTER TABLE tasks ADD COLUMN max_retries INTEGER")
|
|
||||||
|
|
||||||
# task_events gained a run_id column; back-fill it as NULL for
|
# task_events gained a run_id column; back-fill it as NULL for
|
||||||
# historical events (they predate runs and can't be attributed).
|
# historical events (they predate runs and can't be attributed).
|
||||||
ev_cols = {row["name"] for row in conn.execute("PRAGMA table_info(task_events)")}
|
ev_cols = {row["name"] for row in conn.execute("PRAGMA table_info(task_events)")}
|
||||||
@@ -1188,7 +1163,6 @@ def create_task(
|
|||||||
idempotency_key: Optional[str] = None,
|
idempotency_key: Optional[str] = None,
|
||||||
max_runtime_seconds: Optional[int] = None,
|
max_runtime_seconds: Optional[int] = None,
|
||||||
skills: Optional[Iterable[str]] = None,
|
skills: Optional[Iterable[str]] = None,
|
||||||
max_retries: Optional[int] = None,
|
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Create a new task and optionally link it under parent tasks.
|
"""Create a new task and optionally link it under parent tasks.
|
||||||
|
|
||||||
@@ -1302,9 +1276,8 @@ def create_task(
|
|||||||
INSERT INTO tasks (
|
INSERT INTO tasks (
|
||||||
id, title, body, assignee, status, priority,
|
id, title, body, assignee, status, priority,
|
||||||
created_by, created_at, workspace_kind, workspace_path,
|
created_by, created_at, workspace_kind, workspace_path,
|
||||||
tenant, idempotency_key, max_runtime_seconds, skills,
|
tenant, idempotency_key, max_runtime_seconds, skills
|
||||||
max_retries
|
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
|
||||||
""",
|
""",
|
||||||
(
|
(
|
||||||
task_id,
|
task_id,
|
||||||
@@ -1321,7 +1294,6 @@ def create_task(
|
|||||||
idempotency_key,
|
idempotency_key,
|
||||||
int(max_runtime_seconds) if max_runtime_seconds else None,
|
int(max_runtime_seconds) if max_runtime_seconds else None,
|
||||||
json.dumps(skills_list) if skills_list is not None else None,
|
json.dumps(skills_list) if skills_list is not None else None,
|
||||||
int(max_retries) if max_retries is not None else None,
|
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
for pid in parents:
|
for pid in parents:
|
||||||
@@ -1408,7 +1380,7 @@ def assign_task(conn: sqlite3.Connection, task_id: str, profile: Optional[str])
|
|||||||
profile = _canonical_assignee(profile)
|
profile = _canonical_assignee(profile)
|
||||||
with write_txn(conn):
|
with write_txn(conn):
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT status, claim_lock, assignee FROM tasks WHERE id = ?", (task_id,)
|
"SELECT status, claim_lock FROM tasks WHERE id = ?", (task_id,)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if not row:
|
if not row:
|
||||||
return False
|
return False
|
||||||
@@ -1417,17 +1389,7 @@ def assign_task(conn: sqlite3.Connection, task_id: str, profile: Optional[str])
|
|||||||
f"cannot reassign {task_id}: currently running (claimed). "
|
f"cannot reassign {task_id}: currently running (claimed). "
|
||||||
"Wait for completion or reclaim the stale lock first."
|
"Wait for completion or reclaim the stale lock first."
|
||||||
)
|
)
|
||||||
if row["assignee"] != profile:
|
conn.execute("UPDATE tasks SET assignee = ? WHERE id = ?", (profile, task_id))
|
||||||
# The retry guard is scoped to the task/profile combination. A
|
|
||||||
# human reassigning the task is an explicit recovery action, so the
|
|
||||||
# new profile should not inherit the previous profile's streak.
|
|
||||||
conn.execute(
|
|
||||||
"UPDATE tasks SET assignee = ?, consecutive_failures = 0, "
|
|
||||||
"last_failure_error = NULL WHERE id = ?",
|
|
||||||
(profile, task_id),
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
conn.execute("UPDATE tasks SET assignee = ? WHERE id = ?", (profile, task_id))
|
|
||||||
_append_event(conn, task_id, "assigned", {"assignee": profile})
|
_append_event(conn, task_id, "assigned", {"assignee": profile})
|
||||||
return True
|
return True
|
||||||
|
|
||||||
@@ -1897,47 +1859,34 @@ def heartbeat_claim(
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
def release_stale_claims(
|
def release_stale_claims(conn: sqlite3.Connection) -> int:
|
||||||
conn: sqlite3.Connection,
|
|
||||||
*,
|
|
||||||
signal_fn=None,
|
|
||||||
) -> int:
|
|
||||||
"""Reset any ``running`` task whose claim has expired.
|
"""Reset any ``running`` task whose claim has expired.
|
||||||
|
|
||||||
Returns the number of stale claims reclaimed. Safe to call often.
|
Returns the number of stale claims reclaimed. Safe to call often.
|
||||||
"""
|
"""
|
||||||
now = int(time.time())
|
now = int(time.time())
|
||||||
reclaimed = 0
|
reclaimed = 0
|
||||||
stale = conn.execute(
|
with write_txn(conn):
|
||||||
"SELECT id, claim_lock, worker_pid FROM tasks "
|
stale = conn.execute(
|
||||||
"WHERE status = 'running' AND claim_expires IS NOT NULL AND claim_expires < ?",
|
"SELECT id, claim_lock FROM tasks "
|
||||||
(now,),
|
"WHERE status = 'running' AND claim_expires IS NOT NULL AND claim_expires < ?",
|
||||||
).fetchall()
|
(now,),
|
||||||
for row in stale:
|
).fetchall()
|
||||||
termination = _terminate_reclaimed_worker(
|
for row in stale:
|
||||||
row["worker_pid"], row["claim_lock"], signal_fn=signal_fn,
|
conn.execute(
|
||||||
)
|
|
||||||
with write_txn(conn):
|
|
||||||
cur = conn.execute(
|
|
||||||
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
||||||
"claim_expires = NULL, worker_pid = NULL "
|
"claim_expires = NULL, worker_pid = NULL "
|
||||||
"WHERE id = ? AND status = 'running' AND claim_lock IS ? "
|
"WHERE id = ? AND status = 'running'",
|
||||||
"AND claim_expires IS NOT NULL AND claim_expires < ?",
|
(row["id"],),
|
||||||
(row["id"], row["claim_lock"], now),
|
|
||||||
)
|
)
|
||||||
if cur.rowcount != 1:
|
|
||||||
continue
|
|
||||||
run_id = _end_run(
|
run_id = _end_run(
|
||||||
conn, row["id"],
|
conn, row["id"],
|
||||||
outcome="reclaimed", status="reclaimed",
|
outcome="reclaimed", status="reclaimed",
|
||||||
error=f"stale_lock={row['claim_lock']}",
|
error=f"stale_lock={row['claim_lock']}",
|
||||||
metadata=termination,
|
|
||||||
)
|
)
|
||||||
payload = {"stale_lock": row["claim_lock"]}
|
|
||||||
payload.update(termination)
|
|
||||||
_append_event(
|
_append_event(
|
||||||
conn, row["id"], "reclaimed",
|
conn, row["id"], "reclaimed",
|
||||||
payload,
|
{"stale_lock": row["claim_lock"]},
|
||||||
run_id=run_id,
|
run_id=run_id,
|
||||||
)
|
)
|
||||||
reclaimed += 1
|
reclaimed += 1
|
||||||
@@ -1949,7 +1898,6 @@ def reclaim_task(
|
|||||||
task_id: str,
|
task_id: str,
|
||||||
*,
|
*,
|
||||||
reason: Optional[str] = None,
|
reason: Optional[str] = None,
|
||||||
signal_fn=None,
|
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""Operator-driven reclaim: release the claim and reset to ``ready``.
|
"""Operator-driven reclaim: release the claim and reset to ``ready``.
|
||||||
|
|
||||||
@@ -1962,29 +1910,24 @@ def reclaim_task(
|
|||||||
Returns True if a reclaim happened, False if the task isn't in a
|
Returns True if a reclaim happened, False if the task isn't in a
|
||||||
reclaimable state (not running, or doesn't exist).
|
reclaimable state (not running, or doesn't exist).
|
||||||
"""
|
"""
|
||||||
row = conn.execute(
|
|
||||||
"SELECT status, claim_lock, worker_pid FROM tasks WHERE id = ?",
|
|
||||||
(task_id,),
|
|
||||||
).fetchone()
|
|
||||||
if not row:
|
|
||||||
return False
|
|
||||||
if row["status"] != "running" and row["claim_lock"] is None:
|
|
||||||
# Nothing to reclaim — already ready / blocked / done.
|
|
||||||
return False
|
|
||||||
prev_lock = row["claim_lock"]
|
|
||||||
termination = _terminate_reclaimed_worker(
|
|
||||||
row["worker_pid"], prev_lock, signal_fn=signal_fn,
|
|
||||||
)
|
|
||||||
with write_txn(conn):
|
with write_txn(conn):
|
||||||
cur = conn.execute(
|
row = conn.execute(
|
||||||
|
"SELECT status, claim_lock, worker_pid FROM tasks WHERE id = ?",
|
||||||
|
(task_id,),
|
||||||
|
).fetchone()
|
||||||
|
if not row:
|
||||||
|
return False
|
||||||
|
if row["status"] != "running" and row["claim_lock"] is None:
|
||||||
|
# Nothing to reclaim — already ready / blocked / done.
|
||||||
|
return False
|
||||||
|
prev_lock = row["claim_lock"]
|
||||||
|
prev_pid = row["worker_pid"]
|
||||||
|
conn.execute(
|
||||||
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
||||||
"claim_expires = NULL, worker_pid = NULL "
|
"claim_expires = NULL, worker_pid = NULL "
|
||||||
"WHERE id = ? AND status IN ('running', 'ready', 'blocked') "
|
"WHERE id = ? AND status IN ('running', 'ready', 'blocked')",
|
||||||
"AND claim_lock IS ?",
|
(task_id,),
|
||||||
(task_id, prev_lock),
|
|
||||||
)
|
)
|
||||||
if cur.rowcount != 1:
|
|
||||||
return False
|
|
||||||
run_id = _end_run(
|
run_id = _end_run(
|
||||||
conn, task_id,
|
conn, task_id,
|
||||||
outcome="reclaimed", status="reclaimed",
|
outcome="reclaimed", status="reclaimed",
|
||||||
@@ -1992,17 +1935,15 @@ def reclaim_task(
|
|||||||
f"manual_reclaim: {reason}" if reason
|
f"manual_reclaim: {reason}" if reason
|
||||||
else f"manual_reclaim lock={prev_lock}"
|
else f"manual_reclaim lock={prev_lock}"
|
||||||
),
|
),
|
||||||
metadata=termination,
|
|
||||||
)
|
)
|
||||||
payload = {
|
|
||||||
"manual": True,
|
|
||||||
"reason": reason,
|
|
||||||
"prev_lock": prev_lock,
|
|
||||||
}
|
|
||||||
payload.update(termination)
|
|
||||||
_append_event(
|
_append_event(
|
||||||
conn, task_id, "reclaimed",
|
conn, task_id, "reclaimed",
|
||||||
payload,
|
{
|
||||||
|
"manual": True,
|
||||||
|
"reason": reason,
|
||||||
|
"prev_lock": prev_lock,
|
||||||
|
"prev_pid": prev_pid,
|
||||||
|
},
|
||||||
run_id=run_id,
|
run_id=run_id,
|
||||||
)
|
)
|
||||||
# Operator intervention — they've looked at the task, so the
|
# Operator intervention — they've looked at the task, so the
|
||||||
@@ -2503,91 +2444,6 @@ def unblock_task(conn: sqlite3.Connection, task_id: str) -> bool:
|
|||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
def specify_triage_task(
|
|
||||||
conn: sqlite3.Connection,
|
|
||||||
task_id: str,
|
|
||||||
*,
|
|
||||||
title: Optional[str] = None,
|
|
||||||
body: Optional[str] = None,
|
|
||||||
author: Optional[str] = None,
|
|
||||||
) -> bool:
|
|
||||||
"""Flesh out a triage task and promote it to ``todo``.
|
|
||||||
|
|
||||||
Atomically updates ``title`` / ``body`` (when provided) and transitions
|
|
||||||
``status: triage -> todo`` in a single write txn. Returns False when
|
|
||||||
the task is missing or not in the ``triage`` column — callers should
|
|
||||||
surface that as "nothing to specify" rather than an error.
|
|
||||||
|
|
||||||
``todo`` (not ``ready``) is the correct landing column: ``recompute_ready``
|
|
||||||
promotes parent-free / parent-done todos to ``ready`` on the next
|
|
||||||
dispatcher tick, which keeps the normal parent-gating behaviour intact
|
|
||||||
for specified tasks that happen to have open parents.
|
|
||||||
|
|
||||||
``author`` is recorded on an audit comment only when at least one of
|
|
||||||
``title`` / ``body`` actually changed — avoids noisy comment spam for
|
|
||||||
status-only promotions.
|
|
||||||
"""
|
|
||||||
if title is not None and not title.strip():
|
|
||||||
raise ValueError("title cannot be blank")
|
|
||||||
with write_txn(conn):
|
|
||||||
existing = conn.execute(
|
|
||||||
"SELECT title, body FROM tasks WHERE id = ? AND status = 'triage'",
|
|
||||||
(task_id,),
|
|
||||||
).fetchone()
|
|
||||||
if existing is None:
|
|
||||||
return False
|
|
||||||
sets: list[str] = ["status = 'todo'"]
|
|
||||||
params: list[Any] = []
|
|
||||||
changed_fields: list[str] = []
|
|
||||||
if title is not None and title.strip() != (existing["title"] or ""):
|
|
||||||
sets.append("title = ?")
|
|
||||||
params.append(title.strip())
|
|
||||||
changed_fields.append("title")
|
|
||||||
if body is not None and (body or "") != (existing["body"] or ""):
|
|
||||||
sets.append("body = ?")
|
|
||||||
params.append(body)
|
|
||||||
changed_fields.append("body")
|
|
||||||
params.append(task_id)
|
|
||||||
cur = conn.execute(
|
|
||||||
f"UPDATE tasks SET {', '.join(sets)} "
|
|
||||||
f"WHERE id = ? AND status = 'triage'",
|
|
||||||
tuple(params),
|
|
||||||
)
|
|
||||||
if cur.rowcount != 1:
|
|
||||||
return False
|
|
||||||
if changed_fields and author and author.strip():
|
|
||||||
# Inline INSERT (rather than ``add_comment``) because we're
|
|
||||||
# already inside this function's write_txn — nested BEGIN
|
|
||||||
# IMMEDIATE would raise OperationalError. We also skip the
|
|
||||||
# 'commented' event that ``add_comment`` emits, since the
|
|
||||||
# 'specified' event below already records the change.
|
|
||||||
conn.execute(
|
|
||||||
"INSERT INTO task_comments (task_id, author, body, created_at) "
|
|
||||||
"VALUES (?, ?, ?, ?)",
|
|
||||||
(
|
|
||||||
task_id,
|
|
||||||
author.strip(),
|
|
||||||
"Specified — updated "
|
|
||||||
+ ", ".join(changed_fields)
|
|
||||||
+ " and promoted to todo.",
|
|
||||||
int(time.time()),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
_append_event(
|
|
||||||
conn,
|
|
||||||
task_id,
|
|
||||||
"specified",
|
|
||||||
{"changed_fields": changed_fields} if changed_fields else None,
|
|
||||||
)
|
|
||||||
# Outside the write_txn above, so we don't nest BEGIN IMMEDIATE — the
|
|
||||||
# ready-promotion pass opens its own IMMEDIATE txn. This runs the same
|
|
||||||
# logic the dispatcher would on its next tick, so a specified task
|
|
||||||
# with no open parents flips straight to 'ready' here instead of
|
|
||||||
# idling in 'todo' until the next sweep.
|
|
||||||
recompute_ready(conn)
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def archive_task(conn: sqlite3.Connection, task_id: str) -> bool:
|
def archive_task(conn: sqlite3.Connection, task_id: str) -> bool:
|
||||||
with write_txn(conn):
|
with write_txn(conn):
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
@@ -2692,11 +2548,11 @@ def set_workspace_path(
|
|||||||
# Dispatcher (one-shot pass)
|
# Dispatcher (one-shot pass)
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
# After this many consecutive non-success attempts on a task/profile, the
|
# After this many consecutive `spawn_failed` events on a task, the dispatcher
|
||||||
# dispatcher stops retrying and parks the task in ``blocked`` with a reason so
|
# stops retrying and parks the task in ``blocked`` with a reason so a human
|
||||||
# a human can investigate. Prevents retry storms when a worker repeatedly times
|
# can investigate. Prevents the dispatcher from thrashing forever on a task
|
||||||
# out, crashes, or cannot spawn.
|
# whose profile doesn't exist, whose workspace is unmountable, etc.
|
||||||
DEFAULT_FAILURE_LIMIT = 2
|
DEFAULT_FAILURE_LIMIT = 5
|
||||||
# Legacy alias — callers / tests still reference the old name.
|
# Legacy alias — callers / tests still reference the old name.
|
||||||
DEFAULT_SPAWN_FAILURE_LIMIT = DEFAULT_FAILURE_LIMIT
|
DEFAULT_SPAWN_FAILURE_LIMIT = DEFAULT_FAILURE_LIMIT
|
||||||
|
|
||||||
@@ -2731,77 +2587,6 @@ class DispatchResult:
|
|||||||
"""Task ids whose workers exceeded ``max_runtime_seconds``."""
|
"""Task ids whose workers exceeded ``max_runtime_seconds``."""
|
||||||
|
|
||||||
|
|
||||||
# Bounded registry of recently-reaped worker child exits, populated by the
|
|
||||||
# reap loop at the top of ``dispatch_once`` and consulted by
|
|
||||||
# ``detect_crashed_workers`` to classify a dead-pid task.
|
|
||||||
#
|
|
||||||
# Entry: ``pid -> (raw_wait_status, reaped_at_epoch)``. We keep raw status
|
|
||||||
# so both ``os.WIFEXITED`` / ``os.WEXITSTATUS`` and ``os.WIFSIGNALED`` can
|
|
||||||
# be consulted. Entries are trimmed by age (and total size cap as a
|
|
||||||
# belt-and-braces against unbounded growth on exotic platforms).
|
|
||||||
_RECENT_WORKER_EXIT_TTL_SECONDS = 600
|
|
||||||
_RECENT_WORKER_EXITS_MAX = 4096
|
|
||||||
_recent_worker_exits: "dict[int, tuple[int, float]]" = {}
|
|
||||||
|
|
||||||
|
|
||||||
def _record_worker_exit(pid: int, raw_status: int) -> None:
|
|
||||||
"""Record a reaped child's exit status for later classification.
|
|
||||||
|
|
||||||
Called from the reap loop in ``dispatch_once``. Safe to call many
|
|
||||||
times; duplicate pids overwrite (pids can cycle, latest wins).
|
|
||||||
"""
|
|
||||||
if not pid or pid <= 0:
|
|
||||||
return
|
|
||||||
now = time.time()
|
|
||||||
_recent_worker_exits[int(pid)] = (int(raw_status), now)
|
|
||||||
# Age-based trim: drop entries older than the TTL.
|
|
||||||
if len(_recent_worker_exits) > _RECENT_WORKER_EXITS_MAX // 2:
|
|
||||||
cutoff = now - _RECENT_WORKER_EXIT_TTL_SECONDS
|
|
||||||
for _pid in [p for p, (_s, t) in _recent_worker_exits.items() if t < cutoff]:
|
|
||||||
_recent_worker_exits.pop(_pid, None)
|
|
||||||
# Size cap as a final guard.
|
|
||||||
if len(_recent_worker_exits) > _RECENT_WORKER_EXITS_MAX:
|
|
||||||
# Drop oldest half.
|
|
||||||
ordered = sorted(_recent_worker_exits.items(), key=lambda kv: kv[1][1])
|
|
||||||
for _pid, _ in ordered[: len(ordered) // 2]:
|
|
||||||
_recent_worker_exits.pop(_pid, None)
|
|
||||||
|
|
||||||
|
|
||||||
def _classify_worker_exit(pid: int) -> "tuple[str, Optional[int]]":
|
|
||||||
"""Classify a recently-reaped worker by pid.
|
|
||||||
|
|
||||||
Returns ``(kind, code)`` where ``kind`` is one of:
|
|
||||||
|
|
||||||
* ``"clean_exit"`` — ``WIFEXITED`` with ``WEXITSTATUS == 0``. When the
|
|
||||||
task is still ``running`` in the DB, this is a protocol violation
|
|
||||||
(worker exited without calling ``kanban_complete`` / ``kanban_block``)
|
|
||||||
and should be auto-blocked immediately — retrying will just loop.
|
|
||||||
* ``"nonzero_exit"`` — ``WIFEXITED`` with non-zero status. Real error.
|
|
||||||
* ``"signaled"`` — ``WIFSIGNALED`` (OOM killer, SIGKILL, etc). Real crash.
|
|
||||||
* ``"unknown"`` — pid was not in the reap registry (either reaped by
|
|
||||||
something else, or died between reap tick and liveness check). Fall
|
|
||||||
back to existing crashed-counter behavior.
|
|
||||||
|
|
||||||
``code`` is the exit status (for ``clean_exit`` / ``nonzero_exit``) or
|
|
||||||
the signal number (for ``signaled``), or ``None`` for ``unknown``.
|
|
||||||
"""
|
|
||||||
entry = _recent_worker_exits.get(int(pid))
|
|
||||||
if entry is None:
|
|
||||||
return ("unknown", None)
|
|
||||||
raw, _ = entry
|
|
||||||
try:
|
|
||||||
if os.WIFEXITED(raw):
|
|
||||||
code = os.WEXITSTATUS(raw)
|
|
||||||
if code == 0:
|
|
||||||
return ("clean_exit", 0)
|
|
||||||
return ("nonzero_exit", code)
|
|
||||||
if os.WIFSIGNALED(raw):
|
|
||||||
return ("signaled", os.WTERMSIG(raw))
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return ("unknown", None)
|
|
||||||
|
|
||||||
|
|
||||||
def _pid_alive(pid: Optional[int]) -> bool:
|
def _pid_alive(pid: Optional[int]) -> bool:
|
||||||
"""Return True if ``pid`` is still running on this host.
|
"""Return True if ``pid`` is still running on this host.
|
||||||
|
|
||||||
@@ -2835,7 +2620,7 @@ def _pid_alive(pid: Optional[int]) -> bool:
|
|||||||
# where we have a cheap, deterministic process-state probe.
|
# where we have a cheap, deterministic process-state probe.
|
||||||
if sys.platform == "linux":
|
if sys.platform == "linux":
|
||||||
try:
|
try:
|
||||||
with open(f"/proc/{int(pid)}/status", "r", encoding="utf-8") as f:
|
with open(f"/proc/{int(pid)}/status", "r") as f:
|
||||||
for line in f:
|
for line in f:
|
||||||
if line.startswith("State:"):
|
if line.startswith("State:"):
|
||||||
# "State:\tZ (zombie)" → dead
|
# "State:\tZ (zombie)" → dead
|
||||||
@@ -2867,62 +2652,6 @@ def _pid_alive(pid: Optional[int]) -> bool:
|
|||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
def _terminate_reclaimed_worker(
|
|
||||||
pid: Optional[int],
|
|
||||||
claim_lock: Optional[str],
|
|
||||||
*,
|
|
||||||
signal_fn=None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Best-effort host-local worker termination for reclaim paths."""
|
|
||||||
import signal
|
|
||||||
|
|
||||||
info: dict[str, Any] = {
|
|
||||||
"prev_pid": int(pid) if pid else None,
|
|
||||||
"host_local": False,
|
|
||||||
"termination_attempted": False,
|
|
||||||
"terminated": False,
|
|
||||||
"sigkill": False,
|
|
||||||
}
|
|
||||||
if not pid or pid <= 0 or not claim_lock:
|
|
||||||
return info
|
|
||||||
|
|
||||||
host_prefix = f"{_claimer_id().split(':', 1)[0]}:"
|
|
||||||
if not str(claim_lock).startswith(host_prefix):
|
|
||||||
return info
|
|
||||||
info["host_local"] = True
|
|
||||||
|
|
||||||
kill = signal_fn if signal_fn is not None else (
|
|
||||||
os.kill if hasattr(os, "kill") else None
|
|
||||||
)
|
|
||||||
if kill is None:
|
|
||||||
return info
|
|
||||||
|
|
||||||
info["termination_attempted"] = True
|
|
||||||
try:
|
|
||||||
kill(int(pid), signal.SIGTERM)
|
|
||||||
except (ProcessLookupError, OSError):
|
|
||||||
return info
|
|
||||||
|
|
||||||
for _ in range(10):
|
|
||||||
if not _pid_alive(pid):
|
|
||||||
info["terminated"] = True
|
|
||||||
return info
|
|
||||||
time.sleep(0.5)
|
|
||||||
|
|
||||||
if _pid_alive(pid):
|
|
||||||
try:
|
|
||||||
# signal.SIGKILL doesn't exist on Windows; fall back to SIGTERM
|
|
||||||
# (which maps to TerminateProcess via the stdlib shim).
|
|
||||||
_sigkill = getattr(signal, "SIGKILL", signal.SIGTERM)
|
|
||||||
kill(int(pid), _sigkill)
|
|
||||||
info["sigkill"] = True
|
|
||||||
except (ProcessLookupError, OSError):
|
|
||||||
return info
|
|
||||||
|
|
||||||
info["terminated"] = not _pid_alive(pid)
|
|
||||||
return info
|
|
||||||
|
|
||||||
|
|
||||||
def heartbeat_worker(
|
def heartbeat_worker(
|
||||||
conn: sqlite3.Connection,
|
conn: sqlite3.Connection,
|
||||||
task_id: str,
|
task_id: str,
|
||||||
@@ -3038,9 +2767,7 @@ def enforce_max_runtime(
|
|||||||
time.sleep(0.5)
|
time.sleep(0.5)
|
||||||
if _pid_alive(pid):
|
if _pid_alive(pid):
|
||||||
try:
|
try:
|
||||||
# signal.SIGKILL doesn't exist on Windows.
|
kill(pid, signal.SIGKILL)
|
||||||
_sigkill = getattr(signal, "SIGKILL", signal.SIGTERM)
|
|
||||||
kill(pid, _sigkill)
|
|
||||||
killed = True
|
killed = True
|
||||||
except (ProcessLookupError, OSError):
|
except (ProcessLookupError, OSError):
|
||||||
pass
|
pass
|
||||||
@@ -3113,22 +2840,12 @@ def detect_crashed_workers(conn: sqlite3.Connection) -> list[str]:
|
|||||||
are meaningless here. The host-local check is enough because
|
are meaningless here. The host-local check is enough because
|
||||||
``_default_spawn`` always runs the worker on the same host as the
|
``_default_spawn`` always runs the worker on the same host as the
|
||||||
dispatcher (the whole design is single-host).
|
dispatcher (the whole design is single-host).
|
||||||
|
|
||||||
When the reap registry shows the worker exited cleanly (rc=0) but
|
|
||||||
the task was still ``running`` in the DB, treat it as a protocol
|
|
||||||
violation (worker answered conversationally without calling
|
|
||||||
``kanban_complete`` / ``kanban_block``) and trip the circuit breaker
|
|
||||||
on the first occurrence — retrying a worker whose CLI keeps
|
|
||||||
returning 0 without a terminal transition just loops forever.
|
|
||||||
"""
|
"""
|
||||||
crashed: list[str] = []
|
crashed: list[str] = []
|
||||||
# Per-crash details collected inside the main txn, used after it
|
# Per-crash details collected inside the main txn, used after it
|
||||||
# closes to run ``_record_task_failure`` (which needs its own
|
# closes to run ``_record_task_failure`` (which needs its own
|
||||||
# write_txn so can't nest). ``protocol_violation`` flags the
|
# write_txn so can't nest).
|
||||||
# clean-exit-but-still-running case so we can trip the breaker
|
crash_details: list[tuple[str, int, str]] = [] # (task_id, pid, claimer)
|
||||||
# immediately instead of incrementing by 1.
|
|
||||||
crash_details: list[tuple[str, int, str, bool, str]] = []
|
|
||||||
# (task_id, pid, claimer, protocol_violation, error_text)
|
|
||||||
with write_txn(conn):
|
with write_txn(conn):
|
||||||
rows = conn.execute(
|
rows = conn.execute(
|
||||||
"SELECT id, worker_pid, claim_lock FROM tasks "
|
"SELECT id, worker_pid, claim_lock FROM tasks "
|
||||||
@@ -3142,39 +2859,6 @@ def detect_crashed_workers(conn: sqlite3.Connection) -> list[str]:
|
|||||||
continue
|
continue
|
||||||
if _pid_alive(row["worker_pid"]):
|
if _pid_alive(row["worker_pid"]):
|
||||||
continue
|
continue
|
||||||
|
|
||||||
pid = int(row["worker_pid"])
|
|
||||||
kind, code = _classify_worker_exit(pid)
|
|
||||||
if kind == "clean_exit":
|
|
||||||
# Worker subprocess returned 0 but its task is still
|
|
||||||
# ``running`` in the DB — it exited without calling
|
|
||||||
# ``kanban_complete`` / ``kanban_block``. Retrying won't
|
|
||||||
# help.
|
|
||||||
protocol_violation = True
|
|
||||||
error_text = (
|
|
||||||
"worker exited cleanly (rc=0) without calling "
|
|
||||||
"kanban_complete or kanban_block — protocol violation"
|
|
||||||
)
|
|
||||||
event_kind = "protocol_violation"
|
|
||||||
event_payload = {
|
|
||||||
"pid": pid,
|
|
||||||
"claimer": row["claim_lock"],
|
|
||||||
"exit_code": code,
|
|
||||||
}
|
|
||||||
else:
|
|
||||||
protocol_violation = False
|
|
||||||
if kind == "nonzero_exit":
|
|
||||||
error_text = f"pid {pid} exited with code {code}"
|
|
||||||
elif kind == "signaled":
|
|
||||||
error_text = f"pid {pid} killed by signal {code}"
|
|
||||||
else:
|
|
||||||
error_text = f"pid {pid} not alive"
|
|
||||||
event_kind = "crashed"
|
|
||||||
event_payload = {"pid": pid, "claimer": row["claim_lock"]}
|
|
||||||
if code is not None and kind != "unknown":
|
|
||||||
event_payload["exit_kind"] = kind
|
|
||||||
event_payload["exit_code"] = code
|
|
||||||
|
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
||||||
"claim_expires = NULL, worker_pid = NULL "
|
"claim_expires = NULL, worker_pid = NULL "
|
||||||
@@ -3185,47 +2869,34 @@ def detect_crashed_workers(conn: sqlite3.Connection) -> list[str]:
|
|||||||
run_id = _end_run(
|
run_id = _end_run(
|
||||||
conn, row["id"],
|
conn, row["id"],
|
||||||
outcome="crashed", status="crashed",
|
outcome="crashed", status="crashed",
|
||||||
error=error_text,
|
error=f"pid {int(row['worker_pid'])} not alive",
|
||||||
metadata=dict(event_payload),
|
metadata={
|
||||||
|
"pid": int(row["worker_pid"]),
|
||||||
|
"claimer": row["claim_lock"],
|
||||||
|
},
|
||||||
)
|
)
|
||||||
_append_event(
|
_append_event(
|
||||||
conn, row["id"], event_kind,
|
conn, row["id"], "crashed",
|
||||||
event_payload,
|
{"pid": int(row["worker_pid"]), "claimer": row["claim_lock"]},
|
||||||
run_id=run_id,
|
run_id=run_id,
|
||||||
)
|
)
|
||||||
crashed.append(row["id"])
|
crashed.append(row["id"])
|
||||||
crash_details.append(
|
crash_details.append(
|
||||||
(row["id"], pid, row["claim_lock"],
|
(row["id"], int(row["worker_pid"]), row["claim_lock"])
|
||||||
protocol_violation, error_text)
|
|
||||||
)
|
)
|
||||||
# Outside the main txn: increment the unified failure counter for
|
# Outside the main txn: increment the unified failure counter for
|
||||||
# each crashed task. If the breaker trips, the task transitions
|
# each crashed task. If the breaker trips, the task transitions
|
||||||
# ready → blocked with a ``gave_up`` event on top of the ``crashed``
|
# ready → blocked with a ``gave_up`` event on top of the ``crashed``
|
||||||
# event we already emitted.
|
# event we already emitted.
|
||||||
#
|
for tid, pid, claimer in crash_details:
|
||||||
# Protocol-violation crashes force an immediate trip (failure_limit=1)
|
_record_task_failure(
|
||||||
# because clean-exit-without-transition is deterministic: the next
|
|
||||||
# respawn will do exactly the same thing. Better to surface to a
|
|
||||||
# human with a clear reason than to loop ``DEFAULT_FAILURE_LIMIT``
|
|
||||||
# times first.
|
|
||||||
auto_blocked: list[str] = []
|
|
||||||
for tid, pid, claimer, protocol_violation, error_text in crash_details:
|
|
||||||
tripped = _record_task_failure(
|
|
||||||
conn, tid,
|
conn, tid,
|
||||||
error=error_text,
|
error=f"pid {pid} not alive",
|
||||||
outcome="crashed",
|
outcome="crashed",
|
||||||
failure_limit=(1 if protocol_violation else None),
|
|
||||||
release_claim=False,
|
release_claim=False,
|
||||||
end_run=False,
|
end_run=False,
|
||||||
event_payload_extra={"pid": pid, "claimer": claimer},
|
event_payload_extra={"pid": pid, "claimer": claimer},
|
||||||
)
|
)
|
||||||
if tripped:
|
|
||||||
auto_blocked.append(tid)
|
|
||||||
# Stash auto-blocked ids on the function for the dispatch loop to pick up.
|
|
||||||
# Keeps the public return type (``list[str]``) stable for direct callers
|
|
||||||
# and tests that destructure the result; ``dispatch_once`` reads this
|
|
||||||
# side-channel attribute to populate ``DispatchResult.auto_blocked``.
|
|
||||||
detect_crashed_workers._last_auto_blocked = auto_blocked # type: ignore[attr-defined]
|
|
||||||
return crashed
|
return crashed
|
||||||
|
|
||||||
|
|
||||||
@@ -3267,39 +2938,20 @@ def _record_task_failure(
|
|||||||
``event_payload_extra`` merges into the ``gave_up`` event payload
|
``event_payload_extra`` merges into the ``gave_up`` event payload
|
||||||
when the breaker trips, so callers can include outcome-specific
|
when the breaker trips, so callers can include outcome-specific
|
||||||
context (e.g. pid on crash, elapsed on timeout).
|
context (e.g. pid on crash, elapsed on timeout).
|
||||||
|
|
||||||
Resolution order for the effective threshold:
|
|
||||||
1. per-task ``max_retries`` if set (nothing else overrides)
|
|
||||||
2. caller-supplied ``failure_limit`` (gateway passes the config
|
|
||||||
value from ``kanban.failure_limit``; tests pass fixed values)
|
|
||||||
3. ``DEFAULT_FAILURE_LIMIT``
|
|
||||||
"""
|
"""
|
||||||
if failure_limit is None:
|
if failure_limit is None:
|
||||||
failure_limit = DEFAULT_FAILURE_LIMIT
|
failure_limit = DEFAULT_FAILURE_LIMIT
|
||||||
blocked = False
|
blocked = False
|
||||||
with write_txn(conn):
|
with write_txn(conn):
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT consecutive_failures, status, max_retries "
|
"SELECT consecutive_failures, status FROM tasks WHERE id = ?", (task_id,),
|
||||||
"FROM tasks WHERE id = ?", (task_id,),
|
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
return False
|
return False
|
||||||
failures = int(row["consecutive_failures"]) + 1
|
failures = int(row["consecutive_failures"]) + 1
|
||||||
cur_status = row["status"]
|
cur_status = row["status"]
|
||||||
|
|
||||||
# Per-task override wins over both caller-supplied and default
|
if failures >= failure_limit:
|
||||||
# thresholds. None (the common case) falls through.
|
|
||||||
task_override = (
|
|
||||||
row["max_retries"] if "max_retries" in row.keys() else None
|
|
||||||
)
|
|
||||||
if task_override is not None:
|
|
||||||
effective_limit = int(task_override)
|
|
||||||
limit_source = "task"
|
|
||||||
else:
|
|
||||||
effective_limit = int(failure_limit)
|
|
||||||
limit_source = "dispatcher"
|
|
||||||
|
|
||||||
if failures >= effective_limit:
|
|
||||||
# Trip the breaker.
|
# Trip the breaker.
|
||||||
if release_claim:
|
if release_claim:
|
||||||
# Spawn path: still running, also clear claim state.
|
# Spawn path: still running, also clear claim state.
|
||||||
@@ -3327,17 +2979,10 @@ def _record_task_failure(
|
|||||||
conn, task_id,
|
conn, task_id,
|
||||||
outcome="gave_up", status="gave_up",
|
outcome="gave_up", status="gave_up",
|
||||||
error=error[:500],
|
error=error[:500],
|
||||||
metadata={
|
metadata={"failures": failures, "trigger_outcome": outcome},
|
||||||
"failures": failures,
|
|
||||||
"trigger_outcome": outcome,
|
|
||||||
"effective_limit": effective_limit,
|
|
||||||
"limit_source": limit_source,
|
|
||||||
},
|
|
||||||
)
|
)
|
||||||
payload = {
|
payload = {
|
||||||
"failures": failures,
|
"failures": failures,
|
||||||
"effective_limit": effective_limit,
|
|
||||||
"limit_source": limit_source,
|
|
||||||
"error": error[:500],
|
"error": error[:500],
|
||||||
"trigger_outcome": outcome,
|
"trigger_outcome": outcome,
|
||||||
}
|
}
|
||||||
@@ -3505,50 +3150,9 @@ def dispatch_once(
|
|||||||
``board`` pins workspace/log/db resolution for this tick to a specific
|
``board`` pins workspace/log/db resolution for this tick to a specific
|
||||||
board. When omitted, the current-board resolution chain is used.
|
board. When omitted, the current-board resolution chain is used.
|
||||||
"""
|
"""
|
||||||
# Reap zombie children from previously spawned workers.
|
|
||||||
# The gateway-embedded dispatcher is the parent of every worker spawned
|
|
||||||
# via _default_spawn (start_new_session=True only detaches the
|
|
||||||
# controlling tty, not the parent). Without an explicit waitpid, each
|
|
||||||
# completed worker becomes a <defunct> entry that lingers until gateway
|
|
||||||
# exit. WNOHANG keeps this non-blocking; ChildProcessError means no
|
|
||||||
# children to reap. Bounded: at most one tick's worth of completions
|
|
||||||
# can be in <defunct> at once.
|
|
||||||
#
|
|
||||||
# We also record the exit status keyed by pid, so
|
|
||||||
# ``detect_crashed_workers`` can distinguish a worker that exited
|
|
||||||
# cleanly without calling ``kanban_complete`` / ``kanban_block``
|
|
||||||
# (protocol violation — auto-block) from a real crash (OOM killer,
|
|
||||||
# SIGKILL, non-zero exit — existing counter behavior).
|
|
||||||
#
|
|
||||||
# Windows has no zombies / no os.WNOHANG — subprocess.Popen handles
|
|
||||||
# are freed when the Python object is garbage-collected or .wait() is
|
|
||||||
# called explicitly. The kanban dispatcher discards the Popen handle
|
|
||||||
# after spawn (``_default_spawn`` → abandon), so on Windows there's
|
|
||||||
# nothing to reap here — skip the whole block.
|
|
||||||
if os.name != "nt":
|
|
||||||
try:
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
_pid, _status = os.waitpid(-1, os.WNOHANG)
|
|
||||||
except ChildProcessError:
|
|
||||||
break
|
|
||||||
if _pid == 0:
|
|
||||||
break
|
|
||||||
_record_worker_exit(_pid, _status)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
result = DispatchResult()
|
result = DispatchResult()
|
||||||
result.reclaimed = release_stale_claims(conn)
|
result.reclaimed = release_stale_claims(conn)
|
||||||
result.crashed = detect_crashed_workers(conn)
|
result.crashed = detect_crashed_workers(conn)
|
||||||
# detect_crashed_workers stashes protocol-violation auto-blocks on
|
|
||||||
# itself so the public list-return stays stable. Pull them into the
|
|
||||||
# DispatchResult here so telemetry / tests see the trip.
|
|
||||||
_crash_auto_blocked = getattr(
|
|
||||||
detect_crashed_workers, "_last_auto_blocked", []
|
|
||||||
)
|
|
||||||
if _crash_auto_blocked:
|
|
||||||
result.auto_blocked.extend(_crash_auto_blocked)
|
|
||||||
result.timed_out = enforce_max_runtime(conn)
|
result.timed_out = enforce_max_runtime(conn)
|
||||||
result.promoted = recompute_ready(conn)
|
result.promoted = recompute_ready(conn)
|
||||||
|
|
||||||
|
|||||||
@@ -1,265 +0,0 @@
|
|||||||
"""Kanban triage specifier — flesh out a one-liner into a real spec.
|
|
||||||
|
|
||||||
Used by ``hermes kanban specify [task_id | --all]``. Takes a task that
|
|
||||||
lives in the Triage column (a rough idea, typically only a title), calls
|
|
||||||
the auxiliary LLM to produce:
|
|
||||||
|
|
||||||
* A tightened title (optional — only replaces if the model proposes a
|
|
||||||
materially different one)
|
|
||||||
* A concrete body: goal, proposed approach, acceptance criteria
|
|
||||||
|
|
||||||
and then flips the task ``triage -> todo`` via
|
|
||||||
``kanban_db.specify_triage_task``. The dispatcher promotes it to
|
|
||||||
``ready`` on its next tick (or immediately if there are no open parents).
|
|
||||||
|
|
||||||
Design notes
|
|
||||||
------------
|
|
||||||
|
|
||||||
* This module intentionally mirrors ``hermes_cli/goals.py`` — same aux
|
|
||||||
client pattern, same "empty config => skip, don't crash" tolerance.
|
|
||||||
Keeps the surface area tiny and the failure modes predictable.
|
|
||||||
|
|
||||||
* The prompt is a short system + user pair. We ask for JSON with
|
|
||||||
``{title, body}``; if parsing fails, we fall back to treating the
|
|
||||||
whole response as the body and leave the title untouched. No
|
|
||||||
retry loop — one shot, keep cost bounded.
|
|
||||||
|
|
||||||
* Structured output / JSON mode is not requested explicitly so the
|
|
||||||
specifier works on providers that don't implement it. The parse
|
|
||||||
is lenient (tolerates markdown code fences around the JSON).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Optional
|
|
||||||
|
|
||||||
from hermes_cli import kanban_db as kb
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
_SYSTEM_PROMPT = """You are the Kanban triage specifier for the Hermes Agent board.
|
|
||||||
A user dropped a rough idea into the Triage column. Your job is to turn it
|
|
||||||
into a concrete, actionable task spec that an autonomous worker can pick up
|
|
||||||
and execute without further clarification.
|
|
||||||
|
|
||||||
Output a single JSON object with exactly two keys:
|
|
||||||
|
|
||||||
{
|
|
||||||
"title": "<tightened task title, <= 80 chars, imperative voice>",
|
|
||||||
"body": "<multi-line spec, see structure below>"
|
|
||||||
}
|
|
||||||
|
|
||||||
The body MUST include these sections, each prefixed with a bold markdown
|
|
||||||
heading, in this order:
|
|
||||||
|
|
||||||
**Goal** — one sentence, user-facing outcome.
|
|
||||||
**Approach** — 2-5 bullets on how a worker should tackle it.
|
|
||||||
**Acceptance criteria** — checklist of concrete, verifiable conditions.
|
|
||||||
**Out of scope** — short list of things NOT to touch (omit if nothing
|
|
||||||
obvious; never invent scope creep).
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
- Keep the tightened title close in meaning to the original idea — do
|
|
||||||
NOT invent a different project.
|
|
||||||
- If the original idea is already detailed, preserve its substance and
|
|
||||||
just reformat into the sections above.
|
|
||||||
- Never add invented requirements the user didn't hint at.
|
|
||||||
- No preamble, no closing remarks, no code fences around the JSON.
|
|
||||||
- Output only the JSON object and nothing else.
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
_USER_TEMPLATE = """Task id: {task_id}
|
|
||||||
Current title: {title}
|
|
||||||
Current body:
|
|
||||||
{body}
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class SpecifyOutcome:
|
|
||||||
"""Result of specifying a single triage task."""
|
|
||||||
|
|
||||||
task_id: str
|
|
||||||
ok: bool
|
|
||||||
reason: str = ""
|
|
||||||
new_title: Optional[str] = None
|
|
||||||
|
|
||||||
|
|
||||||
def _truncate(text: str, limit: int) -> str:
|
|
||||||
if len(text) <= limit:
|
|
||||||
return text
|
|
||||||
return text[: limit - 1] + "…"
|
|
||||||
|
|
||||||
|
|
||||||
_FENCE_RE = re.compile(r"^\s*```(?:json)?\s*|\s*```\s*$", re.IGNORECASE)
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_json_blob(raw: str) -> Optional[dict]:
|
|
||||||
"""Lenient JSON extraction — tolerates fenced code blocks and
|
|
||||||
leading/trailing whitespace. Returns None if nothing parses."""
|
|
||||||
if not raw:
|
|
||||||
return None
|
|
||||||
stripped = _FENCE_RE.sub("", raw.strip())
|
|
||||||
# Greedy: find the first `{` and last `}` and try that slice.
|
|
||||||
first = stripped.find("{")
|
|
||||||
last = stripped.rfind("}")
|
|
||||||
if first == -1 or last == -1 or last <= first:
|
|
||||||
return None
|
|
||||||
candidate = stripped[first : last + 1]
|
|
||||||
try:
|
|
||||||
val = json.loads(candidate)
|
|
||||||
except (ValueError, json.JSONDecodeError):
|
|
||||||
return None
|
|
||||||
if not isinstance(val, dict):
|
|
||||||
return None
|
|
||||||
return val
|
|
||||||
|
|
||||||
|
|
||||||
def _profile_author() -> str:
|
|
||||||
"""Mirror of ``hermes_cli.kanban._profile_author``. Kept local to
|
|
||||||
avoid a circular import when kanban.py imports this module."""
|
|
||||||
return (
|
|
||||||
os.environ.get("HERMES_PROFILE")
|
|
||||||
or os.environ.get("USER")
|
|
||||||
or "specifier"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def specify_task(
|
|
||||||
task_id: str,
|
|
||||||
*,
|
|
||||||
author: Optional[str] = None,
|
|
||||||
timeout: Optional[int] = None,
|
|
||||||
) -> SpecifyOutcome:
|
|
||||||
"""Specify a single triage task and promote it to ``todo``.
|
|
||||||
|
|
||||||
Returns an outcome describing what happened. Never raises for expected
|
|
||||||
failure modes (task not in triage, no aux client configured, API
|
|
||||||
error, malformed response) — those surface via ``ok=False`` so the
|
|
||||||
``--all`` sweep can continue past individual failures.
|
|
||||||
"""
|
|
||||||
with kb.connect() as conn:
|
|
||||||
task = kb.get_task(conn, task_id)
|
|
||||||
if task is None:
|
|
||||||
return SpecifyOutcome(task_id, False, "unknown task id")
|
|
||||||
if task.status != "triage":
|
|
||||||
return SpecifyOutcome(
|
|
||||||
task_id, False, f"task is not in triage (status={task.status!r})"
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
from agent.auxiliary_client import get_text_auxiliary_client
|
|
||||||
except Exception as exc: # pragma: no cover — import smoke test
|
|
||||||
logger.debug("specify: auxiliary client import failed: %s", exc)
|
|
||||||
return SpecifyOutcome(task_id, False, "auxiliary client unavailable")
|
|
||||||
|
|
||||||
try:
|
|
||||||
client, model = get_text_auxiliary_client("triage_specifier")
|
|
||||||
except Exception as exc:
|
|
||||||
logger.debug("specify: get_text_auxiliary_client failed: %s", exc)
|
|
||||||
return SpecifyOutcome(task_id, False, "auxiliary client unavailable")
|
|
||||||
|
|
||||||
if client is None or not model:
|
|
||||||
return SpecifyOutcome(
|
|
||||||
task_id, False, "no auxiliary client configured"
|
|
||||||
)
|
|
||||||
|
|
||||||
user_msg = _USER_TEMPLATE.format(
|
|
||||||
task_id=task.id,
|
|
||||||
title=_truncate(task.title or "", 400),
|
|
||||||
body=_truncate(task.body or "(no body)", 4000),
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
resp = client.chat.completions.create(
|
|
||||||
model=model,
|
|
||||||
messages=[
|
|
||||||
{"role": "system", "content": _SYSTEM_PROMPT},
|
|
||||||
{"role": "user", "content": user_msg},
|
|
||||||
],
|
|
||||||
temperature=0.3,
|
|
||||||
max_tokens=1500,
|
|
||||||
timeout=timeout or 120,
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.info(
|
|
||||||
"specify: API call failed for %s (%s) — skipping",
|
|
||||||
task_id, exc,
|
|
||||||
)
|
|
||||||
return SpecifyOutcome(
|
|
||||||
task_id, False, f"LLM error: {type(exc).__name__}"
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
raw = resp.choices[0].message.content or ""
|
|
||||||
except Exception:
|
|
||||||
raw = ""
|
|
||||||
|
|
||||||
parsed = _extract_json_blob(raw)
|
|
||||||
|
|
||||||
new_title: Optional[str]
|
|
||||||
new_body: Optional[str]
|
|
||||||
if parsed is None:
|
|
||||||
# Fall back: treat the whole reply as the body, leave title as-is.
|
|
||||||
# Worst case the user edits afterward — still better than stranding
|
|
||||||
# the task in triage on a malformed LLM reply.
|
|
||||||
stripped_raw = raw.strip()
|
|
||||||
if not stripped_raw:
|
|
||||||
return SpecifyOutcome(
|
|
||||||
task_id, False, "LLM returned an empty response"
|
|
||||||
)
|
|
||||||
new_title = None
|
|
||||||
new_body = stripped_raw
|
|
||||||
else:
|
|
||||||
title_val = parsed.get("title")
|
|
||||||
body_val = parsed.get("body")
|
|
||||||
new_title = (
|
|
||||||
title_val.strip()
|
|
||||||
if isinstance(title_val, str) and title_val.strip()
|
|
||||||
else None
|
|
||||||
)
|
|
||||||
new_body = (
|
|
||||||
body_val if isinstance(body_val, str) and body_val.strip() else None
|
|
||||||
)
|
|
||||||
if new_body is None and new_title is None:
|
|
||||||
return SpecifyOutcome(
|
|
||||||
task_id, False, "LLM response missing title and body"
|
|
||||||
)
|
|
||||||
|
|
||||||
with kb.connect() as conn:
|
|
||||||
ok = kb.specify_triage_task(
|
|
||||||
conn,
|
|
||||||
task_id,
|
|
||||||
title=new_title,
|
|
||||||
body=new_body,
|
|
||||||
author=author or _profile_author(),
|
|
||||||
)
|
|
||||||
if not ok:
|
|
||||||
# Race: someone else promoted / archived the task between our
|
|
||||||
# read above and the write. Report, don't crash.
|
|
||||||
return SpecifyOutcome(
|
|
||||||
task_id, False, "task moved out of triage before promotion"
|
|
||||||
)
|
|
||||||
return SpecifyOutcome(task_id, True, "specified", new_title=new_title)
|
|
||||||
|
|
||||||
|
|
||||||
def list_triage_ids(*, tenant: Optional[str] = None) -> list[str]:
|
|
||||||
"""Return task ids currently in the triage column.
|
|
||||||
|
|
||||||
``tenant`` narrows the sweep; ``None`` returns every triage task.
|
|
||||||
"""
|
|
||||||
with kb.connect() as conn:
|
|
||||||
tasks = kb.list_tasks(
|
|
||||||
conn,
|
|
||||||
status="triage",
|
|
||||||
tenant=tenant,
|
|
||||||
include_archived=False,
|
|
||||||
)
|
|
||||||
return [t.id for t in tasks]
|
|
||||||
+34
-144
@@ -43,11 +43,6 @@ Usage:
|
|||||||
hermes claw migrate --dry-run # Preview migration without changes
|
hermes claw migrate --dry-run # Preview migration without changes
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# IMPORTANT: hermes_bootstrap must be the very first import — it sets up
|
|
||||||
# UTF-8 stdio on Windows so print()/subprocess children don't hit
|
|
||||||
# UnicodeEncodeError with non-ASCII characters. No-op on POSIX.
|
|
||||||
import hermes_bootstrap # noqa: F401
|
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
@@ -235,7 +230,6 @@ except Exception:
|
|||||||
pass # best-effort — don't crash if config isn't available yet
|
pass # best-effort — don't crash if config isn't available yet
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
import threading
|
|
||||||
import time as _time
|
import time as _time
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
@@ -3540,7 +3534,7 @@ def _remove_custom_provider(config):
|
|||||||
clear_screen=False,
|
clear_screen=False,
|
||||||
title="Select provider to remove:",
|
title="Select provider to remove:",
|
||||||
)
|
)
|
||||||
idx = menu.show()
|
idx: int | None = menu.show() # ty:ignore[invalid-assignment] - TerminalMenu.show() is always `int | None` when multi_select is False / not provided.
|
||||||
from hermes_cli.curses_ui import flush_stdin
|
from hermes_cli.curses_ui import flush_stdin
|
||||||
|
|
||||||
flush_stdin()
|
flush_stdin()
|
||||||
@@ -3626,7 +3620,7 @@ def _model_flow_named_custom(config, provider_info):
|
|||||||
clear_screen=False,
|
clear_screen=False,
|
||||||
title=f"Select model from {name}:",
|
title=f"Select model from {name}:",
|
||||||
)
|
)
|
||||||
idx = menu.show()
|
idx: int | None = menu.show() # ty:ignore[invalid-assignment] - TerminalMenu.show() is always `int | None` when multi_select is False / not provided.
|
||||||
from hermes_cli.curses_ui import flush_stdin
|
from hermes_cli.curses_ui import flush_stdin
|
||||||
|
|
||||||
flush_stdin()
|
flush_stdin()
|
||||||
@@ -3802,7 +3796,7 @@ def _prompt_reasoning_effort_selection(efforts, current_effort=""):
|
|||||||
clear_screen=False,
|
clear_screen=False,
|
||||||
title="Select reasoning effort:",
|
title="Select reasoning effort:",
|
||||||
)
|
)
|
||||||
idx = menu.show()
|
idx: int | None = menu.show() # ty:ignore[invalid-assignment] - TerminalMenu.show() is always `int | None` when multi_select is False / not provided.
|
||||||
from hermes_cli.curses_ui import flush_stdin
|
from hermes_cli.curses_ui import flush_stdin
|
||||||
|
|
||||||
flush_stdin()
|
flush_stdin()
|
||||||
@@ -6451,45 +6445,6 @@ def _load_installable_optional_extras() -> list[str]:
|
|||||||
return referenced
|
return referenced
|
||||||
|
|
||||||
|
|
||||||
def _run_install_with_heartbeat(
|
|
||||||
cmd: list[str],
|
|
||||||
*,
|
|
||||||
env: dict[str, str] | None = None,
|
|
||||||
heartbeat_interval_seconds: int = 30,
|
|
||||||
) -> None:
|
|
||||||
"""Run dependency install command with periodic heartbeat output.
|
|
||||||
|
|
||||||
Some resolvers/build backends (especially when compiling Rust/C extensions)
|
|
||||||
can stay quiet for minutes. Emit a simple elapsed-time heartbeat so users
|
|
||||||
know ``hermes update`` is still progressing even if pip/uv itself is silent.
|
|
||||||
"""
|
|
||||||
done = threading.Event()
|
|
||||||
start = _time.time()
|
|
||||||
|
|
||||||
def _heartbeat() -> None:
|
|
||||||
# Wait first, then print, so short installs don't emit noise.
|
|
||||||
while not done.wait(heartbeat_interval_seconds):
|
|
||||||
elapsed = int(_time.time() - start)
|
|
||||||
print(
|
|
||||||
f" … still installing dependencies ({elapsed}s elapsed)"
|
|
||||||
" — compiling Rust/C extensions can take several minutes",
|
|
||||||
flush=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
t = threading.Thread(target=_heartbeat, daemon=True)
|
|
||||||
t.start()
|
|
||||||
try:
|
|
||||||
subprocess.run(
|
|
||||||
cmd,
|
|
||||||
cwd=PROJECT_ROOT,
|
|
||||||
check=True,
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
done.set()
|
|
||||||
t.join(timeout=0.2)
|
|
||||||
|
|
||||||
|
|
||||||
def _install_python_dependencies_with_optional_fallback(
|
def _install_python_dependencies_with_optional_fallback(
|
||||||
install_cmd_prefix: list[str],
|
install_cmd_prefix: list[str],
|
||||||
*,
|
*,
|
||||||
@@ -6506,13 +6461,12 @@ def _install_python_dependencies_with_optional_fallback(
|
|||||||
Collecting/Building/Installing step), so keeping it visible costs
|
Collecting/Building/Installing step), so keeping it visible costs
|
||||||
nothing on fast hardware and prevents the "hermes update hangs" reports
|
nothing on fast hardware and prevents the "hermes update hangs" reports
|
||||||
on slow hardware.
|
on slow hardware.
|
||||||
|
|
||||||
We also add periodic heartbeat lines in case the resolver/build backend is
|
|
||||||
itself silent for long stretches.
|
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
_run_install_with_heartbeat(
|
subprocess.run(
|
||||||
install_cmd_prefix + ["install", "-e", ".[all]"],
|
install_cmd_prefix + ["install", "-e", ".[all]"],
|
||||||
|
cwd=PROJECT_ROOT,
|
||||||
|
check=True,
|
||||||
env=env,
|
env=env,
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
@@ -6521,8 +6475,10 @@ def _install_python_dependencies_with_optional_fallback(
|
|||||||
" ⚠ Optional extras failed, reinstalling base dependencies and retrying extras individually..."
|
" ⚠ Optional extras failed, reinstalling base dependencies and retrying extras individually..."
|
||||||
)
|
)
|
||||||
|
|
||||||
_run_install_with_heartbeat(
|
subprocess.run(
|
||||||
install_cmd_prefix + ["install", "-e", "."],
|
install_cmd_prefix + ["install", "-e", "."],
|
||||||
|
cwd=PROJECT_ROOT,
|
||||||
|
check=True,
|
||||||
env=env,
|
env=env,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -6530,8 +6486,10 @@ def _install_python_dependencies_with_optional_fallback(
|
|||||||
installed_extras: list[str] = []
|
installed_extras: list[str] = []
|
||||||
for extra in _load_installable_optional_extras():
|
for extra in _load_installable_optional_extras():
|
||||||
try:
|
try:
|
||||||
_run_install_with_heartbeat(
|
subprocess.run(
|
||||||
install_cmd_prefix + ["install", "-e", f".[{extra}]"],
|
install_cmd_prefix + ["install", "-e", f".[{extra}]"],
|
||||||
|
cwd=PROJECT_ROOT,
|
||||||
|
check=True,
|
||||||
env=env,
|
env=env,
|
||||||
)
|
)
|
||||||
installed_extras.append(extra)
|
installed_extras.append(extra)
|
||||||
@@ -7373,9 +7331,7 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
|||||||
for p in all_profiles:
|
for p in all_profiles:
|
||||||
try:
|
try:
|
||||||
r = seed_profile_skills(p.path, quiet=True)
|
r = seed_profile_skills(p.path, quiet=True)
|
||||||
if r and r.get("skipped_opt_out"):
|
if r:
|
||||||
status = "opted out (--no-skills)"
|
|
||||||
elif r:
|
|
||||||
copied = len(r.get("copied", []))
|
copied = len(r.get("copied", []))
|
||||||
updated = len(r.get("updated", []))
|
updated = len(r.get("updated", []))
|
||||||
modified = len(r.get("user_modified", []))
|
modified = len(r.get("user_modified", []))
|
||||||
@@ -7446,8 +7402,11 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
|||||||
.lower()
|
.lower()
|
||||||
)
|
)
|
||||||
elif not (sys.stdin.isatty() and sys.stdout.isatty()):
|
elif not (sys.stdin.isatty() and sys.stdout.isatty()):
|
||||||
print(" ℹ Non-interactive session — applying safe config migrations.")
|
print(" ℹ Non-interactive session — skipping config migration prompt.")
|
||||||
response = "auto"
|
print(
|
||||||
|
" Run 'hermes config migrate' later to apply any new config/env options."
|
||||||
|
)
|
||||||
|
response = "n"
|
||||||
else:
|
else:
|
||||||
try:
|
try:
|
||||||
response = (
|
response = (
|
||||||
@@ -7458,22 +7417,19 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
|||||||
except EOFError:
|
except EOFError:
|
||||||
response = "n"
|
response = "n"
|
||||||
|
|
||||||
if response in ("", "y", "yes", "auto"):
|
if response in ("", "y", "yes"):
|
||||||
print()
|
print()
|
||||||
# Gateway mode, --yes, and non-interactive update contexts
|
# In gateway mode OR under --yes, run auto-migrations only (no
|
||||||
# (dashboard / web server actions) cannot prompt for API keys.
|
# input() prompts for API keys which would hang the detached
|
||||||
# Still run the non-interactive migration pass before restarting
|
# process / defeat the point of --yes).
|
||||||
# so new default config fields and version bumps are written
|
results = migrate_config(
|
||||||
# before the freshly updated gateway validates config at startup.
|
interactive=not (gateway_mode or assume_yes), quiet=False
|
||||||
interactive_migration = not (
|
|
||||||
gateway_mode or assume_yes or response == "auto"
|
|
||||||
)
|
)
|
||||||
results = migrate_config(interactive=interactive_migration, quiet=False)
|
|
||||||
|
|
||||||
if results["env_added"] or results["config_added"]:
|
if results["env_added"] or results["config_added"]:
|
||||||
print()
|
print()
|
||||||
print("✓ Configuration updated!")
|
print("✓ Configuration updated!")
|
||||||
if (gateway_mode or assume_yes or response == "auto") and missing_env:
|
if (gateway_mode or assume_yes) and missing_env:
|
||||||
print(" ℹ API keys require manual entry: hermes config migrate")
|
print(" ℹ API keys require manual entry: hermes config migrate")
|
||||||
else:
|
else:
|
||||||
print()
|
print()
|
||||||
@@ -7626,7 +7582,7 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
|||||||
# systemd units without SIGUSR1 wiring this wait just times out
|
# systemd units without SIGUSR1 wiring this wait just times out
|
||||||
# and we fall back to ``systemctl restart`` (the old behaviour).
|
# and we fall back to ``systemctl restart`` (the old behaviour).
|
||||||
try:
|
try:
|
||||||
from hermes_constants import (
|
from gateway.restart import (
|
||||||
DEFAULT_GATEWAY_RESTART_DRAIN_TIMEOUT as _DEFAULT_DRAIN,
|
DEFAULT_GATEWAY_RESTART_DRAIN_TIMEOUT as _DEFAULT_DRAIN,
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
@@ -7777,23 +7733,6 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
|||||||
# when the graceful path failed (unit missing
|
# when the graceful path failed (unit missing
|
||||||
# SIGUSR1 wiring, drain exceeded the budget,
|
# SIGUSR1 wiring, drain exceeded the budget,
|
||||||
# restart-policy mismatch).
|
# restart-policy mismatch).
|
||||||
#
|
|
||||||
# Always `reset-failed` first. If systemd's own
|
|
||||||
# auto-restart attempts already parked the unit
|
|
||||||
# in a failed state (transient CHDIR / OOM /
|
|
||||||
# filesystem race after our drain + exit-75),
|
|
||||||
# a plain `systemctl restart` can wedge against
|
|
||||||
# the RestartSec backoff and leave the unit
|
|
||||||
# dead. Clearing the failed state first makes
|
|
||||||
# the restart idempotent. Mirrors the recovery
|
|
||||||
# path in `hermes gateway restart`
|
|
||||||
# (`systemd_restart()`) as of PR #20949.
|
|
||||||
subprocess.run(
|
|
||||||
scope_cmd + ["reset-failed", svc_name],
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=10,
|
|
||||||
)
|
|
||||||
restart = subprocess.run(
|
restart = subprocess.run(
|
||||||
scope_cmd + ["restart", svc_name],
|
scope_cmd + ["restart", svc_name],
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
@@ -7813,19 +7752,10 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
|||||||
else:
|
else:
|
||||||
# Retry once — transient startup failures
|
# Retry once — transient startup failures
|
||||||
# (stale module cache, import race) often
|
# (stale module cache, import race) often
|
||||||
# resolve on the second attempt. Again
|
# resolve on the second attempt.
|
||||||
# clear any failed state first so the
|
|
||||||
# retry isn't blocked by the previous
|
|
||||||
# crash.
|
|
||||||
print(
|
print(
|
||||||
f" ⚠ {svc_name} died after restart, retrying..."
|
f" ⚠ {svc_name} died after restart, retrying..."
|
||||||
)
|
)
|
||||||
subprocess.run(
|
|
||||||
scope_cmd + ["reset-failed", svc_name],
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=10,
|
|
||||||
)
|
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
scope_cmd + ["restart", svc_name],
|
scope_cmd + ["restart", svc_name],
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
@@ -7840,13 +7770,10 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
|||||||
restarted_services.append(svc_name)
|
restarted_services.append(svc_name)
|
||||||
print(f" ✓ {svc_name} recovered on retry")
|
print(f" ✓ {svc_name} recovered on retry")
|
||||||
else:
|
else:
|
||||||
_scope_flag = "--user " if scope == "user" else ""
|
|
||||||
print(
|
print(
|
||||||
f" ✗ {svc_name} failed to stay running after restart.\n"
|
f" ✗ {svc_name} failed to stay running after restart.\n"
|
||||||
f" Check logs: journalctl {_scope_flag}-u {svc_name} --since '2 min ago'\n"
|
f" Check logs: journalctl --user -u {svc_name} --since '2 min ago'\n"
|
||||||
f" Recover manually:\n"
|
f" Restart manually: systemctl {'--user ' if scope == 'user' else ''}restart {svc_name}"
|
||||||
f" systemctl {_scope_flag}reset-failed {svc_name}\n"
|
|
||||||
f" systemctl {_scope_flag}restart {svc_name}"
|
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
print(
|
print(
|
||||||
@@ -7970,15 +7897,10 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
|||||||
print(
|
print(
|
||||||
f" ⚠ {len(_stuck)} gateway process(es) ignored SIGTERM — force-killing"
|
f" ⚠ {len(_stuck)} gateway process(es) ignored SIGTERM — force-killing"
|
||||||
)
|
)
|
||||||
from gateway.status import terminate_pid as _terminate_pid
|
|
||||||
for pid in _stuck:
|
for pid in _stuck:
|
||||||
try:
|
try:
|
||||||
# Routes through taskkill /T /F on Windows,
|
os.kill(pid, _signal.SIGKILL)
|
||||||
# SIGKILL on POSIX — _signal.SIGKILL doesn't
|
except (ProcessLookupError, PermissionError):
|
||||||
# exist on Windows so the old raw os.kill call
|
|
||||||
# used to crash the entire update path.
|
|
||||||
_terminate_pid(pid, force=True)
|
|
||||||
except (ProcessLookupError, PermissionError, OSError):
|
|
||||||
pass
|
pass
|
||||||
# Give the OS a beat to reap the processes so the
|
# Give the OS a beat to reap the processes so the
|
||||||
# watchers see them exit and respawn.
|
# watchers see them exit and respawn.
|
||||||
@@ -8202,7 +8124,6 @@ def cmd_profile(args):
|
|||||||
clone = getattr(args, "clone", False)
|
clone = getattr(args, "clone", False)
|
||||||
clone_all = getattr(args, "clone_all", False)
|
clone_all = getattr(args, "clone_all", False)
|
||||||
no_alias = getattr(args, "no_alias", False)
|
no_alias = getattr(args, "no_alias", False)
|
||||||
no_skills = getattr(args, "no_skills", False)
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
clone_from = getattr(args, "clone_from", None)
|
clone_from = getattr(args, "clone_from", None)
|
||||||
@@ -8213,7 +8134,6 @@ def cmd_profile(args):
|
|||||||
clone_all=clone_all,
|
clone_all=clone_all,
|
||||||
clone_config=clone,
|
clone_config=clone,
|
||||||
no_alias=no_alias,
|
no_alias=no_alias,
|
||||||
no_skills=no_skills,
|
|
||||||
)
|
)
|
||||||
print(f"\nProfile '{name}' created at {profile_dir}")
|
print(f"\nProfile '{name}' created at {profile_dir}")
|
||||||
|
|
||||||
@@ -8238,17 +8158,10 @@ def cmd_profile(args):
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass # Honcho plugin not installed or not configured
|
pass # Honcho plugin not installed or not configured
|
||||||
|
|
||||||
# Seed bundled skills (skip if --clone-all already copied them, or
|
# Seed bundled skills (skip if --clone-all already copied them)
|
||||||
# if --no-skills was passed — in which case seed_profile_skills()
|
|
||||||
# honors the marker file and returns skipped_opt_out=True).
|
|
||||||
if not clone_all:
|
if not clone_all:
|
||||||
result = seed_profile_skills(profile_dir)
|
result = seed_profile_skills(profile_dir)
|
||||||
if result and result.get("skipped_opt_out"):
|
if result:
|
||||||
print(
|
|
||||||
"No bundled skills seeded (--no-skills). "
|
|
||||||
"Delete .no-bundled-skills in the profile to opt back in."
|
|
||||||
)
|
|
||||||
elif result:
|
|
||||||
copied = len(result.get("copied", []))
|
copied = len(result.get("copied", []))
|
||||||
print(f"{copied} bundled skills synced.")
|
print(f"{copied} bundled skills synced.")
|
||||||
else:
|
else:
|
||||||
@@ -8564,13 +8477,6 @@ def _build_provider_choices() -> list[str]:
|
|||||||
|
|
||||||
def main():
|
def main():
|
||||||
"""Main entry point for hermes CLI."""
|
"""Main entry point for hermes CLI."""
|
||||||
# Force UTF-8 stdio on Windows before anything prints. No-op elsewhere.
|
|
||||||
try:
|
|
||||||
from hermes_cli.stdio import configure_windows_stdio
|
|
||||||
configure_windows_stdio()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
from hermes_cli._parser import build_top_level_parser
|
from hermes_cli._parser import build_top_level_parser
|
||||||
|
|
||||||
parser, subparsers, chat_parser = build_top_level_parser()
|
parser, subparsers, chat_parser = build_top_level_parser()
|
||||||
@@ -8773,9 +8679,6 @@ def main():
|
|||||||
help="Target the Linux system-level gateway service",
|
help="Target the Linux system-level gateway service",
|
||||||
)
|
)
|
||||||
|
|
||||||
# gateway list
|
|
||||||
gateway_subparsers.add_parser("list", help="List all profiles and their gateway status")
|
|
||||||
|
|
||||||
# gateway setup
|
# gateway setup
|
||||||
gateway_subparsers.add_parser("setup", help="Configure messaging platforms")
|
gateway_subparsers.add_parser("setup", help="Configure messaging platforms")
|
||||||
|
|
||||||
@@ -10093,15 +9996,7 @@ Examples:
|
|||||||
)
|
)
|
||||||
mcp_add_p.add_argument("name", help="Server name (used as config key)")
|
mcp_add_p.add_argument("name", help="Server name (used as config key)")
|
||||||
mcp_add_p.add_argument("--url", help="HTTP/SSE endpoint URL")
|
mcp_add_p.add_argument("--url", help="HTTP/SSE endpoint URL")
|
||||||
# dest="mcp_command" so this flag does not clobber the top-level
|
mcp_add_p.add_argument("--command", help="Stdio command (e.g. npx)")
|
||||||
# subparser's args.command attribute, which the dispatcher reads to
|
|
||||||
# route to cmd_mcp. Without an explicit dest, argparse derives
|
|
||||||
# dest="command" from the flag name and sets it to None when the
|
|
||||||
# flag is omitted, causing `hermes mcp add ...` to fall through to
|
|
||||||
# interactive chat.
|
|
||||||
mcp_add_p.add_argument(
|
|
||||||
"--command", dest="mcp_command", help="Stdio command (e.g. npx)"
|
|
||||||
)
|
|
||||||
mcp_add_p.add_argument(
|
mcp_add_p.add_argument(
|
||||||
"--args", nargs="*", default=[], help="Arguments for stdio command"
|
"--args", nargs="*", default=[], help="Arguments for stdio command"
|
||||||
)
|
)
|
||||||
@@ -10628,11 +10523,6 @@ Examples:
|
|||||||
profile_create.add_argument(
|
profile_create.add_argument(
|
||||||
"--no-alias", action="store_true", help="Skip wrapper script creation"
|
"--no-alias", action="store_true", help="Skip wrapper script creation"
|
||||||
)
|
)
|
||||||
profile_create.add_argument(
|
|
||||||
"--no-skills",
|
|
||||||
action="store_true",
|
|
||||||
help="Create an empty profile with no bundled skills (opts out of `hermes update` skill sync)",
|
|
||||||
)
|
|
||||||
|
|
||||||
profile_delete = profile_subparsers.add_parser("delete", help="Delete a profile")
|
profile_delete = profile_subparsers.add_parser("delete", help="Delete a profile")
|
||||||
profile_delete.add_argument("profile_name", help="Profile to delete")
|
profile_delete.add_argument("profile_name", help="Profile to delete")
|
||||||
|
|||||||
@@ -221,10 +221,7 @@ def cmd_mcp_add(args):
|
|||||||
"""Add a new MCP server with discovery-first tool selection."""
|
"""Add a new MCP server with discovery-first tool selection."""
|
||||||
name = args.name
|
name = args.name
|
||||||
url = getattr(args, "url", None)
|
url = getattr(args, "url", None)
|
||||||
# Read from `mcp_command` (set by --command via explicit dest) — see
|
command = getattr(args, "command", None)
|
||||||
# mcp_add_p.add_argument("--command", dest="mcp_command", ...) in
|
|
||||||
# hermes_cli/main.py for why the dest is renamed.
|
|
||||||
command = getattr(args, "mcp_command", None)
|
|
||||||
cmd_args = getattr(args, "args", None) or []
|
cmd_args = getattr(args, "args", None) or []
|
||||||
auth_type = getattr(args, "auth", None)
|
auth_type = getattr(args, "auth", None)
|
||||||
preset_name = getattr(args, "preset", None)
|
preset_name = getattr(args, "preset", None)
|
||||||
|
|||||||
@@ -69,7 +69,7 @@ def _install_dependencies(provider_name: str) -> None:
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
import yaml
|
import yaml
|
||||||
with open(yaml_path, encoding="utf-8") as f:
|
with open(yaml_path) as f:
|
||||||
meta = yaml.safe_load(f) or {}
|
meta = yaml.safe_load(f) or {}
|
||||||
except Exception:
|
except Exception:
|
||||||
return
|
return
|
||||||
@@ -377,7 +377,7 @@ def _write_env_vars(env_path: Path, env_writes: dict) -> None:
|
|||||||
if key not in updated_keys:
|
if key not in updated_keys:
|
||||||
new_lines.append(f"{key}={val}")
|
new_lines.append(f"{key}={val}")
|
||||||
|
|
||||||
env_path.write_text("\n".join(new_lines) + "\n", encoding="utf-8")
|
env_path.write_text("\n".join(new_lines) + "\n")
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -173,7 +173,7 @@ def _read_disk_cache() -> tuple[dict[str, Any] | None, float]:
|
|||||||
except (OSError, FileNotFoundError):
|
except (OSError, FileNotFoundError):
|
||||||
return (None, 0.0)
|
return (None, 0.0)
|
||||||
try:
|
try:
|
||||||
with open(path, encoding="utf-8") as fh:
|
with open(path) as fh:
|
||||||
data = json.load(fh)
|
data = json.load(fh)
|
||||||
except (OSError, json.JSONDecodeError):
|
except (OSError, json.JSONDecodeError):
|
||||||
return (None, 0.0)
|
return (None, 0.0)
|
||||||
@@ -187,7 +187,7 @@ def _write_disk_cache(data: dict[str, Any]) -> None:
|
|||||||
try:
|
try:
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||||
with open(tmp, "w", encoding="utf-8") as fh:
|
with open(tmp, "w") as fh:
|
||||||
json.dump(data, fh, indent=2)
|
json.dump(data, fh, indent=2)
|
||||||
fh.write("\n")
|
fh.write("\n")
|
||||||
atomic_replace(tmp, path)
|
atomic_replace(tmp, path)
|
||||||
|
|||||||
@@ -1637,8 +1637,7 @@ def list_authenticated_providers(
|
|||||||
groups[group_key]["models"].append(m)
|
groups[group_key]["models"].append(m)
|
||||||
|
|
||||||
_section4_emitted_slugs: set = set()
|
_section4_emitted_slugs: set = set()
|
||||||
for grp_key, grp in groups.items():
|
for grp in groups.values():
|
||||||
api_url, api_key = grp_key
|
|
||||||
slug = grp["slug"]
|
slug = grp["slug"]
|
||||||
# If the slug is already claimed by a built-in / overlay /
|
# If the slug is already claimed by a built-in / overlay /
|
||||||
# user-provider row (sections 1-3), skip this custom group
|
# user-provider row (sections 1-3), skip this custom group
|
||||||
@@ -1676,18 +1675,6 @@ def list_authenticated_providers(
|
|||||||
_grp_url_norm = _pair_key[1]
|
_grp_url_norm = _pair_key[1]
|
||||||
if _grp_url_norm and _grp_url_norm in _builtin_endpoints:
|
if _grp_url_norm and _grp_url_norm in _builtin_endpoints:
|
||||||
continue
|
continue
|
||||||
# Live model discovery from custom provider endpoints (matches
|
|
||||||
# Section 3 behavior for user ``providers:`` entries).
|
|
||||||
if api_url and api_key:
|
|
||||||
try:
|
|
||||||
from hermes_cli.models import fetch_api_models
|
|
||||||
|
|
||||||
live_models = fetch_api_models(api_key, api_url)
|
|
||||||
if live_models:
|
|
||||||
grp["models"] = live_models
|
|
||||||
grp["total_models"] = len(live_models)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
results.append({
|
results.append({
|
||||||
"slug": slug,
|
"slug": slug,
|
||||||
"name": grp["name"],
|
"name": grp["name"],
|
||||||
|
|||||||
@@ -46,7 +46,6 @@ OPENROUTER_MODELS: list[tuple[str, str]] = [
|
|||||||
("xiaomi/mimo-v2.5-pro", ""),
|
("xiaomi/mimo-v2.5-pro", ""),
|
||||||
("xiaomi/mimo-v2.5", ""),
|
("xiaomi/mimo-v2.5", ""),
|
||||||
("tencent/hy3-preview:free", "free"),
|
("tencent/hy3-preview:free", "free"),
|
||||||
("tencent/hy3-preview", ""),
|
|
||||||
("openai/gpt-5.3-codex", ""),
|
("openai/gpt-5.3-codex", ""),
|
||||||
("google/gemini-3-pro-image-preview", ""),
|
("google/gemini-3-pro-image-preview", ""),
|
||||||
("google/gemini-3-flash-preview", ""),
|
("google/gemini-3-flash-preview", ""),
|
||||||
@@ -417,18 +416,6 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
|||||||
"glm-4.7",
|
"glm-4.7",
|
||||||
"MiniMax-M2.5",
|
"MiniMax-M2.5",
|
||||||
],
|
],
|
||||||
# Alibaba Coding Plan — same platform as alibaba (DashScope coding-intl),
|
|
||||||
# separate provider ID with its own base_url_env_var.
|
|
||||||
"alibaba-coding-plan": [
|
|
||||||
"qwen3.6-plus",
|
|
||||||
"qwen3.5-plus",
|
|
||||||
"qwen3-coder-plus",
|
|
||||||
"qwen3-coder-next",
|
|
||||||
"kimi-k2.5",
|
|
||||||
"glm-5",
|
|
||||||
"glm-4.7",
|
|
||||||
"MiniMax-M2.5",
|
|
||||||
],
|
|
||||||
# Curated HF model list — only agentic models that map to OpenRouter defaults.
|
# Curated HF model list — only agentic models that map to OpenRouter defaults.
|
||||||
"huggingface": [
|
"huggingface": [
|
||||||
"moonshotai/Kimi-K2.5",
|
"moonshotai/Kimi-K2.5",
|
||||||
|
|||||||
@@ -174,7 +174,7 @@ def run_oneshot(
|
|||||||
# Redirect stderr AND stdout to devnull for the entire call tree.
|
# Redirect stderr AND stdout to devnull for the entire call tree.
|
||||||
# We'll print the final response to the real stdout at the end.
|
# We'll print the final response to the real stdout at the end.
|
||||||
real_stdout = sys.stdout
|
real_stdout = sys.stdout
|
||||||
devnull = open(os.devnull, "w", encoding="utf-8")
|
devnull = open(os.devnull, "w")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with redirect_stdout(devnull), redirect_stderr(devnull):
|
with redirect_stdout(devnull), redirect_stderr(devnull):
|
||||||
|
|||||||
@@ -73,24 +73,6 @@ def _cmd_approve(store, platform: str, code: str):
|
|||||||
display = f"{name} ({uid})" if name else uid
|
display = f"{name} ({uid})" if name else uid
|
||||||
print(f"\n Approved! User {display} on {platform} can now use the bot~")
|
print(f"\n Approved! User {display} on {platform} can now use the bot~")
|
||||||
print(" They'll be recognized automatically on their next message.\n")
|
print(" They'll be recognized automatically on their next message.\n")
|
||||||
elif store._is_locked_out(platform):
|
|
||||||
# Disambiguate: approve_code returns None for both invalid codes
|
|
||||||
# and lockout. Tell the operator it's lockout so they don't chase
|
|
||||||
# a "wrong code" rabbit hole (#10195).
|
|
||||||
import time as _time
|
|
||||||
limits = store._load_json(store._rate_limit_path())
|
|
||||||
lockout_until = limits.get(f"_lockout:{platform}", 0)
|
|
||||||
remaining = max(0, int(lockout_until - _time.time()))
|
|
||||||
mins = remaining // 60
|
|
||||||
print(
|
|
||||||
f"\n Platform '{platform}' is locked out after too many failed "
|
|
||||||
f"approval attempts."
|
|
||||||
)
|
|
||||||
print(f" Lockout clears in ~{mins} minute(s).")
|
|
||||||
print(
|
|
||||||
" To reset sooner, delete the '_lockout:{0}' entry from "
|
|
||||||
"~/.hermes/platforms/pairing/_rate_limits.json\n".format(platform)
|
|
||||||
)
|
|
||||||
else:
|
else:
|
||||||
print(f"\n Code '{code}' not found or expired for platform '{platform}'.")
|
print(f"\n Code '{code}' not found or expired for platform '{platform}'.")
|
||||||
print(" Run 'hermes pairing list' to see pending codes.\n")
|
print(" Run 'hermes pairing list' to see pending codes.\n")
|
||||||
|
|||||||
@@ -80,10 +80,6 @@ VALID_HOOKS: Set[str] = {
|
|||||||
"post_tool_call",
|
"post_tool_call",
|
||||||
"transform_terminal_output",
|
"transform_terminal_output",
|
||||||
"transform_tool_result",
|
"transform_tool_result",
|
||||||
# Transform LLM output before it's returned to the user.
|
|
||||||
# Plugins return a string to replace the response text, or None/empty to leave unchanged.
|
|
||||||
# First non-None string wins. Useful for vocabulary/personality transformation.
|
|
||||||
"transform_llm_output",
|
|
||||||
"pre_llm_call",
|
"pre_llm_call",
|
||||||
"post_llm_call",
|
"post_llm_call",
|
||||||
"pre_api_request",
|
"pre_api_request",
|
||||||
@@ -870,7 +866,7 @@ class PluginManager:
|
|||||||
if yaml is None:
|
if yaml is None:
|
||||||
logger.warning("PyYAML not installed – cannot load %s", manifest_file)
|
logger.warning("PyYAML not installed – cannot load %s", manifest_file)
|
||||||
return None
|
return None
|
||||||
data = yaml.safe_load(manifest_file.read_text(encoding="utf-8")) or {}
|
data = yaml.safe_load(manifest_file.read_text()) or {}
|
||||||
|
|
||||||
name = data.get("name", plugin_dir.name)
|
name = data.get("name", plugin_dir.name)
|
||||||
key = f"{prefix}/{plugin_dir.name}" if prefix else name
|
key = f"{prefix}/{plugin_dir.name}" if prefix else name
|
||||||
|
|||||||
@@ -127,7 +127,7 @@ def _read_manifest(plugin_dir: Path) -> dict:
|
|||||||
try:
|
try:
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
with open(manifest_file, encoding="utf-8") as f:
|
with open(manifest_file) as f:
|
||||||
return yaml.safe_load(f) or {}
|
return yaml.safe_load(f) or {}
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning("Failed to read plugin.yaml in %s: %s", plugin_dir, e)
|
logger.warning("Failed to read plugin.yaml in %s: %s", plugin_dir, e)
|
||||||
@@ -703,7 +703,7 @@ def _discover_all_plugins() -> list:
|
|||||||
description = ""
|
description = ""
|
||||||
if yaml:
|
if yaml:
|
||||||
try:
|
try:
|
||||||
with open(manifest_file, encoding="utf-8") as f:
|
with open(manifest_file) as f:
|
||||||
manifest = yaml.safe_load(f) or {}
|
manifest = yaml.safe_load(f) or {}
|
||||||
name = manifest.get("name", d.name)
|
name = manifest.get("name", d.name)
|
||||||
version = manifest.get("version", "")
|
version = manifest.get("version", "")
|
||||||
|
|||||||
+6
-65
@@ -71,22 +71,6 @@ _CLONE_ALL_STRIP = [
|
|||||||
"processes.json",
|
"processes.json",
|
||||||
]
|
]
|
||||||
|
|
||||||
# Marker file written by `hermes profile create --no-skills`. When present in
|
|
||||||
# a profile's root, callers of seed_profile_skills() (fresh-create, `hermes
|
|
||||||
# update`'s all-profile sync, the web dashboard) skip bundled-skill seeding
|
|
||||||
# for that profile. The user can still install skills manually via
|
|
||||||
# `hermes skills install` or drop SKILL.md files into the profile's skills/.
|
|
||||||
# Delete the marker file to opt back in.
|
|
||||||
NO_BUNDLED_SKILLS_MARKER = ".no-bundled-skills"
|
|
||||||
|
|
||||||
|
|
||||||
def has_bundled_skills_opt_out(profile_dir: Path) -> bool:
|
|
||||||
"""Return True if the profile opted out of bundled-skill seeding."""
|
|
||||||
try:
|
|
||||||
return (profile_dir / NO_BUNDLED_SKILLS_MARKER).exists()
|
|
||||||
except OSError:
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def _clone_all_copytree_ignore(source_dir: Path):
|
def _clone_all_copytree_ignore(source_dir: Path):
|
||||||
"""Ignore ``profiles/`` at the root of *source_dir* only.
|
"""Ignore ``profiles/`` at the root of *source_dir* only.
|
||||||
@@ -354,7 +338,7 @@ def _read_config_model(profile_dir: Path) -> tuple:
|
|||||||
return None, None
|
return None, None
|
||||||
try:
|
try:
|
||||||
import yaml
|
import yaml
|
||||||
with open(config_path, "r", encoding="utf-8") as f:
|
with open(config_path, "r") as f:
|
||||||
cfg = yaml.safe_load(f) or {}
|
cfg = yaml.safe_load(f) or {}
|
||||||
model_cfg = cfg.get("model", {})
|
model_cfg = cfg.get("model", {})
|
||||||
if isinstance(model_cfg, str):
|
if isinstance(model_cfg, str):
|
||||||
@@ -443,7 +427,6 @@ def create_profile(
|
|||||||
clone_all: bool = False,
|
clone_all: bool = False,
|
||||||
clone_config: bool = False,
|
clone_config: bool = False,
|
||||||
no_alias: bool = False,
|
no_alias: bool = False,
|
||||||
no_skills: bool = False,
|
|
||||||
) -> Path:
|
) -> Path:
|
||||||
"""Create a new profile directory.
|
"""Create a new profile directory.
|
||||||
|
|
||||||
@@ -461,22 +444,12 @@ def create_profile(
|
|||||||
skills, and selected profile identity files from the source profile.
|
skills, and selected profile identity files from the source profile.
|
||||||
no_alias:
|
no_alias:
|
||||||
If True, skip wrapper script creation.
|
If True, skip wrapper script creation.
|
||||||
no_skills:
|
|
||||||
If True, create an empty profile with no bundled skills, and write
|
|
||||||
a marker file so ``hermes update`` skips re-seeding this profile's
|
|
||||||
skills. Mutually exclusive with ``clone_config``/``clone_all`` (those
|
|
||||||
explicitly copy skills from the source).
|
|
||||||
|
|
||||||
Returns
|
Returns
|
||||||
-------
|
-------
|
||||||
Path
|
Path
|
||||||
The newly created profile directory.
|
The newly created profile directory.
|
||||||
"""
|
"""
|
||||||
if no_skills and (clone_config or clone_all):
|
|
||||||
raise ValueError(
|
|
||||||
"--no-skills is mutually exclusive with --clone / --clone-all "
|
|
||||||
"(cloning explicitly copies skills from the source profile)."
|
|
||||||
)
|
|
||||||
canon = normalize_profile_name(name)
|
canon = normalize_profile_name(name)
|
||||||
validate_profile_name(canon)
|
validate_profile_name(canon)
|
||||||
|
|
||||||
@@ -554,19 +527,6 @@ def create_profile(
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass # best-effort — don't fail profile creation over this
|
pass # best-effort — don't fail profile creation over this
|
||||||
|
|
||||||
# Write the opt-out marker so seed_profile_skills() and `hermes update`'s
|
|
||||||
# all-profile sync loop both skip this profile for bundled-skill seeding.
|
|
||||||
if no_skills:
|
|
||||||
try:
|
|
||||||
(profile_dir / NO_BUNDLED_SKILLS_MARKER).write_text(
|
|
||||||
"This profile opted out of bundled-skill seeding "
|
|
||||||
"(`hermes profile create --no-skills`).\n"
|
|
||||||
"Delete this file to re-enable sync on the next `hermes update`.\n",
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
except OSError:
|
|
||||||
pass # best-effort — the feature still works via the empty skills/ dir
|
|
||||||
|
|
||||||
return profile_dir
|
return profile_dir
|
||||||
|
|
||||||
|
|
||||||
@@ -575,19 +535,7 @@ def seed_profile_skills(profile_dir: Path, quiet: bool = False) -> Optional[dict
|
|||||||
|
|
||||||
Uses subprocess because sync_skills() caches HERMES_HOME at module level.
|
Uses subprocess because sync_skills() caches HERMES_HOME at module level.
|
||||||
Returns the sync result dict, or None on failure.
|
Returns the sync result dict, or None on failure.
|
||||||
|
|
||||||
Profiles that opted out of bundled skills (via ``hermes profile create
|
|
||||||
--no-skills`` — which writes ``.no-bundled-skills`` to the profile root)
|
|
||||||
are skipped and get an empty-result dict so callers can report
|
|
||||||
"opted out" instead of "failed".
|
|
||||||
"""
|
"""
|
||||||
if has_bundled_skills_opt_out(profile_dir):
|
|
||||||
return {
|
|
||||||
"copied": [],
|
|
||||||
"updated": [],
|
|
||||||
"user_modified": [],
|
|
||||||
"skipped_opt_out": True,
|
|
||||||
}
|
|
||||||
project_root = Path(__file__).parent.parent.resolve()
|
project_root = Path(__file__).parent.parent.resolve()
|
||||||
try:
|
try:
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
@@ -758,6 +706,7 @@ def _cleanup_gateway_service(name: str, profile_dir: Path) -> None:
|
|||||||
|
|
||||||
def _stop_gateway_process(profile_dir: Path) -> None:
|
def _stop_gateway_process(profile_dir: Path) -> None:
|
||||||
"""Stop a running gateway process via its PID file."""
|
"""Stop a running gateway process via its PID file."""
|
||||||
|
import signal as _signal
|
||||||
import time as _time
|
import time as _time
|
||||||
|
|
||||||
pid_file = profile_dir / "gateway.pid"
|
pid_file = profile_dir / "gateway.pid"
|
||||||
@@ -768,27 +717,19 @@ def _stop_gateway_process(profile_dir: Path) -> None:
|
|||||||
raw = pid_file.read_text().strip()
|
raw = pid_file.read_text().strip()
|
||||||
data = json.loads(raw) if raw.startswith("{") else {"pid": int(raw)}
|
data = json.loads(raw) if raw.startswith("{") else {"pid": int(raw)}
|
||||||
pid = int(data["pid"])
|
pid = int(data["pid"])
|
||||||
# Route through terminate_pid so Windows uses the appropriate
|
os.kill(pid, _signal.SIGTERM)
|
||||||
# primitive (taskkill / TerminateProcess) — raw os.kill with
|
|
||||||
# _signal.SIGKILL raises AttributeError at import time on Windows,
|
|
||||||
# and raw os.kill with SIGTERM doesn't cascade to child processes
|
|
||||||
# the same way taskkill /T does.
|
|
||||||
from gateway.status import terminate_pid as _terminate_pid
|
|
||||||
_terminate_pid(pid) # graceful first
|
|
||||||
# Wait up to 10s for graceful shutdown
|
# Wait up to 10s for graceful shutdown
|
||||||
for _ in range(20):
|
for _ in range(20):
|
||||||
_time.sleep(0.5)
|
_time.sleep(0.5)
|
||||||
try:
|
try:
|
||||||
os.kill(pid, 0)
|
os.kill(pid, 0)
|
||||||
except (ProcessLookupError, OSError):
|
except ProcessLookupError:
|
||||||
# OSError covers Windows' WinError 87 "invalid parameter"
|
|
||||||
# returned for an invalid/gone PID probe.
|
|
||||||
print(f"✓ Gateway stopped (PID {pid})")
|
print(f"✓ Gateway stopped (PID {pid})")
|
||||||
return
|
return
|
||||||
# Force kill
|
# Force kill
|
||||||
try:
|
try:
|
||||||
_terminate_pid(pid, force=True)
|
os.kill(pid, _signal.SIGKILL)
|
||||||
except (ProcessLookupError, OSError):
|
except ProcessLookupError:
|
||||||
pass
|
pass
|
||||||
print(f"✓ Gateway force-stopped (PID {pid})")
|
print(f"✓ Gateway force-stopped (PID {pid})")
|
||||||
except (ProcessLookupError, PermissionError):
|
except (ProcessLookupError, PermissionError):
|
||||||
|
|||||||
@@ -7,14 +7,11 @@ keystrokes can be fed back in. The only caller today is the
|
|||||||
|
|
||||||
Design constraints:
|
Design constraints:
|
||||||
|
|
||||||
* **POSIX-only.** This module depends on ``fcntl``, ``termios``, and
|
* **POSIX-only.** Hermes Agent supports Windows exclusively via WSL, which
|
||||||
``ptyprocess``, none of which exist on native Windows Python. Native
|
exposes a native POSIX PTY via ``openpty(3)``. Native Windows Python
|
||||||
Windows ConPTY is a different API (Windows 10 build 17763+) and would
|
has no PTY; :class:`PtyUnavailableError` is raised with a user-readable
|
||||||
need a separate Windows implementation (``pywinpty``) — that's tracked
|
install/platform message so the dashboard can render a banner instead of
|
||||||
as a future enhancement. On native Windows, importing this module
|
crashing.
|
||||||
raises :class:`ImportError` and the dashboard's ``/chat`` tab shows a
|
|
||||||
WSL-recommended banner instead of crashing. Every other feature in the
|
|
||||||
dashboard (sessions, jobs, metrics, config editor) works natively.
|
|
||||||
* **Zero Node dependency on the server side.** We use :mod:`ptyprocess`,
|
* **Zero Node dependency on the server side.** We use :mod:`ptyprocess`,
|
||||||
which is a pure-Python wrapper around the OS calls. The browser talks
|
which is a pure-Python wrapper around the OS calls. The browser talks
|
||||||
to the same ``hermes --tui`` binary it would launch from the CLI, so
|
to the same ``hermes --tui`` binary it would launch from the CLI, so
|
||||||
|
|||||||
+4
-60
@@ -84,34 +84,18 @@ def resolve_hermes_bin() -> Optional[str]:
|
|||||||
1. ``sys.argv[0]`` if it resolves to a real executable.
|
1. ``sys.argv[0]`` if it resolves to a real executable.
|
||||||
2. ``shutil.which("hermes")`` on PATH.
|
2. ``shutil.which("hermes")`` on PATH.
|
||||||
3. ``None`` → caller should fall back to ``python -m hermes_cli.main``.
|
3. ``None`` → caller should fall back to ``python -m hermes_cli.main``.
|
||||||
|
|
||||||
Windows note: ``os.access(path, os.X_OK)`` returns True for ``.py`` and
|
|
||||||
``.pyc`` files on Windows (the OS treats anything listed in PATHEXT as
|
|
||||||
executable, and Python files are often registered there). But
|
|
||||||
``subprocess.run([script.py, ...])`` can't actually execute a .py
|
|
||||||
directly — CreateProcessW needs a real .exe, not a script associated
|
|
||||||
with the Python launcher. On Windows we therefore skip the argv[0]
|
|
||||||
fast-path when it points at a .py file and fall through to either
|
|
||||||
``hermes.exe`` on PATH or the ``sys.executable -m hermes_cli.main``
|
|
||||||
fallback.
|
|
||||||
"""
|
"""
|
||||||
argv0 = sys.argv[0]
|
argv0 = sys.argv[0]
|
||||||
_is_windows = sys.platform == "win32"
|
|
||||||
|
|
||||||
def _is_python_script(p: str) -> bool:
|
|
||||||
return p.lower().endswith((".py", ".pyc"))
|
|
||||||
|
|
||||||
# Absolute path to an executable (covers nix store, venv wrappers, etc.)
|
# Absolute path to an executable (covers nix store, venv wrappers, etc.)
|
||||||
if os.path.isabs(argv0) and os.path.isfile(argv0) and os.access(argv0, os.X_OK):
|
if os.path.isabs(argv0) and os.path.isfile(argv0) and os.access(argv0, os.X_OK):
|
||||||
if not (_is_windows and _is_python_script(argv0)):
|
return argv0
|
||||||
return argv0
|
|
||||||
|
|
||||||
# Relative path — resolve against CWD
|
# Relative path — resolve against CWD
|
||||||
if not argv0.startswith("-") and os.path.isfile(argv0):
|
if not argv0.startswith("-") and os.path.isfile(argv0):
|
||||||
abs_path = os.path.abspath(argv0)
|
abs_path = os.path.abspath(argv0)
|
||||||
if os.access(abs_path, os.X_OK):
|
if os.access(abs_path, os.X_OK):
|
||||||
if not (_is_windows and _is_python_script(abs_path)):
|
return abs_path
|
||||||
return abs_path
|
|
||||||
|
|
||||||
# PATH lookup
|
# PATH lookup
|
||||||
path_bin = shutil.which("hermes")
|
path_bin = shutil.which("hermes")
|
||||||
@@ -158,48 +142,8 @@ def relaunch(
|
|||||||
preserve_inherited: bool = True,
|
preserve_inherited: bool = True,
|
||||||
original_argv: Optional[Sequence[str]] = None,
|
original_argv: Optional[Sequence[str]] = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Replace the current process with a fresh hermes invocation.
|
"""Replace the current process with a fresh hermes invocation."""
|
||||||
|
|
||||||
On POSIX we use ``os.execvp`` which replaces the running process with
|
|
||||||
the new one in place — same PID, no double-fork. That's what the
|
|
||||||
relaunch contract wants: "run hermes again as if the user had typed
|
|
||||||
the new argv".
|
|
||||||
|
|
||||||
Windows has no native exec semantics — ``os.execvp`` on Windows
|
|
||||||
*emulates* exec by spawning the child and exiting the parent, but
|
|
||||||
only works when the target is a real Win32 executable. Our target
|
|
||||||
is usually ``hermes.exe`` (a Python console-script shim that wraps
|
|
||||||
``python -m hermes_cli.main``) or a ``.cmd`` batch file, and both
|
|
||||||
raise ``OSError(8, "Exec format error")`` on Windows' execvp.
|
|
||||||
|
|
||||||
The Windows-correct pattern is: spawn the child with ``subprocess.run``
|
|
||||||
(which routes through ``cmd.exe`` via ``shell=False`` + PATHEXT resolution),
|
|
||||||
wait for it to exit, then propagate its exit code via ``sys.exit``.
|
|
||||||
That's functionally equivalent — the user sees "hermes exited, then
|
|
||||||
new hermes started" — just with two PIDs in play instead of one.
|
|
||||||
"""
|
|
||||||
new_argv = build_relaunch_argv(
|
new_argv = build_relaunch_argv(
|
||||||
extra_args, preserve_inherited=preserve_inherited, original_argv=original_argv
|
extra_args, preserve_inherited=preserve_inherited, original_argv=original_argv
|
||||||
)
|
)
|
||||||
if sys.platform == "win32":
|
os.execvp(new_argv[0], new_argv)
|
||||||
# Windows: subprocess + exit, because execvp can't swap to .cmd/.exe shims.
|
|
||||||
import subprocess
|
|
||||||
try:
|
|
||||||
result = subprocess.run(new_argv)
|
|
||||||
sys.exit(result.returncode)
|
|
||||||
except KeyboardInterrupt:
|
|
||||||
sys.exit(130)
|
|
||||||
except OSError as exc:
|
|
||||||
# Surface a helpful error rather than the raw OSError — the
|
|
||||||
# caller used to see ``[Errno 8] Exec format error`` which is
|
|
||||||
# cryptic. Common causes: ``hermes`` not on PATH yet (install
|
|
||||||
# hasn't propagated User PATH into this shell) or a stale shim.
|
|
||||||
print(
|
|
||||||
f"\nHermes relaunch failed: {exc}\n"
|
|
||||||
f"Command: {' '.join(new_argv)}\n"
|
|
||||||
f"Fix: open a new terminal so PATH picks up, then re-run hermes.",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
else:
|
|
||||||
os.execvp(new_argv[0], new_argv)
|
|
||||||
@@ -319,10 +319,9 @@ def _try_resolve_from_custom_pool(
|
|||||||
base_url: str,
|
base_url: str,
|
||||||
provider_label: str,
|
provider_label: str,
|
||||||
api_mode_override: Optional[str] = None,
|
api_mode_override: Optional[str] = None,
|
||||||
provider_name: Optional[str] = None,
|
|
||||||
) -> Optional[Dict[str, Any]]:
|
) -> Optional[Dict[str, Any]]:
|
||||||
"""Check if a credential pool exists for a custom endpoint and return a runtime dict if so."""
|
"""Check if a credential pool exists for a custom endpoint and return a runtime dict if so."""
|
||||||
pool_key = get_custom_provider_pool_key(base_url, provider_name=provider_name)
|
pool_key = get_custom_provider_pool_key(base_url)
|
||||||
if not pool_key:
|
if not pool_key:
|
||||||
return None
|
return None
|
||||||
try:
|
try:
|
||||||
@@ -522,7 +521,7 @@ def _resolve_named_custom_runtime(
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
# Check if a credential pool exists for this custom endpoint
|
# Check if a credential pool exists for this custom endpoint
|
||||||
pool_result = _try_resolve_from_custom_pool(base_url, "custom", custom_provider.get("api_mode"), provider_name=custom_provider.get("name"))
|
pool_result = _try_resolve_from_custom_pool(base_url, "custom", custom_provider.get("api_mode"))
|
||||||
if pool_result:
|
if pool_result:
|
||||||
# Propagate the model name even when using pooled credentials —
|
# Propagate the model name even when using pooled credentials —
|
||||||
# the pool doesn't know about the custom_providers model field.
|
# the pool doesn't know about the custom_providers model field.
|
||||||
@@ -641,11 +640,8 @@ def _resolve_openrouter_runtime(
|
|||||||
|
|
||||||
# For custom endpoints, check if a credential pool exists
|
# For custom endpoints, check if a credential pool exists
|
||||||
if effective_provider == "custom" and base_url:
|
if effective_provider == "custom" and base_url:
|
||||||
# Pass requested_provider so pool lookup prefers name match over base_url,
|
|
||||||
# fixing credential mix-ups when multiple custom providers share a base_url.
|
|
||||||
pool_result = _try_resolve_from_custom_pool(
|
pool_result = _try_resolve_from_custom_pool(
|
||||||
base_url, effective_provider, _parse_api_mode(model_cfg.get("api_mode")),
|
base_url, effective_provider, _parse_api_mode(model_cfg.get("api_mode")),
|
||||||
provider_name=requested_provider if requested_norm != "custom" else None,
|
|
||||||
)
|
)
|
||||||
if pool_result:
|
if pool_result:
|
||||||
return pool_result
|
return pool_result
|
||||||
|
|||||||
+2
-22
@@ -2462,9 +2462,6 @@ def setup_gateway(config: dict):
|
|||||||
launchd_start,
|
launchd_start,
|
||||||
launchd_restart,
|
launchd_restart,
|
||||||
UserSystemdUnavailableError,
|
UserSystemdUnavailableError,
|
||||||
SystemScopeRequiresRootError,
|
|
||||||
_system_scope_wizard_would_need_root,
|
|
||||||
_print_system_scope_remediation,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
service_installed = _is_service_installed()
|
service_installed = _is_service_installed()
|
||||||
@@ -2482,9 +2479,7 @@ def setup_gateway(config: dict):
|
|||||||
print()
|
print()
|
||||||
|
|
||||||
if service_running:
|
if service_running:
|
||||||
if supports_systemd and _system_scope_wizard_would_need_root():
|
if prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
||||||
_print_system_scope_remediation("restart")
|
|
||||||
elif prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
|
||||||
try:
|
try:
|
||||||
if supports_systemd:
|
if supports_systemd:
|
||||||
systemd_restart()
|
systemd_restart()
|
||||||
@@ -2494,19 +2489,10 @@ def setup_gateway(config: dict):
|
|||||||
print_error(" Restart failed — user systemd not reachable:")
|
print_error(" Restart failed — user systemd not reachable:")
|
||||||
for line in str(e).splitlines():
|
for line in str(e).splitlines():
|
||||||
print(f" {line}")
|
print(f" {line}")
|
||||||
except SystemScopeRequiresRootError as e:
|
|
||||||
# Defense in depth: the pre-check above should have
|
|
||||||
# caught this, but a race (unit file appearing mid-run)
|
|
||||||
# could still land here. Previously this exited the
|
|
||||||
# whole wizard via sys.exit(1).
|
|
||||||
print_error(f" Restart failed: {e}")
|
|
||||||
_print_system_scope_remediation("restart")
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print_error(f" Restart failed: {e}")
|
print_error(f" Restart failed: {e}")
|
||||||
elif service_installed:
|
elif service_installed:
|
||||||
if supports_systemd and _system_scope_wizard_would_need_root():
|
if prompt_yes_no(" Start the gateway service?", True):
|
||||||
_print_system_scope_remediation("start")
|
|
||||||
elif prompt_yes_no(" Start the gateway service?", True):
|
|
||||||
try:
|
try:
|
||||||
if supports_systemd:
|
if supports_systemd:
|
||||||
systemd_start()
|
systemd_start()
|
||||||
@@ -2516,9 +2502,6 @@ def setup_gateway(config: dict):
|
|||||||
print_error(" Start failed — user systemd not reachable:")
|
print_error(" Start failed — user systemd not reachable:")
|
||||||
for line in str(e).splitlines():
|
for line in str(e).splitlines():
|
||||||
print(f" {line}")
|
print(f" {line}")
|
||||||
except SystemScopeRequiresRootError as e:
|
|
||||||
print_error(f" Start failed: {e}")
|
|
||||||
_print_system_scope_remediation("start")
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print_error(f" Start failed: {e}")
|
print_error(f" Start failed: {e}")
|
||||||
elif supports_service_manager:
|
elif supports_service_manager:
|
||||||
@@ -2546,9 +2529,6 @@ def setup_gateway(config: dict):
|
|||||||
print_error(" Start failed — user systemd not reachable:")
|
print_error(" Start failed — user systemd not reachable:")
|
||||||
for line in str(e).splitlines():
|
for line in str(e).splitlines():
|
||||||
print(f" {line}")
|
print(f" {line}")
|
||||||
except SystemScopeRequiresRootError as e:
|
|
||||||
print_error(f" Start failed: {e}")
|
|
||||||
_print_system_scope_remediation("start")
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print_error(f" Start failed: {e}")
|
print_error(f" Start failed: {e}")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
|||||||
@@ -1257,7 +1257,7 @@ def do_snapshot_export(output_path: str, console: Optional[Console] = None) -> N
|
|||||||
sys.stdout.write(payload)
|
sys.stdout.write(payload)
|
||||||
else:
|
else:
|
||||||
out = Path(output_path)
|
out = Path(output_path)
|
||||||
out.write_text(payload, encoding="utf-8")
|
out.write_text(payload)
|
||||||
c.print(f"[bold green]Snapshot exported:[/] {out}")
|
c.print(f"[bold green]Snapshot exported:[/] {out}")
|
||||||
c.print(f"[dim]{len(installed)} skill(s), {len(tap_list)} tap(s)[/]\n")
|
c.print(f"[dim]{len(installed)} skill(s), {len(tap_list)} tap(s)[/]\n")
|
||||||
|
|
||||||
@@ -1274,7 +1274,7 @@ def do_snapshot_import(input_path: str, force: bool = False,
|
|||||||
return
|
return
|
||||||
|
|
||||||
try:
|
try:
|
||||||
snapshot = json.loads(inp.read_text(encoding="utf-8"))
|
snapshot = json.loads(inp.read_text())
|
||||||
except json.JSONDecodeError:
|
except json.JSONDecodeError:
|
||||||
c.print(f"[bold red]Error:[/] Invalid JSON in {inp}\n")
|
c.print(f"[bold red]Error:[/] Invalid JSON in {inp}\n")
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -1,252 +0,0 @@
|
|||||||
"""Windows-safe stdio configuration.
|
|
||||||
|
|
||||||
On Windows, Python's ``sys.stdout``/``sys.stderr`` default to the console's
|
|
||||||
active code page (often ``cp1252``, sometimes ``cp437``, occasionally ``cp932``
|
|
||||||
on Japanese locales, etc.). Hermes's banners, tool output feed, and slash
|
|
||||||
command listings all contain Unicode: box-drawing characters (``─┌┐└┘├┤``),
|
|
||||||
mathematical and geometric symbols (``◆ ◇ ◎ ▣ ⚔ ⚖ →``), and user-supplied
|
|
||||||
text in any language. Printing those to a cp1252 console raises
|
|
||||||
``UnicodeEncodeError: 'charmap' codec can't encode character…`` and kills the
|
|
||||||
whole CLI before the REPL even opens.
|
|
||||||
|
|
||||||
The fix is to force UTF-8 on the Python side and also flip the console's
|
|
||||||
code page to UTF-8 (65001). Both matter: Python-level only helps when
|
|
||||||
Python's stdout is a real TTY; code-page flipping lets subprocesses and
|
|
||||||
child Python ``print()`` calls agree on encoding.
|
|
||||||
|
|
||||||
This module is a no-op on every non-Windows platform, and idempotent.
|
|
||||||
Entry points (``cli.py`` ``main``, ``hermes_cli/main.py`` CLI dispatch,
|
|
||||||
``gateway/run.py`` startup) call :func:`configure_windows_stdio` exactly
|
|
||||||
once early in startup.
|
|
||||||
|
|
||||||
Patterns cribbed from Claude Code (``src/utils/platform.ts``), OpenCode
|
|
||||||
(``packages/opencode/src/pty/index.ts`` env injection), and OpenAI Codex
|
|
||||||
(``codex-rs/core/src/unified_exec/process_manager.rs``). None of those
|
|
||||||
actually flip the console code page — they rely on their runtime (Node or
|
|
||||||
Rust) writing UTF-16 to the Win32 console API and letting the terminal
|
|
||||||
sort it out. Python doesn't get that luxury.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
|
|
||||||
__all__ = ["configure_windows_stdio", "is_windows"]
|
|
||||||
|
|
||||||
|
|
||||||
_CONFIGURED = False
|
|
||||||
|
|
||||||
|
|
||||||
def is_windows() -> bool:
|
|
||||||
"""Return True iff running on native Windows (not WSL)."""
|
|
||||||
return sys.platform == "win32"
|
|
||||||
|
|
||||||
|
|
||||||
def _flip_console_code_page_to_utf8() -> None:
|
|
||||||
"""Set the attached console's input and output code pages to UTF-8.
|
|
||||||
|
|
||||||
Uses ``SetConsoleCP`` / ``SetConsoleOutputCP`` via ``ctypes``. Failure
|
|
||||||
is silent — if there's no attached console (e.g. Hermes is running
|
|
||||||
behind a redirected stdout, under a service, or inside a PTY-less CI
|
|
||||||
runner) these calls simply return 0 and we move on.
|
|
||||||
|
|
||||||
CP_UTF8 is 65001.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
import ctypes
|
|
||||||
|
|
||||||
kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined]
|
|
||||||
# Best-effort; if there's no console attached these just fail silently.
|
|
||||||
kernel32.SetConsoleCP(65001)
|
|
||||||
kernel32.SetConsoleOutputCP(65001)
|
|
||||||
except Exception:
|
|
||||||
# ctypes import, missing kernel32, or non-Windows — any failure here
|
|
||||||
# is non-fatal. We've still reconfigured Python's own streams below.
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _reconfigure_stream(stream, *, encoding: str = "utf-8", errors: str = "replace") -> None:
|
|
||||||
"""Reconfigure a text stream to UTF-8 in place.
|
|
||||||
|
|
||||||
Uses ``TextIOWrapper.reconfigure`` (Python 3.7+). If the stream isn't
|
|
||||||
a ``TextIOWrapper`` (e.g. it's been redirected to an ``io.StringIO``
|
|
||||||
during tests), we skip rather than blow up.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
reconfigure = getattr(stream, "reconfigure", None)
|
|
||||||
if reconfigure is None:
|
|
||||||
return
|
|
||||||
reconfigure(encoding=encoding, errors=errors)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def configure_windows_stdio() -> bool:
|
|
||||||
"""Force UTF-8 stdio on Windows. No-op elsewhere.
|
|
||||||
|
|
||||||
Idempotent — safe to call multiple times from different entry points.
|
|
||||||
|
|
||||||
Returns ``True`` if anything was actually changed, ``False`` on
|
|
||||||
non-Windows or on a repeat call.
|
|
||||||
|
|
||||||
Set ``HERMES_DISABLE_WINDOWS_UTF8=1`` in the environment to opt out
|
|
||||||
(for diagnosing encoding-related bugs by forcing the old cp1252 path).
|
|
||||||
|
|
||||||
Also sets a sensible default ``EDITOR`` on Windows if none is already
|
|
||||||
set — see :func:`_default_windows_editor`.
|
|
||||||
"""
|
|
||||||
global _CONFIGURED
|
|
||||||
|
|
||||||
if _CONFIGURED:
|
|
||||||
return False
|
|
||||||
if not is_windows():
|
|
||||||
# Mark configured so repeated calls on POSIX are true no-ops.
|
|
||||||
_CONFIGURED = True
|
|
||||||
return False
|
|
||||||
|
|
||||||
if os.environ.get("HERMES_DISABLE_WINDOWS_UTF8") in ("1", "true", "True", "yes"):
|
|
||||||
_CONFIGURED = True
|
|
||||||
return False
|
|
||||||
|
|
||||||
# Encourage every child Python process spawned by the agent to also use
|
|
||||||
# UTF-8 for its stdio. PYTHONIOENCODING wins over the locale-based
|
|
||||||
# default in subprocesses. Don't override an explicit user setting.
|
|
||||||
os.environ.setdefault("PYTHONIOENCODING", "utf-8")
|
|
||||||
# PYTHONUTF8 = 1 enables UTF-8 Mode globally for any Python subprocess
|
|
||||||
# (PEP 540). Again, don't override an explicit setting.
|
|
||||||
os.environ.setdefault("PYTHONUTF8", "1")
|
|
||||||
|
|
||||||
# Set EDITOR to a working Windows default if neither EDITOR nor VISUAL
|
|
||||||
# is set. prompt_toolkit's ``open_in_editor`` falls back to POSIX-only
|
|
||||||
# paths (``/usr/bin/nano``, ``/usr/bin/vi``) that don't exist on
|
|
||||||
# Windows — Ctrl+X Ctrl+E and ``/edit`` silently do nothing there
|
|
||||||
# otherwise. This happens even with full Git for Windows installed,
|
|
||||||
# so it's not a MinGit-specific issue.
|
|
||||||
_default_editor = _default_windows_editor()
|
|
||||||
if _default_editor and not os.environ.get("EDITOR") and not os.environ.get("VISUAL"):
|
|
||||||
os.environ["EDITOR"] = _default_editor
|
|
||||||
|
|
||||||
# Augment PATH with the Hermes-managed Git install directories so
|
|
||||||
# subprocess calls (bash, rg, grep, etc.) resolve even in sessions
|
|
||||||
# that started before the User PATH broadcast reached them. When
|
|
||||||
# install.ps1 adds these to User PATH via SetEnvironmentVariable,
|
|
||||||
# already-running shells don't see the change — which means hermes
|
|
||||||
# launched from the install session won't find rg / bash / grep
|
|
||||||
# even though they're "installed". Prepending the known paths here
|
|
||||||
# closes that gap. No-op when the paths don't exist (e.g. system-Git
|
|
||||||
# install without Hermes-managed PortableGit).
|
|
||||||
_augment_path_with_known_tools()
|
|
||||||
|
|
||||||
# Flip the console code page first so that any subprocess that
|
|
||||||
# inherits the console (e.g. a launched shell) also sees CP_UTF8.
|
|
||||||
_flip_console_code_page_to_utf8()
|
|
||||||
|
|
||||||
# Reconfigure Python's own stdio wrappers so ``print()`` calls from
|
|
||||||
# this process round-trip emoji / box-drawing / non-Latin text.
|
|
||||||
# ``errors="replace"`` means a genuinely unencodable byte sequence
|
|
||||||
# gets a ``?`` rather than crashing the interpreter — we prefer
|
|
||||||
# degraded output over a stack trace.
|
|
||||||
_reconfigure_stream(sys.stdout)
|
|
||||||
_reconfigure_stream(sys.stderr)
|
|
||||||
# stdin is re-configured for completeness; Hermes's interactive
|
|
||||||
# input path uses prompt_toolkit which manages its own encoding,
|
|
||||||
# but batch/pipe input benefits from UTF-8 decoding on stdin too.
|
|
||||||
_reconfigure_stream(sys.stdin)
|
|
||||||
|
|
||||||
_CONFIGURED = True
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def _default_windows_editor() -> str:
|
|
||||||
"""Return a Windows-appropriate default for ``$EDITOR``.
|
|
||||||
|
|
||||||
Priority order, first match wins:
|
|
||||||
|
|
||||||
1. ``notepad`` — ships with every Windows install, no deps, works as a
|
|
||||||
blocking editor (``subprocess.call(["notepad", file])`` blocks until
|
|
||||||
the user closes the window). This is the "always-works" default.
|
|
||||||
|
|
||||||
The prompt_toolkit buffer's ``open_in_editor`` and Hermes's
|
|
||||||
``hermes config edit`` both honour ``$EDITOR``. Users who prefer a
|
|
||||||
different editor can override:
|
|
||||||
|
|
||||||
- VSCode: ``$env:EDITOR = "code --wait"`` (``--wait`` is critical;
|
|
||||||
without it the editor returns immediately and any input is lost)
|
|
||||||
- Notepad++: ``$env:EDITOR = "'C:\\Program Files\\Notepad++\\notepad++.exe' -multiInst -nosession"``
|
|
||||||
- Neovim: ``$env:EDITOR = "nvim"`` (if installed)
|
|
||||||
|
|
||||||
Set this before launching Hermes (User env var in Windows Settings, or
|
|
||||||
export in a PowerShell profile) and Hermes picks it up automatically.
|
|
||||||
"""
|
|
||||||
import shutil
|
|
||||||
|
|
||||||
# notepad.exe is always in %SystemRoot%\System32 on Windows, so shutil.which
|
|
||||||
# will reliably find it. Return the bare name so prompt_toolkit's shlex
|
|
||||||
# split doesn't trip over a path containing spaces.
|
|
||||||
if shutil.which("notepad"):
|
|
||||||
return "notepad"
|
|
||||||
# On the extreme off-chance notepad is missing (WinPE, Nano Server), fall
|
|
||||||
# back to nothing and let prompt_toolkit's silent no-op do its thing.
|
|
||||||
return ""
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def _augment_path_with_known_tools() -> None:
|
|
||||||
"""Prepend well-known Hermes-managed tool directories to os.environ['PATH'].
|
|
||||||
|
|
||||||
Fixes the "User PATH was just updated but my process can't see it" gap on
|
|
||||||
Windows. When install.ps1 runs, it adds entries like
|
|
||||||
``%LOCALAPPDATA%\\hermes\\git\\bin`` to the User PATH via
|
|
||||||
``SetEnvironmentVariable(..., "User")``. That write propagates to newly
|
|
||||||
*spawned* processes only — already-running shells (including the one the
|
|
||||||
user invokes ``hermes`` from right after install) retain their old PATH.
|
|
||||||
|
|
||||||
Any subprocess Hermes spawns — bash, ``rg``, ``grep``, ``npm`` — inherits
|
|
||||||
that stale PATH and reports commands as missing even though they're on
|
|
||||||
disk. Symptom: ``search_files`` reports "rg/find not available" when
|
|
||||||
the user clearly just installed ripgrep.
|
|
||||||
|
|
||||||
Patch-up strategy: add the known Hermes-managed tool directories to our
|
|
||||||
PATH at startup so subprocess calls resolve correctly. No-op on POSIX
|
|
||||||
and when the directories don't exist. The User PATH broadcast still
|
|
||||||
happens in the background for future shells; this just smooths over
|
|
||||||
the first-launch gap.
|
|
||||||
"""
|
|
||||||
if not is_windows():
|
|
||||||
return
|
|
||||||
|
|
||||||
import shutil as _shutil
|
|
||||||
|
|
||||||
local_appdata = os.environ.get("LOCALAPPDATA", "")
|
|
||||||
if not local_appdata:
|
|
||||||
return
|
|
||||||
|
|
||||||
# Known tool dirs installed by scripts/install.ps1. Kept in sync with
|
|
||||||
# the PATH entries that installer adds to User scope — the two lists
|
|
||||||
# should match so this prefill fully mirrors what a fresh shell would
|
|
||||||
# see on next launch.
|
|
||||||
candidate_dirs = [
|
|
||||||
os.path.join(local_appdata, "hermes", "git", "cmd"),
|
|
||||||
os.path.join(local_appdata, "hermes", "git", "bin"),
|
|
||||||
os.path.join(local_appdata, "hermes", "git", "usr", "bin"),
|
|
||||||
# Hermes venv Scripts directory — host of the hermes.exe shim itself,
|
|
||||||
# also where any pip-installed console scripts land. Usually already
|
|
||||||
# on PATH when the user invokes hermes, but harmless to include.
|
|
||||||
os.path.join(local_appdata, "hermes", "hermes-agent", "venv", "Scripts"),
|
|
||||||
# WinGet packages directory — where ``winget install`` drops CLI
|
|
||||||
# shims by default (ripgrep lands here as rg.exe). Covers the case
|
|
||||||
# of a system-Git install + ripgrep-via-winget that isn't yet on
|
|
||||||
# the spawning shell's PATH.
|
|
||||||
os.path.join(local_appdata, "Microsoft", "WinGet", "Links"),
|
|
||||||
]
|
|
||||||
|
|
||||||
existing = os.environ.get("PATH", "")
|
|
||||||
existing_lower = {p.lower() for p in existing.split(os.pathsep) if p}
|
|
||||||
prepend = []
|
|
||||||
for d in candidate_dirs:
|
|
||||||
if os.path.isdir(d) and d.lower() not in existing_lower:
|
|
||||||
prepend.append(d)
|
|
||||||
|
|
||||||
if prepend:
|
|
||||||
os.environ["PATH"] = os.pathsep.join([*prepend, existing])
|
|
||||||
@@ -308,23 +308,6 @@ TOOL_CATEGORIES = {
|
|||||||
{"key": "SEARXNG_URL", "prompt": "Your SearXNG instance URL (e.g., http://localhost:8080)", "url": "https://searxng.github.io/searxng/"},
|
{"key": "SEARXNG_URL", "prompt": "Your SearXNG instance URL (e.g., http://localhost:8080)", "url": "https://searxng.github.io/searxng/"},
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
{
|
|
||||||
"name": "Brave Search (Free Tier)",
|
|
||||||
"badge": "free tier · search only",
|
|
||||||
"tag": "2,000 queries/mo free — search only (pair with any extract provider)",
|
|
||||||
"web_backend": "brave-free",
|
|
||||||
"env_vars": [
|
|
||||||
{"key": "BRAVE_SEARCH_API_KEY", "prompt": "Brave Search subscription token", "url": "https://brave.com/search/api/"},
|
|
||||||
],
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "DuckDuckGo (ddgs)",
|
|
||||||
"badge": "free · no key · search only",
|
|
||||||
"tag": "Search via the ddgs Python package — no API key (pair with any extract provider)",
|
|
||||||
"web_backend": "ddgs",
|
|
||||||
"env_vars": [],
|
|
||||||
"post_setup": "ddgs",
|
|
||||||
},
|
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
"image_gen": {
|
"image_gen": {
|
||||||
@@ -509,12 +492,8 @@ def _run_post_setup(post_setup_key: str):
|
|||||||
if not node_modules.exists() and npm_bin:
|
if not node_modules.exists() and npm_bin:
|
||||||
_print_info(" Installing Node.js dependencies for browser tools...")
|
_print_info(" Installing Node.js dependencies for browser tools...")
|
||||||
import subprocess
|
import subprocess
|
||||||
# Use the resolved npm_bin absolute path so subprocess.Popen can
|
|
||||||
# execute npm.cmd on Windows (CreateProcessW otherwise rejects
|
|
||||||
# batch shims). On POSIX npm_bin is the plain path — same
|
|
||||||
# behaviour as before.
|
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
[npm_bin, "install", "--silent"],
|
["npm", "install", "--silent"],
|
||||||
capture_output=True, text=True, cwd=str(PROJECT_ROOT)
|
capture_output=True, text=True, cwd=str(PROJECT_ROOT)
|
||||||
)
|
)
|
||||||
if result.returncode == 0:
|
if result.returncode == 0:
|
||||||
@@ -613,13 +592,11 @@ def _run_post_setup(post_setup_key: str):
|
|||||||
|
|
||||||
elif post_setup_key == "camofox":
|
elif post_setup_key == "camofox":
|
||||||
camofox_dir = PROJECT_ROOT / "node_modules" / "@askjo" / "camofox-browser"
|
camofox_dir = PROJECT_ROOT / "node_modules" / "@askjo" / "camofox-browser"
|
||||||
_npm_bin = shutil.which("npm")
|
if not camofox_dir.exists() and shutil.which("npm"):
|
||||||
if not camofox_dir.exists() and _npm_bin:
|
|
||||||
_print_info(" Installing Camofox browser server...")
|
_print_info(" Installing Camofox browser server...")
|
||||||
import subprocess
|
import subprocess
|
||||||
# Absolute npm path so .cmd shim executes on Windows.
|
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
[_npm_bin, "install", "--silent"],
|
["npm", "install", "--silent"],
|
||||||
capture_output=True, text=True, cwd=str(PROJECT_ROOT)
|
capture_output=True, text=True, cwd=str(PROJECT_ROOT)
|
||||||
)
|
)
|
||||||
if result.returncode == 0:
|
if result.returncode == 0:
|
||||||
@@ -692,32 +669,6 @@ def _run_post_setup(post_setup_key: str):
|
|||||||
_print_info(" Full voice list: https://github.com/OHF-Voice/piper1-gpl/blob/main/docs/VOICES.md")
|
_print_info(" Full voice list: https://github.com/OHF-Voice/piper1-gpl/blob/main/docs/VOICES.md")
|
||||||
_print_info(" Switch voices by setting tts.piper.voice in ~/.hermes/config.yaml")
|
_print_info(" Switch voices by setting tts.piper.voice in ~/.hermes/config.yaml")
|
||||||
|
|
||||||
elif post_setup_key == "ddgs":
|
|
||||||
try:
|
|
||||||
__import__("ddgs")
|
|
||||||
_print_success(" ddgs is already installed")
|
|
||||||
except ImportError:
|
|
||||||
import subprocess
|
|
||||||
_print_info(" Installing ddgs (DuckDuckGo search package)...")
|
|
||||||
try:
|
|
||||||
result = subprocess.run(
|
|
||||||
[sys.executable, "-m", "pip", "install", "-U", "ddgs", "--quiet"],
|
|
||||||
capture_output=True, text=True, timeout=300,
|
|
||||||
)
|
|
||||||
if result.returncode == 0:
|
|
||||||
_print_success(" ddgs installed")
|
|
||||||
else:
|
|
||||||
_print_warning(" ddgs install failed:")
|
|
||||||
_print_info(f" {result.stderr.strip()[:300]}")
|
|
||||||
_print_info(" Run manually: python -m pip install -U ddgs")
|
|
||||||
return
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
_print_warning(" ddgs install timed out (>5min)")
|
|
||||||
_print_info(" Run manually: python -m pip install -U ddgs")
|
|
||||||
return
|
|
||||||
_print_info(" No API key required. DuckDuckGo enforces server-side rate limits.")
|
|
||||||
_print_info(" Pair with an extract provider if you also need web_extract.")
|
|
||||||
|
|
||||||
elif post_setup_key == "spotify":
|
elif post_setup_key == "spotify":
|
||||||
# Run the full `hermes auth spotify` flow — if the user has no
|
# Run the full `hermes auth spotify` flow — if the user has no
|
||||||
# client_id yet, this drops them into the interactive wizard
|
# client_id yet, this drops them into the interactive wizard
|
||||||
|
|||||||
+36
-148
@@ -281,8 +281,6 @@ _recorder_lock = threading.Lock()
|
|||||||
# ── Continuous (VAD) state ───────────────────────────────────────────
|
# ── Continuous (VAD) state ───────────────────────────────────────────
|
||||||
_continuous_lock = threading.Lock()
|
_continuous_lock = threading.Lock()
|
||||||
_continuous_active = False
|
_continuous_active = False
|
||||||
_continuous_stopping = False
|
|
||||||
_continuous_auto_restart: bool = True
|
|
||||||
_continuous_recorder: Any = None
|
_continuous_recorder: Any = None
|
||||||
|
|
||||||
# ── TTS-vs-STT feedback guard ────────────────────────────────────────
|
# ── TTS-vs-STT feedback guard ────────────────────────────────────────
|
||||||
@@ -372,43 +370,32 @@ def start_continuous(
|
|||||||
on_silent_limit: Optional[Callable[[], None]] = None,
|
on_silent_limit: Optional[Callable[[], None]] = None,
|
||||||
silence_threshold: int = 200,
|
silence_threshold: int = 200,
|
||||||
silence_duration: float = 3.0,
|
silence_duration: float = 3.0,
|
||||||
auto_restart: bool = True,
|
) -> None:
|
||||||
) -> bool:
|
|
||||||
"""Start a VAD-driven continuous recording loop.
|
"""Start a VAD-driven continuous recording loop.
|
||||||
|
|
||||||
The loop calls ``on_transcript(text)`` each time speech is detected and
|
The loop calls ``on_transcript(text)`` each time speech is detected and
|
||||||
transcribed successfully. If ``auto_restart`` is True, it auto-restarts
|
transcribed successfully, then auto-restarts. After
|
||||||
for the next turn and resets the no-speech counter for that loop. If
|
``_CONTINUOUS_NO_SPEECH_LIMIT`` consecutive silent cycles (no speech
|
||||||
``auto_restart`` is False, the first silence-triggered transcription ends
|
picked up at all) the loop stops itself and calls ``on_silent_limit``
|
||||||
the loop and reports ``"idle"``; no-speech counts are retained across
|
so the UI can reflect "voice off". Idempotent — calling while already
|
||||||
starts so a push-to-talk caller can still enforce the three-strikes guard.
|
active is a no-op.
|
||||||
After ``_CONTINUOUS_NO_SPEECH_LIMIT`` consecutive silent cycles (no speech
|
|
||||||
picked up at all) the loop stops itself and calls ``on_silent_limit`` so the
|
|
||||||
UI can reflect "voice off". Returns False if a previous stop is still
|
|
||||||
transcribing/cleaning up; otherwise returns True. Idempotent — calling while
|
|
||||||
already active is a successful no-op.
|
|
||||||
|
|
||||||
``on_status`` is called with ``"listening"`` / ``"transcribing"`` /
|
``on_status`` is called with ``"listening"`` / ``"transcribing"`` /
|
||||||
``"idle"`` so the UI can show a live indicator.
|
``"idle"`` so the UI can show a live indicator.
|
||||||
"""
|
"""
|
||||||
global _continuous_active, _continuous_recorder, _continuous_auto_restart
|
global _continuous_active, _continuous_recorder
|
||||||
global _continuous_on_transcript, _continuous_on_status, _continuous_on_silent_limit
|
global _continuous_on_transcript, _continuous_on_status, _continuous_on_silent_limit
|
||||||
global _continuous_no_speech_count
|
global _continuous_no_speech_count
|
||||||
|
|
||||||
with _continuous_lock:
|
with _continuous_lock:
|
||||||
if _continuous_active:
|
if _continuous_active:
|
||||||
_debug("start_continuous: already active — no-op")
|
_debug("start_continuous: already active — no-op")
|
||||||
return True
|
return
|
||||||
if _continuous_stopping:
|
|
||||||
_debug("start_continuous: stop/transcribe in progress — busy")
|
|
||||||
return False
|
|
||||||
_continuous_active = True
|
_continuous_active = True
|
||||||
_continuous_auto_restart = auto_restart
|
|
||||||
_continuous_on_transcript = on_transcript
|
_continuous_on_transcript = on_transcript
|
||||||
_continuous_on_status = on_status
|
_continuous_on_status = on_status
|
||||||
_continuous_on_silent_limit = on_silent_limit
|
_continuous_on_silent_limit = on_silent_limit
|
||||||
if auto_restart:
|
_continuous_no_speech_count = 0
|
||||||
_continuous_no_speech_count = 0
|
|
||||||
|
|
||||||
if _continuous_recorder is None:
|
if _continuous_recorder is None:
|
||||||
_continuous_recorder = create_audio_recorder()
|
_continuous_recorder = create_audio_recorder()
|
||||||
@@ -441,18 +428,15 @@ def start_continuous(
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
def stop_continuous() -> None:
|
||||||
def stop_continuous(force_transcribe: bool = False) -> None:
|
|
||||||
"""Stop the active continuous loop and release the microphone.
|
"""Stop the active continuous loop and release the microphone.
|
||||||
|
|
||||||
Idempotent — calling while not active is a no-op. If ``force_transcribe`` is
|
Idempotent — calling while not active is a no-op. Any in-flight
|
||||||
True, the recorder stops synchronously, then transcription/cleanup runs on a
|
transcription completes but its result is discarded (the callback
|
||||||
background thread before reporting ``"idle"``. Otherwise the buffer is
|
checks ``_continuous_active`` before firing).
|
||||||
discarded.
|
|
||||||
"""
|
"""
|
||||||
global _continuous_active, _continuous_on_transcript, _continuous_stopping
|
global _continuous_active, _continuous_on_transcript
|
||||||
global _continuous_on_status, _continuous_on_silent_limit
|
global _continuous_on_status, _continuous_on_silent_limit
|
||||||
global _continuous_recorder, _continuous_no_speech_count
|
global _continuous_recorder, _continuous_no_speech_count
|
||||||
|
|
||||||
@@ -462,98 +446,18 @@ def stop_continuous(force_transcribe: bool = False) -> None:
|
|||||||
_continuous_active = False
|
_continuous_active = False
|
||||||
rec = _continuous_recorder
|
rec = _continuous_recorder
|
||||||
on_status = _continuous_on_status
|
on_status = _continuous_on_status
|
||||||
on_transcript = _continuous_on_transcript
|
|
||||||
on_silent_limit = _continuous_on_silent_limit
|
|
||||||
auto_restart = _continuous_auto_restart
|
|
||||||
track_no_speech = force_transcribe and not auto_restart
|
|
||||||
_continuous_stopping = rec is not None
|
|
||||||
_continuous_on_transcript = None
|
_continuous_on_transcript = None
|
||||||
_continuous_on_status = None
|
_continuous_on_status = None
|
||||||
_continuous_on_silent_limit = None
|
_continuous_on_silent_limit = None
|
||||||
if not track_no_speech:
|
_continuous_no_speech_count = 0
|
||||||
_continuous_no_speech_count = 0
|
|
||||||
|
|
||||||
if rec is not None:
|
if rec is not None:
|
||||||
if force_transcribe and on_transcript:
|
try:
|
||||||
if on_status:
|
# cancel() (not stop()) discards buffered frames — the loop
|
||||||
try:
|
# is over, we don't want to transcribe a half-captured turn.
|
||||||
on_status("transcribing")
|
rec.cancel()
|
||||||
except Exception:
|
except Exception as e:
|
||||||
pass
|
logger.warning("failed to cancel recorder: %s", e)
|
||||||
try:
|
|
||||||
wav_path = rec.stop()
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning("failed to stop recorder: %s", e)
|
|
||||||
try:
|
|
||||||
rec.cancel()
|
|
||||||
except Exception as cancel_error:
|
|
||||||
logger.warning("failed to cancel recorder: %s", cancel_error)
|
|
||||||
wav_path = None
|
|
||||||
|
|
||||||
def _transcribe_and_cleanup():
|
|
||||||
global _continuous_no_speech_count, _continuous_stopping
|
|
||||||
transcript: Optional[str] = None
|
|
||||||
should_halt = False
|
|
||||||
|
|
||||||
try:
|
|
||||||
if wav_path:
|
|
||||||
try:
|
|
||||||
result = transcribe_recording(wav_path)
|
|
||||||
if result.get("success"):
|
|
||||||
text = (result.get("transcript") or "").strip()
|
|
||||||
if text and not is_whisper_hallucination(text):
|
|
||||||
transcript = text
|
|
||||||
finally:
|
|
||||||
if os.path.isfile(wav_path):
|
|
||||||
os.unlink(wav_path)
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning("failed to stop/transcribe recorder: %s", e)
|
|
||||||
finally:
|
|
||||||
if transcript:
|
|
||||||
try:
|
|
||||||
on_transcript(transcript)
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning("on_transcript callback raised: %s", e)
|
|
||||||
|
|
||||||
if track_no_speech:
|
|
||||||
with _continuous_lock:
|
|
||||||
if transcript:
|
|
||||||
_continuous_no_speech_count = 0
|
|
||||||
else:
|
|
||||||
_continuous_no_speech_count += 1
|
|
||||||
should_halt = (
|
|
||||||
_continuous_no_speech_count
|
|
||||||
>= _CONTINUOUS_NO_SPEECH_LIMIT
|
|
||||||
)
|
|
||||||
if should_halt:
|
|
||||||
_continuous_no_speech_count = 0
|
|
||||||
if should_halt and on_silent_limit:
|
|
||||||
try:
|
|
||||||
on_silent_limit()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
_play_beep(frequency=660, count=2)
|
|
||||||
with _continuous_lock:
|
|
||||||
_continuous_stopping = False
|
|
||||||
if on_status:
|
|
||||||
try:
|
|
||||||
on_status("idle")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
threading.Thread(target=_transcribe_and_cleanup, daemon=True).start()
|
|
||||||
return
|
|
||||||
else:
|
|
||||||
try:
|
|
||||||
# cancel() (not stop()) discards buffered frames — the loop
|
|
||||||
# is over, we don't want to transcribe a half-captured turn.
|
|
||||||
rec.cancel()
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning("failed to cancel recorder: %s", e)
|
|
||||||
|
|
||||||
with _continuous_lock:
|
|
||||||
_continuous_stopping = False
|
|
||||||
|
|
||||||
# Audible "recording stopped" cue (CLI parity: same 660 Hz × 2 the
|
# Audible "recording stopped" cue (CLI parity: same 660 Hz × 2 the
|
||||||
# silence-auto-stop path plays).
|
# silence-auto-stop path plays).
|
||||||
@@ -699,39 +603,23 @@ def _continuous_on_silence() -> None:
|
|||||||
_debug("_continuous_on_silence: stopped while waiting for TTS")
|
_debug("_continuous_on_silence: stopped while waiting for TTS")
|
||||||
return
|
return
|
||||||
|
|
||||||
if _continuous_auto_restart:
|
# Restart for the next turn.
|
||||||
# Restart for the next turn.
|
_debug(f"_continuous_on_silence: restarting loop (no_speech={no_speech})")
|
||||||
_debug(f"_continuous_on_silence: restarting loop (no_speech={no_speech})")
|
_play_beep(frequency=880, count=1)
|
||||||
_play_beep(frequency=880, count=1)
|
try:
|
||||||
try:
|
rec.start(on_silence_stop=_continuous_on_silence)
|
||||||
rec.start(on_silence_stop=_continuous_on_silence)
|
except Exception as e:
|
||||||
except Exception as e:
|
logger.error("failed to restart continuous recording: %s", e)
|
||||||
logger.error("failed to restart continuous recording: %s", e)
|
_debug(f"_continuous_on_silence: restart raised {type(e).__name__}: {e}")
|
||||||
_debug(f"_continuous_on_silence: restart raised {type(e).__name__}: {e}")
|
|
||||||
with _continuous_lock:
|
|
||||||
_continuous_active = False
|
|
||||||
if on_status:
|
|
||||||
try:
|
|
||||||
on_status("idle")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return
|
|
||||||
|
|
||||||
if on_status:
|
|
||||||
try:
|
|
||||||
on_status("listening")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
# Do not auto-restart. Clean up state and notify idle.
|
|
||||||
_debug("_continuous_on_silence: auto_restart=False, stopping loop")
|
|
||||||
with _continuous_lock:
|
with _continuous_lock:
|
||||||
_continuous_active = False
|
_continuous_active = False
|
||||||
if on_status:
|
return
|
||||||
try:
|
|
||||||
on_status("idle")
|
if on_status:
|
||||||
except Exception:
|
try:
|
||||||
pass
|
on_status("listening")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
# ── TTS API ──────────────────────────────────────────────────────────
|
# ── TTS API ──────────────────────────────────────────────────────────
|
||||||
|
|||||||
+13
-209
@@ -52,7 +52,7 @@ from gateway.status import get_running_pid, read_runtime_status
|
|||||||
try:
|
try:
|
||||||
from fastapi import FastAPI, HTTPException, Request, WebSocket, WebSocketDisconnect
|
from fastapi import FastAPI, HTTPException, Request, WebSocket, WebSocketDisconnect
|
||||||
from fastapi.middleware.cors import CORSMiddleware
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response
|
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse
|
||||||
from fastapi.staticfiles import StaticFiles
|
from fastapi.staticfiles import StaticFiles
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
except ImportError:
|
except ImportError:
|
||||||
@@ -692,7 +692,7 @@ def _tail_lines(path: Path, n: int) -> List[str]:
|
|||||||
if not path.exists():
|
if not path.exists():
|
||||||
return []
|
return []
|
||||||
try:
|
try:
|
||||||
text = path.read_text(encoding="utf-8", errors="replace")
|
text = path.read_text(errors="replace")
|
||||||
except OSError:
|
except OSError:
|
||||||
return []
|
return []
|
||||||
lines = text.splitlines()
|
lines = text.splitlines()
|
||||||
@@ -1877,8 +1877,8 @@ async def _start_device_code_flow(provider_id: str) -> Dict[str, Any]:
|
|||||||
name=f"oauth-codex-{sid[:6]}",
|
name=f"oauth-codex-{sid[:6]}",
|
||||||
).start()
|
).start()
|
||||||
# Block briefly until the worker has populated the user_code, OR error.
|
# Block briefly until the worker has populated the user_code, OR error.
|
||||||
deadline = time.monotonic() + 10
|
deadline = time.time() + 10
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
with _oauth_sessions_lock:
|
with _oauth_sessions_lock:
|
||||||
s = _oauth_sessions.get(sid)
|
s = _oauth_sessions.get(sid)
|
||||||
if s and (s.get("user_code") or s["status"] != "pending"):
|
if s and (s.get("user_code") or s["status"] != "pending"):
|
||||||
@@ -2012,10 +2012,10 @@ def _codex_full_login_worker(session_id: str) -> None:
|
|||||||
sess["expires_at"] = time.time() + sess["expires_in"]
|
sess["expires_at"] = time.time() + sess["expires_in"]
|
||||||
|
|
||||||
# Step 2: poll until authorized
|
# Step 2: poll until authorized
|
||||||
deadline = time.monotonic() + sess["expires_in"]
|
deadline = time.time() + sess["expires_in"]
|
||||||
code_resp = None
|
code_resp = None
|
||||||
with httpx.Client(timeout=httpx.Timeout(15.0)) as client:
|
with httpx.Client(timeout=httpx.Timeout(15.0)) as client:
|
||||||
while time.monotonic() < deadline:
|
while time.time() < deadline:
|
||||||
time.sleep(poll_interval)
|
time.sleep(poll_interval)
|
||||||
poll = client.post(
|
poll = client.post(
|
||||||
f"{issuer}/api/accounts/deviceauth/token",
|
f"{issuer}/api/accounts/deviceauth/token",
|
||||||
@@ -2173,83 +2173,6 @@ async def cancel_oauth_session(session_id: str, request: Request):
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def _session_latest_descendant(session_id: str):
|
|
||||||
"""Resolve a session id to the newest child leaf session.
|
|
||||||
|
|
||||||
/model may create child sessions. Dashboard refresh should continue the
|
|
||||||
newest child instead of reopening the old parent.
|
|
||||||
"""
|
|
||||||
from hermes_state import SessionDB
|
|
||||||
|
|
||||||
def row_get(row, key, index):
|
|
||||||
if isinstance(row, dict):
|
|
||||||
return row.get(key)
|
|
||||||
try:
|
|
||||||
return row[key]
|
|
||||||
except Exception:
|
|
||||||
try:
|
|
||||||
return row[index]
|
|
||||||
except Exception:
|
|
||||||
return None
|
|
||||||
|
|
||||||
db = SessionDB()
|
|
||||||
try:
|
|
||||||
sid = db.resolve_session_id(session_id)
|
|
||||||
if not sid or not db.get_session(sid):
|
|
||||||
return None, []
|
|
||||||
|
|
||||||
conn = (
|
|
||||||
getattr(db, "conn", None)
|
|
||||||
or getattr(db, "_conn", None)
|
|
||||||
or getattr(db, "connection", None)
|
|
||||||
or getattr(db, "_connection", None)
|
|
||||||
)
|
|
||||||
|
|
||||||
rows = []
|
|
||||||
if conn is not None:
|
|
||||||
raw_rows = conn.execute(
|
|
||||||
"SELECT id, parent_session_id, started_at FROM sessions"
|
|
||||||
).fetchall()
|
|
||||||
for row in raw_rows:
|
|
||||||
rows.append({
|
|
||||||
"id": row_get(row, "id", 0),
|
|
||||||
"parent_session_id": row_get(row, "parent_session_id", 1),
|
|
||||||
"started_at": row_get(row, "started_at", 2),
|
|
||||||
})
|
|
||||||
else:
|
|
||||||
rows = db.list_sessions_rich(limit=10000, offset=0)
|
|
||||||
|
|
||||||
children = {}
|
|
||||||
for row in rows:
|
|
||||||
rid = row.get("id")
|
|
||||||
parent = row.get("parent_session_id")
|
|
||||||
if rid and parent:
|
|
||||||
children.setdefault(parent, []).append(row)
|
|
||||||
|
|
||||||
def started(row):
|
|
||||||
try:
|
|
||||||
return float(row.get("started_at") or 0)
|
|
||||||
except Exception:
|
|
||||||
return 0.0
|
|
||||||
|
|
||||||
current = sid
|
|
||||||
path = [sid]
|
|
||||||
seen = {sid}
|
|
||||||
|
|
||||||
while children.get(current):
|
|
||||||
candidates = [r for r in children[current] if r.get("id") not in seen]
|
|
||||||
if not candidates:
|
|
||||||
break
|
|
||||||
candidates.sort(key=started, reverse=True)
|
|
||||||
current = candidates[0]["id"]
|
|
||||||
path.append(current)
|
|
||||||
seen.add(current)
|
|
||||||
|
|
||||||
return current, path
|
|
||||||
finally:
|
|
||||||
db.close()
|
|
||||||
|
|
||||||
@app.get("/api/sessions/{session_id}")
|
@app.get("/api/sessions/{session_id}")
|
||||||
async def get_session_detail(session_id: str):
|
async def get_session_detail(session_id: str):
|
||||||
from hermes_state import SessionDB
|
from hermes_state import SessionDB
|
||||||
@@ -2264,19 +2187,6 @@ async def get_session_detail(session_id: str):
|
|||||||
db.close()
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/api/sessions/{session_id}/latest-descendant")
|
|
||||||
async def get_session_latest_descendant(session_id: str):
|
|
||||||
latest, path = _session_latest_descendant(session_id)
|
|
||||||
if not latest:
|
|
||||||
raise HTTPException(status_code=404, detail="Session not found")
|
|
||||||
return {
|
|
||||||
"requested_session_id": path[0] if path else session_id,
|
|
||||||
"session_id": latest,
|
|
||||||
"path": path,
|
|
||||||
"changed": bool(path and latest != path[0]),
|
|
||||||
}
|
|
||||||
|
|
||||||
@app.get("/api/sessions/{session_id}/messages")
|
@app.get("/api/sessions/{session_id}/messages")
|
||||||
async def get_session_messages(session_id: str):
|
async def get_session_messages(session_id: str):
|
||||||
from hermes_state import SessionDB
|
from hermes_state import SessionDB
|
||||||
@@ -2456,7 +2366,6 @@ async def delete_cron_job(job_id: str):
|
|||||||
class ProfileCreate(BaseModel):
|
class ProfileCreate(BaseModel):
|
||||||
name: str
|
name: str
|
||||||
clone_from_default: bool = False
|
clone_from_default: bool = False
|
||||||
no_skills: bool = False
|
|
||||||
|
|
||||||
|
|
||||||
class ProfileRename(BaseModel):
|
class ProfileRename(BaseModel):
|
||||||
@@ -2562,13 +2471,11 @@ async def create_profile_endpoint(body: ProfileCreate):
|
|||||||
name=body.name,
|
name=body.name,
|
||||||
clone_from="default" if body.clone_from_default else None,
|
clone_from="default" if body.clone_from_default else None,
|
||||||
clone_config=body.clone_from_default,
|
clone_config=body.clone_from_default,
|
||||||
no_skills=body.no_skills,
|
|
||||||
)
|
)
|
||||||
# Match the CLI's profile-create flow: fresh named profiles get the
|
# Match the CLI's profile-create flow: fresh named profiles get the
|
||||||
# bundled skills installed. When cloning from default, create_profile()
|
# bundled skills installed. When cloning from default, create_profile()
|
||||||
# has already copied the source profile's skills, including any
|
# has already copied the source profile's skills, including any
|
||||||
# user-installed skills. When no_skills=True, create_profile() wrote
|
# user-installed skills.
|
||||||
# the opt-out marker and seed_profile_skills() will no-op.
|
|
||||||
if not body.clone_from_default:
|
if not body.clone_from_default:
|
||||||
profiles_mod.seed_profile_skills(path, quiet=True)
|
profiles_mod.seed_profile_skills(path, quiet=True)
|
||||||
|
|
||||||
@@ -2979,20 +2886,7 @@ async def get_models_analytics(days: int = 30):
|
|||||||
import re
|
import re
|
||||||
import asyncio
|
import asyncio
|
||||||
|
|
||||||
# PTY bridge is POSIX-only (depends on fcntl/termios/ptyprocess). On native
|
from hermes_cli.pty_bridge import PtyBridge, PtyUnavailableError
|
||||||
# Windows the import raises; catch and leave PtyBridge=None so the rest of
|
|
||||||
# the dashboard (sessions, jobs, metrics, config editor) still loads and the
|
|
||||||
# /api/pty endpoint cleanly refuses with a WSL-suggested message.
|
|
||||||
try:
|
|
||||||
from hermes_cli.pty_bridge import PtyBridge, PtyUnavailableError
|
|
||||||
_PTY_BRIDGE_AVAILABLE = True
|
|
||||||
except ImportError as _pty_import_err: # pragma: no cover - Windows-only path
|
|
||||||
PtyBridge = None # type: ignore[assignment]
|
|
||||||
_PTY_BRIDGE_AVAILABLE = False
|
|
||||||
|
|
||||||
class PtyUnavailableError(RuntimeError): # type: ignore[no-redef]
|
|
||||||
"""Stub on platforms where pty_bridge can't be imported."""
|
|
||||||
pass
|
|
||||||
|
|
||||||
_RESIZE_RE = re.compile(rb"\x1b\[RESIZE:(\d+);(\d+)\]")
|
_RESIZE_RE = re.compile(rb"\x1b\[RESIZE:(\d+);(\d+)\]")
|
||||||
_PTY_READ_CHUNK_TIMEOUT = 0.2
|
_PTY_READ_CHUNK_TIMEOUT = 0.2
|
||||||
@@ -3052,18 +2946,8 @@ def _resolve_chat_argv(
|
|||||||
argv, cwd = _make_tui_argv(PROJECT_ROOT / "ui-tui", tui_dev=False)
|
argv, cwd = _make_tui_argv(PROJECT_ROOT / "ui-tui", tui_dev=False)
|
||||||
env = os.environ.copy()
|
env = os.environ.copy()
|
||||||
env.setdefault("NODE_ENV", "production")
|
env.setdefault("NODE_ENV", "production")
|
||||||
# Browser-embedded chat should prefer stable wheel-based scrollback over
|
|
||||||
# native terminal mouse tracking. When mouse tracking is enabled, wheel
|
|
||||||
# events are consumed by the TUI and forwarded as terminal input, which
|
|
||||||
# makes browser-side transcript scrolling feel broken. Keep the terminal
|
|
||||||
# build unchanged for native CLI usage; only disable mouse tracking for
|
|
||||||
# the dashboard PTY path.
|
|
||||||
env.setdefault("HERMES_TUI_DISABLE_MOUSE", "1")
|
|
||||||
|
|
||||||
if resume:
|
if resume:
|
||||||
latest_resume, _latest_path = _session_latest_descendant(resume)
|
|
||||||
if latest_resume:
|
|
||||||
resume = latest_resume
|
|
||||||
env["HERMES_TUI_RESUME"] = resume
|
env["HERMES_TUI_RESUME"] = resume
|
||||||
|
|
||||||
if sidecar_url:
|
if sidecar_url:
|
||||||
@@ -3126,18 +3010,6 @@ async def pty_ws(ws: WebSocket) -> None:
|
|||||||
|
|
||||||
await ws.accept()
|
await ws.accept()
|
||||||
|
|
||||||
# On native Windows, the POSIX PTY bridge can't be imported. Tell the
|
|
||||||
# client and close cleanly rather than pretending the feature works.
|
|
||||||
if not _PTY_BRIDGE_AVAILABLE:
|
|
||||||
await ws.send_text(
|
|
||||||
"\r\n\x1b[31mChat unavailable: the embedded terminal requires a "
|
|
||||||
"POSIX PTY, which native Windows Python doesn't provide.\x1b[0m\r\n"
|
|
||||||
"\x1b[33mInstall Hermes inside WSL2 to use the dashboard's /chat "
|
|
||||||
"tab — the rest of the dashboard works here.\x1b[0m\r\n"
|
|
||||||
)
|
|
||||||
await ws.close(code=1011)
|
|
||||||
return
|
|
||||||
|
|
||||||
# --- spawn PTY ------------------------------------------------------
|
# --- spawn PTY ------------------------------------------------------
|
||||||
resume = ws.query_params.get("resume") or None
|
resume = ws.query_params.get("resume") or None
|
||||||
channel = _channel_or_close_code(ws)
|
channel = _channel_or_close_code(ws)
|
||||||
@@ -3333,42 +3205,12 @@ async def events_ws(ws: WebSocket) -> None:
|
|||||||
_event_channels.pop(channel, None)
|
_event_channels.pop(channel, None)
|
||||||
|
|
||||||
|
|
||||||
def _normalise_prefix(raw: Optional[str]) -> str:
|
|
||||||
"""Normalise an X-Forwarded-Prefix header value.
|
|
||||||
|
|
||||||
Returns a string like ``"/hermes"`` (no trailing slash) or ``""`` when
|
|
||||||
no prefix is set / the header is malformed. We deliberately reject
|
|
||||||
anything containing ``..`` or non-printable bytes so a hostile proxy
|
|
||||||
can't inject HTML via the prefix.
|
|
||||||
"""
|
|
||||||
if not raw:
|
|
||||||
return ""
|
|
||||||
p = raw.strip()
|
|
||||||
if not p:
|
|
||||||
return ""
|
|
||||||
if not p.startswith("/"):
|
|
||||||
p = "/" + p
|
|
||||||
p = p.rstrip("/")
|
|
||||||
if "//" in p or ".." in p or any(c in p for c in ('"', "'", "<", ">", " ", "\n", "\r", "\t")):
|
|
||||||
return ""
|
|
||||||
if len(p) > 64:
|
|
||||||
return ""
|
|
||||||
return p
|
|
||||||
|
|
||||||
|
|
||||||
def mount_spa(application: FastAPI):
|
def mount_spa(application: FastAPI):
|
||||||
"""Mount the built SPA. Falls back to index.html for client-side routing.
|
"""Mount the built SPA. Falls back to index.html for client-side routing.
|
||||||
|
|
||||||
The session token is injected into index.html via a ``<script>`` tag so
|
The session token is injected into index.html via a ``<script>`` tag so
|
||||||
the SPA can authenticate against protected API endpoints without a
|
the SPA can authenticate against protected API endpoints without a
|
||||||
separate (unauthenticated) token-dispensing endpoint.
|
separate (unauthenticated) token-dispensing endpoint.
|
||||||
|
|
||||||
When served behind a path-prefix reverse proxy (e.g.
|
|
||||||
``mission-control.tilos.com/hermes/*`` -> local Caddy -> :9119), the
|
|
||||||
proxy injects ``X-Forwarded-Prefix: /hermes`` on every request. We
|
|
||||||
rewrite the served ``index.html`` so absolute asset URLs (``/assets/...``)
|
|
||||||
and the SPA's runtime ``__HERMES_BASE_PATH__`` honour that prefix
|
|
||||||
without rebuilding the bundle.
|
|
||||||
"""
|
"""
|
||||||
if not WEB_DIST.exists():
|
if not WEB_DIST.exists():
|
||||||
@application.get("/{full_path:path}")
|
@application.get("/{full_path:path}")
|
||||||
@@ -3381,62 +3223,24 @@ def mount_spa(application: FastAPI):
|
|||||||
|
|
||||||
_index_path = WEB_DIST / "index.html"
|
_index_path = WEB_DIST / "index.html"
|
||||||
|
|
||||||
def _serve_index(prefix: str = ""):
|
def _serve_index():
|
||||||
"""Return index.html with the session token + base-path injected.
|
"""Return index.html with the session token injected."""
|
||||||
|
|
||||||
``prefix`` is the normalised ``X-Forwarded-Prefix`` (e.g. ``/hermes``)
|
|
||||||
or empty string when served at root.
|
|
||||||
"""
|
|
||||||
html = _index_path.read_text()
|
html = _index_path.read_text()
|
||||||
chat_js = "true" if _DASHBOARD_EMBEDDED_CHAT_ENABLED else "false"
|
chat_js = "true" if _DASHBOARD_EMBEDDED_CHAT_ENABLED else "false"
|
||||||
token_script = (
|
token_script = (
|
||||||
f'<script>window.__HERMES_SESSION_TOKEN__="{_SESSION_TOKEN}";'
|
f'<script>window.__HERMES_SESSION_TOKEN__="{_SESSION_TOKEN}";'
|
||||||
f"window.__HERMES_DASHBOARD_EMBEDDED_CHAT__={chat_js};"
|
f"window.__HERMES_DASHBOARD_EMBEDDED_CHAT__={chat_js};</script>"
|
||||||
f'window.__HERMES_BASE_PATH__="{prefix}";</script>'
|
|
||||||
)
|
)
|
||||||
if prefix:
|
|
||||||
# Rewrite absolute asset URLs baked into the Vite build so the
|
|
||||||
# browser fetches them through the same proxy prefix.
|
|
||||||
html = html.replace('href="/assets/', f'href="{prefix}/assets/')
|
|
||||||
html = html.replace('src="/assets/', f'src="{prefix}/assets/')
|
|
||||||
html = html.replace('href="/favicon.ico"', f'href="{prefix}/favicon.ico"')
|
|
||||||
html = html.replace('href="/fonts/', f'href="{prefix}/fonts/')
|
|
||||||
html = html.replace('href="/ds-assets/', f'href="{prefix}/ds-assets/')
|
|
||||||
html = html.replace('src="/ds-assets/', f'src="{prefix}/ds-assets/')
|
|
||||||
html = html.replace("</head>", f"{token_script}</head>", 1)
|
html = html.replace("</head>", f"{token_script}</head>", 1)
|
||||||
return HTMLResponse(
|
return HTMLResponse(
|
||||||
html,
|
html,
|
||||||
headers={"Cache-Control": "no-store, no-cache, must-revalidate"},
|
headers={"Cache-Control": "no-store, no-cache, must-revalidate"},
|
||||||
)
|
)
|
||||||
|
|
||||||
# When served behind a path-prefix proxy, the built CSS contains
|
|
||||||
# absolute ``url(/fonts/...)`` and ``url(/ds-assets/...)`` references.
|
|
||||||
# Browsers resolve those against the document origin, which means
|
|
||||||
# under ``/hermes`` they'd hit ``mission-control.tilos.com/fonts/...``
|
|
||||||
# (the MC Pages app), not the Hermes backend. Intercept CSS asset
|
|
||||||
# requests BEFORE the StaticFiles mount and rewrite the absolute paths
|
|
||||||
# when a prefix is in play.
|
|
||||||
@application.get("/assets/{filename}.css")
|
|
||||||
async def serve_css(filename: str, request: Request):
|
|
||||||
css_path = WEB_DIST / "assets" / f"{filename}.css"
|
|
||||||
if not css_path.is_file() or not css_path.resolve().is_relative_to(
|
|
||||||
WEB_DIST.resolve()
|
|
||||||
):
|
|
||||||
return JSONResponse({"error": "not found"}, status_code=404)
|
|
||||||
prefix = _normalise_prefix(request.headers.get("x-forwarded-prefix"))
|
|
||||||
css = css_path.read_text()
|
|
||||||
if prefix:
|
|
||||||
for asset_dir in ("/fonts/", "/fonts-terminal/", "/ds-assets/", "/assets/"):
|
|
||||||
css = css.replace(f"url({asset_dir}", f"url({prefix}{asset_dir}")
|
|
||||||
css = css.replace(f"url(\"{asset_dir}", f"url(\"{prefix}{asset_dir}")
|
|
||||||
css = css.replace(f"url('{asset_dir}", f"url('{prefix}{asset_dir}")
|
|
||||||
return Response(content=css, media_type="text/css")
|
|
||||||
|
|
||||||
application.mount("/assets", StaticFiles(directory=WEB_DIST / "assets"), name="assets")
|
application.mount("/assets", StaticFiles(directory=WEB_DIST / "assets"), name="assets")
|
||||||
|
|
||||||
@application.get("/{full_path:path}")
|
@application.get("/{full_path:path}")
|
||||||
async def serve_spa(full_path: str, request: Request):
|
async def serve_spa(full_path: str):
|
||||||
prefix = _normalise_prefix(request.headers.get("x-forwarded-prefix"))
|
|
||||||
file_path = WEB_DIST / full_path
|
file_path = WEB_DIST / full_path
|
||||||
# Prevent path traversal via url-encoded sequences (%2e%2e/)
|
# Prevent path traversal via url-encoded sequences (%2e%2e/)
|
||||||
if (
|
if (
|
||||||
@@ -3446,7 +3250,7 @@ def mount_spa(application: FastAPI):
|
|||||||
and file_path.is_file()
|
and file_path.is_file()
|
||||||
):
|
):
|
||||||
return FileResponse(file_path)
|
return FileResponse(file_path)
|
||||||
return _serve_index(prefix)
|
return _serve_index()
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
+2
-2
@@ -233,7 +233,7 @@ def is_wsl() -> bool:
|
|||||||
if _wsl_detected is not None:
|
if _wsl_detected is not None:
|
||||||
return _wsl_detected
|
return _wsl_detected
|
||||||
try:
|
try:
|
||||||
with open("/proc/version", "r", encoding="utf-8") as f:
|
with open("/proc/version", "r") as f:
|
||||||
_wsl_detected = "microsoft" in f.read().lower()
|
_wsl_detected = "microsoft" in f.read().lower()
|
||||||
except Exception:
|
except Exception:
|
||||||
_wsl_detected = False
|
_wsl_detected = False
|
||||||
@@ -260,7 +260,7 @@ def is_container() -> bool:
|
|||||||
_container_detected = True
|
_container_detected = True
|
||||||
return True
|
return True
|
||||||
try:
|
try:
|
||||||
with open("/proc/1/cgroup", "r", encoding="utf-8") as f:
|
with open("/proc/1/cgroup", "r") as f:
|
||||||
cgroup = f.read()
|
cgroup = f.read()
|
||||||
if "docker" in cgroup or "podman" in cgroup or "/lxc/" in cgroup:
|
if "docker" in cgroup or "podman" in cgroup or "/lxc/" in cgroup:
|
||||||
_container_detected = True
|
_container_detected = True
|
||||||
|
|||||||
@@ -612,11 +612,6 @@ class SessionDB:
|
|||||||
the caller already holds cumulative totals (gateway path, where the
|
the caller already holds cumulative totals (gateway path, where the
|
||||||
cached agent accumulates across messages).
|
cached agent accumulates across messages).
|
||||||
"""
|
"""
|
||||||
# Ensure the session row exists so the UPDATE doesn't silently affect
|
|
||||||
# 0 rows. Under concurrent load (cron + kanban + delegate_task) the
|
|
||||||
# initial create_session() may have failed due to SQLite locking.
|
|
||||||
# INSERT OR IGNORE is cheap and idempotent.
|
|
||||||
self._insert_session_row(session_id, "unknown", model=model)
|
|
||||||
if absolute:
|
if absolute:
|
||||||
sql = """UPDATE sessions SET
|
sql = """UPDATE sessions SET
|
||||||
input_tokens = ?,
|
input_tokens = ?,
|
||||||
|
|||||||
+1
-1
@@ -50,7 +50,7 @@ def _resolve_timezone_name() -> str:
|
|||||||
import yaml
|
import yaml
|
||||||
config_path = get_config_path()
|
config_path = get_config_path()
|
||||||
if config_path.exists():
|
if config_path.exists():
|
||||||
with open(config_path, encoding="utf-8") as f:
|
with open(config_path) as f:
|
||||||
cfg = yaml.safe_load(f) or {}
|
cfg = yaml.safe_load(f) or {}
|
||||||
tz_cfg = cfg.get("timezone", "")
|
tz_cfg = cfg.get("timezone", "")
|
||||||
if isinstance(tz_cfg, str) and tz_cfg.strip():
|
if isinstance(tz_cfg, str) and tz_cfg.strip():
|
||||||
|
|||||||
+2
-32
@@ -115,25 +115,6 @@ def _load_channel_directory() -> dict:
|
|||||||
return {}
|
return {}
|
||||||
|
|
||||||
|
|
||||||
def _coerce_int(
|
|
||||||
value,
|
|
||||||
*,
|
|
||||||
default: int,
|
|
||||||
minimum: int,
|
|
||||||
maximum: int,
|
|
||||||
) -> int:
|
|
||||||
"""Coerce value to int with fallback and clamping.
|
|
||||||
|
|
||||||
Used at MCP tool boundaries to handle invalid types from external clients.
|
|
||||||
Returns default if value cannot be converted to int.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
coerced = int(value)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
coerced = default
|
|
||||||
return max(minimum, min(coerced, maximum))
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_message_content(msg: dict) -> str:
|
def _extract_message_content(msg: dict) -> str:
|
||||||
"""Extract text content from a message, handling multi-part content."""
|
"""Extract text content from a message, handling multi-part content."""
|
||||||
content = msg.get("content", "")
|
content = msg.get("content", "")
|
||||||
@@ -484,7 +465,6 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
|||||||
limit: Maximum number of conversations to return (default 50)
|
limit: Maximum number of conversations to return (default 50)
|
||||||
search: Optional text to filter conversations by name
|
search: Optional text to filter conversations by name
|
||||||
"""
|
"""
|
||||||
limit = _coerce_int(limit, default=50, minimum=1, maximum=200)
|
|
||||||
entries = _load_sessions_index()
|
entries = _load_sessions_index()
|
||||||
conversations = []
|
conversations = []
|
||||||
|
|
||||||
@@ -572,7 +552,6 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
|||||||
session_key: The session key from conversations_list
|
session_key: The session key from conversations_list
|
||||||
limit: Maximum number of messages to return (default 50, most recent)
|
limit: Maximum number of messages to return (default 50, most recent)
|
||||||
"""
|
"""
|
||||||
limit = _coerce_int(limit, default=50, minimum=1, maximum=200)
|
|
||||||
entries = _load_sessions_index()
|
entries = _load_sessions_index()
|
||||||
entry = entries.get(session_key)
|
entry = entries.get(session_key)
|
||||||
if not entry:
|
if not entry:
|
||||||
@@ -685,8 +664,6 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
|||||||
session_key: Optional filter to one conversation
|
session_key: Optional filter to one conversation
|
||||||
limit: Maximum events to return (default 20)
|
limit: Maximum events to return (default 20)
|
||||||
"""
|
"""
|
||||||
after_cursor = _coerce_int(after_cursor, default=0, minimum=0, maximum=10**18)
|
|
||||||
limit = _coerce_int(limit, default=20, minimum=1, maximum=200)
|
|
||||||
result = bridge.poll_events(
|
result = bridge.poll_events(
|
||||||
after_cursor=after_cursor,
|
after_cursor=after_cursor,
|
||||||
session_key=session_key,
|
session_key=session_key,
|
||||||
@@ -712,17 +689,10 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
|||||||
session_key: Optional filter to one conversation
|
session_key: Optional filter to one conversation
|
||||||
timeout_ms: Maximum wait time in milliseconds (default 30000)
|
timeout_ms: Maximum wait time in milliseconds (default 30000)
|
||||||
"""
|
"""
|
||||||
after_cursor = _coerce_int(after_cursor, default=0, minimum=0, maximum=10**18)
|
|
||||||
timeout_ms = _coerce_int(
|
|
||||||
timeout_ms,
|
|
||||||
default=30000,
|
|
||||||
minimum=0,
|
|
||||||
maximum=300000,
|
|
||||||
) # Cap at 5 minutes
|
|
||||||
event = bridge.wait_for_event(
|
event = bridge.wait_for_event(
|
||||||
after_cursor=after_cursor,
|
after_cursor=after_cursor,
|
||||||
session_key=session_key,
|
session_key=session_key,
|
||||||
timeout_ms=timeout_ms,
|
timeout_ms=min(timeout_ms, 300000), # Cap at 5 minutes
|
||||||
)
|
)
|
||||||
if event:
|
if event:
|
||||||
return json.dumps({"event": event}, indent=2)
|
return json.dumps({"event": event}, indent=2)
|
||||||
@@ -802,7 +772,7 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
|||||||
return json.dumps({"count": len(targets), "channels": targets}, indent=2)
|
return json.dumps({"count": len(targets), "channels": targets}, indent=2)
|
||||||
|
|
||||||
channels = []
|
channels = []
|
||||||
for plat, entries_list in directory.get("platforms", {}).items():
|
for plat, entries_list in directory.items():
|
||||||
if platform and plat.lower() != platform.lower():
|
if platform and plat.lower() != platform.lower():
|
||||||
continue
|
continue
|
||||||
if isinstance(entries_list, list):
|
if isinstance(entries_list, list):
|
||||||
|
|||||||
+6
-6
@@ -730,8 +730,8 @@ def handle_function_call(
|
|||||||
session_id=session_id or "",
|
session_id=session_id or "",
|
||||||
tool_call_id=tool_call_id or "",
|
tool_call_id=tool_call_id or "",
|
||||||
)
|
)
|
||||||
except Exception as _hook_err:
|
except Exception:
|
||||||
logger.debug("pre_tool_call hook error: %s", _hook_err)
|
pass
|
||||||
|
|
||||||
if block_message is not None:
|
if block_message is not None:
|
||||||
return json.dumps({"error": block_message}, ensure_ascii=False)
|
return json.dumps({"error": block_message}, ensure_ascii=False)
|
||||||
@@ -782,8 +782,8 @@ def handle_function_call(
|
|||||||
tool_call_id=tool_call_id or "",
|
tool_call_id=tool_call_id or "",
|
||||||
duration_ms=duration_ms,
|
duration_ms=duration_ms,
|
||||||
)
|
)
|
||||||
except Exception as _hook_err:
|
except Exception:
|
||||||
logger.debug("post_tool_call hook error: %s", _hook_err)
|
pass
|
||||||
|
|
||||||
# Generic tool-result canonicalization seam: plugins receive the
|
# Generic tool-result canonicalization seam: plugins receive the
|
||||||
# final result string (JSON, usually) and may replace it by
|
# final result string (JSON, usually) and may replace it by
|
||||||
@@ -807,8 +807,8 @@ def handle_function_call(
|
|||||||
if isinstance(hook_result, str):
|
if isinstance(hook_result, str):
|
||||||
result = hook_result
|
result = hook_result
|
||||||
break
|
break
|
||||||
except Exception as _hook_err:
|
except Exception:
|
||||||
logger.debug("transform_tool_result hook error: %s", _hook_err)
|
pass
|
||||||
|
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
|||||||
@@ -1,432 +0,0 @@
|
|||||||
---
|
|
||||||
name: 3-statement-model
|
|
||||||
description: Build fully-integrated 3-statement models (IS, BS, CF) in Excel with working capital schedules, D&A roll-forwards, debt schedule, and the plugs that make cash and retained earnings tie. Pairs with excel-author.
|
|
||||||
version: 1.0.0
|
|
||||||
author: Anthropic (adapted by Nous Research)
|
|
||||||
license: Apache-2.0
|
|
||||||
metadata:
|
|
||||||
hermes:
|
|
||||||
tags: [finance, three-statement, income-statement, balance-sheet, cash-flow, excel, openpyxl, modeling]
|
|
||||||
related_skills: [excel-author, pptx-author, dcf-model, lbo-model]
|
|
||||||
---
|
|
||||||
|
|
||||||
## Environment
|
|
||||||
|
|
||||||
This skill assumes **headless openpyxl** — you are producing an .xlsx file on disk.
|
|
||||||
Follow the `excel-author` skill's conventions for cell coloring, formulas, named ranges, and sensitivity tables.
|
|
||||||
Recalculate before delivery: `python /path/to/excel-author/scripts/recalc.py ./out/model.xlsx`.
|
|
||||||
|
|
||||||
# 3-Statement Financial Model Template Completion
|
|
||||||
|
|
||||||
Complete and populate integrated financial model templates with proper linkages between Income Statement, Balance Sheet, and Cash Flow Statement.
|
|
||||||
|
|
||||||
## ⚠️ CRITICAL PRINCIPLES — Read Before Populating Any Template
|
|
||||||
|
|
||||||
**Formulas over hardcodes (non-negotiable):**
|
|
||||||
- Every projection cell, roll-forward, linkage, and subtotal MUST be an Excel formula — never a pre-computed value
|
|
||||||
- When using Python/openpyxl: write formula strings (`ws["D15"] = "=D14*(1+Assumptions!$B$5)"`), NOT computed results (`ws["D15"] = 12500`)
|
|
||||||
- The ONLY cells that should contain hardcoded numbers are: (1) historical actuals, (2) assumption drivers in the Assumptions tab
|
|
||||||
- If you find yourself computing a value in Python and writing the result to a cell — STOP. Write the formula instead.
|
|
||||||
- Why: the model must flex when scenarios toggle or assumptions change. Hardcodes break every downstream integrity check silently.
|
|
||||||
|
|
||||||
**Verify step-by-step with the user:**
|
|
||||||
1. **After mapping the template** → show the user which tabs/sections you've identified and confirm before touching any cells
|
|
||||||
2. **After populating historicals** → show the user the historical block and confirm values/periods match source data
|
|
||||||
3. **After building IS projections** → run the subtotal checks, show the user the projected IS, confirm before moving to BS
|
|
||||||
4. **After building BS** → show the user the balance check (Assets = L+E) for every period, confirm before moving to CF
|
|
||||||
5. **After building CF** → show the user the cash tie-out (CF ending cash = BS cash), confirm before finalizing
|
|
||||||
6. **Do NOT populate the entire model end-to-end and present it complete** — break at each statement, show the work, catch errors early
|
|
||||||
|
|
||||||
## Formatting — Professional Blue/Grey Palette (Default unless template/user specifies otherwise)
|
|
||||||
|
|
||||||
**Keep colors minimal.** Use only blues and greys for cell fills. Do NOT introduce greens, yellows, oranges, or multiple accent colors — a clean model uses restraint.
|
|
||||||
|
|
||||||
| Element | Fill | Font |
|
|
||||||
|---|---|---|
|
|
||||||
| Section headers (IS / BS / CF titles) | Dark blue `#1F4E79` | White bold |
|
|
||||||
| Column headers (FY2024A, FY2025E, etc.) | Light blue `#D9E1F2` | Black bold |
|
|
||||||
| Input cells (historicals, assumption drivers) | Light grey `#F2F2F2` or white | Blue `#0000FF` |
|
|
||||||
| Formula cells | White | Black |
|
|
||||||
| Cross-tab links | White | Green `#008000` |
|
|
||||||
| Check rows / key totals | Medium blue `#BDD7EE` | Black bold |
|
|
||||||
|
|
||||||
**That's 3 blues + 1 grey + white.** If the template has its own color scheme, follow the template instead.
|
|
||||||
|
|
||||||
Font color signals *what* a cell is (input/formula/link). Fill color signals *where* you are (header/data/check).
|
|
||||||
|
|
||||||
## Model Structure
|
|
||||||
|
|
||||||
### Identifying Template Tab Organization
|
|
||||||
|
|
||||||
Templates vary in their tab naming conventions and organization. Before populating, review all tabs to understand the template's structure. Below are common tab names and their typical contents:
|
|
||||||
|
|
||||||
| Common Tab Names | Contents to Look For |
|
|
||||||
|------------------|----------------------|
|
|
||||||
| IS, P&L, Income Statement | Income Statement |
|
|
||||||
| BS, Balance Sheet | Balance Sheet |
|
|
||||||
| CF, CFS, Cash Flow | Cash Flow Statement |
|
|
||||||
| WC, Working Capital | Working Capital Schedule |
|
|
||||||
| DA, D&A, Depreciation, PP&E | Depreciation & Amortization Schedule |
|
|
||||||
| Debt, Debt Schedule | Debt Schedule |
|
|
||||||
| NOL, Tax, DTA | Net Operating Loss Schedule |
|
|
||||||
| Assumptions, Inputs, Drivers | Driver assumptions and inputs |
|
|
||||||
| Checks, Audit, Validation | Error-checking dashboard |
|
|
||||||
|
|
||||||
**Template Review Checklist**
|
|
||||||
- Identify which tabs exist in the template (not all templates include every schedule)
|
|
||||||
- Note any template-specific tabs not listed above
|
|
||||||
- Understand tab dependencies (e.g., which schedules feed into the main statements)
|
|
||||||
- Locate input cells vs. formula cells on each tab
|
|
||||||
|
|
||||||
### Understanding Template Structure
|
|
||||||
|
|
||||||
Before populating a template, familiarize yourself with its existing layout to ensure data is entered in the correct locations and formulas remain intact.
|
|
||||||
|
|
||||||
**Identifying Row Structure**
|
|
||||||
- Locate the model title at top of each tab
|
|
||||||
- Identify section headers and their visual separation
|
|
||||||
- Find the units row indicating $ millions, %, x, etc.
|
|
||||||
- Note column headers distinguishing Actuals vs. Estimates periods
|
|
||||||
- Confirm period labels (e.g., FY2024A, FY2025E)
|
|
||||||
- Identify input cells vs. formula cells (typically distinguished by font color)
|
|
||||||
|
|
||||||
**Identifying Column Structure**
|
|
||||||
- Confirm line item labels in leftmost column
|
|
||||||
- Verify historical years precede projection years
|
|
||||||
- Note the visual border separating historical from projected periods
|
|
||||||
- Check for consistent column order across all tabs
|
|
||||||
|
|
||||||
**Working with Named Ranges**
|
|
||||||
Templates often use named ranges for key inputs and outputs. Before entering data:
|
|
||||||
- Review existing named ranges in the template (Formulas → Name Manager in Excel)
|
|
||||||
- Common named ranges include: Revenue growth rates, cost percentages, key outputs (Net Income, EBITDA, Total Debt, Cash), scenario selector cell
|
|
||||||
- Ensure inputs are entered in cells that feed into these named ranges
|
|
||||||
|
|
||||||
### Projection Period
|
|
||||||
- Templates typically project 5 years forward from last historical year
|
|
||||||
- Verify historical (A) vs. projected (E) columns are clearly separated
|
|
||||||
- Confirm columns use fiscal year notation (e.g., FY2024A, FY2025E)
|
|
||||||
|
|
||||||
## Margin Analysis
|
|
||||||
|
|
||||||
**Note: The following margin analysis should only be performed if prompted by the user or if the template explicitly requires it. If no prompt is given, skip this section.**
|
|
||||||
|
|
||||||
Calculate and display profitability margins on the Income Statement (IS) tab to track operational efficiency and enable peer comparison.
|
|
||||||
|
|
||||||
### Core Margins to Include
|
|
||||||
|
|
||||||
| Margin | Formula | What It Measures |
|
|
||||||
|--------|---------|------------------|
|
|
||||||
| Gross Margin | Gross Profit / Revenue | Pricing power, production efficiency |
|
|
||||||
| EBITDA Margin | EBITDA / Revenue | Core operating profitability |
|
|
||||||
| EBIT Margin | EBIT / Revenue | Operating profitability after D&A |
|
|
||||||
| Net Income Margin | Net Income / Revenue | Bottom-line profitability |
|
|
||||||
|
|
||||||
### Income Statement Layout with Margins
|
|
||||||
|
|
||||||
Display margin percentages directly below each profit line item:
|
|
||||||
- Gross Margin % below Gross Profit
|
|
||||||
- EBIT Margin % below EBIT
|
|
||||||
- EBITDA Margin % below EBITDA
|
|
||||||
- Net Income Margin % below Net Income
|
|
||||||
|
|
||||||
## Credit Metrics
|
|
||||||
|
|
||||||
**Note: The following Credit analysis should only be performed if prompted by the user or if the template explicitly requires it. If no prompt is given, skip this section.**
|
|
||||||
|
|
||||||
Calculate and display credit/leverage metrics on the Balance Sheet (BS) tab to assess financial health, debt capacity, and covenant compliance.
|
|
||||||
|
|
||||||
### Core Credit Metrics to Include
|
|
||||||
|
|
||||||
| Metric | Formula | What It Measures |
|
|
||||||
|--------|---------|------------------|
|
|
||||||
| Total Debt / EBITDA | Total Debt / LTM EBITDA | Leverage multiple |
|
|
||||||
| Net Debt / EBITDA | (Total Debt - Cash) / LTM EBITDA | Leverage net of cash |
|
|
||||||
| Interest Coverage | EBITDA / Interest Expense | Ability to service debt |
|
|
||||||
| Debt / Total Cap | Total Debt / (Total Debt + Equity) | Capital structure |
|
|
||||||
| Debt / Equity | Total Debt / Total Equity | Financial leverage |
|
|
||||||
| Current Ratio | Current Assets / Current Liabilities | Short-term liquidity |
|
|
||||||
| Quick Ratio | (Current Assets - Inventory) / Current Liabilities | Immediate liquidity |
|
|
||||||
|
|
||||||
### Credit Metric Hierarchy Checks
|
|
||||||
|
|
||||||
Validate that Upside shows strongest credit profile:
|
|
||||||
- Leverage: Upside < Base < Downside (lower is better)
|
|
||||||
- Coverage: Upside > Base > Downside (higher is better)
|
|
||||||
- Liquidity: Upside > Base > Downside (higher is better)
|
|
||||||
|
|
||||||
### Covenant Compliance Tracking
|
|
||||||
|
|
||||||
If debt covenants are known, add explicit compliance checks comparing actual metrics to covenant thresholds.
|
|
||||||
|
|
||||||
## Scenario Analysis (Base / Upside / Downside)
|
|
||||||
|
|
||||||
Use a scenario toggle (dropdown) in the Assumptions tab with CHOOSE or INDEX/MATCH formulas.
|
|
||||||
|
|
||||||
| Scenario | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| Base Case | Management guidance or consensus estimates |
|
|
||||||
| Upside Case | Above-guidance growth, margin expansion |
|
|
||||||
| Downside Case | Below-trend growth, margin compression |
|
|
||||||
|
|
||||||
**Key Drivers to Sensitize**: Revenue growth, Gross margin, SG&A %, DSO/DIO/DPO, CapEx %, Interest rate, Tax rate.
|
|
||||||
|
|
||||||
**Scenario Audit Checks**: Toggle switches all statements, BS balances in all scenarios, Cash ties out, Hierarchy holds (Upside > Base > Downside for NI, EBITDA, FCF, margins).
|
|
||||||
|
|
||||||
## SEC Filings Data Extraction
|
|
||||||
|
|
||||||
If the template specifically requires pulling data from SEC filings (10-K, 10-Q), see [references/sec-filings.md](references/sec-filings.md) for detailed extraction guidance. This reference is only needed when populating templates with public company data from regulatory filings.
|
|
||||||
|
|
||||||
## Completing Model Templates
|
|
||||||
|
|
||||||
This section provides general guidance for completing any 3-statement financial model template while preserving existing formulas and ensuring data integrity.
|
|
||||||
|
|
||||||
### Step 1: Analyze the Template Structure
|
|
||||||
|
|
||||||
Before entering any data, thoroughly review the template to understand its architecture:
|
|
||||||
|
|
||||||
**Identify Input vs. Formula Cells**
|
|
||||||
- Look for visual cues (font color, cell shading) that distinguish input cells from formula cells
|
|
||||||
- Common conventions: Blue font = inputs, Black font = formulas, Green font = links to other sheets
|
|
||||||
- Use Excel's Trace Precedents/Dependents (Formulas → Trace Precedents) to understand cell relationships
|
|
||||||
- Check for named ranges that may control key inputs (Formulas → Name Manager)
|
|
||||||
|
|
||||||
**Map the Template's Flow**
|
|
||||||
- Identify which tabs feed into others (e.g., Assumptions → IS → BS → CF)
|
|
||||||
- Note any supporting schedules and their linkages to main statements
|
|
||||||
- Document the template's specific line items and structure before populating
|
|
||||||
|
|
||||||
### Step 2: Filling in Data Without Breaking Formulas
|
|
||||||
|
|
||||||
**Golden Rules for Data Entry**
|
|
||||||
|
|
||||||
| Rule | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| Only edit input cells | Never overwrite cells containing formulas unless intentionally replacing the formula |
|
|
||||||
| Preserve cell references | When copying data, use Paste Values (Ctrl+Shift+V) to avoid overwriting formulas with source formatting |
|
|
||||||
| Match the template's units | Verify if template uses thousands, millions, or actual values before entering data |
|
|
||||||
| Respect sign conventions | Follow the template's existing sign convention (e.g., expenses as positive or negative) |
|
|
||||||
| Check for circular references | If the template uses iterative calculations, ensure Enable Iterative Calculation is turned on |
|
|
||||||
|
|
||||||
**Safe Data Entry Process**
|
|
||||||
1. Identify the exact cells designated for input (usually highlighted or labeled)
|
|
||||||
2. Enter historical data first, then verify formulas are calculating correctly for those periods
|
|
||||||
3. Enter assumption drivers that feed forecast calculations
|
|
||||||
4. Review calculated outputs to confirm formulas are working as intended
|
|
||||||
5. If a formula cell must be modified, document the original formula before making changes
|
|
||||||
|
|
||||||
**Handling Pre-Built Formulas**
|
|
||||||
- If formulas reference cells you haven't populated yet, expect temporary errors (#REF!, #DIV/0!) until all inputs are complete
|
|
||||||
- When formulas produce unexpected results, trace precedents to identify missing or incorrect inputs
|
|
||||||
- Never delete rows/columns without checking for formula dependencies across all tabs
|
|
||||||
|
|
||||||
### Step 3: Validating Formulas
|
|
||||||
|
|
||||||
**Formula Integrity Checks**
|
|
||||||
|
|
||||||
Before relying on template outputs, validate that formulas are functioning correctly:
|
|
||||||
|
|
||||||
| Check Type | Method |
|
|
||||||
|------------|--------|
|
|
||||||
| Trace precedents | Select a formula cell → Formulas → Trace Precedents to verify it references correct inputs |
|
|
||||||
| Trace dependents | Verify key inputs flow to expected output cells |
|
|
||||||
| Evaluate formula | Use Formulas → Evaluate Formula to step through complex calculations |
|
|
||||||
| Check for hardcodes | Projection formulas should reference assumptions, not contain hardcoded values |
|
|
||||||
| Test with known values | Input simple test values to verify formulas produce expected results |
|
|
||||||
| Cross-tab consistency | Ensure the same formula logic applies across all projection periods |
|
|
||||||
|
|
||||||
**Common Formula Issues to Watch For**
|
|
||||||
- Mixed absolute/relative references causing incorrect results when copied across periods
|
|
||||||
- Broken links to external files or deleted ranges (#REF! errors)
|
|
||||||
- Division by zero in early periods before revenue ramps (#DIV/0! errors)
|
|
||||||
- Circular reference warnings (may be intentional for interest calculations)
|
|
||||||
- Inconsistent formulas across projection columns (use Ctrl+\ to find differences)
|
|
||||||
|
|
||||||
**Validating Cross-Tab Linkages**
|
|
||||||
- Confirm values that appear on multiple tabs are linked (not duplicated)
|
|
||||||
- Verify schedule totals tie to corresponding line items on main statements
|
|
||||||
- Check that period labels align across all tabs
|
|
||||||
|
|
||||||
### Step 4: Quality Checks by Sheet
|
|
||||||
|
|
||||||
Perform these validation checks on each sheet after populating the template:
|
|
||||||
|
|
||||||
**Income Statement (IS) Quality Checks**
|
|
||||||
- Revenue figures match source data for historical periods
|
|
||||||
- All expense line items sum to reported totals
|
|
||||||
- Subtotals (Gross Profit, EBIT, EBT, Net Income) calculate correctly
|
|
||||||
- Tax calculation logic is appropriate (handles losses correctly)
|
|
||||||
- Forecast drivers reference assumptions tab (no hardcodes)
|
|
||||||
- Period-over-period changes are directionally reasonable
|
|
||||||
|
|
||||||
**Balance Sheet (BS) Quality Checks**
|
|
||||||
- Assets = Liabilities + Equity for every period (primary check)
|
|
||||||
- Cash balance matches Cash Flow Statement ending cash
|
|
||||||
- Working capital accounts tie to supporting schedules (if applicable)
|
|
||||||
- Retained Earnings rolls forward correctly: Prior RE + Net Income - Dividends +/- Adjustments = Ending RE
|
|
||||||
- Debt balances tie to debt schedule (if applicable)
|
|
||||||
- All balance sheet items have appropriate signs (assets positive, most liabilities positive)
|
|
||||||
|
|
||||||
**Cash Flow Statement (CF) Quality Checks**
|
|
||||||
- Net Income at top of CFO matches Income Statement Net Income
|
|
||||||
- Non-cash add-backs (D&A, SBC, etc.) tie to their source schedules/statements
|
|
||||||
- Working capital changes have correct signs (increase in asset = use of cash = negative)
|
|
||||||
- CapEx ties to PP&E schedule or fixed asset roll-forward
|
|
||||||
- Financing activities tie to changes in debt and equity accounts on BS
|
|
||||||
- Ending Cash matches Balance Sheet Cash
|
|
||||||
- Beginning Cash equals prior period Ending Cash
|
|
||||||
|
|
||||||
**Supporting Schedule Quality Checks**
|
|
||||||
- Opening balances equal prior period closing balances
|
|
||||||
- Roll-forward logic is complete (Beginning + Additions - Deductions = Ending)
|
|
||||||
- Schedule totals tie to main statement line items
|
|
||||||
- Assumptions used in calculations match Assumptions tab
|
|
||||||
|
|
||||||
### Step 5: Cross-Statement Integrity Checks
|
|
||||||
|
|
||||||
After validating individual sheets, confirm the three statements are properly integrated:
|
|
||||||
|
|
||||||
| Check | Formula | Expected Result |
|
|
||||||
|-------|---------|-----------------|
|
|
||||||
| Balance Sheet Balance | Assets - Liabilities - Equity | = 0 |
|
|
||||||
| Cash Tie-Out | CF Ending Cash - BS Cash | = 0 |
|
|
||||||
| Net Income Link | IS Net Income - CF Starting Net Income | = 0 |
|
|
||||||
| Retained Earnings | Prior RE + NI - Dividends - BS Ending RE | = 0 (adjust for SBC/other items as needed) |
|
|
||||||
|
|
||||||
### Step 6: Final Review
|
|
||||||
|
|
||||||
Before considering the model complete:
|
|
||||||
- Toggle through all scenarios (if applicable) to verify checks pass in each case
|
|
||||||
- Review all #REF!, #DIV/0!, #VALUE!, and #NAME? errors and resolve or document
|
|
||||||
- Confirm all input cells have been populated (search for placeholder values)
|
|
||||||
- Verify units are consistent across all tabs
|
|
||||||
- Save a clean version before making any additional modifications
|
|
||||||
|
|
||||||
## Model Validation and Audit
|
|
||||||
|
|
||||||
This section consolidates all validation checks and audit procedures for completed templates.
|
|
||||||
|
|
||||||
### Core Linkages (Must Always Hold)
|
|
||||||
|
|
||||||
See [references/formulas.md](references/formulas.md) for all formula details.
|
|
||||||
|
|
||||||
| Check | Formula | Expected Result |
|
|
||||||
|-------|---------|-----------------|
|
|
||||||
| Balance Sheet Balance | Assets - Liabilities - Equity | = 0 |
|
|
||||||
| Cash Tie-Out | CF Ending Cash - BS Cash | = 0 |
|
|
||||||
| Cash Monthly vs Annual | Closing Cash (Monthly) - Closing Cash (Annual) | = 0 |
|
|
||||||
| Net Income Link | IS Net Income - CF Starting Net Income | = 0 |
|
|
||||||
| Retained Earnings | Prior RE + NI + SBC - Dividends - BS Ending RE | = 0 |
|
|
||||||
| Equity Financing | ΔCommon Stock/APIC (BS) - Equity Issuance (CFF) | = 0 |
|
|
||||||
| Year 0 Equity | Equity Raised (Year 0) - Beginning Equity Capital (Year 1) | = 0 |
|
|
||||||
|
|
||||||
### Sign Convention Reference
|
|
||||||
|
|
||||||
| Statement | Item | Sign Convention |
|
|
||||||
|-----------|------|-----------------|
|
|
||||||
| CFO | D&A, SBC | Positive (add-back) |
|
|
||||||
| CFO | ΔAR (increase) | Negative (use of cash) |
|
|
||||||
| CFO | ΔAP (increase) | Positive (source of cash) |
|
|
||||||
| CFI | CapEx | Negative |
|
|
||||||
| CFF | Debt issuance | Positive |
|
|
||||||
| CFF | Debt repayments | Negative |
|
|
||||||
| CFF | Dividends | Negative |
|
|
||||||
|
|
||||||
### Circular Reference Handling
|
|
||||||
|
|
||||||
Interest expense creates circularity: Interest → Net Income → Cash → Debt Balance → Interest
|
|
||||||
|
|
||||||
Enable iterative calculation in Excel: File → Options → Formulas → Enable iterative calculation. Set maximum iterations to 100, maximum change to 0.001. Add a circuit breaker toggle in Assumptions tab.
|
|
||||||
|
|
||||||
### Check Categories
|
|
||||||
|
|
||||||
**Section 1: Currency Consistency**
|
|
||||||
- Currency identified and documented in Assumptions
|
|
||||||
- All tabs use consistent currency symbol and scale
|
|
||||||
- Units row matches model currency
|
|
||||||
|
|
||||||
**Section 2: Balance Sheet Integrity**
|
|
||||||
- Assets = Liabilities + Equity (for each period)
|
|
||||||
- Formula: Assets - Liabilities - Equity (must = 0)
|
|
||||||
|
|
||||||
**Section 3: Cash Flow Integrity**
|
|
||||||
- Cash ties to BS (CF Ending Cash = BS Cash)
|
|
||||||
- Cash Monthly vs Annual: Closing Cash (Monthly) = Closing Cash (Annual)
|
|
||||||
- NI ties to IS (CF Net Income = IS Net Income)
|
|
||||||
- D&A ties to schedule
|
|
||||||
- SBC ties to IS
|
|
||||||
- ΔAR, ΔInventory, ΔAP tie to WC schedule
|
|
||||||
- CapEx ties to DA schedule
|
|
||||||
|
|
||||||
**Section 4: Retained Earnings**
|
|
||||||
- RE roll-forward check: Prior RE + NI + SBC - Dividends = Ending RE
|
|
||||||
- Show component breakdown for debugging
|
|
||||||
|
|
||||||
**Section 5: Working Capital**
|
|
||||||
- AR, Inventory, AP tie to BS
|
|
||||||
- DSO, DIO, DPO reasonability checks (flag if outside normal ranges)
|
|
||||||
|
|
||||||
**Section 6: Debt Schedule**
|
|
||||||
- Total Debt ties to BS (Current + LT Debt)
|
|
||||||
- Interest calculation ties to IS
|
|
||||||
|
|
||||||
**Section 6b: Equity Financing**
|
|
||||||
- Equity issuance proceeds tie to BS Common Stock/APIC increase
|
|
||||||
- Cash increase from equity = Equity account increase (must balance)
|
|
||||||
- Equity Raise Tie-Out: ΔCommon Stock/APIC (BS) = Equity Issuance (CFF) (must = 0)
|
|
||||||
- Year 0 Equity Tie-Out: Equity Raised (Year 0) = Beginning Equity Capital (Year 1)
|
|
||||||
|
|
||||||
**Section 6c: NOL Schedule**
|
|
||||||
- Beginning NOL (Year 1 / Formation) = 0 (new business starts with zero NOL)
|
|
||||||
- NOL increases only when EBT < 0 (losses must be realized to generate NOL)
|
|
||||||
- DTA ties to BS (NOL Schedule DTA = BS Deferred Tax Asset)
|
|
||||||
- NOL utilization ≤ 80% of EBT (post-2017 federal limitation)
|
|
||||||
- NOL balance is non-negative (cannot utilize more than available)
|
|
||||||
- NOL generated only when EBT < 0
|
|
||||||
- Tax expense = 0 when taxable income ≤ 0
|
|
||||||
|
|
||||||
**Section 7: Scenario Hierarchy**
|
|
||||||
- Absolute metrics: Upside > Base > Downside (NI, EBITDA, FCF)
|
|
||||||
- Margins: Upside > Base > Downside (GM%, EBITDA%, NI%)
|
|
||||||
- Credit metrics: Upside < Base < Downside for leverage (inverted)
|
|
||||||
|
|
||||||
**Section 8: Formula Integrity**
|
|
||||||
- COGS, S&M, G&A, R&D, SBC driven by % of Revenue (no hardcodes)
|
|
||||||
- Consistent formulas across projection years
|
|
||||||
- No #REF!, #DIV/0!, #VALUE! errors
|
|
||||||
|
|
||||||
**Section 9: Credit Metric Thresholds**
|
|
||||||
- Flag metrics as Green/Yellow/Red based on covenant thresholds
|
|
||||||
- Summary of any red flags
|
|
||||||
|
|
||||||
### Master Check Formula
|
|
||||||
|
|
||||||
Aggregate all section statuses into a single master check:
|
|
||||||
- If all sections pass → "✓ ALL CHECKS PASS"
|
|
||||||
- If any section fails → "✗ ERRORS DETECTED - REVIEW BELOW"
|
|
||||||
|
|
||||||
### Quick Debug Workflow
|
|
||||||
|
|
||||||
When Master Status shows errors:
|
|
||||||
1. Scroll to find red-highlighted sections
|
|
||||||
2. Identify which check category has failures
|
|
||||||
3. Navigate to source tab to investigate
|
|
||||||
4. Fix the underlying issue
|
|
||||||
5. Return to Checks tab to verify resolution
|
|
||||||
|
|
||||||
|
|
||||||
## Data sources — MCP first, web fallback
|
|
||||||
|
|
||||||
Many passages below say "use the S&P Kensho MCP / Daloopa MCP / FactSet MCP". Those are commercial financial-data MCPs from the original Cowork plugin context. In Hermes:
|
|
||||||
|
|
||||||
- **If you have any structured financial-data MCP configured** (Hermes supports MCP — see `native-mcp` skill), prefer it for point-in-time comps, precedent transactions, and filings.
|
|
||||||
- **Otherwise**, fall back to:
|
|
||||||
- `web_search` / `web_extract` against SEC EDGAR (`https://www.sec.gov/cgi-bin/browse-edgar`) for US filings
|
|
||||||
- Company IR pages for press releases, earnings decks
|
|
||||||
- `browser_navigate` for interactive data portals
|
|
||||||
- User-provided data (explicitly ask when the context doesn't have it)
|
|
||||||
- **Never fabricate**. If a multiple, precedent, or filing number can't be sourced, flag the cell as `[UNSOURCED]` and surface it to the user.
|
|
||||||
|
|
||||||
## Attribution
|
|
||||||
|
|
||||||
This skill is adapted from Anthropic's Claude for Financial Services plugin suite (Apache-2.0). The Office-JS / Cowork live-Excel paths have been removed; this version targets headless openpyxl via the `excel-author` skill's conventions. Original: https://github.com/anthropics/financial-services
|
|
||||||
@@ -1,118 +0,0 @@
|
|||||||
# Formatting Standards Reference
|
|
||||||
|
|
||||||
| Element | Format |
|
|
||||||
|---------|--------|
|
|
||||||
| Hard-coded inputs | Blue font |
|
|
||||||
| Formulas | Black font |
|
|
||||||
| Links to other sheets | Green font |
|
|
||||||
| Check cells | Red if error, green if balanced |
|
|
||||||
| Negative values | Parentheses, not minus signs |
|
|
||||||
| Currency | No decimals for large figures, 2 decimals for per-share |
|
|
||||||
| Percentages | 1 decimal place |
|
|
||||||
| Headers | Bold, bottom border |
|
|
||||||
| Units row | Include units row below headers ($ millions, %, etc.) |
|
|
||||||
|
|
||||||
## Visual Separation Guidelines
|
|
||||||
|
|
||||||
- Thin vertical border between historical and projected columns
|
|
||||||
- Thick bottom border after section totals (e.g., Total Assets)
|
|
||||||
- Single bottom border for subtotals
|
|
||||||
- Double bottom border for grand totals
|
|
||||||
|
|
||||||
## Total and Subtotal Row Formatting
|
|
||||||
|
|
||||||
All total and subtotal rows must use **bold font formatting** for their numerical values to clearly distinguish aggregated figures from individual line items.
|
|
||||||
|
|
||||||
### Income Statement (P&L) Tab
|
|
||||||
| Row | Formatting |
|
|
||||||
|-----|------------|
|
|
||||||
| Gross Revenue | Bold |
|
|
||||||
| Total Cost of Revenue | Bold |
|
|
||||||
| Gross Profit | Bold |
|
|
||||||
| Total SG&A | Bold |
|
|
||||||
| EBITDA | Bold |
|
|
||||||
| EBIT | Bold |
|
|
||||||
| EBT | Bold |
|
|
||||||
| Net Profit After Tax | Bold |
|
|
||||||
|
|
||||||
### Balance Sheet Tab
|
|
||||||
| Row | Formatting |
|
|
||||||
|-----|------------|
|
|
||||||
| Total Current Assets | Bold |
|
|
||||||
| Total Non-Current Assets | Bold |
|
|
||||||
| Total Other Assets | Bold |
|
|
||||||
| Total Assets | Bold |
|
|
||||||
| Total Current Liabilities | Bold |
|
|
||||||
| Total Non-Current Liabilities | Bold |
|
|
||||||
| Total Equity | Bold |
|
|
||||||
| Total Liabilities and Equity | Bold |
|
|
||||||
|
|
||||||
### Cash Flow Statement Tab
|
|
||||||
| Row | Formatting |
|
|
||||||
|-----|------------|
|
|
||||||
| Cash Generated from Operations Before Working Capital Changes | Bold |
|
|
||||||
| Total Working Capital Changes | Bold |
|
|
||||||
| Net Cash Generated from Operations | Bold |
|
|
||||||
| Net Cash Flow from Investing Activities | Bold |
|
|
||||||
| Net Cash Flow from Financing Activities | Bold |
|
|
||||||
| Closing Cash Balance | Bold |
|
|
||||||
|
|
||||||
**Note:** This list is non-exhaustive. Apply bold formatting to any row that represents a total, subtotal, or summary calculation across the model.
|
|
||||||
|
|
||||||
## Balance Sheet Check Row Formatting
|
|
||||||
|
|
||||||
The Balance Sheet check row (below Total Liabilities and Equity) uses conditional number formatting that displays non-zero values in red. When the balance sheet balances correctly (check = 0), the values display in black or standard formatting.
|
|
||||||
|
|
||||||
| Check Value | Font Color |
|
|
||||||
|-------------|------------|
|
|
||||||
| = 0 (balanced) | Black (standard) |
|
|
||||||
| ≠ 0 (error) | Red |
|
|
||||||
|
|
||||||
**Implementation:** Apply custom number format `[Red][<>0]0.00;[Red][<>0](0.00);0.00` or use Excel conditional formatting with the rule "Cell Value ≠ 0" → Red font.
|
|
||||||
|
|
||||||
## Margin Row Formatting
|
|
||||||
|
|
||||||
| Element | Format |
|
|
||||||
|---------|--------|
|
|
||||||
| Margin % rows | Indent, italics, 1 decimal place |
|
|
||||||
| Positive trend | No special formatting (or subtle green) |
|
|
||||||
| Negative trend | Flag for review (subtle yellow) |
|
|
||||||
| Below peer average | Consider highlighting for discussion |
|
|
||||||
|
|
||||||
## Credit Metric Formatting
|
|
||||||
|
|
||||||
| Element | Format |
|
|
||||||
|---------|--------|
|
|
||||||
| Leverage multiples | 1 decimal with "x" suffix (e.g., 2.5x) |
|
|
||||||
| Percentages | 1 decimal with "%" suffix |
|
|
||||||
| Net Debt negative | Parentheses, indicates net cash position |
|
|
||||||
| Section header | Bold, "CREDIT METRICS" |
|
|
||||||
| Separator line | Thin border above credit metrics section |
|
|
||||||
|
|
||||||
## Credit Metric Threshold Colors
|
|
||||||
|
|
||||||
| Metric | Green | Yellow | Red |
|
|
||||||
|--------|-------|--------|-----|
|
|
||||||
| Total Debt / EBITDA | < 2.5x | 2.5x-4.0x | > 4.0x |
|
|
||||||
| Net Debt / EBITDA | < 2.0x | 2.0x-3.5x | > 3.5x |
|
|
||||||
| Interest Coverage | > 4.0x | 2.5x-4.0x | < 2.5x |
|
|
||||||
| Debt / Total Cap | < 40% | 40%-60% | > 60% |
|
|
||||||
| Current Ratio | > 1.5x | 1.0x-1.5x | < 1.0x |
|
|
||||||
| Quick Ratio | > 1.0x | 0.75x-1.0x | < 0.75x |
|
|
||||||
|
|
||||||
## Conditional Formatting for Checks Tab
|
|
||||||
|
|
||||||
- Cell contains pass indicator → Green fill
|
|
||||||
- Cell contains fail indicator → Red fill
|
|
||||||
- Cell contains warning → Yellow fill
|
|
||||||
- Difference cells = 0 → Light green fill
|
|
||||||
- Difference cells ≠ 0 → Light red fill
|
|
||||||
|
|
||||||
## Margin Reasonability Flags
|
|
||||||
|
|
||||||
- Gross Margin < 0% → ERROR: Review COGS
|
|
||||||
- Gross Margin > 80% → WARNING: Verify revenue/COGS
|
|
||||||
- EBITDA Margin < 0% → FLAG: Operating losses
|
|
||||||
- EBITDA Margin > 50% → WARNING: Unusually high
|
|
||||||
- Net Margin < 0% → FLAG: Net losses (may be acceptable in growth phase)
|
|
||||||
- Net Margin > Gross Margin → ERROR: Formula issue
|
|
||||||
@@ -1,292 +0,0 @@
|
|||||||
# Formula Reference
|
|
||||||
|
|
||||||
**IMPORTANT:** Use the formulas outlined in this reference document unless otherwise specified by the user.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Core Linkages
|
|
||||||
|
|
||||||
```
|
|
||||||
Balance Sheet: Assets = Liabilities + Equity
|
|
||||||
Net Income: IS Net Income → CF Operations (starting point)
|
|
||||||
Cash Flow: ΔCash = CFO + CFI + CFF
|
|
||||||
Cash Tie-Out: Ending Cash (CF) = Cash (BS Asset)
|
|
||||||
Cash Monthly/Annual: Closing Cash (Monthly) = Closing Cash (Annual)
|
|
||||||
Retained Earnings: Prior RE + Net Income - Dividends = Ending RE
|
|
||||||
Equity Raise: ΔCommon Stock/APIC (BS) = Equity Issuance (CFF)
|
|
||||||
Year 0 Equity: Equity Raised (Year 0) = Beginning Equity (Year 1)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Gross Profit Calculation
|
|
||||||
|
|
||||||
**IMPORTANT:** Gross Profit must be calculated from Net Revenue, not Gross Revenue.
|
|
||||||
|
|
||||||
```
|
|
||||||
Net Revenue - Cost of Revenue = Gross Profit
|
|
||||||
```
|
|
||||||
|
|
||||||
| Term | Definition |
|
|
||||||
|------|------------|
|
|
||||||
| Gross Revenue | Total revenue before any deductions |
|
|
||||||
| Net Revenue | Gross Revenue - Returns - Allowances - Discounts |
|
|
||||||
| Cost of Revenue | Direct costs attributable to production of goods/services sold |
|
|
||||||
| Gross Profit | Net Revenue - Cost of Revenue |
|
|
||||||
|
|
||||||
**Note:** Always use Net Revenue (also called "Net Sales" or simply "Revenue" on most financial statements) as the starting point for profitability calculations. Gross Revenue overstates the true top-line performance.
|
|
||||||
|
|
||||||
## Margin Formulas
|
|
||||||
|
|
||||||
```
|
|
||||||
Gross Margin % = Gross Profit / Net Revenue
|
|
||||||
EBITDA = EBIT + D&A (or = Gross Profit - OpEx)
|
|
||||||
EBITDA Margin % = EBITDA / Net Revenue
|
|
||||||
EBIT Margin % = EBIT / Net Revenue
|
|
||||||
Net Income Margin % = Net Income / Net Revenue
|
|
||||||
```
|
|
||||||
|
|
||||||
## Credit Metric Formulas
|
|
||||||
|
|
||||||
```
|
|
||||||
Total Debt = Current Portion of Debt + Long-Term Debt
|
|
||||||
Net Debt = Total Debt - Cash
|
|
||||||
Total Debt / EBITDA = Total Debt / EBITDA (from IS)
|
|
||||||
Net Debt / EBITDA = Net Debt / EBITDA (from IS)
|
|
||||||
Interest Coverage = EBITDA / Interest Expense (from IS)
|
|
||||||
Net Int Exp % Debt = Net Interest Expense / Long-Term Debt
|
|
||||||
Debt / Total Cap = Total Debt / (Total Debt + Total Equity)
|
|
||||||
Debt / Equity = Total Debt / Total Equity
|
|
||||||
Current Ratio = Total Current Assets / Total Current Liabilities
|
|
||||||
Quick Ratio = (Total Current Assets - Inventory) / Total Current Liabilities
|
|
||||||
```
|
|
||||||
|
|
||||||
## Forecast Formulas (% of Net Revenue Method)
|
|
||||||
|
|
||||||
```
|
|
||||||
Cost of Revenue (Forecast) = Net Revenue × Cost of Revenue % Assumption
|
|
||||||
S&M (Forecast) = Net Revenue × S&M % Assumption
|
|
||||||
G&A (Forecast) = Net Revenue × G&A % Assumption
|
|
||||||
R&D (Forecast) = Net Revenue × R&D % Assumption
|
|
||||||
SBC (Forecast) = Net Revenue × SBC % Assumption
|
|
||||||
```
|
|
||||||
|
|
||||||
## Working Capital Formulas
|
|
||||||
|
|
||||||
```
|
|
||||||
Accounts Receivable
|
|
||||||
Prior AR
|
|
||||||
+ Revenue (from IS)
|
|
||||||
- Cash Collections (plug)
|
|
||||||
= Ending AR
|
|
||||||
DSO = (AR / Revenue) × 365
|
|
||||||
|
|
||||||
Inventory
|
|
||||||
Prior Inventory
|
|
||||||
+ Purchases (plug)
|
|
||||||
- COGS (from IS)
|
|
||||||
= Ending Inventory
|
|
||||||
DIO = (Inventory / COGS) × 365
|
|
||||||
|
|
||||||
Accounts Payable
|
|
||||||
Prior AP
|
|
||||||
+ Purchases (from Inventory calc)
|
|
||||||
- Cash Payments (plug)
|
|
||||||
= Ending AP
|
|
||||||
DPO = (AP / COGS) × 365
|
|
||||||
|
|
||||||
Net Working Capital = AR + Inventory - AP
|
|
||||||
ΔWC = Current NWC - Prior NWC
|
|
||||||
```
|
|
||||||
|
|
||||||
## D&A Schedule Formulas
|
|
||||||
|
|
||||||
```
|
|
||||||
Beginning PP&E (Gross)
|
|
||||||
+ CapEx
|
|
||||||
= Ending PP&E (Gross)
|
|
||||||
|
|
||||||
Beginning Accumulated Depreciation
|
|
||||||
+ Depreciation Expense
|
|
||||||
= Ending Accumulated Depreciation
|
|
||||||
|
|
||||||
PP&E (Net) = Gross PP&E - Accumulated Depreciation
|
|
||||||
```
|
|
||||||
|
|
||||||
## Debt Schedule Formulas
|
|
||||||
|
|
||||||
```
|
|
||||||
Beginning Debt Balance
|
|
||||||
+ New Borrowings
|
|
||||||
- Repayments
|
|
||||||
= Ending Debt Balance
|
|
||||||
|
|
||||||
Interest Expense = Avg Debt Balance × Interest Rate
|
|
||||||
(Use beginning balance to avoid circularity, or iterate if circular refs enabled)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Retained Earnings Formula
|
|
||||||
|
|
||||||
```
|
|
||||||
Beginning Retained Earnings
|
|
||||||
+ Net Income (from IS)
|
|
||||||
+ Stock-Based Compensation (SBC) (from IS)
|
|
||||||
- Dividends
|
|
||||||
= Ending Retained Earnings
|
|
||||||
```
|
|
||||||
|
|
||||||
## NOL (Net Operating Loss) Schedule Formulas
|
|
||||||
|
|
||||||
```
|
|
||||||
NOL CARRYFORWARD SCHEDULE
|
|
||||||
|
|
||||||
Beginning NOL Balance (Year 1 / Formation = 0)
|
|
||||||
+ NOL Generated (if EBT < 0, then ABS(EBT), else 0)
|
|
||||||
- NOL Utilized (limited by taxable income and utilization cap)
|
|
||||||
= Ending NOL Balance
|
|
||||||
|
|
||||||
STARTING BALANCE RULE
|
|
||||||
|
|
||||||
For a new business or first modeled period:
|
|
||||||
Beginning NOL Balance = 0
|
|
||||||
NOL can only increase through realized losses (EBT < 0)
|
|
||||||
NOL cannot be created from thin air or assumed
|
|
||||||
|
|
||||||
NOL UTILIZATION CALCULATION
|
|
||||||
|
|
||||||
Pre-Tax Income (EBT)
|
|
||||||
If EBT > 0:
|
|
||||||
NOL Available = Beginning NOL Balance
|
|
||||||
Utilization Limit = EBT × 80% (post-2017 federal limit)
|
|
||||||
NOL Utilized = MIN(NOL Available, Utilization Limit)
|
|
||||||
Taxable Income = EBT - NOL Utilized
|
|
||||||
If EBT ≤ 0:
|
|
||||||
NOL Utilized = 0
|
|
||||||
Taxable Income = 0
|
|
||||||
NOL Generated = ABS(EBT)
|
|
||||||
|
|
||||||
TAX CALCULATION WITH NOL
|
|
||||||
|
|
||||||
Taxes Payable = MAX(0, Taxable Income × Tax Rate)
|
|
||||||
(Taxes cannot be negative; losses create NOL asset instead)
|
|
||||||
|
|
||||||
DEFERRED TAX ASSET (DTA) FOR NOL
|
|
||||||
|
|
||||||
DTA - NOL Carryforward = Ending NOL Balance × Tax Rate
|
|
||||||
ΔDTA = Current DTA - Prior DTA
|
|
||||||
(Increase in DTA = non-cash benefit on CF)
|
|
||||||
(Decrease in DTA = non-cash expense on CF)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Balance Sheet Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
ASSETS
|
|
||||||
Cash (from CF ending cash)
|
|
||||||
Accounts Receivable (from WC)
|
|
||||||
Inventory (from WC)
|
|
||||||
Total Current Assets
|
|
||||||
|
|
||||||
PP&E, Net (from DA)
|
|
||||||
Deferred Tax Asset - NOL (from NOL schedule)
|
|
||||||
Total Non-Current Assets
|
|
||||||
Total Assets
|
|
||||||
|
|
||||||
LIABILITIES
|
|
||||||
Accounts Payable (from WC)
|
|
||||||
Current Portion of Debt (from Debt)
|
|
||||||
Total Current Liabilities
|
|
||||||
|
|
||||||
Long-Term Debt (from Debt)
|
|
||||||
Total Liabilities
|
|
||||||
|
|
||||||
EQUITY
|
|
||||||
Common Stock
|
|
||||||
Retained Earnings (from RE schedule)
|
|
||||||
Total Equity
|
|
||||||
|
|
||||||
CHECK: Assets - Liabilities - Equity = 0
|
|
||||||
```
|
|
||||||
|
|
||||||
## Cash Flow Statement Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
CASH FROM OPERATIONS (CFO)
|
|
||||||
Net Income (LINK: IS)
|
|
||||||
+ D&A (LINK: DA schedule)
|
|
||||||
+ Stock-Based Compensation (SBC) (LINK: IS or Assumptions)
|
|
||||||
- ΔDTA (Deferred Tax Asset) (LINK: NOL schedule; increase in DTA = use of cash)
|
|
||||||
- ΔAR (LINK: WC)
|
|
||||||
- ΔInventory (LINK: WC)
|
|
||||||
+ ΔAP (LINK: WC)
|
|
||||||
= CFO
|
|
||||||
|
|
||||||
CASH FROM INVESTING (CFI)
|
|
||||||
- CapEx (LINK: DA schedule)
|
|
||||||
= CFI
|
|
||||||
|
|
||||||
CASH FROM FINANCING (CFF)
|
|
||||||
+ Debt Issuance (LINK: Debt)
|
|
||||||
- Debt Repayment (LINK: Debt)
|
|
||||||
+ Equity Issuance (LINK: BS Common Stock/APIC)
|
|
||||||
- Dividends (LINK: RE schedule)
|
|
||||||
= CFF
|
|
||||||
|
|
||||||
Net Change in Cash = CFO + CFI + CFF
|
|
||||||
Beginning Cash
|
|
||||||
+ Net Change in Cash
|
|
||||||
= Ending Cash (LINK TO: BS Cash)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Income Statement Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
Net Revenue
|
|
||||||
Growth %
|
|
||||||
(-) Cost of Revenue
|
|
||||||
% of Net Revenue
|
|
||||||
────────────────
|
|
||||||
Gross Profit (= Net Revenue - Cost of Revenue)
|
|
||||||
Gross Margin %
|
|
||||||
|
|
||||||
(-) S&M
|
|
||||||
% of Net Revenue
|
|
||||||
(-) G&A
|
|
||||||
% of Net Revenue
|
|
||||||
(-) R&D
|
|
||||||
% of Net Revenue
|
|
||||||
(-) D&A
|
|
||||||
(-) SBC
|
|
||||||
% of Net Revenue
|
|
||||||
────────────────
|
|
||||||
EBIT
|
|
||||||
EBIT Margin %
|
|
||||||
|
|
||||||
EBITDA
|
|
||||||
EBITDA Margin %
|
|
||||||
|
|
||||||
(-) Interest Expense
|
|
||||||
────────────────
|
|
||||||
EBT (Pre-Tax Income)
|
|
||||||
(-) NOL Utilization (from NOL schedule, reduces taxable income)
|
|
||||||
────────────────
|
|
||||||
Taxable Income
|
|
||||||
(-) Taxes (Taxable Income × Tax Rate)
|
|
||||||
────────────────
|
|
||||||
Net Income
|
|
||||||
Net Income Margin %
|
|
||||||
```
|
|
||||||
|
|
||||||
## Check Formulas
|
|
||||||
|
|
||||||
```
|
|
||||||
BS Balance Check: = Assets - Liabilities - Equity (must = 0)
|
|
||||||
Cash Tie-Out: = BS Cash - CF Ending Cash (must = 0)
|
|
||||||
RE Roll-Forward: = Prior RE + NI + SBC - Div - BS RE (must = 0)
|
|
||||||
DTA Tie-Out: = NOL Schedule DTA - BS DTA (must = 0)
|
|
||||||
Equity Raise Tie-Out: = ΔCommon Stock/APIC (BS) - Equity Issuance (CFF) (must = 0)
|
|
||||||
Year 0 Equity Tie-Out: = Equity Raised (Year 0) - Beginning Equity (Year 1) (must = 0)
|
|
||||||
Cash Monthly vs Annual: = Closing Cash (Monthly) - Closing Cash (Annual) (must = 0)
|
|
||||||
NOL Utilization Cap: = NOL Utilized ≤ EBT × 80% (must be TRUE for post-2017)
|
|
||||||
NOL Non-Negative: = Ending NOL Balance ≥ 0 (must be TRUE)
|
|
||||||
NOL Starting Balance: = Beginning NOL (Year 1) = 0 (must be TRUE for new business)
|
|
||||||
NOL Accumulation: = NOL increases only when EBT < 0 (losses generate NOL)
|
|
||||||
```
|
|
||||||
@@ -1,125 +0,0 @@
|
|||||||
# SEC Filings Data Extraction Reference
|
|
||||||
|
|
||||||
**When to Use:** Only reference this file when a model template specifically requires pulling data from SEC filings (10-K, 10-Q). For templates that provide data directly or use other data sources, this reference is not needed.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Extracting Data from SEC Filings (10-K / 10-Q)
|
|
||||||
|
|
||||||
When populating a model template with public company data, extract financials directly from SEC filings.
|
|
||||||
|
|
||||||
### Step 1: Locate the Filing
|
|
||||||
|
|
||||||
1. Use SEC EDGAR: `https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=[TICKER]&type=10-K`
|
|
||||||
2. For quarterly data, use `type=10-Q`
|
|
||||||
|
|
||||||
### Step 2: Identify Filing Currency
|
|
||||||
|
|
||||||
Before extracting data, identify the reporting currency:
|
|
||||||
- Check the cover page or header for reporting currency
|
|
||||||
- Look at statement headers (e.g., "in thousands of U.S. dollars")
|
|
||||||
- Review Note 1 (Summary of Significant Accounting Policies)
|
|
||||||
|
|
||||||
**Common Currency Indicators**
|
|
||||||
|
|
||||||
| Indicator | Currency |
|
|
||||||
|-----------|----------|
|
|
||||||
| $, USD | US Dollar |
|
|
||||||
| €, EUR | Euro |
|
|
||||||
| £, GBP | British Pound |
|
|
||||||
| ¥, JPY | Japanese Yen |
|
|
||||||
| ¥, CNY, RMB | Chinese Yuan |
|
|
||||||
| CHF | Swiss Franc |
|
|
||||||
| CAD, C$ | Canadian Dollar |
|
|
||||||
|
|
||||||
Set model currency to match filing; document in Assumptions tab.
|
|
||||||
|
|
||||||
### Step 3: Navigate to Financial Statements
|
|
||||||
|
|
||||||
Within the 10-K or 10-Q, locate:
|
|
||||||
- **Item 8** (10-K) or **Item 1** (10-Q): Financial Statements
|
|
||||||
- Key sections to extract:
|
|
||||||
- Consolidated Statements of Operations (Income Statement)
|
|
||||||
- Consolidated Balance Sheets
|
|
||||||
- Consolidated Statements of Cash Flows
|
|
||||||
- Notes to Financial Statements (for schedule details)
|
|
||||||
|
|
||||||
### Step 4: Data Extraction Mapping
|
|
||||||
|
|
||||||
**Income Statement (from Consolidated Statements of Operations)**
|
|
||||||
|
|
||||||
| Filing Line Item | Model Line Item |
|
|
||||||
|------------------|-----------------|
|
|
||||||
| Net revenues / Net sales | Revenue |
|
|
||||||
| Cost of goods sold | COGS |
|
|
||||||
| Selling, general and administrative | SG&A |
|
|
||||||
| Depreciation and amortization | D&A |
|
|
||||||
| Interest expense, net | Interest Expense |
|
|
||||||
| Income tax expense | Taxes |
|
|
||||||
| Net income | Net Income |
|
|
||||||
|
|
||||||
**Balance Sheet (from Consolidated Balance Sheets)**
|
|
||||||
|
|
||||||
| Filing Line Item | Model Line Item |
|
|
||||||
|------------------|-----------------|
|
|
||||||
| Cash and cash equivalents | Cash |
|
|
||||||
| Accounts receivable, net | AR |
|
|
||||||
| Inventories | Inventory |
|
|
||||||
| Property, plant and equipment, net | PP&E (Net) |
|
|
||||||
| Total assets | Total Assets |
|
|
||||||
| Accounts payable | AP |
|
|
||||||
| Short-term debt / Current portion of LT debt | Current Debt |
|
|
||||||
| Long-term debt | LT Debt |
|
|
||||||
| Retained earnings | Retained Earnings |
|
|
||||||
| Total stockholders' equity | Total Equity |
|
|
||||||
|
|
||||||
**Cash Flow Statement (from Consolidated Statements of Cash Flows)**
|
|
||||||
|
|
||||||
| Filing Line Item | Model Line Item |
|
|
||||||
|------------------|-----------------|
|
|
||||||
| Net income | Net Income |
|
|
||||||
| Depreciation and amortization | D&A |
|
|
||||||
| Changes in accounts receivable | ΔAR |
|
|
||||||
| Changes in inventories | ΔInventory |
|
|
||||||
| Changes in accounts payable | ΔAP |
|
|
||||||
| Capital expenditures | CapEx |
|
|
||||||
| Proceeds from issuance of common stock | Equity Issuance |
|
|
||||||
| Proceeds from / Repayments of debt | Debt activity |
|
|
||||||
| Dividends paid | Dividends |
|
|
||||||
|
|
||||||
### Step 5: Extract Supporting Detail from Notes
|
|
||||||
|
|
||||||
For schedules, pull from Notes to Financial Statements:
|
|
||||||
- **Note: Debt** → Maturity schedule, interest rates, covenants
|
|
||||||
- **Note: Property, Plant & Equipment** → Gross PP&E, accumulated depreciation, useful lives
|
|
||||||
- **Note: Revenue** → Segment breakdowns, geographic splits
|
|
||||||
- **Note: Leases** → Operating vs. finance lease obligations
|
|
||||||
|
|
||||||
### Step 6: Historical Data Requirements
|
|
||||||
|
|
||||||
Extract 3 years of historical data minimum:
|
|
||||||
- 10-K provides 3 years of IS/CF, 2 years of BS
|
|
||||||
- For 3rd year BS, pull from prior year's 10-K
|
|
||||||
- Use 10-Qs to fill in quarterly granularity if needed
|
|
||||||
|
|
||||||
### Data Extraction Checklist
|
|
||||||
|
|
||||||
- Identify reporting currency and scale (thousands, millions)
|
|
||||||
- 3 years historical Income Statement
|
|
||||||
- 3 years historical Cash Flow Statement
|
|
||||||
- 3 years historical Balance Sheet
|
|
||||||
- Verify IS Net Income = CF starting Net Income (each year)
|
|
||||||
- Verify BS Cash = CF Ending Cash (each year)
|
|
||||||
- Extract debt maturity schedule from notes
|
|
||||||
- Extract D&A detail or useful life assumptions
|
|
||||||
- Note any non-recurring / one-time items to normalize
|
|
||||||
|
|
||||||
### Handling Common Filing Variations
|
|
||||||
|
|
||||||
| Variation | How to Handle |
|
|
||||||
|-----------|---------------|
|
|
||||||
| D&A embedded in COGS/SG&A | Pull D&A from Cash Flow Statement |
|
|
||||||
| "Other" line items are material | Check notes for breakdown |
|
|
||||||
| Restatements | Use restated figures, note in assumptions |
|
|
||||||
| Fiscal year ≠ calendar year | Label with fiscal year end (e.g., FYE Jan 2025) |
|
|
||||||
| Non-USD reporting currency | Adapt model currency to match filing |
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user