Compare commits
235 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ec1714e71f | |||
| e0c03defd5 | |||
| 9c914c01c8 | |||
| 6098272454 | |||
| bf43f6cfdd | |||
| f5ec30dfe6 | |||
| 8798bea31f | |||
| 668e4b8d7e | |||
| fab984c7f8 | |||
| f0d2516a30 | |||
| 2e403bd0a4 | |||
| 2c7b479d16 | |||
| 225b57f314 | |||
| 4d7e72e14d | |||
| 787d964ea1 | |||
| cf9b2df57a | |||
| eeb723fff2 | |||
| 1da89528e7 | |||
| 5486ad2f2a | |||
| fda234a210 | |||
| 7f369bfe55 | |||
| c80fa728bd | |||
| 292f468366 | |||
| d87c7b99e2 | |||
| cff821e2dc | |||
| 2214ab1073 | |||
| 9076a2e74e | |||
| 24d48ffb82 | |||
| 732a6c45fa | |||
| dc5ef1ac8e | |||
| da18fd084a | |||
| 54c0b10d14 | |||
| 04193cf71c | |||
| cdc0a47dd5 | |||
| 7e2af0c2e8 | |||
| 733e297b8a | |||
| 498bfc7bc1 | |||
| 2564132a1f | |||
| 812ce0b987 | |||
| 1d2029b2b7 | |||
| 04918345ea | |||
| 4de3ef38b1 | |||
| a1fe5f473d | |||
| bbff2f6345 | |||
| 162ad3dd16 | |||
| f4de3810ef | |||
| 74c9c0eec9 | |||
| 898b6d7d55 | |||
| fb4f953569 | |||
| 5c08b851df | |||
| 5b121c6e35 | |||
| de584cd1dd | |||
| 9feaeb632b | |||
| ac51c4c1ad | |||
| ff09853235 | |||
| 145e8ec237 | |||
| 1baab8771a | |||
| 62c2f5d8d2 | |||
| 43cf72a458 | |||
| be87a96296 | |||
| 44cd79e798 | |||
| af9336d575 | |||
| c8e3e39185 | |||
| dd2dc2bddf | |||
| 4ee6c3349a | |||
| d5fcc83922 | |||
| 12a0f5901c | |||
| e0a2b08768 | |||
| 5a3e5b23d2 | |||
| 3924cb408b | |||
| 69d025e4a7 | |||
| f5c9bb582c | |||
| cd3ef685c4 | |||
| 6a4ecc0a9f | |||
| 76d2dcdc8e | |||
| fc88eec926 | |||
| e795b7e3ab | |||
| a78e622dfe | |||
| 52e2777821 | |||
| 6769060ae2 | |||
| ec9d0e26d4 | |||
| 30c9990175 | |||
| edbbc96b55 | |||
| 2c1921241c | |||
| f9b4b8af34 | |||
| f481395d4c | |||
| a1f85ef2b9 | |||
| 8ad117a3d6 | |||
| 33563df027 | |||
| 4d4807585a | |||
| 0efc547962 | |||
| 5d9061148f | |||
| 755b74fc2d | |||
| f7b71aa0da | |||
| 8aa30407c2 | |||
| 80548f9a4f | |||
| 25187ca05c | |||
| a9ebee5f02 | |||
| 4d32f40306 | |||
| 926402dd13 | |||
| 5909526a06 | |||
| 46d1fc16ab | |||
| 9575bce6ca | |||
| b7a97cd44f | |||
| 98ca0694d6 | |||
| fcd619cae4 | |||
| a9c7bdaea6 | |||
| b739fcdfce | |||
| cfe019c782 | |||
| 5533ad7644 | |||
| fd13b7d2b9 | |||
| 6ea4a6a740 | |||
| fa582749e1 | |||
| b93c9f6393 | |||
| 498c01406f | |||
| 2f2f654486 | |||
| 4876959a19 | |||
| f648c2e3aa | |||
| d12be46df8 | |||
| c2d6b385f1 | |||
| 7244a1f0d3 | |||
| a494a614d0 | |||
| 31f22890ea | |||
| 8cef149131 | |||
| 9442a8fa22 | |||
| 84287b0de8 | |||
| afbcca0f06 | |||
| 5cf703245b | |||
| 5ead126709 | |||
| 14f38822fa | |||
| 6e46f99e7e | |||
| 8dcdc3cbc2 | |||
| 2021c18655 | |||
| e73508979f | |||
| 80717a157f | |||
| 5c045b8f6c | |||
| ef1e565570 | |||
| 8308d18339 | |||
| 2c14d3b9b0 | |||
| 5b24c0fa85 | |||
| ae1f058b3c | |||
| 47bf5d7ecb | |||
| c3be6ec184 | |||
| 6e250a55de | |||
| b12a5a72b0 | |||
| e9685a5cf7 | |||
| b9f1ac8c10 | |||
| 647f95b422 | |||
| 0d3593e514 | |||
| 0214858ef5 | |||
| 12289c2630 | |||
| c4a7992317 | |||
| 3c439ec681 | |||
| 61d9e3366d | |||
| fe4748ede8 | |||
| a5f116fc3f | |||
| 36ad97337a | |||
| 43a6645718 | |||
| d8d57fb2f6 | |||
| 6b3a9b4bfa | |||
| 6b9f7140bb | |||
| bda7b240b4 | |||
| 3a82172dd5 | |||
| f5a232af84 | |||
| 103e11926f | |||
| e38ea38079 | |||
| 3c8154e62c | |||
| 6d9b30632d | |||
| eef23354a5 | |||
| 7cbef2bd42 | |||
| 8aceef539f | |||
| a0758cd1e9 | |||
| fdb9e0f6a6 | |||
| 699c770e5c | |||
| aa9a2091f6 | |||
| 2b6345cee3 | |||
| 187951ec6b | |||
| 7137cccbd1 | |||
| abe5a3c937 | |||
| 4e27e4e05a | |||
| e82f3b0c41 | |||
| d34f03c32a | |||
| 8d363f8d54 | |||
| f0dd5b9c10 | |||
| 4f364c4e99 | |||
| 333598cb0e | |||
| 176b93575a | |||
| 5bf12eb44a | |||
| 69692039e9 | |||
| 042eb930e2 | |||
| 991df4ef81 | |||
| 8b32a9d0f1 | |||
| fb1ce793e6 | |||
| d856f4535d | |||
| ecaafe5f22 | |||
| 3a0d52d579 | |||
| 2e00bcaaab | |||
| 6e8f1e09a9 | |||
| 73d6371762 | |||
| 8a96fa48c1 | |||
| 429e78589b | |||
| a84e56d4c6 | |||
| 38b1c7dce5 | |||
| 961a3535fa | |||
| fad684b1f3 | |||
| 233bfd3621 | |||
| 411cfa26e3 | |||
| 595e906698 | |||
| b49a3f8474 | |||
| 06f24351c5 | |||
| 63bd690a50 | |||
| 40b51c93a2 | |||
| bf843adf05 | |||
| 7c0766e06a | |||
| d4de7d4179 | |||
| fce58cbe2e | |||
| 11b9b146f1 | |||
| 1f27ca638f | |||
| aa5690342b | |||
| 7d36e8346b | |||
| a5c9c83b78 | |||
| 595bcc89fc | |||
| 033e533d05 | |||
| e7eb07cec7 | |||
| bd0c54d171 | |||
| 51f9953e69 | |||
| 49c3c2e0d3 | |||
| 45cbf93899 | |||
| 5a3cadf6eb | |||
| d797755a1c | |||
| 3cdbf334d5 | |||
| 04cf4788cc | |||
| 5ccab51fa8 | |||
| 53a024994a | |||
| f4031df05d |
@@ -423,3 +423,24 @@ IMAGE_TOOLS_DEBUG=false
|
||||
# TEAMS_HOME_CHANNEL= # Default channel/chat ID for cron delivery
|
||||
# TEAMS_HOME_CHANNEL_NAME= # Display name for the home channel
|
||||
# TEAMS_PORT=3978 # Webhook listen port (Bot Framework default)
|
||||
|
||||
# =============================================================================
|
||||
# GOOGLE CHAT INTEGRATION
|
||||
# =============================================================================
|
||||
# Connects via Cloud Pub/Sub pull subscription (no public URL required).
|
||||
# Setup walkthrough: website/docs/user-guide/messaging/google_chat.md.
|
||||
# 1. Create a GCP project, enable the Google Chat API and Cloud Pub/Sub.
|
||||
# 2. Create a Service Account with roles/pubsub.subscriber on the
|
||||
# subscription (NOT project-wide); download the JSON key.
|
||||
# 3. Configure your Chat app at console.cloud.google.com/apis/credentials
|
||||
# → Google Chat API → Configuration → Cloud Pub/Sub topic.
|
||||
# 4. (Optional, for native attachment delivery) Each user runs
|
||||
# `/setup-files` once in their own DM after Pub/Sub is wired up.
|
||||
#
|
||||
# GOOGLE_CHAT_PROJECT_ID= # GCP project hosting the topic (or set GOOGLE_CLOUD_PROJECT)
|
||||
# GOOGLE_CHAT_SUBSCRIPTION_NAME= # Full path: projects/<id>/subscriptions/<name>
|
||||
# GOOGLE_CHAT_SERVICE_ACCOUNT_JSON= # Path to SA JSON (or set GOOGLE_APPLICATION_CREDENTIALS)
|
||||
# GOOGLE_CHAT_ALLOWED_USERS= # Comma-separated emails allowed to talk to the bot
|
||||
# GOOGLE_CHAT_ALLOW_ALL_USERS=false # Set true to skip the allowlist
|
||||
# GOOGLE_CHAT_HOME_CHANNEL= # Default space (spaces/XXXX) for cron delivery
|
||||
# GOOGLE_CHAT_HOME_CHANNEL_NAME= # Display name for the home channel
|
||||
|
||||
@@ -16,9 +16,13 @@ on:
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
# Top-level concurrency: do NOT cancel in-flight builds when a new push lands.
|
||||
# Every commit deserves its own SHA-tagged image in the registry, and we guard
|
||||
# the :latest tag in a separate job below (with its own concurrency group) so
|
||||
# a slow run can't clobber :latest with older bits.
|
||||
concurrency:
|
||||
group: docker-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
build-and-push:
|
||||
@@ -26,11 +30,18 @@ jobs:
|
||||
if: github.repository == 'NousResearch/hermes-agent'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 60
|
||||
outputs:
|
||||
pushed_sha_tag: ${{ steps.mark_pushed.outputs.pushed }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
submodules: recursive
|
||||
# Fetch enough history to run `git merge-base --is-ancestor` in the
|
||||
# move-latest job. That job reuses this checkout via its own
|
||||
# actions/checkout call, but commits reachable from main up to ~1000
|
||||
# back are plenty for any realistic race window.
|
||||
fetch-depth: 1000
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
|
||||
@@ -54,19 +65,31 @@ jobs:
|
||||
|
||||
- name: Test image starts
|
||||
run: |
|
||||
mkdir -p /tmp/hermes-test
|
||||
sudo chown -R 10000:10000 /tmp/hermes-test
|
||||
# The image runs as the hermes user (UID 10000). GitHub Actions
|
||||
# creates /tmp/hermes-test root-owned by default, which hermes
|
||||
# can't write to — chown it to match the in-container UID before
|
||||
# bind-mounting. Real users doing `docker run -v ~/.hermes:...`
|
||||
# with their own UID hit the same issue and have their own
|
||||
# remediations (HERMES_UID env var, or chown locally).
|
||||
mkdir -p /tmp/hermes-test
|
||||
sudo chown -R 10000:10000 /tmp/hermes-test
|
||||
docker run --rm \
|
||||
-v /tmp/hermes-test:/opt/data \
|
||||
--entrypoint /opt/hermes/docker/entrypoint.sh \
|
||||
nousresearch/hermes-agent:test --help
|
||||
|
||||
- name: Test dashboard subcommand
|
||||
run: |
|
||||
mkdir -p /tmp/hermes-test
|
||||
sudo chown -R 10000:10000 /tmp/hermes-test
|
||||
# Verify the dashboard subcommand is included in the Docker image.
|
||||
# This prevents regressions like #9153 where the dashboard command
|
||||
# was present in source but missing from the published image.
|
||||
docker run --rm \
|
||||
-v /tmp/hermes-test:/opt/data \
|
||||
--entrypoint /opt/hermes/docker/entrypoint.sh \
|
||||
nousresearch/hermes-agent:test dashboard --help
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main' || github.event_name == 'release'
|
||||
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
|
||||
@@ -74,7 +97,12 @@ jobs:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Push multi-arch image (main branch)
|
||||
# Always push a per-commit SHA tag on main. This is race-free because
|
||||
# every commit has a unique SHA — concurrent runs can't clobber each
|
||||
# other here. We also embed the git SHA as an OCI label so the
|
||||
# move-latest job (below) can read it back off the registry's `:latest`.
|
||||
- name: Push multi-arch image with SHA tag (main branch)
|
||||
id: push_sha
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
||||
with:
|
||||
@@ -82,10 +110,17 @@ jobs:
|
||||
file: Dockerfile
|
||||
push: true
|
||||
platforms: linux/amd64,linux/arm64
|
||||
tags: nousresearch/hermes-agent:latest
|
||||
tags: nousresearch/hermes-agent:sha-${{ github.sha }}
|
||||
labels: |
|
||||
org.opencontainers.image.revision=${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Mark SHA tag pushed
|
||||
id: mark_pushed
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
run: echo "pushed=true" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Push multi-arch image (release)
|
||||
if: github.event_name == 'release'
|
||||
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
||||
@@ -97,3 +132,119 @@ jobs:
|
||||
tags: nousresearch/hermes-agent:${{ github.event.release.tag_name }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
# Second job: moves `:latest` to point at the SHA tag the first job pushed.
|
||||
#
|
||||
# Has its own concurrency group with `cancel-in-progress: true`, which
|
||||
# gives us the serialization we need: if a newer push arrives while an
|
||||
# older run is mid-way through this job, the older run is cancelled
|
||||
# before it can clobber `:latest`. Combined with the ancestor check
|
||||
# below, this means `:latest` only ever moves forward in git history.
|
||||
move-latest:
|
||||
if: |
|
||||
github.repository == 'NousResearch/hermes-agent'
|
||||
&& github.event_name == 'push'
|
||||
&& github.ref == 'refs/heads/main'
|
||||
&& needs.build-and-push.outputs.pushed_sha_tag == 'true'
|
||||
needs: build-and-push
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
concurrency:
|
||||
group: docker-move-latest-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
fetch-depth: 1000
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# Read the git revision label off the current `:latest` manifest, then
|
||||
# use `git merge-base --is-ancestor` to check whether our commit is a
|
||||
# descendant of it. If `:latest` doesn't exist yet, or its label is
|
||||
# missing, we treat that as "safe to publish". If another run already
|
||||
# advanced `:latest` past us (or diverged), we skip and leave it alone.
|
||||
- name: Decide whether to move :latest
|
||||
id: latest_check
|
||||
run: |
|
||||
set -euo pipefail
|
||||
image=nousresearch/hermes-agent
|
||||
|
||||
# Pull the JSON for the linux/amd64 sub-manifest's config and extract
|
||||
# the OCI revision label with jq — Go template field access can't
|
||||
# handle dots in map keys, so using json+jq is the robust route.
|
||||
image_json=$(
|
||||
docker buildx imagetools inspect "${image}:latest" \
|
||||
--format '{{ json (index .Image "linux/amd64") }}' \
|
||||
2>/dev/null || true
|
||||
)
|
||||
|
||||
if [ -z "${image_json}" ]; then
|
||||
echo "No existing :latest (or inspect failed) — safe to publish."
|
||||
echo "push_latest=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
current_sha=$(
|
||||
printf '%s' "${image_json}" \
|
||||
| jq -r '.config.Labels."org.opencontainers.image.revision" // ""'
|
||||
)
|
||||
|
||||
if [ -z "${current_sha}" ]; then
|
||||
echo "Registry :latest has no revision label — safe to publish."
|
||||
echo "push_latest=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Registry :latest is at ${current_sha}"
|
||||
echo "This run is at ${GITHUB_SHA}"
|
||||
|
||||
if [ "${current_sha}" = "${GITHUB_SHA}" ]; then
|
||||
echo ":latest already points at our SHA — nothing to do."
|
||||
echo "push_latest=false" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Make sure we have the :latest commit locally for merge-base.
|
||||
if ! git cat-file -e "${current_sha}^{commit}" 2>/dev/null; then
|
||||
git fetch --no-tags --prune origin \
|
||||
"+refs/heads/main:refs/remotes/origin/main" \
|
||||
|| true
|
||||
fi
|
||||
|
||||
if ! git cat-file -e "${current_sha}^{commit}" 2>/dev/null; then
|
||||
echo "Registry :latest points at an unknown commit (${current_sha}); refusing to overwrite."
|
||||
echo "push_latest=false" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Our SHA must be a descendant of the current :latest to be safe.
|
||||
if git merge-base --is-ancestor "${current_sha}" "${GITHUB_SHA}"; then
|
||||
echo "Our commit is a descendant of :latest — safe to advance."
|
||||
echo "push_latest=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "Another run advanced :latest past us (or diverged) — leaving it alone."
|
||||
echo "push_latest=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
# Retag the already-pushed SHA manifest as :latest. This is a registry-
|
||||
# side operation — no rebuild, no layer re-push — so it's quick and
|
||||
# atomic per-tag. The ancestor check above plus the cancel-in-progress
|
||||
# concurrency on this job together guarantee we only ever move :latest
|
||||
# forward in git history.
|
||||
- name: Move :latest to this SHA
|
||||
if: steps.latest_check.outputs.push_latest == 'true'
|
||||
run: |
|
||||
set -euo pipefail
|
||||
image=nousresearch/hermes-agent
|
||||
docker buildx imagetools create \
|
||||
--tag "${image}:latest" \
|
||||
"${image}:sha-${GITHUB_SHA}"
|
||||
|
||||
@@ -48,16 +48,12 @@ jobs:
|
||||
|
||||
- name: Determine base ref
|
||||
id: base
|
||||
env:
|
||||
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
run: |
|
||||
# For PRs, diff against the PR's pinned parent commit
|
||||
# (github.event.pull_request.base.sha — snapshot at PR open time,
|
||||
# so later pushes to main don't leak into the diff).
|
||||
# For PRs, diff against the merge base with the target branch.
|
||||
# For pushes to main, diff against the previous commit on main.
|
||||
if [ "${{ github.event_name }}" = "pull_request" ]; then
|
||||
BASE_SHA="${PR_BASE_SHA}"
|
||||
BASE_REF="PR base (${BASE_SHA:0:7})"
|
||||
BASE_SHA=$(git merge-base "origin/${{ github.base_ref }}" HEAD)
|
||||
BASE_REF="origin/${{ github.base_ref }}"
|
||||
else
|
||||
BASE_SHA=$(git rev-parse HEAD~1 2>/dev/null || git rev-parse HEAD)
|
||||
BASE_REF="HEAD~1"
|
||||
@@ -121,9 +117,6 @@ jobs:
|
||||
name: lint-reports
|
||||
path: .lint-reports/
|
||||
retention-days: 14
|
||||
# .lint-reports/ is a dotfile-prefixed directory, and upload-artifact@v4
|
||||
# skips hidden files by default (breaking change from v3). Opt back in.
|
||||
include-hidden-files: true
|
||||
|
||||
- name: Post / update PR comment
|
||||
if: github.event_name == 'pull_request'
|
||||
|
||||
+16
-9
@@ -106,6 +106,11 @@ hermes chat -q "Hello"
|
||||
### Run tests
|
||||
|
||||
```bash
|
||||
# Preferred — matches CI (hermetic env, 4 xdist workers); see AGENTS.md
|
||||
scripts/run_tests.sh
|
||||
|
||||
# Alternative (activate the venv first). The wrapper is still recommended
|
||||
# for parity with GitHub Actions before you open a PR:
|
||||
pytest tests/ -v
|
||||
```
|
||||
|
||||
@@ -286,16 +291,18 @@ registry.register(
|
||||
)
|
||||
```
|
||||
|
||||
Then add the import to `model_tools.py` in the `_modules` list:
|
||||
**Wire into a toolset (required):** Built-in tools are auto-discovered: any
|
||||
`tools/*.py` file that contains a top-level `registry.register(...)` call is
|
||||
imported by `discover_builtin_tools()` in `tools/registry.py` when `model_tools`
|
||||
loads. There is **no** manual import list in `model_tools.py` to maintain.
|
||||
|
||||
```python
|
||||
_modules = [
|
||||
# ... existing modules ...
|
||||
"tools.my_tool",
|
||||
]
|
||||
```
|
||||
You must still add the tool name to the appropriate list in `toolsets.py`
|
||||
(for example `_HERMES_CORE_TOOLS` or a dedicated toolset); otherwise the tool
|
||||
registers but is never exposed to the agent. If you introduce a new toolset,
|
||||
add it in `toolsets.py` and wire it into the relevant platform presets.
|
||||
|
||||
If it's a new toolset, add it to `toolsets.py` and to the relevant platform presets.
|
||||
See `AGENTS.md` (section **Adding New Tools**) for profile-aware paths and
|
||||
plugin vs core guidance.
|
||||
|
||||
---
|
||||
|
||||
@@ -595,7 +602,7 @@ refactor/description # Code restructuring
|
||||
|
||||
### Before submitting
|
||||
|
||||
1. **Run tests**: `pytest tests/ -v`
|
||||
1. **Run tests**: `scripts/run_tests.sh` (recommended; same as CI) or `pytest tests/ -v` with the project venv activated
|
||||
2. **Test manually**: Run `hermes` and exercise the code path you changed
|
||||
3. **Check cross-platform impact**: If you touch file I/O, process management, or terminal handling, consider macOS, Linux, and WSL2
|
||||
4. **Keep PRs focused**: One logical change per PR. Don't mix a bug fix with a refactor with a new feature.
|
||||
|
||||
+7
-1
@@ -66,8 +66,14 @@ RUN cd web && npm run build && \
|
||||
# ---------- Permissions ----------
|
||||
# Make install dir world-readable so any HERMES_UID can read it at runtime.
|
||||
# The venv needs to be traversable too.
|
||||
# node_modules trees additionally need to be writable by the hermes user
|
||||
# so the runtime `npm install` triggered by _tui_need_npm_install() in
|
||||
# hermes_cli/main.py succeeds (see #18800). /opt/hermes/web is build-time
|
||||
# only (HERMES_WEB_DIST points at hermes_cli/web_dist) and is intentionally
|
||||
# not chowned here.
|
||||
USER root
|
||||
RUN chmod -R a+rX /opt/hermes
|
||||
RUN chmod -R a+rX /opt/hermes && \
|
||||
chown -R hermes:hermes /opt/hermes/ui-tui /opt/hermes/node_modules
|
||||
# Start as root so the entrypoint can usermod/groupmod + gosu.
|
||||
# If HERMES_UID is unset, the entrypoint drops to the default hermes user (10000).
|
||||
|
||||
|
||||
@@ -30,15 +30,27 @@ Use any model you want — [Nous Portal](https://portal.nousresearch.com), [Open
|
||||
|
||||
## Quick Install
|
||||
|
||||
### Linux, macOS, WSL2, Termux
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
|
||||
```
|
||||
|
||||
Works on Linux, macOS, WSL2, and Android via Termux. The installer handles the platform-specific setup for you.
|
||||
### Windows (native, PowerShell)
|
||||
|
||||
Run this in PowerShell:
|
||||
|
||||
```powershell
|
||||
irm https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.ps1 | iex
|
||||
```
|
||||
|
||||
The installer handles everything: uv, Python 3.11, Node.js, ripgrep, ffmpeg, **and a portable Git Bash** (MinGit, unpacked to `%LOCALAPPDATA%\hermes\git` — no admin required, completely isolated from any system Git install). Hermes uses this bundled Git Bash to run shell commands.
|
||||
|
||||
If you already have Git installed, the installer detects it and uses that instead. Otherwise a ~45MB MinGit download is all you need — it won't touch or interfere with any system Git.
|
||||
|
||||
> **Android / Termux:** The tested manual path is documented in the [Termux guide](https://hermes-agent.nousresearch.com/docs/getting-started/termux). On Termux, Hermes installs a curated `.[termux]` extra because the full `.[all]` extra currently pulls Android-incompatible voice dependencies.
|
||||
>
|
||||
> **Windows:** Native Windows is not supported. Please install [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install) and run the command above.
|
||||
> **Windows:** Native Windows is supported — the PowerShell one-liner above installs everything. If you'd rather use WSL2, the Linux command works there too. Native Windows install lives under `%LOCALAPPDATA%\hermes`; WSL2 installs under `~/.hermes` as on Linux. The only Hermes feature that currently needs WSL2 specifically is the browser-based dashboard chat pane (it uses a POSIX PTY — classic CLI and gateway both run natively).
|
||||
|
||||
After installation:
|
||||
|
||||
@@ -155,13 +167,13 @@ Manual path (equivalent to the above):
|
||||
|
||||
```bash
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
uv venv venv --python 3.11
|
||||
source venv/bin/activate
|
||||
uv venv .venv --python 3.11
|
||||
source .venv/bin/activate
|
||||
uv pip install -e ".[all,dev]"
|
||||
scripts/run_tests.sh
|
||||
```
|
||||
|
||||
> **RL Training (optional):** The RL/Atropos integration (`environments/`) ships via the `atroposlib` and `tinker` dependencies pulled in by `.[all,dev]` — no submodule setup required.
|
||||
> **RL Training (optional):** The RL/Atropos integration (`environments/`) — see [`CONTRIBUTING.md`](https://github.com/NousResearch/hermes-agent/blob/main/CONTRIBUTING.md#development-setup) for the full setup.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,641 @@
|
||||
# Hermes Agent v0.13.0 (v2026.5.7)
|
||||
|
||||
**Release Date:** May 7, 2026
|
||||
**Since v0.12.0:** 864 commits · 588 merged PRs · 829 files changed · 128,366 insertions · 282 issues closed (13 P0, 36 P1) · 295 community contributors (including co-authors)
|
||||
|
||||
> The Tenacity Release — Hermes Agent now finishes what it starts. Kanban ships as a durable multi-agent board (heartbeat, reclaim, zombie detection, auto-block on incomplete exit, per-task retries, hallucination recovery). `/goal` keeps the agent locked on a target across turns (Ralph loop). Checkpoints v2 rewrites state persistence with real pruning. Gateway auto-resumes interrupted sessions after restart. Cron grows a `no_agent` watchdog mode. A security wave closes 8 P0s — redaction is now ON by default, Discord role-allowlists are guild-scoped, WhatsApp rejects strangers by default, and TOCTOU windows close across auth.json and MCP OAuth. Google Chat becomes the 20th platform. Providers become a pluggable surface. Seven i18n locales ship.
|
||||
|
||||
---
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Multi-agent Kanban — delegate to an AI team that actually finishes** — Spin up a durable board, drop tasks on it, and let multiple Hermes workers pick them up, hand off, and close them out. Heartbeats, reclaim, zombie detection, retry budgets, and a hallucination gate keep the team honest. One install, many kanbans. ([#17805](https://github.com/NousResearch/hermes-agent/pull/17805), [#19653](https://github.com/NousResearch/hermes-agent/pull/19653), [#20232](https://github.com/NousResearch/hermes-agent/pull/20232), [#20332](https://github.com/NousResearch/hermes-agent/pull/20332), [#21330](https://github.com/NousResearch/hermes-agent/pull/21330), [#21183](https://github.com/NousResearch/hermes-agent/pull/21183), [#21214](https://github.com/NousResearch/hermes-agent/pull/21214))
|
||||
|
||||
- **`/goal` — the agent doesn't forget what you asked it to do** — Lock the agent onto a target and it stays on task across turns. The Ralph loop as a first-class primitive. ([#18262](https://github.com/NousResearch/hermes-agent/pull/18262), [#18275](https://github.com/NousResearch/hermes-agent/pull/18275), [#21287](https://github.com/NousResearch/hermes-agent/pull/21287))
|
||||
|
||||
- **Show it a video** — new `video_analyze` tool for native video understanding on Gemini and compatible multimodal models. (@alt-glitch) ([#19301](https://github.com/NousResearch/hermes-agent/pull/19301))
|
||||
|
||||
- **Clone a voice** — xAI Custom Voices lands as a TTS provider with voice cloning support. (@alt-glitch) ([#18776](https://github.com/NousResearch/hermes-agent/pull/18776))
|
||||
|
||||
- **Hermes speaks your language** — static gateway + CLI messages translate to 7 locales: Chinese, Japanese, German, Spanish, French, Ukrainian, and Turkish. Docs site gains a Chinese (zh-Hans) locale. ([#20231](https://github.com/NousResearch/hermes-agent/pull/20231), [#20329](https://github.com/NousResearch/hermes-agent/pull/20329), [#20467](https://github.com/NousResearch/hermes-agent/pull/20467), [#20474](https://github.com/NousResearch/hermes-agent/pull/20474), [#20430](https://github.com/NousResearch/hermes-agent/pull/20430), [#20431](https://github.com/NousResearch/hermes-agent/pull/20431))
|
||||
|
||||
- **Google Chat — the 20th messaging platform** — plus a generic platform-plugin hooks surface so third-party adapters drop in without touching core (IRC and Teams migrated). ([#21306](https://github.com/NousResearch/hermes-agent/pull/21306), [#21331](https://github.com/NousResearch/hermes-agent/pull/21331))
|
||||
|
||||
- **Sessions survive restarts** — gateway bounces mid-agent, `/update` restarts, source-file reloads — conversations auto-resume when the gateway comes back. ([#21192](https://github.com/NousResearch/hermes-agent/pull/21192))
|
||||
|
||||
- **Security wave — 8 P0 closures** — redaction ON by default, Discord role-allowlists guild-scoped (CVSS 8.1 cross-guild DM bypass closed), WhatsApp rejects strangers by default, TOCTOU windows closed across `auth.json` and MCP OAuth, browser enforces cloud-metadata SSRF floor, cron prompt-injection scans assembled skill content, `hermes debug share` redacts at upload. ([#21193](https://github.com/NousResearch/hermes-agent/pull/21193), [#21241](https://github.com/NousResearch/hermes-agent/pull/21241), [#21291](https://github.com/NousResearch/hermes-agent/pull/21291), [#21176](https://github.com/NousResearch/hermes-agent/pull/21176), [#21194](https://github.com/NousResearch/hermes-agent/pull/21194), [#21228](https://github.com/NousResearch/hermes-agent/pull/21228), [#21350](https://github.com/NousResearch/hermes-agent/pull/21350), [#19318](https://github.com/NousResearch/hermes-agent/pull/19318))
|
||||
|
||||
- **Checkpoints v2** — state persistence rewritten. Real pruning, disk guardrails, no more orphan shadow repos. ([#20709](https://github.com/NousResearch/hermes-agent/pull/20709))
|
||||
|
||||
- **The agent lints its own writes** — post-write delta lint on `write_file` + `patch`. Python, JSON, YAML, TOML. Syntax errors surface immediately instead of shipping downstream. ([#20191](https://github.com/NousResearch/hermes-agent/pull/20191))
|
||||
|
||||
- **`no_agent` cron mode — script-only watchdog** — cron jobs can now skip the agent entirely and just run a script. Empty stdout is silent, non-empty gets delivered verbatim. ([#19709](https://github.com/NousResearch/hermes-agent/pull/19709))
|
||||
|
||||
- **Platform allowlists everywhere** — `allowed_channels` / `allowed_chats` / `allowed_rooms` config across Slack, Telegram, Mattermost, Matrix, and DingTalk. ([#21251](https://github.com/NousResearch/hermes-agent/pull/21251))
|
||||
|
||||
- **Providers are now plugins** — `ProviderProfile` ABC + `plugins/model-providers/`. Drop in third-party providers without touching core. ([#20324](https://github.com/NousResearch/hermes-agent/pull/20324))
|
||||
|
||||
- **API server — long-term memory per session** — `X-Hermes-Session-Key` header gives memory providers a stable session identifier. ([#20199](https://github.com/NousResearch/hermes-agent/pull/20199))
|
||||
|
||||
- **MCP levels up** — SSE transport with OAuth forwarding, stale-pipe retries, image results surface as MEDIA tags instead of getting dropped, keepalive on long-lived lifecycle waits. ([#21227](https://github.com/NousResearch/hermes-agent/pull/21227), [#21323](https://github.com/NousResearch/hermes-agent/pull/21323), [#21289](https://github.com/NousResearch/hermes-agent/pull/21289), [#21328](https://github.com/NousResearch/hermes-agent/pull/21328), [#20209](https://github.com/NousResearch/hermes-agent/pull/20209))
|
||||
|
||||
- **Curator grows subcommands** — `hermes curator archive`, `prune`, `list-archived`. Manual `hermes curator run` is synchronous now — you see results without polling. ([#20200](https://github.com/NousResearch/hermes-agent/pull/20200), [#21236](https://github.com/NousResearch/hermes-agent/pull/21236), [#21216](https://github.com/NousResearch/hermes-agent/pull/21216))
|
||||
|
||||
- **ACP — `/steer` and `/queue`** — direct the in-flight agent or queue follow-ups from Zed, VS Code, or JetBrains. Plus atomic session persistence and reasoning-metadata preservation across restarts. (@HenkDz) ([#18114](https://github.com/NousResearch/hermes-agent/pull/18114), [#20279](https://github.com/NousResearch/hermes-agent/pull/20279), [#20296](https://github.com/NousResearch/hermes-agent/pull/20296), [#20433](https://github.com/NousResearch/hermes-agent/pull/20433))
|
||||
|
||||
- **TUI glow-up** — `/model` picker matches `hermes model` with inline auth (@austinpickett), collapsible startup banner sections (@kshitijk4poor), context-compression counter in the status bar. ([#18117](https://github.com/NousResearch/hermes-agent/pull/18117), [#20625](https://github.com/NousResearch/hermes-agent/pull/20625), [#21218](https://github.com/NousResearch/hermes-agent/pull/21218))
|
||||
|
||||
- **Dashboard grows up** — Plugins page (manage, enable/disable, auth status) (@austinpickett), Profiles management page (@vincez-hms-coder), sortable analytics tables, reverse-proxy support via `X-Forwarded-Prefix`, new `default-large` 18px theme. ([#18095](https://github.com/NousResearch/hermes-agent/pull/18095), [#16419](https://github.com/NousResearch/hermes-agent/pull/16419), [#18192](https://github.com/NousResearch/hermes-agent/pull/18192), [#21296](https://github.com/NousResearch/hermes-agent/pull/21296), [#20820](https://github.com/NousResearch/hermes-agent/pull/20820))
|
||||
|
||||
- **SearXNG + split web tools** — SearXNG ships as a native search-only backend; web tools now let you pick different backends per capability (search vs extract vs browse). (@kshitijk4poor) ([#20823](https://github.com/NousResearch/hermes-agent/pull/20823), [#20061](https://github.com/NousResearch/hermes-agent/pull/20061), [#20841](https://github.com/NousResearch/hermes-agent/pull/20841))
|
||||
|
||||
- **OpenRouter response caching** — explicit cache control for models that expose it. (@kshitijk4poor) ([#19132](https://github.com/NousResearch/hermes-agent/pull/19132))
|
||||
|
||||
- **`[[as_document]]` — skill media-routing directive** — skills can force the gateway to deliver output as a document on platforms that support it. ([#21210](https://github.com/NousResearch/hermes-agent/pull/21210))
|
||||
|
||||
- **`transform_llm_output` plugin hook** — new lifecycle hook that lets plugins reshape or filter LLM output before it hits the conversation. Useful for context-window reducers and content filters. ([#21235](https://github.com/NousResearch/hermes-agent/pull/21235))
|
||||
|
||||
- **Nous OAuth persists across profiles** — shared token store: sign in once, every profile inherits the session. ([#19712](https://github.com/NousResearch/hermes-agent/pull/19712))
|
||||
|
||||
- **QQBot — native approval keyboards** — feature parity with Telegram / Discord approval UX. Chunked upload, quoted attachments. ([#21342](https://github.com/NousResearch/hermes-agent/pull/21342), [#21353](https://github.com/NousResearch/hermes-agent/pull/21353))
|
||||
|
||||
- **6 new optional skills** — Shopify (Admin + Storefront GraphQL), here.now, shop-app personal shopping assistant, Anthropic financial-services bundle, kanban-video-orchestrator (@SHL0MS), searxng-search (@kshitijk4poor). ([#18116](https://github.com/NousResearch/hermes-agent/pull/18116), [#18170](https://github.com/NousResearch/hermes-agent/pull/18170), [#20702](https://github.com/NousResearch/hermes-agent/pull/20702), [#21180](https://github.com/NousResearch/hermes-agent/pull/21180), [#19281](https://github.com/NousResearch/hermes-agent/pull/19281), [#20841](https://github.com/NousResearch/hermes-agent/pull/20841))
|
||||
|
||||
- **New models** — `deepseek/deepseek-v4-pro`, `x-ai/grok-4.3`, `openrouter/owl-alpha` (free), `tencent/hy3-preview` (@Contentment003111), Arcee Trinity Large Thinking temperature + compression overrides. ([#20495](https://github.com/NousResearch/hermes-agent/pull/20495), [#20497](https://github.com/NousResearch/hermes-agent/pull/20497), [#18071](https://github.com/NousResearch/hermes-agent/pull/18071), [#21077](https://github.com/NousResearch/hermes-agent/pull/21077), [#20473](https://github.com/NousResearch/hermes-agent/pull/20473))
|
||||
|
||||
- **100 fresh CLI startup tips** — the random tip banner gets 100 new entries covering cron, kanban, curator, plugins, and lesser-known flags. ([#20168](https://github.com/NousResearch/hermes-agent/pull/20168))
|
||||
|
||||
---
|
||||
|
||||
## 🧩 Multi-Agent Kanban (Durable)
|
||||
|
||||
### New — durable multi-profile collaboration board
|
||||
- **`feat(kanban): durable multi-profile collaboration board`** — post-revert reimplementation, multi-profile by design ([#17805](https://github.com/NousResearch/hermes-agent/pull/17805))
|
||||
- **Multi-project boards** — one install, many kanbans ([#19653](https://github.com/NousResearch/hermes-agent/pull/19653), [#19679](https://github.com/NousResearch/hermes-agent/pull/19679))
|
||||
- **Share board, workspaces, and worker logs across profiles** ([#19378](https://github.com/NousResearch/hermes-agent/pull/19378))
|
||||
- **Hallucination gate + recovery UX for worker-created-card claims** (closes #20017) ([#20232](https://github.com/NousResearch/hermes-agent/pull/20232))
|
||||
- **Generic diagnostics engine for task distress signals** ([#20332](https://github.com/NousResearch/hermes-agent/pull/20332))
|
||||
- **Per-task `max_retries` override** (supersedes #20972) ([#21330](https://github.com/NousResearch/hermes-agent/pull/21330))
|
||||
- **Multiline textarea for inline-create title** (salvage of #20970) ([#21243](https://github.com/NousResearch/hermes-agent/pull/21243))
|
||||
|
||||
### Kanban Dashboard
|
||||
- **Workspace kind + path inputs in inline create form** ([#19679](https://github.com/NousResearch/hermes-agent/pull/19679))
|
||||
- **Per-platform home-channel notification toggles** ([#19864](https://github.com/NousResearch/hermes-agent/pull/19864))
|
||||
- **Sharper home-channel toggle contrast + drop → running action** ([#19916](https://github.com/NousResearch/hermes-agent/pull/19916))
|
||||
- Fix: reject direct status transition to 'running' via dashboard API (salvage of #19554) ([#19705](https://github.com/NousResearch/hermes-agent/pull/19705))
|
||||
- Fix: dashboard board pin authoritative over server current file (#20879) ([#21230](https://github.com/NousResearch/hermes-agent/pull/21230))
|
||||
- Fix: treat dashboard event-stream cancellation as normal shutdown (#20790) ([#21222](https://github.com/NousResearch/hermes-agent/pull/21222))
|
||||
- Fix: filter dashboard board by selected tenant (#19817) ([#21349](https://github.com/NousResearch/hermes-agent/pull/21349))
|
||||
- Fix: code/pre styling theme-immune across all themes (#21086) ([#21247](https://github.com/NousResearch/hermes-agent/pull/21247))
|
||||
- Fix: reset `<code>` background inside dashboard board ([#20687](https://github.com/NousResearch/hermes-agent/pull/20687))
|
||||
- Fix: preserve dashboard completion summaries + add kanban edit (salvages #20016) ([#20195](https://github.com/NousResearch/hermes-agent/pull/20195))
|
||||
- Fix: avoid fragile failure-column renames (salvage #20848) (@kshitijk4poor) ([#20855](https://github.com/NousResearch/hermes-agent/pull/20855))
|
||||
|
||||
### Worker lifecycle + reliability
|
||||
- **Heartbeat + reclaim + zombie + retry-cap fixes** (#21147, #21141, #21169, #20881) ([#21183](https://github.com/NousResearch/hermes-agent/pull/21183))
|
||||
- **Auto-block workers that exit without completing + shutdown race** (#20894) ([#21214](https://github.com/NousResearch/hermes-agent/pull/21214))
|
||||
- **Detect darwin zombie workers** (salvages #20023) ([#20188](https://github.com/NousResearch/hermes-agent/pull/20188))
|
||||
- **Unify failure counter across spawn/timeout/crash outcomes** ([#20410](https://github.com/NousResearch/hermes-agent/pull/20410))
|
||||
- **Enforce worker task-ownership on destructive tool calls** ([#19713](https://github.com/NousResearch/hermes-agent/pull/19713))
|
||||
- **Drop worker identity claim from KANBAN_GUIDANCE** ([#19427](https://github.com/NousResearch/hermes-agent/pull/19427))
|
||||
- Fix: skip dispatch for tasks assigned to non-profile lanes (salvages #20105, #20134) ([#20165](https://github.com/NousResearch/hermes-agent/pull/20165))
|
||||
- Fix: include default profile in on-disk assignee enumeration (salvages #20123) ([#20170](https://github.com/NousResearch/hermes-agent/pull/20170))
|
||||
- Fix: ignore stale current board pointers (salvages #20063) ([#20183](https://github.com/NousResearch/hermes-agent/pull/20183))
|
||||
- Fix: profile discovery ignores HERMES_HOME in custom-root deployments (@jackey8616) ([#19020](https://github.com/NousResearch/hermes-agent/pull/19020))
|
||||
- Fix: allow orchestrator profiles to see kanban tools via toolsets config ([#19606](https://github.com/NousResearch/hermes-agent/pull/19606))
|
||||
|
||||
### Batch salvages
|
||||
- Tier-1 batch — metadata test, max_spawn config, run-id lifecycle guard (salvages #19522 #19556 #19829) ([#20440](https://github.com/NousResearch/hermes-agent/pull/20440))
|
||||
- Tier-2 batch — doctor, started_at, parent-guard, latest_summary, selects, linked-children ([#20448](https://github.com/NousResearch/hermes-agent/pull/20448))
|
||||
|
||||
### Documentation
|
||||
- Backfill multi-board refs in reference docs ([#19704](https://github.com/NousResearch/hermes-agent/pull/19704))
|
||||
- Document `/kanban` slash command ([#19584](https://github.com/NousResearch/hermes-agent/pull/19584))
|
||||
- Document recommended handoff evidence metadata (salvage #19512) ([#20415](https://github.com/NousResearch/hermes-agent/pull/20415))
|
||||
- Fix orchestrator + worker skill setup instructions (@helix4u) ([#20958](https://github.com/NousResearch/hermes-agent/pull/20958), [#20960](https://github.com/NousResearch/hermes-agent/pull/20960))
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Persistent Goals, Checkpoints & Session Durability
|
||||
|
||||
### `/goal` — persistent cross-turn goals (Ralph loop)
|
||||
- **`feat: /goal — persistent cross-turn goals`** ([#18262](https://github.com/NousResearch/hermes-agent/pull/18262))
|
||||
- **Docs page — Persistent Goals (/goal)** ([#18275](https://github.com/NousResearch/hermes-agent/pull/18275))
|
||||
- Fix: honor configured goal turn budget (salvage #19423) ([#21287](https://github.com/NousResearch/hermes-agent/pull/21287))
|
||||
|
||||
### Checkpoints v2
|
||||
- **Single-store rewrite with real pruning + disk guardrails** ([#20709](https://github.com/NousResearch/hermes-agent/pull/20709))
|
||||
|
||||
### Session durability
|
||||
- **Auto-resume interrupted sessions after gateway restart** (salvage #20888) ([#21192](https://github.com/NousResearch/hermes-agent/pull/21192))
|
||||
- **Preserve pending update prompts across restarts** ([#20160](https://github.com/NousResearch/hermes-agent/pull/20160))
|
||||
- **Preserve home-channel thread targets across restart notifications** (salvage #18440) ([#19271](https://github.com/NousResearch/hermes-agent/pull/19271))
|
||||
- **Preserve thread routing from cached live session sources** ([#21206](https://github.com/NousResearch/hermes-agent/pull/21206))
|
||||
- **Preserve assistant metadata when branching sessions** ([#18222](https://github.com/NousResearch/hermes-agent/pull/18222))
|
||||
- **Preserve thread routing for /update progress and prompts** ([#18193](https://github.com/NousResearch/hermes-agent/pull/18193))
|
||||
- **Preserve document type when merging queued events** ([#18215](https://github.com/NousResearch/hermes-agent/pull/18215))
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ Security & Reliability
|
||||
|
||||
### Security hardening (8 P0 closures)
|
||||
- **Enable secret redaction by default** (#17691, #20785) ([#21193](https://github.com/NousResearch/hermes-agent/pull/21193))
|
||||
- **Discord — scope `DISCORD_ALLOWED_ROLES` to originating guild** (#12136, CVSS 8.1) ([#21241](https://github.com/NousResearch/hermes-agent/pull/21241))
|
||||
- **WhatsApp — reject strangers by default, never respond in self-chat** (#8389) ([#21291](https://github.com/NousResearch/hermes-agent/pull/21291))
|
||||
- **MCP OAuth — close TOCTOU window when saving credentials** ([#21176](https://github.com/NousResearch/hermes-agent/pull/21176))
|
||||
- **`hermes_cli/auth.py` — close TOCTOU window in credential writers** ([#21194](https://github.com/NousResearch/hermes-agent/pull/21194))
|
||||
- **Browser — enforce cloud-metadata SSRF floor in hybrid routing** (#16234) ([#21228](https://github.com/NousResearch/hermes-agent/pull/21228))
|
||||
- **`hermes debug share` — redact log content at upload time** (@GodsBoy) ([#19318](https://github.com/NousResearch/hermes-agent/pull/19318))
|
||||
- **Cron — scan assembled prompt including skill content for prompt injection** (#3968) ([#21350](https://github.com/NousResearch/hermes-agent/pull/21350))
|
||||
- **Restore .env/auth.json/state.db with 0600 perms** ([#19699](https://github.com/NousResearch/hermes-agent/pull/19699))
|
||||
- **SRI integrity for dashboard plugin scripts** (salvage #19389) ([#21277](https://github.com/NousResearch/hermes-agent/pull/21277))
|
||||
- **Bind Meet node server to localhost, restrict token file to owner read** ([#19597](https://github.com/NousResearch/hermes-agent/pull/19597))
|
||||
- **Extend sensitive-write target to cover shell RC and credential files** ([#19282](https://github.com/NousResearch/hermes-agent/pull/19282))
|
||||
- **Harden YOLO mode env parsing against quoted-bool strings** ([#18214](https://github.com/NousResearch/hermes-agent/pull/18214))
|
||||
- **OSV-Scanner CI + Dependabot for github-actions only** ([#20037](https://github.com/NousResearch/hermes-agent/pull/20037))
|
||||
|
||||
### Reliability — critical bug closures
|
||||
- **CLI crash on startup — `Invalid key 'c-S-c'`** (P0, prompt_toolkit doesn't support Shift modifier) ([#19895](https://github.com/NousResearch/hermes-agent/pull/19895), [#19919](https://github.com/NousResearch/hermes-agent/pull/19919))
|
||||
- **CLOSE_WAIT fd leak audit** — httpx keepalive + WhatsApp aiohttp leak + Feishu hygiene (#18451) ([#18766](https://github.com/NousResearch/hermes-agent/pull/18766))
|
||||
- **Gateway creates AIAgent with empty OpenRouter API key when OPENROUTER_API_KEY is missing** (#20982) — fallback providers correctly honored
|
||||
- **Background review + curator protected from overwriting bundled/hub skills** (#20273) ([#20194](https://github.com/NousResearch/hermes-agent/pull/20194))
|
||||
- **TUI compression continuation — ghost sessions with incomplete metadata** (#20001)
|
||||
- **`hermes mcp add` silently launches chat instead of registering MCP server** (#19785) ([#21204](https://github.com/NousResearch/hermes-agent/pull/21204))
|
||||
- **Background review agent runtime propagation** — provider/model/credentials now actually inherit from parent
|
||||
- **Inbound document host paths translated to container paths for Docker backend** (salvage #19048) ([#21184](https://github.com/NousResearch/hermes-agent/pull/21184))
|
||||
- **Matrix gateway race between auto-redaction and message delivery with high-speed models** (#19075)
|
||||
- **`/new` during active agent session never sends response on Telegram** (#18912)
|
||||
|
||||
---
|
||||
|
||||
## 📱 Messaging Platforms (Gateway)
|
||||
|
||||
### New platform
|
||||
- **Google Chat — 20th platform** + generic `env_enablement_fn` / `cron_deliver_env_var` platform-plugin hooks (IRC + Teams migrated) ([#21306](https://github.com/NousResearch/hermes-agent/pull/21306), [#21331](https://github.com/NousResearch/hermes-agent/pull/21331))
|
||||
|
||||
### Cross-platform
|
||||
- **`allowed_{channels,chats,rooms}` whitelist** — Slack (salvage #7401), Telegram, Mattermost, Matrix, DingTalk ([#21251](https://github.com/NousResearch/hermes-agent/pull/21251))
|
||||
- **Per-platform `gateway_restart_notification` flag** ([#20892](https://github.com/NousResearch/hermes-agent/pull/20892))
|
||||
- **`busy_ack_enabled` config — suppress ack messages** ([#18194](https://github.com/NousResearch/hermes-agent/pull/18194))
|
||||
- **Auto-delete slash-command system notices after TTL** ([#18266](https://github.com/NousResearch/hermes-agent/pull/18266))
|
||||
- **Opt-in cleanup of temporary progress bubbles** ([#21186](https://github.com/NousResearch/hermes-agent/pull/21186))
|
||||
- **`[[as_document]]` directive — skill media routing** (salvage #19069) ([#21210](https://github.com/NousResearch/hermes-agent/pull/21210))
|
||||
- **`hermes gateway list` — cross-profile status** (salvage #19129) ([#21225](https://github.com/NousResearch/hermes-agent/pull/21225))
|
||||
- **Auto-resume interrupted sessions after restart** (salvage #20888) ([#21192](https://github.com/NousResearch/hermes-agent/pull/21192))
|
||||
- **Atomic restart markers + Windows runtime-lock offset** (#17842) ([#18179](https://github.com/NousResearch/hermes-agent/pull/18179))
|
||||
- Fix: `config.yaml` wins over `.env` for agent/display/timezone settings ([#18764](https://github.com/NousResearch/hermes-agent/pull/18764))
|
||||
- Fix: auto-restart when source files change out from under us (#17648) ([#18409](https://github.com/NousResearch/hermes-agent/pull/18409))
|
||||
- Fix: use git HEAD SHA for stale-code check, not file mtimes ([#19740](https://github.com/NousResearch/hermes-agent/pull/19740))
|
||||
- Fix: shutdown + restart hygiene — drain timeout, false-fatal, success log ([#18761](https://github.com/NousResearch/hermes-agent/pull/18761))
|
||||
- Fix: preserve max_turns after env reload (salvage #19183) ([#21240](https://github.com/NousResearch/hermes-agent/pull/21240))
|
||||
- Fix: exclude ancestor PIDs from gateway process scan ([#19586](https://github.com/NousResearch/hermes-agent/pull/19586))
|
||||
- Fix: move quick-command alias dispatch before built-ins ([#19588](https://github.com/NousResearch/hermes-agent/pull/19588))
|
||||
- Fix: show other profiles in 'gateway status' to prevent confusion ([#19582](https://github.com/NousResearch/hermes-agent/pull/19582))
|
||||
- Fix: include external_dirs skills in Telegram/Discord slash commands (salvage #8790) ([#18741](https://github.com/NousResearch/hermes-agent/pull/18741))
|
||||
- Fix: match disabled/optional skills by frontmatter slug, not dir name ([#18753](https://github.com/NousResearch/hermes-agent/pull/18753))
|
||||
- Fix: read /status token totals from SessionDB (#17158) ([#18206](https://github.com/NousResearch/hermes-agent/pull/18206))
|
||||
- Fix: snapshot callback generation after agent binds it, not before ([#18219](https://github.com/NousResearch/hermes-agent/pull/18219))
|
||||
- Fix: re-inject topic-bound skill after /new or /reset ([#18205](https://github.com/NousResearch/hermes-agent/pull/18205))
|
||||
- Fix: isolate pending native image paths by session ([#18202](https://github.com/NousResearch/hermes-agent/pull/18202))
|
||||
- Fix: clear queued reload skills notes on new/resume/branch ([#19431](https://github.com/NousResearch/hermes-agent/pull/19431))
|
||||
- Fix: hide required-arg commands from Telegram menu ([#19400](https://github.com/NousResearch/hermes-agent/pull/19400))
|
||||
- Fix: bridge top-level `require_mention` to Telegram config ([#19429](https://github.com/NousResearch/hermes-agent/pull/19429))
|
||||
- Fix: suppress duplicate voice transcripts ([#19428](https://github.com/NousResearch/hermes-agent/pull/19428))
|
||||
- Fix: show friendly error when service is not installed ([#19707](https://github.com/NousResearch/hermes-agent/pull/19707))
|
||||
- Fix: read context_length from custom_providers in session info header ([#19708](https://github.com/NousResearch/hermes-agent/pull/19708))
|
||||
- Fix: preserve WSL interop PATH in systemd units ([#19867](https://github.com/NousResearch/hermes-agent/pull/19867))
|
||||
- Fix: handle planned service stops (salvage #19876) ([#19936](https://github.com/NousResearch/hermes-agent/pull/19936))
|
||||
- Fix: keep DoH-confirmed Telegram IPs that match system DNS (salvage #17043) ([#20175](https://github.com/NousResearch/hermes-agent/pull/20175))
|
||||
- Fix: load `reply_to_mode` from config.yaml for Discord + Telegram (salvage #17117) ([#20171](https://github.com/NousResearch/hermes-agent/pull/20171))
|
||||
- Fix: tolerate malformed HERMES_HUMAN_DELAY_* env vars (salvage #16933) ([#20217](https://github.com/NousResearch/hermes-agent/pull/20217))
|
||||
- Fix: deterministic thread eviction preserves newest entries (salvage #13639) ([#20285](https://github.com/NousResearch/hermes-agent/pull/20285))
|
||||
- Fix: don't dead-end setup wizard when only system-scope unit is installed ([#20905](https://github.com/NousResearch/hermes-agent/pull/20905))
|
||||
- Fix: wait for systemd restart readiness + harden Discord slash-command sync ([#20949](https://github.com/NousResearch/hermes-agent/pull/20949))
|
||||
- Fix: avoid duplicated Responses history (salvage #18995) ([#21185](https://github.com/NousResearch/hermes-agent/pull/21185))
|
||||
- Fix: surface bootstrap failures to stderr (salvage #21157) ([#21278](https://github.com/NousResearch/hermes-agent/pull/21278))
|
||||
- Fix: log agent task failures instead of silently losing usage data (salvage #21159) ([#21274](https://github.com/NousResearch/hermes-agent/pull/21274))
|
||||
- Fix: log runtime-status write failures with rate-limiting (salvage #21158) ([#21285](https://github.com/NousResearch/hermes-agent/pull/21285))
|
||||
- Fix: reset-failed before every fallback restart so the gateway can't get stranded ([#21371](https://github.com/NousResearch/hermes-agent/pull/21371))
|
||||
- Fix: Telegram — preserve `thread_id=1` for forum General typing indicator ([#21390](https://github.com/NousResearch/hermes-agent/pull/21390))
|
||||
- Fix: batch critical fixes — session resume, /new race, HA WebSocket scheme (@kshitijk4poor) ([#19182](https://github.com/NousResearch/hermes-agent/pull/19182))
|
||||
|
||||
### Telegram
|
||||
- **DM user-managed multi-session topics** (salvage of #19185) ([#19206](https://github.com/NousResearch/hermes-agent/pull/19206))
|
||||
|
||||
### Discord
|
||||
- **Message deletion action** (salvage #19052) ([#21197](https://github.com/NousResearch/hermes-agent/pull/21197))
|
||||
- Fix: allow `free_response_channels` to override `DISCORD_IGNORE_NO_MENTION` ([#19629](https://github.com/NousResearch/hermes-agent/pull/19629))
|
||||
|
||||
### Slack
|
||||
- Fix: ephemeral slash-command ack, private notice delivery, format_message fixes (@kshitijk4poor) ([#18198](https://github.com/NousResearch/hermes-agent/pull/18198))
|
||||
|
||||
### WhatsApp
|
||||
- Fix: load WhatsApp home channel from env overrides ([#18190](https://github.com/NousResearch/hermes-agent/pull/18190))
|
||||
|
||||
### Feishu
|
||||
- **Operator-configurable bot admission and mention policy** ([#18208](https://github.com/NousResearch/hermes-agent/pull/18208))
|
||||
- Fix: force text mode for markdown tables (salvage of #13723 by @WuTianyi123) ([#20275](https://github.com/NousResearch/hermes-agent/pull/20275))
|
||||
|
||||
### Matrix + Email
|
||||
- Fix: `/sethome` on Matrix and Email now persists across restarts ([#18272](https://github.com/NousResearch/hermes-agent/pull/18272))
|
||||
|
||||
### Teams
|
||||
- **Docs + feat: sidebar + threading with group-chat fallback** ([#20042](https://github.com/NousResearch/hermes-agent/pull/20042))
|
||||
|
||||
### Weixin
|
||||
- Fix: deduplicate Weixin messages by content fingerprint ([#19742](https://github.com/NousResearch/hermes-agent/pull/19742))
|
||||
|
||||
### QQBot
|
||||
- **Port SDK improvements in-tree — chunked upload, approval keyboards, quoted attachments** ([#21342](https://github.com/NousResearch/hermes-agent/pull/21342))
|
||||
- **Wire native tool-approval UX via inline keyboards** ([#21353](https://github.com/NousResearch/hermes-agent/pull/21353))
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Core Agent & Architecture
|
||||
|
||||
### Provider & Model Support
|
||||
|
||||
#### Pluggable providers
|
||||
- **ProviderProfile ABC + `plugins/model-providers/`** — inference providers are now a pluggable surface (salvage of #14424) ([#20324](https://github.com/NousResearch/hermes-agent/pull/20324))
|
||||
- **`list_picker_providers`** — credential-filtered picker (salvage #13561) ([#20298](https://github.com/NousResearch/hermes-agent/pull/20298))
|
||||
- **Remove `/provider` alias for `/model`** ([#20358](https://github.com/NousResearch/hermes-agent/pull/20358))
|
||||
- **Shared Hermes dotenv loader across CLI + plugins** (salvage #13660) ([#20281](https://github.com/NousResearch/hermes-agent/pull/20281))
|
||||
- **Nous OAuth persisted across profiles via shared token store** ([#19712](https://github.com/NousResearch/hermes-agent/pull/19712))
|
||||
|
||||
#### New models
|
||||
- `deepseek/deepseek-v4-pro` added to OpenRouter + Nous Portal ([#20495](https://github.com/NousResearch/hermes-agent/pull/20495))
|
||||
- `x-ai/grok-4.3` added to OpenRouter + Nous Portal ([#20497](https://github.com/NousResearch/hermes-agent/pull/20497))
|
||||
- `openrouter/owl-alpha` (free tier) added to curated OpenRouter list ([#18071](https://github.com/NousResearch/hermes-agent/pull/18071))
|
||||
- `tencent/hy3-preview` paid route on OpenRouter (@Contentment003111) ([#21077](https://github.com/NousResearch/hermes-agent/pull/21077))
|
||||
- Arcee Trinity Large Thinking — temperature + compression overrides ([#20473](https://github.com/NousResearch/hermes-agent/pull/20473))
|
||||
- Rename `x-ai/grok-4.20-beta` to `x-ai/grok-4.20` ([#19640](https://github.com/NousResearch/hermes-agent/pull/19640))
|
||||
- Demote Vercel AI Gateway to bottom of provider picker ([#18112](https://github.com/NousResearch/hermes-agent/pull/18112))
|
||||
|
||||
#### Provider configuration
|
||||
- **OpenRouter — response caching support** (@kshitijk4poor) ([#19132](https://github.com/NousResearch/hermes-agent/pull/19132))
|
||||
- **`image_gen.model` from config.yaml honored** (salvage #19376) ([#21273](https://github.com/NousResearch/hermes-agent/pull/21273))
|
||||
- Fix: honor runtime default model during delegate provider resolution (@johnncenae) ([#17587](https://github.com/NousResearch/hermes-agent/pull/17587))
|
||||
- Fix: avoid Bedrock credential probe in provider picker (@helix4u) ([#18998](https://github.com/NousResearch/hermes-agent/pull/18998))
|
||||
- Fix: drop stale env-var override of persisted provider for cron ([#19627](https://github.com/NousResearch/hermes-agent/pull/19627))
|
||||
- Fix: auxiliary curator api_key/base_url into runtime resolution ([#19421](https://github.com/NousResearch/hermes-agent/pull/19421))
|
||||
|
||||
### Agent Loop & Conversation
|
||||
- **`video_analyze` — native video understanding tool** (@alt-glitch) ([#19301](https://github.com/NousResearch/hermes-agent/pull/19301))
|
||||
- **Show context compression count in status bar** (CLI + TUI) ([#21218](https://github.com/NousResearch/hermes-agent/pull/21218))
|
||||
- **Isolate `get_tool_definitions` quiet_mode cache + dedup LCM injection** (#17335) ([#17889](https://github.com/NousResearch/hermes-agent/pull/17889))
|
||||
- Fix: warning-first tool-call loop guardrails ([#18227](https://github.com/NousResearch/hermes-agent/pull/18227))
|
||||
- Fix: break permanent empty-response loop from orphan tool-tail ([#21385](https://github.com/NousResearch/hermes-agent/pull/21385))
|
||||
- Fix: propagate ContextVars to concurrent tool worker threads (salvage #16660) ([#18123](https://github.com/NousResearch/hermes-agent/pull/18123))
|
||||
- Fix: surface self-improvement review summaries across CLI, TUI, and gateway ([#18073](https://github.com/NousResearch/hermes-agent/pull/18073))
|
||||
- Fix: serialize concurrent `hermes_tools` RPC calls from `execute_code` ([#17894](https://github.com/NousResearch/hermes-agent/pull/17894), [#17902](https://github.com/NousResearch/hermes-agent/pull/17902))
|
||||
- Fix: include system prompt + tool schemas in token estimates for compression ([#18265](https://github.com/NousResearch/hermes-agent/pull/18265))
|
||||
|
||||
### Compression
|
||||
- Fix: skip non-string tool content in dedup pass to prevent AttributeError ([#19398](https://github.com/NousResearch/hermes-agent/pull/19398))
|
||||
- Fix: reset `_summary_failure_cooldown_until` on session reset ([#19622](https://github.com/NousResearch/hermes-agent/pull/19622))
|
||||
- Fix: trigger fallback on timeout errors alongside model-unavailable errors ([#19665](https://github.com/NousResearch/hermes-agent/pull/19665))
|
||||
- Fix: `_prune_old_tool_results` boundary direction ([#19725](https://github.com/NousResearch/hermes-agent/pull/19725))
|
||||
- Fix: soften summary prompt for content filters (salvage #19456) ([#21302](https://github.com/NousResearch/hermes-agent/pull/21302))
|
||||
|
||||
### Delegate
|
||||
- Fix: inherit parent fallback_chain in `_build_child_agent` ([#19601](https://github.com/NousResearch/hermes-agent/pull/19601))
|
||||
- Fix: guard `_load_config()` against `delegation: null` in config.yaml ([#19662](https://github.com/NousResearch/hermes-agent/pull/19662))
|
||||
- Fix: inherit parent api_key when `delegation.base_url` set without `delegation.api_key` ([#19741](https://github.com/NousResearch/hermes-agent/pull/19741))
|
||||
- Fix: expand composite toolsets before intersection (salvage #19455) ([#21300](https://github.com/NousResearch/hermes-agent/pull/21300))
|
||||
- Fix: correct ACP docs — Claude Code CLI has no --acp flag (salvage #19058) ([#21201](https://github.com/NousResearch/hermes-agent/pull/21201))
|
||||
|
||||
### Session & Memory
|
||||
- **Hindsight — probe API for `update_mode='append'` to dedupe across processes** (@nicoloboschi) ([#20222](https://github.com/NousResearch/hermes-agent/pull/20222))
|
||||
|
||||
### Curator
|
||||
- **`hermes curator archive` and `prune` subcommands** ([#20200](https://github.com/NousResearch/hermes-agent/pull/20200))
|
||||
- **`hermes curator list-archived`** (#20651) ([#21236](https://github.com/NousResearch/hermes-agent/pull/21236))
|
||||
- **Synchronous manual `hermes curator run`** (#20555) ([#21216](https://github.com/NousResearch/hermes-agent/pull/21216))
|
||||
- Fix: preserve `last_report_path` in state ([#18169](https://github.com/NousResearch/hermes-agent/pull/18169))
|
||||
- Fix: rewrite cron job skill refs after consolidation ([#18253](https://github.com/NousResearch/hermes-agent/pull/18253))
|
||||
- Fix: defer first run + `--dry-run` preview (#18373) ([#18389](https://github.com/NousResearch/hermes-agent/pull/18389))
|
||||
- Fix: authoritative `absorbed_into` on delete + restore cron skill links on rollback (#18671) ([#18731](https://github.com/NousResearch/hermes-agent/pull/18731))
|
||||
- Fix: prevent false-positive consolidation from substring matching ([#19573](https://github.com/NousResearch/hermes-agent/pull/19573))
|
||||
- Fix: only mark agent-created for background-review sediment ([#19621](https://github.com/NousResearch/hermes-agent/pull/19621))
|
||||
- Fix: protect hub skills by frontmatter name ([#20194](https://github.com/NousResearch/hermes-agent/pull/20194))
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Tool System
|
||||
|
||||
### File tools
|
||||
- **Post-write delta lint on `write_file` + `patch`** — in-proc linters for Python, JSON, YAML, TOML ([#20191](https://github.com/NousResearch/hermes-agent/pull/20191))
|
||||
|
||||
### Cron
|
||||
- **`no_agent` mode — script-only cron jobs (watchdog pattern)** ([#19709](https://github.com/NousResearch/hermes-agent/pull/19709))
|
||||
- **`context_from` chaining docs** (salvage #15724) ([#20394](https://github.com/NousResearch/hermes-agent/pull/20394))
|
||||
- Fix: treat non-dict origin as missing instead of crashing tick ([#19283](https://github.com/NousResearch/hermes-agent/pull/19283))
|
||||
- Fix: bump skill usage when cron jobs load skills ([#19433](https://github.com/NousResearch/hermes-agent/pull/19433))
|
||||
- Fix: recover null `next_run_at` jobs ([#19576](https://github.com/NousResearch/hermes-agent/pull/19576))
|
||||
- Fix: skip AI call when prerun script produces no output ([#19628](https://github.com/NousResearch/hermes-agent/pull/19628))
|
||||
- Fix: expand config.yaml refs during job execution ([#19872](https://github.com/NousResearch/hermes-agent/pull/19872))
|
||||
- Fix: serialize `get_due_jobs` writes to prevent parallel state corruption ([#19874](https://github.com/NousResearch/hermes-agent/pull/19874))
|
||||
- Fix: initialize MCP servers before constructing the cron AIAgent ([#21354](https://github.com/NousResearch/hermes-agent/pull/21354))
|
||||
|
||||
### MCP
|
||||
- **SSE transport support** (salvage #19135) ([#21227](https://github.com/NousResearch/hermes-agent/pull/21227))
|
||||
- **Forward OAuth auth + bump `sse_read_timeout` on SSE transport** ([#21323](https://github.com/NousResearch/hermes-agent/pull/21323))
|
||||
- **Retry stale pipe transport failures as session-expired** ([#21289](https://github.com/NousResearch/hermes-agent/pull/21289))
|
||||
- **Surface image tool results as MEDIA tags instead of dropping them** ([#21328](https://github.com/NousResearch/hermes-agent/pull/21328))
|
||||
- **Periodic keepalive to `_wait_for_lifecycle_event`** (salvage #17016) ([#20209](https://github.com/NousResearch/hermes-agent/pull/20209))
|
||||
- Fix: reconnect on terminated sessions ([#19380](https://github.com/NousResearch/hermes-agent/pull/19380))
|
||||
- Fix: decouple AnyUrl import from mcp dependency ([#19695](https://github.com/NousResearch/hermes-agent/pull/19695))
|
||||
- Fix: `mcp add --command` gets distinct argparse dest ([#21204](https://github.com/NousResearch/hermes-agent/pull/21204))
|
||||
- Fix: clear stale thread interrupt before MCP discovery ([#21276](https://github.com/NousResearch/hermes-agent/pull/21276))
|
||||
- Fix: report configured timeout in MCP call errors ([#21281](https://github.com/NousResearch/hermes-agent/pull/21281))
|
||||
- Fix: include exception type in error messages when str(exc) is empty (salvage #19425) ([#21292](https://github.com/NousResearch/hermes-agent/pull/21292))
|
||||
- Fix: re-raise CancelledError explicitly in `MCPServerTask.run` ([#21318](https://github.com/NousResearch/hermes-agent/pull/21318))
|
||||
- Fix: coerce numeric tool args defensively in `mcp_serve` ([#21329](https://github.com/NousResearch/hermes-agent/pull/21329))
|
||||
- Fix: gate utility stubs on server-advertised capabilities ([#21347](https://github.com/NousResearch/hermes-agent/pull/21347))
|
||||
|
||||
### Browser
|
||||
- Fix: allow explicit CDP override without local agent-browser ([#19670](https://github.com/NousResearch/hermes-agent/pull/19670))
|
||||
- Fix: inject `--no-sandbox` for root + AppArmor userns restrictions ([#19747](https://github.com/NousResearch/hermes-agent/pull/19747))
|
||||
- Fix: tighten Lightpanda fallback edge cases (@kshitijk4poor) ([#20672](https://github.com/NousResearch/hermes-agent/pull/20672))
|
||||
|
||||
### Web tools
|
||||
- **Per-capability backend selection — search/extract split** (@kshitijk4poor) ([#20061](https://github.com/NousResearch/hermes-agent/pull/20061))
|
||||
- **SearXNG native search-only backend** (@kshitijk4poor) ([#20823](https://github.com/NousResearch/hermes-agent/pull/20823))
|
||||
|
||||
### Approval / Tool gating
|
||||
- Fix: wake blocked gateway approvals on session cleanup ([#18171](https://github.com/NousResearch/hermes-agent/pull/18171))
|
||||
- Fix: harden YOLO mode env parsing against quoted-bool strings ([#18214](https://github.com/NousResearch/hermes-agent/pull/18214))
|
||||
- Fix: extend sensitive write target to cover shell RC and credential files ([#19282](https://github.com/NousResearch/hermes-agent/pull/19282))
|
||||
|
||||
---
|
||||
|
||||
## 🔌 Plugin System
|
||||
|
||||
- **`transform_llm_output` plugin hook** (salvage of #20813) ([#21235](https://github.com/NousResearch/hermes-agent/pull/21235))
|
||||
- **Document `env_enablement_fn` + `cron_deliver_env_var` platform-plugin hooks** ([#21331](https://github.com/NousResearch/hermes-agent/pull/21331))
|
||||
- **Pluggable surfaces coverage — model-provider guide, full plugin map, opt-in fix** ([#20749](https://github.com/NousResearch/hermes-agent/pull/20749))
|
||||
- **Plugin-authoring gaps — image-gen provider guide + publishing a skill tap** ([#20800](https://github.com/NousResearch/hermes-agent/pull/20800))
|
||||
|
||||
---
|
||||
|
||||
## 🧩 Skills Ecosystem
|
||||
|
||||
### New optional skills
|
||||
- **Shopify** — Admin + Storefront GraphQL optional skill ([#18116](https://github.com/NousResearch/hermes-agent/pull/18116))
|
||||
- **here.now** — optional skill ([#18170](https://github.com/NousResearch/hermes-agent/pull/18170))
|
||||
- **shop-app** — personal shopping assistant (optional) ([#20702](https://github.com/NousResearch/hermes-agent/pull/20702))
|
||||
- **Anthropic financial-services bundle** — ported as optional finance skills ([#21180](https://github.com/NousResearch/hermes-agent/pull/21180))
|
||||
- **kanban-video-orchestrator** — creative optional skill (@SHL0MS) ([#19281](https://github.com/NousResearch/hermes-agent/pull/19281))
|
||||
- **searxng-search** — optional skill + Web Search + Extract docs page (@kshitijk4poor) ([#20841](https://github.com/NousResearch/hermes-agent/pull/20841), [#20844](https://github.com/NousResearch/hermes-agent/pull/20844))
|
||||
|
||||
### Skill UX
|
||||
- **Linear skill — add Documents support + Python helper script** ([#20752](https://github.com/NousResearch/hermes-agent/pull/20752))
|
||||
- **Modernize Obsidian skill to use file tools** (salvage #19332) ([#20413](https://github.com/NousResearch/hermes-agent/pull/20413))
|
||||
- **Default custom tool creation to plugins** (@kshitijk4poor) ([#19755](https://github.com/NousResearch/hermes-agent/pull/19755))
|
||||
- **skill_commands cache — rescan on platform scope changes** (salvage #14570 by @LeonSGP43) ([#18739](https://github.com/NousResearch/hermes-agent/pull/18739))
|
||||
- **Skills — additional rescan paths in skill_commands cache** (salvage #19042) ([#21181](https://github.com/NousResearch/hermes-agent/pull/21181))
|
||||
- Fix: regression tests for non-dict metadata in `extract_skill_conditions` ([#18213](https://github.com/NousResearch/hermes-agent/pull/18213))
|
||||
- Docs: explain restoring bundled skills (salvage #19254) ([#20404](https://github.com/NousResearch/hermes-agent/pull/20404))
|
||||
- Docs: document `hermes skills reset` subcommand (salvage #11544) ([#20395](https://github.com/NousResearch/hermes-agent/pull/20395))
|
||||
- Docs: himalaya v1.2.0 `folder.aliases` syntax ([#19882](https://github.com/NousResearch/hermes-agent/pull/19882))
|
||||
- Point agent at `hermes-agent` skill + docs site sync ([#20390](https://github.com/NousResearch/hermes-agent/pull/20390))
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ CLI & User Experience
|
||||
|
||||
### CLI
|
||||
- **`/new` accepts optional session name argument** (salvage of #19555) ([#19637](https://github.com/NousResearch/hermes-agent/pull/19637))
|
||||
- **100 new CLI startup tips** ([#20168](https://github.com/NousResearch/hermes-agent/pull/20168))
|
||||
- **`display.language` — static message translation** (zh/ja/de/es) ([#20231](https://github.com/NousResearch/hermes-agent/pull/20231))
|
||||
- **French (fr) locale** (@Foolafroos) ([#20329](https://github.com/NousResearch/hermes-agent/pull/20329))
|
||||
- **Ukrainian (uk) locale** ([#20467](https://github.com/NousResearch/hermes-agent/pull/20467))
|
||||
- **Turkish (tr) locale** ([#20474](https://github.com/NousResearch/hermes-agent/pull/20474))
|
||||
- Fix: recover classic CLI output after resize (@helix4u) ([#20444](https://github.com/NousResearch/hermes-agent/pull/20444))
|
||||
- Fix: complete absolute paths as paths (@helix4u) ([#19930](https://github.com/NousResearch/hermes-agent/pull/19930))
|
||||
- Fix: resolve lazy session creation regressions (#18370 fallout) (@alt-glitch) ([#20363](https://github.com/NousResearch/hermes-agent/pull/20363))
|
||||
- Fix: local backend CLI always uses launch directory (@alt-glitch) ([#19334](https://github.com/NousResearch/hermes-agent/pull/19334))
|
||||
- Refactor: drop dead c-S-c key binding (follow-up to #19895) ([#19919](https://github.com/NousResearch/hermes-agent/pull/19919))
|
||||
|
||||
### TUI (Ink)
|
||||
- **`/model` picker overhaul to match `hermes model` with inline auth** (@austinpickett) ([#18117](https://github.com/NousResearch/hermes-agent/pull/18117))
|
||||
- **Collapsible sections in startup banner** — skills, system prompt, MCP (@kshitijk4poor) ([#20625](https://github.com/NousResearch/hermes-agent/pull/20625))
|
||||
- **Show context compression count in status bar** ([#21218](https://github.com/NousResearch/hermes-agent/pull/21218))
|
||||
- Perf: reduce overlay render churn with focused selectors (@OutThisLife) ([#20393](https://github.com/NousResearch/hermes-agent/pull/20393))
|
||||
- Fix: restore voice push-to-talk parity (salvage of #16189 by @Montbra) (@OutThisLife) ([#20897](https://github.com/NousResearch/hermes-agent/pull/20897))
|
||||
- Fix: kanban button (@austinpickett) ([#18358](https://github.com/NousResearch/hermes-agent/pull/18358))
|
||||
|
||||
### Dashboard
|
||||
- **Plugins page — manage, enable/disable, auth status** (@austinpickett) ([#18095](https://github.com/NousResearch/hermes-agent/pull/18095))
|
||||
- **Profiles management page** (@vincez-hms-coder) ([#16419](https://github.com/NousResearch/hermes-agent/pull/16419))
|
||||
- **Interactive column sorting in analytics tables** ([#18192](https://github.com/NousResearch/hermes-agent/pull/18192))
|
||||
- **`default-large` built-in theme with 18px base size** ([#20820](https://github.com/NousResearch/hermes-agent/pull/20820))
|
||||
- **Support serving under URL prefix via `X-Forwarded-Prefix`** (salvage #19450) ([#21296](https://github.com/NousResearch/hermes-agent/pull/21296))
|
||||
- **Launch dashboard as side-process via `HERMES_DASHBOARD=1` in Docker** (@benbarclay) ([#19540](https://github.com/NousResearch/hermes-agent/pull/19540))
|
||||
- Fix: dashboard theme layout shift (@AllardQuek) ([#17232](https://github.com/NousResearch/hermes-agent/pull/17232))
|
||||
- Fix: gateway model picker current context (@helix4u) ([#20513](https://github.com/NousResearch/hermes-agent/pull/20513))
|
||||
|
||||
### Update + setup
|
||||
- **`hermes update --yes/-y` to skip interactive prompts** ([#18261](https://github.com/NousResearch/hermes-agent/pull/18261))
|
||||
- **Restart manual profile gateways after update** ([#18178](https://github.com/NousResearch/hermes-agent/pull/18178))
|
||||
|
||||
### Profiles
|
||||
- **`--no-skills` flag for empty profile creation** ([#20986](https://github.com/NousResearch/hermes-agent/pull/20986))
|
||||
|
||||
---
|
||||
|
||||
## 🎵 Voice, Image & Media
|
||||
|
||||
- **xAI Custom Voices — voice cloning** (@alt-glitch) ([#18776](https://github.com/NousResearch/hermes-agent/pull/18776))
|
||||
- **Achievements — share card render on unlocked badges** ([#19657](https://github.com/NousResearch/hermes-agent/pull/19657))
|
||||
- **Refresh systemd unit on gateway boot (not just start/restart)** (@alt-glitch) ([#19684](https://github.com/NousResearch/hermes-agent/pull/19684))
|
||||
|
||||
---
|
||||
|
||||
## 🔗 API Server & Remote Access
|
||||
|
||||
- **`X-Hermes-Session-Key` header for long-term memory scoping** (closes #20060) ([#20199](https://github.com/NousResearch/hermes-agent/pull/20199))
|
||||
|
||||
---
|
||||
|
||||
## 🧰 ACP Adapter (VS Code / Zed / JetBrains)
|
||||
|
||||
- **`/steer` and `/queue` slash commands** (@HenkDz) ([#18114](https://github.com/NousResearch/hermes-agent/pull/18114))
|
||||
- Fix: translate Windows cwd for WSL sessions (salvage #18128) ([#18233](https://github.com/NousResearch/hermes-agent/pull/18233))
|
||||
- Fix: run `/steer` as a regular prompt on idle sessions ([#18258](https://github.com/NousResearch/hermes-agent/pull/18258))
|
||||
- Fix: route Zed thoughts to reasoning + polish tool/context rendering ([#19139](https://github.com/NousResearch/hermes-agent/pull/19139))
|
||||
- Fix: atomic session persistence via `replace_messages` (salvage #13675) ([#20279](https://github.com/NousResearch/hermes-agent/pull/20279))
|
||||
- Fix: preserve assistant reasoning metadata in session persistence (salvage #13575) ([#20296](https://github.com/NousResearch/hermes-agent/pull/20296))
|
||||
- Docs: update VS Code setup for ACP Client extension (salvage #12495) ([#20433](https://github.com/NousResearch/hermes-agent/pull/20433))
|
||||
|
||||
---
|
||||
|
||||
## 🐳 Docker
|
||||
|
||||
- **Launch dashboard as side-process via `HERMES_DASHBOARD=1`** (@benbarclay) ([#19540](https://github.com/NousResearch/hermes-agent/pull/19540))
|
||||
- **Refuse root gateway runs in official image** (salvage #19215) ([#21250](https://github.com/NousResearch/hermes-agent/pull/21250))
|
||||
- **Chown runtime `node_modules` trees to hermes user** (salvage #19303) ([#21267](https://github.com/NousResearch/hermes-agent/pull/21267))
|
||||
- Fix: exclude compose/profile runtime state from build context ([#19626](https://github.com/NousResearch/hermes-agent/pull/19626))
|
||||
- CI: don't cancel overlapping builds, guard `:latest` (@ethernet8023) ([#20890](https://github.com/NousResearch/hermes-agent/pull/20890))
|
||||
- Test: align Dockerfile contract tests with simplified TUI flow (salvage #19024) ([#21174](https://github.com/NousResearch/hermes-agent/pull/21174))
|
||||
- Docs: connect to local inference servers (vLLM, Ollama) (salvage #12335) ([#20407](https://github.com/NousResearch/hermes-agent/pull/20407))
|
||||
- Docs: document `API_SERVER_*` env vars (salvage #11758) ([#20409](https://github.com/NousResearch/hermes-agent/pull/20409))
|
||||
- Docs: clarify Docker terminal backend is a single persistent container ([#20003](https://github.com/NousResearch/hermes-agent/pull/20003))
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Notable Bug Fixes
|
||||
|
||||
### Agent
|
||||
- Fix: recover lazy session creation regressions (#18370 fallout) (@alt-glitch) ([#20363](https://github.com/NousResearch/hermes-agent/pull/20363))
|
||||
- Fix: propagate ContextVars to concurrent tool worker threads (salvage #16660) ([#18123](https://github.com/NousResearch/hermes-agent/pull/18123))
|
||||
- Fix: warning-first tool-call loop guardrails ([#18227](https://github.com/NousResearch/hermes-agent/pull/18227))
|
||||
- Fix: surface self-improvement review summaries across CLI, TUI, and gateway ([#18073](https://github.com/NousResearch/hermes-agent/pull/18073))
|
||||
|
||||
### Gateway streaming
|
||||
- Fix: harden StreamingConfig bool and numeric coercion (@simbam99) ([#16463](https://github.com/NousResearch/hermes-agent/pull/16463))
|
||||
|
||||
### Model
|
||||
- Fix: avoid Bedrock credential probe in provider picker (@helix4u) ([#18998](https://github.com/NousResearch/hermes-agent/pull/18998))
|
||||
|
||||
### Doctor
|
||||
- Fix: check global agent-browser when local install not found ([#19671](https://github.com/NousResearch/hermes-agent/pull/19671))
|
||||
- Test: kimi-coding-cn provider validation regression ([#19734](https://github.com/NousResearch/hermes-agent/pull/19734))
|
||||
|
||||
### Update
|
||||
- Fix: patch `isatty` on real streams to fix xdist-flaky `--yes` tests (salvage #19026) ([#21175](https://github.com/NousResearch/hermes-agent/pull/21175))
|
||||
- Fix: teach restart-mocks about the post-update survivor sweep (salvage #19031) ([#21177](https://github.com/NousResearch/hermes-agent/pull/21177))
|
||||
|
||||
### Auth
|
||||
- Fix: acp preserve assistant reasoning metadata ([#20296](https://github.com/NousResearch/hermes-agent/pull/20296))
|
||||
|
||||
### Redact
|
||||
- Fix: add `code_file` param to skip false-positive ENV/JSON patterns ([#19715](https://github.com/NousResearch/hermes-agent/pull/19715))
|
||||
|
||||
### Email
|
||||
- Fix: quoted-relative file-drop paths + Date header on tool email path ([#19646](https://github.com/NousResearch/hermes-agent/pull/19646))
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
- **ACP — accept prompt persistence kwargs in MCP E2E mocks** (@stephenschoettler) ([#18047](https://github.com/NousResearch/hermes-agent/pull/18047))
|
||||
- **Toolsets — include kanban in expected post-#17805 toolset assertions** (@briandevans) ([#18122](https://github.com/NousResearch/hermes-agent/pull/18122))
|
||||
- **Agent — cover max-iterations summary message sanitization** ([#19580](https://github.com/NousResearch/hermes-agent/pull/19580))
|
||||
- **run_agent — `-inf` and `nan` regression coverage for `_coerce_number`** ([#19703](https://github.com/NousResearch/hermes-agent/pull/19703))
|
||||
|
||||
---
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
### Major docs additions
|
||||
- **`llms.txt` + `llms-full.txt` — agent-friendly ingestion** ([#18276](https://github.com/NousResearch/hermes-agent/pull/18276))
|
||||
- **User Stories and Use Cases collage page** ([#18282](https://github.com/NousResearch/hermes-agent/pull/18282))
|
||||
- **Persistent Goals (/goal) feature page** ([#18275](https://github.com/NousResearch/hermes-agent/pull/18275))
|
||||
- **Windows (WSL2) guide expansion** — filesystem, networking, services, pitfalls ([#20748](https://github.com/NousResearch/hermes-agent/pull/20748))
|
||||
- **Chinese (zh-CN) README translation** (salvage #13508) ([#20431](https://github.com/NousResearch/hermes-agent/pull/20431))
|
||||
- **zh-Hans Docusaurus locale** + Tool Gateway / image-gen / WSL quickstart translations (salvage #11728) ([#20430](https://github.com/NousResearch/hermes-agent/pull/20430))
|
||||
- **Tool Gateway docs restructure** — lead with what it does, config moved to bottom ([#20827](https://github.com/NousResearch/hermes-agent/pull/20827))
|
||||
- **Quickstart — Onchain AI Garage Hermes tutorials playlist** ([#20192](https://github.com/NousResearch/hermes-agent/pull/20192))
|
||||
- **Open WebUI bootstrap script** (salvage #9566) ([#20427](https://github.com/NousResearch/hermes-agent/pull/20427))
|
||||
- **Local Ollama setup guide** (salvage #5842) ([#20426](https://github.com/NousResearch/hermes-agent/pull/20426))
|
||||
- **Google Gemini guide** (salvage #17450) ([#20401](https://github.com/NousResearch/hermes-agent/pull/20401))
|
||||
- **Custom model aliases for /model command** ([#20475](https://github.com/NousResearch/hermes-agent/pull/20475))
|
||||
- **Together/Groq/Perplexity cookbook via `custom_providers`** (salvage #15214) ([#20400](https://github.com/NousResearch/hermes-agent/pull/20400))
|
||||
- **Doubao speech integration examples** (TTS + STT) (salvage #18065) ([#20418](https://github.com/NousResearch/hermes-agent/pull/20418))
|
||||
- **WSL-to-Windows Chrome MCP bridge** (salvage #8313) ([#20428](https://github.com/NousResearch/hermes-agent/pull/20428))
|
||||
- **Hermes skills docs sync** — slash commands + durable-systems section ([#20390](https://github.com/NousResearch/hermes-agent/pull/20390))
|
||||
- **AGENTS.md — curator/cron/delegation/toolsets + fix plugin tree** ([#20226](https://github.com/NousResearch/hermes-agent/pull/20226))
|
||||
- **Bedrock quickstart entry + fallback comment + deployment link** (salvage #11093) ([#20397](https://github.com/NousResearch/hermes-agent/pull/20397))
|
||||
|
||||
### Docs polish
|
||||
- Collapse exploding skills tree to a single Skills node ([#18259](https://github.com/NousResearch/hermes-agent/pull/18259))
|
||||
- Clarify `session_search` auxiliary model docs ([#19593](https://github.com/NousResearch/hermes-agent/pull/19593))
|
||||
- Open WebUI Quick Setup gap fill ([#19654](https://github.com/NousResearch/hermes-agent/pull/19654))
|
||||
- Default custom tool creation to plugins (@kshitijk4poor) ([#19755](https://github.com/NousResearch/hermes-agent/pull/19755))
|
||||
- Clarify Telegram group chat troubleshooting (salvage #18672) ([#20416](https://github.com/NousResearch/hermes-agent/pull/20416))
|
||||
- Codex OAuth auth prerequisite clarification (salvage #18688) ([#20417](https://github.com/NousResearch/hermes-agent/pull/20417))
|
||||
- Discord Server Members Intent + SSRC-mapping drift + /voice join slash Choice (salvage #11350) ([#20411](https://github.com/NousResearch/hermes-agent/pull/20411))
|
||||
- Document `ctx.dispatch_tool()` (salvage #10955) ([#20391](https://github.com/NousResearch/hermes-agent/pull/20391))
|
||||
- Document `hermes webhook subscribe --deliver-only` (salvage #12612) ([#20392](https://github.com/NousResearch/hermes-agent/pull/20392))
|
||||
- Document `hermes import` reference (salvage #14711) ([#20396](https://github.com/NousResearch/hermes-agent/pull/20396))
|
||||
- Document per-provider TTS `max_text_length` caps (salvage #13825) ([#20389](https://github.com/NousResearch/hermes-agent/pull/20389))
|
||||
- Clarify supported prompt customization surfaces (salvage #19987) ([#20383](https://github.com/NousResearch/hermes-agent/pull/20383))
|
||||
- Correct `web_extract` summarizer timeout comment (salvage #20051) ([#20381](https://github.com/NousResearch/hermes-agent/pull/20381))
|
||||
- Fix fallback provider config paths (salvage #20033) ([#20382](https://github.com/NousResearch/hermes-agent/pull/20382))
|
||||
- Fix misleading RL install-extras claim (salvage #19080) ([#21213](https://github.com/NousResearch/hermes-agent/pull/21213))
|
||||
- Clarify API server tool execution locality (salvage #19117) ([#21223](https://github.com/NousResearch/hermes-agent/pull/21223))
|
||||
- Prefer `.venv` to match AGENTS.md and scripts/run_tests.sh (@xxxigm) ([#21334](https://github.com/NousResearch/hermes-agent/pull/21334))
|
||||
- Align tool discovery + test runner with AGENTS.md (@xxxigm) ([#20791](https://github.com/NousResearch/hermes-agent/pull/20791))
|
||||
- Align terminal-backend count and naming across docs and code (salvage #19044) ([#20402](https://github.com/NousResearch/hermes-agent/pull/20402))
|
||||
- Refresh stale platform counts (salvage #19053) ([#20403](https://github.com/NousResearch/hermes-agent/pull/20403))
|
||||
|
||||
---
|
||||
|
||||
## 👥 Contributors
|
||||
|
||||
### Core
|
||||
- **@teknium1** — salvage, triage, review, feature work, and release management
|
||||
|
||||
### Top Community Contributors
|
||||
|
||||
- **@kshitijk4poor** (21 PRs) — SearXNG native search backend, per-capability backend selection, collapsible TUI startup banner, Slack ephemeral ack + format fixes, Lightpanda fallback hardening, searxng-search optional skill + Web Search + Extract docs, default custom tool creation to plugins, kanban failure-column fix
|
||||
- **@alt-glitch** (13 PRs) — video_analyze tool, xAI Custom Voices (voice cloning), local-backend CLI launch-directory fix, lazy-session creation regression recovery, systemd unit refresh on gateway boot
|
||||
- **@OutThisLife** (9 PRs) — TUI perf — overlay render churn reduction, voice push-to-talk parity restoration (salvaging @Montbra)
|
||||
- **@helix4u** (6 PRs) — Classic CLI output recovery after resize, absolute-path TUI completion, gateway model picker current-context fix, Bedrock credential probe avoidance, kanban docs fixes
|
||||
- **@ethernet8023** (3 PRs) — Docker CI — don't cancel overlapping builds, :latest guard
|
||||
- **@benbarclay** (3 PRs) — Docker — launch dashboard as side-process via HERMES_DASHBOARD=1
|
||||
- **@austinpickett** (3 PRs) — Dashboard Plugins page, TUI /model picker overhaul with inline auth, kanban button fix
|
||||
- **@sprmn24** (2 PRs) — Contributor (2 PRs)
|
||||
- **@asheriif** (2 PRs) — Contributor (2 PRs)
|
||||
- **@xxxigm** (2 PRs) — Contributing docs — .venv preference and test runner alignment with AGENTS.md
|
||||
- **@stephenschoettler** (1 PR) — ACP — MCP E2E mock kwargs
|
||||
- **@vincez-hms-coder** (1 PR) — Dashboard — Profiles management page
|
||||
- **@cdanis** (1 PR) — Contributor
|
||||
- **@briandevans** (1 PR) — Toolsets test — kanban assertions post-#17805
|
||||
- **@heyitsaamir** (1 PR) — Contributor
|
||||
|
||||
### All Contributors
|
||||
|
||||
Thanks to everyone who contributed to v0.13.0 — commits, co-authored work, and salvaged PRs. 295 contributors in one week.
|
||||
|
||||
@0oAstro, @0xDevNinja, @0xharryriddle, @0xKingBack, @0xsir0000, @0xyg3n, @0z1-ghb, @abhinav11082001-stack,
|
||||
@acc001k, @acesjohnny, @adamludwin, @adybag14-cyber, @agentlinker, @agilejava, @ai-ag2026, @AJV20,
|
||||
@alanxchen85, @albert748, @AllardQuek, @alt-glitch, @altmazza0-star, @ambition0802, @amitgaur, @amroessam,
|
||||
@andrewhosf, @Asce66, @asheriif, @ashermorse, @asimons81, @Aslaaen, @Asunfly, @atongrun, @austinpickett,
|
||||
@banditburai, @barteqpl, @Bartok9, @Beandon13, @beardthelion, @beibi9966, @benbarclay, @binhnt92, @bjianhang,
|
||||
@BlackJulySnow, @bobashopcashier, @bogerman1, @Bongulielmi, @Brecht-H, @briandevans, @brooklynnicholson,
|
||||
@c3115644151, @camaragon, @CashWilliams, @CCClelo, @cdanis, @CES4751, @cg2aigc, @changchun989, @ChanlerDev,
|
||||
@CharlieKerfoot, @chengoak, @chenyunbo411, @chinadbo, @CIRWEL, @cixuuz, @cmcgrabby-hue, @colorcross,
|
||||
@Contentment003111, @CoreyNoDream, @counterposition, @curiouscleo, @DaniuXie, @deep-name, @dengtaoyuan450-a11y,
|
||||
@discodirector, @donramon77, @dpaluy, @ee-blog, @ehz0ah, @el-analista, @elmatadorgh, @EmelyanenkoK,
|
||||
@Emidomenge, @emozilla, @Es1la, @EthanGuo-coder, @etherman-os, @ethernet8023, @EvilDrag0n, @exxmen, @Fearvox,
|
||||
@Feranmi10, @firefly, @flobo3, @fmercurio, @Foolafroos, @formulahendry, @franksong2702, @ggnnggez, @GinWU05,
|
||||
@giwaov, @glesperance, @gnanirahulnutakki, @GodsBoy, @Gosuj, @Grey0202, @guillaumemeyer, @Gutslabs, @h0tp-ftw,
|
||||
@haidao1919, @halmisen, @happy5318, @hedirman, @helix4u, @hendrixfreire, @HenkDz, @hex-clawd, @heyitsaamir,
|
||||
@hharry11, @Hinotoi-agent, @holynn-q, @hrkzogw, @Hypn0sis, @Hypnus-Yuan, @ideathinklab01-source, @IMHaoyan,
|
||||
@Interstellar-code, @ishardo, @jacdevos, @jackey8616, @JanCong, @jasonoutland, @jatingodnani, @JayGwod,
|
||||
@jethac, @JezzaHehn, @JiaDe-Wu, @jjjojoj, @jkausel-ai, @John-tip, @johnncenae, @jrusso1020, @jslizar,
|
||||
@JTroyerOvermatch, @julysir, @Junass1, @JustinUssuri, @Kailigithub, @keepcalmqqf, @kiala9, @konsisumer,
|
||||
@kowenhaoai, @Krionex, @kshitijk4poor, @kyan12, @leavrcn, @leon7609, @LeonSGP43, @leprincep35700, @lhysdl,
|
||||
@likejudy, @lisanhu, @liu-collab, @liuguangyong93, @liuhao1024, @LucianoSP, @luoyuctl, @luyao618, @M3RCUR2Y,
|
||||
@maciekczech, @Magicray1217, @magicray1217, @MaHaoHao-ch, @malaiwah, @manateelazycat, @masonjames, @megastary,
|
||||
@memosr, @MichaelWDanko, @mikeyobrien, @millerc79, @Mind-Dragon, @mioimotoai-lgtm, @misery-hl, @molvikar,
|
||||
@momowind, @Montbra, @MottledShadow, @mrbob-git, @mrcharlesiv, @mrcoferland, @ms-alan, @mwnickerson,
|
||||
@nazirulhafiy, @nftpoetrist, @nicoloboschi, @nightq, @nikolay-bratanov, @NikolayGusev-astra, @nocturnum91,
|
||||
@noOne-list, @nouseman666, @novax635, @npmisantosh, @nudiltoys-cmyk, @olisikh, @oluwadareab12, @Oxidane-bot,
|
||||
@pama0227, @pander, @pasevin, @paul-tian, @pdonizete, @perlowja, @pingchesu, @PratikRai0101, @priveperfumes,
|
||||
@probepark, @QifengKuang, @quocanh261997, @qWaitCrypto, @qxxaa, @r266-tech, @rames-jusso, @revaraver,
|
||||
@Ricardo-M-L, @rob-maron, @Roy-oss1, @rxdxxxx, @SandroHub013, @Sanjays2402, @Sertug17, @shashwatgokhe,
|
||||
@shellybotmoyer, @SHL0MS, @SimbaKingjoe, @simbam99, @simplenamebox-ops, @socrates1024, @sonic-netizen,
|
||||
@sprmn24, @steezkelly, @stephen0110, @stephenschoettler, @stevenchanin, @stevenchouai, @stormhierta,
|
||||
@subtract0, @suncokret12, @swithek, @taeng0204, @TakeshiSawaguchi, @tangyuanjc, @TheEpTic, @thelumiereguy,
|
||||
@Tkander1715, @tmdgusya, @Tranquil-Flow, @TruaShamu, @UgwujaGeorge, @valda, @vincez-hms-coder, @VinVC,
|
||||
@vominh1919, @wabrent, @WadydX, @wanazhar, @WanderWang, @warabe1122, @web-dev0521, @WideLee, @willy-scr,
|
||||
@wmagev, @WuTianyi123, @wxst, @wysie, @Wysie, @xsfX20, @xxxigm, @xyiy001, @YanzhongSu, @ygd58, @Yoimex,
|
||||
@yuehei, @Yukipukii1, @yuqianma, @YX234, @zeejaytan, @zhanggttry, @zhao0112, @zng8418, @zons-zhaozhy, @Zyproth
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: [v2026.4.30...v2026.5.7](https://github.com/NousResearch/hermes-agent/compare/v2026.4.30...v2026.5.7)
|
||||
@@ -13,6 +13,10 @@ Usage::
|
||||
hermes-acp
|
||||
"""
|
||||
|
||||
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
|
||||
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
|
||||
import hermes_bootstrap # noqa: F401
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import sys
|
||||
|
||||
+288
-2
@@ -3,13 +3,16 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import contextvars
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from collections import defaultdict, deque
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
from typing import Any, Deque, Optional
|
||||
from urllib.parse import unquote, urlparse
|
||||
|
||||
import acp
|
||||
from acp.schema import (
|
||||
@@ -18,6 +21,7 @@ from acp.schema import (
|
||||
AuthenticateResponse,
|
||||
AvailableCommand,
|
||||
AvailableCommandsUpdate,
|
||||
BlobResourceContents,
|
||||
ClientCapabilities,
|
||||
EmbeddedResourceContentBlock,
|
||||
ForkSessionResponse,
|
||||
@@ -46,6 +50,7 @@ from acp.schema import (
|
||||
SessionResumeCapabilities,
|
||||
SessionInfo,
|
||||
TextContentBlock,
|
||||
TextResourceContents,
|
||||
UnstructuredCommandInput,
|
||||
Usage,
|
||||
UsageUpdate,
|
||||
@@ -83,6 +88,272 @@ _executor = ThreadPoolExecutor(max_workers=4, thread_name_prefix="acp-agent")
|
||||
# does not expose a client-side limit, so this is a fixed cap that clients
|
||||
# paginate against using `cursor` / `next_cursor`.
|
||||
_LIST_SESSIONS_PAGE_SIZE = 50
|
||||
_MAX_ACP_RESOURCE_BYTES = 512 * 1024
|
||||
_TEXT_RESOURCE_MIME_PREFIXES = ("text/",)
|
||||
_TEXT_RESOURCE_MIME_TYPES = {
|
||||
"application/json",
|
||||
"application/javascript",
|
||||
"application/typescript",
|
||||
"application/xml",
|
||||
"application/x-yaml",
|
||||
"application/yaml",
|
||||
"application/toml",
|
||||
"application/sql",
|
||||
}
|
||||
|
||||
|
||||
def _resource_display_name(uri: str, name: str | None = None, title: str | None = None) -> str:
|
||||
"""Human-readable attachment name for prompt context."""
|
||||
raw_name = (name or "").strip()
|
||||
raw_title = (title or "").strip()
|
||||
if raw_title and raw_name and raw_title != raw_name:
|
||||
return f"{raw_title} ({raw_name})"
|
||||
if raw_title:
|
||||
return raw_title
|
||||
if raw_name:
|
||||
return raw_name
|
||||
parsed = urlparse(uri)
|
||||
candidate = parsed.path if parsed.scheme else uri
|
||||
return Path(unquote(candidate)).name or uri or "resource"
|
||||
|
||||
|
||||
def _is_text_resource(mime_type: str | None) -> bool:
|
||||
mime = (mime_type or "").split(";", 1)[0].strip().lower()
|
||||
if not mime:
|
||||
return False
|
||||
return mime.startswith(_TEXT_RESOURCE_MIME_PREFIXES) or mime in _TEXT_RESOURCE_MIME_TYPES
|
||||
|
||||
|
||||
def _is_image_resource(mime_type: str | None) -> bool:
|
||||
mime = (mime_type or "").split(";", 1)[0].strip().lower()
|
||||
return mime.startswith("image/")
|
||||
|
||||
|
||||
def _guess_image_mime_from_path(path: Path) -> str | None:
|
||||
suffix = path.suffix.lower()
|
||||
return {
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".gif": "image/gif",
|
||||
".webp": "image/webp",
|
||||
".bmp": "image/bmp",
|
||||
".svg": "image/svg+xml",
|
||||
}.get(suffix)
|
||||
|
||||
|
||||
def _image_data_url(data: bytes, mime_type: str) -> str:
|
||||
return f"data:{mime_type};base64,{base64.b64encode(data).decode('ascii')}"
|
||||
|
||||
|
||||
def _path_from_file_uri(uri: str) -> Path | None:
|
||||
"""Convert local file URIs/paths from ACP clients into a readable Path.
|
||||
|
||||
Zed may send POSIX file URIs from Linux/WSL workspaces or Windows-ish paths
|
||||
when launched through wsl.exe. Translate the common Windows drive form to
|
||||
/mnt/<drive>/... so Hermes running in WSL can read it.
|
||||
"""
|
||||
raw = (uri or "").strip()
|
||||
if not raw:
|
||||
return None
|
||||
|
||||
parsed = urlparse(raw)
|
||||
if parsed.scheme and parsed.scheme != "file":
|
||||
return None
|
||||
|
||||
if parsed.scheme == "file":
|
||||
if parsed.netloc and parsed.netloc not in {"", "localhost"}:
|
||||
return None
|
||||
path_text = unquote(parsed.path or "")
|
||||
else:
|
||||
path_text = unquote(raw)
|
||||
|
||||
# file:///C:/Users/... or C:\Users\...
|
||||
if len(path_text) >= 3 and path_text[0] == "/" and path_text[2] == ":" and path_text[1].isalpha():
|
||||
drive = path_text[1].lower()
|
||||
rest = path_text[3:].lstrip("/\\").replace("\\", "/")
|
||||
return Path("/mnt") / drive / rest
|
||||
if len(path_text) >= 2 and path_text[1] == ":" and path_text[0].isalpha():
|
||||
drive = path_text[0].lower()
|
||||
rest = path_text[2:].lstrip("/\\").replace("\\", "/")
|
||||
return Path("/mnt") / drive / rest
|
||||
|
||||
return Path(path_text)
|
||||
|
||||
|
||||
def _decode_text_bytes(data: bytes, mime_type: str | None) -> str | None:
|
||||
"""Decode resource bytes if they are probably text; return None for binary."""
|
||||
if b"\x00" in data and not _is_text_resource(mime_type):
|
||||
return None
|
||||
for encoding in ("utf-8-sig", "utf-8", "latin-1"):
|
||||
try:
|
||||
return data.decode(encoding)
|
||||
except UnicodeDecodeError:
|
||||
continue
|
||||
return data.decode("utf-8", errors="replace")
|
||||
|
||||
|
||||
def _format_resource_text(
|
||||
*,
|
||||
uri: str,
|
||||
body: str,
|
||||
name: str | None = None,
|
||||
title: str | None = None,
|
||||
note: str | None = None,
|
||||
) -> str:
|
||||
display = _resource_display_name(uri, name=name, title=title)
|
||||
header = f"[Attached file: {display}]"
|
||||
if note:
|
||||
header += f" ({note})"
|
||||
return f"{header}\nURI: {uri}\n\n{body}"
|
||||
|
||||
|
||||
def _resource_link_to_parts(block: ResourceContentBlock) -> list[dict[str, Any]]:
|
||||
"""Convert an ACP resource_link block to OpenAI content parts.
|
||||
|
||||
Returns a list of {"type": "text", ...} and/or {"type": "image_url", ...}
|
||||
parts. Image resources produce an image_url part with a small text header
|
||||
so the model knows which attachment it is. Non-image resources return a
|
||||
single text part with the inlined file body (or a binary-omit note).
|
||||
"""
|
||||
uri = str(getattr(block, "uri", "") or "").strip()
|
||||
if not uri:
|
||||
return []
|
||||
|
||||
name = str(getattr(block, "name", "") or "").strip() or None
|
||||
title = str(getattr(block, "title", "") or "").strip() or None
|
||||
mime_type = str(getattr(block, "mime_type", "") or "").strip() or None
|
||||
path = _path_from_file_uri(uri)
|
||||
|
||||
if path is None:
|
||||
return [{
|
||||
"type": "text",
|
||||
"text": _format_resource_text(
|
||||
uri=uri,
|
||||
name=name,
|
||||
title=title,
|
||||
body="[Resource link only; Hermes cannot read non-file ACP resource URIs directly.]",
|
||||
),
|
||||
}]
|
||||
|
||||
# Image files: emit a short text header + image_url data URL so vision
|
||||
# models can see the attachment instead of a "binary omitted" note.
|
||||
image_mime = mime_type if _is_image_resource(mime_type) else _guess_image_mime_from_path(path)
|
||||
if image_mime and _is_image_resource(image_mime):
|
||||
try:
|
||||
size = path.stat().st_size
|
||||
if size > _MAX_ACP_RESOURCE_BYTES:
|
||||
return [{
|
||||
"type": "text",
|
||||
"text": _format_resource_text(
|
||||
uri=uri,
|
||||
name=name,
|
||||
title=title,
|
||||
body=f"[Image too large to inline: {size} bytes, cap={_MAX_ACP_RESOURCE_BYTES}]",
|
||||
),
|
||||
}]
|
||||
with path.open("rb") as fh:
|
||||
data = fh.read()
|
||||
except OSError as exc:
|
||||
logger.warning("ACP image resource read failed: %s", uri, exc_info=True)
|
||||
return [{
|
||||
"type": "text",
|
||||
"text": _format_resource_text(
|
||||
uri=uri,
|
||||
name=name,
|
||||
title=title,
|
||||
body=f"[Could not read attached image: {exc}]",
|
||||
),
|
||||
}]
|
||||
display = _resource_display_name(uri, name=name, title=title)
|
||||
return [
|
||||
{"type": "text", "text": f"[Attached image: {display}]\nURI: {uri}"},
|
||||
{"type": "image_url", "image_url": {"url": _image_data_url(data, image_mime)}},
|
||||
]
|
||||
|
||||
try:
|
||||
size = path.stat().st_size
|
||||
read_size = min(size, _MAX_ACP_RESOURCE_BYTES)
|
||||
with path.open("rb") as fh:
|
||||
data = fh.read(read_size)
|
||||
text = _decode_text_bytes(data, mime_type)
|
||||
if text is None:
|
||||
return [{
|
||||
"type": "text",
|
||||
"text": _format_resource_text(
|
||||
uri=uri,
|
||||
name=name,
|
||||
title=title,
|
||||
body=f"[Binary file omitted: {size} bytes, mime={mime_type or 'unknown'}]",
|
||||
),
|
||||
}]
|
||||
note = None
|
||||
if size > _MAX_ACP_RESOURCE_BYTES:
|
||||
note = f"truncated to {_MAX_ACP_RESOURCE_BYTES} of {size} bytes"
|
||||
return [{
|
||||
"type": "text",
|
||||
"text": _format_resource_text(uri=uri, name=name, title=title, body=text, note=note),
|
||||
}]
|
||||
except OSError as exc:
|
||||
logger.warning("ACP resource read failed: %s", uri, exc_info=True)
|
||||
return [{
|
||||
"type": "text",
|
||||
"text": _format_resource_text(
|
||||
uri=uri,
|
||||
name=name,
|
||||
title=title,
|
||||
body=f"[Could not read attached file: {exc}]",
|
||||
),
|
||||
}]
|
||||
|
||||
|
||||
def _embedded_resource_to_parts(block: EmbeddedResourceContentBlock) -> list[dict[str, Any]]:
|
||||
resource = getattr(block, "resource", None)
|
||||
if resource is None:
|
||||
return []
|
||||
|
||||
uri = str(getattr(resource, "uri", "") or "").strip()
|
||||
mime_type = str(getattr(resource, "mime_type", "") or "").strip() or None
|
||||
|
||||
if isinstance(resource, TextResourceContents):
|
||||
return [{"type": "text", "text": _format_resource_text(uri=uri, body=resource.text)}]
|
||||
|
||||
if isinstance(resource, BlobResourceContents):
|
||||
blob = resource.blob or ""
|
||||
try:
|
||||
data = base64.b64decode(blob, validate=True)
|
||||
except Exception:
|
||||
data = blob.encode("utf-8", errors="replace")
|
||||
|
||||
# Image blobs go through as image_url so vision models can see them.
|
||||
if _is_image_resource(mime_type):
|
||||
if len(data) > _MAX_ACP_RESOURCE_BYTES:
|
||||
return [{
|
||||
"type": "text",
|
||||
"text": _format_resource_text(
|
||||
uri=uri,
|
||||
body=f"[Embedded image too large to inline: {len(data)} bytes, cap={_MAX_ACP_RESOURCE_BYTES}]",
|
||||
),
|
||||
}]
|
||||
display = _resource_display_name(uri)
|
||||
return [
|
||||
{"type": "text", "text": f"[Attached image: {display}]" + (f"\nURI: {uri}" if uri else "")},
|
||||
{"type": "image_url", "image_url": {"url": _image_data_url(data, mime_type or "image/png")}},
|
||||
]
|
||||
|
||||
text = _decode_text_bytes(data[:_MAX_ACP_RESOURCE_BYTES], mime_type)
|
||||
if text is None:
|
||||
body = f"[Binary embedded file omitted: {len(data)} bytes, mime={mime_type or 'unknown'}]"
|
||||
else:
|
||||
body = text
|
||||
if len(data) > _MAX_ACP_RESOURCE_BYTES:
|
||||
body += f"\n\n[Truncated to {_MAX_ACP_RESOURCE_BYTES} of {len(data)} bytes]"
|
||||
return [{"type": "text", "text": _format_resource_text(uri=uri, body=body)}]
|
||||
|
||||
text = getattr(resource, "text", None)
|
||||
if text:
|
||||
return [{"type": "text", "text": _format_resource_text(uri=uri, body=str(text))}]
|
||||
return []
|
||||
|
||||
|
||||
def _extract_text(
|
||||
@@ -144,6 +415,20 @@ def _content_blocks_to_openai_user_content(
|
||||
if image_part is not None:
|
||||
parts.append(image_part)
|
||||
continue
|
||||
if isinstance(block, ResourceContentBlock):
|
||||
resource_parts = _resource_link_to_parts(block)
|
||||
for part in resource_parts:
|
||||
parts.append(part)
|
||||
if part.get("type") == "text":
|
||||
text_parts.append(part["text"])
|
||||
continue
|
||||
if isinstance(block, EmbeddedResourceContentBlock):
|
||||
resource_parts = _embedded_resource_to_parts(block)
|
||||
for part in resource_parts:
|
||||
parts.append(part)
|
||||
if part.get("type") == "text":
|
||||
text_parts.append(part["text"])
|
||||
continue
|
||||
|
||||
if not parts:
|
||||
return _extract_text(prompt)
|
||||
@@ -803,6 +1088,7 @@ class HermesACPAgent(acp.Agent):
|
||||
|
||||
user_text = _extract_text(prompt).strip()
|
||||
user_content = _content_blocks_to_openai_user_content(prompt)
|
||||
text_only_prompt = all(isinstance(block, TextContentBlock) for block in prompt)
|
||||
has_content = bool(user_text) or (
|
||||
isinstance(user_content, list) and bool(user_content)
|
||||
)
|
||||
@@ -821,7 +1107,7 @@ class HermesACPAgent(acp.Agent):
|
||||
# silently append to state.queued_prompts and respond with
|
||||
# "No active turn — queued for the next turn", which looks like
|
||||
# /queue even though the user never typed /queue.
|
||||
if isinstance(user_content, str) and user_text.startswith("/steer"):
|
||||
if text_only_prompt and isinstance(user_content, str) and user_text.startswith("/steer"):
|
||||
steer_text = user_text.split(maxsplit=1)[1].strip() if len(user_text.split(maxsplit=1)) > 1 else ""
|
||||
interrupted_prompt = ""
|
||||
rewrite_idle = False
|
||||
@@ -846,7 +1132,7 @@ class HermesACPAgent(acp.Agent):
|
||||
# Slash commands are text-only; if the client included images/resources,
|
||||
# send the whole multimodal prompt to the agent instead of treating it as
|
||||
# an ACP command.
|
||||
if isinstance(user_content, str) and user_text.startswith("/"):
|
||||
if text_only_prompt and isinstance(user_content, str) and user_text.startswith("/"):
|
||||
response_text = self._handle_slash_command(user_text, state)
|
||||
if response_text is not None:
|
||||
if self._conn:
|
||||
|
||||
+35
-32
@@ -231,33 +231,30 @@ def _supports_fast_mode(model: str) -> bool:
|
||||
return any(v in model for v in _FAST_MODE_SUPPORTED_SUBSTRINGS)
|
||||
|
||||
|
||||
# Beta headers for enhanced features (sent with ALL auth types).
|
||||
# As of Opus 4.7 (2026-04-16), the first two are GA on Claude 4.6+ — the
|
||||
# Beta headers for enhanced features that are safe on ordinary/native Anthropic
|
||||
# requests. As of Opus 4.7 (2026-04-16), these are GA on Claude 4.6+ — the
|
||||
# beta headers are still accepted (harmless no-op) but not required. Kept
|
||||
# here so older Claude (4.5, 4.1) + third-party Anthropic-compat endpoints
|
||||
# that still gate on the headers continue to get the enhanced features.
|
||||
# here so older Claude (4.5, 4.1) + compatible endpoints that still gate on
|
||||
# the headers continue to get the enhanced features.
|
||||
#
|
||||
# ``context-1m-2025-08-07`` unlocks the 1M context window on Claude Opus 4.6/4.7
|
||||
# and Sonnet 4.6 when served via AWS Bedrock or Azure AI Foundry. 1M is GA on
|
||||
# native Anthropic (api.anthropic.com) for Opus 4.6+, but Bedrock/Azure still
|
||||
# gate it behind this beta header as of 2026-04 — without it Bedrock caps Opus
|
||||
# at 200K even though model_metadata.py advertises 1M. The header is a harmless
|
||||
# no-op on endpoints where 1M is GA.
|
||||
# Do NOT include ``context-1m-2025-08-07`` here. Anthropic returns HTTP 400
|
||||
# ("long context beta is not yet available for this subscription") for
|
||||
# accounts without the long-context beta, which breaks normal short auxiliary
|
||||
# calls like title generation/session summarization.
|
||||
#
|
||||
# Migration guide: remove these if you no longer support ≤4.5 models or once
|
||||
# Bedrock/Azure promote 1M to GA.
|
||||
# ``context-1m-2025-08-07`` is still required to unlock the 1M context window
|
||||
# on Claude Opus 4.6/4.7 and Sonnet 4.6 when served via AWS Bedrock or Azure
|
||||
# AI Foundry. Add it only for those endpoint-specific paths below.
|
||||
_COMMON_BETAS = [
|
||||
"interleaved-thinking-2025-05-14",
|
||||
"fine-grained-tool-streaming-2025-05-14",
|
||||
"context-1m-2025-08-07",
|
||||
]
|
||||
# MiniMax's Anthropic-compatible endpoints fail tool-use requests when
|
||||
# the fine-grained tool streaming beta is present. Omit it so tool calls
|
||||
# fall back to the provider's default response path.
|
||||
_TOOL_STREAMING_BETA = "fine-grained-tool-streaming-2025-05-14"
|
||||
# 1M context beta — see comment on _COMMON_BETAS above. Stripped for
|
||||
# Bearer-auth (MiniMax) endpoints since they host their own models and
|
||||
# unknown Anthropic beta headers risk request rejection.
|
||||
# 1M context beta. Native Anthropic does not get this by default because some
|
||||
# subscriptions reject it, but Bedrock/Azure still need it for 1M context.
|
||||
_CONTEXT_1M_BETA = "context-1m-2025-08-07"
|
||||
|
||||
# Fast mode beta — enables the ``speed: "fast"`` request parameter for
|
||||
@@ -476,6 +473,14 @@ def _requires_bearer_auth(base_url: str | None) -> bool:
|
||||
return normalized.startswith(("https://api.minimax.io/anthropic", "https://api.minimaxi.com/anthropic"))
|
||||
|
||||
|
||||
def _base_url_needs_context_1m_beta(base_url: str | None) -> bool:
|
||||
"""Return True for endpoints that still gate 1M context behind a beta."""
|
||||
normalized = _normalize_base_url_text(base_url).lower()
|
||||
if not normalized:
|
||||
return False
|
||||
return "azure.com" in normalized
|
||||
|
||||
|
||||
def _common_betas_for_base_url(
|
||||
base_url: str | None,
|
||||
*,
|
||||
@@ -485,27 +490,25 @@ def _common_betas_for_base_url(
|
||||
|
||||
MiniMax's Anthropic-compatible endpoints (Bearer-auth) reject requests
|
||||
that include Anthropic's ``fine-grained-tool-streaming`` beta — every
|
||||
tool-use message triggers a connection error. Strip that beta for
|
||||
Bearer-auth endpoints while keeping all other betas intact.
|
||||
tool-use message triggers a connection error.
|
||||
|
||||
The ``context-1m-2025-08-07`` beta is also stripped for Bearer-auth
|
||||
endpoints — MiniMax hosts its own models, not Claude, so the header is
|
||||
irrelevant at best and risks request rejection at worst.
|
||||
The ``context-1m-2025-08-07`` beta is not sent to native Anthropic by
|
||||
default because some subscriptions reject it. Add it only for endpoint
|
||||
families that still require it for 1M context, currently Azure AI Foundry.
|
||||
Bedrock uses its own client helper below and opts in explicitly.
|
||||
|
||||
``drop_context_1m_beta=True`` additionally strips the 1M-context beta on
|
||||
otherwise-unrelated endpoints. The OAuth retry path flips this flag after
|
||||
a subscription rejects the beta with
|
||||
"The long context beta is not yet available for this subscription" so
|
||||
subsequent requests in the same session don't repeat the probe. See the
|
||||
reactive recovery loop in ``run_agent.py`` and issue-comment history on
|
||||
PR #17680 for the full rationale.
|
||||
``drop_context_1m_beta=True`` strips the 1M-context beta from any path that
|
||||
would otherwise include it after a subscription/endpoint rejects the beta.
|
||||
"""
|
||||
betas = list(_COMMON_BETAS)
|
||||
if _base_url_needs_context_1m_beta(base_url) and not drop_context_1m_beta:
|
||||
betas.append(_CONTEXT_1M_BETA)
|
||||
if _requires_bearer_auth(base_url):
|
||||
_stripped = {_TOOL_STREAMING_BETA, _CONTEXT_1M_BETA}
|
||||
return [b for b in _COMMON_BETAS if b not in _stripped]
|
||||
return [b for b in betas if b not in _stripped]
|
||||
if drop_context_1m_beta:
|
||||
return [b for b in _COMMON_BETAS if b != _CONTEXT_1M_BETA]
|
||||
return _COMMON_BETAS
|
||||
return [b for b in betas if b != _CONTEXT_1M_BETA]
|
||||
return betas
|
||||
|
||||
|
||||
def build_anthropic_client(
|
||||
@@ -642,7 +645,7 @@ def build_anthropic_bedrock_client(region: str):
|
||||
return _anthropic_sdk.AnthropicBedrock(
|
||||
aws_region=region,
|
||||
timeout=Timeout(timeout=900.0, connect=10.0),
|
||||
default_headers={"anthropic-beta": ",".join(_COMMON_BETAS)},
|
||||
default_headers={"anthropic-beta": ",".join([*_COMMON_BETAS, _CONTEXT_1M_BETA])},
|
||||
)
|
||||
|
||||
|
||||
|
||||
+127
-6
@@ -455,6 +455,12 @@ def _to_openai_base_url(base_url: str) -> str:
|
||||
"""
|
||||
url = str(base_url or "").strip().rstrip("/")
|
||||
if url.endswith("/anthropic"):
|
||||
# ZAI (open.bigmodel.cn) uses /api/anthropic for Anthropic wire
|
||||
# but /api/paas/v4 for OpenAI wire — the generic /v1 rewrite is wrong.
|
||||
if "open.bigmodel.cn" in url or "bigmodel" in url:
|
||||
rewritten = url[: -len("/anthropic")] + "/paas/v4"
|
||||
logger.debug("Auxiliary client: rewrote ZAI base URL %s → %s", url, rewritten)
|
||||
return rewritten
|
||||
rewritten = url[: -len("/anthropic")] + "/v1"
|
||||
logger.debug("Auxiliary client: rewrote base URL %s → %s", url, rewritten)
|
||||
return rewritten
|
||||
@@ -596,6 +602,14 @@ class _CodexCompletionsAdapter:
|
||||
"store": False,
|
||||
}
|
||||
|
||||
# Preserve the chat.completions timeout contract. This adapter is used
|
||||
# by auxiliary calls such as context compression; if the timeout is not
|
||||
# forwarded and enforced, a Codex Responses stream can sit behind a
|
||||
# dead-looking CLI until the user force-interrupts the whole session.
|
||||
timeout = kwargs.get("timeout")
|
||||
if timeout is not None:
|
||||
resp_kwargs["timeout"] = timeout
|
||||
|
||||
# Note: the Codex endpoint (chatgpt.com/backend-api/codex) does NOT
|
||||
# support max_output_tokens or temperature — omit to avoid 400 errors.
|
||||
|
||||
@@ -653,6 +667,37 @@ class _CodexCompletionsAdapter:
|
||||
text_parts: List[str] = []
|
||||
tool_calls_raw: List[Any] = []
|
||||
usage = None
|
||||
total_timeout = timeout if isinstance(timeout, (int, float)) and timeout > 0 else None
|
||||
deadline = time.monotonic() + float(total_timeout) if total_timeout else None
|
||||
timed_out = threading.Event()
|
||||
timeout_timer: Optional[threading.Timer] = None
|
||||
|
||||
def _timeout_message() -> str:
|
||||
return f"Codex auxiliary Responses stream exceeded {float(total_timeout):.1f}s total timeout"
|
||||
|
||||
def _close_client_on_timeout() -> None:
|
||||
timed_out.set()
|
||||
close = getattr(self._client, "close", None)
|
||||
if callable(close):
|
||||
try:
|
||||
close()
|
||||
except Exception:
|
||||
logger.debug("Codex auxiliary: client close during timeout failed", exc_info=True)
|
||||
|
||||
def _check_cancelled() -> None:
|
||||
if deadline is not None and time.monotonic() >= deadline:
|
||||
timed_out.set()
|
||||
raise TimeoutError(_timeout_message())
|
||||
try:
|
||||
from tools.interrupt import is_interrupted
|
||||
if is_interrupted():
|
||||
raise InterruptedError("Codex auxiliary Responses stream interrupted")
|
||||
except InterruptedError:
|
||||
raise
|
||||
except Exception:
|
||||
# Interrupt state is a best-effort UX hook; never make it a
|
||||
# new failure mode for auxiliary calls.
|
||||
pass
|
||||
|
||||
try:
|
||||
# Collect output items and text deltas during streaming —
|
||||
@@ -661,8 +706,14 @@ class _CodexCompletionsAdapter:
|
||||
collected_output_items: List[Any] = []
|
||||
collected_text_deltas: List[str] = []
|
||||
has_function_calls = False
|
||||
if total_timeout:
|
||||
timeout_timer = threading.Timer(float(total_timeout), _close_client_on_timeout)
|
||||
timeout_timer.daemon = True
|
||||
timeout_timer.start()
|
||||
_check_cancelled()
|
||||
with self._client.responses.stream(**resp_kwargs) as stream:
|
||||
for _event in stream:
|
||||
_check_cancelled()
|
||||
_etype = getattr(_event, "type", "")
|
||||
if _etype == "response.output_item.done":
|
||||
_done = getattr(_event, "item", None)
|
||||
@@ -674,6 +725,7 @@ class _CodexCompletionsAdapter:
|
||||
collected_text_deltas.append(_delta)
|
||||
elif "function_call" in _etype:
|
||||
has_function_calls = True
|
||||
_check_cancelled()
|
||||
final = stream.get_final_response()
|
||||
|
||||
# Backfill empty output from collected stream events
|
||||
@@ -733,8 +785,13 @@ class _CodexCompletionsAdapter:
|
||||
total_tokens=getattr(resp_usage, "total_tokens", 0),
|
||||
)
|
||||
except Exception as exc:
|
||||
if timed_out.is_set():
|
||||
raise TimeoutError(_timeout_message()) from exc
|
||||
logger.debug("Codex auxiliary Responses API call failed: %s", exc)
|
||||
raise
|
||||
finally:
|
||||
if timeout_timer is not None:
|
||||
timeout_timer.cancel()
|
||||
|
||||
content = "".join(text_parts).strip() or None
|
||||
|
||||
@@ -828,7 +885,14 @@ class _AnthropicCompletionsAdapter:
|
||||
model = kwargs.get("model", self._model)
|
||||
tools = kwargs.get("tools")
|
||||
tool_choice = kwargs.get("tool_choice")
|
||||
max_tokens = kwargs.get("max_tokens") or kwargs.get("max_completion_tokens") or 2000
|
||||
# ZAI's Anthropic-compatible endpoint rejects max_tokens on vision
|
||||
# models (glm-4v-flash etc.) with error code 1210. When the caller
|
||||
# signals this by setting _skip_zai_max_tokens in kwargs, omit it.
|
||||
_skip_mt = kwargs.pop("_skip_zai_max_tokens", False)
|
||||
if _skip_mt:
|
||||
max_tokens = None
|
||||
else:
|
||||
max_tokens = kwargs.get("max_tokens") or kwargs.get("max_completion_tokens") or 2000
|
||||
temperature = kwargs.get("temperature")
|
||||
|
||||
normalized_tool_choice = None
|
||||
@@ -2835,6 +2899,33 @@ def resolve_vision_provider_client(
|
||||
)
|
||||
return _finalize(requested, sync_client, default_model)
|
||||
|
||||
# ZAI vision models must use the OpenAI-compatible endpoint, not the
|
||||
# Anthropic-compatible one (which may be the main-runtime default).
|
||||
# The Anthropic wire rejects max_tokens on multimodal calls (error 1210),
|
||||
# while the OpenAI wire handles it correctly.
|
||||
if requested == "zai" and not resolved_base_url:
|
||||
zai_openai_urls = [
|
||||
"https://open.bigmodel.cn/api/paas/v4",
|
||||
"https://api.z.ai/api/paas/v4",
|
||||
]
|
||||
for _zai_url in zai_openai_urls:
|
||||
client, final_model = _get_cached_client(
|
||||
requested, resolved_model, async_mode,
|
||||
base_url=_zai_url,
|
||||
api_key=resolved_api_key or None,
|
||||
api_mode="chat_completions",
|
||||
is_vision=True,
|
||||
)
|
||||
if client is not None:
|
||||
return _finalize(requested, client, final_model)
|
||||
# Fallback: try without explicit base_url (old behavior)
|
||||
client, final_model = _get_cached_client(requested, resolved_model, async_mode,
|
||||
api_mode=resolved_api_mode,
|
||||
is_vision=True)
|
||||
if client is None:
|
||||
return requested, None, None
|
||||
return requested, client, final_model
|
||||
|
||||
client, final_model = _get_cached_client(requested, resolved_model, async_mode,
|
||||
api_mode=resolved_api_mode,
|
||||
is_vision=True)
|
||||
@@ -2862,10 +2953,11 @@ def auxiliary_max_tokens_param(value: int) -> dict:
|
||||
"""
|
||||
custom_base = _current_custom_base_url()
|
||||
or_key = os.getenv("OPENROUTER_API_KEY")
|
||||
# Only use max_completion_tokens for direct OpenAI custom endpoints
|
||||
# Use max_completion_tokens for direct OpenAI-compatible providers that reject
|
||||
# max_tokens on newer GPT-4o/o-series/GPT-5-style models.
|
||||
if (not or_key
|
||||
and _read_nous_auth() is None
|
||||
and base_url_hostname(custom_base) == "api.openai.com"):
|
||||
and base_url_hostname(custom_base) in {"api.openai.com", "api.githubcopilot.com"}):
|
||||
return {"max_completion_tokens": value}
|
||||
return {"max_tokens": value}
|
||||
|
||||
@@ -3393,7 +3485,16 @@ def _build_call_kwargs(
|
||||
if max_tokens is not None:
|
||||
# Codex adapter handles max_tokens internally; OpenRouter/Nous use max_tokens.
|
||||
# Direct OpenAI api.openai.com with newer models needs max_completion_tokens.
|
||||
if provider == "custom":
|
||||
# ZAI vision models (glm-4v-flash, glm-4v-plus, etc.) reject max_tokens with
|
||||
# error code 1210 ("API 调用参数有误") on multimodal requests — skip it.
|
||||
_model_lower = (model or "").lower()
|
||||
_skip_max_tokens = (
|
||||
provider == "zai"
|
||||
and ("4v" in _model_lower or "5v" in _model_lower or "-v" in _model_lower)
|
||||
)
|
||||
if _skip_max_tokens:
|
||||
pass # ZAI vision models do not accept max_tokens
|
||||
elif provider == "custom":
|
||||
custom_base = base_url or _current_custom_base_url()
|
||||
if base_url_hostname(custom_base) == "api.openai.com":
|
||||
kwargs["max_completion_tokens"] = max_tokens
|
||||
@@ -3624,13 +3725,23 @@ def call_llm(
|
||||
kwargs = retry_kwargs
|
||||
|
||||
err_str = str(first_err)
|
||||
# ZAI vision models (glm-4v-flash etc.) return error code 1210
|
||||
# ("API 调用参数有误") when max_tokens is passed on multimodal
|
||||
# calls. The error message does NOT contain "max_tokens" so the
|
||||
# generic retry below never fires. Detect the ZAI-specific error
|
||||
# and strip max_tokens before retrying.
|
||||
_is_zai_param_error = (
|
||||
"1210" in err_str
|
||||
and "bigmodel" in str(getattr(client, "base_url", ""))
|
||||
)
|
||||
if max_tokens is not None and (
|
||||
"max_tokens" in err_str
|
||||
or "unsupported_parameter" in err_str
|
||||
or _is_unsupported_parameter_error(first_err, "max_tokens")
|
||||
or _is_zai_param_error
|
||||
):
|
||||
kwargs.pop("max_tokens", None)
|
||||
kwargs["max_completion_tokens"] = max_tokens
|
||||
kwargs.pop("max_completion_tokens", None)
|
||||
try:
|
||||
return _validate_llm_response(
|
||||
client.chat.completions.create(**kwargs), task)
|
||||
@@ -3930,13 +4041,23 @@ async def async_call_llm(
|
||||
kwargs = retry_kwargs
|
||||
|
||||
err_str = str(first_err)
|
||||
# ZAI vision models (glm-4v-flash etc.) return error code 1210
|
||||
# ("API 调用参数有误") when max_tokens is passed on multimodal
|
||||
# calls. The error message does NOT contain "max_tokens" so the
|
||||
# generic retry below never fires. Detect the ZAI-specific error
|
||||
# and strip max_tokens before retrying.
|
||||
_is_zai_param_error = (
|
||||
"1210" in err_str
|
||||
and "bigmodel" in str(getattr(client, "base_url", ""))
|
||||
)
|
||||
if max_tokens is not None and (
|
||||
"max_tokens" in err_str
|
||||
or "unsupported_parameter" in err_str
|
||||
or _is_unsupported_parameter_error(first_err, "max_tokens")
|
||||
or _is_zai_param_error
|
||||
):
|
||||
kwargs.pop("max_tokens", None)
|
||||
kwargs["max_completion_tokens"] = max_tokens
|
||||
kwargs.pop("max_completion_tokens", None)
|
||||
try:
|
||||
return _validate_llm_response(
|
||||
await client.chat.completions.create(**kwargs), task)
|
||||
|
||||
@@ -631,11 +631,18 @@ def normalize_converse_response(response: Dict) -> SimpleNamespace:
|
||||
stop_reason = response.get("stopReason", "end_turn")
|
||||
|
||||
text_parts = []
|
||||
reasoning_parts = []
|
||||
tool_calls = []
|
||||
|
||||
for block in content_blocks:
|
||||
if "text" in block:
|
||||
text_parts.append(block["text"])
|
||||
elif "reasoningContent" in block:
|
||||
reasoning = block["reasoningContent"]
|
||||
if isinstance(reasoning, dict):
|
||||
thinking_text = reasoning.get("text", "")
|
||||
if thinking_text:
|
||||
reasoning_parts.append(str(thinking_text))
|
||||
elif "toolUse" in block:
|
||||
tu = block["toolUse"]
|
||||
tool_calls.append(SimpleNamespace(
|
||||
@@ -652,6 +659,7 @@ def normalize_converse_response(response: Dict) -> SimpleNamespace:
|
||||
role="assistant",
|
||||
content="\n".join(text_parts) if text_parts else None,
|
||||
tool_calls=tool_calls if tool_calls else None,
|
||||
reasoning_content="\n\n".join(reasoning_parts) if reasoning_parts else None,
|
||||
)
|
||||
|
||||
# Build usage stats
|
||||
@@ -732,6 +740,7 @@ def stream_converse_with_callbacks(
|
||||
``normalize_converse_response()``.
|
||||
"""
|
||||
text_parts: List[str] = []
|
||||
reasoning_parts: List[str] = []
|
||||
tool_calls: List[SimpleNamespace] = []
|
||||
current_tool: Optional[Dict] = None
|
||||
current_text_buffer: List[str] = []
|
||||
@@ -777,8 +786,10 @@ def stream_converse_with_callbacks(
|
||||
reasoning = delta["reasoningContent"]
|
||||
if isinstance(reasoning, dict):
|
||||
thinking_text = reasoning.get("text", "")
|
||||
if thinking_text and on_reasoning_delta:
|
||||
on_reasoning_delta(thinking_text)
|
||||
if thinking_text:
|
||||
reasoning_parts.append(str(thinking_text))
|
||||
if on_reasoning_delta:
|
||||
on_reasoning_delta(thinking_text)
|
||||
|
||||
elif "contentBlockStop" in event:
|
||||
if current_tool is not None:
|
||||
@@ -817,6 +828,7 @@ def stream_converse_with_callbacks(
|
||||
role="assistant",
|
||||
content="\n".join(text_parts) if text_parts else None,
|
||||
tool_calls=tool_calls if tool_calls else None,
|
||||
reasoning_content="\n\n".join(reasoning_parts) if reasoning_parts else None,
|
||||
)
|
||||
|
||||
usage = SimpleNamespace(
|
||||
|
||||
+10
-12
@@ -6,8 +6,7 @@ protecting head and tail context.
|
||||
|
||||
Improvements over v2:
|
||||
- Structured summary template with Resolved/Pending question tracking
|
||||
- Summarizer preamble: "Do not respond to any questions" (from OpenCode)
|
||||
- Handoff framing: "different assistant" (from Codex) to create separation
|
||||
- Filter-safe summarizer preamble that treats prior turns as source material
|
||||
- "Remaining Work" replaces "Next Steps" to avoid reading as active instructions
|
||||
- Clear separator when summary merges into tail message
|
||||
- Iterative summary updates (preserves info across multiple compactions)
|
||||
@@ -755,15 +754,14 @@ class ContextCompressor(ContextEngine):
|
||||
content_to_summarize = self._serialize_for_summary(turns_to_summarize)
|
||||
|
||||
# Preamble shared by both first-compaction and iterative-update prompts.
|
||||
# Inspired by OpenCode's "do not respond to any questions" instruction
|
||||
# and Codex's "another language model" framing.
|
||||
# Keep the wording deliberately plain: Azure/OpenAI-compatible content
|
||||
# filters have flagged stronger "injection" / "do not respond" framing.
|
||||
_summarizer_preamble = (
|
||||
"You are a summarization agent creating a context checkpoint. "
|
||||
"Your output will be injected as reference material for a DIFFERENT "
|
||||
"assistant that continues the conversation. "
|
||||
"Do NOT respond to any questions or requests in the conversation — "
|
||||
"only output the structured summary. "
|
||||
"Do NOT include any preamble, greeting, or prefix. "
|
||||
"Treat the conversation turns below as source material for a "
|
||||
"compact record of prior work. "
|
||||
"Produce only the structured summary; do not add a greeting, "
|
||||
"preamble, or prefix. "
|
||||
"Write the summary in the same language the user was using in the "
|
||||
"conversation — do not translate or switch to English. "
|
||||
"NEVER include API keys, tokens, passwords, secrets, credentials, "
|
||||
@@ -777,7 +775,7 @@ class ContextCompressor(ContextEngine):
|
||||
[THE SINGLE MOST IMPORTANT FIELD. Copy the user's most recent request or
|
||||
task assignment verbatim — the exact words they used. If multiple tasks
|
||||
were requested and only some are done, list only the ones NOT yet completed.
|
||||
The next assistant must pick up exactly here. Example:
|
||||
Continuation should pick up exactly here. Example:
|
||||
"User asked: 'Now refactor the auth module to use JWT instead of sessions'"
|
||||
If no outstanding task exists, write "None."]
|
||||
|
||||
@@ -814,7 +812,7 @@ Be specific with file paths, commands, line numbers, and results.]
|
||||
[Important technical decisions and WHY they were made]
|
||||
|
||||
## Resolved Questions
|
||||
[Questions the user asked that were ALREADY answered — include the answer so the next assistant does not re-answer them]
|
||||
[Questions the user asked that were ALREADY answered — include the answer so it is not repeated]
|
||||
|
||||
## Pending User Asks
|
||||
[Questions or requests from the user that have NOT yet been answered or fulfilled. If none, write "None."]
|
||||
@@ -851,7 +849,7 @@ Update the summary using this exact structure. PRESERVE all existing information
|
||||
# First compaction: summarize from scratch
|
||||
prompt = f"""{_summarizer_preamble}
|
||||
|
||||
Create a structured handoff summary for a different assistant that will continue this conversation after earlier turns are compacted. The next assistant should be able to understand what happened without re-reading the original turns.
|
||||
Create a structured checkpoint summary for the conversation after earlier turns are compacted. The summary should preserve enough detail for continuity without re-reading the original turns.
|
||||
|
||||
TURNS TO SUMMARIZE:
|
||||
{content_to_summarize}
|
||||
|
||||
@@ -477,8 +477,8 @@ class CopilotACPClient:
|
||||
proc.stdin.write(json.dumps(payload) + "\n")
|
||||
proc.stdin.flush()
|
||||
|
||||
deadline = time.time() + timeout_seconds
|
||||
while time.time() < deadline:
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
while time.monotonic() < deadline:
|
||||
if proc.poll() is not None:
|
||||
break
|
||||
try:
|
||||
|
||||
@@ -68,8 +68,10 @@ SUPPORTED_POOL_STRATEGIES = {
|
||||
}
|
||||
|
||||
# Cooldown before retrying an exhausted credential.
|
||||
# 429 (rate-limited) and 402 (billing/quota) both cool down after 1 hour.
|
||||
# Transient 401 auth failures cool down briefly so single-key setups can recover.
|
||||
# 429 (rate-limited), 402 (billing/quota), and other failures cool down after 1 hour.
|
||||
# Provider-supplied reset_at timestamps override these defaults.
|
||||
EXHAUSTED_TTL_401_SECONDS = 5 * 60 # 5 minutes
|
||||
EXHAUSTED_TTL_429_SECONDS = 60 * 60 # 1 hour
|
||||
EXHAUSTED_TTL_DEFAULT_SECONDS = 60 * 60 # 1 hour
|
||||
|
||||
@@ -190,6 +192,8 @@ def _is_manual_source(source: str) -> bool:
|
||||
|
||||
def _exhausted_ttl(error_code: Optional[int]) -> int:
|
||||
"""Return cooldown seconds based on the HTTP status that caused exhaustion."""
|
||||
if error_code == 401:
|
||||
return EXHAUSTED_TTL_401_SECONDS
|
||||
if error_code == 429:
|
||||
return EXHAUSTED_TTL_429_SECONDS
|
||||
return EXHAUSTED_TTL_DEFAULT_SECONDS
|
||||
@@ -305,14 +309,29 @@ def _iter_custom_providers(config: Optional[dict] = None):
|
||||
yield _normalize_custom_pool_name(name), entry
|
||||
|
||||
|
||||
def get_custom_provider_pool_key(base_url: str) -> Optional[str]:
|
||||
def get_custom_provider_pool_key(base_url: str, provider_name: Optional[str] = None) -> Optional[str]:
|
||||
"""Look up the custom_providers list in config.yaml and return 'custom:<name>' for a matching base_url.
|
||||
|
||||
When provider_name is given, prefer matching by name first (solving the case where
|
||||
multiple custom providers share the same base_url but have different API keys).
|
||||
Falls back to base_url matching when no name match is found.
|
||||
|
||||
Returns None if no match is found.
|
||||
"""
|
||||
if not base_url:
|
||||
return None
|
||||
normalized_url = base_url.strip().rstrip("/")
|
||||
|
||||
# When a provider name is given, try to match by name first.
|
||||
# This fixes the P1 bug where two custom providers sharing the same
|
||||
# base_url always resolve to the first one's credentials.
|
||||
if provider_name:
|
||||
normalized_name = _normalize_custom_pool_name(provider_name)
|
||||
for norm_name, entry in _iter_custom_providers():
|
||||
if norm_name == normalized_name:
|
||||
return f"{CUSTOM_POOL_PREFIX}{norm_name}"
|
||||
|
||||
# Fall back to base_url matching (original behavior)
|
||||
for norm_name, entry in _iter_custom_providers():
|
||||
entry_url = str(entry.get("base_url") or "").strip().rstrip("/")
|
||||
if entry_url and entry_url == normalized_url:
|
||||
|
||||
+1
-1
@@ -1607,7 +1607,7 @@ def _run_llm_review(prompt: str) -> Dict[str, Any]:
|
||||
# terminal. The background-thread runner also hides it; this
|
||||
# belt-and-suspenders path matters when a caller invokes
|
||||
# run_curator_review(synchronous=True) from the CLI.
|
||||
with open(os.devnull, "w") as _devnull, \
|
||||
with open(os.devnull, "w", encoding="utf-8") as _devnull, \
|
||||
contextlib.redirect_stdout(_devnull), \
|
||||
contextlib.redirect_stderr(_devnull):
|
||||
conv_result = review_agent.run_conversation(user_message=prompt)
|
||||
|
||||
+4
-2
@@ -852,13 +852,15 @@ def get_cute_tool_message(
|
||||
s = str(s)
|
||||
if _tool_preview_max_len == 0:
|
||||
return s # no limit
|
||||
return (s[:n-3] + "...") if len(s) > n else s
|
||||
limit = _tool_preview_max_len
|
||||
return (s[:limit-3] + "...") if len(s) > limit else s
|
||||
|
||||
def _path(p, n=35):
|
||||
p = str(p)
|
||||
if _tool_preview_max_len == 0:
|
||||
return p # no limit
|
||||
return ("..." + p[-(n-3):]) if len(p) > n else p
|
||||
limit = _tool_preview_max_len
|
||||
return ("..." + p[-(limit-3):]) if len(p) > limit else p
|
||||
|
||||
def _wrap(line: str) -> str:
|
||||
"""Apply skin tool prefix and failure suffix."""
|
||||
|
||||
+78
-13
@@ -144,7 +144,51 @@ def decide_image_input_mode(
|
||||
# it fires, which is cheaper than permanent quality loss.
|
||||
|
||||
|
||||
def _guess_mime(path: Path) -> str:
|
||||
def _sniff_mime_from_bytes(raw: bytes) -> Optional[str]:
|
||||
"""Detect image MIME from magic bytes. Returns None if unrecognised.
|
||||
|
||||
Filename-based detection (``mimetypes.guess_type``) is unreliable when
|
||||
upstream platforms lie about content-type. Discord, for example, can
|
||||
serve a PNG with ``content_type=image/webp`` for proxied/animated
|
||||
stickers, custom emoji previews, or images uploaded via certain bots.
|
||||
Anthropic strictly validates that declared media_type matches the
|
||||
actual bytes and returns HTTP 400 on mismatch, so we sniff to be safe.
|
||||
"""
|
||||
if not raw:
|
||||
return None
|
||||
# PNG: 89 50 4E 47 0D 0A 1A 0A
|
||||
if raw.startswith(b"\x89PNG\r\n\x1a\n"):
|
||||
return "image/png"
|
||||
# JPEG: FF D8 FF
|
||||
if raw.startswith(b"\xff\xd8\xff"):
|
||||
return "image/jpeg"
|
||||
# GIF87a / GIF89a
|
||||
if raw[:6] in (b"GIF87a", b"GIF89a"):
|
||||
return "image/gif"
|
||||
# WEBP: "RIFF" .... "WEBP"
|
||||
if len(raw) >= 12 and raw[:4] == b"RIFF" and raw[8:12] == b"WEBP":
|
||||
return "image/webp"
|
||||
# BMP: "BM"
|
||||
if raw.startswith(b"BM"):
|
||||
return "image/bmp"
|
||||
# HEIC/HEIF: ftypheic / ftypheix / ftypmif1 / ftypmsf1 etc.
|
||||
if len(raw) >= 12 and raw[4:8] == b"ftyp" and raw[8:12] in (
|
||||
b"heic", b"heix", b"hevc", b"hevx", b"mif1", b"msf1", b"heim", b"heis",
|
||||
):
|
||||
return "image/heic"
|
||||
return None
|
||||
|
||||
|
||||
def _guess_mime(path: Path, raw: Optional[bytes] = None) -> str:
|
||||
"""Return image MIME type for *path*.
|
||||
|
||||
If *raw* bytes are provided, magic-byte sniffing wins (authoritative).
|
||||
Otherwise we fall back to ``mimetypes`` then suffix-based defaults.
|
||||
"""
|
||||
if raw is not None:
|
||||
sniffed = _sniff_mime_from_bytes(raw)
|
||||
if sniffed:
|
||||
return sniffed
|
||||
mime, _ = mimetypes.guess_type(str(path))
|
||||
if mime and mime.startswith("image/"):
|
||||
return mime
|
||||
@@ -178,7 +222,7 @@ def _file_to_data_url(path: Path) -> Optional[str]:
|
||||
except Exception as exc:
|
||||
logger.warning("image_routing: failed to read %s — %s", path, exc)
|
||||
return None
|
||||
mime = _guess_mime(path)
|
||||
mime = _guess_mime(path, raw=raw)
|
||||
b64 = base64.b64encode(raw).decode("ascii")
|
||||
return f"data:{mime};base64,{b64}"
|
||||
|
||||
@@ -190,24 +234,30 @@ def build_native_content_parts(
|
||||
"""Build an OpenAI-style ``content`` list for a user turn.
|
||||
|
||||
Shape:
|
||||
[{"type": "text", "text": "..."},
|
||||
[{"type": "text", "text": "...\\n\\n[Image attached at: /local/path]"},
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}},
|
||||
...]
|
||||
|
||||
The local path of each successfully attached image is appended to the
|
||||
text part as ``[Image attached at: <path>]``. The model still sees the
|
||||
pixels via the ``image_url`` part (full native vision); the path note
|
||||
just gives it a string handle so MCP/skill tools that take an image
|
||||
path or URL argument can be invoked on the same image without an
|
||||
extra round-trip. This parallels the text-mode hint produced by
|
||||
``Runner._enrich_message_with_vision`` (``vision_analyze using image_url:
|
||||
<path>``) so behaviour is consistent across both image input modes.
|
||||
|
||||
Images are attached at their native size. If a provider rejects the
|
||||
request because an image is too large (e.g. Anthropic's 5 MB per-image
|
||||
ceiling), the agent's retry loop transparently shrinks and retries
|
||||
once — see ``run_agent._try_shrink_image_parts_in_messages``.
|
||||
|
||||
Returns (content_parts, skipped_paths). Skipped paths are files that
|
||||
couldn't be read from disk.
|
||||
couldn't be read from disk and are NOT advertised in the path hints.
|
||||
"""
|
||||
parts: List[Dict[str, Any]] = []
|
||||
skipped: List[str] = []
|
||||
|
||||
text = (user_text or "").strip()
|
||||
if text:
|
||||
parts.append({"type": "text", "text": text})
|
||||
image_parts: List[Dict[str, Any]] = []
|
||||
attached_paths: List[str] = []
|
||||
|
||||
for raw_path in image_paths:
|
||||
p = Path(raw_path)
|
||||
@@ -218,15 +268,30 @@ def build_native_content_parts(
|
||||
if not data_url:
|
||||
skipped.append(str(raw_path))
|
||||
continue
|
||||
parts.append({
|
||||
image_parts.append({
|
||||
"type": "image_url",
|
||||
"image_url": {"url": data_url},
|
||||
})
|
||||
attached_paths.append(str(raw_path))
|
||||
|
||||
# If the text was empty, add a neutral prompt so the turn isn't just images.
|
||||
if not text and any(p.get("type") == "image_url" for p in parts):
|
||||
parts.insert(0, {"type": "text", "text": "What do you see in this image?"})
|
||||
text = (user_text or "").strip()
|
||||
|
||||
# If at least one image attached, build a single text part that combines
|
||||
# the user's caption (or a neutral default) with one path hint per image.
|
||||
if attached_paths:
|
||||
base_text = text or "What do you see in this image?"
|
||||
path_hints = "\n".join(
|
||||
f"[Image attached at: {p}]" for p in attached_paths
|
||||
)
|
||||
combined_text = f"{base_text}\n\n{path_hints}"
|
||||
parts: List[Dict[str, Any]] = [{"type": "text", "text": combined_text}]
|
||||
parts.extend(image_parts)
|
||||
return parts, skipped
|
||||
|
||||
# No images successfully attached — fall back to plain text-only behaviour.
|
||||
parts = []
|
||||
if text:
|
||||
parts.append({"type": "text", "text": text})
|
||||
return parts, skipped
|
||||
|
||||
|
||||
|
||||
@@ -754,7 +754,7 @@ def _load_context_cache() -> Dict[str, int]:
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
with open(path) as f:
|
||||
with open(path, encoding="utf-8") as f:
|
||||
data = yaml.safe_load(f) or {}
|
||||
return data.get("context_lengths", {})
|
||||
except Exception as e:
|
||||
@@ -776,7 +776,7 @@ def save_context_length(model: str, base_url: str, length: int) -> None:
|
||||
path = _get_context_cache_path()
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(path, "w") as f:
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
yaml.dump({"context_lengths": cache}, f, default_flow_style=False)
|
||||
logger.info("Cached context length %s -> %s tokens", key, f"{length:,}")
|
||||
except Exception as e:
|
||||
@@ -800,7 +800,7 @@ def _invalidate_cached_context_length(model: str, base_url: str) -> None:
|
||||
path = _get_context_cache_path()
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(path, "w") as f:
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
yaml.dump({"context_lengths": cache}, f, default_flow_style=False)
|
||||
except Exception as e:
|
||||
logger.debug("Failed to invalidate context length cache entry %s: %s", key, e)
|
||||
|
||||
+9
-5
@@ -381,14 +381,18 @@ def get_model_capabilities(provider: str, model: str) -> Optional[ModelCapabilit
|
||||
|
||||
# Extract capability flags (default to False if missing)
|
||||
supports_tools = bool(entry.get("tool_call", False))
|
||||
# Vision: check both the `attachment` flag and `modalities.input` for "image".
|
||||
# Some models (e.g. gemma-4) list image in input modalities but not attachment.
|
||||
# Vision: prefer explicit `modalities.input` when models.dev provides it.
|
||||
# The older `attachment` flag can be stale or too broad for image routing;
|
||||
# fall back to it only when the input modalities are absent/invalid.
|
||||
input_mods = entry.get("modalities", {})
|
||||
if isinstance(input_mods, dict):
|
||||
input_mods = input_mods.get("input", [])
|
||||
input_mods = input_mods.get("input")
|
||||
else:
|
||||
input_mods = []
|
||||
supports_vision = bool(entry.get("attachment", False)) or "image" in input_mods
|
||||
input_mods = None
|
||||
if isinstance(input_mods, list):
|
||||
supports_vision = "image" in input_mods
|
||||
else:
|
||||
supports_vision = bool(entry.get("attachment", False))
|
||||
supports_reasoning = bool(entry.get("reasoning", False))
|
||||
|
||||
# Extract limits
|
||||
|
||||
@@ -144,7 +144,7 @@ def nous_rate_limit_remaining() -> Optional[float]:
|
||||
"""
|
||||
path = _state_path()
|
||||
try:
|
||||
with open(path) as f:
|
||||
with open(path, encoding="utf-8") as f:
|
||||
state = json.load(f)
|
||||
reset_at = state.get("reset_at", 0)
|
||||
remaining = reset_at - time.time()
|
||||
|
||||
+9
-6
@@ -56,12 +56,15 @@ _SENSITIVE_BODY_KEYS = frozenset({
|
||||
})
|
||||
|
||||
# Snapshot at import time so runtime env mutations (e.g. LLM-generated
|
||||
# `export HERMES_REDACT_SECRETS=true`) cannot enable/disable redaction
|
||||
# mid-session. OFF by default — user must opt in via
|
||||
# `security.redact_secrets: true` in config.yaml (bridged to this env var
|
||||
# in hermes_cli/main.py and gateway/run.py) or `HERMES_REDACT_SECRETS=true`
|
||||
# in ~/.hermes/.env.
|
||||
_REDACT_ENABLED = os.getenv("HERMES_REDACT_SECRETS", "").lower() in ("1", "true", "yes", "on")
|
||||
# `export HERMES_REDACT_SECRETS=false`) cannot disable redaction
|
||||
# mid-session. ON by default — secure default per issue #17691. Users who
|
||||
# need raw credential values in tool output (e.g. working on the redactor
|
||||
# itself) can opt out via `security.redact_secrets: false` in config.yaml
|
||||
# (bridged to this env var in hermes_cli/main.py, gateway/run.py, and
|
||||
# cli.py) or `HERMES_REDACT_SECRETS=false` in ~/.hermes/.env. An opt-out
|
||||
# warning is logged at gateway and CLI startup so operators see the
|
||||
# downgrade — see `_log_redaction_status()` in gateway/run.py and cli.py.
|
||||
_REDACT_ENABLED = os.getenv("HERMES_REDACT_SECRETS", "true").lower() in ("1", "true", "yes", "on")
|
||||
|
||||
# Known API key prefixes -- match the prefix + contiguous token chars
|
||||
_PREFIX_PATTERNS = [
|
||||
|
||||
@@ -617,7 +617,7 @@ def _locked_update_approvals() -> Iterator[Dict[str, Any]]:
|
||||
save_allowlist(data)
|
||||
return
|
||||
|
||||
with open(lock_path, "a+") as lock_fh:
|
||||
with open(lock_path, "a+", encoding="utf-8") as lock_fh:
|
||||
fcntl.flock(lock_fh.fileno(), fcntl.LOCK_EX)
|
||||
try:
|
||||
data = load_allowlist()
|
||||
|
||||
+159
-14
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from decimal import Decimal
|
||||
@@ -82,6 +83,121 @@ _UTC_NOW = lambda: datetime.now(timezone.utc)
|
||||
# Official docs snapshot entries. Models whose published pricing and cache
|
||||
# semantics are stable enough to encode exactly.
|
||||
_OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
# ── Anthropic Claude 4.7 ─────────────────────────────────────────────
|
||||
# Opus 4.5/4.6/4.7 share $5/$25 pricing (new tokenizer, up to 35% more
|
||||
# tokens for the same text).
|
||||
# Source: https://platform.claude.com/docs/en/about-claude/pricing
|
||||
(
|
||||
"anthropic",
|
||||
"claude-opus-4-7",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("5.00"),
|
||||
output_cost_per_million=Decimal("25.00"),
|
||||
cache_read_cost_per_million=Decimal("0.50"),
|
||||
cache_write_cost_per_million=Decimal("6.25"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
"claude-opus-4-7-20250507",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("5.00"),
|
||||
output_cost_per_million=Decimal("25.00"),
|
||||
cache_read_cost_per_million=Decimal("0.50"),
|
||||
cache_write_cost_per_million=Decimal("6.25"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
# ── Anthropic Claude 4.6 ─────────────────────────────────────────────
|
||||
(
|
||||
"anthropic",
|
||||
"claude-opus-4-6",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("5.00"),
|
||||
output_cost_per_million=Decimal("25.00"),
|
||||
cache_read_cost_per_million=Decimal("0.50"),
|
||||
cache_write_cost_per_million=Decimal("6.25"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
"claude-opus-4-6-20250414",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("5.00"),
|
||||
output_cost_per_million=Decimal("25.00"),
|
||||
cache_read_cost_per_million=Decimal("0.50"),
|
||||
cache_write_cost_per_million=Decimal("6.25"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
"claude-sonnet-4-6",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("3.00"),
|
||||
output_cost_per_million=Decimal("15.00"),
|
||||
cache_read_cost_per_million=Decimal("0.30"),
|
||||
cache_write_cost_per_million=Decimal("3.75"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
"claude-sonnet-4-6-20250414",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("3.00"),
|
||||
output_cost_per_million=Decimal("15.00"),
|
||||
cache_read_cost_per_million=Decimal("0.30"),
|
||||
cache_write_cost_per_million=Decimal("3.75"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
# ── Anthropic Claude 4.5 ─────────────────────────────────────────────
|
||||
(
|
||||
"anthropic",
|
||||
"claude-opus-4-5",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("5.00"),
|
||||
output_cost_per_million=Decimal("25.00"),
|
||||
cache_read_cost_per_million=Decimal("0.50"),
|
||||
cache_write_cost_per_million=Decimal("6.25"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
"claude-sonnet-4-5",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("3.00"),
|
||||
output_cost_per_million=Decimal("15.00"),
|
||||
cache_read_cost_per_million=Decimal("0.30"),
|
||||
cache_write_cost_per_million=Decimal("3.75"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
"claude-haiku-4-5",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("1.00"),
|
||||
output_cost_per_million=Decimal("5.00"),
|
||||
cache_read_cost_per_million=Decimal("0.10"),
|
||||
cache_write_cost_per_million=Decimal("1.25"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
# ── Anthropic Claude 4 / 4.1 ─────────────────────────────────────────
|
||||
(
|
||||
"anthropic",
|
||||
"claude-opus-4-20250514",
|
||||
@@ -91,8 +207,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
cache_read_cost_per_million=Decimal("1.50"),
|
||||
cache_write_cost_per_million=Decimal("18.75"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||
pricing_version="anthropic-prompt-caching-2026-03-16",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
@@ -103,8 +219,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
cache_read_cost_per_million=Decimal("0.30"),
|
||||
cache_write_cost_per_million=Decimal("3.75"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||
pricing_version="anthropic-prompt-caching-2026-03-16",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
# OpenAI
|
||||
(
|
||||
@@ -184,7 +300,7 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
source_url="https://openai.com/api/pricing/",
|
||||
pricing_version="openai-pricing-2026-03-16",
|
||||
),
|
||||
# Anthropic older models (pre-4.6 generation)
|
||||
# ── Anthropic older models (pre-4.5 generation) ────────────────────────
|
||||
(
|
||||
"anthropic",
|
||||
"claude-3-5-sonnet-20241022",
|
||||
@@ -194,8 +310,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
cache_read_cost_per_million=Decimal("0.30"),
|
||||
cache_write_cost_per_million=Decimal("3.75"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||
pricing_version="anthropic-pricing-2026-03-16",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
@@ -206,8 +322,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
cache_read_cost_per_million=Decimal("0.08"),
|
||||
cache_write_cost_per_million=Decimal("1.00"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||
pricing_version="anthropic-pricing-2026-03-16",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
@@ -218,8 +334,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
cache_read_cost_per_million=Decimal("1.50"),
|
||||
cache_write_cost_per_million=Decimal("18.75"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||
pricing_version="anthropic-pricing-2026-03-16",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
(
|
||||
"anthropic",
|
||||
@@ -230,8 +346,8 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
cache_read_cost_per_million=Decimal("0.03"),
|
||||
cache_write_cost_per_million=Decimal("0.30"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching",
|
||||
pricing_version="anthropic-pricing-2026-03-16",
|
||||
source_url="https://platform.claude.com/docs/en/about-claude/pricing",
|
||||
pricing_version="anthropic-pricing-2026-05",
|
||||
),
|
||||
# DeepSeek
|
||||
(
|
||||
@@ -426,8 +542,37 @@ def resolve_billing_route(
|
||||
return BillingRoute(provider=provider_name or "unknown", model=model.split("/")[-1] if model else "", base_url=base_url or "", billing_mode="unknown")
|
||||
|
||||
|
||||
def _normalize_anthropic_model_name(model: str) -> str:
|
||||
"""Normalize Anthropic model name variants to canonical form.
|
||||
|
||||
Handles:
|
||||
- Dot notation: claude-opus-4.7 → claude-opus-4-7
|
||||
- Short aliases: claude-opus-4.7 → claude-opus-4-7
|
||||
- Strips anthropic/ prefix if present
|
||||
"""
|
||||
name = model.lower().strip()
|
||||
if name.startswith("anthropic/"):
|
||||
name = name[len("anthropic/"):]
|
||||
# Normalize dots to dashes in version numbers (e.g. 4.7 → 4-7, 4.6 → 4-6)
|
||||
# But preserve the rest of the name structure
|
||||
name = re.sub(r"(\d+)\.(\d+)", r"\1-\2", name)
|
||||
return name
|
||||
|
||||
|
||||
def _lookup_official_docs_pricing(route: BillingRoute) -> Optional[PricingEntry]:
|
||||
return _OFFICIAL_DOCS_PRICING.get((route.provider, route.model.lower()))
|
||||
model = route.model.lower()
|
||||
# Direct lookup first
|
||||
entry = _OFFICIAL_DOCS_PRICING.get((route.provider, model))
|
||||
if entry:
|
||||
return entry
|
||||
# Try normalized name for Anthropic (handles dot-notation like opus-4.7)
|
||||
if route.provider == "anthropic":
|
||||
normalized = _normalize_anthropic_model_name(model)
|
||||
if normalized != model:
|
||||
entry = _OFFICIAL_DOCS_PRICING.get((route.provider, normalized))
|
||||
if entry:
|
||||
return entry
|
||||
return None
|
||||
|
||||
|
||||
def _openrouter_pricing_entry(route: BillingRoute) -> Optional[PricingEntry]:
|
||||
|
||||
@@ -20,6 +20,10 @@ Usage:
|
||||
python batch_runner.py --dataset_file=data.jsonl --batch_size=10 --run_name=my_run --distribution=image_gen
|
||||
"""
|
||||
|
||||
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
|
||||
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
|
||||
import hermes_bootstrap # noqa: F401
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
|
||||
+19
-1
@@ -601,7 +601,7 @@ agent:
|
||||
# - A preset like "hermes-cli" or "hermes-telegram" (curated tool set)
|
||||
# - A list of individual toolsets to compose your own (see list below)
|
||||
#
|
||||
# Supported platform keys: cli, telegram, discord, whatsapp, slack, qqbot, teams
|
||||
# Supported platform keys: cli, telegram, discord, whatsapp, slack, qqbot, teams, google_chat
|
||||
#
|
||||
# Examples:
|
||||
#
|
||||
@@ -632,6 +632,7 @@ agent:
|
||||
# homeassistant: hermes-homeassistant (same as telegram)
|
||||
# qqbot: hermes-qqbot (same as telegram)
|
||||
# teams: hermes-teams (same as telegram)
|
||||
# google_chat: hermes-google_chat (same as telegram)
|
||||
#
|
||||
platform_toolsets:
|
||||
cli: [hermes-cli]
|
||||
@@ -644,6 +645,7 @@ platform_toolsets:
|
||||
qqbot: [hermes-qqbot]
|
||||
yuanbao: [hermes-yuanbao]
|
||||
teams: [hermes-teams]
|
||||
google_chat: [hermes-google_chat]
|
||||
|
||||
# =============================================================================
|
||||
# Gateway Platform Settings
|
||||
@@ -875,6 +877,22 @@ display:
|
||||
# Toggle at runtime with /verbose in the CLI
|
||||
tool_progress: all
|
||||
|
||||
# Auto-cleanup of temporary progress bubbles after the final response lands.
|
||||
# On platforms that support message deletion (currently Telegram), this
|
||||
# removes the tool-progress bubble, "⏳ Still working..." notices, and
|
||||
# context-pressure status messages once the final reply has been delivered —
|
||||
# keeping long-running turns visible live, then tidy afterward. Failed runs
|
||||
# leave the bubbles in place as breadcrumbs. Off by default.
|
||||
# Per-platform override: display.platforms.telegram.cleanup_progress
|
||||
# true: Delete tracked progress/status bubbles on successful turn
|
||||
# false: Leave everything in place (default)
|
||||
# Example:
|
||||
# display:
|
||||
# platforms:
|
||||
# telegram:
|
||||
# cleanup_progress: true
|
||||
cleanup_progress: false
|
||||
|
||||
# Gateway-only natural mid-turn assistant updates.
|
||||
# When true, completed assistant status messages are sent as separate chat
|
||||
# messages. This is independent of tool_progress and gateway streaming.
|
||||
|
||||
@@ -9,10 +9,13 @@ Usage:
|
||||
python cli.py # Start interactive mode with all tools
|
||||
python cli.py --toolsets web,terminal # Start with specific toolsets
|
||||
python cli.py --skills hermes-agent-dev,github-auth
|
||||
python cli.py -q "your question" # Single query mode
|
||||
python cli.py --list-tools # List available tools and exit
|
||||
"""
|
||||
|
||||
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
|
||||
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
|
||||
import hermes_bootstrap # noqa: F401
|
||||
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
@@ -728,8 +731,43 @@ def _run_cleanup():
|
||||
_active_worktree: Optional[Dict[str, str]] = None
|
||||
|
||||
|
||||
def _normalize_git_bash_path(p: Optional[str]) -> Optional[str]:
|
||||
"""Translate a Git Bash-style path (``/c/Users/...``) to the native
|
||||
Windows form (``C:\\Users\\...``) that Python's ``subprocess.Popen``
|
||||
and ``pathlib.Path`` accept.
|
||||
|
||||
No-op on non-Windows and for paths that already look native. Git on
|
||||
native Windows normally emits forward-slash Windows paths
|
||||
(``C:/Users/...``) which both bash and Python handle, but certain
|
||||
configurations (Git Bash shells, MSYS2, WSL-mounted repos) surface
|
||||
``/c/...`` or ``/cygdrive/c/...`` variants.
|
||||
"""
|
||||
if not p:
|
||||
return p
|
||||
if sys.platform != "win32":
|
||||
return p
|
||||
import re as _re
|
||||
# /c/Users/... or /C/Users/...
|
||||
m = _re.match(r"^/([a-zA-Z])/(.*)$", p)
|
||||
if m:
|
||||
drive, rest = m.group(1), m.group(2)
|
||||
return f"{drive.upper()}:\\{rest.replace('/', chr(92))}"
|
||||
# /cygdrive/c/... or /mnt/c/...
|
||||
m = _re.match(r"^/(?:cygdrive|mnt)/([a-zA-Z])/(.*)$", p)
|
||||
if m:
|
||||
drive, rest = m.group(1), m.group(2)
|
||||
return f"{drive.upper()}:\\{rest.replace('/', chr(92))}"
|
||||
return p
|
||||
|
||||
|
||||
def _git_repo_root() -> Optional[str]:
|
||||
"""Return the git repo root for CWD, or None if not in a repo."""
|
||||
"""Return the git repo root for CWD, or None if not in a repo.
|
||||
|
||||
Runs through :func:`_normalize_git_bash_path` so callers can pass
|
||||
the result directly to ``Path``/``subprocess.Popen(cwd=...)`` on
|
||||
Windows without hitting ``C:\\c\\Users\\...`` style resolution
|
||||
mistakes.
|
||||
"""
|
||||
import subprocess
|
||||
try:
|
||||
result = subprocess.run(
|
||||
@@ -737,7 +775,7 @@ def _git_repo_root() -> Optional[str]:
|
||||
capture_output=True, text=True, timeout=5,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return result.stdout.strip()
|
||||
return _normalize_git_bash_path(result.stdout.strip())
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
@@ -781,7 +819,7 @@ def _setup_worktree(repo_root: str = None) -> Optional[Dict[str, str]]:
|
||||
try:
|
||||
existing = gitignore.read_text() if gitignore.exists() else ""
|
||||
if _ignore_entry not in existing.splitlines():
|
||||
with open(gitignore, "a") as f:
|
||||
with open(gitignore, "a", encoding="utf-8") as f:
|
||||
if existing and not existing.endswith("\n"):
|
||||
f.write("\n")
|
||||
f.write(f"{_ignore_entry}\n")
|
||||
@@ -832,10 +870,39 @@ def _setup_worktree(repo_root: str = None) -> Optional[Dict[str, str]]:
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(str(src), str(dst))
|
||||
elif src.is_dir():
|
||||
# Symlink directories (faster, saves disk)
|
||||
# Symlink directories (faster, saves disk). On Windows,
|
||||
# symlink creation requires Developer Mode or elevation,
|
||||
# and fails with OSError otherwise — fall back to a
|
||||
# recursive copy so the worktree is still usable. The
|
||||
# copy is slower and uses disk, but it doesn't require
|
||||
# admin and matches the Linux/macOS symlink outcome
|
||||
# functionally.
|
||||
if not dst.exists():
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.symlink(str(src_resolved), str(dst))
|
||||
try:
|
||||
os.symlink(str(src_resolved), str(dst))
|
||||
except (OSError, NotImplementedError) as _sym_err:
|
||||
if sys.platform == "win32":
|
||||
logger.info(
|
||||
".worktreeinclude: symlink failed (%s) — "
|
||||
"falling back to copytree on Windows.",
|
||||
_sym_err,
|
||||
)
|
||||
try:
|
||||
shutil.copytree(
|
||||
str(src_resolved),
|
||||
str(dst),
|
||||
symlinks=True,
|
||||
dirs_exist_ok=False,
|
||||
)
|
||||
except Exception as _copy_err:
|
||||
logger.warning(
|
||||
".worktreeinclude: copy fallback "
|
||||
"also failed for %s -> %s: %s",
|
||||
src, dst, _copy_err,
|
||||
)
|
||||
else:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.debug("Error copying .worktreeinclude entries: %s", e)
|
||||
|
||||
@@ -1408,7 +1475,13 @@ def _cprint(text: str):
|
||||
|
||||
import asyncio as _asyncio
|
||||
try:
|
||||
current_loop = _asyncio.get_event_loop_policy().get_event_loop()
|
||||
# Use get_running_loop() instead of get_event_loop() to avoid the
|
||||
# DeprecationWarning / RuntimeWarning emitted by Python 3.10+ when
|
||||
# get_event_loop() is called from a thread that has no current event
|
||||
# loop set (e.g. the process_loop background thread). Fixes #19285.
|
||||
current_loop = _asyncio.get_running_loop()
|
||||
except RuntimeError:
|
||||
current_loop = None
|
||||
except Exception:
|
||||
current_loop = None
|
||||
# Same thread as the app's loop → safe to print directly.
|
||||
@@ -2074,7 +2147,7 @@ def save_config_value(key_path: str, value: any) -> bool:
|
||||
|
||||
# Load existing config
|
||||
if config_path.exists():
|
||||
with open(config_path, 'r') as f:
|
||||
with open(config_path, 'r', encoding="utf-8") as f:
|
||||
config = yaml.safe_load(f) or {}
|
||||
else:
|
||||
config = {}
|
||||
@@ -2571,6 +2644,15 @@ class HermesCLI:
|
||||
return "class:status-bar-warn"
|
||||
return "class:status-bar-good"
|
||||
|
||||
@staticmethod
|
||||
def _compression_count_style(count: int) -> str:
|
||||
"""Return a style class reflecting context compression pressure."""
|
||||
if count >= 10:
|
||||
return "class:status-bar-bad"
|
||||
if count >= 5:
|
||||
return "class:status-bar-warn"
|
||||
return "class:status-bar-dim"
|
||||
|
||||
def _build_context_bar(self, percent_used: Optional[int], width: int = 10) -> str:
|
||||
safe_percent = max(0, min(100, percent_used or 0))
|
||||
filled = round((safe_percent / 100) * width)
|
||||
@@ -2854,6 +2936,9 @@ class HermesCLI:
|
||||
return self._trim_status_bar_text(text, width)
|
||||
if width < 76:
|
||||
parts = [f"⚕ {snapshot['model_short']}", percent_label]
|
||||
compressions = snapshot.get("compressions", 0)
|
||||
if compressions:
|
||||
parts.append(f"🗜️ {compressions}")
|
||||
parts.append(duration_label)
|
||||
return self._trim_status_bar_text(" · ".join(parts), width)
|
||||
|
||||
@@ -2864,7 +2949,10 @@ class HermesCLI:
|
||||
else:
|
||||
context_label = "ctx --"
|
||||
|
||||
compressions = snapshot.get("compressions", 0)
|
||||
parts = [f"⚕ {snapshot['model_short']}", context_label, percent_label]
|
||||
if compressions:
|
||||
parts.append(f"🗜️ {compressions}")
|
||||
parts.append(duration_label)
|
||||
prompt_elapsed = snapshot.get("prompt_elapsed")
|
||||
if prompt_elapsed:
|
||||
@@ -2898,15 +2986,21 @@ class HermesCLI:
|
||||
percent = snapshot["context_percent"]
|
||||
percent_label = f"{percent}%" if percent is not None else "--"
|
||||
if width < 76:
|
||||
compressions = snapshot.get("compressions", 0)
|
||||
frags = [
|
||||
("class:status-bar", " ⚕ "),
|
||||
("class:status-bar-strong", snapshot["model_short"]),
|
||||
("class:status-bar-dim", " · "),
|
||||
(self._status_bar_context_style(percent), percent_label),
|
||||
]
|
||||
if compressions:
|
||||
frags.append(("class:status-bar-dim", " · "))
|
||||
frags.append((self._compression_count_style(compressions), f"🗜️ {compressions}"))
|
||||
frags.extend([
|
||||
("class:status-bar-dim", " · "),
|
||||
("class:status-bar-dim", duration_label),
|
||||
("class:status-bar", " "),
|
||||
]
|
||||
])
|
||||
else:
|
||||
if snapshot["context_length"]:
|
||||
ctx_total = _format_context_length(snapshot["context_length"])
|
||||
@@ -2916,6 +3010,7 @@ class HermesCLI:
|
||||
context_label = "ctx --"
|
||||
|
||||
bar_style = self._status_bar_context_style(percent)
|
||||
compressions = snapshot.get("compressions", 0)
|
||||
frags = [
|
||||
("class:status-bar", " ⚕ "),
|
||||
("class:status-bar-strong", snapshot["model_short"]),
|
||||
@@ -2925,9 +3020,14 @@ class HermesCLI:
|
||||
(bar_style, self._build_context_bar(percent)),
|
||||
("class:status-bar-dim", " "),
|
||||
(bar_style, percent_label),
|
||||
]
|
||||
if compressions:
|
||||
frags.append(("class:status-bar-dim", " │ "))
|
||||
frags.append((self._compression_count_style(compressions), f"🗜️ {compressions}"))
|
||||
frags.extend([
|
||||
("class:status-bar-dim", " │ "),
|
||||
("class:status-bar-dim", duration_label),
|
||||
]
|
||||
])
|
||||
# Position 7: per-prompt elapsed timer (live or frozen)
|
||||
prompt_elapsed = snapshot.get("prompt_elapsed")
|
||||
if prompt_elapsed:
|
||||
@@ -7958,6 +8058,7 @@ class HermesCLI:
|
||||
output_tokens = getattr(agent, "session_output_tokens", 0) or 0
|
||||
cache_read_tokens = getattr(agent, "session_cache_read_tokens", 0) or 0
|
||||
cache_write_tokens = getattr(agent, "session_cache_write_tokens", 0) or 0
|
||||
reasoning_tokens = getattr(agent, "session_reasoning_tokens", 0) or 0
|
||||
prompt = agent.session_prompt_tokens
|
||||
completion = agent.session_completion_tokens
|
||||
total = agent.session_total_tokens
|
||||
@@ -7989,6 +8090,8 @@ class HermesCLI:
|
||||
print(f" Cache read tokens: {cache_read_tokens:>10,}")
|
||||
print(f" Cache write tokens: {cache_write_tokens:>10,}")
|
||||
print(f" Output tokens: {output_tokens:>10,}")
|
||||
if reasoning_tokens:
|
||||
print(f" ↳ Reasoning (subset): {reasoning_tokens:>10,}")
|
||||
print(f" Prompt tokens (total): {prompt:>10,}")
|
||||
print(f" Completion tokens: {completion:>10,}")
|
||||
print(f" Total tokens: {total:>10,}")
|
||||
@@ -9670,7 +9773,7 @@ class HermesCLI:
|
||||
# Debug: log to file (stdout may be devnull from redirect_stdout)
|
||||
try:
|
||||
_dbg = _hermes_home / "interrupt_debug.log"
|
||||
with open(_dbg, "a") as _f:
|
||||
with open(_dbg, "a", encoding="utf-8") as _f:
|
||||
_f.write(f"{time.strftime('%H:%M:%S')} interrupt fired: msg={str(interrupt_msg)[:60]!r}, "
|
||||
f"children={len(self.agent._active_children)}, "
|
||||
f"parent._interrupt={self.agent._interrupt_requested}\n")
|
||||
@@ -10213,6 +10316,24 @@ class HermesCLI:
|
||||
_welcome_text = "Welcome to Hermes Agent! Type your message or /help for commands."
|
||||
_welcome_color = "#FFF8DC"
|
||||
self._console_print(f"[{_welcome_color}]{_welcome_text}[/]")
|
||||
|
||||
# Redaction opt-out warning (#17691): ON by default, loud when off.
|
||||
# The redactor snapshots its state at import time so any toggle now
|
||||
# won't affect the running process — we just want the operator to
|
||||
# see that they're running without the safety net.
|
||||
try:
|
||||
_redact_raw = os.getenv("HERMES_REDACT_SECRETS", "true")
|
||||
if _redact_raw.lower() not in ("1", "true", "yes", "on"):
|
||||
self._console_print(
|
||||
"[bold red]⚠ Secret redaction is DISABLED[/] "
|
||||
f"(HERMES_REDACT_SECRETS={_redact_raw}). "
|
||||
"API keys and tokens may appear verbatim in chat output, "
|
||||
"session JSONs, and logs. Set "
|
||||
"[cyan]security.redact_secrets: true[/] in config.yaml "
|
||||
"to re-enable."
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
# First-time OpenClaw-residue banner — fires once if ~/.openclaw/ exists
|
||||
# after an OpenClaw→Hermes migration (especially migrations done by
|
||||
# OpenClaw's own tool, which doesn't archive the source directory).
|
||||
@@ -10484,7 +10605,7 @@ class HermesCLI:
|
||||
# Debug: log to file when message enters interrupt queue
|
||||
try:
|
||||
_dbg = _hermes_home / "interrupt_debug.log"
|
||||
with open(_dbg, "a") as _f:
|
||||
with open(_dbg, "a", encoding="utf-8") as _f:
|
||||
_f.write(f"{time.strftime('%H:%M:%S')} ENTER: queued interrupt msg={str(payload)[:60]!r}, "
|
||||
f"agent_running={self._agent_running}\n")
|
||||
except Exception:
|
||||
@@ -12145,8 +12266,12 @@ class HermesCLI:
|
||||
# Set the custom handler on prompt_toolkit's event loop
|
||||
try:
|
||||
import asyncio as _aio
|
||||
_loop = _aio.get_event_loop()
|
||||
# Use get_running_loop() to avoid DeprecationWarning on
|
||||
# Python 3.10+ when called outside an async context.
|
||||
_loop = _aio.get_running_loop()
|
||||
_loop.set_exception_handler(_suppress_closed_loop_errors)
|
||||
except RuntimeError:
|
||||
pass # No running loop -- nothing to patch
|
||||
except Exception:
|
||||
pass
|
||||
app.run()
|
||||
@@ -12284,6 +12409,15 @@ def main(
|
||||
"""
|
||||
global _active_worktree
|
||||
|
||||
# Force UTF-8 stdio on Windows before any banner/print() runs — the
|
||||
# Rich console prints Unicode box-drawing characters that would
|
||||
# UnicodeEncodeError on cp1252. No-op on Linux/macOS.
|
||||
try:
|
||||
from hermes_cli.stdio import configure_windows_stdio
|
||||
configure_windows_stdio()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Signal to terminal_tool that we're in interactive mode
|
||||
# This enables interactive sudo password prompts with timeout
|
||||
os.environ["HERMES_INTERACTIVE"] = "1"
|
||||
@@ -12481,7 +12615,18 @@ def main(
|
||||
):
|
||||
cli.session_id = cli.agent.session_id
|
||||
response = result.get("final_response", "") if isinstance(result, dict) else str(result)
|
||||
if response:
|
||||
# Surface backend errors that produced no visible output
|
||||
# (e.g. invalid model slug → provider 4xx). Mirrors the
|
||||
# interactive CLI path. Write to stderr so piped stdout
|
||||
# stays clean for automation wrappers.
|
||||
if (
|
||||
not response
|
||||
and isinstance(result, dict)
|
||||
and result.get("error")
|
||||
and (result.get("failed") or result.get("partial"))
|
||||
):
|
||||
print(f"Error: {result['error']}", file=sys.stderr)
|
||||
elif response:
|
||||
print(response)
|
||||
# Session ID goes to stderr so piped stdout is clean.
|
||||
print(f"\nsession_id: {cli.session_id}", file=sys.stderr)
|
||||
|
||||
+171
-10
@@ -14,6 +14,7 @@ import contextvars
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
@@ -41,6 +42,19 @@ from hermes_time import now as _hermes_now
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CronPromptInjectionBlocked(Exception):
|
||||
"""Raised by _build_job_prompt when the fully-assembled prompt trips the
|
||||
injection scanner. Caught in run_job so the operator sees a clean
|
||||
"job blocked" delivery instead of the scheduler crashing.
|
||||
|
||||
Assembled-prompt scanning (including loaded skill content) plugs the
|
||||
gap from #3968: create-time scanning only covers the user-supplied
|
||||
prompt field; skill content loaded at runtime was never scanned, so a
|
||||
malicious skill could carry an injection payload that reached the
|
||||
non-interactive (auto-approve) cron agent.
|
||||
"""
|
||||
|
||||
|
||||
def _resolve_cron_enabled_toolsets(job: dict, cfg: dict) -> list[str] | None:
|
||||
"""Resolve the toolset list for a cron job.
|
||||
|
||||
@@ -152,9 +166,54 @@ def _resolve_origin(job: dict) -> Optional[dict]:
|
||||
return None
|
||||
|
||||
|
||||
def _plugin_cron_env_var(platform_name: str) -> str:
|
||||
"""Return the cron home-channel env var registered by a plugin platform.
|
||||
|
||||
Falls through the platform registry so plugins that set
|
||||
``cron_deliver_env_var`` on their ``PlatformEntry`` get cron delivery
|
||||
support without editing this module.
|
||||
"""
|
||||
try:
|
||||
from hermes_cli.plugins import discover_plugins
|
||||
discover_plugins() # idempotent
|
||||
from gateway.platform_registry import platform_registry
|
||||
entry = platform_registry.get(platform_name.lower())
|
||||
if entry and entry.cron_deliver_env_var:
|
||||
return entry.cron_deliver_env_var
|
||||
except Exception:
|
||||
pass
|
||||
return ""
|
||||
|
||||
|
||||
def _is_known_delivery_platform(platform_name: str) -> bool:
|
||||
"""Whether ``platform_name`` is a valid cron delivery target.
|
||||
|
||||
Hardcoded built-ins in ``_KNOWN_DELIVERY_PLATFORMS`` are checked first;
|
||||
plugin platforms registered via ``PlatformEntry`` are accepted if they
|
||||
provide a ``cron_deliver_env_var``.
|
||||
"""
|
||||
name = platform_name.lower()
|
||||
if name in _KNOWN_DELIVERY_PLATFORMS:
|
||||
return True
|
||||
return bool(_plugin_cron_env_var(name))
|
||||
|
||||
|
||||
def _resolve_home_env_var(platform_name: str) -> str:
|
||||
"""Return the env var name for a platform's cron home channel.
|
||||
|
||||
Built-in platforms are in ``_HOME_TARGET_ENV_VARS``; plugin platforms are
|
||||
resolved from the platform registry.
|
||||
"""
|
||||
name = platform_name.lower()
|
||||
env_var = _HOME_TARGET_ENV_VARS.get(name)
|
||||
if env_var:
|
||||
return env_var
|
||||
return _plugin_cron_env_var(name)
|
||||
|
||||
|
||||
def _get_home_target_chat_id(platform_name: str) -> str:
|
||||
"""Return the configured home target chat/room ID for a delivery platform."""
|
||||
env_var = _HOME_TARGET_ENV_VARS.get(platform_name.lower())
|
||||
env_var = _resolve_home_env_var(platform_name)
|
||||
if not env_var:
|
||||
return ""
|
||||
value = os.getenv(env_var, "")
|
||||
@@ -167,7 +226,7 @@ def _get_home_target_chat_id(platform_name: str) -> str:
|
||||
|
||||
def _get_home_target_thread_id(platform_name: str) -> Optional[str]:
|
||||
"""Return the optional thread/topic ID for a platform home target."""
|
||||
env_var = _HOME_TARGET_ENV_VARS.get(platform_name.lower())
|
||||
env_var = _resolve_home_env_var(platform_name)
|
||||
if not env_var:
|
||||
return None
|
||||
value = os.getenv(f"{env_var}_THREAD_ID", "").strip()
|
||||
@@ -178,6 +237,24 @@ def _get_home_target_thread_id(platform_name: str) -> Optional[str]:
|
||||
return value or None
|
||||
|
||||
|
||||
def _iter_home_target_platforms():
|
||||
"""Iterate built-in + plugin platform names that expose a home channel.
|
||||
|
||||
Used by the ``deliver=origin`` fallback when the job has no origin.
|
||||
"""
|
||||
for name in _HOME_TARGET_ENV_VARS:
|
||||
yield name
|
||||
try:
|
||||
from hermes_cli.plugins import discover_plugins
|
||||
discover_plugins() # idempotent
|
||||
from gateway.platform_registry import platform_registry
|
||||
for entry in platform_registry.plugin_entries():
|
||||
if entry.cron_deliver_env_var and entry.name not in _HOME_TARGET_ENV_VARS:
|
||||
yield entry.name
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[dict]:
|
||||
"""Resolve one concrete auto-delivery target for a cron job."""
|
||||
|
||||
@@ -195,7 +272,7 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
}
|
||||
# Origin missing (e.g. job created via API/script) — try each
|
||||
# platform's home channel as a fallback instead of silently dropping.
|
||||
for platform_name in _HOME_TARGET_ENV_VARS:
|
||||
for platform_name in _iter_home_target_platforms():
|
||||
chat_id = _get_home_target_chat_id(platform_name)
|
||||
if chat_id:
|
||||
logger.info(
|
||||
@@ -251,7 +328,7 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
"thread_id": origin.get("thread_id"),
|
||||
}
|
||||
|
||||
if platform_name.lower() not in _KNOWN_DELIVERY_PLATFORMS:
|
||||
if not _is_known_delivery_platform(platform_name):
|
||||
return None
|
||||
chat_id = _get_home_target_chat_id(platform_name)
|
||||
if not chat_id:
|
||||
@@ -638,7 +715,21 @@ def _run_job_script(script_path: str) -> tuple[bool, str]:
|
||||
# choice explicit here keeps the allowed surface small and auditable.
|
||||
suffix = path.suffix.lower()
|
||||
if suffix in (".sh", ".bash"):
|
||||
argv = ["/bin/bash", str(path)]
|
||||
# Resolve bash dynamically so Windows (Git Bash) and Linux/macOS
|
||||
# all work. On native Windows without Git for Windows installed
|
||||
# shutil.which returns None — fall back to a clear error rather
|
||||
# than a FileNotFoundError with a confusing "[WinError 2]"
|
||||
# traceback.
|
||||
_bash = shutil.which("bash") or (
|
||||
"/bin/bash" if os.path.isfile("/bin/bash") else None
|
||||
)
|
||||
if _bash is None:
|
||||
return False, (
|
||||
f"Cannot run .sh/.bash script {path.name!r}: bash not found on PATH. "
|
||||
"On Windows, install Git for Windows (which ships Git Bash) "
|
||||
"or rewrite the script as Python (.py)."
|
||||
)
|
||||
argv = [_bash, str(path)]
|
||||
else:
|
||||
argv = [sys.executable, str(path)]
|
||||
|
||||
@@ -805,7 +896,7 @@ def _build_job_prompt(job: dict, prerun_script: Optional[tuple] = None) -> str:
|
||||
|
||||
skill_names = [str(name).strip() for name in skills if str(name).strip()]
|
||||
if not skill_names:
|
||||
return prompt
|
||||
return _scan_assembled_cron_prompt(prompt, job)
|
||||
|
||||
from tools.skills_tool import skill_view
|
||||
from tools.skill_usage import bump_use
|
||||
@@ -848,7 +939,32 @@ def _build_job_prompt(job: dict, prerun_script: Optional[tuple] = None) -> str:
|
||||
|
||||
if prompt:
|
||||
parts.extend(["", f"The user has provided the following instruction alongside the skill invocation: {prompt}"])
|
||||
return "\n".join(parts)
|
||||
return _scan_assembled_cron_prompt("\n".join(parts), job)
|
||||
|
||||
|
||||
def _scan_assembled_cron_prompt(assembled: str, job: dict) -> str:
|
||||
"""Scan the fully-assembled cron prompt (including skill content) for
|
||||
injection patterns. Raises ``CronPromptInjectionBlocked`` when a match
|
||||
fires so ``run_job`` can surface a clear refusal to the operator.
|
||||
|
||||
Plugs the #3968 gap: ``_scan_cron_prompt`` runs on the user-supplied
|
||||
prompt at create/update, but skill content is loaded from disk at
|
||||
runtime and was never scanned. Since cron runs non-interactively
|
||||
(auto-approves tool calls), a malicious skill carrying an injection
|
||||
payload bypassed every gate.
|
||||
"""
|
||||
from tools.cronjob_tools import _scan_cron_prompt
|
||||
|
||||
scan_error = _scan_cron_prompt(assembled)
|
||||
if scan_error:
|
||||
job_label = job.get("name") or job.get("id") or "<unknown>"
|
||||
logger.warning(
|
||||
"Cron job '%s': assembled prompt blocked by injection scanner — %s",
|
||||
job_label,
|
||||
scan_error,
|
||||
)
|
||||
raise CronPromptInjectionBlocked(scan_error)
|
||||
return assembled
|
||||
|
||||
|
||||
def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||
@@ -1003,7 +1119,31 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||
)
|
||||
return True, silent_doc, SILENT_MARKER, None
|
||||
|
||||
prompt = _build_job_prompt(job, prerun_script=prerun_script)
|
||||
try:
|
||||
prompt = _build_job_prompt(job, prerun_script=prerun_script)
|
||||
except CronPromptInjectionBlocked as block_exc:
|
||||
# Assembled prompt (user prompt + loaded skill content) tripped the
|
||||
# injection scanner. Refuse to run the agent this tick and surface
|
||||
# a clear failure to the operator so they see WHY the scheduled job
|
||||
# didn't run and can audit the offending skill.
|
||||
logger.warning(
|
||||
"Job '%s' (ID: %s): blocked by prompt-injection scanner — %s",
|
||||
job_name, job_id, block_exc,
|
||||
)
|
||||
blocked_doc = (
|
||||
f"# Cron Job: {job_name}\n\n"
|
||||
f"**Job ID:** {job_id}\n"
|
||||
f"**Run Time:** {_hermes_now().strftime('%Y-%m-%d %H:%M:%S')}\n"
|
||||
f"**Status:** BLOCKED\n\n"
|
||||
"The assembled prompt (user prompt + loaded skill content) tripped "
|
||||
"the cron injection scanner and the agent was NOT run.\n\n"
|
||||
f"**Scanner result:** {block_exc}\n\n"
|
||||
"Audit the skill(s) attached to this job for prompt-injection "
|
||||
"payloads or invisible-unicode markers. If the skill is legitimate "
|
||||
"and the match is a false positive, rephrase the content to avoid "
|
||||
"the threat pattern (`tools/cronjob_tools.py::_CRON_THREAT_PATTERNS`)."
|
||||
)
|
||||
return False, blocked_doc, "", str(block_exc)
|
||||
if prompt is None:
|
||||
logger.info("Job '%s': script produced no output, skipping AI call.", job_name)
|
||||
return True, "", SILENT_MARKER, None
|
||||
@@ -1088,7 +1228,7 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||
import yaml
|
||||
_cfg_path = str(_get_hermes_home() / "config.yaml")
|
||||
if os.path.exists(_cfg_path):
|
||||
with open(_cfg_path) as _f:
|
||||
with open(_cfg_path, encoding="utf-8") as _f:
|
||||
_cfg = yaml.safe_load(_f) or {}
|
||||
_cfg = _expand_env_vars(_cfg)
|
||||
_model_cfg = _cfg.get("model", {})
|
||||
@@ -1198,6 +1338,27 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||
except Exception as e:
|
||||
logger.debug("Job '%s': failed to load credential pool for %s: %s", job_id, runtime_provider, e)
|
||||
|
||||
# Initialize MCP servers so configured mcp_servers are available to
|
||||
# the agent's tool registry before AIAgent is constructed. Without
|
||||
# this, cron jobs never saw any MCP tools — only the gateway / CLI
|
||||
# paths called discover_mcp_tools() at startup. Idempotent: subsequent
|
||||
# ticks short-circuit on already-connected servers inside
|
||||
# register_mcp_servers(). Non-fatal on failure: a broken MCP server
|
||||
# shouldn't kill an otherwise-working cron job. See #4219.
|
||||
try:
|
||||
from tools.mcp_tool import discover_mcp_tools
|
||||
_mcp_tools = discover_mcp_tools()
|
||||
if _mcp_tools:
|
||||
logger.info(
|
||||
"Job '%s': %d MCP tool(s) available",
|
||||
job_id, len(_mcp_tools),
|
||||
)
|
||||
except Exception as _mcp_exc:
|
||||
logger.warning(
|
||||
"Job '%s': MCP initialization failed (non-fatal): %s",
|
||||
job_id, _mcp_exc,
|
||||
)
|
||||
|
||||
agent = AIAgent(
|
||||
model=model,
|
||||
api_key=runtime.get("api_key"),
|
||||
@@ -1450,7 +1611,7 @@ def tick(verbose: bool = True, adapters=None, loop=None) -> int:
|
||||
# Cross-platform file locking: fcntl on Unix, msvcrt on Windows
|
||||
lock_fd = None
|
||||
try:
|
||||
lock_fd = open(lock_file, "w")
|
||||
lock_fd = open(lock_file, "w", encoding="utf-8")
|
||||
if fcntl:
|
||||
fcntl.flock(lock_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
elif msvcrt:
|
||||
|
||||
@@ -14,6 +14,9 @@
|
||||
# keys; exposing it on LAN without auth is unsafe. If you want remote
|
||||
# access, use an SSH tunnel or put it behind a reverse proxy that
|
||||
# adds authentication — do NOT pass --insecure --host 0.0.0.0.
|
||||
# - If you override entrypoint, keep /opt/hermes/docker/entrypoint.sh in
|
||||
# the command chain. It drops root to the hermes user before gateway
|
||||
# files such as gateway.lock are created.
|
||||
# - The gateway's API server is off unless you uncomment API_SERVER_KEY
|
||||
# and API_SERVER_HOST. See docs/user-guide/api-server.md before doing
|
||||
# this on an internet-facing host.
|
||||
@@ -41,6 +44,15 @@ services:
|
||||
# - TEAMS_TENANT_ID=${TEAMS_TENANT_ID}
|
||||
# - TEAMS_ALLOWED_USERS=${TEAMS_ALLOWED_USERS}
|
||||
# - TEAMS_PORT=${TEAMS_PORT:-3978}
|
||||
# Google Chat — uncomment and fill in to enable the Google Chat gateway.
|
||||
# See website/docs/user-guide/messaging/google_chat.md for the full setup.
|
||||
# The SA JSON path must point to a file mounted into the container —
|
||||
# add a volume entry above (e.g. ``- ~/.hermes/google-chat-sa.json:/secrets/google-chat-sa.json:ro``)
|
||||
# then set GOOGLE_CHAT_SERVICE_ACCOUNT_JSON to that mount path.
|
||||
# - GOOGLE_CHAT_PROJECT_ID=${GOOGLE_CHAT_PROJECT_ID}
|
||||
# - GOOGLE_CHAT_SUBSCRIPTION_NAME=${GOOGLE_CHAT_SUBSCRIPTION_NAME}
|
||||
# - GOOGLE_CHAT_SERVICE_ACCOUNT_JSON=${GOOGLE_CHAT_SERVICE_ACCOUNT_JSON}
|
||||
# - GOOGLE_CHAT_ALLOWED_USERS=${GOOGLE_CHAT_ALLOWED_USERS}
|
||||
command: ["gateway", "run"]
|
||||
|
||||
dashboard:
|
||||
|
||||
@@ -365,7 +365,7 @@ class TerminalBench2EvalEnv(HermesAgentBaseEnv):
|
||||
os.makedirs(log_dir, exist_ok=True)
|
||||
run_ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
self._streaming_path = os.path.join(log_dir, f"samples_{run_ts}.jsonl")
|
||||
self._streaming_file = open(self._streaming_path, "w")
|
||||
self._streaming_file = open(self._streaming_path, "w", encoding="utf-8")
|
||||
self._streaming_lock = __import__("threading").Lock()
|
||||
print(f" Streaming results to: {self._streaming_path}")
|
||||
|
||||
|
||||
@@ -422,7 +422,7 @@ class YCBenchEvalEnv(HermesAgentBaseEnv):
|
||||
os.makedirs(log_dir, exist_ok=True)
|
||||
run_ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
self._streaming_path = os.path.join(log_dir, f"samples_{run_ts}.jsonl")
|
||||
self._streaming_file = open(self._streaming_path, "w")
|
||||
self._streaming_file = open(self._streaming_path, "w", encoding="utf-8")
|
||||
self._streaming_lock = threading.Lock()
|
||||
|
||||
print(f"\nYC-Bench eval matrix: {len(self.all_eval_items)} runs")
|
||||
|
||||
+82
-5
@@ -809,6 +809,12 @@ def load_gateway_config() -> GatewayConfig:
|
||||
os.environ["SLACK_FREE_RESPONSE_CHANNELS"] = str(frc)
|
||||
if "reactions" in slack_cfg and not os.getenv("SLACK_REACTIONS"):
|
||||
os.environ["SLACK_REACTIONS"] = str(slack_cfg["reactions"]).lower()
|
||||
# allowed_channels: if set, bot ONLY responds in these channels (whitelist)
|
||||
ac = slack_cfg.get("allowed_channels")
|
||||
if ac is not None and not os.getenv("SLACK_ALLOWED_CHANNELS"):
|
||||
if isinstance(ac, list):
|
||||
ac = ",".join(str(v) for v in ac)
|
||||
os.environ["SLACK_ALLOWED_CHANNELS"] = str(ac)
|
||||
|
||||
# Discord settings → env vars (env vars take precedence)
|
||||
discord_cfg = yaml_cfg.get("discord", {})
|
||||
@@ -893,6 +899,12 @@ def load_gateway_config() -> GatewayConfig:
|
||||
if isinstance(frc, list):
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["TELEGRAM_FREE_RESPONSE_CHATS"] = str(frc)
|
||||
# allowed_chats: if set, bot ONLY responds in these group chats (whitelist)
|
||||
ac = telegram_cfg.get("allowed_chats")
|
||||
if ac is not None and not os.getenv("TELEGRAM_ALLOWED_CHATS"):
|
||||
if isinstance(ac, list):
|
||||
ac = ",".join(str(v) for v in ac)
|
||||
os.environ["TELEGRAM_ALLOWED_CHATS"] = str(ac)
|
||||
ignored_threads = telegram_cfg.get("ignored_threads")
|
||||
if ignored_threads is not None and not os.getenv("TELEGRAM_IGNORED_THREADS"):
|
||||
if isinstance(ignored_threads, list):
|
||||
@@ -976,12 +988,35 @@ def load_gateway_config() -> GatewayConfig:
|
||||
if isinstance(frc, list):
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["DINGTALK_FREE_RESPONSE_CHATS"] = str(frc)
|
||||
# allowed_chats: if set, bot ONLY responds in these group chats (whitelist)
|
||||
ac = dingtalk_cfg.get("allowed_chats")
|
||||
if ac is not None and not os.getenv("DINGTALK_ALLOWED_CHATS"):
|
||||
if isinstance(ac, list):
|
||||
ac = ",".join(str(v) for v in ac)
|
||||
os.environ["DINGTALK_ALLOWED_CHATS"] = str(ac)
|
||||
allowed = dingtalk_cfg.get("allowed_users")
|
||||
if allowed is not None and not os.getenv("DINGTALK_ALLOWED_USERS"):
|
||||
if isinstance(allowed, list):
|
||||
allowed = ",".join(str(v) for v in allowed)
|
||||
os.environ["DINGTALK_ALLOWED_USERS"] = str(allowed)
|
||||
|
||||
# Mattermost settings → env vars (env vars take precedence)
|
||||
mattermost_cfg = yaml_cfg.get("mattermost", {})
|
||||
if isinstance(mattermost_cfg, dict):
|
||||
if "require_mention" in mattermost_cfg and not os.getenv("MATTERMOST_REQUIRE_MENTION"):
|
||||
os.environ["MATTERMOST_REQUIRE_MENTION"] = str(mattermost_cfg["require_mention"]).lower()
|
||||
frc = mattermost_cfg.get("free_response_channels")
|
||||
if frc is not None and not os.getenv("MATTERMOST_FREE_RESPONSE_CHANNELS"):
|
||||
if isinstance(frc, list):
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["MATTERMOST_FREE_RESPONSE_CHANNELS"] = str(frc)
|
||||
# allowed_channels: if set, bot ONLY responds in these channels (whitelist)
|
||||
ac = mattermost_cfg.get("allowed_channels")
|
||||
if ac is not None and not os.getenv("MATTERMOST_ALLOWED_CHANNELS"):
|
||||
if isinstance(ac, list):
|
||||
ac = ",".join(str(v) for v in ac)
|
||||
os.environ["MATTERMOST_ALLOWED_CHANNELS"] = str(ac)
|
||||
|
||||
# Matrix settings → env vars (env vars take precedence)
|
||||
matrix_cfg = yaml_cfg.get("matrix", {})
|
||||
if isinstance(matrix_cfg, dict):
|
||||
@@ -992,6 +1027,12 @@ def load_gateway_config() -> GatewayConfig:
|
||||
if isinstance(frc, list):
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["MATRIX_FREE_RESPONSE_ROOMS"] = str(frc)
|
||||
# allowed_rooms: if set, bot ONLY responds in these rooms (whitelist)
|
||||
ar = matrix_cfg.get("allowed_rooms")
|
||||
if ar is not None and not os.getenv("MATRIX_ALLOWED_ROOMS"):
|
||||
if isinstance(ar, list):
|
||||
ar = ",".join(str(v) for v in ar)
|
||||
os.environ["MATRIX_ALLOWED_ROOMS"] = str(ar)
|
||||
if "auto_thread" in matrix_cfg and not os.getenv("MATRIX_AUTO_THREAD"):
|
||||
os.environ["MATRIX_AUTO_THREAD"] = str(matrix_cfg["auto_thread"]).lower()
|
||||
if "dm_mention_threads" in matrix_cfg and not os.getenv("MATRIX_DM_MENTION_THREADS"):
|
||||
@@ -1152,10 +1193,17 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
|
||||
# WhatsApp (typically uses different auth mechanism)
|
||||
whatsapp_enabled = os.getenv("WHATSAPP_ENABLED", "").lower() in ("true", "1", "yes")
|
||||
if whatsapp_enabled:
|
||||
if Platform.WHATSAPP not in config.platforms:
|
||||
config.platforms[Platform.WHATSAPP] = PlatformConfig()
|
||||
config.platforms[Platform.WHATSAPP].enabled = True
|
||||
whatsapp_disabled_explicitly = os.getenv("WHATSAPP_ENABLED", "").lower() in ("false", "0", "no")
|
||||
if Platform.WHATSAPP in config.platforms:
|
||||
# YAML config exists — respect explicit disable
|
||||
wa_cfg = config.platforms[Platform.WHATSAPP]
|
||||
if whatsapp_disabled_explicitly:
|
||||
wa_cfg.enabled = False
|
||||
elif whatsapp_enabled:
|
||||
wa_cfg.enabled = True
|
||||
# else: keep whatever the YAML set
|
||||
elif whatsapp_enabled:
|
||||
config.platforms[Platform.WHATSAPP] = PlatformConfig(enabled=True)
|
||||
whatsapp_home = os.getenv("WHATSAPP_HOME_CHANNEL")
|
||||
if whatsapp_home and Platform.WHATSAPP in config.platforms:
|
||||
config.platforms[Platform.WHATSAPP].home_channel = HomeChannel(
|
||||
@@ -1616,7 +1664,10 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
# Registry-driven enable for plugin platforms. Built-ins have explicit
|
||||
# blocks above; plugins expose check_fn() which is the single source of
|
||||
# truth for "are my env vars set?". When it returns True, ensure the
|
||||
# platform is enabled so start() will create its adapter.
|
||||
# platform is enabled so start() will create its adapter. Plugins that
|
||||
# need to seed ``PlatformConfig.extra`` from env vars (e.g. Google Chat's
|
||||
# project_id / subscription_name) can supply ``env_enablement_fn`` on
|
||||
# their PlatformEntry — called here BEFORE adapter construction.
|
||||
try:
|
||||
from hermes_cli.plugins import discover_plugins
|
||||
discover_plugins() # idempotent
|
||||
@@ -1632,5 +1683,31 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
if platform not in config.platforms:
|
||||
config.platforms[platform] = PlatformConfig()
|
||||
config.platforms[platform].enabled = True
|
||||
# Seed extras from env if the plugin opted in.
|
||||
if entry.env_enablement_fn is not None:
|
||||
try:
|
||||
seed = entry.env_enablement_fn()
|
||||
except Exception as e:
|
||||
logger.debug(
|
||||
"env_enablement_fn for %s raised: %s", entry.name, e
|
||||
)
|
||||
seed = None
|
||||
if isinstance(seed, dict) and seed:
|
||||
# Extract the home_channel dict (if provided) so we wire it
|
||||
# up as a proper HomeChannel dataclass. Everything else is
|
||||
# merged into ``extra``.
|
||||
home = seed.pop("home_channel", None)
|
||||
config.platforms[platform].extra.update(seed)
|
||||
if isinstance(home, dict) and home.get("chat_id"):
|
||||
config.platforms[platform].home_channel = HomeChannel(
|
||||
platform=platform,
|
||||
chat_id=str(home["chat_id"]),
|
||||
name=str(home.get("name") or "Home"),
|
||||
thread_id=(
|
||||
str(home["thread_id"])
|
||||
if home.get("thread_id")
|
||||
else None
|
||||
),
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug("Plugin platform enable pass failed: %s", e)
|
||||
|
||||
@@ -35,6 +35,12 @@ _GLOBAL_DEFAULTS: dict[str, Any] = {
|
||||
"show_reasoning": False,
|
||||
"tool_preview_length": 0,
|
||||
"streaming": None, # None = follow top-level streaming config
|
||||
# When true, delete tool-progress / "Still working..." / status bubbles
|
||||
# after the final response lands on platforms that support message
|
||||
# deletion (e.g. Telegram). Off by default — progress is still shown
|
||||
# live, just cleaned up after success so the chat doesn't fill up with
|
||||
# stale breadcrumbs. Failed runs leave bubbles in place as breadcrumbs.
|
||||
"cleanup_progress": False,
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -188,6 +194,10 @@ def _normalise(setting: str, value: Any) -> Any:
|
||||
if isinstance(value, str):
|
||||
return value.lower() in ("true", "1", "yes", "on")
|
||||
return bool(value)
|
||||
if setting == "cleanup_progress":
|
||||
if isinstance(value, str):
|
||||
return value.lower() in ("true", "1", "yes", "on")
|
||||
return bool(value)
|
||||
if setting == "tool_preview_length":
|
||||
try:
|
||||
return int(value)
|
||||
|
||||
+12
-1
@@ -195,12 +195,23 @@ class PairingStore:
|
||||
"""
|
||||
Approve a pairing code. Adds the user to the approved list.
|
||||
|
||||
Returns {user_id, user_name} on success, None if code is invalid/expired.
|
||||
Returns {user_id, user_name} on success, None if code is
|
||||
invalid/expired OR the platform is currently locked out after
|
||||
``MAX_FAILED_ATTEMPTS`` failed approvals (#10195). Callers can
|
||||
disambiguate with ``_is_locked_out(platform)``.
|
||||
"""
|
||||
with self._lock:
|
||||
self._cleanup_expired(platform)
|
||||
code = code.upper().strip()
|
||||
|
||||
# Lockout check — must run before the pending lookup so a
|
||||
# valid code (e.g. one already sitting in pending) cannot be
|
||||
# accepted once the lockout fires. Without this, the lockout
|
||||
# only blocks `generate_code`, not `approve_code` — nullifying
|
||||
# the brute-force protection for any code already issued.
|
||||
if self._is_locked_out(platform):
|
||||
return None
|
||||
|
||||
pending = self._load_json(self._pending_path(platform))
|
||||
if code not in pending:
|
||||
self._record_failed_attempt(platform)
|
||||
|
||||
@@ -110,6 +110,21 @@ class PlatformEntry:
|
||||
# Do not use markdown."). Empty string = no hint.
|
||||
platform_hint: str = ""
|
||||
|
||||
# ── Env-driven auto-configuration ──
|
||||
# Optional: read env vars, return a dict of ``PlatformConfig.extra`` fields
|
||||
# to seed when the platform is auto-enabled. Called during
|
||||
# ``_apply_env_overrides`` BEFORE the adapter is constructed, so
|
||||
# ``gateway status`` etc. can reflect env-only configuration without
|
||||
# instantiating the adapter. Return ``None`` (or an empty dict) to skip.
|
||||
# Signature: () -> Optional[dict[str, Any]]
|
||||
env_enablement_fn: Optional[Callable[[], Optional[dict]]] = None
|
||||
|
||||
# Optional: home-channel env var name for cron/notification delivery
|
||||
# (e.g. ``"IRC_HOME_CHANNEL"``). When set, ``cron.scheduler`` treats this
|
||||
# platform as a valid ``deliver=<name>`` target and reads the env var to
|
||||
# resolve the default chat/room ID. Empty = no cron home-channel support.
|
||||
cron_deliver_env_var: str = ""
|
||||
|
||||
|
||||
class PlatformRegistry:
|
||||
"""Central registry of platform adapters.
|
||||
|
||||
@@ -4,18 +4,34 @@ There are two ways to add a platform to the Hermes gateway:
|
||||
|
||||
## Plugin Path (Recommended for Community/Third-Party)
|
||||
|
||||
Create a plugin directory in `~/.hermes/plugins/` with a `PLUGIN.yaml` and
|
||||
`adapter.py`. The adapter inherits from `BasePlatformAdapter` and registers
|
||||
via `ctx.register_platform()` in the `register(ctx)` entry point. This
|
||||
requires **zero changes to core Hermes code**.
|
||||
Create a plugin directory in `~/.hermes/plugins/` (or under `plugins/platforms/`
|
||||
for bundled plugins) with a `plugin.yaml` and `adapter.py`. The adapter
|
||||
inherits from `BasePlatformAdapter` and registers via
|
||||
`ctx.register_platform()` in the `register(ctx)` entry point. This requires
|
||||
**zero changes to core Hermes code**.
|
||||
|
||||
The plugin system automatically handles: adapter creation, config parsing,
|
||||
user authorization, cron delivery, send_message routing, system prompt hints,
|
||||
status display, gateway setup, and more.
|
||||
|
||||
See `plugins/platforms/irc/` for a complete reference implementation, and
|
||||
**Three optional hooks cover the edges most adapters need:**
|
||||
|
||||
- `env_enablement_fn: () -> Optional[dict]` — seeds `PlatformConfig.extra`
|
||||
(and an optional `home_channel` dict) from env vars BEFORE the adapter is
|
||||
constructed. Without this, env-only setups don't surface in
|
||||
`hermes gateway status` or `get_connected_platforms()` until the SDK
|
||||
instantiates.
|
||||
- `cron_deliver_env_var: str` — name of the `*_HOME_CHANNEL` env var. When
|
||||
set, `deliver=<name>` cron jobs route to this var without editing
|
||||
`cron/scheduler.py`'s hardcoded sets.
|
||||
- `plugin.yaml` `requires_env` / `optional_env` rich-dict entries —
|
||||
auto-populate `OPTIONAL_ENV_VARS` in `hermes_cli/config.py` so the setup
|
||||
wizard surfaces proper descriptions, prompts, password flags, and URLs.
|
||||
|
||||
See `plugins/platforms/irc/`, `plugins/platforms/teams/`, and
|
||||
`plugins/platforms/google_chat/` for complete working examples, and
|
||||
`website/docs/developer-guide/adding-platform-adapters.md` for the full
|
||||
plugin guide with code examples.
|
||||
plugin guide with code examples and hook documentation.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -917,6 +917,16 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
"type": "bearer",
|
||||
"required": bool(self._api_key),
|
||||
},
|
||||
"runtime": {
|
||||
"mode": "server_agent",
|
||||
"tool_execution": "server",
|
||||
"split_runtime": False,
|
||||
"description": (
|
||||
"The API server creates a server-side Hermes AIAgent; "
|
||||
"tools execute on the API-server host unless a future "
|
||||
"explicit split-runtime mode is enabled."
|
||||
),
|
||||
},
|
||||
"features": {
|
||||
"chat_completions": True,
|
||||
"chat_completions_streaming": True,
|
||||
@@ -1316,8 +1326,8 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
try:
|
||||
result, agent_usage = await agent_task
|
||||
usage = agent_usage or usage
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as exc:
|
||||
logger.warning("Agent task %s failed, usage data lost: %s", completion_id, exc)
|
||||
|
||||
# Finish chunk
|
||||
finish_chunk = {
|
||||
@@ -1888,12 +1898,12 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
"output_tokens": usage.get("output_tokens", 0),
|
||||
"total_tokens": usage.get("total_tokens", 0),
|
||||
}
|
||||
full_history = list(conversation_history)
|
||||
full_history.append({"role": "user", "content": user_message})
|
||||
if isinstance(result, dict) and result.get("messages"):
|
||||
full_history.extend(result["messages"])
|
||||
else:
|
||||
full_history.append({"role": "assistant", "content": final_response_text})
|
||||
full_history = self._build_response_conversation_history(
|
||||
conversation_history,
|
||||
user_message,
|
||||
result,
|
||||
final_response_text,
|
||||
)
|
||||
_persist_response_snapshot(
|
||||
completed_env,
|
||||
conversation_history_snapshot=full_history,
|
||||
@@ -2192,17 +2202,22 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
|
||||
# Build the full conversation history for storage
|
||||
# (includes tool calls from the agent run)
|
||||
full_history = list(conversation_history)
|
||||
full_history.append({"role": "user", "content": user_message})
|
||||
# Add agent's internal messages if available
|
||||
agent_messages = result.get("messages", [])
|
||||
if agent_messages:
|
||||
full_history.extend(agent_messages)
|
||||
else:
|
||||
full_history.append({"role": "assistant", "content": final_response})
|
||||
full_history = self._build_response_conversation_history(
|
||||
conversation_history,
|
||||
user_message,
|
||||
result,
|
||||
final_response,
|
||||
)
|
||||
|
||||
# Build output items (includes tool calls + final message)
|
||||
output_items = self._extract_output_items(result)
|
||||
# Build output items from the current turn only. AIAgent returns a
|
||||
# full transcript in result["messages"], while older/mocked paths may
|
||||
# return only the current turn suffix.
|
||||
output_start_index = self._response_messages_turn_start_index(
|
||||
conversation_history,
|
||||
user_message,
|
||||
result,
|
||||
)
|
||||
output_items = self._extract_output_items(result, start_index=output_start_index)
|
||||
|
||||
response_data = {
|
||||
"id": response_id,
|
||||
@@ -2494,17 +2509,70 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _extract_output_items(result: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Build the full output item array from the agent's messages.
|
||||
def _build_response_conversation_history(
|
||||
conversation_history: List[Dict[str, Any]],
|
||||
user_message: Any,
|
||||
result: Dict[str, Any],
|
||||
final_response: Any,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Build the stored Responses transcript without duplicating history."""
|
||||
prior = list(conversation_history)
|
||||
current_user = {"role": "user", "content": user_message}
|
||||
agent_messages = result.get("messages") if isinstance(result, dict) else None
|
||||
|
||||
Walks *result["messages"]* and emits:
|
||||
if isinstance(agent_messages, list) and agent_messages:
|
||||
turn_start = APIServerAdapter._response_messages_turn_start_index(
|
||||
conversation_history,
|
||||
user_message,
|
||||
result,
|
||||
)
|
||||
if turn_start:
|
||||
return list(agent_messages)
|
||||
|
||||
full_history = prior
|
||||
full_history.append(current_user)
|
||||
full_history.extend(agent_messages)
|
||||
return full_history
|
||||
|
||||
full_history = prior
|
||||
full_history.append(current_user)
|
||||
full_history.append({"role": "assistant", "content": final_response})
|
||||
return full_history
|
||||
|
||||
@staticmethod
|
||||
def _response_messages_turn_start_index(
|
||||
conversation_history: List[Dict[str, Any]],
|
||||
user_message: Any,
|
||||
result: Dict[str, Any],
|
||||
) -> int:
|
||||
"""Detect transcript-shaped result["messages"] and return turn start."""
|
||||
agent_messages = result.get("messages") if isinstance(result, dict) else None
|
||||
if not isinstance(agent_messages, list) or not agent_messages:
|
||||
return 0
|
||||
|
||||
prior = list(conversation_history)
|
||||
current_user = {"role": "user", "content": user_message}
|
||||
expected_prefix = prior + [current_user]
|
||||
if agent_messages[:len(expected_prefix)] == expected_prefix:
|
||||
return len(expected_prefix)
|
||||
if prior and agent_messages[:len(prior)] == prior:
|
||||
return len(prior)
|
||||
return 0
|
||||
|
||||
@staticmethod
|
||||
def _extract_output_items(result: Dict[str, Any], start_index: int = 0) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Build the output item array from the agent's messages.
|
||||
|
||||
Walks *result["messages"]* starting at *start_index* and emits:
|
||||
- ``function_call`` items for each tool_call on assistant messages
|
||||
- ``function_call_output`` items for each tool-role message
|
||||
- a final ``message`` item with the assistant's text reply
|
||||
"""
|
||||
items: List[Dict[str, Any]] = []
|
||||
messages = result.get("messages", [])
|
||||
if start_index > 0:
|
||||
messages = messages[start_index:]
|
||||
|
||||
for msg in messages:
|
||||
role = msg.get("role")
|
||||
|
||||
+114
-26
@@ -1304,37 +1304,52 @@ class BasePlatformAdapter(ABC):
|
||||
self._fatal_error_code = None
|
||||
self._fatal_error_message = None
|
||||
self._fatal_error_retryable = True
|
||||
try:
|
||||
from gateway.status import write_runtime_status
|
||||
write_runtime_status(platform=self.platform.value, platform_state="connected", error_code=None, error_message=None)
|
||||
except Exception:
|
||||
pass
|
||||
self._write_runtime_status_safe("connected", platform_state="connected", error_code=None, error_message=None)
|
||||
|
||||
def _mark_disconnected(self) -> None:
|
||||
self._running = False
|
||||
if self.has_fatal_error:
|
||||
return
|
||||
try:
|
||||
from gateway.status import write_runtime_status
|
||||
write_runtime_status(platform=self.platform.value, platform_state="disconnected", error_code=None, error_message=None)
|
||||
except Exception:
|
||||
pass
|
||||
self._write_runtime_status_safe("disconnected", platform_state="disconnected", error_code=None, error_message=None)
|
||||
|
||||
def _set_fatal_error(self, code: str, message: str, *, retryable: bool) -> None:
|
||||
self._running = False
|
||||
self._fatal_error_code = code
|
||||
self._fatal_error_message = message
|
||||
self._fatal_error_retryable = retryable
|
||||
self._write_runtime_status_safe("fatal", platform_state="fatal", error_code=code, error_message=message)
|
||||
|
||||
def _write_runtime_status_safe(self, context: str, **kwargs) -> None:
|
||||
"""Write runtime status; log first failure per context at warning, rest at debug.
|
||||
|
||||
Status writes can fail on permissions, ENOSPC, missing status dir, etc.
|
||||
A persistently failing status dir used to be silent (``except: pass``).
|
||||
Logging every failure would spam the log on reconnect loops, so this
|
||||
surfaces the first failure per (platform, context) at warning level and
|
||||
downgrades subsequent failures to debug.
|
||||
"""
|
||||
try:
|
||||
from gateway.status import write_runtime_status
|
||||
write_runtime_status(
|
||||
platform=self.platform.value,
|
||||
platform_state="fatal",
|
||||
error_code=code,
|
||||
error_message=message,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
write_runtime_status(platform=self.platform.value, **kwargs)
|
||||
except Exception as exc:
|
||||
# Use getattr so object.__new__(...) test harnesses that skip __init__
|
||||
# don't blow up on attribute access.
|
||||
logged = getattr(self, "_status_write_logged", None)
|
||||
if logged is None:
|
||||
logged = set()
|
||||
try:
|
||||
self._status_write_logged = logged
|
||||
except Exception:
|
||||
pass
|
||||
key = (self.platform.value, context)
|
||||
if key not in logged:
|
||||
logger.warning(
|
||||
"Failed to write runtime status (%s) for %s: %s (further failures at debug level)",
|
||||
context, self.platform.value, exc,
|
||||
)
|
||||
logged.add(key)
|
||||
else:
|
||||
logger.debug("Failed to write runtime status (%s) for %s: %s", context, self.platform.value, exc)
|
||||
|
||||
async def _notify_fatal_error(self) -> None:
|
||||
handler = self._fatal_error_handler
|
||||
@@ -1874,23 +1889,38 @@ class BasePlatformAdapter(ABC):
|
||||
def extract_media(content: str) -> Tuple[List[Tuple[str, bool]], str]:
|
||||
"""
|
||||
Extract MEDIA:<path> tags and [[audio_as_voice]] directives from response text.
|
||||
|
||||
|
||||
The TTS tool returns responses like:
|
||||
[[audio_as_voice]]
|
||||
MEDIA:/path/to/audio.ogg
|
||||
|
||||
|
||||
Skills that produce large/lossless images (e.g. info-graph, where a
|
||||
rendered JPG is 1-2 MB but Telegram's sendPhoto recompresses to
|
||||
~200 KB at 1280px) can use ``[[as_document]]`` to request unmodified
|
||||
delivery via sendDocument instead of sendPhoto/sendMediaGroup. The
|
||||
directive is detected at the dispatch sites (which have access to the
|
||||
original response); this method just strips it so it never leaks into
|
||||
user-visible text. Per-file granularity is intentionally not exposed —
|
||||
when an agent emits ``[[as_document]]`` once, every image path in the
|
||||
same response is delivered as a document, mirroring the all-or-nothing
|
||||
scope of ``[[audio_as_voice]]``.
|
||||
|
||||
Args:
|
||||
content: The response text to scan.
|
||||
|
||||
|
||||
Returns:
|
||||
Tuple of (list of (path, is_voice) pairs, cleaned content with tags removed).
|
||||
"""
|
||||
media = []
|
||||
cleaned = content
|
||||
|
||||
|
||||
# Check for [[audio_as_voice]] directive
|
||||
has_voice_tag = "[[audio_as_voice]]" in content
|
||||
cleaned = cleaned.replace("[[audio_as_voice]]", "")
|
||||
# Strip [[as_document]] directive — callers inspect the original
|
||||
# ``content`` for it (so they can still react to it); here we just
|
||||
# keep it out of the user-visible cleaned text.
|
||||
cleaned = cleaned.replace("[[as_document]]", "")
|
||||
|
||||
# Extract MEDIA:<path> tags, allowing optional whitespace after the colon
|
||||
# and quoted/backticked paths for LLM-formatted outputs.
|
||||
@@ -2096,9 +2126,52 @@ class BasePlatformAdapter(ABC):
|
||||
|
||||
``generation`` lets callers tie the callback to a specific gateway run
|
||||
generation so stale runs cannot clear callbacks owned by a fresher run.
|
||||
|
||||
If a callback for the same ``session_key`` (and generation, when set)
|
||||
is already registered, the new callback is chained — both fire, in
|
||||
registration order, with per-callback exception isolation. This lets
|
||||
independent features (background-review release + temporary-bubble
|
||||
cleanup) coexist without clobbering each other. Stale-generation
|
||||
callers never overwrite a fresher generation's slot.
|
||||
"""
|
||||
if not session_key or not callable(callback):
|
||||
return
|
||||
|
||||
existing = self._post_delivery_callbacks.get(session_key)
|
||||
if existing is not None:
|
||||
if isinstance(existing, tuple) and len(existing) == 2:
|
||||
existing_gen, existing_cb = existing
|
||||
else:
|
||||
existing_gen, existing_cb = None, existing
|
||||
# Stale-generation registrations never overwrite a fresher slot.
|
||||
if (
|
||||
existing_gen is not None
|
||||
and generation is not None
|
||||
and int(generation) < int(existing_gen)
|
||||
):
|
||||
return
|
||||
# Same-or-newer generation: chain with the existing callback so
|
||||
# both fire in registration order.
|
||||
if callable(existing_cb) and (
|
||||
existing_gen is None
|
||||
or generation is None
|
||||
or int(existing_gen) == int(generation)
|
||||
):
|
||||
_prev = existing_cb
|
||||
_new = callback
|
||||
|
||||
def _chained() -> None:
|
||||
try:
|
||||
_prev()
|
||||
except Exception:
|
||||
logger.debug("Post-delivery callback failed", exc_info=True)
|
||||
try:
|
||||
_new()
|
||||
except Exception:
|
||||
logger.debug("Post-delivery callback failed", exc_info=True)
|
||||
|
||||
callback = _chained
|
||||
|
||||
if generation is None:
|
||||
self._post_delivery_callbacks[session_key] = callback
|
||||
else:
|
||||
@@ -2772,13 +2845,21 @@ class BasePlatformAdapter(ABC):
|
||||
if not response:
|
||||
logger.debug("[%s] Handler returned empty/None response for %s", self.name, event.source.chat_id)
|
||||
if response:
|
||||
# Capture [[as_document]] before extract_media strips it, so the
|
||||
# dispatch partition below can route image-extension files
|
||||
# through send_document instead of send_multiple_images. Used
|
||||
# by skills that produce large/lossless images (e.g. info-graph)
|
||||
# where Telegram's sendPhoto recompression destroys legibility.
|
||||
force_document_attachments = "[[as_document]]" in response
|
||||
|
||||
# Extract MEDIA:<path> tags (from TTS tool) before other processing
|
||||
media_files, response = self.extract_media(response)
|
||||
|
||||
|
||||
# Extract image URLs and send them as native platform attachments
|
||||
images, text_content = self.extract_images(response)
|
||||
# Strip any remaining internal directives from message body (fixes #1561)
|
||||
text_content = text_content.replace("[[audio_as_voice]]", "").strip()
|
||||
text_content = text_content.replace("[[as_document]]", "").strip()
|
||||
text_content = re.sub(r"MEDIA:\s*\S+", "", text_content).strip()
|
||||
if images:
|
||||
logger.info("[%s] extract_images found %d image(s) in response (%d chars)", self.name, len(images), len(response))
|
||||
@@ -2880,19 +2961,26 @@ class BasePlatformAdapter(ABC):
|
||||
_IMAGE_EXTS = {'.jpg', '.jpeg', '.png', '.webp', '.gif'}
|
||||
|
||||
# Partition images out of media_files + local_files so they
|
||||
# can be sent as a single batch (Signal RPC)
|
||||
# can be sent as a single batch (Signal RPC). When
|
||||
# ``[[as_document]]`` was set on the original response, image
|
||||
# files skip the photo path and route to send_document below
|
||||
# so they're delivered with original bytes (no Telegram
|
||||
# sendPhoto recompression).
|
||||
from urllib.parse import quote as _quote
|
||||
_image_paths: list = []
|
||||
_non_image_media: list = []
|
||||
for media_path, is_voice in media_files:
|
||||
_ext = Path(media_path).suffix.lower()
|
||||
if _ext in _IMAGE_EXTS and not is_voice:
|
||||
if (_ext in _IMAGE_EXTS
|
||||
and not is_voice
|
||||
and not force_document_attachments):
|
||||
_image_paths.append(media_path)
|
||||
else:
|
||||
_non_image_media.append((media_path, is_voice))
|
||||
_non_image_local: list = []
|
||||
for file_path in local_files:
|
||||
if Path(file_path).suffix.lower() in _IMAGE_EXTS:
|
||||
if (Path(file_path).suffix.lower() in _IMAGE_EXTS
|
||||
and not force_document_attachments):
|
||||
_image_paths.append(file_path)
|
||||
else:
|
||||
_non_image_local.append(file_path)
|
||||
|
||||
@@ -365,6 +365,20 @@ class DingTalkAdapter(BasePlatformAdapter):
|
||||
return {str(part).strip() for part in raw if str(part).strip()}
|
||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
||||
|
||||
def _dingtalk_allowed_chats(self) -> Set[str]:
|
||||
"""Return the whitelist of group chat IDs the bot will respond in.
|
||||
|
||||
When non-empty, group messages from chats NOT in this set are silently
|
||||
ignored — even if the bot is @mentioned. DMs are never filtered.
|
||||
Empty set means no restriction (fully backward compatible).
|
||||
"""
|
||||
raw = self.config.extra.get("allowed_chats") if self.config.extra else None
|
||||
if raw is None:
|
||||
raw = os.getenv("DINGTALK_ALLOWED_CHATS", "")
|
||||
if isinstance(raw, list):
|
||||
return {str(part).strip() for part in raw if str(part).strip()}
|
||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
||||
|
||||
def _compile_mention_patterns(self) -> List[re.Pattern]:
|
||||
"""Compile optional regex wake-word patterns for group triggers."""
|
||||
patterns = self.config.extra.get("mention_patterns") if self.config.extra else None
|
||||
@@ -443,13 +457,21 @@ class DingTalkAdapter(BasePlatformAdapter):
|
||||
|
||||
DMs remain unrestricted (subject to ``allowed_users`` which is enforced
|
||||
earlier). Group messages are accepted when:
|
||||
- the chat passes the ``allowed_chats`` whitelist (when set)
|
||||
- the chat is explicitly allowlisted in ``free_response_chats``
|
||||
- ``require_mention`` is disabled
|
||||
- the bot is @mentioned (``is_in_at_list``)
|
||||
- the text matches a configured regex wake-word pattern
|
||||
|
||||
When ``allowed_chats`` is non-empty, it acts as a hard gate — messages
|
||||
from any group chat not in the list are ignored regardless of the
|
||||
other rules.
|
||||
"""
|
||||
if not is_group:
|
||||
return True
|
||||
allowed = self._dingtalk_allowed_chats()
|
||||
if allowed and chat_id and chat_id not in allowed:
|
||||
return False
|
||||
if chat_id and chat_id in self._dingtalk_free_response_chats():
|
||||
return True
|
||||
if not self._dingtalk_require_mention():
|
||||
|
||||
+348
-42
@@ -10,6 +10,8 @@ Uses discord.py library for:
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import struct
|
||||
@@ -24,6 +26,10 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
VALID_THREAD_AUTO_ARCHIVE_MINUTES = {60, 1440, 4320, 10080}
|
||||
_DISCORD_COMMAND_SYNC_POLICIES = {"safe", "bulk", "off"}
|
||||
_DISCORD_COMMAND_SYNC_STATE_SUBDIR = "gateway"
|
||||
_DISCORD_COMMAND_SYNC_STATE_FILENAME = "discord_command_sync_state.json"
|
||||
_DISCORD_COMMAND_SYNC_MUTATION_INTERVAL_SECONDS = 4.5
|
||||
_DISCORD_COMMAND_SYNC_MAX_RATE_LIMIT_SLEEP_SECONDS = 30.0
|
||||
|
||||
try:
|
||||
import discord
|
||||
@@ -45,6 +51,7 @@ from gateway.config import Platform, PlatformConfig
|
||||
import re
|
||||
|
||||
from gateway.platforms.helpers import MessageDeduplicator, ThreadParticipationTracker
|
||||
from utils import atomic_json_write
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter,
|
||||
MessageEvent,
|
||||
@@ -470,6 +477,34 @@ class VoiceReceiver:
|
||||
pass
|
||||
|
||||
|
||||
def _read_dm_role_auth_guild() -> Optional[int]:
|
||||
"""Return the guild ID opted-in for DM role-based auth, or None.
|
||||
|
||||
Reads ``discord.dm_role_auth_guild`` from config.yaml. This is
|
||||
deliberately a config.yaml-only setting (not an env var): per repo
|
||||
policy, ``~/.hermes/.env`` is for secrets only, and this is a
|
||||
behavioral setting. Guild IDs aren't secrets.
|
||||
|
||||
Accepts ints or numeric strings in the config. Anything else
|
||||
(empty, malformed, None) returns None, which keeps the secure
|
||||
default (DM role-auth disabled).
|
||||
"""
|
||||
try:
|
||||
from hermes_cli.config import read_raw_config
|
||||
cfg = read_raw_config() or {}
|
||||
discord_cfg = cfg.get("discord", {}) or {}
|
||||
raw = discord_cfg.get("dm_role_auth_guild")
|
||||
except Exception:
|
||||
return None
|
||||
if raw is None or raw == "":
|
||||
return None
|
||||
try:
|
||||
guild_id = int(raw)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return guild_id if guild_id > 0 else None
|
||||
|
||||
|
||||
class DiscordAdapter(BasePlatformAdapter):
|
||||
"""
|
||||
Discord bot adapter.
|
||||
@@ -694,7 +729,17 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
# human-user allowlist below (bots aren't in it).
|
||||
else:
|
||||
# Non-bot: enforce the configured user/role allowlists.
|
||||
if not self._is_allowed_user(str(message.author.id), message.author):
|
||||
# Pass guild + is_dm so role checks are scoped to the
|
||||
# originating guild (prevents cross-guild DM bypass, see
|
||||
# _is_allowed_user docstring).
|
||||
_msg_guild = getattr(message, "guild", None)
|
||||
_is_dm = isinstance(message.channel, discord.DMChannel) or _msg_guild is None
|
||||
if not self._is_allowed_user(
|
||||
str(message.author.id),
|
||||
message.author,
|
||||
guild=_msg_guild,
|
||||
is_dm=_is_dm,
|
||||
):
|
||||
return
|
||||
|
||||
# Multi-agent filtering: if the message mentions specific bots
|
||||
@@ -825,6 +870,167 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
|
||||
logger.info("[%s] Disconnected", self.name)
|
||||
|
||||
def _command_sync_state_path(self) -> _Path:
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
directory = get_hermes_home() / _DISCORD_COMMAND_SYNC_STATE_SUBDIR
|
||||
try:
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
return directory / _DISCORD_COMMAND_SYNC_STATE_FILENAME
|
||||
|
||||
def _read_command_sync_state(self) -> dict:
|
||||
try:
|
||||
path = self._command_sync_state_path()
|
||||
if not path.exists():
|
||||
return {}
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
except Exception:
|
||||
return {}
|
||||
return data if isinstance(data, dict) else {}
|
||||
|
||||
def _write_command_sync_state(self, state: dict) -> None:
|
||||
atomic_json_write(
|
||||
self._command_sync_state_path(),
|
||||
state,
|
||||
indent=None,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
|
||||
def _command_sync_state_key(self, app_id: Any) -> str:
|
||||
return str(app_id or "unknown")
|
||||
|
||||
def _desired_command_sync_fingerprint(self) -> str:
|
||||
tree = self._client.tree if self._client else None
|
||||
desired = []
|
||||
if tree is not None:
|
||||
desired = [
|
||||
self._canonicalize_app_command_payload(command.to_dict(tree))
|
||||
for command in tree.get_commands()
|
||||
]
|
||||
desired.sort(key=lambda item: (item.get("type", 1), item.get("name", "")))
|
||||
payload = json.dumps(desired, sort_keys=True, separators=(",", ":"))
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||
|
||||
def _command_sync_skip_reason(self, app_id: Any, fingerprint: str) -> Optional[str]:
|
||||
entry = self._read_command_sync_state().get(self._command_sync_state_key(app_id))
|
||||
if not isinstance(entry, dict):
|
||||
return None
|
||||
now = time.time()
|
||||
retry_after_until = float(entry.get("retry_after_until") or 0)
|
||||
if retry_after_until > now:
|
||||
remaining = max(1, int(retry_after_until - now))
|
||||
return f"Discord asked us to wait before syncing slash commands; retry in {remaining}s"
|
||||
if entry.get("fingerprint") == fingerprint and entry.get("last_success_at"):
|
||||
return "same slash-command fingerprint already synced"
|
||||
return None
|
||||
|
||||
def _record_command_sync_attempt(self, app_id: Any, fingerprint: str) -> None:
|
||||
state = self._read_command_sync_state()
|
||||
state[self._command_sync_state_key(app_id)] = {
|
||||
**(
|
||||
state.get(self._command_sync_state_key(app_id))
|
||||
if isinstance(state.get(self._command_sync_state_key(app_id)), dict)
|
||||
else {}
|
||||
),
|
||||
"fingerprint": fingerprint,
|
||||
"last_attempt_at": time.time(),
|
||||
}
|
||||
self._write_command_sync_state(state)
|
||||
|
||||
def _record_command_sync_rate_limit(self, app_id: Any, fingerprint: str, retry_after: float) -> None:
|
||||
retry_after = max(1.0, float(retry_after))
|
||||
state = self._read_command_sync_state()
|
||||
state[self._command_sync_state_key(app_id)] = {
|
||||
**(
|
||||
state.get(self._command_sync_state_key(app_id))
|
||||
if isinstance(state.get(self._command_sync_state_key(app_id)), dict)
|
||||
else {}
|
||||
),
|
||||
"fingerprint": fingerprint,
|
||||
"last_attempt_at": time.time(),
|
||||
"retry_after_until": time.time() + retry_after,
|
||||
"retry_after": retry_after,
|
||||
}
|
||||
self._write_command_sync_state(state)
|
||||
|
||||
def _record_command_sync_success(self, app_id: Any, fingerprint: str, summary: dict) -> None:
|
||||
state = self._read_command_sync_state()
|
||||
state[self._command_sync_state_key(app_id)] = {
|
||||
"fingerprint": fingerprint,
|
||||
"last_attempt_at": time.time(),
|
||||
"last_success_at": time.time(),
|
||||
"summary": summary,
|
||||
}
|
||||
self._write_command_sync_state(state)
|
||||
|
||||
@staticmethod
|
||||
def _extract_discord_retry_after(exc: BaseException) -> Optional[float]:
|
||||
value = getattr(exc, "retry_after", None)
|
||||
if value is not None:
|
||||
try:
|
||||
return max(1.0, float(value))
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
response = getattr(exc, "response", None)
|
||||
headers = getattr(response, "headers", None)
|
||||
if headers:
|
||||
for key in ("Retry-After", "X-RateLimit-Reset-After"):
|
||||
try:
|
||||
raw = headers.get(key)
|
||||
except Exception:
|
||||
raw = None
|
||||
if raw is None:
|
||||
continue
|
||||
try:
|
||||
return max(1.0, float(raw))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _is_discord_rate_limit(exc: BaseException) -> bool:
|
||||
"""True only for exceptions that look like Discord 429 rate limits.
|
||||
|
||||
Narrower than ``hasattr(exc, 'retry_after')``: discord.py's own
|
||||
``RateLimited`` exception and any HTTPException with status 429
|
||||
qualify. This prevents suppressing unrelated failures that happen
|
||||
to expose a ``retry_after`` attribute."""
|
||||
# discord.py emits RateLimited / HTTPException subclasses for 429s.
|
||||
# Guard with isinstance-of-class so a mocked ``discord`` module
|
||||
# (where attrs are MagicMocks, not types) doesn't trip isinstance.
|
||||
if DISCORD_AVAILABLE and discord is not None:
|
||||
for attr_name in ("RateLimited", "HTTPException"):
|
||||
cls = getattr(discord, attr_name, None)
|
||||
if not isinstance(cls, type):
|
||||
continue
|
||||
if isinstance(exc, cls):
|
||||
if attr_name == "RateLimited":
|
||||
return True
|
||||
status = getattr(exc, "status", None)
|
||||
if status == 429:
|
||||
return True
|
||||
# Fallback duck-type: something named like a rate-limit with a
|
||||
# numeric retry_after. Covers mocked clients in tests and exotic
|
||||
# transports, without swallowing arbitrary exceptions.
|
||||
name = type(exc).__name__.lower()
|
||||
if ("ratelimit" in name or "rate_limit" in name) and getattr(exc, "retry_after", None) is not None:
|
||||
return True
|
||||
response = getattr(exc, "response", None)
|
||||
status = getattr(response, "status", None) or getattr(response, "status_code", None)
|
||||
if status == 429:
|
||||
return True
|
||||
return False
|
||||
|
||||
def _command_sync_mutation_interval_seconds(self) -> float:
|
||||
return _DISCORD_COMMAND_SYNC_MUTATION_INTERVAL_SECONDS
|
||||
|
||||
async def _sleep_between_command_sync_mutations(self) -> None:
|
||||
interval = self._command_sync_mutation_interval_seconds()
|
||||
if interval > 0:
|
||||
await asyncio.sleep(interval)
|
||||
|
||||
async def _run_post_connect_initialization(self) -> None:
|
||||
"""Finish non-critical startup work after Discord is connected."""
|
||||
if not self._client:
|
||||
@@ -840,14 +1046,46 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
logger.info("[%s] Synced %d slash command(s) via bulk tree sync", self.name, len(synced))
|
||||
return
|
||||
|
||||
# Discord's per-app command-management bucket is ~5 writes / 20 s,
|
||||
# so a mass-prune-plus-upsert reconcile (e.g. 77 orphans + 30
|
||||
# desired = 107 writes) takes several minutes of forced waits.
|
||||
# A flat 30 s budget blew up reliably under bucket pressure and
|
||||
# left slash commands broken for ~60 min until the bucket fully
|
||||
# recovered. Use a wide ceiling; the cap still guards against a
|
||||
# true hang. (#16713)
|
||||
summary = await asyncio.wait_for(self._safe_sync_slash_commands(), timeout=600)
|
||||
app_id = getattr(self._client, "application_id", None) or getattr(getattr(self._client, "user", None), "id", None)
|
||||
fingerprint = self._desired_command_sync_fingerprint()
|
||||
skip_reason = self._command_sync_skip_reason(app_id, fingerprint)
|
||||
if skip_reason:
|
||||
logger.info("[%s] Skipping Discord slash command sync: %s", self.name, skip_reason)
|
||||
return
|
||||
self._record_command_sync_attempt(app_id, fingerprint)
|
||||
|
||||
http = getattr(self._client, "http", None)
|
||||
has_ratelimit_timeout = http is not None and hasattr(http, "max_ratelimit_timeout")
|
||||
previous_ratelimit_timeout = getattr(http, "max_ratelimit_timeout", None) if has_ratelimit_timeout else None
|
||||
if has_ratelimit_timeout:
|
||||
http.max_ratelimit_timeout = _DISCORD_COMMAND_SYNC_MAX_RATE_LIMIT_SLEEP_SECONDS
|
||||
|
||||
try:
|
||||
# Discord's per-app command-management bucket is small, and
|
||||
# discord.py can otherwise sit inside one long retry sleep
|
||||
# before surfacing the 429. Keep the whole sync bounded and
|
||||
# persist Discord's retry-after when it refuses the batch.
|
||||
summary = await asyncio.wait_for(self._safe_sync_slash_commands(), timeout=600)
|
||||
except Exception as e:
|
||||
if not self._is_discord_rate_limit(e):
|
||||
raise
|
||||
retry_after = self._extract_discord_retry_after(e)
|
||||
if retry_after is None:
|
||||
# Rate-limited but no retry-after signal — back off for a
|
||||
# conservative default so we don't slam the bucket again.
|
||||
retry_after = _DISCORD_COMMAND_SYNC_MAX_RATE_LIMIT_SLEEP_SECONDS
|
||||
self._record_command_sync_rate_limit(app_id, fingerprint, retry_after)
|
||||
logger.warning(
|
||||
"[%s] Discord rate-limited slash command sync; retrying after %.0fs",
|
||||
self.name,
|
||||
retry_after,
|
||||
)
|
||||
return
|
||||
finally:
|
||||
if has_ratelimit_timeout:
|
||||
http.max_ratelimit_timeout = previous_ratelimit_timeout
|
||||
|
||||
self._record_command_sync_success(app_id, fingerprint, summary)
|
||||
logger.info(
|
||||
"[%s] Safely reconciled %d slash command(s): unchanged=%d updated=%d recreated=%d created=%d deleted=%d",
|
||||
self.name,
|
||||
@@ -1009,11 +1247,20 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
created = 0
|
||||
deleted = 0
|
||||
http = self._client.http
|
||||
mutation_count = 0
|
||||
|
||||
async def mutate(call, *args):
|
||||
nonlocal mutation_count
|
||||
if mutation_count:
|
||||
await self._sleep_between_command_sync_mutations()
|
||||
result = await call(*args)
|
||||
mutation_count += 1
|
||||
return result
|
||||
|
||||
for key, desired in desired_by_key.items():
|
||||
current = existing_by_key.pop(key, None)
|
||||
if current is None:
|
||||
await http.upsert_global_command(app_id, desired)
|
||||
await mutate(http.upsert_global_command, app_id, desired)
|
||||
created += 1
|
||||
continue
|
||||
|
||||
@@ -1025,16 +1272,16 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
continue
|
||||
|
||||
if self._patchable_app_command_payload(current_existing_payload) == self._patchable_app_command_payload(desired):
|
||||
await http.delete_global_command(app_id, current.id)
|
||||
await http.upsert_global_command(app_id, desired)
|
||||
await mutate(http.delete_global_command, app_id, current.id)
|
||||
await mutate(http.upsert_global_command, app_id, desired)
|
||||
recreated += 1
|
||||
continue
|
||||
|
||||
await http.edit_global_command(app_id, current.id, desired)
|
||||
await mutate(http.edit_global_command, app_id, current.id, desired)
|
||||
updated += 1
|
||||
|
||||
for current in existing_by_key.values():
|
||||
await http.delete_global_command(app_id, current.id)
|
||||
await mutate(http.delete_global_command, app_id, current.id)
|
||||
deleted += 1
|
||||
|
||||
return {
|
||||
@@ -1854,8 +2101,16 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
pass
|
||||
|
||||
completed = receiver.check_silence()
|
||||
# Voice inputs always originate from a specific guild
|
||||
# (guild_id is in scope). Pass it so role checks are
|
||||
# guild-scoped and not cross-guild.
|
||||
_vc_guild = self._client.get_guild(guild_id) if self._client is not None else None
|
||||
for user_id, pcm_data in completed:
|
||||
if not self._is_allowed_user(str(user_id)):
|
||||
if not self._is_allowed_user(
|
||||
str(user_id),
|
||||
guild=_vc_guild,
|
||||
is_dm=False,
|
||||
):
|
||||
continue
|
||||
await self._process_voice_input(guild_id, user_id, pcm_data)
|
||||
except asyncio.CancelledError:
|
||||
@@ -1898,13 +2153,32 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _is_allowed_user(self, user_id: str, author=None) -> bool:
|
||||
def _is_allowed_user(
|
||||
self,
|
||||
user_id: str,
|
||||
author=None,
|
||||
*,
|
||||
guild=None,
|
||||
is_dm: bool = False,
|
||||
) -> bool:
|
||||
"""Check if user is allowed via DISCORD_ALLOWED_USERS or DISCORD_ALLOWED_ROLES.
|
||||
|
||||
Uses OR semantics: if the user matches EITHER allowlist, they're allowed.
|
||||
If both allowlists are empty, everyone is allowed (backwards compatible).
|
||||
When author is a Member, checks .roles directly; otherwise falls back
|
||||
to scanning the bot's mutual guilds for a Member record.
|
||||
|
||||
Role checks are **scoped to the guild the message originated from**.
|
||||
For DMs (no guild context), role-based auth is disabled by default and
|
||||
only user-ID allowlist applies. Set ``discord.dm_role_auth_guild``
|
||||
in config.yaml to a specific guild ID to opt-in: role membership in
|
||||
that one guild will authorize DMs. This prevents cross-guild
|
||||
privilege escalation where a user with the configured role in any
|
||||
shared public server could DM the bot and pass the allowlist.
|
||||
|
||||
Args:
|
||||
user_id: Author ID as a string.
|
||||
author: Optional Member/User object for in-guild role lookup.
|
||||
guild: The guild the message arrived in (None for DMs).
|
||||
is_dm: True if the message came from a DM channel.
|
||||
"""
|
||||
# ``getattr`` fallbacks here guard against test fixtures that build
|
||||
# an adapter via ``object.__new__(DiscordAdapter)`` and skip __init__
|
||||
@@ -1915,31 +2189,54 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
has_roles = bool(allowed_roles)
|
||||
if not has_users and not has_roles:
|
||||
return True
|
||||
# Check user ID allowlist
|
||||
# Check user ID allowlist (works for both DMs and guild messages)
|
||||
if has_users and user_id in allowed_users:
|
||||
return True
|
||||
# Check role allowlist
|
||||
if has_roles:
|
||||
# Try direct role check from Member object
|
||||
direct_roles = getattr(author, "roles", None) if author is not None else None
|
||||
if direct_roles:
|
||||
if any(getattr(r, "id", None) in allowed_roles for r in direct_roles):
|
||||
return True
|
||||
# Fallback: scan mutual guilds for member's roles
|
||||
if self._client is not None:
|
||||
try:
|
||||
uid_int = int(user_id)
|
||||
except (TypeError, ValueError):
|
||||
uid_int = None
|
||||
if uid_int is not None:
|
||||
for guild in self._client.guilds:
|
||||
m = guild.get_member(uid_int)
|
||||
if m is None:
|
||||
continue
|
||||
m_roles = getattr(m, "roles", None) or []
|
||||
if any(getattr(r, "id", None) in allowed_roles for r in m_roles):
|
||||
return True
|
||||
return False
|
||||
# Role allowlist is only consulted when configured.
|
||||
if not has_roles:
|
||||
return False
|
||||
|
||||
# DM path: roles require explicit opt-in via
|
||||
# ``discord.dm_role_auth_guild`` in config.yaml. Without this, a
|
||||
# user with the configured role in ANY mutual guild could DM the
|
||||
# bot and bypass the allowlist (cross-guild leakage).
|
||||
if is_dm or guild is None:
|
||||
dm_guild_id = _read_dm_role_auth_guild()
|
||||
if dm_guild_id is None:
|
||||
return False
|
||||
if self._client is None:
|
||||
return False
|
||||
dm_guild = self._client.get_guild(dm_guild_id)
|
||||
if dm_guild is None:
|
||||
return False
|
||||
try:
|
||||
uid_int = int(user_id)
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
m = dm_guild.get_member(uid_int)
|
||||
if m is None:
|
||||
return False
|
||||
m_roles = getattr(m, "roles", None) or []
|
||||
return any(getattr(r, "id", None) in allowed_roles for r in m_roles)
|
||||
|
||||
# Guild path: role check is scoped to THIS guild only.
|
||||
# 1) Prefer the direct Member object passed in (correct guild by construction).
|
||||
direct_roles = getattr(author, "roles", None) if author is not None else None
|
||||
author_guild = getattr(author, "guild", None)
|
||||
if direct_roles and (author_guild is None or author_guild.id == guild.id):
|
||||
if any(getattr(r, "id", None) in allowed_roles for r in direct_roles):
|
||||
return True
|
||||
# 2) Fallback: resolve the Member in the message's guild only — NEVER
|
||||
# scan other mutual guilds (that is the cross-guild bypass bug).
|
||||
try:
|
||||
uid_int = int(user_id)
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
m = guild.get_member(uid_int)
|
||||
if m is None:
|
||||
return False
|
||||
m_roles = getattr(m, "roles", None) or []
|
||||
return any(getattr(r, "id", None) in allowed_roles for r in m_roles)
|
||||
|
||||
# ── Slash command authorization ─────────────────────────────────────
|
||||
# Slash commands (``_run_simple_slash`` and ``_handle_thread_create_slash``)
|
||||
@@ -2036,7 +2333,16 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
return (True, None)
|
||||
|
||||
user_id = str(user.id)
|
||||
if not self._is_allowed_user(user_id, author=user):
|
||||
# Pass guild + is_dm so role check is scoped to the originating
|
||||
# guild and cross-guild DM bypass (#12136) can't land via the
|
||||
# slash surface either.
|
||||
interaction_guild = getattr(interaction, "guild", None)
|
||||
if not self._is_allowed_user(
|
||||
user_id,
|
||||
author=user,
|
||||
guild=interaction_guild,
|
||||
is_dm=in_dm,
|
||||
):
|
||||
return (
|
||||
False,
|
||||
"user not in DISCORD_ALLOWED_USERS / DISCORD_ALLOWED_ROLES",
|
||||
|
||||
@@ -4591,12 +4591,12 @@ def _poll_registration(
|
||||
Returns dict with app_id, app_secret, domain, open_id on success.
|
||||
Returns None on failure.
|
||||
"""
|
||||
deadline = time.time() + expire_in
|
||||
deadline = time.monotonic() + expire_in
|
||||
current_domain = domain
|
||||
domain_switched = False
|
||||
poll_count = 0
|
||||
|
||||
while time.time() < deadline:
|
||||
while time.monotonic() < deadline:
|
||||
base_url = _accounts_base_url(current_domain)
|
||||
try:
|
||||
res = _post_registration(base_url, {
|
||||
|
||||
+87
-12
@@ -17,7 +17,8 @@ Environment variables:
|
||||
MATRIX_REACTIONS Set "false" to disable processing lifecycle reactions
|
||||
(eyes/checkmark/cross). Default: true
|
||||
MATRIX_REQUIRE_MENTION Require @mention in rooms (default: true)
|
||||
MATRIX_FREE_RESPONSE_ROOMS Comma-separated room IDs exempt from mention requirement
|
||||
MATRIX_FREE_RESPONSE_ROOMS Comma-separated room IDs exempt from mention requirement (alias of matrix.free_response_rooms)
|
||||
MATRIX_ALLOWED_ROOMS Comma-separated room IDs; if set, bot ONLY responds in these rooms (whitelist, DMs exempt; alias of matrix.allowed_rooms)
|
||||
MATRIX_AUTO_THREAD Auto-create threads for room messages (default: true)
|
||||
MATRIX_DM_AUTO_THREAD Auto-create threads for DM messages (default: false)
|
||||
MATRIX_RECOVERY_KEY Recovery key for cross-signing verification after device key rotation
|
||||
@@ -343,10 +344,29 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
self._require_mention: bool = os.getenv(
|
||||
"MATRIX_REQUIRE_MENTION", "true"
|
||||
).lower() not in ("false", "0", "no")
|
||||
free_rooms_raw = os.getenv("MATRIX_FREE_RESPONSE_ROOMS", "")
|
||||
self._free_rooms: Set[str] = {
|
||||
r.strip() for r in free_rooms_raw.split(",") if r.strip()
|
||||
}
|
||||
free_rooms_raw = config.extra.get("free_response_rooms")
|
||||
if free_rooms_raw is None:
|
||||
free_rooms_raw = os.getenv("MATRIX_FREE_RESPONSE_ROOMS", "")
|
||||
if isinstance(free_rooms_raw, list):
|
||||
self._free_rooms: Set[str] = {
|
||||
str(r).strip() for r in free_rooms_raw if str(r).strip()
|
||||
}
|
||||
else:
|
||||
self._free_rooms: Set[str] = {
|
||||
r.strip() for r in str(free_rooms_raw).split(",") if r.strip()
|
||||
}
|
||||
# If non-empty, bot ONLY responds in these rooms (whitelist); DMs exempt.
|
||||
allowed_rooms_raw = config.extra.get("allowed_rooms")
|
||||
if allowed_rooms_raw is None:
|
||||
allowed_rooms_raw = os.getenv("MATRIX_ALLOWED_ROOMS", "")
|
||||
if isinstance(allowed_rooms_raw, list):
|
||||
self._allowed_rooms: Set[str] = {
|
||||
str(r).strip() for r in allowed_rooms_raw if str(r).strip()
|
||||
}
|
||||
else:
|
||||
self._allowed_rooms: Set[str] = {
|
||||
r.strip() for r in str(allowed_rooms_raw).split(",") if r.strip()
|
||||
}
|
||||
self._auto_thread: bool = os.getenv("MATRIX_AUTO_THREAD", "true").lower() in (
|
||||
"true",
|
||||
"1",
|
||||
@@ -364,6 +384,12 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
"MATRIX_REACTIONS", "true"
|
||||
).lower() not in ("false", "0", "no")
|
||||
self._pending_reactions: dict[tuple[str, str], str] = {}
|
||||
# Delay before redacting reactions so Matrix homeservers have time to
|
||||
# deliver the final message event without tripping "missing event"
|
||||
# errors in some clients. 5s is empirically safe; not user-tunable —
|
||||
# if that changes, add a config.yaml entry rather than an env var.
|
||||
self._reaction_redaction_delay_seconds = 5.0
|
||||
self._reaction_redaction_tasks: Set[asyncio.Task] = set()
|
||||
|
||||
# Proxy support — resolve once at init, reuse for all HTTP traffic.
|
||||
self._proxy_url: str | None = resolve_proxy_url(platform_env_var="MATRIX_PROXY")
|
||||
@@ -851,6 +877,14 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
except (asyncio.CancelledError, Exception):
|
||||
pass
|
||||
|
||||
redaction_tasks = list(self._reaction_redaction_tasks)
|
||||
for task in redaction_tasks:
|
||||
if not task.done():
|
||||
task.cancel()
|
||||
if redaction_tasks:
|
||||
await asyncio.gather(*redaction_tasks, return_exceptions=True)
|
||||
self._reaction_redaction_tasks.clear()
|
||||
|
||||
# Close the SQLite crypto store database.
|
||||
if hasattr(self, "_crypto_db") and self._crypto_db:
|
||||
try:
|
||||
@@ -1559,6 +1593,18 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
|
||||
# Require-mention gating.
|
||||
if not is_dm:
|
||||
# allowed_rooms check (whitelist — must pass before other gating).
|
||||
# When set, messages from rooms NOT in this whitelist are silently
|
||||
# ignored, even if @mentioned. DMs are already excluded above.
|
||||
if self._allowed_rooms and room_id not in self._allowed_rooms:
|
||||
logger.debug(
|
||||
"Matrix: ignoring message %s in %s — room not in "
|
||||
"MATRIX_ALLOWED_ROOMS whitelist",
|
||||
event_id,
|
||||
room_id,
|
||||
)
|
||||
return None
|
||||
|
||||
is_free_room = room_id in self._free_rooms
|
||||
in_bot_thread = bool(thread_id and thread_id in self._threads)
|
||||
if self._require_mention and not is_free_room and not in_bot_thread:
|
||||
@@ -1929,6 +1975,35 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
"""Remove a reaction by redacting its event."""
|
||||
return await self.redact_message(room_id, reaction_event_id, reason)
|
||||
|
||||
def _schedule_reaction_redaction(
|
||||
self,
|
||||
room_id: str,
|
||||
reaction_event_id: str,
|
||||
reason: str = "",
|
||||
) -> None:
|
||||
"""Redact a reaction after a short delay so message delivery settles."""
|
||||
|
||||
async def _redact_later() -> None:
|
||||
try:
|
||||
if self._reaction_redaction_delay_seconds:
|
||||
await asyncio.sleep(self._reaction_redaction_delay_seconds)
|
||||
if not await self._redact_reaction(room_id, reaction_event_id, reason):
|
||||
logger.debug(
|
||||
"Matrix: failed to redact reaction %s", reaction_event_id
|
||||
)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as exc:
|
||||
logger.debug(
|
||||
"Matrix: delayed reaction redaction failed for %s: %s",
|
||||
reaction_event_id,
|
||||
exc,
|
||||
)
|
||||
|
||||
task = asyncio.create_task(_redact_later())
|
||||
self._reaction_redaction_tasks.add(task)
|
||||
task.add_done_callback(self._reaction_redaction_tasks.discard)
|
||||
|
||||
async def on_processing_start(self, event: MessageEvent) -> None:
|
||||
"""Add eyes reaction when the agent starts processing a message."""
|
||||
if not self._reactions_enabled:
|
||||
@@ -1957,8 +2032,11 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
reaction_key = (room_id, msg_id)
|
||||
if reaction_key in self._pending_reactions:
|
||||
eyes_event_id = self._pending_reactions.pop(reaction_key)
|
||||
if not await self._redact_reaction(room_id, eyes_event_id):
|
||||
logger.debug("Matrix: failed to redact eyes reaction %s", eyes_event_id)
|
||||
self._schedule_reaction_redaction(
|
||||
room_id,
|
||||
eyes_event_id,
|
||||
"processing complete",
|
||||
)
|
||||
await self._send_reaction(
|
||||
room_id,
|
||||
msg_id,
|
||||
@@ -2037,11 +2115,8 @@ class MatrixAdapter(BasePlatformAdapter):
|
||||
) -> None:
|
||||
"""Redact the bot's seed ✅/❎ reactions, leaving only the user's reaction."""
|
||||
for emoji, evt_id in prompt.bot_reaction_events.items():
|
||||
try:
|
||||
await self.redact_message(room_id, evt_id, "approval resolved")
|
||||
logger.debug("Matrix: redacted bot reaction %s (%s)", emoji, evt_id)
|
||||
except Exception as exc:
|
||||
logger.debug("Matrix: failed to redact bot reaction %s: %s", emoji, exc)
|
||||
self._schedule_reaction_redaction(room_id, evt_id, "approval resolved")
|
||||
logger.debug("Matrix: scheduled bot reaction redaction %s (%s)", emoji, evt_id)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Text message aggregation (handles Matrix client-side splits)
|
||||
|
||||
@@ -706,10 +706,30 @@ class MattermostAdapter(BasePlatformAdapter):
|
||||
message_text = post.get("message", "")
|
||||
|
||||
# Mention-gating for non-DM channels.
|
||||
# Config (env vars):
|
||||
# MATTERMOST_REQUIRE_MENTION: Require @mention in channels (default: true)
|
||||
# MATTERMOST_FREE_RESPONSE_CHANNELS: Channel IDs where bot responds without mention
|
||||
# Config (config.yaml `mattermost.*` with env-var fallback):
|
||||
# require_mention / MATTERMOST_REQUIRE_MENTION: Require @mention in channels (default: true)
|
||||
# free_response_channels / MATTERMOST_FREE_RESPONSE_CHANNELS: Channel IDs where bot responds without mention
|
||||
# allowed_channels / MATTERMOST_ALLOWED_CHANNELS: If set, bot ONLY responds in these channels (whitelist)
|
||||
if channel_type_raw != "D":
|
||||
# allowed_channels check (whitelist — must pass before other gating).
|
||||
# When set, messages from channels NOT in this list are silently
|
||||
# ignored, even if @mentioned. DMs are already excluded above.
|
||||
allowed_raw = self.config.extra.get("allowed_channels") if self.config.extra else None
|
||||
if allowed_raw is None:
|
||||
allowed_raw = os.getenv("MATTERMOST_ALLOWED_CHANNELS", "")
|
||||
if isinstance(allowed_raw, list):
|
||||
allowed_channels = {str(c).strip() for c in allowed_raw if str(c).strip()}
|
||||
else:
|
||||
allowed_channels = {
|
||||
c.strip() for c in str(allowed_raw).split(",") if c.strip()
|
||||
}
|
||||
if allowed_channels and channel_id not in allowed_channels:
|
||||
logger.debug(
|
||||
"Mattermost: ignoring message in non-allowed channel: %s",
|
||||
channel_id,
|
||||
)
|
||||
return
|
||||
|
||||
require_mention = os.getenv(
|
||||
"MATTERMOST_REQUIRE_MENTION", "true"
|
||||
).lower() not in ("false", "0", "no")
|
||||
|
||||
@@ -34,6 +34,27 @@ from .crypto import decrypt_secret, generate_bind_key # noqa: F401
|
||||
# -- Utils -----------------------------------------------------------------
|
||||
from .utils import build_user_agent, get_api_headers, coerce_list # noqa: F401
|
||||
|
||||
# -- Chunked upload --------------------------------------------------------
|
||||
from .chunked_upload import ( # noqa: F401
|
||||
ChunkedUploader,
|
||||
UploadDailyLimitExceededError,
|
||||
UploadFileTooLargeError,
|
||||
)
|
||||
|
||||
# -- Inline keyboards ------------------------------------------------------
|
||||
from .keyboards import ( # noqa: F401
|
||||
ApprovalRequest,
|
||||
ApprovalSender,
|
||||
InlineKeyboard,
|
||||
InteractionEvent,
|
||||
build_approval_keyboard,
|
||||
build_approval_text,
|
||||
build_update_prompt_keyboard,
|
||||
parse_approval_button_data,
|
||||
parse_interaction_event,
|
||||
parse_update_prompt_button_data,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
# adapter
|
||||
"QQAdapter",
|
||||
@@ -52,4 +73,19 @@ __all__ = [
|
||||
"build_user_agent",
|
||||
"get_api_headers",
|
||||
"coerce_list",
|
||||
# chunked upload
|
||||
"ChunkedUploader",
|
||||
"UploadDailyLimitExceededError",
|
||||
"UploadFileTooLargeError",
|
||||
# keyboards
|
||||
"ApprovalRequest",
|
||||
"ApprovalSender",
|
||||
"InlineKeyboard",
|
||||
"InteractionEvent",
|
||||
"build_approval_keyboard",
|
||||
"build_approval_text",
|
||||
"build_update_prompt_keyboard",
|
||||
"parse_approval_button_data",
|
||||
"parse_interaction_event",
|
||||
"parse_update_prompt_button_data",
|
||||
]
|
||||
|
||||
@@ -41,7 +41,7 @@ import time
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
from typing import Any, Awaitable, Callable, Dict, List, Optional, Tuple
|
||||
from urllib.parse import urlparse
|
||||
|
||||
try:
|
||||
@@ -119,6 +119,22 @@ from gateway.platforms.qqbot.utils import (
|
||||
coerce_list as _coerce_list_impl,
|
||||
build_user_agent,
|
||||
)
|
||||
from gateway.platforms.qqbot.chunked_upload import (
|
||||
ChunkedUploader,
|
||||
UploadDailyLimitExceededError,
|
||||
UploadFileTooLargeError,
|
||||
)
|
||||
from gateway.platforms.qqbot.keyboards import (
|
||||
ApprovalRequest,
|
||||
ApprovalSender,
|
||||
InlineKeyboard,
|
||||
InteractionEvent,
|
||||
build_approval_keyboard,
|
||||
build_update_prompt_keyboard,
|
||||
parse_approval_button_data,
|
||||
parse_interaction_event,
|
||||
parse_update_prompt_button_data,
|
||||
)
|
||||
|
||||
|
||||
def check_qq_requirements() -> bool:
|
||||
@@ -208,6 +224,22 @@ class QQAdapter(BasePlatformAdapter):
|
||||
# Upload cache: content_hash -> {file_info, file_uuid, expires_at}
|
||||
self._upload_cache: Dict[str, Dict[str, Any]] = {}
|
||||
|
||||
# Inline-keyboard interaction routing. The callback (if set) is invoked
|
||||
# for every INTERACTION_CREATE event after the adapter has already
|
||||
# ACKed it. Callers (gateway wiring for approvals / update prompts)
|
||||
# register via set_interaction_callback().
|
||||
self._interaction_callback: Optional[
|
||||
Callable[[InteractionEvent], Awaitable[None]]
|
||||
] = None
|
||||
|
||||
# Default interaction dispatcher: routes approval-button clicks to
|
||||
# tools.approval.resolve_gateway_approval() and update-prompt clicks
|
||||
# to ~/.hermes/.update_response. Set here so the cross-adapter gateway
|
||||
# contract (send_exec_approval / send_update_prompt) works out of the
|
||||
# box; callers can override with set_interaction_callback(None) or
|
||||
# register a custom handler.
|
||||
self._interaction_callback = self._default_interaction_dispatch
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Properties
|
||||
# ------------------------------------------------------------------
|
||||
@@ -759,6 +791,8 @@ class QQAdapter(BasePlatformAdapter):
|
||||
"GUILD_AT_MESSAGE_CREATE",
|
||||
):
|
||||
asyncio.create_task(self._on_message(t, d))
|
||||
elif t == "INTERACTION_CREATE":
|
||||
self._create_task(self._on_interaction(d))
|
||||
else:
|
||||
logger.debug("[%s] Unhandled dispatch: %s", self._log_tag, t)
|
||||
return
|
||||
@@ -832,6 +866,206 @@ class QQAdapter(BasePlatformAdapter):
|
||||
elif event_type == "DIRECT_MESSAGE_CREATE":
|
||||
await self._handle_dm_message(d, msg_id, content, author, timestamp)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Inline-keyboard interactions (INTERACTION_CREATE)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def set_interaction_callback(
|
||||
self,
|
||||
callback: Optional[Callable[[InteractionEvent], Awaitable[None]]],
|
||||
) -> None:
|
||||
"""Register (or clear) the interaction callback.
|
||||
|
||||
Invoked once per ``INTERACTION_CREATE`` event *after* the adapter has
|
||||
ACKed the interaction. The callback is responsible for routing the
|
||||
button click to the right subsystem (approval resolver, update-prompt
|
||||
resolver, etc.) based on the ``button_data`` payload.
|
||||
"""
|
||||
self._interaction_callback = callback
|
||||
|
||||
async def _on_interaction(self, d: Any) -> None:
|
||||
"""Handle an ``INTERACTION_CREATE`` event.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
1. Parse the raw payload into an :class:`InteractionEvent`.
|
||||
2. ACK the interaction (``PUT /interactions/{id}``) so the client
|
||||
stops showing a loading indicator on the button.
|
||||
3. Dispatch to the registered interaction callback, if any.
|
||||
"""
|
||||
if not isinstance(d, dict):
|
||||
return
|
||||
try:
|
||||
event = parse_interaction_event(d)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"[%s] Failed to parse INTERACTION_CREATE: %s", self._log_tag, exc
|
||||
)
|
||||
return
|
||||
|
||||
if not event.id:
|
||||
logger.warning(
|
||||
"[%s] INTERACTION_CREATE missing id, skipping ACK", self._log_tag
|
||||
)
|
||||
return
|
||||
|
||||
# ACK the interaction promptly — per the QQ docs the client will show
|
||||
# an error icon on the button if we don't respond quickly.
|
||||
try:
|
||||
await self._acknowledge_interaction(event.id)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"[%s] Failed to ACK interaction %s: %s",
|
||||
self._log_tag, event.id, exc,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"[%s] Interaction: scene=%s button_data=%r operator=%s",
|
||||
self._log_tag, event.scene, event.button_data, event.operator_openid,
|
||||
)
|
||||
|
||||
callback = self._interaction_callback
|
||||
if callback is None:
|
||||
logger.debug(
|
||||
"[%s] No interaction callback registered; dropping button "
|
||||
"click %r",
|
||||
self._log_tag, event.button_data,
|
||||
)
|
||||
return
|
||||
try:
|
||||
await callback(event)
|
||||
except Exception as exc:
|
||||
logger.error(
|
||||
"[%s] Interaction callback raised: %s",
|
||||
self._log_tag, exc, exc_info=True,
|
||||
)
|
||||
|
||||
async def _acknowledge_interaction(
|
||||
self,
|
||||
interaction_id: str,
|
||||
code: int = 0,
|
||||
) -> None:
|
||||
"""ACK a button interaction via ``PUT /interactions/{id}``.
|
||||
|
||||
:param interaction_id: The ``id`` field from the
|
||||
``INTERACTION_CREATE`` event.
|
||||
:param code: Response code (``0`` = success).
|
||||
"""
|
||||
if not self._http_client:
|
||||
raise RuntimeError("HTTP client not initialized — not connected?")
|
||||
token = await self._ensure_token()
|
||||
headers = {
|
||||
"Authorization": f"QQBot {token}",
|
||||
"Content-Type": "application/json",
|
||||
"User-Agent": build_user_agent(),
|
||||
}
|
||||
resp = await self._http_client.put(
|
||||
f"{API_BASE}/interactions/{interaction_id}",
|
||||
headers=headers,
|
||||
json={"code": code},
|
||||
timeout=DEFAULT_API_TIMEOUT,
|
||||
)
|
||||
if resp.status_code >= 400:
|
||||
raise RuntimeError(
|
||||
f"Interaction ACK failed [{resp.status_code}]: "
|
||||
f"{resp.text[:200]}"
|
||||
)
|
||||
|
||||
# Mapping from QQ keyboard button decisions → the ``choice`` vocabulary
|
||||
# accepted by ``tools.approval.resolve_gateway_approval``. QQ's 3-button
|
||||
# layout (mobile-space constraint) collapses "session" and "always" into
|
||||
# a single "always" button; users wanting session-only approval can fall
|
||||
# back to the ``/approve session`` text command.
|
||||
_APPROVAL_BUTTON_TO_CHOICE = {
|
||||
"allow-once": "once",
|
||||
"allow-always": "always",
|
||||
"deny": "deny",
|
||||
}
|
||||
|
||||
async def _default_interaction_dispatch(
|
||||
self,
|
||||
event: InteractionEvent,
|
||||
) -> None:
|
||||
"""Route ``INTERACTION_CREATE`` button clicks to the right subsystem.
|
||||
|
||||
- ``approve:<session_key>:<decision>`` →
|
||||
:func:`tools.approval.resolve_gateway_approval`
|
||||
(unblocks the agent thread waiting on a dangerous-command approval).
|
||||
- ``update_prompt:<answer>`` →
|
||||
writes the answer to ``~/.hermes/.update_response`` for the
|
||||
detached ``hermes update --gateway`` process to consume.
|
||||
- Anything else is logged at DEBUG and ignored.
|
||||
|
||||
Installed as the adapter's default interaction callback in
|
||||
``__init__``. Callers can replace via
|
||||
:meth:`set_interaction_callback` to route clicks elsewhere (or pass
|
||||
``None`` to drop them entirely).
|
||||
"""
|
||||
button_data = event.button_data
|
||||
if not button_data:
|
||||
return
|
||||
|
||||
approval = parse_approval_button_data(button_data)
|
||||
if approval is not None:
|
||||
session_key, decision = approval
|
||||
choice = self._APPROVAL_BUTTON_TO_CHOICE.get(decision)
|
||||
if choice is None:
|
||||
logger.warning(
|
||||
"[%s] Unknown approval decision %r (session=%s)",
|
||||
self._log_tag, decision, session_key,
|
||||
)
|
||||
return
|
||||
try:
|
||||
# Import lazily to keep the adapter importable in tests that
|
||||
# don't exercise the approval subsystem.
|
||||
from tools.approval import resolve_gateway_approval
|
||||
count = resolve_gateway_approval(session_key, choice)
|
||||
logger.info(
|
||||
"[%s] Button resolved %d approval(s) for session %s "
|
||||
"(choice=%s, operator=%s)",
|
||||
self._log_tag, count, session_key, choice,
|
||||
event.operator_openid,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error(
|
||||
"[%s] resolve_gateway_approval failed for session %s: %s",
|
||||
self._log_tag, session_key, exc,
|
||||
)
|
||||
return
|
||||
|
||||
update_answer = parse_update_prompt_button_data(button_data)
|
||||
if update_answer is not None:
|
||||
self._write_update_response(update_answer, event.operator_openid)
|
||||
return
|
||||
|
||||
logger.debug(
|
||||
"[%s] Unrecognised button_data %r from interaction %s",
|
||||
self._log_tag, button_data, event.id,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _write_update_response(answer: str, operator: str = "") -> None:
|
||||
"""Atomically write the update-prompt answer to ``.update_response``.
|
||||
|
||||
Mirrors the Discord / Telegram / Feishu adapters: the detached
|
||||
``hermes update --gateway`` watcher polls this file for a ``y``/``n``
|
||||
response to its interactive prompts (stash-restore, config migration).
|
||||
Writes via ``tmp + rename`` so a partial write can't fool the reader.
|
||||
"""
|
||||
try:
|
||||
from hermes_constants import get_hermes_home
|
||||
home = get_hermes_home()
|
||||
response_path = home / ".update_response"
|
||||
tmp = response_path.with_suffix(".tmp")
|
||||
tmp.write_text(answer)
|
||||
tmp.replace(response_path)
|
||||
logger.info(
|
||||
"QQ update prompt answered %r by %s",
|
||||
answer, operator or "(unknown)",
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error("Failed to write update response: %s", exc)
|
||||
|
||||
async def _handle_c2c_message(
|
||||
self,
|
||||
d: Dict[str, Any],
|
||||
@@ -900,6 +1134,13 @@ class QQAdapter(BasePlatformAdapter):
|
||||
len(voice_transcripts),
|
||||
)
|
||||
|
||||
# Merge any quoted-message context (message_type=103 → msg_elements[0]).
|
||||
quoted = await self._process_quoted_context(d)
|
||||
text = self._merge_quote_into(text, quoted["quote_block"])
|
||||
if quoted["image_urls"]:
|
||||
image_urls = image_urls + quoted["image_urls"]
|
||||
image_media_types = image_media_types + quoted["image_media_types"]
|
||||
|
||||
if not text.strip() and not image_urls:
|
||||
return
|
||||
|
||||
@@ -958,6 +1199,13 @@ class QQAdapter(BasePlatformAdapter):
|
||||
else attachment_info
|
||||
)
|
||||
|
||||
# Merge any quoted-message context (message_type=103 → msg_elements[0]).
|
||||
quoted = await self._process_quoted_context(d)
|
||||
text = self._merge_quote_into(text, quoted["quote_block"])
|
||||
if quoted["image_urls"]:
|
||||
image_urls = image_urls + quoted["image_urls"]
|
||||
image_media_types = image_media_types + quoted["image_media_types"]
|
||||
|
||||
if not text.strip() and not image_urls:
|
||||
return
|
||||
|
||||
@@ -1025,6 +1273,13 @@ class QQAdapter(BasePlatformAdapter):
|
||||
else attachment_info
|
||||
)
|
||||
|
||||
# Merge any quoted-message context (message_type=103 → msg_elements[0]).
|
||||
quoted = await self._process_quoted_context(d)
|
||||
text = self._merge_quote_into(text, quoted["quote_block"])
|
||||
if quoted["image_urls"]:
|
||||
image_urls = image_urls + quoted["image_urls"]
|
||||
image_media_types = image_media_types + quoted["image_media_types"]
|
||||
|
||||
if not text.strip() and not image_urls:
|
||||
return
|
||||
|
||||
@@ -1089,6 +1344,13 @@ class QQAdapter(BasePlatformAdapter):
|
||||
else attachment_info
|
||||
)
|
||||
|
||||
# Merge any quoted-message context (message_type=103 → msg_elements[0]).
|
||||
quoted = await self._process_quoted_context(d)
|
||||
text = self._merge_quote_into(text, quoted["quote_block"])
|
||||
if quoted["image_urls"]:
|
||||
image_urls = image_urls + quoted["image_urls"]
|
||||
image_media_types = image_media_types + quoted["image_media_types"]
|
||||
|
||||
if not text.strip() and not image_urls:
|
||||
return
|
||||
|
||||
@@ -1109,6 +1371,113 @@ class QQAdapter(BasePlatformAdapter):
|
||||
)
|
||||
await self.handle_message(event)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Quoted-message handling
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _process_quoted_context(
|
||||
self,
|
||||
d: Dict[str, Any],
|
||||
) -> Dict[str, Any]:
|
||||
"""Process the quoted message a user is replying to.
|
||||
|
||||
When a user replies while quoting another message, the platform sets
|
||||
``message_type = 103`` and pushes the referenced message's content and
|
||||
attachments inside ``msg_elements[0]``. The old adapter ignored
|
||||
``msg_elements`` entirely, so:
|
||||
|
||||
- Quoted text was surfaced only when the user typed something of
|
||||
their own — bare quote-replies showed nothing.
|
||||
- Quoted attachments (images, voice, files) were never downloaded
|
||||
or described.
|
||||
- Quoted voice messages specifically produced no transcript, so the
|
||||
LLM had no way to see what the user was referring to.
|
||||
|
||||
This method parses ``msg_elements`` and runs the quoted attachments
|
||||
through the same :meth:`_process_attachments` pipeline as the main
|
||||
message body, so quoted voice messages get STT transcripts and
|
||||
quoted images are cached identically.
|
||||
|
||||
:param d: Raw inbound message dict (from the WS dispatch payload).
|
||||
:returns: Dict with keys:
|
||||
|
||||
- ``quote_block``: string to prepend to the user's text body
|
||||
(empty when there's nothing quoted).
|
||||
- ``image_urls``: list of cached quoted-image paths.
|
||||
- ``image_media_types``: parallel list of image MIME types.
|
||||
"""
|
||||
empty = {
|
||||
"quote_block": "",
|
||||
"image_urls": [],
|
||||
"image_media_types": [],
|
||||
}
|
||||
# Short-circuit: only message_type 103 indicates a quote.
|
||||
try:
|
||||
if int(d.get("message_type", 0) or 0) != 103:
|
||||
return empty
|
||||
except (TypeError, ValueError):
|
||||
return empty
|
||||
|
||||
elements = d.get("msg_elements")
|
||||
if not isinstance(elements, list) or not elements:
|
||||
return empty
|
||||
|
||||
# msg_elements[0] carries the referenced message. Additional elements
|
||||
# (if any) are very rare in practice; we concatenate their text and
|
||||
# union their attachments for completeness.
|
||||
quoted_text_parts: List[str] = []
|
||||
all_attachments: List[Dict[str, Any]] = []
|
||||
for elem in elements:
|
||||
if not isinstance(elem, dict):
|
||||
continue
|
||||
etext = str(elem.get("content", "")).strip()
|
||||
if etext:
|
||||
quoted_text_parts.append(etext)
|
||||
eatts = elem.get("attachments")
|
||||
if isinstance(eatts, list):
|
||||
for a in eatts:
|
||||
if isinstance(a, dict):
|
||||
all_attachments.append(a)
|
||||
|
||||
att_result = await self._process_attachments(all_attachments)
|
||||
quoted_voice = att_result.get("voice_transcripts") or []
|
||||
quoted_info = att_result.get("attachment_info") or ""
|
||||
quoted_images = att_result.get("image_urls") or []
|
||||
quoted_image_types = att_result.get("image_media_types") or []
|
||||
|
||||
lines: List[str] = []
|
||||
if quoted_text_parts:
|
||||
lines.append(" ".join(quoted_text_parts))
|
||||
for t in quoted_voice:
|
||||
lines.append(t)
|
||||
if quoted_info:
|
||||
lines.append(quoted_info)
|
||||
|
||||
if not lines and not quoted_images:
|
||||
return empty
|
||||
|
||||
if lines:
|
||||
quote_block = "[Quoted message]:\n" + "\n".join(lines)
|
||||
else:
|
||||
# Images-only quote: give the LLM at least a marker so it knows
|
||||
# context was referenced.
|
||||
quote_block = "[Quoted message]: (image)"
|
||||
|
||||
return {
|
||||
"quote_block": quote_block,
|
||||
"image_urls": quoted_images,
|
||||
"image_media_types": quoted_image_types,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _merge_quote_into(text: str, quote_block: str) -> str:
|
||||
"""Prepend ``quote_block`` to *text*, separated by a blank line."""
|
||||
if not quote_block:
|
||||
return text
|
||||
if text.strip():
|
||||
return f"{quote_block}\n\n{text}".strip()
|
||||
return quote_block
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Attachment processing
|
||||
# ------------------------------------------------------------------
|
||||
@@ -1992,26 +2361,44 @@ class QQAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error=error_msg, retryable=retryable)
|
||||
|
||||
async def _send_c2c_text(
|
||||
self, openid: str, content: str, reply_to: Optional[str] = None
|
||||
self,
|
||||
openid: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
keyboard: Optional[InlineKeyboard] = None,
|
||||
) -> SendResult:
|
||||
"""Send text to a C2C user via REST API."""
|
||||
"""Send text to a C2C user via REST API.
|
||||
|
||||
:param keyboard: Optional inline keyboard attached to the message.
|
||||
"""
|
||||
self._next_msg_seq(reply_to or openid)
|
||||
body = self._build_text_body(content, reply_to)
|
||||
if reply_to:
|
||||
body["msg_id"] = reply_to
|
||||
if keyboard is not None:
|
||||
body["keyboard"] = keyboard.to_dict()
|
||||
|
||||
data = await self._api_request("POST", f"/v2/users/{openid}/messages", body)
|
||||
msg_id = str(data.get("id", uuid.uuid4().hex[:12]))
|
||||
return SendResult(success=True, message_id=msg_id, raw_response=data)
|
||||
|
||||
async def _send_group_text(
|
||||
self, group_openid: str, content: str, reply_to: Optional[str] = None
|
||||
self,
|
||||
group_openid: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
keyboard: Optional[InlineKeyboard] = None,
|
||||
) -> SendResult:
|
||||
"""Send text to a group via REST API."""
|
||||
"""Send text to a group via REST API.
|
||||
|
||||
:param keyboard: Optional inline keyboard attached to the message.
|
||||
"""
|
||||
self._next_msg_seq(reply_to or group_openid)
|
||||
body = self._build_text_body(content, reply_to)
|
||||
if reply_to:
|
||||
body["msg_id"] = reply_to
|
||||
if keyboard is not None:
|
||||
body["keyboard"] = keyboard.to_dict()
|
||||
|
||||
data = await self._api_request(
|
||||
"POST", f"/v2/groups/{group_openid}/messages", body
|
||||
@@ -2031,6 +2418,156 @@ class QQAdapter(BasePlatformAdapter):
|
||||
msg_id = str(data.get("id", uuid.uuid4().hex[:12]))
|
||||
return SendResult(success=True, message_id=msg_id, raw_response=data)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Inline-keyboard outbound helpers (approval / update-prompt flows)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def send_with_keyboard(
|
||||
self,
|
||||
chat_id: str,
|
||||
content: str,
|
||||
keyboard: InlineKeyboard,
|
||||
reply_to: Optional[str] = None,
|
||||
) -> SendResult:
|
||||
"""Send a single text message with an inline keyboard attached.
|
||||
|
||||
Unlike :meth:`send`, this does NOT split long content into chunks —
|
||||
a keyboard message has exactly one interactive surface, and splitting
|
||||
would orphan the buttons from the first chunk. Callers should keep
|
||||
approval/update-prompt bodies short.
|
||||
|
||||
Guild (channel) chats don't support inline keyboards; returns a
|
||||
non-retryable failure for those.
|
||||
"""
|
||||
if not self.is_connected:
|
||||
if not await self._wait_for_reconnection():
|
||||
return SendResult(
|
||||
success=False, error="Not connected", retryable=True
|
||||
)
|
||||
|
||||
chat_type = self._guess_chat_type(chat_id)
|
||||
formatted = self.format_message(content)
|
||||
truncated = formatted[: self.MAX_MESSAGE_LENGTH]
|
||||
try:
|
||||
if chat_type == "c2c":
|
||||
return await self._send_c2c_text(
|
||||
chat_id, truncated, reply_to, keyboard=keyboard,
|
||||
)
|
||||
if chat_type == "group":
|
||||
return await self._send_group_text(
|
||||
chat_id, truncated, reply_to, keyboard=keyboard,
|
||||
)
|
||||
return SendResult(
|
||||
success=False,
|
||||
error=(
|
||||
f"Inline keyboards not supported for chat_type "
|
||||
f"{chat_type!r}"
|
||||
),
|
||||
retryable=False,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error(
|
||||
"[%s] send_with_keyboard failed: %s", self._log_tag, exc
|
||||
)
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
async def send_approval_request(
|
||||
self,
|
||||
chat_id: str,
|
||||
req: ApprovalRequest,
|
||||
reply_to: Optional[str] = None,
|
||||
) -> SendResult:
|
||||
"""Send a 3-button approval request (``allow-once / allow-always / deny``).
|
||||
|
||||
The rendered text comes from :func:`build_approval_text`; callers can
|
||||
override by passing a custom :class:`ApprovalRequest`.
|
||||
|
||||
Users click the button → ``INTERACTION_CREATE`` fires → the adapter's
|
||||
registered :meth:`set_interaction_callback` handler decodes
|
||||
``button_data`` via :func:`parse_approval_button_data`.
|
||||
"""
|
||||
from gateway.platforms.qqbot.keyboards import build_approval_text
|
||||
return await self.send_with_keyboard(
|
||||
chat_id,
|
||||
build_approval_text(req),
|
||||
build_approval_keyboard(req.session_key),
|
||||
reply_to=reply_to,
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Cross-adapter gateway contract — send_exec_approval + send_update_prompt
|
||||
# ------------------------------------------------------------------
|
||||
#
|
||||
# These mirror the signatures that gateway/run.py detects on the adapter
|
||||
# class (e.g. type(adapter).send_exec_approval, type(adapter).send_update_prompt)
|
||||
# for button-based approval / update-confirm UX. Discord, Telegram, Slack,
|
||||
# Matrix, and Feishu already implement the same contract.
|
||||
|
||||
async def send_exec_approval(
|
||||
self,
|
||||
chat_id: str,
|
||||
command: str,
|
||||
session_key: str,
|
||||
description: str = "dangerous command",
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
"""Send a button-based exec-approval prompt for a dangerous command.
|
||||
|
||||
Called by ``gateway/run.py``'s ``_approval_notify_sync`` when the
|
||||
agent is blocked waiting for approval. Button clicks resolve via
|
||||
:func:`tools.approval.resolve_gateway_approval` — dispatched by the
|
||||
adapter's interaction callback (:meth:`_default_interaction_dispatch`).
|
||||
"""
|
||||
del metadata # QQ doesn't have thread_id / DM targeting overrides.
|
||||
|
||||
# Use the reply-to message for passive-message context when we have one.
|
||||
# QQ requires a msg_id on outbound messages to a user we've never
|
||||
# seen; the last inbound msg_id is the natural choice.
|
||||
msg_id = self._last_msg_id.get(chat_id)
|
||||
|
||||
req = ApprovalRequest(
|
||||
session_key=session_key,
|
||||
title=f"Execute this command?",
|
||||
description=description,
|
||||
command_preview=command,
|
||||
timeout_sec=self._APPROVAL_TIMEOUT_SECONDS,
|
||||
)
|
||||
return await self.send_approval_request(
|
||||
chat_id, req, reply_to=msg_id,
|
||||
)
|
||||
|
||||
_APPROVAL_TIMEOUT_SECONDS = 300 # matches gateway's default gateway_timeout
|
||||
|
||||
async def send_update_prompt(
|
||||
self,
|
||||
chat_id: str,
|
||||
prompt: str,
|
||||
default: str = "",
|
||||
session_key: str = "",
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
"""Send a Yes/No update-confirmation prompt with inline buttons.
|
||||
|
||||
Matches the cross-adapter contract used by
|
||||
``gateway/run.py``'s ``hermes update --gateway`` watcher. Button
|
||||
clicks surface as ``INTERACTION_CREATE`` with
|
||||
``button_data = 'update_prompt:y'`` or ``'update_prompt:n'``;
|
||||
the adapter's interaction callback writes the answer to
|
||||
``~/.hermes/.update_response`` so the detached update process
|
||||
can read it.
|
||||
"""
|
||||
del session_key, metadata # present for contract parity only.
|
||||
|
||||
default_hint = f" (default: {default})" if default else ""
|
||||
content = f"⚕ **Update Needs Your Input**\n\n{prompt}{default_hint}"
|
||||
msg_id = self._last_msg_id.get(chat_id)
|
||||
return await self.send_with_keyboard(
|
||||
chat_id,
|
||||
content,
|
||||
build_update_prompt_keyboard(),
|
||||
reply_to=msg_id,
|
||||
)
|
||||
|
||||
def _build_text_body(
|
||||
self, content: str, reply_to: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
@@ -2160,42 +2697,62 @@ class QQAdapter(BasePlatformAdapter):
|
||||
reply_to: Optional[str] = None,
|
||||
file_name: Optional[str] = None,
|
||||
) -> SendResult:
|
||||
"""Upload media and send as a native message."""
|
||||
"""Upload media and send as a native message.
|
||||
|
||||
Upload strategy:
|
||||
|
||||
- **HTTP(S) URLs** → single ``POST /v2/{users|groups}/{id}/files``
|
||||
with ``url=...``. The QQ platform fetches the URL directly; fastest
|
||||
path when the source is already hosted.
|
||||
- **Local files** → three-step chunked upload (prepare / PUT parts /
|
||||
complete). Handles files up to the platform's ~100 MB per-file
|
||||
limit without the ~10 MB inline-base64 cap of the old adapter.
|
||||
"""
|
||||
if not self.is_connected:
|
||||
if not await self._wait_for_reconnection():
|
||||
return SendResult(success=False, error="Not connected", retryable=True)
|
||||
|
||||
try:
|
||||
# Resolve media source
|
||||
data, content_type, resolved_name = await self._load_media(
|
||||
media_source, file_name
|
||||
chat_type = self._guess_chat_type(chat_id)
|
||||
if chat_type == "guild":
|
||||
# Guild channels don't support native media upload in the same way.
|
||||
return SendResult(
|
||||
success=False,
|
||||
error="Guild media send not supported via this path",
|
||||
)
|
||||
|
||||
# Route
|
||||
chat_type = self._guess_chat_type(chat_id)
|
||||
|
||||
if chat_type == "guild":
|
||||
# Guild channels don't support native media upload in the same way
|
||||
# Send as URL fallback
|
||||
return SendResult(
|
||||
success=False, error="Guild media send not supported via this path"
|
||||
try:
|
||||
if self._is_url(media_source):
|
||||
# URL upload — let the platform fetch it directly.
|
||||
resolved_name = (
|
||||
file_name
|
||||
or Path(urlparse(media_source).path).name
|
||||
or "media"
|
||||
)
|
||||
upload = await self._upload_media(
|
||||
chat_type,
|
||||
chat_id,
|
||||
file_type,
|
||||
url=media_source,
|
||||
srv_send_msg=False,
|
||||
file_name=resolved_name if file_type == MEDIA_TYPE_FILE else None,
|
||||
)
|
||||
else:
|
||||
# Local file — chunked upload (prepare / PUT parts / complete).
|
||||
resolved_name, upload = await self._upload_local_file(
|
||||
chat_type,
|
||||
chat_id,
|
||||
media_source,
|
||||
file_type,
|
||||
file_name,
|
||||
)
|
||||
|
||||
# Upload
|
||||
upload = await self._upload_media(
|
||||
chat_type,
|
||||
chat_id,
|
||||
file_type,
|
||||
file_data=data if not self._is_url(media_source) else None,
|
||||
url=media_source if self._is_url(media_source) else None,
|
||||
srv_send_msg=False,
|
||||
file_name=resolved_name if file_type == MEDIA_TYPE_FILE else None,
|
||||
)
|
||||
|
||||
file_info = upload.get("file_info")
|
||||
file_info = upload.get("file_info") or (
|
||||
upload.get("data", {}) or {}
|
||||
).get("file_info")
|
||||
if not file_info:
|
||||
return SendResult(
|
||||
success=False, error=f"Upload returned no file_info: {upload}"
|
||||
success=False,
|
||||
error=f"Upload returned no file_info: {upload}",
|
||||
)
|
||||
|
||||
# Send media message
|
||||
@@ -2224,10 +2781,86 @@ class QQAdapter(BasePlatformAdapter):
|
||||
message_id=str(send_data.get("id", uuid.uuid4().hex[:12])),
|
||||
raw_response=send_data,
|
||||
)
|
||||
except UploadDailyLimitExceededError as exc:
|
||||
# Non-retryable: daily quota hit. Give the caller actionable text
|
||||
# so the model can compose a helpful reply.
|
||||
logger.warning(
|
||||
"[%s] Daily upload limit exceeded for %s (%s)",
|
||||
self._log_tag, exc.file_name, exc.file_size_human,
|
||||
)
|
||||
return SendResult(
|
||||
success=False,
|
||||
error=(
|
||||
f"QQ daily upload limit exceeded for {exc.file_name!r} "
|
||||
f"({exc.file_size_human}). Retry tomorrow."
|
||||
),
|
||||
retryable=False,
|
||||
)
|
||||
except UploadFileTooLargeError as exc:
|
||||
logger.warning(
|
||||
"[%s] File too large: %s (%s, platform limit %s)",
|
||||
self._log_tag, exc.file_name, exc.file_size_human, exc.limit_human,
|
||||
)
|
||||
return SendResult(
|
||||
success=False,
|
||||
error=(
|
||||
f"{exc.file_name!r} ({exc.file_size_human}) exceeds the "
|
||||
f"QQ per-file upload limit ({exc.limit_human})."
|
||||
),
|
||||
retryable=False,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error("[%s] Media send failed: %s", self._log_tag, exc)
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
async def _upload_local_file(
|
||||
self,
|
||||
chat_type: str,
|
||||
chat_id: str,
|
||||
media_source: str,
|
||||
file_type: int,
|
||||
file_name: Optional[str],
|
||||
) -> Tuple[str, Dict[str, Any]]:
|
||||
"""Chunked-upload a local file and return ``(resolved_name, complete_response)``.
|
||||
|
||||
The returned ``complete_response`` contains the ``file_info`` token
|
||||
that goes into the subsequent RichMedia message body.
|
||||
|
||||
:raises UploadDailyLimitExceededError: On biz_code 40093002.
|
||||
:raises UploadFileTooLargeError: When the file exceeds the platform limit.
|
||||
:raises FileNotFoundError: If the path does not exist.
|
||||
:raises ValueError: If the path looks like a placeholder (``<path>``).
|
||||
:raises RuntimeError: If the HTTP client is not initialized.
|
||||
"""
|
||||
if not self._http_client:
|
||||
raise RuntimeError("HTTP client not initialized — not connected?")
|
||||
|
||||
local_path = Path(media_source).expanduser()
|
||||
if not local_path.is_absolute():
|
||||
local_path = (Path.cwd() / local_path).resolve()
|
||||
|
||||
if not local_path.exists() or not local_path.is_file():
|
||||
if media_source.startswith("<") or len(media_source) < 3:
|
||||
raise ValueError(
|
||||
f"Invalid media source (looks like a placeholder): {media_source!r}"
|
||||
)
|
||||
raise FileNotFoundError(f"Media file not found: {local_path}")
|
||||
|
||||
resolved_name = file_name or local_path.name
|
||||
uploader = ChunkedUploader(
|
||||
api_request=self._api_request,
|
||||
http_put=self._http_client.put,
|
||||
log_tag=self._log_tag,
|
||||
)
|
||||
complete = await uploader.upload(
|
||||
chat_type=chat_type,
|
||||
target_id=chat_id,
|
||||
file_path=str(local_path),
|
||||
file_type=file_type,
|
||||
file_name=resolved_name,
|
||||
)
|
||||
return resolved_name, complete
|
||||
|
||||
async def _load_media(
|
||||
self, source: str, file_name: Optional[str] = None
|
||||
) -> Tuple[str, str, str]:
|
||||
|
||||
@@ -0,0 +1,603 @@
|
||||
"""QQ Bot chunked upload flow.
|
||||
|
||||
The QQ v2 API caps inline base64 uploads (``file_data`` / ``url``) at ~10 MB.
|
||||
For files between 10 MB and ~100 MB we have to use the three-step chunked
|
||||
upload flow::
|
||||
|
||||
1. POST /v2/{users|groups}/{id}/upload_prepare
|
||||
→ returns upload_id, block_size, and an array of pre-signed COS part URLs.
|
||||
2. For each part:
|
||||
PUT the part bytes to its pre-signed COS URL,
|
||||
then POST /v2/{users|groups}/{id}/upload_part_finish to acknowledge.
|
||||
3. POST /v2/{users|groups}/{id}/files with {"upload_id": ...}
|
||||
→ returns the ``file_info`` token the caller uses in a RichMedia
|
||||
message.
|
||||
|
||||
Error-code semantics (from the QQ Bot v2 API spec):
|
||||
|
||||
- ``40093001`` — ``upload_part_finish`` retryable. Retry until the server-provided
|
||||
``retry_timeout`` elapses (or a local cap).
|
||||
- ``40093002`` — daily cumulative upload quota exceeded. Not retryable; surface
|
||||
as :class:`UploadDailyLimitExceededError` so the caller can build a
|
||||
user-friendly reply.
|
||||
|
||||
Exceptions:
|
||||
|
||||
- :class:`UploadDailyLimitExceededError` — daily quota hit (non-retryable).
|
||||
- :class:`UploadFileTooLargeError` — file exceeds the platform per-file limit.
|
||||
- :class:`RuntimeError` — generic upload failure (network, part PUT, complete).
|
||||
|
||||
Ported from WideLee's qqbot-agent-sdk v1.2.2 (``media_loader.py::ChunkedUploader``)
|
||||
so the heavy-upload path stays in-tree. Authorship preserved via Co-authored-by.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import functools
|
||||
import hashlib
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Awaitable, Callable, Dict, List, Optional
|
||||
|
||||
from gateway.platforms.qqbot.constants import FILE_UPLOAD_TIMEOUT
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ── Error codes ──────────────────────────────────────────────────────
|
||||
_BIZ_CODE_DAILY_LIMIT = 40093002 # upload_prepare: daily cumulative limit
|
||||
_BIZ_CODE_PART_RETRYABLE = 40093001 # upload_part_finish: transient
|
||||
|
||||
# ── Part upload tuning ───────────────────────────────────────────────
|
||||
_DEFAULT_CONCURRENT_PARTS = 1
|
||||
_MAX_CONCURRENT_PARTS = 10
|
||||
|
||||
_PART_UPLOAD_TIMEOUT = 300.0 # 5 minutes per COS PUT
|
||||
_PART_UPLOAD_MAX_RETRIES = 2
|
||||
_PART_FINISH_RETRY_INTERVAL = 1.0
|
||||
_PART_FINISH_DEFAULT_TIMEOUT = 120.0
|
||||
_PART_FINISH_MAX_TIMEOUT = 600.0
|
||||
|
||||
_COMPLETE_UPLOAD_MAX_RETRIES = 2
|
||||
_COMPLETE_UPLOAD_BASE_DELAY = 2.0
|
||||
|
||||
# First 10,002,432 bytes used for the ``md5_10m`` hash (per QQ API spec).
|
||||
_MD5_10M_SIZE = 10_002_432
|
||||
|
||||
|
||||
# ── Exceptions ───────────────────────────────────────────────────────
|
||||
|
||||
class UploadDailyLimitExceededError(Exception):
|
||||
"""Raised when ``upload_prepare`` returns biz_code 40093002.
|
||||
|
||||
The daily cumulative upload quota for this bot has been reached. Callers
|
||||
should surface :attr:`file_name` + :attr:`file_size_human` so the model
|
||||
can compose a helpful reply.
|
||||
"""
|
||||
|
||||
def __init__(self, file_name: str, file_size: int, message: str = "") -> None:
|
||||
self.file_name = file_name
|
||||
self.file_size = file_size
|
||||
super().__init__(
|
||||
message or f"Daily upload limit exceeded for {file_name!r}"
|
||||
)
|
||||
|
||||
@property
|
||||
def file_size_human(self) -> str:
|
||||
return format_size(self.file_size)
|
||||
|
||||
|
||||
class UploadFileTooLargeError(Exception):
|
||||
"""Raised when a file exceeds the platform per-file size limit."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
file_name: str,
|
||||
file_size: int,
|
||||
limit_bytes: int = 0,
|
||||
message: str = "",
|
||||
) -> None:
|
||||
self.file_name = file_name
|
||||
self.file_size = file_size
|
||||
self.limit_bytes = limit_bytes
|
||||
limit_str = f" ({format_size(limit_bytes)})" if limit_bytes else ""
|
||||
super().__init__(
|
||||
message
|
||||
or (
|
||||
f"File {file_name!r} ({format_size(file_size)}) "
|
||||
f"exceeds platform limit{limit_str}"
|
||||
)
|
||||
)
|
||||
|
||||
@property
|
||||
def file_size_human(self) -> str:
|
||||
return format_size(self.file_size)
|
||||
|
||||
@property
|
||||
def limit_human(self) -> str:
|
||||
return format_size(self.limit_bytes) if self.limit_bytes else "unknown"
|
||||
|
||||
|
||||
# ── Progress tracking ────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class _UploadProgress:
|
||||
total_parts: int = 0
|
||||
total_bytes: int = 0
|
||||
completed_parts: int = 0
|
||||
uploaded_bytes: int = 0
|
||||
|
||||
|
||||
# ── Prepare-response shape ───────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class _PreparePart:
|
||||
index: int
|
||||
presigned_url: str
|
||||
block_size: int = 0
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PrepareResult:
|
||||
upload_id: str
|
||||
block_size: int
|
||||
parts: List[_PreparePart]
|
||||
concurrency: int = _DEFAULT_CONCURRENT_PARTS
|
||||
retry_timeout: float = 0.0
|
||||
|
||||
|
||||
def _parse_prepare_response(raw: Dict[str, Any]) -> _PrepareResult:
|
||||
"""Parse the upload_prepare API response into a normalized shape.
|
||||
|
||||
The API may return the response directly or wrapped in ``data``.
|
||||
"""
|
||||
src = raw.get("data") if isinstance(raw.get("data"), dict) else raw
|
||||
upload_id = str(src.get("upload_id", ""))
|
||||
if not upload_id:
|
||||
raise ValueError(
|
||||
f"upload_prepare response missing upload_id: {str(raw)[:200]}"
|
||||
)
|
||||
block_size = int(src.get("block_size", 0))
|
||||
raw_parts = src.get("parts") or src.get("part_list") or []
|
||||
if not isinstance(raw_parts, list) or not raw_parts:
|
||||
raise ValueError(
|
||||
f"upload_prepare response missing parts: {str(raw)[:200]}"
|
||||
)
|
||||
parts: List[_PreparePart] = []
|
||||
for p in raw_parts:
|
||||
if not isinstance(p, dict):
|
||||
continue
|
||||
parts.append(
|
||||
_PreparePart(
|
||||
index=int(p.get("part_index") or p.get("index") or 0),
|
||||
presigned_url=str(
|
||||
p.get("presigned_url") or p.get("url") or ""
|
||||
),
|
||||
block_size=int(p.get("block_size", 0)),
|
||||
)
|
||||
)
|
||||
return _PrepareResult(
|
||||
upload_id=upload_id,
|
||||
block_size=block_size,
|
||||
parts=parts,
|
||||
concurrency=int(src.get("concurrency", _DEFAULT_CONCURRENT_PARTS)) or _DEFAULT_CONCURRENT_PARTS,
|
||||
retry_timeout=float(src.get("retry_timeout", 0.0) or 0.0),
|
||||
)
|
||||
|
||||
|
||||
# ── Chunked upload driver ────────────────────────────────────────────
|
||||
|
||||
ApiRequestFn = Callable[..., Awaitable[Dict[str, Any]]]
|
||||
"""Signature of the adapter's ``_api_request`` callable.
|
||||
|
||||
We pass the bound method in rather than importing the adapter, to avoid
|
||||
circular imports and keep this module testable in isolation.
|
||||
"""
|
||||
|
||||
|
||||
class ChunkedUploader:
|
||||
"""Run the prepare → PUT parts → complete sequence.
|
||||
|
||||
:param api_request: Bound ``_api_request(method, path, body=..., timeout=...)``
|
||||
coroutine from the adapter. Must raise ``RuntimeError`` with the biz_code
|
||||
embedded in the message on API errors.
|
||||
:param http_put: Coroutine ``(url, data, headers, timeout) -> response`` for
|
||||
COS part uploads. Typically wraps ``httpx.AsyncClient.put``.
|
||||
:param log_tag: Log prefix.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
api_request: ApiRequestFn,
|
||||
http_put: Callable[..., Awaitable[Any]],
|
||||
log_tag: str = "QQBot",
|
||||
) -> None:
|
||||
self._api_request = api_request
|
||||
self._http_put = http_put
|
||||
self._log_tag = log_tag
|
||||
|
||||
async def upload(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
file_path: str,
|
||||
file_type: int,
|
||||
file_name: str,
|
||||
) -> Dict[str, Any]:
|
||||
"""Run the full chunked upload and return the ``complete_upload`` response.
|
||||
|
||||
:param chat_type: ``'c2c'`` or ``'group'``.
|
||||
:param target_id: User or group openid.
|
||||
:param file_path: Absolute path to a local file.
|
||||
:param file_type: ``MEDIA_TYPE_*`` constant.
|
||||
:param file_name: Original filename (for upload_prepare).
|
||||
:returns: The raw response dict from ``complete_upload`` — contains
|
||||
``file_info`` that the caller uses in a RichMedia message body.
|
||||
:raises UploadDailyLimitExceededError: On biz_code 40093002.
|
||||
:raises UploadFileTooLargeError: When the file exceeds the platform limit.
|
||||
:raises RuntimeError: On other API or I/O failures.
|
||||
"""
|
||||
if chat_type not in ("c2c", "group"):
|
||||
raise ValueError(
|
||||
f"ChunkedUploader: unsupported chat_type {chat_type!r}"
|
||||
)
|
||||
|
||||
path = Path(file_path)
|
||||
file_size = path.stat().st_size
|
||||
|
||||
logger.info(
|
||||
"[%s] Chunked upload start: file=%s size=%s type=%d",
|
||||
self._log_tag, file_name, format_size(file_size), file_type,
|
||||
)
|
||||
|
||||
# Step 1: compute hashes (blocking I/O → executor).
|
||||
hashes = await asyncio.get_running_loop().run_in_executor(
|
||||
None, _compute_file_hashes, file_path, file_size
|
||||
)
|
||||
|
||||
# Step 2: upload_prepare.
|
||||
prepare = await self._prepare(
|
||||
chat_type, target_id, file_type, file_name, file_size, hashes
|
||||
)
|
||||
max_concurrent = min(prepare.concurrency, _MAX_CONCURRENT_PARTS)
|
||||
retry_timeout = min(
|
||||
prepare.retry_timeout if prepare.retry_timeout > 0 else _PART_FINISH_DEFAULT_TIMEOUT,
|
||||
_PART_FINISH_MAX_TIMEOUT,
|
||||
)
|
||||
logger.info(
|
||||
"[%s] Prepared: upload_id=%s block_size=%s parts=%d concurrency=%d",
|
||||
self._log_tag, prepare.upload_id, format_size(prepare.block_size),
|
||||
len(prepare.parts), max_concurrent,
|
||||
)
|
||||
|
||||
progress = _UploadProgress(
|
||||
total_parts=len(prepare.parts),
|
||||
total_bytes=file_size,
|
||||
)
|
||||
|
||||
# Step 3: PUT each part + notify.
|
||||
tasks: List[Callable[[], Awaitable[None]]] = [
|
||||
functools.partial(
|
||||
self._upload_one_part,
|
||||
chat_type=chat_type,
|
||||
target_id=target_id,
|
||||
file_path=file_path,
|
||||
file_size=file_size,
|
||||
upload_id=prepare.upload_id,
|
||||
rsp_block_size=prepare.block_size,
|
||||
part=part,
|
||||
retry_timeout=retry_timeout,
|
||||
progress=progress,
|
||||
)
|
||||
for part in prepare.parts
|
||||
]
|
||||
await _run_with_concurrency(tasks, max_concurrent)
|
||||
|
||||
logger.info(
|
||||
"[%s] All %d parts uploaded, completing…",
|
||||
self._log_tag, len(prepare.parts),
|
||||
)
|
||||
|
||||
# Step 4: complete_upload (retry on transient errors).
|
||||
return await self._complete(chat_type, target_id, prepare.upload_id)
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
# Step 1 — upload_prepare
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
|
||||
async def _prepare(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
file_type: int,
|
||||
file_name: str,
|
||||
file_size: int,
|
||||
hashes: Dict[str, str],
|
||||
) -> _PrepareResult:
|
||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
||||
path = f"{base}/{target_id}/upload_prepare"
|
||||
body = {
|
||||
"file_type": file_type,
|
||||
"file_name": file_name,
|
||||
"file_size": file_size,
|
||||
"md5": hashes["md5"],
|
||||
"sha1": hashes["sha1"],
|
||||
"md5_10m": hashes["md5_10m"],
|
||||
}
|
||||
try:
|
||||
raw = await self._api_request(
|
||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
||||
)
|
||||
except RuntimeError as exc:
|
||||
err_msg = str(exc)
|
||||
if f"{_BIZ_CODE_DAILY_LIMIT}" in err_msg:
|
||||
raise UploadDailyLimitExceededError(
|
||||
file_name, file_size, err_msg
|
||||
) from exc
|
||||
raise
|
||||
return _parse_prepare_response(raw)
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
# Step 2 — PUT one part + part_finish
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
|
||||
async def _upload_one_part(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
file_path: str,
|
||||
file_size: int,
|
||||
upload_id: str,
|
||||
rsp_block_size: int,
|
||||
part: _PreparePart,
|
||||
retry_timeout: float,
|
||||
progress: _UploadProgress,
|
||||
) -> None:
|
||||
"""PUT one part to COS, then call ``upload_part_finish``."""
|
||||
part_index = part.index
|
||||
# Per-part block_size wins; fall back to the response-level value.
|
||||
actual_block_size = part.block_size if part.block_size > 0 else rsp_block_size
|
||||
offset = (part_index - 1) * rsp_block_size
|
||||
length = min(actual_block_size, file_size - offset)
|
||||
|
||||
# Read this slice of the file (blocking → executor).
|
||||
data = await asyncio.get_running_loop().run_in_executor(
|
||||
None, _read_file_chunk, file_path, offset, length
|
||||
)
|
||||
md5_hex = hashlib.md5(data).hexdigest()
|
||||
|
||||
logger.debug(
|
||||
"[%s] Part %d/%d: uploading %s (offset=%d md5=%s)",
|
||||
self._log_tag, part_index, progress.total_parts,
|
||||
format_size(length), offset, md5_hex,
|
||||
)
|
||||
|
||||
await self._put_to_presigned_url(
|
||||
part.presigned_url, data, part_index, progress.total_parts
|
||||
)
|
||||
await self._part_finish_with_retry(
|
||||
chat_type, target_id, upload_id,
|
||||
part_index, length, md5_hex, retry_timeout,
|
||||
)
|
||||
|
||||
progress.completed_parts += 1
|
||||
progress.uploaded_bytes += length
|
||||
logger.debug(
|
||||
"[%s] Part %d/%d done (%d/%d total)",
|
||||
self._log_tag, part_index, progress.total_parts,
|
||||
progress.completed_parts, progress.total_parts,
|
||||
)
|
||||
|
||||
async def _put_to_presigned_url(
|
||||
self,
|
||||
url: str,
|
||||
data: bytes,
|
||||
part_index: int,
|
||||
total_parts: int,
|
||||
) -> None:
|
||||
"""PUT part data to a pre-signed COS URL with retry."""
|
||||
last_exc: Optional[Exception] = None
|
||||
for attempt in range(_PART_UPLOAD_MAX_RETRIES + 1):
|
||||
try:
|
||||
resp = await asyncio.wait_for(
|
||||
self._http_put(
|
||||
url,
|
||||
data=data,
|
||||
headers={"Content-Length": str(len(data))},
|
||||
),
|
||||
timeout=_PART_UPLOAD_TIMEOUT,
|
||||
)
|
||||
# Caller's http_put is expected to return an httpx-like response.
|
||||
status = getattr(resp, "status_code", 0)
|
||||
if 200 <= status < 300:
|
||||
logger.debug(
|
||||
"[%s] PUT part %d/%d: %d OK",
|
||||
self._log_tag, part_index, total_parts, status,
|
||||
)
|
||||
return
|
||||
body_preview = ""
|
||||
try:
|
||||
body_preview = getattr(resp, "text", "")[:200]
|
||||
except Exception: # pragma: no cover — defensive
|
||||
pass
|
||||
raise RuntimeError(
|
||||
f"COS PUT returned {status}: {body_preview}"
|
||||
)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt < _PART_UPLOAD_MAX_RETRIES:
|
||||
delay = 1.0 * (2 ** attempt)
|
||||
logger.warning(
|
||||
"[%s] PUT part %d/%d attempt %d failed, retry in %.1fs: %s",
|
||||
self._log_tag, part_index, total_parts,
|
||||
attempt + 1, delay, exc,
|
||||
)
|
||||
await asyncio.sleep(delay)
|
||||
raise RuntimeError(
|
||||
f"Part {part_index}/{total_parts} upload failed after "
|
||||
f"{_PART_UPLOAD_MAX_RETRIES + 1} attempts: {last_exc}"
|
||||
)
|
||||
|
||||
async def _part_finish_with_retry(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
upload_id: str,
|
||||
part_index: int,
|
||||
block_size: int,
|
||||
md5: str,
|
||||
retry_timeout: float,
|
||||
) -> None:
|
||||
"""Call ``upload_part_finish``, retrying on biz_code 40093001."""
|
||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
||||
path = f"{base}/{target_id}/upload_part_finish"
|
||||
body = {
|
||||
"upload_id": upload_id,
|
||||
"part_index": part_index,
|
||||
"block_size": block_size,
|
||||
"md5": md5,
|
||||
}
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
start = loop.time()
|
||||
attempt = 0
|
||||
while True:
|
||||
try:
|
||||
await self._api_request(
|
||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
||||
)
|
||||
return
|
||||
except RuntimeError as exc:
|
||||
err_msg = str(exc)
|
||||
if f"{_BIZ_CODE_PART_RETRYABLE}" not in err_msg:
|
||||
raise
|
||||
elapsed = loop.time() - start
|
||||
if elapsed >= retry_timeout:
|
||||
raise RuntimeError(
|
||||
f"upload_part_finish persistent retry timed out "
|
||||
f"after {retry_timeout:.0f}s ({attempt} retries): {exc}"
|
||||
) from exc
|
||||
attempt += 1
|
||||
logger.debug(
|
||||
"[%s] part_finish retryable error, attempt %d, "
|
||||
"elapsed=%.1fs: %s",
|
||||
self._log_tag, attempt, elapsed, exc,
|
||||
)
|
||||
await asyncio.sleep(_PART_FINISH_RETRY_INTERVAL)
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
# Step 3 — complete_upload
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
|
||||
async def _complete(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
upload_id: str,
|
||||
) -> Dict[str, Any]:
|
||||
"""Call ``complete_upload`` with retry.
|
||||
|
||||
This reuses the ``/files`` endpoint (same as the simple URL-based upload)
|
||||
but signals the chunked-completion path by sending only ``upload_id``.
|
||||
"""
|
||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
||||
path = f"{base}/{target_id}/files"
|
||||
body = {"upload_id": upload_id}
|
||||
|
||||
last_exc: Optional[Exception] = None
|
||||
for attempt in range(_COMPLETE_UPLOAD_MAX_RETRIES + 1):
|
||||
try:
|
||||
return await self._api_request(
|
||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
||||
)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt < _COMPLETE_UPLOAD_MAX_RETRIES:
|
||||
delay = _COMPLETE_UPLOAD_BASE_DELAY * (2 ** attempt)
|
||||
logger.warning(
|
||||
"[%s] complete_upload attempt %d failed, "
|
||||
"retry in %.1fs: %s",
|
||||
self._log_tag, attempt + 1, delay, exc,
|
||||
)
|
||||
await asyncio.sleep(delay)
|
||||
raise RuntimeError(
|
||||
f"complete_upload failed after "
|
||||
f"{_COMPLETE_UPLOAD_MAX_RETRIES + 1} attempts: {last_exc}"
|
||||
)
|
||||
|
||||
|
||||
# ── Helpers (module-level for testability) ───────────────────────────
|
||||
|
||||
def format_size(size_bytes: int) -> str:
|
||||
"""Return a human-readable file size string (e.g. ``'12.3 MB'``)."""
|
||||
size = float(size_bytes)
|
||||
for unit in ("B", "KB", "MB", "GB"):
|
||||
if size < 1024.0:
|
||||
return f"{size:.1f} {unit}"
|
||||
size /= 1024.0
|
||||
return f"{size:.1f} TB"
|
||||
|
||||
|
||||
def _read_file_chunk(file_path: str, offset: int, length: int) -> bytes:
|
||||
"""Read *length* bytes from *file_path* starting at *offset*.
|
||||
|
||||
:raises IOError: If fewer bytes were read than expected (truncated file).
|
||||
"""
|
||||
with open(file_path, "rb") as fh:
|
||||
fh.seek(offset)
|
||||
data = fh.read(length)
|
||||
if len(data) != length:
|
||||
raise IOError(
|
||||
f"Short read from {file_path}: expected {length} bytes at "
|
||||
f"offset {offset}, got {len(data)} (file may be truncated)"
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def _compute_file_hashes(file_path: str, file_size: int) -> Dict[str, str]:
|
||||
"""Compute md5, sha1, and md5_10m in a single pass."""
|
||||
md5 = hashlib.md5()
|
||||
sha1 = hashlib.sha1()
|
||||
md5_10m = hashlib.md5()
|
||||
|
||||
need_10m = file_size > _MD5_10M_SIZE
|
||||
bytes_read = 0
|
||||
|
||||
with open(file_path, "rb") as fh:
|
||||
while True:
|
||||
chunk = fh.read(65536)
|
||||
if not chunk:
|
||||
break
|
||||
md5.update(chunk)
|
||||
sha1.update(chunk)
|
||||
if need_10m:
|
||||
remaining = _MD5_10M_SIZE - bytes_read
|
||||
if remaining > 0:
|
||||
md5_10m.update(chunk[:remaining])
|
||||
bytes_read += len(chunk)
|
||||
|
||||
full_md5 = md5.hexdigest()
|
||||
return {
|
||||
"md5": full_md5,
|
||||
"sha1": sha1.hexdigest(),
|
||||
# For small files the "10m" hash is just the full md5.
|
||||
"md5_10m": md5_10m.hexdigest() if need_10m else full_md5,
|
||||
}
|
||||
|
||||
|
||||
async def _run_with_concurrency(
|
||||
tasks: List[Callable[[], Awaitable[None]]],
|
||||
concurrency: int,
|
||||
) -> None:
|
||||
"""Run a list of thunks with a bounded number in flight at once."""
|
||||
if concurrency < 1:
|
||||
concurrency = 1
|
||||
sem = asyncio.Semaphore(concurrency)
|
||||
|
||||
async def _wrap(thunk: Callable[[], Awaitable[None]]) -> None:
|
||||
async with sem:
|
||||
await thunk()
|
||||
|
||||
await asyncio.gather(*(_wrap(t) for t in tasks))
|
||||
@@ -0,0 +1,473 @@
|
||||
"""QQ Bot inline keyboards + approval / update-prompt senders.
|
||||
|
||||
QQ Bot v2 supports attaching inline keyboards to outbound messages. When a
|
||||
user clicks a button, the platform dispatches an ``INTERACTION_CREATE``
|
||||
gateway event containing the button's ``data`` payload. The bot must ACK the
|
||||
interaction promptly via ``PUT /interactions/{id}`` or the user sees an
|
||||
error indicator on the button.
|
||||
|
||||
This module provides:
|
||||
|
||||
- :class:`InlineKeyboard` + button dataclasses — serialized into the
|
||||
``keyboard`` field of the outbound message body.
|
||||
- :func:`build_approval_keyboard` — 3-button ✅ once / ⭐ always / ❌ deny
|
||||
keyboard for tool-approval flows.
|
||||
- :func:`build_update_prompt_keyboard` — Yes/No keyboard for update confirms.
|
||||
- :func:`parse_approval_button_data` / :func:`parse_update_prompt_button_data`
|
||||
— decode the ``button_data`` payload from ``INTERACTION_CREATE``.
|
||||
- :class:`ApprovalRequest` + :class:`ApprovalSender` — high-level helper that
|
||||
builds an approval message with keyboard and posts it to a c2c / group chat.
|
||||
|
||||
``button_data`` formats::
|
||||
|
||||
approve:<session_key>:<decision> # decision = allow-once|allow-always|deny
|
||||
update_prompt:<answer> # answer = y|n
|
||||
|
||||
Ported from WideLee's qqbot-agent-sdk v1.2.2 (``approval.py`` + ``dto.py``
|
||||
keyboard types). Authorship preserved via Co-authored-by.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Awaitable, Callable, Dict, List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ── button_data prefixes + patterns ──────────────────────────────────
|
||||
|
||||
APPROVAL_BUTTON_PREFIX = "approve:"
|
||||
UPDATE_PROMPT_PREFIX = "update_prompt:"
|
||||
|
||||
# Pattern: approve:<session_key>:<decision>
|
||||
# session_key may itself contain colons (e.g. agent:main:qqbot:c2c:OPENID),
|
||||
# so the session_key group is greedy but trails the decision.
|
||||
_APPROVAL_DATA_RE = re.compile(
|
||||
r"^approve:(.+):(allow-once|allow-always|deny)$"
|
||||
)
|
||||
|
||||
# Pattern: update_prompt:y | update_prompt:n
|
||||
_UPDATE_PROMPT_RE = re.compile(r"^update_prompt:(y|n)$")
|
||||
|
||||
|
||||
# ── Keyboard dataclasses ─────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class KeyboardButtonPermission:
|
||||
"""Button permission metadata. ``type=2`` means all users can click."""
|
||||
type: int = 2
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {"type": self.type}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardButtonAction:
|
||||
"""What happens when the button is clicked.
|
||||
|
||||
:param type: ``1`` (Callback — triggers ``INTERACTION_CREATE``) or
|
||||
``2`` (Link — opens a URL).
|
||||
:param data: Payload delivered in ``data.resolved.button_data`` when
|
||||
``type=1``.
|
||||
:param permission: :class:`KeyboardButtonPermission`.
|
||||
:param click_limit: Max clicks per user (``1`` = single-use).
|
||||
"""
|
||||
type: int
|
||||
data: str
|
||||
permission: KeyboardButtonPermission = field(
|
||||
default_factory=KeyboardButtonPermission
|
||||
)
|
||||
click_limit: int = 1
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"type": self.type,
|
||||
"data": self.data,
|
||||
"permission": self.permission.to_dict(),
|
||||
"click_limit": self.click_limit,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardButtonRenderData:
|
||||
"""Visual rendering of a button.
|
||||
|
||||
:param label: Pre-click label.
|
||||
:param visited_label: Post-click label (button stays greyed in place).
|
||||
:param style: ``0`` = grey, ``1`` = blue.
|
||||
"""
|
||||
label: str
|
||||
visited_label: str
|
||||
style: int = 1
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"label": self.label,
|
||||
"visited_label": self.visited_label,
|
||||
"style": self.style,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardButton:
|
||||
"""One button in a keyboard.
|
||||
|
||||
:param group_id: Buttons sharing a ``group_id`` are mutually exclusive —
|
||||
clicking one greys the rest.
|
||||
"""
|
||||
id: str
|
||||
render_data: KeyboardButtonRenderData
|
||||
action: KeyboardButtonAction
|
||||
group_id: str = "default"
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"render_data": self.render_data.to_dict(),
|
||||
"action": self.action.to_dict(),
|
||||
"group_id": self.group_id,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardRow:
|
||||
buttons: List[KeyboardButton] = field(default_factory=list)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {"buttons": [b.to_dict() for b in self.buttons]}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardContent:
|
||||
rows: List[KeyboardRow] = field(default_factory=list)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {"rows": [r.to_dict() for r in self.rows]}
|
||||
|
||||
|
||||
@dataclass
|
||||
class InlineKeyboard:
|
||||
"""Top-level keyboard payload — goes into ``MessageToCreate.keyboard``."""
|
||||
content: KeyboardContent = field(default_factory=KeyboardContent)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {"content": self.content.to_dict()}
|
||||
|
||||
|
||||
# ── INTERACTION_CREATE parsing ───────────────────────────────────────
|
||||
|
||||
def parse_approval_button_data(button_data: str) -> Optional[tuple[str, str]]:
|
||||
"""Parse approval ``button_data`` into ``(session_key, decision)``.
|
||||
|
||||
:param button_data: Raw ``data.resolved.button_data`` from
|
||||
``INTERACTION_CREATE``.
|
||||
:returns: ``(session_key, decision)`` or ``None`` if not an approval button.
|
||||
"""
|
||||
m = _APPROVAL_DATA_RE.match(button_data or "")
|
||||
if not m:
|
||||
return None
|
||||
return m.group(1), m.group(2)
|
||||
|
||||
|
||||
def parse_update_prompt_button_data(button_data: str) -> Optional[str]:
|
||||
"""Parse update-prompt ``button_data`` into ``'y'`` or ``'n'``."""
|
||||
m = _UPDATE_PROMPT_RE.match(button_data or "")
|
||||
if not m:
|
||||
return None
|
||||
return m.group(1)
|
||||
|
||||
|
||||
# ── Keyboard builders ────────────────────────────────────────────────
|
||||
|
||||
def _make_callback_button(
|
||||
btn_id: str,
|
||||
label: str,
|
||||
visited_label: str,
|
||||
data: str,
|
||||
style: int,
|
||||
group_id: str,
|
||||
) -> KeyboardButton:
|
||||
return KeyboardButton(
|
||||
id=btn_id,
|
||||
render_data=KeyboardButtonRenderData(
|
||||
label=label,
|
||||
visited_label=visited_label,
|
||||
style=style,
|
||||
),
|
||||
action=KeyboardButtonAction(type=1, data=data),
|
||||
group_id=group_id,
|
||||
)
|
||||
|
||||
|
||||
def build_approval_keyboard(session_key: str) -> InlineKeyboard:
|
||||
"""Build the 3-button approval keyboard.
|
||||
|
||||
Layout: ``[✅ 允许一次] [⭐ 始终允许] [❌ 拒绝]`` — all three share
|
||||
``group_id='approval'`` so clicking one greys out the rest.
|
||||
|
||||
:param session_key: Embedded into ``button_data`` so the decision
|
||||
routes back to the right pending approval.
|
||||
"""
|
||||
return InlineKeyboard(
|
||||
content=KeyboardContent(
|
||||
rows=[
|
||||
KeyboardRow(buttons=[
|
||||
_make_callback_button(
|
||||
btn_id="allow",
|
||||
label="✅ 允许一次",
|
||||
visited_label="已允许",
|
||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:allow-once",
|
||||
style=1,
|
||||
group_id="approval",
|
||||
),
|
||||
_make_callback_button(
|
||||
btn_id="always",
|
||||
label="⭐ 始终允许",
|
||||
visited_label="已始终允许",
|
||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:allow-always",
|
||||
style=1,
|
||||
group_id="approval",
|
||||
),
|
||||
_make_callback_button(
|
||||
btn_id="deny",
|
||||
label="❌ 拒绝",
|
||||
visited_label="已拒绝",
|
||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:deny",
|
||||
style=0,
|
||||
group_id="approval",
|
||||
),
|
||||
]),
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def build_update_prompt_keyboard() -> InlineKeyboard:
|
||||
"""Build a Yes/No keyboard for update confirmation prompts."""
|
||||
return InlineKeyboard(
|
||||
content=KeyboardContent(
|
||||
rows=[
|
||||
KeyboardRow(buttons=[
|
||||
_make_callback_button(
|
||||
btn_id="yes",
|
||||
label="✓ 确认",
|
||||
visited_label="已确认",
|
||||
data=f"{UPDATE_PROMPT_PREFIX}y",
|
||||
style=1,
|
||||
group_id="update_prompt",
|
||||
),
|
||||
_make_callback_button(
|
||||
btn_id="no",
|
||||
label="✗ 取消",
|
||||
visited_label="已取消",
|
||||
data=f"{UPDATE_PROMPT_PREFIX}n",
|
||||
style=0,
|
||||
group_id="update_prompt",
|
||||
),
|
||||
]),
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# ── ApprovalRequest + text builder ───────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class ApprovalRequest:
|
||||
"""Structured approval-request display data.
|
||||
|
||||
:param session_key: Routes the decision back to the waiting caller.
|
||||
:param title: Short title at the top.
|
||||
:param description: Optional longer description.
|
||||
:param command_preview: Command text (exec approvals).
|
||||
:param cwd: Working directory (exec approvals).
|
||||
:param tool_name: Tool name (plugin approvals).
|
||||
:param severity: ``'critical' | 'info' | ''``.
|
||||
:param timeout_sec: Seconds until the approval expires.
|
||||
"""
|
||||
session_key: str
|
||||
title: str
|
||||
description: str = ""
|
||||
command_preview: str = ""
|
||||
cwd: str = ""
|
||||
tool_name: str = ""
|
||||
severity: str = ""
|
||||
timeout_sec: int = 120
|
||||
|
||||
|
||||
def build_approval_text(req: ApprovalRequest) -> str:
|
||||
"""Render an :class:`ApprovalRequest` into the message body (markdown)."""
|
||||
if req.command_preview or req.cwd:
|
||||
return _build_exec_text(req)
|
||||
return _build_plugin_text(req)
|
||||
|
||||
|
||||
def _build_exec_text(req: ApprovalRequest) -> str:
|
||||
lines: List[str] = ["🔐 **命令执行审批**", ""]
|
||||
if req.command_preview:
|
||||
preview = req.command_preview[:300]
|
||||
lines.append(f"```\n{preview}\n```")
|
||||
if req.cwd:
|
||||
lines.append(f"📁 目录: {req.cwd}")
|
||||
if req.title and req.title != req.command_preview:
|
||||
lines.append(f"📋 {req.title}")
|
||||
if req.description:
|
||||
lines.append(f"📝 {req.description}")
|
||||
lines.append("")
|
||||
lines.append(f"⏱️ 超时: {req.timeout_sec} 秒")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _build_plugin_text(req: ApprovalRequest) -> str:
|
||||
icon = (
|
||||
"🔴" if req.severity == "critical"
|
||||
else "🔵" if req.severity == "info"
|
||||
else "🟡"
|
||||
)
|
||||
lines: List[str] = [f"{icon} **审批请求**", ""]
|
||||
lines.append(f"📋 {req.title}")
|
||||
if req.description:
|
||||
lines.append(f"📝 {req.description}")
|
||||
if req.tool_name:
|
||||
lines.append(f"🔧 工具: {req.tool_name}")
|
||||
lines.append("")
|
||||
lines.append(f"⏱️ 超时: {req.timeout_sec} 秒")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# ── ApprovalSender ───────────────────────────────────────────────────
|
||||
|
||||
PostMessageFn = Callable[..., Awaitable[Dict[str, Any]]]
|
||||
"""Signature of an async POST to ``/v2/{users|groups}/{id}/messages``.
|
||||
|
||||
Implementations accept a body dict and return the raw API response.
|
||||
"""
|
||||
|
||||
|
||||
class ApprovalSender:
|
||||
"""Send an approval-request message with an inline keyboard.
|
||||
|
||||
Decoupled from the adapter via callables so it can be unit-tested in
|
||||
isolation. Pass the adapter's ``_send_message_with_keyboard`` helper
|
||||
(or any equivalent) as ``post_message``.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
post_c2c: PostMessageFn,
|
||||
post_group: PostMessageFn,
|
||||
log_tag: str = "QQBot",
|
||||
) -> None:
|
||||
self._post_c2c = post_c2c
|
||||
self._post_group = post_group
|
||||
self._log_tag = log_tag
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_type: str,
|
||||
chat_id: str,
|
||||
req: ApprovalRequest,
|
||||
msg_id: Optional[str] = None,
|
||||
) -> bool:
|
||||
"""Send an approval message to *chat_id*.
|
||||
|
||||
:param chat_type: ``'c2c'`` or ``'group'``.
|
||||
:param chat_id: User openid or group openid.
|
||||
:param req: :class:`ApprovalRequest`.
|
||||
:param msg_id: Reply-to message id (required for passive messages).
|
||||
:returns: ``True`` on success, ``False`` on failure.
|
||||
"""
|
||||
text = build_approval_text(req)
|
||||
keyboard = build_approval_keyboard(req.session_key)
|
||||
|
||||
logger.info(
|
||||
"[%s] Sending approval request to %s:%s (session=%.20s…)",
|
||||
self._log_tag, chat_type, chat_id, req.session_key,
|
||||
)
|
||||
|
||||
try:
|
||||
if chat_type == "c2c":
|
||||
await self._post_c2c(chat_id, text, msg_id, keyboard)
|
||||
elif chat_type == "group":
|
||||
await self._post_group(chat_id, text, msg_id, keyboard)
|
||||
else:
|
||||
logger.warning(
|
||||
"[%s] Approval: unsupported chat_type %r",
|
||||
self._log_tag, chat_type,
|
||||
)
|
||||
return False
|
||||
logger.info(
|
||||
"[%s] Approval message sent to %s:%s",
|
||||
self._log_tag, chat_type, chat_id,
|
||||
)
|
||||
return True
|
||||
except Exception as exc:
|
||||
logger.error(
|
||||
"[%s] Failed to send approval message to %s:%s: %s",
|
||||
self._log_tag, chat_type, chat_id, exc,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
# ── INTERACTION_CREATE event shape ───────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class InteractionEvent:
|
||||
"""Parsed ``INTERACTION_CREATE`` event payload.
|
||||
|
||||
See https://bot.q.qq.com/wiki/develop/api-v2/dev-prepare/interface-framework/event-emit.html
|
||||
"""
|
||||
id: str = ""
|
||||
"""Interaction event id — required for the ``PUT /interactions/{id}`` ACK."""
|
||||
|
||||
type: int = 0
|
||||
"""Event type code (``11`` = message button)."""
|
||||
|
||||
chat_type: int = 0
|
||||
"""``0`` = guild, ``1`` = group, ``2`` = c2c."""
|
||||
|
||||
scene: str = ""
|
||||
"""``'guild'`` | ``'group'`` | ``'c2c'`` — human-readable scene."""
|
||||
|
||||
group_openid: str = ""
|
||||
group_member_openid: str = ""
|
||||
user_openid: str = ""
|
||||
channel_id: str = ""
|
||||
guild_id: str = ""
|
||||
|
||||
button_data: str = ""
|
||||
button_id: str = ""
|
||||
resolver_user_id: str = ""
|
||||
|
||||
@property
|
||||
def operator_openid(self) -> str:
|
||||
"""Best available operator openid (group → member; c2c → user)."""
|
||||
return (
|
||||
self.group_member_openid
|
||||
or self.user_openid
|
||||
or self.resolver_user_id
|
||||
)
|
||||
|
||||
|
||||
def parse_interaction_event(raw: Dict[str, Any]) -> InteractionEvent:
|
||||
"""Parse a raw ``INTERACTION_CREATE`` dispatch payload (``d``)."""
|
||||
data_raw = raw.get("data") or {}
|
||||
resolved = data_raw.get("resolved") or {}
|
||||
scene_code = int(raw.get("chat_type", 0) or 0)
|
||||
scene = {0: "guild", 1: "group", 2: "c2c"}.get(scene_code, "")
|
||||
return InteractionEvent(
|
||||
id=str(raw.get("id", "")),
|
||||
type=int(data_raw.get("type", 0) or 0),
|
||||
chat_type=scene_code,
|
||||
scene=scene,
|
||||
group_openid=str(raw.get("group_openid", "")),
|
||||
group_member_openid=str(raw.get("group_member_openid", "")),
|
||||
user_openid=str(raw.get("user_openid", "")),
|
||||
channel_id=str(raw.get("channel_id", "")),
|
||||
guild_id=str(raw.get("guild_id", "")),
|
||||
button_data=str(resolved.get("button_data", "")),
|
||||
button_id=str(resolved.get("button_id", "")),
|
||||
resolver_user_id=str(resolved.get("user_id", "")),
|
||||
)
|
||||
@@ -1887,6 +1887,12 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
is_thread_reply = bool(event_thread_ts and event_thread_ts != ts)
|
||||
|
||||
if not is_dm and bot_uid:
|
||||
# Check allowed channels — if set, only respond in these channels (whitelist)
|
||||
allowed_channels = self._slack_allowed_channels()
|
||||
if allowed_channels and channel_id not in allowed_channels:
|
||||
logger.debug("[Slack] Ignoring message in non-allowed channel: %s", channel_id)
|
||||
return
|
||||
|
||||
if channel_id in self._slack_free_response_channels():
|
||||
pass # Free-response channel — always process
|
||||
elif not self._slack_require_mention():
|
||||
@@ -2924,3 +2930,19 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
if s:
|
||||
return {part.strip() for part in s.split(",") if part.strip()}
|
||||
return set()
|
||||
|
||||
def _slack_allowed_channels(self) -> set:
|
||||
"""Return the whitelist of channel IDs the bot will respond in.
|
||||
|
||||
When non-empty, messages from channels NOT in this set are silently
|
||||
ignored — even if the bot is @mentioned. DMs are never filtered.
|
||||
Empty set means no restriction (fully backward compatible).
|
||||
"""
|
||||
raw = self.config.extra.get("allowed_channels")
|
||||
if raw is None:
|
||||
raw = os.getenv("SLACK_ALLOWED_CHANNELS", "")
|
||||
if isinstance(raw, list):
|
||||
return {str(part).strip() for part in raw if str(part).strip()}
|
||||
if isinstance(raw, str) and raw.strip():
|
||||
return {part.strip() for part in raw.split(",") if part.strip()}
|
||||
return set()
|
||||
|
||||
+104
-21
@@ -86,6 +86,22 @@ from gateway.platforms.telegram_network import (
|
||||
)
|
||||
from utils import atomic_replace
|
||||
|
||||
_TELEGRAM_IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".gif"}
|
||||
_TELEGRAM_IMAGE_MIME_TO_EXT = {
|
||||
"image/png": ".png",
|
||||
"image/jpeg": ".jpg",
|
||||
"image/jpg": ".jpg",
|
||||
"image/webp": ".webp",
|
||||
"image/gif": ".gif",
|
||||
}
|
||||
_TELEGRAM_IMAGE_EXT_TO_MIME = {
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".webp": "image/webp",
|
||||
".gif": "image/gif",
|
||||
}
|
||||
|
||||
|
||||
def check_telegram_requirements() -> bool:
|
||||
"""Check if Telegram dependencies are available."""
|
||||
@@ -353,10 +369,14 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
|
||||
@classmethod
|
||||
def _message_thread_id_for_typing(cls, thread_id: Optional[str]) -> Optional[int]:
|
||||
# Mirrors _message_thread_id_for_send: the General forum topic (thread id
|
||||
# "1") is represented as "no thread id" on the wire. User-created topics
|
||||
# keep their real id so typing stays scoped to that topic.
|
||||
if not thread_id or str(thread_id) == cls._GENERAL_TOPIC_THREAD_ID:
|
||||
# Asymmetric with _message_thread_id_for_send on purpose. Telegram's
|
||||
# sendMessage and sendChatAction treat thread id "1" (the forum General
|
||||
# topic) differently: sends reject message_thread_id=1 and must omit it,
|
||||
# but sendChatAction needs message_thread_id=1 to place the typing
|
||||
# bubble in the General topic (omitting it hides the bubble entirely
|
||||
# from the client's view of that topic). Preserve the real id here —
|
||||
# sends still map "1" → None via _message_thread_id_for_send.
|
||||
if not thread_id:
|
||||
return None
|
||||
return int(thread_id)
|
||||
|
||||
@@ -724,7 +744,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
return
|
||||
|
||||
import yaml as _yaml
|
||||
with open(config_path, "r") as f:
|
||||
with open(config_path, "r", encoding="utf-8") as f:
|
||||
config = _yaml.safe_load(f) or {}
|
||||
|
||||
# Navigate to platforms.telegram.extra.dm_topics
|
||||
@@ -2755,6 +2775,20 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
return {str(part).strip() for part in raw if str(part).strip()}
|
||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
||||
|
||||
def _telegram_allowed_chats(self) -> set[str]:
|
||||
"""Return the whitelist of group/supergroup chat IDs the bot will respond in.
|
||||
|
||||
When non-empty, group messages from chats NOT in this set are silently
|
||||
ignored — even if the bot is @mentioned. DMs are never filtered.
|
||||
Empty set means no restriction (fully backward compatible).
|
||||
"""
|
||||
raw = self.config.extra.get("allowed_chats")
|
||||
if raw is None:
|
||||
raw = os.getenv("TELEGRAM_ALLOWED_CHATS", "")
|
||||
if isinstance(raw, list):
|
||||
return {str(part).strip() for part in raw if str(part).strip()}
|
||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
||||
|
||||
def _telegram_ignored_threads(self) -> set[int]:
|
||||
raw = self.config.extra.get("ignored_threads")
|
||||
if raw is None:
|
||||
@@ -2903,13 +2937,16 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
"""Apply Telegram group trigger rules.
|
||||
|
||||
DMs remain unrestricted. Group/supergroup messages are accepted when:
|
||||
- the chat passes the ``allowed_chats`` whitelist (when set)
|
||||
- the chat is explicitly allowlisted in ``free_response_chats``
|
||||
- ``require_mention`` is disabled
|
||||
- the message replies to the bot
|
||||
- the bot is @mentioned
|
||||
- the text/caption matches a configured regex wake-word pattern
|
||||
|
||||
When ``require_mention`` is enabled, slash commands are not given
|
||||
When ``allowed_chats`` is non-empty, it acts as a hard gate — messages
|
||||
from any chat not in the list are ignored regardless of the other
|
||||
rules. When ``require_mention`` is enabled, slash commands are not given
|
||||
special treatment — they must pass the same mention/reply checks
|
||||
as any other group message. Users can still trigger commands via
|
||||
the Telegram bot menu (``/command@botname``) or by explicitly
|
||||
@@ -2918,6 +2955,14 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
"""
|
||||
if not self._is_group_chat(message):
|
||||
return True
|
||||
# allowed_chats check (whitelist — must pass before other gating).
|
||||
# When set, group messages from chats NOT in this whitelist are
|
||||
# silently ignored, even if @mentioned. DMs are already excluded above.
|
||||
allowed = self._telegram_allowed_chats()
|
||||
if allowed:
|
||||
chat_id_str = str(getattr(getattr(message, "chat", None), "id", ""))
|
||||
if chat_id_str not in allowed:
|
||||
return False
|
||||
thread_id = getattr(message, "message_thread_id", None)
|
||||
if thread_id is not None:
|
||||
try:
|
||||
@@ -3239,10 +3284,59 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
_, ext = os.path.splitext(original_filename)
|
||||
ext = ext.lower()
|
||||
|
||||
# Normalize mime_type for robust comparisons (some clients send
|
||||
# uppercase like "IMAGE/PNG").
|
||||
doc_mime = (doc.mime_type or "").lower()
|
||||
|
||||
# If no extension from filename, reverse-lookup from MIME type
|
||||
if not ext and doc.mime_type:
|
||||
mime_to_ext = {v: k for k, v in SUPPORTED_DOCUMENT_TYPES.items()}
|
||||
ext = mime_to_ext.get(doc.mime_type, "")
|
||||
if not ext and doc_mime:
|
||||
ext = _TELEGRAM_IMAGE_MIME_TO_EXT.get(doc_mime, "")
|
||||
if not ext:
|
||||
mime_to_ext = {v: k for k, v in SUPPORTED_DOCUMENT_TYPES.items()}
|
||||
ext = mime_to_ext.get(doc_mime, "")
|
||||
|
||||
# Check file size early so image documents cannot bypass the
|
||||
# document size limit by taking the image path.
|
||||
MAX_DOC_BYTES = 20 * 1024 * 1024
|
||||
if not doc.file_size or doc.file_size > MAX_DOC_BYTES:
|
||||
event.text = (
|
||||
"The document is too large or its size could not be verified. "
|
||||
"Maximum: 20 MB."
|
||||
)
|
||||
logger.info("[Telegram] Document too large: %s bytes", doc.file_size)
|
||||
await self.handle_message(event)
|
||||
return
|
||||
|
||||
# Telegram may deliver screenshots/photos as documents. If the
|
||||
# payload is actually an image, route it through the image cache
|
||||
# and batching path instead of rejecting it as a document.
|
||||
if ext in _TELEGRAM_IMAGE_EXTENSIONS or doc_mime.startswith("image/"):
|
||||
file_obj = await doc.get_file()
|
||||
image_bytes = await file_obj.download_as_bytearray()
|
||||
image_ext = ext if ext in _TELEGRAM_IMAGE_EXTENSIONS else _TELEGRAM_IMAGE_MIME_TO_EXT.get(doc_mime, ".jpg")
|
||||
try:
|
||||
cached_path = cache_image_from_bytes(bytes(image_bytes), ext=image_ext)
|
||||
except ValueError as e:
|
||||
logger.warning("[Telegram] Failed to cache image document: %s", e, exc_info=True)
|
||||
event.text = (
|
||||
f"Image document '{original_filename or doc_mime or ext or 'unknown'}' "
|
||||
"could not be read as an image."
|
||||
)
|
||||
await self.handle_message(event)
|
||||
return
|
||||
|
||||
event.message_type = MessageType.PHOTO
|
||||
event.media_urls = [cached_path]
|
||||
event.media_types = [doc_mime if doc_mime.startswith("image/") else _TELEGRAM_IMAGE_EXT_TO_MIME.get(image_ext, "image/jpeg")]
|
||||
logger.info("[Telegram] Cached user image-document at %s", cached_path)
|
||||
|
||||
media_group_id = getattr(msg, "media_group_id", None)
|
||||
if media_group_id:
|
||||
await self._queue_media_group_event(str(media_group_id), event)
|
||||
else:
|
||||
batch_key = self._photo_batch_key(event, msg)
|
||||
self._enqueue_photo_event(batch_key, event)
|
||||
return
|
||||
|
||||
if not ext and doc.mime_type:
|
||||
video_mime_to_ext = {v: k for k, v in SUPPORTED_VIDEO_TYPES.items()}
|
||||
@@ -3270,17 +3364,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
await self.handle_message(event)
|
||||
return
|
||||
|
||||
# Check file size (Telegram Bot API limit: 20 MB)
|
||||
MAX_DOC_BYTES = 20 * 1024 * 1024
|
||||
if not doc.file_size or doc.file_size > MAX_DOC_BYTES:
|
||||
event.text = (
|
||||
"The document is too large or its size could not be verified. "
|
||||
"Maximum: 20 MB."
|
||||
)
|
||||
logger.info("[Telegram] Document too large: %s bytes", doc.file_size)
|
||||
await self.handle_message(event)
|
||||
return
|
||||
|
||||
# Download and cache
|
||||
file_obj = await doc.get_file()
|
||||
doc_bytes = await file_obj.download_as_bytearray()
|
||||
@@ -3433,7 +3516,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
return
|
||||
|
||||
import yaml as _yaml
|
||||
with open(config_path, "r") as f:
|
||||
with open(config_path, "r", encoding="utf-8") as f:
|
||||
config = _yaml.safe_load(f) or {}
|
||||
|
||||
dm_topics = (
|
||||
|
||||
@@ -59,6 +59,29 @@ DEFAULT_PORT = 8644
|
||||
_INSECURE_NO_AUTH = "INSECURE_NO_AUTH"
|
||||
_DYNAMIC_ROUTES_FILENAME = "webhook_subscriptions.json"
|
||||
|
||||
# Hostnames/IP literals that only serve connections originating on the same
|
||||
# machine. Anything else is treated as a public bind for safety-rail purposes.
|
||||
_LOOPBACK_HOSTS = frozenset({
|
||||
"127.0.0.1",
|
||||
"localhost",
|
||||
"::1",
|
||||
"ip6-localhost",
|
||||
"ip6-loopback",
|
||||
})
|
||||
|
||||
|
||||
def _is_loopback_host(host: str) -> bool:
|
||||
"""True when `host` binds only to the local machine.
|
||||
|
||||
Covers IPv4 loopback, the standard `localhost` alias, IPv6 loopback in
|
||||
both bracketed and bare form, and the common Debian-style aliases. Any
|
||||
falsy value (empty string, None) is conservatively treated as non-loopback
|
||||
because an unset host usually means the platform-default public bind.
|
||||
"""
|
||||
if not host:
|
||||
return False
|
||||
return host.strip().lower() in _LOOPBACK_HOSTS
|
||||
|
||||
|
||||
def check_webhook_requirements() -> bool:
|
||||
"""Check if webhook adapter dependencies are available."""
|
||||
@@ -126,6 +149,17 @@ class WebhookAdapter(BasePlatformAdapter):
|
||||
f"For testing without auth, set secret to '{_INSECURE_NO_AUTH}'."
|
||||
)
|
||||
|
||||
# Safety rail: refuse to start if INSECURE_NO_AUTH is combined with a
|
||||
# non-loopback bind. The escape hatch is for local testing only;
|
||||
# serving an unauthenticated route on a public interface is a
|
||||
# deployment-grade footgun we'd rather crash early than ship.
|
||||
if secret == _INSECURE_NO_AUTH and not _is_loopback_host(self._host):
|
||||
raise ValueError(
|
||||
f"[webhook] Route '{name}' uses INSECURE_NO_AUTH secret "
|
||||
f"but is bound to non-loopback host '{self._host}'. "
|
||||
f"INSECURE_NO_AUTH is for local testing only. "
|
||||
f"Refusing to start to prevent accidental exposure."
|
||||
)
|
||||
# deliver_only routes bypass the agent — the POST body becomes a
|
||||
# direct push notification via the configured delivery target.
|
||||
# Validate up-front so misconfiguration surfaces at startup rather
|
||||
|
||||
@@ -37,6 +37,7 @@ import logging
|
||||
import mimetypes
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
@@ -1562,12 +1563,11 @@ def qr_scan_for_bot_info(
|
||||
print(" Fetching configuration results...", end="", flush=True)
|
||||
|
||||
# ── Step 3: Poll for result ──
|
||||
import time
|
||||
deadline = time.time() + timeout_seconds
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
query_url = f"{_QR_QUERY_URL}?scode={urllib.parse.quote(scode)}"
|
||||
poll_count = 0
|
||||
|
||||
while time.time() < deadline:
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
req = urllib.request.Request(query_url, headers={"User-Agent": "HermesAgent/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
|
||||
+81
-22
@@ -23,6 +23,7 @@ import re
|
||||
import secrets
|
||||
import struct
|
||||
import tempfile
|
||||
import textwrap
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
@@ -32,6 +33,8 @@ from urllib.parse import quote, urlparse
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
WEIXIN_COPY_LINE_WIDTH = 120
|
||||
|
||||
try:
|
||||
import aiohttp
|
||||
|
||||
@@ -548,17 +551,21 @@ async def _upload_ciphertext(
|
||||
Accepts either a constructed CDN URL (from upload_param) or a direct
|
||||
upload_full_url — both use POST with the raw ciphertext as the body.
|
||||
"""
|
||||
timeout = aiohttp.ClientTimeout(total=120)
|
||||
async with session.post(upload_url, data=ciphertext, headers={"Content-Type": "application/octet-stream"}, timeout=timeout) as response:
|
||||
if response.status == 200:
|
||||
encrypted_param = response.headers.get("x-encrypted-param")
|
||||
if encrypted_param:
|
||||
await response.read()
|
||||
return encrypted_param
|
||||
# Use asyncio.wait_for() instead of aiohttp ClientTimeout to avoid
|
||||
# "Timeout context manager should be used inside a task" errors when
|
||||
# invoked via asyncio.run_coroutine_threadsafe() from cron jobs.
|
||||
async def _do_upload() -> str:
|
||||
async with session.post(upload_url, data=ciphertext, headers={"Content-Type": "application/octet-stream"}) as response:
|
||||
if response.status == 200:
|
||||
encrypted_param = response.headers.get("x-encrypted-param")
|
||||
if encrypted_param:
|
||||
await response.read()
|
||||
return encrypted_param
|
||||
raw = await response.text()
|
||||
raise RuntimeError(f"CDN upload missing x-encrypted-param header: {raw[:200]}")
|
||||
raw = await response.text()
|
||||
raise RuntimeError(f"CDN upload missing x-encrypted-param header: {raw[:200]}")
|
||||
raw = await response.text()
|
||||
raise RuntimeError(f"CDN upload HTTP {response.status}: {raw[:200]}")
|
||||
raise RuntimeError(f"CDN upload HTTP {response.status}: {raw[:200]}")
|
||||
return await asyncio.wait_for(_do_upload(), timeout=120)
|
||||
|
||||
|
||||
async def _download_bytes(
|
||||
@@ -567,10 +574,13 @@ async def _download_bytes(
|
||||
url: str,
|
||||
timeout_seconds: float = 60.0,
|
||||
) -> bytes:
|
||||
timeout = aiohttp.ClientTimeout(total=timeout_seconds)
|
||||
async with session.get(url, timeout=timeout) as response:
|
||||
response.raise_for_status()
|
||||
return await response.read()
|
||||
# Use asyncio.wait_for() instead of aiohttp ClientTimeout to avoid
|
||||
# "Timeout context manager should be used inside a task" errors.
|
||||
async def _do_download() -> bytes:
|
||||
async with session.get(url) as response:
|
||||
response.raise_for_status()
|
||||
return await response.read()
|
||||
return await asyncio.wait_for(_do_download(), timeout=timeout_seconds)
|
||||
|
||||
|
||||
_WEIXIN_CDN_ALLOWLIST: frozenset[str] = frozenset(
|
||||
@@ -724,6 +734,46 @@ def _normalize_markdown_blocks(content: str) -> str:
|
||||
return "\n".join(result).strip()
|
||||
|
||||
|
||||
def _wrap_copy_friendly_lines_for_weixin(content: str) -> str:
|
||||
"""Wrap long display lines that are hard to copy in WeChat clients."""
|
||||
if not content:
|
||||
return content
|
||||
|
||||
wrapped: List[str] = []
|
||||
in_code_block = False
|
||||
|
||||
for raw_line in content.splitlines():
|
||||
line = raw_line.rstrip()
|
||||
stripped = line.strip()
|
||||
|
||||
if _FENCE_RE.match(stripped):
|
||||
in_code_block = not in_code_block
|
||||
wrapped.append(line)
|
||||
continue
|
||||
|
||||
if (
|
||||
in_code_block
|
||||
or len(line) <= WEIXIN_COPY_LINE_WIDTH
|
||||
or not stripped
|
||||
or stripped.startswith("|")
|
||||
or _TABLE_RULE_RE.match(stripped)
|
||||
):
|
||||
wrapped.append(line)
|
||||
continue
|
||||
|
||||
wrapped_lines = textwrap.wrap(
|
||||
line,
|
||||
width=WEIXIN_COPY_LINE_WIDTH,
|
||||
break_long_words=False,
|
||||
break_on_hyphens=False,
|
||||
replace_whitespace=False,
|
||||
drop_whitespace=True,
|
||||
)
|
||||
wrapped.extend(wrapped_lines or [line])
|
||||
|
||||
return "\n".join(wrapped).strip()
|
||||
|
||||
|
||||
def _split_markdown_blocks(content: str) -> List[str]:
|
||||
if not content:
|
||||
return []
|
||||
@@ -1037,11 +1087,11 @@ async def qr_login(
|
||||
except Exception as _qr_exc:
|
||||
print(f"(终端二维码渲染失败: {_qr_exc},请直接打开上面的二维码链接)")
|
||||
|
||||
deadline = time.time() + timeout_seconds
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
current_base_url = ILINK_BASE_URL
|
||||
refresh_count = 0
|
||||
|
||||
while time.time() < deadline:
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
status_resp = await _api_get(
|
||||
session,
|
||||
@@ -1216,7 +1266,12 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
logger.debug("[%s] Token lock unavailable (non-fatal): %s", self.name, exc)
|
||||
|
||||
self._poll_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
||||
self._send_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
||||
# Disable aiohttp's built-in ClientTimeout (total=None) to prevent
|
||||
# "Timeout context manager should be used inside a task" errors when
|
||||
# send() is invoked via asyncio.run_coroutine_threadsafe() from cron.
|
||||
# Timeout is managed externally via asyncio.wait_for() in _api_post/_api_get.
|
||||
_no_aiohttp_timeout = aiohttp.ClientTimeout(total=None, connect=None, sock_connect=None, sock_read=None)
|
||||
self._send_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector(), timeout=_no_aiohttp_timeout)
|
||||
self._token_store.restore(self._account_id)
|
||||
self._poll_task = asyncio.create_task(self._poll_loop(), name="weixin-poll")
|
||||
self._mark_connected()
|
||||
@@ -1824,10 +1879,14 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
raise ValueError(f"Blocked unsafe URL (SSRF protection): {url}")
|
||||
|
||||
assert self._send_session is not None
|
||||
async with self._send_session.get(url, timeout=aiohttp.ClientTimeout(total=30)) as response:
|
||||
response.raise_for_status()
|
||||
data = await response.read()
|
||||
suffix = Path(url.split("?", 1)[0]).suffix or ".bin"
|
||||
# Use asyncio.wait_for() instead of aiohttp ClientTimeout to avoid
|
||||
# "Timeout context manager should be used inside a task" errors.
|
||||
async def _do_fetch():
|
||||
async with self._send_session.get(url) as response:
|
||||
response.raise_for_status()
|
||||
return await response.read()
|
||||
data = await asyncio.wait_for(_do_fetch(), timeout=30)
|
||||
suffix = Path(url.split("?", 1)[0]).suffix or ".bin"
|
||||
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as handle:
|
||||
handle.write(data)
|
||||
return handle.name
|
||||
@@ -2006,7 +2065,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
def format_message(self, content: Optional[str]) -> str:
|
||||
if content is None:
|
||||
return ""
|
||||
return _normalize_markdown_blocks(content)
|
||||
return _wrap_copy_friendly_lines_for_weixin(_normalize_markdown_blocks(content))
|
||||
|
||||
|
||||
async def send_weixin_direct(
|
||||
|
||||
+110
-13
@@ -21,6 +21,8 @@ import logging
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
|
||||
_IS_WINDOWS = platform.system() == "Windows"
|
||||
@@ -54,19 +56,77 @@ def _kill_port_process(port: int) -> None:
|
||||
except subprocess.SubprocessError:
|
||||
pass
|
||||
else:
|
||||
result = subprocess.run(
|
||||
["fuser", f"{port}/tcp"],
|
||||
capture_output=True, timeout=5,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
subprocess.run(
|
||||
["fuser", "-k", f"{port}/tcp"],
|
||||
# Try fuser first (Linux), fall back to lsof (macOS / WSL2)
|
||||
killed = False
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["fuser", f"{port}/tcp"],
|
||||
capture_output=True, timeout=5,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
subprocess.run(
|
||||
["fuser", "-k", f"{port}/tcp"],
|
||||
capture_output=True, timeout=5,
|
||||
)
|
||||
killed = True
|
||||
except FileNotFoundError:
|
||||
pass # fuser not installed
|
||||
|
||||
if not killed:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["lsof", "-ti", f":{port}"],
|
||||
capture_output=True, text=True, timeout=5,
|
||||
)
|
||||
for pid_str in result.stdout.strip().splitlines():
|
||||
try:
|
||||
os.kill(int(pid_str), signal.SIGTERM)
|
||||
except (ValueError, ProcessLookupError, PermissionError):
|
||||
pass
|
||||
except FileNotFoundError:
|
||||
pass # lsof not installed either
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _kill_stale_bridge_by_pidfile(session_path: Path) -> None:
|
||||
"""Kill a bridge process recorded in a PID file from a previous run.
|
||||
|
||||
The bridge writes ``bridge.pid`` into the session directory when it
|
||||
starts. If the gateway crashed without a clean shutdown the old bridge
|
||||
process becomes orphaned — this helper finds and kills it.
|
||||
"""
|
||||
pid_file = session_path / "bridge.pid"
|
||||
if not pid_file.exists():
|
||||
return
|
||||
try:
|
||||
pid = int(pid_file.read_text().strip())
|
||||
except (ValueError, OSError, TypeError):
|
||||
try:
|
||||
pid_file.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
return
|
||||
try:
|
||||
os.kill(pid, 0) # check existence
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
logger.info("[whatsapp] Killed stale bridge PID %d from pidfile", pid)
|
||||
except (ProcessLookupError, PermissionError, OSError):
|
||||
pass
|
||||
try:
|
||||
pid_file.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _write_bridge_pidfile(session_path: Path, pid: int) -> None:
|
||||
"""Write the bridge PID to a file for later cleanup."""
|
||||
try:
|
||||
(session_path / "bridge.pid").write_text(str(pid))
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _terminate_bridge_process(proc, *, force: bool = False) -> None:
|
||||
"""Terminate the bridge process using process-tree semantics where possible."""
|
||||
if _IS_WINDOWS:
|
||||
@@ -118,10 +178,15 @@ def check_whatsapp_requirements() -> bool:
|
||||
|
||||
WhatsApp requires a Node.js bridge for most implementations.
|
||||
"""
|
||||
# Check for Node.js
|
||||
# Check for Node.js. Resolve via shutil.which so we respect PATHEXT
|
||||
# (node.exe vs node) and get a meaningful "not installed" signal
|
||||
# instead of spawning a cmd flash on Windows.
|
||||
_node = shutil.which("node")
|
||||
if not _node:
|
||||
return False
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["node", "--version"],
|
||||
[_node, "--version"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5
|
||||
@@ -158,6 +223,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
# WhatsApp message limits — practical UX limit, not protocol max.
|
||||
# WhatsApp allows ~65K but long messages are unreadable on mobile.
|
||||
MAX_MESSAGE_LENGTH = 4096
|
||||
DEFAULT_REPLY_PREFIX = "⚕ *Hermes Agent*\n────────────\n"
|
||||
|
||||
# Default bridge location relative to the hermes-agent install
|
||||
_DEFAULT_BRIDGE_DIR = Path(__file__).resolve().parents[2] / "scripts" / "whatsapp-bridge"
|
||||
@@ -193,6 +259,25 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
# notification before the normal "✓ whatsapp disconnected" fires.
|
||||
self._shutting_down: bool = False
|
||||
|
||||
def _effective_reply_prefix(self) -> str:
|
||||
"""Return the prefix the Node bridge will add in self-chat mode."""
|
||||
whatsapp_mode = os.getenv("WHATSAPP_MODE", "self-chat")
|
||||
if whatsapp_mode != "self-chat":
|
||||
return ""
|
||||
if self._reply_prefix is not None:
|
||||
return self._reply_prefix.replace("\\n", "\n")
|
||||
env_prefix = os.getenv("WHATSAPP_REPLY_PREFIX")
|
||||
if env_prefix is not None:
|
||||
return env_prefix.replace("\\n", "\n")
|
||||
return self.DEFAULT_REPLY_PREFIX
|
||||
|
||||
def _outgoing_chunk_limit(self) -> int:
|
||||
"""Reserve room for the bridge-side prefix so final WhatsApp text fits."""
|
||||
prefix_len = len(self._effective_reply_prefix())
|
||||
# Keep enough space for truncate_message's pagination indicator and
|
||||
# code-fence repair even if a user configures a very long prefix.
|
||||
return max(1024, self.MAX_MESSAGE_LENGTH - prefix_len)
|
||||
|
||||
def _whatsapp_require_mention(self) -> bool:
|
||||
configured = self.config.extra.get("require_mention")
|
||||
if configured is not None:
|
||||
@@ -385,9 +470,13 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
bridge_dir = bridge_path.parent
|
||||
if not (bridge_dir / "node_modules").exists():
|
||||
print(f"[{self.name}] Installing WhatsApp bridge dependencies...")
|
||||
# Resolve npm path so Windows can execute the .cmd shim.
|
||||
# shutil.which honours PATHEXT; on POSIX it returns the
|
||||
# plain executable path.
|
||||
_npm_bin = shutil.which("npm") or "npm"
|
||||
try:
|
||||
install_result = subprocess.run(
|
||||
["npm", "install", "--silent"],
|
||||
[_npm_bin, "install", "--silent"],
|
||||
cwd=str(bridge_dir),
|
||||
capture_output=True,
|
||||
text=True,
|
||||
@@ -428,6 +517,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
pass # Bridge not running, start a new one
|
||||
|
||||
# Kill any orphaned bridge from a previous gateway run
|
||||
_kill_stale_bridge_by_pidfile(self._session_path)
|
||||
_kill_port_process(self._bridge_port)
|
||||
await asyncio.sleep(1)
|
||||
|
||||
@@ -436,7 +526,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
# messages are preserved for troubleshooting.
|
||||
whatsapp_mode = os.getenv("WHATSAPP_MODE", "self-chat")
|
||||
self._bridge_log = self._session_path.parent / "bridge.log"
|
||||
bridge_log_fh = open(self._bridge_log, "a")
|
||||
bridge_log_fh = open(self._bridge_log, "a", encoding="utf-8")
|
||||
self._bridge_log_fh = bridge_log_fh
|
||||
|
||||
# Build bridge subprocess environment.
|
||||
@@ -459,6 +549,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
preexec_fn=None if _IS_WINDOWS else os.setsid,
|
||||
env=bridge_env,
|
||||
)
|
||||
_write_bridge_pidfile(self._session_path, self._bridge_process.pid)
|
||||
|
||||
# Wait for the bridge to connect to WhatsApp.
|
||||
# Phase 1: wait for the HTTP server to come up (up to 15s).
|
||||
@@ -609,6 +700,12 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
# Bridge was not started by us, don't kill it
|
||||
print(f"[{self.name}] Disconnecting (external bridge left running)")
|
||||
|
||||
# Clean up PID file
|
||||
try:
|
||||
(self._session_path / "bridge.pid").unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
# Cancel the poll task explicitly
|
||||
if self._poll_task and not self._poll_task.done():
|
||||
self._poll_task.cancel()
|
||||
@@ -713,7 +810,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
|
||||
# Format and chunk the message
|
||||
formatted = self.format_message(content)
|
||||
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
|
||||
chunks = self.truncate_message(formatted, self._outgoing_chunk_limit())
|
||||
|
||||
last_message_id = None
|
||||
for chunk in chunks:
|
||||
@@ -1073,7 +1170,7 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
if file_size > MAX_TEXT_INJECT_BYTES:
|
||||
print(f"[{self.name}] Skipping text injection for {doc_path} ({file_size} bytes > {MAX_TEXT_INJECT_BYTES})", flush=True)
|
||||
continue
|
||||
content = Path(doc_path).read_text(errors="replace")
|
||||
content = Path(doc_path).read_text(encoding="utf-8", errors="replace")
|
||||
fname = Path(doc_path).name
|
||||
# Remove the doc_<hex>_ prefix for display
|
||||
display_name = fname
|
||||
|
||||
+538
-68
@@ -13,6 +13,10 @@ Usage:
|
||||
python cli.py --gateway
|
||||
"""
|
||||
|
||||
# IMPORTANT: hermes_bootstrap must be the very first import — UTF-8 stdio
|
||||
# on Windows. No-op on POSIX. See hermes_bootstrap.py for full rationale.
|
||||
import hermes_bootstrap # noqa: F401
|
||||
|
||||
import asyncio
|
||||
import dataclasses
|
||||
import inspect
|
||||
@@ -258,13 +262,18 @@ def _ensure_ssl_certs() -> None:
|
||||
return
|
||||
|
||||
def _home_target_env_var(platform_name: str) -> str:
|
||||
"""Return the configured home-target env var for a platform."""
|
||||
from cron.scheduler import _HOME_TARGET_ENV_VARS
|
||||
"""Return the configured home-target env var for a platform.
|
||||
|
||||
return _HOME_TARGET_ENV_VARS.get(
|
||||
platform_name.lower(),
|
||||
f"{platform_name.upper()}_HOME_CHANNEL",
|
||||
)
|
||||
Consults built-in ``_HOME_TARGET_ENV_VARS`` first, then the plugin
|
||||
registry via ``cron.scheduler._resolve_home_env_var``, then falls back
|
||||
to ``<PLATFORM>_HOME_CHANNEL`` for unknown names.
|
||||
"""
|
||||
from cron.scheduler import _resolve_home_env_var
|
||||
|
||||
resolved = _resolve_home_env_var(platform_name)
|
||||
if resolved:
|
||||
return resolved
|
||||
return f"{platform_name.upper()}_HOME_CHANNEL"
|
||||
|
||||
|
||||
def _home_thread_env_var(platform_name: str) -> str:
|
||||
@@ -299,6 +308,36 @@ _env_path = _hermes_home / '.env'
|
||||
load_hermes_dotenv(hermes_home=_hermes_home, project_env=Path(__file__).resolve().parents[1] / '.env')
|
||||
|
||||
|
||||
def _reload_runtime_env_preserving_config_authority() -> None:
|
||||
"""Reload .env for fresh credentials without letting stale .env override config.
|
||||
|
||||
Gateway processes are long-lived, so per-turn code reloads ~/.hermes/.env to
|
||||
pick up rotated API keys. config.yaml remains authoritative for agent budget
|
||||
settings such as agent.max_turns; otherwise a stale HERMES_MAX_ITERATIONS in
|
||||
.env can replace the startup bridge on later turns.
|
||||
"""
|
||||
load_hermes_dotenv(
|
||||
hermes_home=_hermes_home,
|
||||
project_env=Path(__file__).resolve().parents[1] / '.env',
|
||||
)
|
||||
|
||||
config_path = _hermes_home / 'config.yaml'
|
||||
if not config_path.exists():
|
||||
return
|
||||
try:
|
||||
import yaml as _yaml
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
cfg = _yaml.safe_load(f) or {}
|
||||
from hermes_cli.config import _expand_env_vars
|
||||
cfg = _expand_env_vars(cfg)
|
||||
except Exception:
|
||||
return
|
||||
|
||||
agent_cfg = cfg.get("agent", {})
|
||||
if isinstance(agent_cfg, dict) and "max_turns" in agent_cfg:
|
||||
os.environ["HERMES_MAX_ITERATIONS"] = str(agent_cfg["max_turns"])
|
||||
|
||||
|
||||
_DOCKER_VOLUME_SPEC_RE = re.compile(r"^(?P<host>.+):(?P<container>/[^:]+?)(?::(?P<options>[^:]+))?$")
|
||||
_DOCKER_MEDIA_OUTPUT_CONTAINER_PATHS = {"/output", "/outputs"}
|
||||
|
||||
@@ -468,22 +507,22 @@ try:
|
||||
_network_cfg = (_cfg if '_cfg' in dir() else {}).get("network", {})
|
||||
if isinstance(_network_cfg, dict) and _network_cfg.get("force_ipv4"):
|
||||
apply_ipv4_preference(force=True)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _bootstrap_exc:
|
||||
print(f" Warning: IPv4 preference application failed: {_bootstrap_exc}", file=sys.stderr)
|
||||
|
||||
# Validate config structure early — log warnings so gateway operators see problems
|
||||
try:
|
||||
from hermes_cli.config import print_config_warnings
|
||||
print_config_warnings()
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _bootstrap_exc:
|
||||
print(f" Warning: config validation failed: {_bootstrap_exc}", file=sys.stderr)
|
||||
|
||||
# Warn if user has deprecated MESSAGING_CWD / TERMINAL_CWD in .env
|
||||
try:
|
||||
from hermes_cli.config import warn_deprecated_cwd_env_vars
|
||||
warn_deprecated_cwd_env_vars()
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _bootstrap_exc:
|
||||
print(f" Warning: deprecation check failed: {_bootstrap_exc}", file=sys.stderr)
|
||||
|
||||
# Gateway runs in quiet mode - suppress debug output and use cwd directly (no temp dirs)
|
||||
os.environ["HERMES_QUIET"] = "1"
|
||||
@@ -613,7 +652,11 @@ def _try_resolve_fallback_provider() -> dict | None:
|
||||
explicit_base_url=entry.get("base_url"),
|
||||
explicit_api_key=entry.get("api_key"),
|
||||
)
|
||||
logger.info("Fallback provider resolved: %s", runtime.get("provider"))
|
||||
logger.info(
|
||||
"Fallback provider resolved: %s model=%s",
|
||||
runtime.get("provider"),
|
||||
entry.get("model"),
|
||||
)
|
||||
return {
|
||||
"api_key": runtime.get("api_key"),
|
||||
"base_url": runtime.get("base_url"),
|
||||
@@ -622,6 +665,7 @@ def _try_resolve_fallback_provider() -> dict | None:
|
||||
"command": runtime.get("command"),
|
||||
"args": list(runtime.get("args") or []),
|
||||
"credential_pool": runtime.get("credential_pool"),
|
||||
"model": entry.get("model"),
|
||||
}
|
||||
except Exception as fb_exc:
|
||||
logger.debug("Fallback entry %s failed: %s", entry.get("provider"), fb_exc)
|
||||
@@ -985,6 +1029,26 @@ def _normalize_empty_agent_response(
|
||||
return response
|
||||
|
||||
|
||||
def _should_clear_resume_pending_after_turn(agent_result: dict) -> bool:
|
||||
"""Return True only when a gateway turn really completed successfully.
|
||||
|
||||
Restart recovery uses ``resume_pending`` as a durable marker for sessions
|
||||
interrupted during gateway drain. A soft interrupt can still bubble out as
|
||||
a syntactically normal agent result with an empty final response; clearing
|
||||
the marker in that case loses the recovery signal and startup auto-resume
|
||||
has nothing to schedule.
|
||||
"""
|
||||
if not isinstance(agent_result, dict):
|
||||
return False
|
||||
if agent_result.get("interrupted"):
|
||||
return False
|
||||
if agent_result.get("failed") or agent_result.get("partial") or agent_result.get("error"):
|
||||
return False
|
||||
if agent_result.get("completed") is False:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
class GatewayRunner:
|
||||
"""
|
||||
Main gateway controller.
|
||||
@@ -1066,6 +1130,13 @@ class GatewayRunner:
|
||||
self._pending_native_image_paths_by_session: Dict[str, List[str]] = {}
|
||||
self._busy_ack_ts: Dict[str, float] = {} # last busy-ack timestamp per session (debounce)
|
||||
self._session_run_generation: Dict[str, int] = {}
|
||||
# LRU cache of live SessionSources keyed by session_key. Used by
|
||||
# fallback routing paths (shutdown notifications, synthetic
|
||||
# background-process events) when the persisted origin is missing
|
||||
# and _parse_session_key can't recover thread_id. Capped so it
|
||||
# cannot grow unbounded over a long-running gateway lifetime.
|
||||
self._session_sources: "OrderedDict[str, SessionSource]" = OrderedDict()
|
||||
self._session_sources_max = 512
|
||||
|
||||
# Cache AIAgent instances per session to preserve prompt caching.
|
||||
# Without this, a new AIAgent is created per message, rebuilding the
|
||||
@@ -1604,6 +1675,14 @@ class GatewayRunner:
|
||||
)
|
||||
|
||||
runtime_kwargs = _resolve_runtime_agent_kwargs()
|
||||
runtime_model = runtime_kwargs.pop("model", None)
|
||||
if runtime_model:
|
||||
logger.info(
|
||||
"Runtime provider supplied explicit model override: %s -> %s",
|
||||
model,
|
||||
runtime_model,
|
||||
)
|
||||
model = runtime_model
|
||||
if override and resolved_session_key:
|
||||
model, runtime_kwargs = self._apply_session_model_override(
|
||||
resolved_session_key, model, runtime_kwargs
|
||||
@@ -2431,6 +2510,9 @@ class GatewayRunner:
|
||||
e,
|
||||
)
|
||||
|
||||
if source is None:
|
||||
source = self._get_cached_session_source(session_key)
|
||||
|
||||
if source is not None:
|
||||
platform_str = source.platform.value
|
||||
chat_id = str(source.chat_id)
|
||||
@@ -2491,7 +2573,12 @@ class GatewayRunner:
|
||||
platform_str, chat_id, e,
|
||||
)
|
||||
|
||||
for platform, adapter in self.adapters.items():
|
||||
# Snapshot adapters up front: adapter.send() can hit a fatal error
|
||||
# path that pops the adapter from self.adapters (see _handle_fatal
|
||||
# elsewhere), which would otherwise trigger
|
||||
# ``RuntimeError: dictionary changed size during iteration`` —
|
||||
# observed in a user report during gateway shutdown.
|
||||
for platform, adapter in list(self.adapters.items()):
|
||||
home = self.config.get_home_channel(platform)
|
||||
if not home or not home.chat_id:
|
||||
continue
|
||||
@@ -2701,6 +2788,48 @@ class GatewayRunner:
|
||||
return
|
||||
|
||||
current_pid = os.getpid()
|
||||
|
||||
# On Windows there's no bash/setsid chain — spawn a tiny Python
|
||||
# watcher directly via sys.executable instead. The watcher polls
|
||||
# current_pid, waits for our exit, then runs `hermes gateway
|
||||
# restart` with detach flags so the respawn survives the CLI
|
||||
# that triggered the /restart command closing its console.
|
||||
if sys.platform == "win32":
|
||||
import textwrap
|
||||
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
|
||||
|
||||
cmd_argv = [*hermes_cmd, "gateway", "restart"]
|
||||
watcher = textwrap.dedent(
|
||||
"""
|
||||
import os, subprocess, sys, time
|
||||
pid = int(sys.argv[1])
|
||||
cmd = sys.argv[2:]
|
||||
deadline = time.monotonic() + 120
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except (ProcessLookupError, PermissionError, OSError):
|
||||
break
|
||||
time.sleep(0.2)
|
||||
_CREATE_NEW_PROCESS_GROUP = 0x00000200
|
||||
_DETACHED_PROCESS = 0x00000008
|
||||
_CREATE_NO_WINDOW = 0x08000000
|
||||
subprocess.Popen(
|
||||
cmd,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
creationflags=_CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW,
|
||||
)
|
||||
"""
|
||||
).strip()
|
||||
subprocess.Popen(
|
||||
[sys.executable, "-c", watcher, str(current_pid), *cmd_argv],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
**windows_detach_popen_kwargs(),
|
||||
)
|
||||
return
|
||||
|
||||
cmd = " ".join(shlex.quote(part) for part in hermes_cmd)
|
||||
shell_cmd = (
|
||||
f"while kill -0 {current_pid} 2>/dev/null; do sleep 0.2; done; "
|
||||
@@ -2739,6 +2868,83 @@ class GatewayRunner:
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
return True
|
||||
|
||||
# Drain-timeout reasons set by _stop_impl() when a still-running turn is
|
||||
# force-interrupted; "restart_interrupted" is set by
|
||||
# SessionStore.suspend_recently_active() on crash recovery (no
|
||||
# .clean_shutdown marker). All three mean "the agent was mid-turn and
|
||||
# we killed it" — eligible for startup auto-resume.
|
||||
_AUTO_RESUME_REASONS = frozenset(
|
||||
{"restart_timeout", "shutdown_timeout", "restart_interrupted"}
|
||||
)
|
||||
|
||||
def _schedule_resume_pending_sessions(self) -> int:
|
||||
"""Auto-continue fresh restart-interrupted sessions after startup.
|
||||
|
||||
``resume_pending`` already preserves the transcript AND the existing
|
||||
``_is_resume_pending`` branch in ``_handle_message_with_agent``
|
||||
injects a reason-aware recovery system note on the next turn. This
|
||||
method closes the UX gap by synthesizing that next turn once
|
||||
adapters are back online — the event text is empty so the existing
|
||||
injection path owns the wording and we never double up.
|
||||
|
||||
Adapters that are not yet ready (adapter missing from
|
||||
``self.adapters``) are skipped silently; their sessions stay
|
||||
``resume_pending`` and will auto-resume on the next real user
|
||||
message, or on the next gateway startup.
|
||||
"""
|
||||
window = _auto_continue_freshness_window()
|
||||
try:
|
||||
with self.session_store._lock: # noqa: SLF001 — snapshot under lock
|
||||
self.session_store._ensure_loaded_locked() # noqa: SLF001
|
||||
candidates = [
|
||||
entry for entry in self.session_store._entries.values() # noqa: SLF001
|
||||
if entry.resume_pending
|
||||
and not entry.suspended
|
||||
and entry.origin is not None
|
||||
and entry.resume_reason in self._AUTO_RESUME_REASONS
|
||||
]
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to enumerate resume-pending sessions: %s", exc)
|
||||
return 0
|
||||
|
||||
now = datetime.now()
|
||||
scheduled = 0
|
||||
for entry in candidates:
|
||||
marker = entry.last_resume_marked_at or entry.updated_at
|
||||
if marker is not None and (now - marker).total_seconds() > window:
|
||||
continue
|
||||
|
||||
source = entry.origin
|
||||
adapter = self.adapters.get(source.platform)
|
||||
if adapter is None:
|
||||
logger.debug(
|
||||
"Skipping auto-resume for %s: adapter not ready for %s",
|
||||
entry.session_key,
|
||||
getattr(source.platform, "value", source.platform),
|
||||
)
|
||||
continue
|
||||
|
||||
# Empty-text internal event — the _is_resume_pending branch in
|
||||
# _handle_message_with_agent prepends the proper reason-aware
|
||||
# system note before the turn runs.
|
||||
event = MessageEvent(
|
||||
text="",
|
||||
message_type=MessageType.TEXT,
|
||||
source=source,
|
||||
internal=True,
|
||||
)
|
||||
task = asyncio.create_task(adapter.handle_message(event))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
scheduled += 1
|
||||
|
||||
if scheduled:
|
||||
logger.info(
|
||||
"Scheduled auto-resume for %d restart-interrupted session(s)",
|
||||
scheduled,
|
||||
)
|
||||
return scheduled
|
||||
|
||||
async def start(self) -> bool:
|
||||
"""
|
||||
Start the gateway and all configured platform adapters.
|
||||
@@ -2763,6 +2969,29 @@ class GatewayRunner:
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
# Redaction status: ON by default (#17691). Surface a prominent
|
||||
# warning if an operator has explicitly opted out so they don't
|
||||
# forget the downgrade is active — the redactor snapshots its
|
||||
# state at import time, so this log line is the source of truth
|
||||
# for this process's lifetime.
|
||||
try:
|
||||
_redact_raw = os.getenv("HERMES_REDACT_SECRETS", "true")
|
||||
_redact_on = _redact_raw.lower() in ("1", "true", "yes", "on")
|
||||
if _redact_on:
|
||||
logger.info(
|
||||
"Secret redaction: ENABLED (tool output, logs, and chat "
|
||||
"responses are scrubbed before delivery)"
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"Secret redaction: DISABLED (HERMES_REDACT_SECRETS=%s). "
|
||||
"API keys and tokens may appear verbatim in chat output, "
|
||||
"session JSONs, and logs. Set security.redact_secrets: true "
|
||||
"in config.yaml to re-enable.",
|
||||
_redact_raw,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
from hermes_cli.profiles import get_active_profile_name
|
||||
_profile = get_active_profile_name()
|
||||
@@ -3127,6 +3356,12 @@ class GatewayRunner:
|
||||
skip_targets=skip_home_targets,
|
||||
)
|
||||
|
||||
# Automatically continue fresh sessions that were interrupted by the
|
||||
# previous gateway restart/shutdown. The resume_pending flag is cleared
|
||||
# by the normal successful-turn path, so a failed auto-resume remains
|
||||
# visible for manual recovery on the next user message.
|
||||
self._schedule_resume_pending_sessions()
|
||||
|
||||
# Drain any recovered process watchers (from crash recovery checkpoint)
|
||||
try:
|
||||
from tools.process_registry import process_registry
|
||||
@@ -3645,6 +3880,24 @@ class GatewayRunner:
|
||||
if max_spawn is not None:
|
||||
logger.info(f"kanban dispatcher: max_spawn={max_spawn}")
|
||||
|
||||
raw_failure_limit = kanban_cfg.get("failure_limit", _kb.DEFAULT_FAILURE_LIMIT)
|
||||
try:
|
||||
failure_limit = int(raw_failure_limit)
|
||||
except (TypeError, ValueError):
|
||||
logger.warning(
|
||||
"kanban dispatcher: invalid kanban.failure_limit=%r; using default %d",
|
||||
raw_failure_limit,
|
||||
_kb.DEFAULT_FAILURE_LIMIT,
|
||||
)
|
||||
failure_limit = _kb.DEFAULT_FAILURE_LIMIT
|
||||
if failure_limit < 1:
|
||||
logger.warning(
|
||||
"kanban dispatcher: kanban.failure_limit=%r is below 1; using default %d",
|
||||
raw_failure_limit,
|
||||
_kb.DEFAULT_FAILURE_LIMIT,
|
||||
)
|
||||
failure_limit = _kb.DEFAULT_FAILURE_LIMIT
|
||||
|
||||
# Initial delay so the gateway finishes wiring adapters before the
|
||||
# dispatcher spawns workers (those workers may hit gateway notify
|
||||
# subscriptions etc.). Matches the notifier watcher's delay.
|
||||
@@ -3673,7 +3926,12 @@ class GatewayRunner:
|
||||
_kb.init_db(board=slug) # idempotent, handles first-run
|
||||
except Exception:
|
||||
pass
|
||||
return _kb.dispatch_once(conn, board=slug, max_spawn=max_spawn)
|
||||
return _kb.dispatch_once(
|
||||
conn,
|
||||
board=slug,
|
||||
max_spawn=max_spawn,
|
||||
failure_limit=failure_limit,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception("kanban dispatcher: tick failed on board %s", slug)
|
||||
return None
|
||||
@@ -5757,6 +6015,7 @@ class GatewayRunner:
|
||||
|
||||
if event.media_urls and event.message_type == MessageType.DOCUMENT:
|
||||
import mimetypes as _mimetypes
|
||||
from tools.credential_files import to_agent_visible_cache_path
|
||||
|
||||
_TEXT_EXTENSIONS = {".txt", ".md", ".csv", ".log", ".json", ".xml", ".yaml", ".yml", ".toml", ".ini", ".cfg"}
|
||||
for i, path in enumerate(event.media_urls):
|
||||
@@ -5777,16 +6036,21 @@ class GatewayRunner:
|
||||
display_name = parts[2] if len(parts) >= 3 else basename
|
||||
display_name = re.sub(r'[^\w.\- ]', '_', display_name)
|
||||
|
||||
# Translate host cache path to in-container path if running under Docker backend.
|
||||
# This ensures the agent receives a path it can open inside its sandbox, as the
|
||||
# cache directories are auto-mounted at /root/.hermes/cache/* by get_cache_directory_mounts().
|
||||
agent_path = to_agent_visible_cache_path(path)
|
||||
|
||||
if mtype.startswith("text/"):
|
||||
context_note = (
|
||||
f"[The user sent a text document: '{display_name}'. "
|
||||
f"Its content has been included below. "
|
||||
f"The file is also saved at: {path}]"
|
||||
f"The file is also saved at: {agent_path}]"
|
||||
)
|
||||
else:
|
||||
context_note = (
|
||||
f"[The user sent a document: '{display_name}'. "
|
||||
f"The file is saved at: {path}. "
|
||||
f"The file is saved at: {agent_path}. "
|
||||
f"Ask the user what they'd like you to do with it.]"
|
||||
)
|
||||
message_text = f"{context_note}\n\n{message_text}"
|
||||
@@ -5851,6 +6115,41 @@ class GatewayRunner:
|
||||
return []
|
||||
return list(pending_native.pop(session_key, []) or [])
|
||||
|
||||
def _cache_session_source(self, session_key: str, source) -> None:
|
||||
if not session_key or source is None:
|
||||
return
|
||||
cached_sources = getattr(self, "_session_sources", None)
|
||||
if cached_sources is None:
|
||||
cached_sources = OrderedDict()
|
||||
self._session_sources = cached_sources
|
||||
try:
|
||||
cached_sources[session_key] = dataclasses.replace(source)
|
||||
except Exception:
|
||||
logger.debug("Failed to cache live session source for %s", session_key, exc_info=True)
|
||||
return
|
||||
# LRU: mark as most-recently-used and trim to max size.
|
||||
try:
|
||||
cached_sources.move_to_end(session_key)
|
||||
max_size = getattr(self, "_session_sources_max", 512)
|
||||
while len(cached_sources) > max_size:
|
||||
cached_sources.popitem(last=False)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _get_cached_session_source(self, session_key: str):
|
||||
if not session_key:
|
||||
return None
|
||||
cached_sources = getattr(self, "_session_sources", None)
|
||||
if not cached_sources:
|
||||
return None
|
||||
source = cached_sources.get(session_key)
|
||||
if source is not None:
|
||||
try:
|
||||
cached_sources.move_to_end(session_key)
|
||||
except Exception:
|
||||
pass
|
||||
return source
|
||||
|
||||
async def _handle_message_with_agent(self, event, source, _quick_key: str, run_generation: int):
|
||||
"""Inner handler that runs under the _running_agents sentinel guard."""
|
||||
_msg_start_time = time.time()
|
||||
@@ -5865,6 +6164,7 @@ class GatewayRunner:
|
||||
# Get or create session
|
||||
session_entry = self.session_store.get_or_create_session(source)
|
||||
session_key = session_entry.session_key
|
||||
self._cache_session_source(session_key, source)
|
||||
if self._is_telegram_topic_lane(source):
|
||||
try:
|
||||
binding = self._session_db.get_telegram_topic_binding(
|
||||
@@ -6501,7 +6801,7 @@ class GatewayRunner:
|
||||
# shutdown) — the turn ran to completion, so recovery
|
||||
# succeeded and subsequent messages should no longer receive
|
||||
# the restart-interruption system note.
|
||||
if session_key:
|
||||
if session_key and _should_clear_resume_pending_after_turn(agent_result):
|
||||
self._clear_restart_failure_count(session_key)
|
||||
try:
|
||||
self.session_store.clear_resume_pending(session_key)
|
||||
@@ -8082,6 +8382,27 @@ class GatewayRunner:
|
||||
# ────────────────────────────────────────────────────────────────
|
||||
# /goal — persistent cross-turn goals (Ralph-style loop)
|
||||
# ────────────────────────────────────────────────────────────────
|
||||
def _goal_max_turns_from_config(self) -> int:
|
||||
"""Resolve the configured /goal turn budget for gateway sessions.
|
||||
|
||||
GatewayRunner.config is a GatewayConfig dataclass, not the full
|
||||
user config mapping. Top-level config blocks such as ``goals`` are
|
||||
therefore only available through hermes_cli.config.load_config().
|
||||
"""
|
||||
try:
|
||||
goals_cfg = (
|
||||
(self.config or {}).get("goals", {})
|
||||
if isinstance(self.config, dict)
|
||||
else getattr(self.config, "goals", {}) or {}
|
||||
)
|
||||
if not goals_cfg:
|
||||
from hermes_cli.config import load_config
|
||||
|
||||
goals_cfg = (load_config() or {}).get("goals") or {}
|
||||
return int(goals_cfg.get("max_turns", 20) or 20)
|
||||
except Exception:
|
||||
return 20
|
||||
|
||||
def _get_goal_manager_for_event(self, event: "MessageEvent"):
|
||||
"""Return a GoalManager bound to the session for this gateway event.
|
||||
|
||||
@@ -8101,15 +8422,7 @@ class GatewayRunner:
|
||||
sid = getattr(session_entry, "session_id", None) or ""
|
||||
if not sid:
|
||||
return None, None
|
||||
try:
|
||||
goals_cfg = (
|
||||
(self.config or {}).get("goals", {})
|
||||
if isinstance(self.config, dict)
|
||||
else getattr(self.config, "goals", {}) or {}
|
||||
)
|
||||
max_turns = int(goals_cfg.get("max_turns", 20) or 20)
|
||||
except Exception:
|
||||
max_turns = 20
|
||||
max_turns = self._goal_max_turns_from_config()
|
||||
return GoalManager(session_id=sid, default_max_turns=max_turns), session_entry
|
||||
|
||||
async def _handle_goal_command(self, event: "MessageEvent") -> str:
|
||||
@@ -8209,15 +8522,7 @@ class GatewayRunner:
|
||||
if not sid:
|
||||
return
|
||||
|
||||
try:
|
||||
goals_cfg = (
|
||||
(self.config or {}).get("goals", {})
|
||||
if isinstance(self.config, dict)
|
||||
else getattr(self.config, "goals", {}) or {}
|
||||
)
|
||||
max_turns = int(goals_cfg.get("max_turns", 20) or 20)
|
||||
except Exception:
|
||||
max_turns = 20
|
||||
max_turns = self._goal_max_turns_from_config()
|
||||
|
||||
mgr = GoalManager(session_id=sid, default_max_turns=max_turns)
|
||||
if not mgr.is_active():
|
||||
@@ -8760,6 +9065,12 @@ class GatewayRunner:
|
||||
from urllib.parse import quote as _quote
|
||||
|
||||
try:
|
||||
# Capture [[as_document]] before extract_media strips it, so the
|
||||
# dispatch partition below can route image-extension files
|
||||
# through send_document (preserving bytes) instead of
|
||||
# send_multiple_images (Telegram sendPhoto recompresses to ~1280px).
|
||||
force_document_attachments = "[[as_document]]" in response
|
||||
|
||||
media_files, _ = adapter.extract_media(response)
|
||||
_, cleaned = adapter.extract_images(response)
|
||||
local_files, _ = adapter.extract_local_files(cleaned)
|
||||
@@ -8772,19 +9083,24 @@ class GatewayRunner:
|
||||
_IMAGE_EXTS = {'.jpg', '.jpeg', '.png', '.webp', '.gif'}
|
||||
|
||||
# Partition out images so they can be sent as a single batch
|
||||
# (e.g. Signal's multi-attachment RPC)
|
||||
# (e.g. Signal's multi-attachment RPC). When [[as_document]] was
|
||||
# set, image-extension files skip the photo path and route to
|
||||
# send_document below — preserving original bytes.
|
||||
image_paths: list = []
|
||||
non_image_media: list = []
|
||||
for media_path, is_voice in media_files:
|
||||
ext = Path(media_path).suffix.lower()
|
||||
if ext in _IMAGE_EXTS and not is_voice:
|
||||
if (ext in _IMAGE_EXTS
|
||||
and not is_voice
|
||||
and not force_document_attachments):
|
||||
image_paths.append(media_path)
|
||||
else:
|
||||
non_image_media.append((media_path, is_voice))
|
||||
|
||||
non_image_local: list = []
|
||||
for file_path in local_files:
|
||||
if Path(file_path).suffix.lower() in _IMAGE_EXTS:
|
||||
if (Path(file_path).suffix.lower() in _IMAGE_EXTS
|
||||
and not force_document_attachments):
|
||||
image_paths.append(file_path)
|
||||
else:
|
||||
non_image_local.append(file_path)
|
||||
@@ -11035,30 +11351,78 @@ class GatewayRunner:
|
||||
# where systemd-run --user fails due to missing D-Bus session).
|
||||
# PYTHONUNBUFFERED ensures output is flushed line-by-line so the
|
||||
# gateway can stream it to the messenger in near-real-time.
|
||||
hermes_cmd_str = " ".join(shlex.quote(part) for part in hermes_cmd)
|
||||
update_cmd = (
|
||||
f"PYTHONUNBUFFERED=1 {hermes_cmd_str} update --gateway"
|
||||
f" > {shlex.quote(str(output_path))} 2>&1; "
|
||||
f"status=$?; printf '%s' \"$status\" > {shlex.quote(str(exit_code_path))}"
|
||||
)
|
||||
# Spawn `hermes update --gateway` detached so it survives gateway restart.
|
||||
# --gateway enables file-based IPC for interactive prompts (stash
|
||||
# restore, config migration) so the gateway can forward them to the
|
||||
# user instead of silently skipping them.
|
||||
# Use setsid for portable session detach (works under system services
|
||||
# where systemd-run --user fails due to missing D-Bus session).
|
||||
# PYTHONUNBUFFERED ensures output is flushed line-by-line so the
|
||||
# gateway can stream it to the messenger in near-real-time.
|
||||
#
|
||||
# Windows: no bash/setsid chain. Run `hermes update --gateway`
|
||||
# directly via sys.executable; redirect stdout/stderr to the same
|
||||
# output files via Popen file handles; write the exit code in a
|
||||
# follow-up write. A tiny Python watcher would be cleaner but
|
||||
# we're already inside gateway/run.py's update path which is async,
|
||||
# so the simplest correct thing is: launch an inline Python helper
|
||||
# that runs the command and writes both outputs.
|
||||
try:
|
||||
setsid_bin = shutil.which("setsid")
|
||||
if setsid_bin:
|
||||
# Preferred: setsid creates a new session, fully detached
|
||||
if sys.platform == "win32":
|
||||
import textwrap
|
||||
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
|
||||
|
||||
# hermes_cmd is a list of argv parts we can pass directly
|
||||
# (no shell-quoting needed).
|
||||
helper = textwrap.dedent(
|
||||
"""
|
||||
import os, subprocess, sys
|
||||
output_path = sys.argv[1]
|
||||
exit_code_path = sys.argv[2]
|
||||
cmd = sys.argv[3:]
|
||||
env = dict(os.environ)
|
||||
env["PYTHONUNBUFFERED"] = "1"
|
||||
with open(output_path, "wb") as f:
|
||||
proc = subprocess.Popen(cmd, stdout=f, stderr=subprocess.STDOUT, env=env)
|
||||
rc = proc.wait()
|
||||
with open(exit_code_path, "w") as f:
|
||||
f.write(str(rc))
|
||||
"""
|
||||
).strip()
|
||||
subprocess.Popen(
|
||||
[setsid_bin, "bash", "-c", update_cmd],
|
||||
[
|
||||
sys.executable, "-c", helper,
|
||||
str(output_path), str(exit_code_path),
|
||||
*hermes_cmd, "update", "--gateway",
|
||||
],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
start_new_session=True,
|
||||
**windows_detach_popen_kwargs(),
|
||||
)
|
||||
else:
|
||||
# Fallback: start_new_session=True calls os.setsid() in child
|
||||
subprocess.Popen(
|
||||
["bash", "-c", update_cmd],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
start_new_session=True,
|
||||
hermes_cmd_str = " ".join(shlex.quote(part) for part in hermes_cmd)
|
||||
update_cmd = (
|
||||
f"PYTHONUNBUFFERED=1 {hermes_cmd_str} update --gateway"
|
||||
f" > {shlex.quote(str(output_path))} 2>&1; "
|
||||
f"status=$?; printf '%s' \"$status\" > {shlex.quote(str(exit_code_path))}"
|
||||
)
|
||||
setsid_bin = shutil.which("setsid")
|
||||
if setsid_bin:
|
||||
# Preferred: setsid creates a new session, fully detached
|
||||
subprocess.Popen(
|
||||
[setsid_bin, "bash", "-c", update_cmd],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
start_new_session=True,
|
||||
)
|
||||
else:
|
||||
# Fallback: start_new_session=True calls os.setsid() in child
|
||||
subprocess.Popen(
|
||||
["bash", "-c", update_cmd],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
start_new_session=True,
|
||||
)
|
||||
except Exception as e:
|
||||
pending_path.unlink(missing_ok=True)
|
||||
exit_code_path.unlink(missing_ok=True)
|
||||
@@ -11739,6 +12103,10 @@ class GatewayRunner:
|
||||
exc,
|
||||
)
|
||||
|
||||
cached_source = self._get_cached_session_source(session_key)
|
||||
if cached_source is not None:
|
||||
return cached_source
|
||||
|
||||
_parsed = _parse_session_key(session_key)
|
||||
if _parsed:
|
||||
derived_platform = _parsed["platform"]
|
||||
@@ -11982,6 +12350,7 @@ class GatewayRunner:
|
||||
# Add more here as new baked-at-construction config settings are added.
|
||||
_CACHE_BUSTING_CONFIG_KEYS: tuple = (
|
||||
("model", "context_length"),
|
||||
("model", "max_tokens"),
|
||||
("compression", "enabled"),
|
||||
("compression", "threshold"),
|
||||
("compression", "target_ratio"),
|
||||
@@ -12839,6 +13208,24 @@ class GatewayRunner:
|
||||
last_tool = [None] # Mutable container for tracking in closure
|
||||
last_progress_msg = [None] # Track last message for dedup
|
||||
repeat_count = [0] # How many times the same message repeated
|
||||
|
||||
# Auto-cleanup of temporary progress bubbles (Telegram + any adapter
|
||||
# that implements ``delete_message``). When enabled via
|
||||
# ``display.platforms.<platform>.cleanup_progress: true``, message IDs
|
||||
# from the tool-progress / "Still working..." / status-callback bubbles
|
||||
# are collected here and deleted after the final response lands.
|
||||
# Failed runs skip cleanup so the bubbles remain as breadcrumbs.
|
||||
_cleanup_progress = bool(
|
||||
resolve_display_setting(user_config, platform_key, "cleanup_progress")
|
||||
)
|
||||
_cleanup_adapter = self.adapters.get(source.platform) if _cleanup_progress else None
|
||||
if _cleanup_adapter is not None and (
|
||||
type(_cleanup_adapter).delete_message is BasePlatformAdapter.delete_message
|
||||
):
|
||||
# Adapter doesn't support deletion — silently disable.
|
||||
_cleanup_progress = False
|
||||
_cleanup_adapter = None
|
||||
_cleanup_msg_ids: List[str] = []
|
||||
# First-touch onboarding latch: fires at most once per run, even if
|
||||
# several tools exceed the threshold.
|
||||
long_tool_hint_fired = [False]
|
||||
@@ -13087,12 +13474,18 @@ class GatewayRunner:
|
||||
adapter.name,
|
||||
)
|
||||
can_edit = False
|
||||
await adapter.send(
|
||||
_flood_result = await adapter.send(
|
||||
chat_id=source.chat_id,
|
||||
content=msg,
|
||||
reply_to=_progress_reply_to,
|
||||
metadata=_progress_metadata,
|
||||
)
|
||||
if (
|
||||
_cleanup_progress
|
||||
and getattr(_flood_result, "success", False)
|
||||
and getattr(_flood_result, "message_id", None)
|
||||
):
|
||||
_cleanup_msg_ids.append(str(_flood_result.message_id))
|
||||
else:
|
||||
if can_edit:
|
||||
# First tool: send all accumulated text as new message
|
||||
@@ -13113,6 +13506,8 @@ class GatewayRunner:
|
||||
)
|
||||
if result.success and result.message_id:
|
||||
progress_msg_id = result.message_id
|
||||
if _cleanup_progress:
|
||||
_cleanup_msg_ids.append(str(result.message_id))
|
||||
|
||||
_last_edit_ts = time.monotonic()
|
||||
|
||||
@@ -13226,7 +13621,7 @@ class GatewayRunner:
|
||||
if not _status_adapter or not _run_still_current():
|
||||
return
|
||||
try:
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
_fut = asyncio.run_coroutine_threadsafe(
|
||||
_status_adapter.send(
|
||||
_status_chat_id,
|
||||
message,
|
||||
@@ -13234,6 +13629,16 @@ class GatewayRunner:
|
||||
),
|
||||
_loop_for_step,
|
||||
)
|
||||
if _cleanup_progress:
|
||||
def _track_status_id(fut) -> None:
|
||||
try:
|
||||
res = fut.result()
|
||||
except Exception:
|
||||
return
|
||||
mid = getattr(res, "message_id", None)
|
||||
if getattr(res, "success", False) and mid:
|
||||
_cleanup_msg_ids.append(str(mid))
|
||||
_fut.add_done_callback(_track_status_id)
|
||||
except Exception as _e:
|
||||
logger.debug("status_callback error (%s): %s", event_type, _e)
|
||||
|
||||
@@ -13267,13 +13672,9 @@ class GatewayRunner:
|
||||
combined_ephemeral = (combined_ephemeral + "\n\n" + self._ephemeral_system_prompt).strip()
|
||||
|
||||
# Re-read .env and config for fresh credentials (gateway is long-lived,
|
||||
# keys may change without restart).
|
||||
try:
|
||||
load_dotenv(_env_path, override=True, encoding="utf-8")
|
||||
except UnicodeDecodeError:
|
||||
load_dotenv(_env_path, override=True, encoding="latin-1")
|
||||
except Exception:
|
||||
pass
|
||||
# keys may change without restart). Keep config.yaml authoritative for
|
||||
# runtime budget settings bridged into env vars.
|
||||
_reload_runtime_env_preserving_config_authority()
|
||||
|
||||
try:
|
||||
model, runtime_kwargs = self._resolve_session_agent_runtime(
|
||||
@@ -13841,6 +14242,11 @@ class GatewayRunner:
|
||||
"messages": result.get("messages", []),
|
||||
"api_calls": result.get("api_calls", 0),
|
||||
"failed": result.get("failed", False),
|
||||
"partial": result.get("partial", False),
|
||||
"completed": result.get("completed"),
|
||||
"interrupted": result.get("interrupted", False),
|
||||
"interrupt_message": result.get("interrupt_message"),
|
||||
"error": result.get("error"),
|
||||
"compression_exhausted": result.get("compression_exhausted", False),
|
||||
"tools": tools_holder[0] or [],
|
||||
"history_offset": len(agent_history),
|
||||
@@ -13956,6 +14362,11 @@ class GatewayRunner:
|
||||
"last_reasoning": result.get("last_reasoning"),
|
||||
"messages": result_holder[0].get("messages", []) if result_holder[0] else [],
|
||||
"api_calls": result_holder[0].get("api_calls", 0) if result_holder[0] else 0,
|
||||
"completed": result_holder[0].get("completed") if result_holder[0] else None,
|
||||
"interrupted": result_holder[0].get("interrupted", False) if result_holder[0] else False,
|
||||
"partial": result_holder[0].get("partial", False) if result_holder[0] else False,
|
||||
"error": result_holder[0].get("error") if result_holder[0] else None,
|
||||
"interrupt_message": result_holder[0].get("interrupt_message") if result_holder[0] else None,
|
||||
"tools": tools_holder[0] or [],
|
||||
"history_offset": _effective_history_offset,
|
||||
"last_prompt_tokens": _last_prompt_toks,
|
||||
@@ -14094,11 +14505,17 @@ class GatewayRunner:
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
await _notify_adapter.send(
|
||||
_notify_res = await _notify_adapter.send(
|
||||
source.chat_id,
|
||||
f"⏳ Still working... ({_elapsed_mins} min elapsed{_status_detail})",
|
||||
metadata=_status_thread_metadata,
|
||||
)
|
||||
if (
|
||||
_cleanup_progress
|
||||
and getattr(_notify_res, "success", False)
|
||||
and getattr(_notify_res, "message_id", None)
|
||||
):
|
||||
_cleanup_msg_ids.append(str(_notify_res.message_id))
|
||||
except Exception as _ne:
|
||||
logger.debug("Long-running notification error: %s", _ne)
|
||||
|
||||
@@ -14572,7 +14989,49 @@ class GatewayRunner:
|
||||
_previewed,
|
||||
)
|
||||
response["already_sent"] = True
|
||||
|
||||
|
||||
# Schedule deletion of tracked temporary progress bubbles after the
|
||||
# final response lands. Failed runs skip this so bubbles remain as
|
||||
# breadcrumbs for the user to see what work happened. Only fires on
|
||||
# adapters that support ``delete_message`` (see init above); failures
|
||||
# are swallowed — deletion is best-effort.
|
||||
if (
|
||||
_cleanup_progress
|
||||
and _cleanup_adapter is not None
|
||||
and _cleanup_msg_ids
|
||||
and session_key
|
||||
and isinstance(response, dict)
|
||||
and not response.get("failed")
|
||||
and hasattr(_cleanup_adapter, "register_post_delivery_callback")
|
||||
):
|
||||
_ids_snapshot = list(_cleanup_msg_ids)
|
||||
_chat_id_snapshot = source.chat_id
|
||||
_adapter_snapshot = _cleanup_adapter
|
||||
_loop_snapshot = asyncio.get_running_loop()
|
||||
|
||||
def _cleanup_temp_bubbles() -> None:
|
||||
async def _delete_all() -> None:
|
||||
for _mid in _ids_snapshot:
|
||||
try:
|
||||
await _adapter_snapshot.delete_message(
|
||||
_chat_id_snapshot, _mid
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
asyncio.run_coroutine_threadsafe(_delete_all(), _loop_snapshot)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
_cleanup_adapter.register_post_delivery_callback(
|
||||
session_key,
|
||||
_cleanup_temp_bubbles,
|
||||
generation=run_generation,
|
||||
)
|
||||
except Exception as _rpe:
|
||||
logger.debug("Post-delivery cleanup registration failed: %s", _rpe)
|
||||
|
||||
return response
|
||||
|
||||
|
||||
@@ -14735,7 +15194,10 @@ async def start_gateway(config: Optional[GatewayConfig] = None, replace: bool =
|
||||
try:
|
||||
os.kill(existing_pid, 0)
|
||||
time.sleep(0.5)
|
||||
except (ProcessLookupError, PermissionError):
|
||||
except (ProcessLookupError, PermissionError, OSError):
|
||||
# OSError covers Windows' WinError 87 "invalid parameter"
|
||||
# for an already-gone PID — without this the probe loop
|
||||
# busy-spins for the full 10s on every --replace start.
|
||||
break # Process is gone
|
||||
else:
|
||||
# Still alive after 10s — force kill
|
||||
@@ -15020,6 +15482,14 @@ async def start_gateway(config: Optional[GatewayConfig] = None, replace: bool =
|
||||
|
||||
def main():
|
||||
"""CLI entry point for the gateway."""
|
||||
# Force UTF-8 stdio on Windows — gateway logs and startup banner would
|
||||
# otherwise UnicodeEncodeError on cp1252 consoles. No-op on POSIX.
|
||||
try:
|
||||
from hermes_cli.stdio import configure_windows_stdio
|
||||
configure_windows_stdio()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description="Hermes Gateway - Multi-platform messaging")
|
||||
|
||||
+3
-3
@@ -113,7 +113,7 @@ def _get_process_start_time(pid: int) -> Optional[int]:
|
||||
stat_path = Path(f"/proc/{pid}/stat")
|
||||
try:
|
||||
# Field 22 in /proc/<pid>/stat is process start time (clock ticks).
|
||||
return int(stat_path.read_text().split()[21])
|
||||
return int(stat_path.read_text(encoding="utf-8").split()[21])
|
||||
except (FileNotFoundError, IndexError, PermissionError, ValueError, OSError):
|
||||
return None
|
||||
|
||||
@@ -197,7 +197,7 @@ def _read_json_file(path: Path) -> Optional[dict[str, Any]]:
|
||||
if not path.exists():
|
||||
return None
|
||||
try:
|
||||
raw = path.read_text().strip()
|
||||
raw = path.read_text(encoding="utf-8").strip()
|
||||
except OSError:
|
||||
return None
|
||||
if not raw:
|
||||
@@ -523,7 +523,7 @@ def acquire_scoped_lock(scope: str, identity: str, metadata: Optional[dict[str,
|
||||
try:
|
||||
_proc_status = Path(f"/proc/{existing_pid}/status")
|
||||
if _proc_status.exists():
|
||||
for _line in _proc_status.read_text().splitlines():
|
||||
for _line in _proc_status.read_text(encoding="utf-8").splitlines():
|
||||
if _line.startswith("State:"):
|
||||
_state = _line.split()[1]
|
||||
if _state in ("T", "t"): # stopped or tracing stop
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
"""Windows UTF-8 bootstrap for Hermes entry points.
|
||||
|
||||
Python on Windows has two long-standing text-encoding footguns:
|
||||
|
||||
1. ``sys.stdout`` / ``sys.stderr`` are bound to the console code page
|
||||
(``cp1252`` on US-locale installs), so ``print("café")`` crashes with
|
||||
``UnicodeEncodeError: 'charmap' codec can't encode character``.
|
||||
|
||||
2. Child processes spawned via ``subprocess`` don't know to use UTF-8
|
||||
unless ``PYTHONUTF8`` and/or ``PYTHONIOENCODING`` are set in their
|
||||
environment — so any Python subprocess (the execute_code sandbox,
|
||||
delegation children, linter subprocesses, etc.) inherits the same
|
||||
cp1252 defaults and hits the same UnicodeEncodeError.
|
||||
|
||||
This module fixes both on Windows *only* — POSIX is untouched. It
|
||||
should be imported at the very top of every Hermes entry point
|
||||
(``hermes``, ``hermes-agent``, ``hermes-acp``, ``python -m gateway.run``,
|
||||
``batch_runner.py``, ``cron/scheduler.py``) before any other imports
|
||||
that might do file I/O or print to stdout.
|
||||
|
||||
What this module does on Windows:
|
||||
|
||||
- Sets ``os.environ["PYTHONUTF8"] = "1"`` (PEP 540 UTF-8 mode) so
|
||||
every child process we spawn uses UTF-8 for ``open()`` and stdio.
|
||||
- Sets ``os.environ["PYTHONIOENCODING"] = "utf-8"`` for belt-and-
|
||||
suspenders — some tools read this instead of / in addition to
|
||||
``PYTHONUTF8``.
|
||||
- Reconfigures ``sys.stdout`` / ``sys.stderr`` to UTF-8 in the current
|
||||
process, using the ``reconfigure()`` API (Python 3.7+). This fixes
|
||||
``print("café")`` in the parent without a re-exec.
|
||||
|
||||
What this module does NOT do:
|
||||
|
||||
- It does not re-exec Python with ``-X utf8``, so ``open()`` calls in
|
||||
the *current* process still default to locale encoding. Those need
|
||||
an explicit ``encoding="utf-8"`` at the call site (lint rule
|
||||
``PLW1514`` / ``PYI058``). Ruff is the right tool for that sweep.
|
||||
|
||||
What this module does on POSIX:
|
||||
|
||||
- Nothing. POSIX systems are already UTF-8 by default in 99% of cases,
|
||||
and we don't want to touch ``LANG``/``LC_*`` behavior that users may
|
||||
have configured intentionally. If someone hits a C/POSIX locale on
|
||||
Linux, they can export ``PYTHONUTF8=1`` themselves — we won't override.
|
||||
|
||||
Idempotent: safe to call multiple times. ``_bootstrap_once`` guards
|
||||
against double-reconfigure.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
_IS_WINDOWS = sys.platform == "win32"
|
||||
_bootstrap_applied = False
|
||||
|
||||
|
||||
def apply_windows_utf8_bootstrap() -> bool:
|
||||
"""Apply the Windows UTF-8 bootstrap if we're on Windows.
|
||||
|
||||
Returns True if bootstrap was applied (i.e. we're on Windows and
|
||||
haven't already done this), False otherwise. The return value is
|
||||
advisory — callers normally don't need it, but tests may want to
|
||||
assert the path was taken.
|
||||
|
||||
Idempotent: subsequent calls after the first are a no-op.
|
||||
"""
|
||||
global _bootstrap_applied
|
||||
|
||||
if not _IS_WINDOWS:
|
||||
return False
|
||||
if _bootstrap_applied:
|
||||
return False
|
||||
|
||||
# 1. Child processes inherit these and run in UTF-8 mode.
|
||||
# We use setdefault() rather than overwriting so the user can
|
||||
# explicitly opt out by setting PYTHONUTF8=0 in their environment
|
||||
# (or PYTHONIOENCODING=something-else) if they really want to.
|
||||
os.environ.setdefault("PYTHONUTF8", "1")
|
||||
os.environ.setdefault("PYTHONIOENCODING", "utf-8")
|
||||
|
||||
# 2. Reconfigure the current process's stdio to UTF-8. Needed
|
||||
# because os.environ changes don't retroactively rebind sys.stdout
|
||||
# — those were bound at interpreter startup based on the console
|
||||
# code page. ``reconfigure`` is a TextIOWrapper method since 3.7.
|
||||
#
|
||||
# errors="replace" means that if we ever *read* something from
|
||||
# stdin that isn't UTF-8 (unlikely but possible with piped input
|
||||
# from legacy tools), we'll get U+FFFD replacement chars rather
|
||||
# than a crash. Output is pure UTF-8.
|
||||
for stream_name in ("stdout", "stderr"):
|
||||
stream = getattr(sys, stream_name, None)
|
||||
if stream is None:
|
||||
continue
|
||||
reconfigure = getattr(stream, "reconfigure", None)
|
||||
if reconfigure is None:
|
||||
# Not a TextIOWrapper (could be redirected to a BytesIO in
|
||||
# tests, or a non-standard stream in some embedded cases).
|
||||
# Skip silently — the env-var fix is still in effect for
|
||||
# child processes, which is the bigger win.
|
||||
continue
|
||||
try:
|
||||
reconfigure(encoding="utf-8", errors="replace")
|
||||
except (OSError, ValueError):
|
||||
# Already closed, or someone replaced it with something
|
||||
# non-reconfigurable. Non-fatal.
|
||||
pass
|
||||
|
||||
# stdin is reconfigured separately with errors="replace" too — input
|
||||
# from a legacy pipe shouldn't crash the process.
|
||||
stdin = getattr(sys, "stdin", None)
|
||||
if stdin is not None:
|
||||
reconfigure = getattr(stdin, "reconfigure", None)
|
||||
if reconfigure is not None:
|
||||
try:
|
||||
reconfigure(encoding="utf-8", errors="replace")
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
|
||||
_bootstrap_applied = True
|
||||
return True
|
||||
|
||||
|
||||
# Apply on import — entry points just need ``import hermes_bootstrap``
|
||||
# (or ``from hermes_bootstrap import apply_windows_utf8_bootstrap``) at
|
||||
# the very top of their module, before importing anything else. The
|
||||
# import side effect does the right thing.
|
||||
apply_windows_utf8_bootstrap()
|
||||
@@ -14,8 +14,8 @@ Provides subcommands for:
|
||||
import os
|
||||
import sys
|
||||
|
||||
__version__ = "0.12.0"
|
||||
__release_date__ = "2026.4.30"
|
||||
__version__ = "0.13.0"
|
||||
__release_date__ = "2026.5.7"
|
||||
|
||||
|
||||
def _ensure_utf8():
|
||||
|
||||
@@ -70,6 +70,9 @@ Examples:
|
||||
hermes logs --since 1h Lines from the last hour
|
||||
hermes debug share Upload debug report for support
|
||||
hermes update Update to latest version
|
||||
hermes dashboard Start web UI dashboard (port 9119)
|
||||
hermes dashboard --stop Stop running dashboard processes
|
||||
hermes dashboard --status List running dashboard processes
|
||||
|
||||
For more help on a command:
|
||||
hermes <command> --help
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
"""Windows subprocess compatibility helpers.
|
||||
|
||||
Hermes is developed on Linux / macOS and tested natively on Windows too.
|
||||
Several common subprocess patterns break silently-or-loudly on Windows:
|
||||
|
||||
* ``["npm", "install", ...]`` — on Windows ``npm`` is ``npm.cmd``, a batch
|
||||
shim. ``subprocess.Popen(["npm", ...])`` fails with WinError 193
|
||||
("not a valid Win32 application") because CreateProcessW can't run a
|
||||
``.cmd`` file without ``shell=True`` or PATHEXT resolution.
|
||||
|
||||
* ``start_new_session=True`` — on POSIX, this maps to ``os.setsid()`` and
|
||||
actually detaches the child. On Windows it's silently ignored; the
|
||||
Windows equivalent is ``CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS``
|
||||
creationflags, which Python only applies when you pass them explicitly.
|
||||
|
||||
* Console-window flashes — every ``subprocess.Popen`` of a ``.exe`` on
|
||||
Windows spawns a cmd window briefly unless ``CREATE_NO_WINDOW`` is
|
||||
passed. Cosmetic but jarring for background daemons.
|
||||
|
||||
This module centralizes the platform-branching logic so the rest of the
|
||||
codebase doesn't sprinkle ``if sys.platform == "win32":`` everywhere.
|
||||
|
||||
**All helpers are no-ops on non-Windows** — calling them in Linux/macOS
|
||||
code paths is safe by design. That's the "do no damage on POSIX"
|
||||
guarantee.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import Optional, Sequence
|
||||
|
||||
__all__ = [
|
||||
"IS_WINDOWS",
|
||||
"resolve_node_command",
|
||||
"windows_detach_flags",
|
||||
"windows_hide_flags",
|
||||
"windows_detach_popen_kwargs",
|
||||
]
|
||||
|
||||
|
||||
IS_WINDOWS = sys.platform == "win32"
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Node ecosystem launcher resolution
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def resolve_node_command(name: str, argv: Sequence[str]) -> list[str]:
|
||||
"""Resolve a Node-ecosystem command name to an absolute-path argv.
|
||||
|
||||
On Windows, commands like ``npm``, ``npx``, ``yarn``, ``pnpm``,
|
||||
``playwright``, ``prettier`` ship as ``.cmd`` files (batch shims).
|
||||
``subprocess.Popen(["npm", "install"])`` fails with WinError 193
|
||||
because CreateProcessW doesn't execute batch files directly.
|
||||
|
||||
``shutil.which(name)`` *does* resolve ``.cmd`` via PATHEXT and returns
|
||||
the fully-qualified path — which CreateProcessW accepts because the
|
||||
extension tells Windows to route through ``cmd.exe /c``.
|
||||
|
||||
On POSIX ``shutil.which`` also returns a fully-qualified path when
|
||||
found. That's a small change from bare-name resolution (the OS does
|
||||
its own PATH search) but functionally identical and has the side
|
||||
benefit of making the argv reproducible in logs.
|
||||
|
||||
Behavior when the command is not on PATH:
|
||||
- On Windows: return the bare name — caller can still try with
|
||||
``shell=True`` as a last resort, OR the subsequent Popen will
|
||||
raise FileNotFoundError with a readable error we want to surface.
|
||||
- On POSIX: same. Bare ``npm`` on a Linux box without npm installed
|
||||
fails the same way it did before this function existed.
|
||||
|
||||
Args:
|
||||
name: The command name to resolve (``npm``, ``npx``, ``node`` …).
|
||||
argv: The remaining arguments. Must NOT include ``name`` itself —
|
||||
this function builds the full argv list.
|
||||
|
||||
Returns:
|
||||
A list suitable for passing to subprocess.Popen/run/call.
|
||||
"""
|
||||
resolved = shutil.which(name)
|
||||
if resolved:
|
||||
return [resolved, *argv]
|
||||
return [name, *argv]
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Detached / hidden process creation
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
# Win32 CreationFlags — defined here rather than imported from subprocess
|
||||
# because CREATE_NO_WINDOW and DETACHED_PROCESS aren't guaranteed to be
|
||||
# present on stdlib subprocess on older Pythons or non-Windows builds.
|
||||
_CREATE_NEW_PROCESS_GROUP = 0x00000200
|
||||
_DETACHED_PROCESS = 0x00000008
|
||||
_CREATE_NO_WINDOW = 0x08000000
|
||||
|
||||
|
||||
def windows_detach_flags() -> int:
|
||||
"""Return Win32 creationflags that detach a child from the parent
|
||||
console and process group. 0 on non-Windows.
|
||||
|
||||
Pair with ``start_new_session=False`` (default) when calling
|
||||
subprocess.Popen — on POSIX use ``start_new_session=True`` instead,
|
||||
which maps to ``os.setsid()`` in the child.
|
||||
|
||||
Rationale:
|
||||
- ``CREATE_NEW_PROCESS_GROUP`` — child has its own process group so
|
||||
Ctrl+C in the parent console doesn't propagate.
|
||||
- ``DETACHED_PROCESS`` — child has no console at all. Necessary for
|
||||
background daemons (gateway watchers, update respawners) because
|
||||
without it, closing the console kills the child.
|
||||
- ``CREATE_NO_WINDOW`` — suppress the brief cmd flash that would
|
||||
otherwise appear when launching a console app. Redundant with
|
||||
DETACHED_PROCESS but explicit for clarity.
|
||||
"""
|
||||
if not IS_WINDOWS:
|
||||
return 0
|
||||
return _CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW
|
||||
|
||||
|
||||
def windows_hide_flags() -> int:
|
||||
"""Return Win32 creationflags that merely hide the child's console
|
||||
window without detaching the child. 0 on non-Windows.
|
||||
|
||||
Use for short-lived console apps spawned as part of a larger
|
||||
operation (``taskkill``, ``where``, version probes) where we want no
|
||||
flash but also want to collect stdout/exit code synchronously.
|
||||
|
||||
The key difference from :func:`windows_detach_flags`: NO
|
||||
``DETACHED_PROCESS`` — the child still inherits stdio handles so
|
||||
``capture_output=True`` works. ``DETACHED_PROCESS`` would sever
|
||||
stdio and break stdout capture.
|
||||
"""
|
||||
if not IS_WINDOWS:
|
||||
return 0
|
||||
return _CREATE_NO_WINDOW
|
||||
|
||||
|
||||
def windows_detach_popen_kwargs() -> dict:
|
||||
"""Return a dict of Popen kwargs that detach a child on Windows and
|
||||
fall back to the POSIX equivalent (``start_new_session=True``) on
|
||||
Linux/macOS.
|
||||
|
||||
Usage pattern:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
subprocess.Popen(
|
||||
argv,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
stdin=subprocess.DEVNULL,
|
||||
close_fds=True,
|
||||
**windows_detach_popen_kwargs(),
|
||||
)
|
||||
|
||||
This replaces the unsafe-on-Windows pattern:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
subprocess.Popen(..., start_new_session=True)
|
||||
|
||||
which silently fails to detach on Windows (the flag is accepted but
|
||||
has no effect — the child stays attached to the parent's console
|
||||
and dies when the console closes).
|
||||
"""
|
||||
if IS_WINDOWS:
|
||||
return {"creationflags": windows_detach_flags()}
|
||||
return {"start_new_session": True}
|
||||
+375
-191
@@ -853,36 +853,48 @@ def _auth_lock_path() -> Path:
|
||||
|
||||
_auth_lock_holder = threading.local()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _auth_store_lock(timeout_seconds: float = AUTH_LOCK_TIMEOUT_SECONDS):
|
||||
"""Cross-process advisory lock for auth.json reads+writes. Reentrant."""
|
||||
# Reentrant: if this thread already holds the lock, just yield.
|
||||
if getattr(_auth_lock_holder, "depth", 0) > 0:
|
||||
_auth_lock_holder.depth += 1
|
||||
def _file_lock(
|
||||
lock_path: Path,
|
||||
holder: threading.local,
|
||||
timeout_seconds: float,
|
||||
timeout_message: str,
|
||||
):
|
||||
"""Cross-process advisory flock helper.
|
||||
|
||||
Reentrant per-thread via ``holder.depth``. Falls back to a depth-only
|
||||
guard when neither ``fcntl`` nor ``msvcrt`` is available (rare).
|
||||
Callers supply their own ``threading.local`` so independent locks
|
||||
(e.g. profile auth.json vs shared Nous store) don't share reentrancy
|
||||
state — that would let one lock's reentrant acquisition silently skip
|
||||
the other's kernel-level flock.
|
||||
"""
|
||||
if getattr(holder, "depth", 0) > 0:
|
||||
holder.depth += 1
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
_auth_lock_holder.depth -= 1
|
||||
holder.depth -= 1
|
||||
return
|
||||
|
||||
lock_path = _auth_lock_path()
|
||||
lock_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if fcntl is None and msvcrt is None:
|
||||
_auth_lock_holder.depth = 1
|
||||
holder.depth = 1
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
_auth_lock_holder.depth = 0
|
||||
holder.depth = 0
|
||||
return
|
||||
|
||||
# On Windows, msvcrt.locking needs the file to have content and the
|
||||
# file pointer at position 0. Ensure the lock file has at least 1 byte.
|
||||
# file pointer at position 0. Ensure the lock file has at least 1 byte.
|
||||
if msvcrt and (not lock_path.exists() or lock_path.stat().st_size == 0):
|
||||
lock_path.write_text(" ", encoding="utf-8")
|
||||
|
||||
with lock_path.open("r+" if msvcrt else "a+") as lock_file:
|
||||
deadline = time.time() + max(1.0, timeout_seconds)
|
||||
deadline = time.monotonic() + max(1.0, timeout_seconds)
|
||||
while True:
|
||||
try:
|
||||
if fcntl:
|
||||
@@ -892,15 +904,15 @@ def _auth_store_lock(timeout_seconds: float = AUTH_LOCK_TIMEOUT_SECONDS):
|
||||
msvcrt.locking(lock_file.fileno(), msvcrt.LK_NBLCK, 1)
|
||||
break
|
||||
except (BlockingIOError, OSError, PermissionError):
|
||||
if time.time() >= deadline:
|
||||
raise TimeoutError("Timed out waiting for auth store lock")
|
||||
if time.monotonic() >= deadline:
|
||||
raise TimeoutError(timeout_message)
|
||||
time.sleep(0.05)
|
||||
|
||||
_auth_lock_holder.depth = 1
|
||||
holder.depth = 1
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
_auth_lock_holder.depth = 0
|
||||
holder.depth = 0
|
||||
if fcntl:
|
||||
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
|
||||
elif msvcrt:
|
||||
@@ -911,6 +923,25 @@ def _auth_store_lock(timeout_seconds: float = AUTH_LOCK_TIMEOUT_SECONDS):
|
||||
pass
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _auth_store_lock(timeout_seconds: float = AUTH_LOCK_TIMEOUT_SECONDS):
|
||||
"""Cross-process advisory lock for auth.json reads+writes. Reentrant.
|
||||
|
||||
Lock ordering invariant: when this lock is held together with
|
||||
``_nous_shared_store_lock``, acquire ``_auth_store_lock`` FIRST
|
||||
(outer) and the shared Nous lock SECOND (inner). All runtime
|
||||
refresh paths follow this order; violating it risks deadlock
|
||||
against a concurrent import on the shared store.
|
||||
"""
|
||||
with _file_lock(
|
||||
_auth_lock_path(),
|
||||
_auth_lock_holder,
|
||||
timeout_seconds,
|
||||
"Timed out waiting for auth store lock",
|
||||
):
|
||||
yield
|
||||
|
||||
|
||||
def _load_auth_store(auth_file: Optional[Path] = None) -> Dict[str, Any]:
|
||||
auth_file = auth_file or _auth_file_path()
|
||||
if not auth_file.exists():
|
||||
@@ -954,12 +985,27 @@ def _load_auth_store(auth_file: Optional[Path] = None) -> Dict[str, Any]:
|
||||
def _save_auth_store(auth_store: Dict[str, Any]) -> Path:
|
||||
auth_file = _auth_file_path()
|
||||
auth_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
# Tighten parent dir to 0o700 so siblings can't traverse to creds.
|
||||
# No-op on Windows (POSIX mode bits not enforced); ignore failures.
|
||||
try:
|
||||
os.chmod(auth_file.parent, 0o700)
|
||||
except OSError:
|
||||
pass
|
||||
auth_store["version"] = AUTH_STORE_VERSION
|
||||
auth_store["updated_at"] = datetime.now(timezone.utc).isoformat()
|
||||
payload = json.dumps(auth_store, indent=2) + "\n"
|
||||
tmp_path = auth_file.with_name(f"{auth_file.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
|
||||
try:
|
||||
with tmp_path.open("w", encoding="utf-8") as handle:
|
||||
# Create with 0o600 atomically via os.open(O_EXCL) + fdopen to close
|
||||
# the TOCTOU window where default umask (often 0o644) briefly exposed
|
||||
# OAuth tokens to other local users between open() and chmod().
|
||||
# Mirrors agent/google_oauth.py (#19673) and tools/mcp_oauth.py (#21148).
|
||||
fd = os.open(
|
||||
str(tmp_path),
|
||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
|
||||
stat.S_IRUSR | stat.S_IWUSR,
|
||||
)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as handle:
|
||||
handle.write(payload)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
@@ -1523,10 +1569,33 @@ def _read_qwen_cli_tokens() -> Dict[str, Any]:
|
||||
def _save_qwen_cli_tokens(tokens: Dict[str, Any]) -> Path:
|
||||
auth_path = _qwen_cli_auth_path()
|
||||
auth_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp_path = auth_path.with_suffix(".tmp")
|
||||
tmp_path.write_text(json.dumps(tokens, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||
os.chmod(tmp_path, stat.S_IRUSR | stat.S_IWUSR)
|
||||
tmp_path.replace(auth_path)
|
||||
try:
|
||||
os.chmod(auth_path.parent, 0o700)
|
||||
except OSError:
|
||||
pass
|
||||
# Per-process random temp suffix avoids collisions between concurrent
|
||||
# writers and stale leftovers from a crashed prior write.
|
||||
tmp_path = auth_path.with_name(f"{auth_path.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
|
||||
# Create with 0o600 atomically via os.open(O_EXCL) — closes the TOCTOU
|
||||
# window where write_text() + post-write chmod briefly exposed tokens
|
||||
# at process umask (typically 0o644). See #19673, #21148.
|
||||
fd = os.open(
|
||||
str(tmp_path),
|
||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
|
||||
stat.S_IRUSR | stat.S_IWUSR,
|
||||
)
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(tokens, indent=2, sort_keys=True) + "\n")
|
||||
fh.flush()
|
||||
os.fsync(fh.fileno())
|
||||
atomic_replace(tmp_path, auth_path)
|
||||
finally:
|
||||
try:
|
||||
if tmp_path.exists():
|
||||
tmp_path.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
return auth_path
|
||||
|
||||
|
||||
@@ -1943,9 +2012,9 @@ def _spotify_wait_for_callback(
|
||||
|
||||
thread = threading.Thread(target=server.serve_forever, kwargs={"poll_interval": 0.1}, daemon=True)
|
||||
thread.start()
|
||||
deadline = time.time() + max(5.0, timeout_seconds)
|
||||
deadline = time.monotonic() + max(5.0, timeout_seconds)
|
||||
try:
|
||||
while time.time() < deadline:
|
||||
while time.monotonic() < deadline:
|
||||
if result["code"] or result["error"]:
|
||||
return result
|
||||
time.sleep(0.1)
|
||||
@@ -2708,10 +2777,10 @@ def _poll_for_token(
|
||||
poll_interval: int,
|
||||
) -> Dict[str, Any]:
|
||||
"""Poll the token endpoint until the user approves or the code expires."""
|
||||
deadline = time.time() + max(1, expires_in)
|
||||
deadline = time.monotonic() + max(1, expires_in)
|
||||
current_interval = max(1, min(poll_interval, DEVICE_AUTH_POLL_INTERVAL_CAP_SECONDS))
|
||||
|
||||
while time.time() < deadline:
|
||||
while time.monotonic() < deadline:
|
||||
response = client.post(
|
||||
f"{portal_base_url}/api/oauth/token",
|
||||
data={
|
||||
@@ -2769,6 +2838,7 @@ def _poll_for_token(
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
NOUS_SHARED_STORE_FILENAME = "nous_auth.json"
|
||||
_nous_shared_lock_holder = threading.local()
|
||||
|
||||
|
||||
def _nous_shared_auth_dir() -> Path:
|
||||
@@ -2808,6 +2878,69 @@ def _nous_shared_store_path() -> Path:
|
||||
return path
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _nous_shared_store_lock(timeout_seconds: float = AUTH_LOCK_TIMEOUT_SECONDS):
|
||||
"""Cross-profile lock for the shared Nous OAuth store.
|
||||
|
||||
Lock ordering invariant: if both this and ``_auth_store_lock`` need
|
||||
to be held, acquire ``_auth_store_lock`` FIRST. All runtime refresh
|
||||
paths follow this order. The one exception is
|
||||
``_try_import_shared_nous_state``, which holds this lock alone for
|
||||
the entire refresh+mint cycle so concurrent imports on sibling
|
||||
profiles can't race on the single-use shared refresh token; that
|
||||
helper must NOT be called with ``_auth_store_lock`` already held.
|
||||
"""
|
||||
try:
|
||||
lock_path = _nous_shared_store_path().with_suffix(".lock")
|
||||
except RuntimeError:
|
||||
# No HERMES_HOME yet (pre-setup): fall through without locking.
|
||||
yield
|
||||
return
|
||||
|
||||
with _file_lock(
|
||||
lock_path,
|
||||
_nous_shared_lock_holder,
|
||||
timeout_seconds,
|
||||
"Timed out waiting for shared Nous auth lock",
|
||||
):
|
||||
yield
|
||||
|
||||
|
||||
def _merge_shared_nous_oauth_state(state: Dict[str, Any]) -> bool:
|
||||
"""Copy fresher shared OAuth tokens into a profile-local Nous state."""
|
||||
shared = _read_shared_nous_state()
|
||||
if not shared:
|
||||
return False
|
||||
|
||||
shared_refresh = shared.get("refresh_token")
|
||||
if not isinstance(shared_refresh, str) or not shared_refresh.strip():
|
||||
return False
|
||||
|
||||
local_refresh = state.get("refresh_token")
|
||||
shared_access_exp = _parse_iso_timestamp(shared.get("expires_at")) or 0.0
|
||||
local_access_exp = _parse_iso_timestamp(state.get("expires_at")) or 0.0
|
||||
refresh_changed = shared_refresh.strip() != str(local_refresh or "").strip()
|
||||
fresher_access = shared_access_exp > local_access_exp
|
||||
if not refresh_changed and not fresher_access:
|
||||
return False
|
||||
|
||||
for key in (
|
||||
"access_token",
|
||||
"refresh_token",
|
||||
"token_type",
|
||||
"scope",
|
||||
"client_id",
|
||||
"portal_base_url",
|
||||
"inference_base_url",
|
||||
"obtained_at",
|
||||
"expires_at",
|
||||
):
|
||||
value = shared.get(key)
|
||||
if value not in (None, ""):
|
||||
state[key] = value
|
||||
return True
|
||||
|
||||
|
||||
def _write_shared_nous_state(state: Dict[str, Any]) -> None:
|
||||
"""Persist a minimal copy of the Nous OAuth state to the shared store.
|
||||
|
||||
@@ -2840,15 +2973,34 @@ def _write_shared_nous_state(state: Dict[str, Any]) -> None:
|
||||
"updated_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
try:
|
||||
path = _nous_shared_store_path()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||
tmp.write_text(json.dumps(shared, indent=2, sort_keys=True))
|
||||
try:
|
||||
os.chmod(tmp, 0o600)
|
||||
except OSError:
|
||||
pass
|
||||
os.replace(tmp, path)
|
||||
with _nous_shared_store_lock():
|
||||
path = _nous_shared_store_path()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
os.chmod(path.parent, 0o700)
|
||||
except OSError:
|
||||
pass
|
||||
tmp = path.with_name(f"{path.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
|
||||
# Create with 0o600 atomically via os.open(O_EXCL) — closes the TOCTOU
|
||||
# window where write_text() + post-write chmod briefly exposed Nous
|
||||
# refresh_token at process umask. See #19673, #21148.
|
||||
fd = os.open(
|
||||
str(tmp),
|
||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
|
||||
stat.S_IRUSR | stat.S_IWUSR,
|
||||
)
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(shared, indent=2, sort_keys=True))
|
||||
fh.flush()
|
||||
os.fsync(fh.fileno())
|
||||
os.replace(tmp, path)
|
||||
finally:
|
||||
try:
|
||||
if tmp.exists():
|
||||
tmp.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
_oauth_trace(
|
||||
"nous_shared_store_written",
|
||||
path=str(path),
|
||||
@@ -2905,36 +3057,38 @@ def _try_import_shared_nous_state(
|
||||
etc.) — caller should then fall through to the normal device-code
|
||||
flow.
|
||||
"""
|
||||
shared = _read_shared_nous_state()
|
||||
if not shared:
|
||||
return None
|
||||
|
||||
# Build a full state dict so refresh_nous_oauth_from_state has every
|
||||
# field it needs. force_refresh=True gets us a fresh access_token
|
||||
# for this profile; force_mint=True gets us a fresh agent_key.
|
||||
state: Dict[str, Any] = {
|
||||
"access_token": shared.get("access_token"),
|
||||
"refresh_token": shared.get("refresh_token"),
|
||||
"client_id": shared.get("client_id") or DEFAULT_NOUS_CLIENT_ID,
|
||||
"portal_base_url": shared.get("portal_base_url") or DEFAULT_NOUS_PORTAL_URL,
|
||||
"inference_base_url": shared.get("inference_base_url") or DEFAULT_NOUS_INFERENCE_URL,
|
||||
"token_type": shared.get("token_type") or "Bearer",
|
||||
"scope": shared.get("scope") or DEFAULT_NOUS_SCOPE,
|
||||
"obtained_at": shared.get("obtained_at"),
|
||||
"expires_at": shared.get("expires_at"),
|
||||
"agent_key": None,
|
||||
"agent_key_expires_at": None,
|
||||
"tls": {"insecure": False, "ca_bundle": None},
|
||||
}
|
||||
|
||||
try:
|
||||
refreshed = refresh_nous_oauth_from_state(
|
||||
state,
|
||||
min_key_ttl_seconds=min_key_ttl_seconds,
|
||||
timeout_seconds=timeout_seconds,
|
||||
force_refresh=True,
|
||||
force_mint=True,
|
||||
)
|
||||
with _nous_shared_store_lock(timeout_seconds=max(timeout_seconds + 5.0, AUTH_LOCK_TIMEOUT_SECONDS)):
|
||||
shared = _read_shared_nous_state()
|
||||
if not shared:
|
||||
return None
|
||||
|
||||
# Build a full state dict so refresh_nous_oauth_from_state has every
|
||||
# field it needs. force_refresh=True gets us a fresh access_token
|
||||
# for this profile; force_mint=True gets us a fresh agent_key.
|
||||
state: Dict[str, Any] = {
|
||||
"access_token": shared.get("access_token"),
|
||||
"refresh_token": shared.get("refresh_token"),
|
||||
"client_id": shared.get("client_id") or DEFAULT_NOUS_CLIENT_ID,
|
||||
"portal_base_url": shared.get("portal_base_url") or DEFAULT_NOUS_PORTAL_URL,
|
||||
"inference_base_url": shared.get("inference_base_url") or DEFAULT_NOUS_INFERENCE_URL,
|
||||
"token_type": shared.get("token_type") or "Bearer",
|
||||
"scope": shared.get("scope") or DEFAULT_NOUS_SCOPE,
|
||||
"obtained_at": shared.get("obtained_at"),
|
||||
"expires_at": shared.get("expires_at"),
|
||||
"agent_key": None,
|
||||
"agent_key_expires_at": None,
|
||||
"tls": {"insecure": False, "ca_bundle": None},
|
||||
}
|
||||
|
||||
refreshed = refresh_nous_oauth_from_state(
|
||||
state,
|
||||
min_key_ttl_seconds=min_key_ttl_seconds,
|
||||
timeout_seconds=timeout_seconds,
|
||||
force_refresh=True,
|
||||
force_mint=True,
|
||||
)
|
||||
_write_shared_nous_state(refreshed)
|
||||
except AuthError as exc:
|
||||
_oauth_trace(
|
||||
"nous_shared_import_failed",
|
||||
@@ -3136,59 +3290,65 @@ def resolve_nous_access_token(
|
||||
client_id = str(state.get("client_id") or DEFAULT_NOUS_CLIENT_ID)
|
||||
verify = _resolve_verify(insecure=insecure, ca_bundle=ca_bundle, auth_state=state)
|
||||
|
||||
access_token = state.get("access_token")
|
||||
refresh_token = state.get("refresh_token")
|
||||
if not isinstance(access_token, str) or not access_token:
|
||||
raise AuthError(
|
||||
"No access token found for Nous Portal login.",
|
||||
provider="nous",
|
||||
relogin_required=True,
|
||||
)
|
||||
with _nous_shared_store_lock(timeout_seconds=max(timeout_seconds + 5.0, AUTH_LOCK_TIMEOUT_SECONDS)):
|
||||
merged_shared = _merge_shared_nous_oauth_state(state)
|
||||
access_token = state.get("access_token")
|
||||
refresh_token = state.get("refresh_token")
|
||||
if not isinstance(access_token, str) or not access_token:
|
||||
raise AuthError(
|
||||
"No access token found for Nous Portal login.",
|
||||
provider="nous",
|
||||
relogin_required=True,
|
||||
)
|
||||
|
||||
if not _is_expiring(state.get("expires_at"), refresh_skew_seconds):
|
||||
return access_token
|
||||
if not _is_expiring(state.get("expires_at"), refresh_skew_seconds):
|
||||
if merged_shared:
|
||||
_save_provider_state(auth_store, "nous", state)
|
||||
_save_auth_store(auth_store)
|
||||
return access_token
|
||||
|
||||
if not isinstance(refresh_token, str) or not refresh_token:
|
||||
raise AuthError(
|
||||
"Session expired and no refresh token is available.",
|
||||
provider="nous",
|
||||
relogin_required=True,
|
||||
)
|
||||
if not isinstance(refresh_token, str) or not refresh_token:
|
||||
raise AuthError(
|
||||
"Session expired and no refresh token is available.",
|
||||
provider="nous",
|
||||
relogin_required=True,
|
||||
)
|
||||
|
||||
timeout = httpx.Timeout(timeout_seconds if timeout_seconds else 15.0)
|
||||
with httpx.Client(
|
||||
timeout=timeout,
|
||||
headers={"Accept": "application/json"},
|
||||
verify=verify,
|
||||
) as client:
|
||||
refreshed = _refresh_access_token(
|
||||
client=client,
|
||||
portal_base_url=portal_base_url,
|
||||
client_id=client_id,
|
||||
refresh_token=refresh_token,
|
||||
)
|
||||
timeout = httpx.Timeout(timeout_seconds if timeout_seconds else 15.0)
|
||||
with httpx.Client(
|
||||
timeout=timeout,
|
||||
headers={"Accept": "application/json"},
|
||||
verify=verify,
|
||||
) as client:
|
||||
refreshed = _refresh_access_token(
|
||||
client=client,
|
||||
portal_base_url=portal_base_url,
|
||||
client_id=client_id,
|
||||
refresh_token=refresh_token,
|
||||
)
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||
state["access_token"] = refreshed["access_token"]
|
||||
state["refresh_token"] = refreshed.get("refresh_token") or refresh_token
|
||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||
state["obtained_at"] = now.isoformat()
|
||||
state["expires_in"] = access_ttl
|
||||
state["expires_at"] = datetime.fromtimestamp(
|
||||
now.timestamp() + access_ttl,
|
||||
tz=timezone.utc,
|
||||
).isoformat()
|
||||
state["portal_base_url"] = portal_base_url
|
||||
state["client_id"] = client_id
|
||||
state["tls"] = {
|
||||
"insecure": verify is False,
|
||||
"ca_bundle": verify if isinstance(verify, str) else None,
|
||||
}
|
||||
_save_provider_state(auth_store, "nous", state)
|
||||
_save_auth_store(auth_store)
|
||||
return state["access_token"]
|
||||
now = datetime.now(timezone.utc)
|
||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||
state["access_token"] = refreshed["access_token"]
|
||||
state["refresh_token"] = refreshed.get("refresh_token") or refresh_token
|
||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||
state["obtained_at"] = now.isoformat()
|
||||
state["expires_in"] = access_ttl
|
||||
state["expires_at"] = datetime.fromtimestamp(
|
||||
now.timestamp() + access_ttl,
|
||||
tz=timezone.utc,
|
||||
).isoformat()
|
||||
state["portal_base_url"] = portal_base_url
|
||||
state["client_id"] = client_id
|
||||
state["tls"] = {
|
||||
"insecure": verify is False,
|
||||
"ca_bundle": verify if isinstance(verify, str) else None,
|
||||
}
|
||||
_save_provider_state(auth_store, "nous", state)
|
||||
_save_auth_store(auth_store)
|
||||
_write_shared_nous_state(state)
|
||||
return state["access_token"]
|
||||
|
||||
|
||||
def refresh_nous_oauth_pure(
|
||||
@@ -3456,46 +3616,53 @@ def resolve_nous_runtime_credentials(
|
||||
|
||||
# Step 1: refresh access token if expiring
|
||||
if _is_expiring(state.get("expires_at"), ACCESS_TOKEN_REFRESH_SKEW_SECONDS):
|
||||
if not isinstance(refresh_token, str) or not refresh_token:
|
||||
raise AuthError("Session expired and no refresh token is available.",
|
||||
provider="nous", relogin_required=True)
|
||||
with _nous_shared_store_lock(timeout_seconds=max(timeout_seconds + 5.0, AUTH_LOCK_TIMEOUT_SECONDS)):
|
||||
if _merge_shared_nous_oauth_state(state):
|
||||
access_token = state.get("access_token")
|
||||
refresh_token = state.get("refresh_token")
|
||||
_persist_state("post_shared_merge_access_expiring")
|
||||
|
||||
_oauth_trace(
|
||||
"refresh_start",
|
||||
sequence_id=sequence_id,
|
||||
reason="access_expiring",
|
||||
refresh_token_fp=_token_fingerprint(refresh_token),
|
||||
)
|
||||
refreshed = _refresh_access_token(
|
||||
client=client, portal_base_url=portal_base_url,
|
||||
client_id=client_id, refresh_token=refresh_token,
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||
previous_refresh_token = refresh_token
|
||||
state["access_token"] = refreshed["access_token"]
|
||||
state["refresh_token"] = refreshed.get("refresh_token") or refresh_token
|
||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||
refreshed_url = _optional_base_url(refreshed.get("inference_base_url"))
|
||||
if refreshed_url:
|
||||
inference_base_url = refreshed_url
|
||||
state["obtained_at"] = now.isoformat()
|
||||
state["expires_in"] = access_ttl
|
||||
state["expires_at"] = datetime.fromtimestamp(
|
||||
now.timestamp() + access_ttl, tz=timezone.utc
|
||||
).isoformat()
|
||||
access_token = state["access_token"]
|
||||
refresh_token = state["refresh_token"]
|
||||
_oauth_trace(
|
||||
"refresh_success",
|
||||
sequence_id=sequence_id,
|
||||
reason="access_expiring",
|
||||
previous_refresh_token_fp=_token_fingerprint(previous_refresh_token),
|
||||
new_refresh_token_fp=_token_fingerprint(refresh_token),
|
||||
)
|
||||
# Persist immediately so downstream mint failures cannot drop rotated refresh tokens.
|
||||
_persist_state("post_refresh_access_expiring")
|
||||
if _is_expiring(state.get("expires_at"), ACCESS_TOKEN_REFRESH_SKEW_SECONDS):
|
||||
if not isinstance(refresh_token, str) or not refresh_token:
|
||||
raise AuthError("Session expired and no refresh token is available.",
|
||||
provider="nous", relogin_required=True)
|
||||
|
||||
_oauth_trace(
|
||||
"refresh_start",
|
||||
sequence_id=sequence_id,
|
||||
reason="access_expiring",
|
||||
refresh_token_fp=_token_fingerprint(refresh_token),
|
||||
)
|
||||
refreshed = _refresh_access_token(
|
||||
client=client, portal_base_url=portal_base_url,
|
||||
client_id=client_id, refresh_token=refresh_token,
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||
previous_refresh_token = refresh_token
|
||||
state["access_token"] = refreshed["access_token"]
|
||||
state["refresh_token"] = refreshed.get("refresh_token") or refresh_token
|
||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||
refreshed_url = _optional_base_url(refreshed.get("inference_base_url"))
|
||||
if refreshed_url:
|
||||
inference_base_url = refreshed_url
|
||||
state["obtained_at"] = now.isoformat()
|
||||
state["expires_in"] = access_ttl
|
||||
state["expires_at"] = datetime.fromtimestamp(
|
||||
now.timestamp() + access_ttl, tz=timezone.utc
|
||||
).isoformat()
|
||||
access_token = state["access_token"]
|
||||
refresh_token = state["refresh_token"]
|
||||
_oauth_trace(
|
||||
"refresh_success",
|
||||
sequence_id=sequence_id,
|
||||
reason="access_expiring",
|
||||
previous_refresh_token_fp=_token_fingerprint(previous_refresh_token),
|
||||
new_refresh_token_fp=_token_fingerprint(refresh_token),
|
||||
)
|
||||
# Persist immediately so downstream mint failures cannot drop rotated refresh tokens.
|
||||
_persist_state("post_refresh_access_expiring")
|
||||
|
||||
# Step 2: mint agent key if missing/expiring
|
||||
used_cached_key = False
|
||||
@@ -3528,41 +3695,47 @@ def resolve_nous_runtime_credentials(
|
||||
and isinstance(latest_refresh_token, str)
|
||||
and latest_refresh_token
|
||||
):
|
||||
_oauth_trace(
|
||||
"refresh_start",
|
||||
sequence_id=sequence_id,
|
||||
reason="mint_retry_after_invalid_token",
|
||||
refresh_token_fp=_token_fingerprint(latest_refresh_token),
|
||||
)
|
||||
refreshed = _refresh_access_token(
|
||||
client=client, portal_base_url=portal_base_url,
|
||||
client_id=client_id, refresh_token=latest_refresh_token,
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||
state["access_token"] = refreshed["access_token"]
|
||||
state["refresh_token"] = refreshed.get("refresh_token") or latest_refresh_token
|
||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||
refreshed_url = _optional_base_url(refreshed.get("inference_base_url"))
|
||||
if refreshed_url:
|
||||
inference_base_url = refreshed_url
|
||||
state["obtained_at"] = now.isoformat()
|
||||
state["expires_in"] = access_ttl
|
||||
state["expires_at"] = datetime.fromtimestamp(
|
||||
now.timestamp() + access_ttl, tz=timezone.utc
|
||||
).isoformat()
|
||||
access_token = state["access_token"]
|
||||
refresh_token = state["refresh_token"]
|
||||
_oauth_trace(
|
||||
"refresh_success",
|
||||
sequence_id=sequence_id,
|
||||
reason="mint_retry_after_invalid_token",
|
||||
previous_refresh_token_fp=_token_fingerprint(latest_refresh_token),
|
||||
new_refresh_token_fp=_token_fingerprint(refresh_token),
|
||||
)
|
||||
# Persist retry refresh immediately for crash safety and cross-process visibility.
|
||||
_persist_state("post_refresh_mint_retry")
|
||||
with _nous_shared_store_lock(timeout_seconds=max(timeout_seconds + 5.0, AUTH_LOCK_TIMEOUT_SECONDS)):
|
||||
if _merge_shared_nous_oauth_state(state):
|
||||
access_token = state.get("access_token")
|
||||
latest_refresh_token = state.get("refresh_token")
|
||||
_persist_state("post_shared_merge_mint_retry")
|
||||
else:
|
||||
_oauth_trace(
|
||||
"refresh_start",
|
||||
sequence_id=sequence_id,
|
||||
reason="mint_retry_after_invalid_token",
|
||||
refresh_token_fp=_token_fingerprint(latest_refresh_token),
|
||||
)
|
||||
refreshed = _refresh_access_token(
|
||||
client=client, portal_base_url=portal_base_url,
|
||||
client_id=client_id, refresh_token=latest_refresh_token,
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
access_ttl = _coerce_ttl_seconds(refreshed.get("expires_in"))
|
||||
state["access_token"] = refreshed["access_token"]
|
||||
state["refresh_token"] = refreshed.get("refresh_token") or latest_refresh_token
|
||||
state["token_type"] = refreshed.get("token_type") or state.get("token_type") or "Bearer"
|
||||
state["scope"] = refreshed.get("scope") or state.get("scope")
|
||||
refreshed_url = _optional_base_url(refreshed.get("inference_base_url"))
|
||||
if refreshed_url:
|
||||
inference_base_url = refreshed_url
|
||||
state["obtained_at"] = now.isoformat()
|
||||
state["expires_in"] = access_ttl
|
||||
state["expires_at"] = datetime.fromtimestamp(
|
||||
now.timestamp() + access_ttl, tz=timezone.utc
|
||||
).isoformat()
|
||||
access_token = state["access_token"]
|
||||
refresh_token = state["refresh_token"]
|
||||
_oauth_trace(
|
||||
"refresh_success",
|
||||
sequence_id=sequence_id,
|
||||
reason="mint_retry_after_invalid_token",
|
||||
previous_refresh_token_fp=_token_fingerprint(latest_refresh_token),
|
||||
new_refresh_token_fp=_token_fingerprint(refresh_token),
|
||||
)
|
||||
# Persist retry refresh immediately for crash safety and cross-process visibility.
|
||||
_persist_state("post_refresh_mint_retry")
|
||||
|
||||
mint_payload = _mint_agent_key(
|
||||
client=client, portal_base_url=portal_base_url,
|
||||
@@ -4058,6 +4231,14 @@ def _config_provider_matches(provider_id: Optional[str]) -> bool:
|
||||
return _get_config_provider() == provider_id.strip().lower()
|
||||
|
||||
|
||||
def _should_reset_config_provider_on_logout(provider_id: Optional[str]) -> bool:
|
||||
"""Return True when logout should reset the model provider config."""
|
||||
if not provider_id:
|
||||
return False
|
||||
normalized = provider_id.strip().lower()
|
||||
return normalized in PROVIDER_REGISTRY and _config_provider_matches(normalized)
|
||||
|
||||
|
||||
def _logout_default_provider_from_config() -> Optional[str]:
|
||||
"""Fallback logout target when auth.json has no active provider.
|
||||
|
||||
@@ -4216,7 +4397,7 @@ def _prompt_model_selection(
|
||||
clear_screen=False,
|
||||
title=effective_title,
|
||||
)
|
||||
idx: int | None = menu.show() # ty:ignore[invalid-assignment] - TerminalMenu.show() is always `int | None` when multi_select is False / not provided.
|
||||
idx = menu.show()
|
||||
from hermes_cli.curses_ui import flush_stdin
|
||||
flush_stdin()
|
||||
if idx is None:
|
||||
@@ -5143,15 +5324,18 @@ def logout_command(args) -> None:
|
||||
print("No provider is currently logged in.")
|
||||
return
|
||||
|
||||
config_matches = _config_provider_matches(target)
|
||||
should_reset_config = _should_reset_config_provider_on_logout(target)
|
||||
provider_name = get_auth_provider_display_name(target)
|
||||
|
||||
if clear_provider_auth(target) or config_matches:
|
||||
_reset_config_provider()
|
||||
if clear_provider_auth(target) or should_reset_config:
|
||||
if should_reset_config:
|
||||
_reset_config_provider()
|
||||
print(f"Logged out of {provider_name}.")
|
||||
if os.getenv("OPENROUTER_API_KEY"):
|
||||
if should_reset_config and os.getenv("OPENROUTER_API_KEY"):
|
||||
print("Hermes will use OpenRouter for inference.")
|
||||
else:
|
||||
elif should_reset_config:
|
||||
print("Run `hermes model` or configure an API key to use Hermes.")
|
||||
else:
|
||||
print("Model provider configuration was unchanged.")
|
||||
else:
|
||||
print(f"No auth state found for {provider_name}.")
|
||||
|
||||
@@ -573,7 +573,7 @@ def create_quick_snapshot(
|
||||
"total_size": sum(manifest.values()),
|
||||
"files": manifest,
|
||||
}
|
||||
with open(snap_dir / "manifest.json", "w") as f:
|
||||
with open(snap_dir / "manifest.json", "w", encoding="utf-8") as f:
|
||||
json.dump(meta, f, indent=2)
|
||||
|
||||
# Auto-prune
|
||||
@@ -599,7 +599,7 @@ def list_quick_snapshots(
|
||||
manifest_path = d / "manifest.json"
|
||||
if manifest_path.exists():
|
||||
try:
|
||||
with open(manifest_path) as f:
|
||||
with open(manifest_path, encoding="utf-8") as f:
|
||||
results.append(json.load(f))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
results.append({"id": d.name, "file_count": 0, "total_size": 0})
|
||||
@@ -629,7 +629,7 @@ def restore_quick_snapshot(
|
||||
if not manifest_path.exists():
|
||||
return False
|
||||
|
||||
with open(manifest_path) as f:
|
||||
with open(manifest_path, encoding="utf-8") as f:
|
||||
meta = json.load(f)
|
||||
|
||||
restored = 0
|
||||
|
||||
@@ -157,9 +157,9 @@ COMMAND_REGISTRY: list[CommandDef] = [
|
||||
CommandDef("cron", "Manage scheduled tasks", "Tools & Skills",
|
||||
cli_only=True, args_hint="[subcommand]",
|
||||
subcommands=("list", "add", "create", "edit", "pause", "resume", "run", "remove")),
|
||||
CommandDef("curator", "Background skill maintenance (status, run, pin, archive)",
|
||||
CommandDef("curator", "Background skill maintenance (status, run, pin, archive, list-archived)",
|
||||
"Tools & Skills", args_hint="[subcommand]",
|
||||
subcommands=("status", "run", "pause", "resume", "pin", "unpin", "restore")),
|
||||
subcommands=("status", "run", "pause", "resume", "pin", "unpin", "restore", "list-archived")),
|
||||
CommandDef("kanban", "Multi-profile collaboration board (tasks, links, comments)",
|
||||
"Tools & Skills", args_hint="[subcommand]",
|
||||
subcommands=("list", "ls", "show", "create", "assign", "link", "unlink",
|
||||
|
||||
+169
-19
@@ -212,7 +212,7 @@ def get_container_exec_info() -> Optional[dict]:
|
||||
|
||||
try:
|
||||
info = {}
|
||||
with open(container_mode_file, "r") as f:
|
||||
with open(container_mode_file, "r", encoding="utf-8") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if "=" in line and not line.startswith("#"):
|
||||
@@ -297,7 +297,7 @@ def _is_container() -> bool:
|
||||
return True
|
||||
# LXC / cgroup-based detection
|
||||
try:
|
||||
with open("/proc/1/cgroup", "r") as f:
|
||||
with open("/proc/1/cgroup", "r", encoding="utf-8") as f:
|
||||
cgroup_content = f.read()
|
||||
if "docker" in cgroup_content or "lxc" in cgroup_content or "kubepods" in cgroup_content:
|
||||
return True
|
||||
@@ -780,6 +780,19 @@ DEFAULT_CONFIG = {
|
||||
"timeout": 30,
|
||||
"extra_body": {},
|
||||
},
|
||||
# Triage specifier — flesh out a rough one-liner in the Kanban
|
||||
# Triage column into a concrete spec, then promote it to ``todo``.
|
||||
# Invoked by ``hermes kanban specify`` (single id or --all). Set a
|
||||
# cheap, capable model here (gemini-flash works well); the main
|
||||
# model is overkill for short spec expansion.
|
||||
"triage_specifier": {
|
||||
"provider": "auto",
|
||||
"model": "",
|
||||
"base_url": "",
|
||||
"api_key": "",
|
||||
"timeout": 120,
|
||||
"extra_body": {},
|
||||
},
|
||||
# Curator — skill-usage review fork. Timeout is generous because the
|
||||
# review pass can take several minutes on reasoning models (umbrella
|
||||
# building over hundreds of candidate skills). "auto" = use main chat
|
||||
@@ -1100,6 +1113,14 @@ DEFAULT_CONFIG = {
|
||||
# Empty string means use server-local time.
|
||||
"timezone": "",
|
||||
|
||||
# Slack platform settings (gateway mode)
|
||||
"slack": {
|
||||
"require_mention": True, # Require @mention to respond in channels
|
||||
"free_response_channels": "", # Comma-separated channel IDs where bot responds without mention
|
||||
"allowed_channels": "", # If set, bot ONLY responds in these channel IDs (whitelist)
|
||||
"channel_prompts": {}, # Per-channel ephemeral system prompts
|
||||
},
|
||||
|
||||
# Discord platform settings (gateway mode)
|
||||
"discord": {
|
||||
"require_mention": True, # Require @mention to respond in server channels
|
||||
@@ -1108,6 +1129,12 @@ DEFAULT_CONFIG = {
|
||||
"auto_thread": True, # Auto-create threads on @mention in channels (like Slack)
|
||||
"reactions": True, # Add 👀/✅/❌ reactions to messages during processing
|
||||
"channel_prompts": {}, # Per-channel ephemeral system prompts (forum parents apply to child threads)
|
||||
# Opt-in DM role-based auth (#12136). By default, DISCORD_ALLOWED_ROLES
|
||||
# authorizes only guild messages in the role's own guild — DMs require
|
||||
# DISCORD_ALLOWED_USERS. Set dm_role_auth_guild to a guild ID to also
|
||||
# authorize DMs from members of that one trusted guild holding the
|
||||
# allowed role. Unset / empty / 0 = secure default (DM role-auth off).
|
||||
"dm_role_auth_guild": "",
|
||||
# discord / discord_admin tools: restrict which actions the agent may call.
|
||||
# Default (empty) = all actions allowed (subject to bot privileged intents).
|
||||
# Accepts comma-separated string ("list_guilds,list_channels,fetch_messages")
|
||||
@@ -1130,18 +1157,24 @@ DEFAULT_CONFIG = {
|
||||
"telegram": {
|
||||
"reactions": False, # Add 👀/✅/❌ reactions to messages during processing
|
||||
"channel_prompts": {}, # Per-chat/topic ephemeral system prompts (topics inherit from parent group)
|
||||
},
|
||||
|
||||
# Slack platform settings (gateway mode)
|
||||
"slack": {
|
||||
"channel_prompts": {}, # Per-channel ephemeral system prompts
|
||||
"allowed_chats": "", # If set, bot ONLY responds in these group/supergroup chat IDs (whitelist)
|
||||
},
|
||||
|
||||
# Mattermost platform settings (gateway mode)
|
||||
"mattermost": {
|
||||
"require_mention": True, # Require @mention to respond in channels
|
||||
"free_response_channels": "", # Comma-separated channel IDs where bot responds without mention
|
||||
"allowed_channels": "", # If set, bot ONLY responds in these channel IDs (whitelist)
|
||||
"channel_prompts": {}, # Per-channel ephemeral system prompts
|
||||
},
|
||||
|
||||
# Matrix platform settings (gateway mode)
|
||||
"matrix": {
|
||||
"require_mention": True, # Require @mention to respond in rooms
|
||||
"free_response_rooms": "", # Comma-separated room IDs where bot responds without mention
|
||||
"allowed_rooms": "", # If set, bot ONLY responds in these room IDs (whitelist)
|
||||
},
|
||||
|
||||
# Approval mode for dangerous commands:
|
||||
# manual — always prompt the user (default)
|
||||
# smart — use auxiliary LLM to auto-approve low-risk commands, prompt for high-risk
|
||||
@@ -1191,7 +1224,7 @@ DEFAULT_CONFIG = {
|
||||
# Pre-exec security scanning via tirith
|
||||
"security": {
|
||||
"allow_private_urls": False, # Allow requests to private/internal IPs (for OpenWrt, proxies, VPNs)
|
||||
"redact_secrets": False,
|
||||
"redact_secrets": True,
|
||||
"tirith_enabled": True,
|
||||
"tirith_path": "tirith",
|
||||
"tirith_timeout": 5,
|
||||
@@ -1230,6 +1263,10 @@ DEFAULT_CONFIG = {
|
||||
# Seconds between dispatcher ticks (idle or not). Lower = snappier
|
||||
# pickup of newly-ready tasks; higher = less SQL pressure.
|
||||
"dispatch_interval_seconds": 60,
|
||||
# Auto-block after this many consecutive non-success attempts for the
|
||||
# same task/profile (spawn_failed, timed_out, or crashed). Reassignment
|
||||
# resets the streak for the new profile.
|
||||
"failure_limit": 2,
|
||||
},
|
||||
|
||||
# execute_code settings — controls the tool used for programmatic tool calls.
|
||||
@@ -1840,6 +1877,14 @@ OPTIONAL_ENV_VARS = {
|
||||
"password": False,
|
||||
"category": "tool",
|
||||
},
|
||||
"BRAVE_SEARCH_API_KEY": {
|
||||
"description": "Brave Search API subscription token (free tier: 2,000 queries/mo)",
|
||||
"prompt": "Brave Search subscription token",
|
||||
"url": "https://brave.com/search/api/",
|
||||
"tools": ["web_search"],
|
||||
"password": True,
|
||||
"category": "tool",
|
||||
},
|
||||
"BROWSERBASE_API_KEY": {
|
||||
"description": "Browserbase API key for cloud browser (optional — local browser works without this)",
|
||||
"prompt": "Browserbase API key",
|
||||
@@ -3407,7 +3452,7 @@ def migrate_config(interactive: bool = True, quiet: bool = False) -> Dict[str, A
|
||||
if not manifest_file.exists():
|
||||
continue
|
||||
try:
|
||||
with open(manifest_file) as _mf:
|
||||
with open(manifest_file, encoding="utf-8") as _mf:
|
||||
manifest = yaml.safe_load(_mf) or {}
|
||||
except Exception:
|
||||
manifest = {}
|
||||
@@ -3974,10 +4019,10 @@ def load_config() -> Dict[str, Any]:
|
||||
|
||||
_SECURITY_COMMENT = """
|
||||
# ── Security ──────────────────────────────────────────────────────────
|
||||
# Secret redaction is OFF by default — tool output (terminal stdout,
|
||||
# read_file results, web content) passes through unmodified. Set
|
||||
# redact_secrets to true to mask strings that look like API keys, tokens,
|
||||
# and passwords before they enter the model context and logs.
|
||||
# Secret redaction is ON by default — strings that look like API keys,
|
||||
# tokens, and passwords are masked in tool output, logs, and chat
|
||||
# responses before the model or user ever sees them. Set redact_secrets
|
||||
# to false to disable (e.g. when developing the redactor itself).
|
||||
# tirith pre-exec scanning is enabled by default when the tirith binary
|
||||
# is available. Configure via security.tirith_* keys or env vars
|
||||
# (TIRITH_ENABLED, TIRITH_BIN, TIRITH_TIMEOUT, TIRITH_FAIL_OPEN).
|
||||
@@ -4017,8 +4062,8 @@ _FALLBACK_COMMENT = """
|
||||
|
||||
_COMMENTED_SECTIONS = """
|
||||
# ── Security ──────────────────────────────────────────────────────────
|
||||
# Secret redaction is OFF by default. Set to true to mask strings that
|
||||
# look like API keys, tokens, and passwords in tool output and logs.
|
||||
# Secret redaction is ON by default. Set to false to pass tool output,
|
||||
# logs, and chat responses through unmodified (e.g. for redactor dev).
|
||||
#
|
||||
# security:
|
||||
# redact_secrets: true
|
||||
@@ -4651,11 +4696,19 @@ def edit_config():
|
||||
|
||||
# Find editor
|
||||
editor = os.getenv('EDITOR') or os.getenv('VISUAL')
|
||||
|
||||
|
||||
if not editor:
|
||||
# Try common editors
|
||||
for cmd in ['nano', 'vim', 'vi', 'code', 'notepad']:
|
||||
import shutil
|
||||
# Try common editors — order is platform-aware so Windows users
|
||||
# land on a working editor (notepad) even without Git Bash or nano
|
||||
# installed. On POSIX, prefer nano/vim over code/notepad because
|
||||
# it's more likely to be present on headless / server systems.
|
||||
import shutil
|
||||
import sys as _sys
|
||||
if _sys.platform == "win32":
|
||||
candidates = ['notepad', 'code', 'vim', 'vi', 'nano']
|
||||
else:
|
||||
candidates = ['nano', 'vim', 'vi', 'code', 'notepad']
|
||||
for cmd in candidates:
|
||||
if shutil.which(cmd):
|
||||
editor = cmd
|
||||
break
|
||||
@@ -4937,3 +4990,100 @@ def _inject_profile_env_vars() -> None:
|
||||
|
||||
# Eagerly inject so that OPTIONAL_ENV_VARS is fully populated at import time.
|
||||
_inject_profile_env_vars()
|
||||
|
||||
|
||||
# ── Platform-plugin env var injection ────────────────────────────────────────
|
||||
# Bundled platform plugins under ``plugins/platforms/*/plugin.yaml`` declare
|
||||
# their required env vars via ``requires_env``. This mirror of
|
||||
# ``_inject_profile_env_vars`` surfaces them in ``hermes config`` UI so users
|
||||
# can configure Teams / IRC / Google Chat without the core repo ever needing
|
||||
# to know they exist.
|
||||
#
|
||||
# Each ``requires_env`` entry may be a bare string (name only) or a dict:
|
||||
#
|
||||
# requires_env:
|
||||
# - TEAMS_CLIENT_ID # minimal
|
||||
# - name: TEAMS_CLIENT_SECRET # rich
|
||||
# description: "Teams bot client secret"
|
||||
# url: "https://portal.azure.com/"
|
||||
# password: true
|
||||
# prompt: "Teams client secret"
|
||||
#
|
||||
# An optional ``optional_env`` block surfaces non-required vars the same way
|
||||
# (e.g. allowlist, home channel).
|
||||
|
||||
_platform_plugin_env_vars_injected = False
|
||||
|
||||
|
||||
def _inject_platform_plugin_env_vars() -> None:
|
||||
"""Populate OPTIONAL_ENV_VARS from bundled platform plugin manifests.
|
||||
|
||||
Called once at module load time. Idempotent — repeated calls are no-ops.
|
||||
Failures are swallowed so a malformed plugin.yaml can't break CLI import.
|
||||
"""
|
||||
global _platform_plugin_env_vars_injected
|
||||
if _platform_plugin_env_vars_injected:
|
||||
return
|
||||
_platform_plugin_env_vars_injected = True
|
||||
try:
|
||||
import yaml # type: ignore
|
||||
|
||||
# Resolve the bundled plugins dir from this file's location so the
|
||||
# injector works regardless of CWD.
|
||||
repo_root = Path(__file__).resolve().parents[1]
|
||||
platforms_dir = repo_root / "plugins" / "platforms"
|
||||
if not platforms_dir.is_dir():
|
||||
return
|
||||
for child in platforms_dir.iterdir():
|
||||
if not child.is_dir():
|
||||
continue
|
||||
manifest_path = child / "plugin.yaml"
|
||||
if not manifest_path.exists():
|
||||
manifest_path = child / "plugin.yml"
|
||||
if not manifest_path.exists():
|
||||
continue
|
||||
try:
|
||||
with open(manifest_path, "r", encoding="utf-8") as f:
|
||||
manifest = yaml.safe_load(f) or {}
|
||||
except Exception:
|
||||
continue
|
||||
label = manifest.get("label") or manifest.get("name") or child.name
|
||||
# Merge required + optional env var declarations.
|
||||
entries = list(manifest.get("requires_env") or [])
|
||||
entries.extend(manifest.get("optional_env") or [])
|
||||
for entry in entries:
|
||||
if isinstance(entry, str):
|
||||
name = entry
|
||||
meta: dict = {}
|
||||
elif isinstance(entry, dict) and entry.get("name"):
|
||||
name = entry["name"]
|
||||
meta = entry
|
||||
else:
|
||||
continue
|
||||
if name in OPTIONAL_ENV_VARS:
|
||||
continue # hardcoded entry wins (back-compat)
|
||||
# Heuristic: anything named *TOKEN, *SECRET, *KEY, *PASSWORD
|
||||
# is a password field unless explicitly overridden.
|
||||
name_upper = name.upper()
|
||||
is_secret = bool(meta.get("password") or meta.get("secret"))
|
||||
if not is_secret and not meta.get("password") is False:
|
||||
is_secret = any(
|
||||
name_upper.endswith(suf)
|
||||
for suf in ("_TOKEN", "_SECRET", "_KEY", "_PASSWORD", "_JSON")
|
||||
)
|
||||
OPTIONAL_ENV_VARS[name] = {
|
||||
"description": (
|
||||
meta.get("description")
|
||||
or f"{label} configuration"
|
||||
),
|
||||
"prompt": meta.get("prompt") or name,
|
||||
"url": meta.get("url") or None,
|
||||
"password": is_secret,
|
||||
"category": meta.get("category") or "messaging",
|
||||
}
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# Eagerly inject so that platform plugin env vars show up in the setup wizard.
|
||||
_inject_platform_plugin_env_vars()
|
||||
|
||||
@@ -212,9 +212,9 @@ def copilot_device_code_login(
|
||||
print(" Waiting for authorization...", end="", flush=True)
|
||||
|
||||
# Step 3: Poll for completion
|
||||
deadline = time.time() + timeout_seconds
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
|
||||
while time.time() < deadline:
|
||||
while time.monotonic() < deadline:
|
||||
time.sleep(interval + _DEVICE_CODE_POLL_SAFETY_MARGIN)
|
||||
|
||||
poll_data = urllib.parse.urlencode({
|
||||
|
||||
+37
-8
@@ -12,6 +12,7 @@ from __future__ import annotations
|
||||
import argparse
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
|
||||
@@ -57,7 +58,8 @@ def _cmd_status(args) -> int:
|
||||
print(f" last summary: {summary}")
|
||||
_report = state.get("last_report_path")
|
||||
if _report:
|
||||
print(f" last report: {_report}")
|
||||
suffix = "" if Path(_report).exists() else " (missing)"
|
||||
print(f" last report: {_report}{suffix}")
|
||||
_ih = curator.get_interval_hours()
|
||||
_interval_label = (
|
||||
f"{_ih // 24}d" if _ih % 24 == 0 and _ih >= 24
|
||||
@@ -161,6 +163,8 @@ def _cmd_run(args) -> int:
|
||||
return 1
|
||||
|
||||
dry = bool(getattr(args, "dry_run", False))
|
||||
background = bool(getattr(args, "background", False))
|
||||
synchronous = bool(getattr(args, "synchronous", False)) or not background
|
||||
if dry:
|
||||
print("curator: running DRY-RUN (report only, no mutations)...")
|
||||
else:
|
||||
@@ -171,7 +175,7 @@ def _cmd_run(args) -> int:
|
||||
|
||||
result = curator.run_curator_review(
|
||||
on_summary=_on_summary,
|
||||
synchronous=bool(args.synchronous),
|
||||
synchronous=synchronous,
|
||||
dry_run=dry,
|
||||
)
|
||||
auto = result.get("auto_transitions", {})
|
||||
@@ -188,13 +192,19 @@ def _cmd_run(args) -> int:
|
||||
f"archived={auto.get('archived', 0)} "
|
||||
f"reactivated={auto.get('reactivated', 0)}"
|
||||
)
|
||||
if not args.synchronous:
|
||||
if not synchronous:
|
||||
print("llm pass running in background — check `hermes curator status` later")
|
||||
if dry:
|
||||
print(
|
||||
"dry-run: no changes applied. When the report lands, read it with "
|
||||
"`hermes curator status` and run `hermes curator run` (no flag) to apply."
|
||||
)
|
||||
if synchronous:
|
||||
print(
|
||||
"dry-run: no changes applied. Read the report with "
|
||||
"`hermes curator status` and run `hermes curator run` (no flag) to apply."
|
||||
)
|
||||
else:
|
||||
print(
|
||||
"dry-run: no changes applied. When the report lands, read it with "
|
||||
"`hermes curator status` and run `hermes curator run` (no flag) to apply."
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
@@ -442,6 +452,18 @@ def _cmd_rollback(args) -> int:
|
||||
return 1
|
||||
|
||||
|
||||
def _cmd_list_archived(args) -> int:
|
||||
"""List archived (recoverable) skills."""
|
||||
from tools import skill_usage
|
||||
names = skill_usage.list_archived_skill_names()
|
||||
if not names:
|
||||
print("curator: no archived skills")
|
||||
return 0
|
||||
for name in names:
|
||||
print(name)
|
||||
return 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# argparse wiring (called from hermes_cli.main)
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -461,7 +483,11 @@ def register_cli(parent: argparse.ArgumentParser) -> None:
|
||||
p_run = subs.add_parser("run", help="Trigger a curator review now")
|
||||
p_run.add_argument(
|
||||
"--sync", "--synchronous", dest="synchronous", action="store_true",
|
||||
help="Wait for the LLM review pass to finish (default: background thread)",
|
||||
help="Wait for the LLM review pass to finish (default for manual runs)",
|
||||
)
|
||||
p_run.add_argument(
|
||||
"--background", dest="background", action="store_true",
|
||||
help="Start the LLM review pass in a background thread and return immediately",
|
||||
)
|
||||
p_run.add_argument(
|
||||
"--dry-run", dest="dry_run", action="store_true",
|
||||
@@ -488,6 +514,9 @@ def register_cli(parent: argparse.ArgumentParser) -> None:
|
||||
p_restore.add_argument("skill", help="Skill name")
|
||||
p_restore.set_defaults(func=_cmd_restore)
|
||||
|
||||
subs.add_parser("list-archived", help="List archived skills") \
|
||||
.set_defaults(func=_cmd_list_archived)
|
||||
|
||||
p_archive = subs.add_parser(
|
||||
"archive",
|
||||
help="Manually archive a skill (move to .archive/, excluded from prompt)",
|
||||
|
||||
+31
-4
@@ -91,6 +91,15 @@ def _termux_browser_setup_steps(node_installed: bool) -> list[str]:
|
||||
return steps
|
||||
|
||||
|
||||
def _termux_install_all_fallback_notes() -> list[str]:
|
||||
return [
|
||||
"Termux install profile: use .[termux-all] for broad compatibility (installer default on Termux).",
|
||||
"Matrix E2EE extra is excluded on Termux (python-olm currently fails to build).",
|
||||
"Local faster-whisper extra is excluded on Termux (ctranslate2/av build path unavailable).",
|
||||
"STT fallback: use Groq Whisper (set GROQ_API_KEY) or OpenAI Whisper (set VOICE_TOOLS_OPENAI_KEY).",
|
||||
]
|
||||
|
||||
|
||||
def _has_provider_env_config(content: str) -> bool:
|
||||
"""Return True when ~/.hermes/.env contains provider auth/base URL settings."""
|
||||
return any(key in content for key in _PROVIDER_ENV_HINTS)
|
||||
@@ -589,7 +598,7 @@ def run_doctor(args):
|
||||
# Detect stale root-level model keys (known bug source — PR #4329)
|
||||
try:
|
||||
import yaml
|
||||
with open(config_path) as f:
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
raw_config = yaml.safe_load(f) or {}
|
||||
stale_root_keys = [k for k in ("provider", "base_url") if k in raw_config and isinstance(raw_config[k], str)]
|
||||
if stale_root_keys:
|
||||
@@ -1050,7 +1059,8 @@ def run_doctor(args):
|
||||
check_warn("Node.js not found", "(optional, needed for browser tools)")
|
||||
|
||||
# npm audit for all Node.js packages
|
||||
if _safe_which("npm"):
|
||||
_npm_bin = _safe_which("npm")
|
||||
if _npm_bin:
|
||||
npm_dirs = [
|
||||
(PROJECT_ROOT, "Browser tools (agent-browser)"),
|
||||
(PROJECT_ROOT / "scripts" / "whatsapp-bridge", "WhatsApp bridge"),
|
||||
@@ -1059,8 +1069,10 @@ def run_doctor(args):
|
||||
if not (npm_dir / "node_modules").exists():
|
||||
continue
|
||||
try:
|
||||
# Use resolved absolute path so Windows can execute
|
||||
# npm.cmd (CreateProcessW can't run bare .cmd names).
|
||||
audit_result = subprocess.run(
|
||||
["npm", "audit", "--json"],
|
||||
[_npm_bin, "audit", "--json"],
|
||||
cwd=str(npm_dir),
|
||||
capture_output=True, text=True, timeout=30,
|
||||
)
|
||||
@@ -1084,6 +1096,11 @@ def run_doctor(args):
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if _is_termux():
|
||||
check_info("Termux compatibility fallbacks:")
|
||||
for note in _termux_install_all_fallback_notes():
|
||||
check_info(note)
|
||||
|
||||
# =========================================================================
|
||||
# Check: API connectivity
|
||||
# =========================================================================
|
||||
@@ -1225,6 +1242,16 @@ def run_doctor(args):
|
||||
headers=_headers,
|
||||
timeout=10,
|
||||
)
|
||||
if (
|
||||
_pname == "Alibaba/DashScope"
|
||||
and not _base
|
||||
and _resp.status_code == 401
|
||||
):
|
||||
_resp = httpx.get(
|
||||
"https://dashscope.aliyuncs.com/compatible-mode/v1/models",
|
||||
headers=_headers,
|
||||
timeout=10,
|
||||
)
|
||||
if _resp.status_code == 200:
|
||||
print(f"\r {color('✓', Colors.GREEN)} {_label} ")
|
||||
elif _resp.status_code == 401:
|
||||
@@ -1372,7 +1399,7 @@ def run_doctor(args):
|
||||
import yaml as _yaml
|
||||
_mem_cfg_path = HERMES_HOME / "config.yaml"
|
||||
if _mem_cfg_path.exists():
|
||||
with open(_mem_cfg_path) as _f:
|
||||
with open(_mem_cfg_path, encoding="utf-8") as _f:
|
||||
_raw_cfg = _yaml.safe_load(_f) or {}
|
||||
_active_memory_provider = (_raw_cfg.get("memory") or {}).get("provider", "")
|
||||
except Exception:
|
||||
|
||||
+369
-47
@@ -232,6 +232,10 @@ def _graceful_restart_via_sigusr1(pid: int, drain_timeout: float) -> bool:
|
||||
# Process still exists but we can't signal it. Treat as alive
|
||||
# so the caller falls back.
|
||||
pass
|
||||
except OSError:
|
||||
# Windows raises OSError (WinError 87 "invalid parameter") for
|
||||
# a gone PID — treat the same as ProcessLookupError.
|
||||
return True
|
||||
_time.sleep(0.5)
|
||||
# Drain didn't finish in time.
|
||||
return False
|
||||
@@ -441,6 +445,25 @@ def launch_detached_profile_gateway_restart(profile: str, old_pid: int) -> bool:
|
||||
if old_pid <= 0:
|
||||
return False
|
||||
|
||||
# The watcher is a tiny Python subprocess that polls the old PID and
|
||||
# respawns the gateway once it's gone. Both legs of the chain need
|
||||
# platform-appropriate detach semantics:
|
||||
#
|
||||
# POSIX — ``start_new_session=True`` (os.setsid in the child) detaches
|
||||
# from the parent's process group so Ctrl+C in the CLI doesn't
|
||||
# propagate and the watcher/gateway survive the CLI exiting.
|
||||
#
|
||||
# Windows — ``start_new_session`` is silently accepted but does NOT
|
||||
# detach. The watcher stays attached to the CLI's console and dies
|
||||
# when the user closes the terminal, leaving ``hermes update`` users
|
||||
# with no running gateway until they re-invoke ``hermes gateway``
|
||||
# manually. The Win32 equivalent is the ``CREATE_NEW_PROCESS_GROUP |
|
||||
# DETACHED_PROCESS | CREATE_NO_WINDOW`` creationflags bundle.
|
||||
#
|
||||
# ``windows_detach_popen_kwargs()`` returns the right kwargs for the
|
||||
# host platform and is a no-op on POSIX (just ``start_new_session=True``).
|
||||
from hermes_cli._subprocess_compat import windows_detach_popen_kwargs
|
||||
|
||||
watcher = textwrap.dedent(
|
||||
"""
|
||||
import os
|
||||
@@ -458,22 +481,39 @@ def launch_detached_profile_gateway_restart(profile: str, old_pid: int) -> bool:
|
||||
break
|
||||
except PermissionError:
|
||||
pass
|
||||
except OSError:
|
||||
# Windows: gone PID raises OSError (WinError 87).
|
||||
break
|
||||
time.sleep(0.2)
|
||||
subprocess.Popen(
|
||||
cmd,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
start_new_session=True,
|
||||
)
|
||||
|
||||
# Platform-appropriate detach for the respawned gateway. On POSIX
|
||||
# start_new_session=True maps to os.setsid; on Windows we need
|
||||
# explicit creationflags because start_new_session is a no-op there.
|
||||
_popen_kwargs = {
|
||||
"stdout": subprocess.DEVNULL,
|
||||
"stderr": subprocess.DEVNULL,
|
||||
}
|
||||
if sys.platform == "win32":
|
||||
_CREATE_NEW_PROCESS_GROUP = 0x00000200
|
||||
_DETACHED_PROCESS = 0x00000008
|
||||
_CREATE_NO_WINDOW = 0x08000000
|
||||
_popen_kwargs["creationflags"] = (
|
||||
_CREATE_NEW_PROCESS_GROUP | _DETACHED_PROCESS | _CREATE_NO_WINDOW
|
||||
)
|
||||
else:
|
||||
_popen_kwargs["start_new_session"] = True
|
||||
subprocess.Popen(cmd, **_popen_kwargs)
|
||||
"""
|
||||
).strip()
|
||||
|
||||
try:
|
||||
# Same platform-aware detach for the watcher process itself — so
|
||||
# closing the user's terminal doesn't kill the watcher.
|
||||
subprocess.Popen(
|
||||
[sys.executable, "-c", watcher, str(old_pid), *_gateway_run_args_for_profile(profile)],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
start_new_session=True,
|
||||
**windows_detach_popen_kwargs(),
|
||||
)
|
||||
except OSError:
|
||||
return False
|
||||
@@ -505,6 +545,7 @@ def _read_systemd_unit_properties(
|
||||
"SubState",
|
||||
"Result",
|
||||
"ExecMainStatus",
|
||||
"MainPID",
|
||||
),
|
||||
) -> dict[str, str]:
|
||||
"""Return selected ``systemctl show`` properties for the gateway unit."""
|
||||
@@ -538,6 +579,41 @@ def _read_systemd_unit_properties(
|
||||
return parsed
|
||||
|
||||
|
||||
def _systemd_main_pid_from_props(props: dict[str, str]) -> int | None:
|
||||
try:
|
||||
pid = int(props.get("MainPID", "0") or "0")
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return pid if pid > 0 else None
|
||||
|
||||
|
||||
def _systemd_main_pid(system: bool = False) -> int | None:
|
||||
return _systemd_main_pid_from_props(_read_systemd_unit_properties(system=system))
|
||||
|
||||
|
||||
def _read_gateway_runtime_status() -> dict | None:
|
||||
try:
|
||||
from gateway.status import read_runtime_status
|
||||
|
||||
state = read_runtime_status()
|
||||
except Exception:
|
||||
return None
|
||||
return state if isinstance(state, dict) else None
|
||||
|
||||
|
||||
def _gateway_runtime_status_for_pid(pid: int | None) -> dict | None:
|
||||
if not pid:
|
||||
return None
|
||||
state = _read_gateway_runtime_status()
|
||||
if not state:
|
||||
return None
|
||||
try:
|
||||
state_pid = int(state.get("pid", 0) or 0)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return state if state_pid == pid else None
|
||||
|
||||
|
||||
def _wait_for_systemd_service_restart(
|
||||
*,
|
||||
system: bool = False,
|
||||
@@ -549,9 +625,10 @@ def _wait_for_systemd_service_restart(
|
||||
|
||||
svc = get_service_name()
|
||||
scope_label = _service_scope_label(system).capitalize()
|
||||
deadline = time.time() + timeout
|
||||
deadline = time.monotonic() + timeout
|
||||
printed_runtime_wait = False
|
||||
|
||||
while time.time() < deadline:
|
||||
while time.monotonic() < deadline:
|
||||
props = _read_systemd_unit_properties(system=system)
|
||||
active_state = props.get("ActiveState", "")
|
||||
sub_state = props.get("SubState", "")
|
||||
@@ -562,19 +639,32 @@ def _wait_for_systemd_service_restart(
|
||||
new_pid = get_running_pid()
|
||||
except Exception:
|
||||
new_pid = None
|
||||
if not new_pid:
|
||||
new_pid = _systemd_main_pid_from_props(props)
|
||||
|
||||
if active_state == "active":
|
||||
if new_pid and (previous_pid is None or new_pid != previous_pid):
|
||||
print(f"✓ {scope_label} service restarted (PID {new_pid})")
|
||||
return True
|
||||
if previous_pid is None:
|
||||
print(f"✓ {scope_label} service restarted")
|
||||
return True
|
||||
runtime_state = _gateway_runtime_status_for_pid(new_pid)
|
||||
gateway_state = (runtime_state or {}).get("gateway_state")
|
||||
if gateway_state == "running":
|
||||
print(f"✓ {scope_label} service restarted (PID {new_pid})")
|
||||
return True
|
||||
if gateway_state == "startup_failed":
|
||||
reason = (runtime_state or {}).get("exit_reason") or "startup failed"
|
||||
print(f"⚠ {scope_label} service process restarted (PID {new_pid}), but gateway startup failed: {reason}")
|
||||
return False
|
||||
if not printed_runtime_wait:
|
||||
print(f"⏳ {scope_label} service process started (PID {new_pid}); waiting for gateway runtime...")
|
||||
printed_runtime_wait = True
|
||||
|
||||
if active_state == "activating" and sub_state == "auto-restart":
|
||||
time.sleep(1)
|
||||
continue
|
||||
|
||||
if _systemd_unit_is_start_limited(props):
|
||||
_print_systemd_start_limit_wait(system=system)
|
||||
return False
|
||||
|
||||
time.sleep(2)
|
||||
|
||||
print(
|
||||
@@ -585,6 +675,46 @@ def _wait_for_systemd_service_restart(
|
||||
return False
|
||||
|
||||
|
||||
def _systemd_unit_is_start_limited(props: dict[str, str]) -> bool:
|
||||
result = props.get("Result", "").lower()
|
||||
sub_state = props.get("SubState", "").lower()
|
||||
return result == "start-limit-hit" or sub_state == "start-limit-hit"
|
||||
|
||||
|
||||
def _systemd_error_indicates_start_limit(exc: subprocess.CalledProcessError) -> bool:
|
||||
parts: list[str] = []
|
||||
for attr in ("stderr", "stdout", "output"):
|
||||
value = getattr(exc, attr, None)
|
||||
if not value:
|
||||
continue
|
||||
if isinstance(value, bytes):
|
||||
value = value.decode(errors="replace")
|
||||
parts.append(str(value))
|
||||
text = "\n".join(parts).lower()
|
||||
return (
|
||||
"start-limit-hit" in text
|
||||
or "start request repeated too quickly" in text
|
||||
or "start-limit" in text
|
||||
)
|
||||
|
||||
|
||||
def _systemd_service_is_start_limited(system: bool = False) -> bool:
|
||||
return _systemd_unit_is_start_limited(_read_systemd_unit_properties(system=system))
|
||||
|
||||
|
||||
def _print_systemd_start_limit_wait(system: bool = False) -> None:
|
||||
svc = get_service_name()
|
||||
scope_label = _service_scope_label(system).capitalize()
|
||||
scope_flag = " --system" if system else ""
|
||||
systemctl_prefix = "systemctl " if system else "systemctl --user "
|
||||
journal_prefix = "journalctl " if system else "journalctl --user "
|
||||
print(f"⏳ {scope_label} service is temporarily rate-limited by systemd.")
|
||||
print(" systemd is refusing another immediate start after repeated exits.")
|
||||
print(f" Wait for the start-limit window to expire, then run: {'sudo ' if system else ''}hermes gateway restart{scope_flag}")
|
||||
print(f" Or clear the failed state manually: {systemctl_prefix}reset-failed {svc}")
|
||||
print(f" Check logs: {journal_prefix}-u {svc} -l --since '5 min ago'")
|
||||
|
||||
|
||||
def _recover_pending_systemd_restart(system: bool = False, previous_pid: int | None = None) -> bool:
|
||||
"""Recover a planned service restart that is stuck in systemd state."""
|
||||
props = _read_systemd_unit_properties(system=system)
|
||||
@@ -740,6 +870,46 @@ def _print_other_profiles_gateway_status() -> None:
|
||||
pass
|
||||
|
||||
|
||||
def _gateway_list() -> None:
|
||||
"""List all profiles and their gateway running status.
|
||||
|
||||
Provides a single-command overview of every known profile and whether
|
||||
its gateway is currently running, so multi-profile users don't have to
|
||||
check each profile individually.
|
||||
"""
|
||||
try:
|
||||
from hermes_cli.profiles import list_profiles, get_active_profile_name
|
||||
except Exception:
|
||||
print("Unable to list profiles.")
|
||||
return
|
||||
|
||||
profiles = list_profiles()
|
||||
if not profiles:
|
||||
print("No profiles found.")
|
||||
return
|
||||
|
||||
current = get_active_profile_name()
|
||||
|
||||
print("Gateways:")
|
||||
for prof in profiles:
|
||||
marker = "✓" if prof.gateway_running else "✗"
|
||||
label = prof.name
|
||||
if prof.name == current:
|
||||
label += " (current)"
|
||||
parts = [f" {marker} {label:<24s}"]
|
||||
if prof.gateway_running:
|
||||
try:
|
||||
from gateway.status import get_running_pid
|
||||
pid = get_running_pid(prof.path / "gateway.pid", cleanup_stale=False)
|
||||
if pid:
|
||||
parts.append(f"PID {pid}")
|
||||
except Exception:
|
||||
pass
|
||||
else:
|
||||
parts.append("not running")
|
||||
print(" — ".join(parts))
|
||||
|
||||
|
||||
def kill_gateway_processes(force: bool = False, exclude_pids: set | None = None,
|
||||
all_profiles: bool = False) -> int:
|
||||
"""Kill any running gateway processes. Returns count killed.
|
||||
@@ -805,7 +975,8 @@ def stop_profile_gateway() -> bool:
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
_time.sleep(0.5)
|
||||
except (ProcessLookupError, PermissionError):
|
||||
except (ProcessLookupError, PermissionError, OSError):
|
||||
# OSError covers Windows' WinError 87 for gone PIDs.
|
||||
break
|
||||
|
||||
if get_running_pid() is None:
|
||||
@@ -967,6 +1138,27 @@ class UserSystemdUnavailableError(RuntimeError):
|
||||
"""
|
||||
|
||||
|
||||
class SystemScopeRequiresRootError(RuntimeError):
|
||||
"""Raised when a system-scope gateway operation is attempted as non-root.
|
||||
|
||||
System-scope units live in ``/etc/systemd/system/`` and require root for
|
||||
install / uninstall / start / stop / restart via ``systemctl``. The
|
||||
previous behavior was ``sys.exit(1)`` which blew past the wizard's
|
||||
``except Exception`` guards and dumped the user at a bare shell prompt
|
||||
with no guidance. Raising a typed exception lets callers that can
|
||||
recover (the setup wizard) print actionable remediation instead, while
|
||||
``gateway_command`` still exits 1 with the same message for the direct
|
||||
CLI path.
|
||||
|
||||
``args[0]`` carries the user-facing message, ``args[1]`` the action name.
|
||||
``str(e)`` returns only the message (not the tuple repr) so format
|
||||
strings like ``f"Failed: {e}"`` render cleanly.
|
||||
"""
|
||||
|
||||
def __str__(self) -> str:
|
||||
return self.args[0] if self.args else ""
|
||||
|
||||
|
||||
def _user_dbus_socket_path() -> Path:
|
||||
"""Return the expected per-user D-Bus socket path (regardless of existence)."""
|
||||
xdg = os.environ.get("XDG_RUNTIME_DIR") or f"/run/user/{os.getuid()}"
|
||||
@@ -1382,8 +1574,10 @@ def print_systemd_scope_conflict_warning() -> None:
|
||||
|
||||
def _require_root_for_system_service(action: str) -> None:
|
||||
if os.geteuid() != 0:
|
||||
print(f"System gateway {action} requires root. Re-run with sudo.")
|
||||
sys.exit(1)
|
||||
raise SystemScopeRequiresRootError(
|
||||
f"System gateway {action} requires root. Re-run with sudo.",
|
||||
action,
|
||||
)
|
||||
|
||||
|
||||
def _system_service_identity(run_as_user: str | None = None) -> tuple[str, str, str]:
|
||||
@@ -1930,6 +2124,47 @@ def _select_systemd_scope(system: bool = False) -> bool:
|
||||
return get_systemd_unit_path(system=True).exists() and not get_systemd_unit_path(system=False).exists()
|
||||
|
||||
|
||||
def _system_scope_wizard_would_need_root(system: bool = False) -> bool:
|
||||
"""True when the setup wizard is about to trigger a system-scope operation
|
||||
as a non-root user.
|
||||
|
||||
Replicates the decision ``_select_systemd_scope`` makes inside
|
||||
``systemd_start`` / ``systemd_restart`` / ``systemd_stop`` so the wizard
|
||||
can detect the dead-end BEFORE prompting, rather than letting
|
||||
``SystemScopeRequiresRootError`` propagate out and leave the user
|
||||
staring at a bare shell.
|
||||
"""
|
||||
if os.geteuid() == 0:
|
||||
return False
|
||||
return _select_systemd_scope(system=system)
|
||||
|
||||
|
||||
def _print_system_scope_remediation(action: str) -> None:
|
||||
"""Print actionable remediation when the wizard skips a system-scope
|
||||
prompt because the user isn't root. Keeps the wizard flowing instead of
|
||||
aborting.
|
||||
"""
|
||||
svc = get_service_name()
|
||||
print_warning(
|
||||
f"Gateway is installed as a system-wide service — "
|
||||
f"{action} requires root."
|
||||
)
|
||||
print_info(" Options:")
|
||||
print_info(f" 1. {action.capitalize()} it this time:")
|
||||
if action == "start":
|
||||
print_info(f" sudo systemctl start {svc}")
|
||||
elif action == "stop":
|
||||
print_info(f" sudo systemctl stop {svc}")
|
||||
elif action == "restart":
|
||||
print_info(f" sudo systemctl restart {svc}")
|
||||
else:
|
||||
print_info(f" sudo systemctl {action} {svc}")
|
||||
print_info(" 2. Switch to a per-user service (recommended for personal use):")
|
||||
print_info(" sudo hermes gateway uninstall --system")
|
||||
print_info(" hermes gateway install")
|
||||
print_info(" hermes gateway start")
|
||||
|
||||
|
||||
def _get_restart_drain_timeout() -> float:
|
||||
"""Return the configured gateway restart drain timeout in seconds."""
|
||||
raw = os.getenv("HERMES_RESTART_DRAIN_TIMEOUT", "").strip()
|
||||
@@ -2071,41 +2306,52 @@ def systemd_restart(system: bool = False):
|
||||
refresh_systemd_unit_if_needed(system=system)
|
||||
from gateway.status import get_running_pid
|
||||
|
||||
pid = get_running_pid()
|
||||
if pid is not None and _request_gateway_self_restart(pid):
|
||||
import time
|
||||
pid = get_running_pid() or _systemd_main_pid(system=system)
|
||||
if pid is not None:
|
||||
scope_label = _service_scope_label(system).capitalize()
|
||||
svc = get_service_name()
|
||||
drain_timeout = _get_restart_drain_timeout()
|
||||
|
||||
# Phase 1: wait for old process to exit (drain + shutdown)
|
||||
print(f"⏳ {scope_label} service draining active work...")
|
||||
deadline = time.time() + 90
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
time.sleep(1)
|
||||
except (ProcessLookupError, PermissionError):
|
||||
break # old process is gone
|
||||
else:
|
||||
print(f"⚠ Old process (PID {pid}) still alive after 90s")
|
||||
print(f"⏳ {scope_label} service restarting gracefully (PID {pid})...")
|
||||
if _graceful_restart_via_sigusr1(pid, drain_timeout + 5):
|
||||
# The gateway exits with code 75 for a planned service restart.
|
||||
# RestartSec can otherwise delay the relaunch even though the
|
||||
# operator asked for an immediate restart, so kick the unit once
|
||||
# the old PID has exited and then wait for the replacement PID.
|
||||
_run_systemctl(
|
||||
["reset-failed", svc],
|
||||
system=system,
|
||||
check=False,
|
||||
timeout=30,
|
||||
)
|
||||
_run_systemctl(
|
||||
["restart", svc],
|
||||
system=system,
|
||||
check=False,
|
||||
timeout=90,
|
||||
)
|
||||
if _wait_for_systemd_service_restart(system=system, previous_pid=pid):
|
||||
return
|
||||
if _systemd_service_is_start_limited(system=system):
|
||||
return
|
||||
|
||||
# The gateway exits with code 75 for a planned service restart.
|
||||
# systemd can sit in the RestartSec window or even wedge itself into a
|
||||
# failed/rate-limited state if the operator asks for another restart in
|
||||
# the middle of that handoff. Clear any stale failed state and kick the
|
||||
# unit immediately so `hermes gateway restart` behaves idempotently.
|
||||
print(
|
||||
f"⚠ Graceful restart did not complete within {int(drain_timeout + 5)}s; "
|
||||
"forcing a service restart..."
|
||||
)
|
||||
_run_systemctl(
|
||||
["reset-failed", svc],
|
||||
system=system,
|
||||
check=False,
|
||||
timeout=30,
|
||||
)
|
||||
_run_systemctl(
|
||||
["start", svc],
|
||||
system=system,
|
||||
check=False,
|
||||
timeout=90,
|
||||
)
|
||||
try:
|
||||
_run_systemctl(["restart", svc], system=system, check=True, timeout=90)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
if _systemd_error_indicates_start_limit(exc) or _systemd_service_is_start_limited(system=system):
|
||||
_print_systemd_start_limit_wait(system=system)
|
||||
return
|
||||
raise
|
||||
_wait_for_systemd_service_restart(system=system, previous_pid=pid)
|
||||
return
|
||||
|
||||
@@ -2118,8 +2364,14 @@ def systemd_restart(system: bool = False):
|
||||
check=False,
|
||||
timeout=30,
|
||||
)
|
||||
_run_systemctl(["reload-or-restart", get_service_name()], system=system, check=True, timeout=90)
|
||||
print(f"✓ {_service_scope_label(system).capitalize()} service restarted")
|
||||
try:
|
||||
_run_systemctl(["restart", get_service_name()], system=system, check=True, timeout=90)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
if _systemd_error_indicates_start_limit(exc) or _systemd_service_is_start_limited(system=system):
|
||||
_print_systemd_start_limit_wait(system=system)
|
||||
return
|
||||
raise
|
||||
_wait_for_systemd_service_restart(system=system, previous_pid=pid)
|
||||
|
||||
|
||||
|
||||
@@ -2191,6 +2443,10 @@ def systemd_status(deep: bool = False, system: bool = False, full: bool = False)
|
||||
result_code = unit_props.get("Result", "")
|
||||
if active_state == "activating" and sub_state == "auto-restart":
|
||||
print(" ⏳ Restart pending: systemd is waiting to relaunch the gateway")
|
||||
elif _systemd_unit_is_start_limited(unit_props):
|
||||
print(" ⏳ Restart pending: systemd is temporarily rate-limiting starts")
|
||||
print(f" Run after the start-limit window expires: {'sudo ' if system else ''}hermes gateway restart{scope_flag}")
|
||||
print(f" Or clear it manually: systemctl {'--user ' if not system else ''}reset-failed {get_service_name()}")
|
||||
elif active_state == "failed" and exec_main_status == str(GATEWAY_SERVICE_RESTART_EXIT_CODE):
|
||||
print(" ⚠ Planned restart is stuck in systemd failed state (exit 75)")
|
||||
print(f" Run: systemctl {'--user ' if not system else ''}reset-failed {get_service_name()} && {'sudo ' if system else ''}hermes gateway start{scope_flag}")
|
||||
@@ -2555,6 +2811,42 @@ def launchd_status(deep: bool = False):
|
||||
# Gateway Runner
|
||||
# =============================================================================
|
||||
|
||||
def _truthy_env(value: str | None) -> bool:
|
||||
return str(value or "").strip().lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def _is_official_docker_checkout() -> bool:
|
||||
return (
|
||||
str(PROJECT_ROOT) == "/opt/hermes"
|
||||
and (PROJECT_ROOT / "docker" / "entrypoint.sh").is_file()
|
||||
)
|
||||
|
||||
|
||||
def _guard_official_docker_root_gateway() -> None:
|
||||
"""Refuse gateway startup when the official Docker privilege drop was bypassed."""
|
||||
if not hasattr(os, "geteuid") or os.geteuid() != 0:
|
||||
return
|
||||
if _truthy_env(os.getenv("HERMES_ALLOW_ROOT_GATEWAY")):
|
||||
return
|
||||
if not _is_official_docker_checkout():
|
||||
return
|
||||
|
||||
print_error(
|
||||
"Refusing to run the Hermes gateway as root inside the official Docker image."
|
||||
)
|
||||
print(
|
||||
" The image entrypoint normally drops privileges to the 'hermes' user. "
|
||||
"If you override entrypoint in Docker Compose, include "
|
||||
"/opt/hermes/docker/entrypoint.sh before the Hermes command."
|
||||
)
|
||||
print(
|
||||
" Running the gateway as root can leave root-owned files in "
|
||||
"$HERMES_HOME and break later non-root dashboard/gateway runs."
|
||||
)
|
||||
print(" Set HERMES_ALLOW_ROOT_GATEWAY=1 only if you intentionally accept this risk.")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def run_gateway(verbose: int = 0, quiet: bool = False, replace: bool = False):
|
||||
"""Run the gateway in foreground.
|
||||
|
||||
@@ -2565,6 +2857,7 @@ def run_gateway(verbose: int = 0, quiet: bool = False, replace: bool = False):
|
||||
This prevents systemd restart loops when the old process
|
||||
hasn't fully exited yet.
|
||||
"""
|
||||
_guard_official_docker_root_gateway()
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
# Refresh the systemd unit definition on every boot so that restart
|
||||
@@ -4115,7 +4408,9 @@ def gateway_setup():
|
||||
print_success("Gateway service is installed and running.")
|
||||
elif service_installed:
|
||||
print_warning("Gateway service is installed but not running.")
|
||||
if prompt_yes_no(" Start it now?", True):
|
||||
if supports_systemd_services() and _system_scope_wizard_would_need_root():
|
||||
_print_system_scope_remediation("start")
|
||||
elif prompt_yes_no(" Start it now?", True):
|
||||
try:
|
||||
if supports_systemd_services():
|
||||
systemd_start()
|
||||
@@ -4125,6 +4420,12 @@ def gateway_setup():
|
||||
print_error(" Failed to start — user systemd not reachable:")
|
||||
for line in str(e).splitlines():
|
||||
print(f" {line}")
|
||||
except SystemScopeRequiresRootError as e:
|
||||
# Defense in depth: the pre-check above should have caught
|
||||
# this, but handle the race/edge case gracefully instead of
|
||||
# letting the exception escape the wizard.
|
||||
print_error(f" Failed to start: {e}")
|
||||
_print_system_scope_remediation("start")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print_error(f" Failed to start: {e}")
|
||||
else:
|
||||
@@ -4174,7 +4475,9 @@ def gateway_setup():
|
||||
service_running = _is_service_running()
|
||||
|
||||
if service_running:
|
||||
if prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
||||
if supports_systemd_services() and _system_scope_wizard_would_need_root():
|
||||
_print_system_scope_remediation("restart")
|
||||
elif prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
||||
try:
|
||||
if supports_systemd_services():
|
||||
systemd_restart()
|
||||
@@ -4187,10 +4490,15 @@ def gateway_setup():
|
||||
print_error(" Restart failed — user systemd not reachable:")
|
||||
for line in str(e).splitlines():
|
||||
print(f" {line}")
|
||||
except SystemScopeRequiresRootError as e:
|
||||
print_error(f" Restart failed: {e}")
|
||||
_print_system_scope_remediation("restart")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print_error(f" Restart failed: {e}")
|
||||
elif service_installed:
|
||||
if prompt_yes_no(" Start the gateway service?", True):
|
||||
if supports_systemd_services() and _system_scope_wizard_would_need_root():
|
||||
_print_system_scope_remediation("start")
|
||||
elif prompt_yes_no(" Start the gateway service?", True):
|
||||
try:
|
||||
if supports_systemd_services():
|
||||
systemd_start()
|
||||
@@ -4200,6 +4508,9 @@ def gateway_setup():
|
||||
print_error(" Start failed — user systemd not reachable:")
|
||||
for line in str(e).splitlines():
|
||||
print(f" {line}")
|
||||
except SystemScopeRequiresRootError as e:
|
||||
print_error(f" Start failed: {e}")
|
||||
_print_system_scope_remediation("start")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print_error(f" Start failed: {e}")
|
||||
else:
|
||||
@@ -4273,6 +4584,14 @@ def gateway_command(args):
|
||||
for line in str(e).splitlines():
|
||||
print(f" {line}")
|
||||
sys.exit(1)
|
||||
except SystemScopeRequiresRootError as e:
|
||||
# The direct ``hermes gateway install|uninstall|start|stop|restart``
|
||||
# path lands here when the user typed a system-scope action without
|
||||
# sudo. Same exit code as before — just gives the wizard a way to
|
||||
# intercept the same condition with friendlier guidance before the
|
||||
# error is raised.
|
||||
print(str(e))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def _gateway_command_inner(args):
|
||||
@@ -4597,6 +4916,9 @@ def _gateway_command_inner(args):
|
||||
# Show other profiles' gateway status for multi-profile awareness
|
||||
_print_other_profiles_gateway_status()
|
||||
|
||||
elif subcmd == "list":
|
||||
_gateway_list()
|
||||
|
||||
elif subcmd == "migrate-legacy":
|
||||
# Stop, disable, and remove legacy Hermes gateway unit files from
|
||||
# pre-rename installs (e.g. hermes.service). Profile units and
|
||||
|
||||
+1
-1
@@ -205,7 +205,7 @@ def _cmd_test(args) -> None:
|
||||
|
||||
if getattr(args, "payload_file", None):
|
||||
try:
|
||||
custom = json.loads(Path(args.payload_file).read_text())
|
||||
custom = json.loads(Path(args.payload_file).read_text(encoding="utf-8"))
|
||||
if isinstance(custom, dict):
|
||||
payload.update(custom)
|
||||
else:
|
||||
|
||||
+150
-2
@@ -70,6 +70,7 @@ def _task_to_dict(t: kb.Task) -> dict[str, Any]:
|
||||
"completed_at": t.completed_at,
|
||||
"result": t.result,
|
||||
"skills": list(t.skills) if t.skills else [],
|
||||
"max_retries": t.max_retries,
|
||||
}
|
||||
|
||||
|
||||
@@ -284,6 +285,15 @@ def build_parser(parent_subparsers: argparse._SubParsersAction) -> argparse.Argu
|
||||
"(repeatable). Appended to the built-in "
|
||||
"kanban-worker skill. Example: "
|
||||
"--skill translation --skill github-code-review")
|
||||
p_create.add_argument("--max-retries", type=int, default=None,
|
||||
metavar="N",
|
||||
help="Per-task override for the consecutive-failure "
|
||||
"circuit breaker. Trip on the Nth failure — "
|
||||
"e.g. --max-retries 1 blocks on the first "
|
||||
"failure (no retries), --max-retries 3 allows "
|
||||
"two retries. Omit to use the dispatcher's "
|
||||
"kanban.failure_limit config "
|
||||
f"(default {kb.DEFAULT_FAILURE_LIMIT}).")
|
||||
p_create.add_argument("--json", action="store_true", help="Emit JSON output")
|
||||
|
||||
# --- list ---
|
||||
@@ -443,8 +453,8 @@ def build_parser(parent_subparsers: argparse._SubParsersAction) -> argparse.Argu
|
||||
help="Cap number of spawns this pass")
|
||||
p_disp.add_argument("--failure-limit", type=int,
|
||||
default=kb.DEFAULT_SPAWN_FAILURE_LIMIT,
|
||||
help=f"Auto-block a task after this many consecutive spawn failures "
|
||||
f"(default: {kb.DEFAULT_SPAWN_FAILURE_LIMIT})")
|
||||
help=f"Auto-block a task after this many consecutive non-success attempts "
|
||||
f"(spawn_failed, timed_out, or crashed; default: {kb.DEFAULT_SPAWN_FAILURE_LIMIT})")
|
||||
p_disp.add_argument("--json", action="store_true")
|
||||
|
||||
# --- daemon (deprecated) ---
|
||||
@@ -560,6 +570,42 @@ def build_parser(parent_subparsers: argparse._SubParsersAction) -> argparse.Argu
|
||||
)
|
||||
p_ctx.add_argument("task_id")
|
||||
|
||||
# --- specify --- (triage → todo via auxiliary LLM)
|
||||
p_specify = sub.add_parser(
|
||||
"specify",
|
||||
help="Flesh out a triage-column task into a concrete spec "
|
||||
"(title + body) and promote it to todo. Uses the auxiliary "
|
||||
"LLM configured under auxiliary.triage_specifier.",
|
||||
)
|
||||
p_specify.add_argument(
|
||||
"task_id",
|
||||
nargs="?",
|
||||
default=None,
|
||||
help="Task id to specify (required unless --all is given)",
|
||||
)
|
||||
p_specify.add_argument(
|
||||
"--all",
|
||||
dest="all_triage",
|
||||
action="store_true",
|
||||
help="Specify every task currently in the triage column",
|
||||
)
|
||||
p_specify.add_argument(
|
||||
"--tenant",
|
||||
default=None,
|
||||
help="When used with --all, restrict the sweep to this tenant",
|
||||
)
|
||||
p_specify.add_argument(
|
||||
"--author",
|
||||
default=None,
|
||||
help="Author name recorded on the audit comment "
|
||||
"(default: $HERMES_PROFILE or 'specifier')",
|
||||
)
|
||||
p_specify.add_argument(
|
||||
"--json",
|
||||
action="store_true",
|
||||
help="Emit one JSON object per task on stdout",
|
||||
)
|
||||
|
||||
# --- gc ---
|
||||
p_gc = sub.add_parser(
|
||||
"gc", help="Garbage-collect archived-task workspaces, old events, and old logs",
|
||||
@@ -674,6 +720,7 @@ def kanban_command(args: argparse.Namespace) -> int:
|
||||
"notify-list": _cmd_notify_list,
|
||||
"notify-unsubscribe": _cmd_notify_unsubscribe,
|
||||
"context": _cmd_context,
|
||||
"specify": _cmd_specify,
|
||||
"gc": _cmd_gc,
|
||||
}
|
||||
handler = handlers.get(action)
|
||||
@@ -982,6 +1029,14 @@ def _cmd_create(args: argparse.Namespace) -> int:
|
||||
except ValueError as exc:
|
||||
print(f"kanban: --max-runtime: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
max_retries = getattr(args, "max_retries", None)
|
||||
if max_retries is not None and max_retries < 1:
|
||||
print(
|
||||
f"kanban: --max-retries must be >= 1 (got {max_retries}); "
|
||||
"use 1 to trip on the first failure.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
with kb.connect() as conn:
|
||||
task_id = kb.create_task(
|
||||
conn,
|
||||
@@ -998,6 +1053,7 @@ def _cmd_create(args: argparse.Namespace) -> int:
|
||||
idempotency_key=getattr(args, "idempotency_key", None),
|
||||
max_runtime_seconds=max_runtime,
|
||||
skills=getattr(args, "skills", None) or None,
|
||||
max_retries=max_retries,
|
||||
)
|
||||
task = kb.get_task(conn, task_id)
|
||||
if getattr(args, "json", False):
|
||||
@@ -1125,6 +1181,23 @@ def _cmd_show(args: argparse.Namespace) -> int:
|
||||
(f" @ {task.workspace_path}" if task.workspace_path else ""))
|
||||
if task.skills:
|
||||
print(f" skills: {', '.join(task.skills)}")
|
||||
# Effective retry threshold. Show the per-task override if set,
|
||||
# otherwise the dispatcher's resolved value from config (or the
|
||||
# default if config doesn't set it either). Helps operators see
|
||||
# why a task auto-blocked earlier/later than they expected.
|
||||
if task.max_retries is not None:
|
||||
print(f" max-retries: {task.max_retries} (task)")
|
||||
else:
|
||||
try:
|
||||
from hermes_cli.config import load_config
|
||||
cfg = load_config()
|
||||
cfg_val = (cfg.get("kanban", {}) or {}).get("failure_limit")
|
||||
except Exception:
|
||||
cfg_val = None
|
||||
if cfg_val is not None and int(cfg_val) != kb.DEFAULT_FAILURE_LIMIT:
|
||||
print(f" max-retries: {int(cfg_val)} (config kanban.failure_limit)")
|
||||
else:
|
||||
print(f" max-retries: {kb.DEFAULT_FAILURE_LIMIT} (default)")
|
||||
print(f" created: {_fmt_ts(task.created_at)} by {task.created_by or '-'}")
|
||||
|
||||
# Diagnostics section — surface active distress signals at the top
|
||||
@@ -1657,6 +1730,7 @@ def _cmd_daemon(args: argparse.Namespace) -> int:
|
||||
" kanban:\n"
|
||||
" dispatch_in_gateway: true # default\n"
|
||||
" dispatch_interval_seconds: 60\n"
|
||||
" failure_limit: 2 # consecutive non-success attempts before auto-block\n"
|
||||
"\n"
|
||||
"Running both the gateway AND this standalone daemon will\n"
|
||||
"race for claims. If you truly need the old standalone\n"
|
||||
@@ -1943,6 +2017,80 @@ def _cmd_context(args: argparse.Namespace) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_specify(args: argparse.Namespace) -> int:
|
||||
"""Flesh out a triage task (or all of them) via auxiliary LLM,
|
||||
then promote to todo. Thin wrapper over ``kanban_specify``."""
|
||||
from hermes_cli import kanban_specify as spec
|
||||
|
||||
all_flag = bool(getattr(args, "all_triage", False))
|
||||
tenant = getattr(args, "tenant", None)
|
||||
author = getattr(args, "author", None) or _profile_author()
|
||||
want_json = bool(getattr(args, "json", False))
|
||||
|
||||
if args.task_id and all_flag:
|
||||
print(
|
||||
"kanban: pass either a task id OR --all, not both",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
|
||||
if all_flag:
|
||||
ids = spec.list_triage_ids(tenant=tenant)
|
||||
if not ids:
|
||||
msg = (
|
||||
"No triage tasks"
|
||||
+ (f" for tenant {tenant!r}" if tenant else "")
|
||||
+ "."
|
||||
)
|
||||
if want_json:
|
||||
print(json.dumps({"specified": 0, "total": 0}))
|
||||
else:
|
||||
print(msg)
|
||||
return 0
|
||||
elif args.task_id:
|
||||
ids = [args.task_id]
|
||||
else:
|
||||
print(
|
||||
"kanban: specify requires a task id or --all",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
|
||||
ok_count = 0
|
||||
fail_count = 0
|
||||
for tid in ids:
|
||||
outcome = spec.specify_task(tid, author=author)
|
||||
if outcome.ok:
|
||||
ok_count += 1
|
||||
else:
|
||||
fail_count += 1
|
||||
if want_json:
|
||||
print(json.dumps({
|
||||
"task_id": outcome.task_id,
|
||||
"ok": outcome.ok,
|
||||
"reason": outcome.reason,
|
||||
"new_title": outcome.new_title,
|
||||
}))
|
||||
else:
|
||||
if outcome.ok:
|
||||
title_suffix = (
|
||||
f" — retitled: {outcome.new_title!r}"
|
||||
if outcome.new_title
|
||||
else ""
|
||||
)
|
||||
print(f"Specified {outcome.task_id} → todo{title_suffix}")
|
||||
else:
|
||||
print(
|
||||
f"kanban: specify {outcome.task_id}: {outcome.reason}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
if not all_flag:
|
||||
return 0 if ok_count == 1 else 1
|
||||
# --all: succeed if at least one promotion landed; exit 1 only when
|
||||
# every candidate failed (honest signal for scripts).
|
||||
return 0 if (ok_count > 0 or not ids) else 1
|
||||
|
||||
|
||||
def _cmd_gc(args: argparse.Namespace) -> int:
|
||||
"""Remove scratch workspaces of archived tasks, prune old events, and
|
||||
delete old worker logs."""
|
||||
|
||||
+456
-60
@@ -595,6 +595,14 @@ class Task:
|
||||
# JSON array of skill names. None = use only the defaults; empty
|
||||
# list = explicitly no extra skills.
|
||||
skills: Optional[list] = None
|
||||
# Per-task override for the consecutive-failure circuit breaker.
|
||||
# The value is the failure count at which the breaker trips — e.g.
|
||||
# ``max_retries=1`` blocks on the first failure (zero retries),
|
||||
# ``max_retries=3`` blocks on the third (two retries allowed).
|
||||
# ``None`` (the common case) falls through to the dispatcher-level
|
||||
# ``kanban.failure_limit`` config, and then to ``DEFAULT_FAILURE_LIMIT``.
|
||||
# Name matches the ``--max-retries`` CLI flag on ``kanban create``.
|
||||
max_retries: Optional[int] = None
|
||||
|
||||
@classmethod
|
||||
def from_row(cls, row: sqlite3.Row) -> "Task":
|
||||
@@ -656,6 +664,9 @@ class Task:
|
||||
row["current_step_key"] if "current_step_key" in keys else None
|
||||
),
|
||||
skills=skills_value,
|
||||
max_retries=(
|
||||
row["max_retries"] if "max_retries" in keys else None
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -776,7 +787,13 @@ CREATE TABLE IF NOT EXISTS tasks (
|
||||
-- Force-loaded skills for the worker on this task, stored as JSON.
|
||||
-- Appended to the dispatcher's built-in `--skills kanban-worker`.
|
||||
-- NULL or empty array = no extras.
|
||||
skills TEXT
|
||||
skills TEXT,
|
||||
-- Per-task override for the consecutive-failure circuit breaker.
|
||||
-- The value is the failure count at which the breaker trips — e.g.
|
||||
-- ``max_retries=1`` blocks on the first failure. NULL (the common
|
||||
-- case) falls through to the dispatcher-level ``kanban.failure_limit``
|
||||
-- config and then ``DEFAULT_FAILURE_LIMIT``.
|
||||
max_retries INTEGER
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS task_links (
|
||||
@@ -1008,6 +1025,14 @@ def _migrate_add_optional_columns(conn: sqlite3.Connection) -> None:
|
||||
# for existing rows.
|
||||
conn.execute("ALTER TABLE tasks ADD COLUMN skills TEXT")
|
||||
|
||||
if "max_retries" not in cols:
|
||||
# Per-task override for the consecutive-failure circuit breaker.
|
||||
# NULL = fall through to the dispatcher-level ``kanban.failure_limit``
|
||||
# config, then ``DEFAULT_FAILURE_LIMIT``. Existing rows get NULL,
|
||||
# which is the correct default (they keep the global behaviour
|
||||
# they were getting before the column existed).
|
||||
conn.execute("ALTER TABLE tasks ADD COLUMN max_retries INTEGER")
|
||||
|
||||
# task_events gained a run_id column; back-fill it as NULL for
|
||||
# historical events (they predate runs and can't be attributed).
|
||||
ev_cols = {row["name"] for row in conn.execute("PRAGMA table_info(task_events)")}
|
||||
@@ -1163,6 +1188,7 @@ def create_task(
|
||||
idempotency_key: Optional[str] = None,
|
||||
max_runtime_seconds: Optional[int] = None,
|
||||
skills: Optional[Iterable[str]] = None,
|
||||
max_retries: Optional[int] = None,
|
||||
) -> str:
|
||||
"""Create a new task and optionally link it under parent tasks.
|
||||
|
||||
@@ -1276,8 +1302,9 @@ def create_task(
|
||||
INSERT INTO tasks (
|
||||
id, title, body, assignee, status, priority,
|
||||
created_by, created_at, workspace_kind, workspace_path,
|
||||
tenant, idempotency_key, max_runtime_seconds, skills
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
tenant, idempotency_key, max_runtime_seconds, skills,
|
||||
max_retries
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
task_id,
|
||||
@@ -1294,6 +1321,7 @@ def create_task(
|
||||
idempotency_key,
|
||||
int(max_runtime_seconds) if max_runtime_seconds else None,
|
||||
json.dumps(skills_list) if skills_list is not None else None,
|
||||
int(max_retries) if max_retries is not None else None,
|
||||
),
|
||||
)
|
||||
for pid in parents:
|
||||
@@ -1380,7 +1408,7 @@ def assign_task(conn: sqlite3.Connection, task_id: str, profile: Optional[str])
|
||||
profile = _canonical_assignee(profile)
|
||||
with write_txn(conn):
|
||||
row = conn.execute(
|
||||
"SELECT status, claim_lock FROM tasks WHERE id = ?", (task_id,)
|
||||
"SELECT status, claim_lock, assignee FROM tasks WHERE id = ?", (task_id,)
|
||||
).fetchone()
|
||||
if not row:
|
||||
return False
|
||||
@@ -1389,7 +1417,17 @@ def assign_task(conn: sqlite3.Connection, task_id: str, profile: Optional[str])
|
||||
f"cannot reassign {task_id}: currently running (claimed). "
|
||||
"Wait for completion or reclaim the stale lock first."
|
||||
)
|
||||
conn.execute("UPDATE tasks SET assignee = ? WHERE id = ?", (profile, task_id))
|
||||
if row["assignee"] != profile:
|
||||
# The retry guard is scoped to the task/profile combination. A
|
||||
# human reassigning the task is an explicit recovery action, so the
|
||||
# new profile should not inherit the previous profile's streak.
|
||||
conn.execute(
|
||||
"UPDATE tasks SET assignee = ?, consecutive_failures = 0, "
|
||||
"last_failure_error = NULL WHERE id = ?",
|
||||
(profile, task_id),
|
||||
)
|
||||
else:
|
||||
conn.execute("UPDATE tasks SET assignee = ? WHERE id = ?", (profile, task_id))
|
||||
_append_event(conn, task_id, "assigned", {"assignee": profile})
|
||||
return True
|
||||
|
||||
@@ -1859,34 +1897,47 @@ def heartbeat_claim(
|
||||
return False
|
||||
|
||||
|
||||
def release_stale_claims(conn: sqlite3.Connection) -> int:
|
||||
def release_stale_claims(
|
||||
conn: sqlite3.Connection,
|
||||
*,
|
||||
signal_fn=None,
|
||||
) -> int:
|
||||
"""Reset any ``running`` task whose claim has expired.
|
||||
|
||||
Returns the number of stale claims reclaimed. Safe to call often.
|
||||
"""
|
||||
now = int(time.time())
|
||||
reclaimed = 0
|
||||
with write_txn(conn):
|
||||
stale = conn.execute(
|
||||
"SELECT id, claim_lock FROM tasks "
|
||||
"WHERE status = 'running' AND claim_expires IS NOT NULL AND claim_expires < ?",
|
||||
(now,),
|
||||
).fetchall()
|
||||
for row in stale:
|
||||
conn.execute(
|
||||
stale = conn.execute(
|
||||
"SELECT id, claim_lock, worker_pid FROM tasks "
|
||||
"WHERE status = 'running' AND claim_expires IS NOT NULL AND claim_expires < ?",
|
||||
(now,),
|
||||
).fetchall()
|
||||
for row in stale:
|
||||
termination = _terminate_reclaimed_worker(
|
||||
row["worker_pid"], row["claim_lock"], signal_fn=signal_fn,
|
||||
)
|
||||
with write_txn(conn):
|
||||
cur = conn.execute(
|
||||
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
||||
"claim_expires = NULL, worker_pid = NULL "
|
||||
"WHERE id = ? AND status = 'running'",
|
||||
(row["id"],),
|
||||
"WHERE id = ? AND status = 'running' AND claim_lock IS ? "
|
||||
"AND claim_expires IS NOT NULL AND claim_expires < ?",
|
||||
(row["id"], row["claim_lock"], now),
|
||||
)
|
||||
if cur.rowcount != 1:
|
||||
continue
|
||||
run_id = _end_run(
|
||||
conn, row["id"],
|
||||
outcome="reclaimed", status="reclaimed",
|
||||
error=f"stale_lock={row['claim_lock']}",
|
||||
metadata=termination,
|
||||
)
|
||||
payload = {"stale_lock": row["claim_lock"]}
|
||||
payload.update(termination)
|
||||
_append_event(
|
||||
conn, row["id"], "reclaimed",
|
||||
{"stale_lock": row["claim_lock"]},
|
||||
payload,
|
||||
run_id=run_id,
|
||||
)
|
||||
reclaimed += 1
|
||||
@@ -1898,6 +1949,7 @@ def reclaim_task(
|
||||
task_id: str,
|
||||
*,
|
||||
reason: Optional[str] = None,
|
||||
signal_fn=None,
|
||||
) -> bool:
|
||||
"""Operator-driven reclaim: release the claim and reset to ``ready``.
|
||||
|
||||
@@ -1910,24 +1962,29 @@ def reclaim_task(
|
||||
Returns True if a reclaim happened, False if the task isn't in a
|
||||
reclaimable state (not running, or doesn't exist).
|
||||
"""
|
||||
row = conn.execute(
|
||||
"SELECT status, claim_lock, worker_pid FROM tasks WHERE id = ?",
|
||||
(task_id,),
|
||||
).fetchone()
|
||||
if not row:
|
||||
return False
|
||||
if row["status"] != "running" and row["claim_lock"] is None:
|
||||
# Nothing to reclaim — already ready / blocked / done.
|
||||
return False
|
||||
prev_lock = row["claim_lock"]
|
||||
termination = _terminate_reclaimed_worker(
|
||||
row["worker_pid"], prev_lock, signal_fn=signal_fn,
|
||||
)
|
||||
with write_txn(conn):
|
||||
row = conn.execute(
|
||||
"SELECT status, claim_lock, worker_pid FROM tasks WHERE id = ?",
|
||||
(task_id,),
|
||||
).fetchone()
|
||||
if not row:
|
||||
return False
|
||||
if row["status"] != "running" and row["claim_lock"] is None:
|
||||
# Nothing to reclaim — already ready / blocked / done.
|
||||
return False
|
||||
prev_lock = row["claim_lock"]
|
||||
prev_pid = row["worker_pid"]
|
||||
conn.execute(
|
||||
cur = conn.execute(
|
||||
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
||||
"claim_expires = NULL, worker_pid = NULL "
|
||||
"WHERE id = ? AND status IN ('running', 'ready', 'blocked')",
|
||||
(task_id,),
|
||||
"WHERE id = ? AND status IN ('running', 'ready', 'blocked') "
|
||||
"AND claim_lock IS ?",
|
||||
(task_id, prev_lock),
|
||||
)
|
||||
if cur.rowcount != 1:
|
||||
return False
|
||||
run_id = _end_run(
|
||||
conn, task_id,
|
||||
outcome="reclaimed", status="reclaimed",
|
||||
@@ -1935,15 +1992,17 @@ def reclaim_task(
|
||||
f"manual_reclaim: {reason}" if reason
|
||||
else f"manual_reclaim lock={prev_lock}"
|
||||
),
|
||||
metadata=termination,
|
||||
)
|
||||
payload = {
|
||||
"manual": True,
|
||||
"reason": reason,
|
||||
"prev_lock": prev_lock,
|
||||
}
|
||||
payload.update(termination)
|
||||
_append_event(
|
||||
conn, task_id, "reclaimed",
|
||||
{
|
||||
"manual": True,
|
||||
"reason": reason,
|
||||
"prev_lock": prev_lock,
|
||||
"prev_pid": prev_pid,
|
||||
},
|
||||
payload,
|
||||
run_id=run_id,
|
||||
)
|
||||
# Operator intervention — they've looked at the task, so the
|
||||
@@ -2444,6 +2503,91 @@ def unblock_task(conn: sqlite3.Connection, task_id: str) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def specify_triage_task(
|
||||
conn: sqlite3.Connection,
|
||||
task_id: str,
|
||||
*,
|
||||
title: Optional[str] = None,
|
||||
body: Optional[str] = None,
|
||||
author: Optional[str] = None,
|
||||
) -> bool:
|
||||
"""Flesh out a triage task and promote it to ``todo``.
|
||||
|
||||
Atomically updates ``title`` / ``body`` (when provided) and transitions
|
||||
``status: triage -> todo`` in a single write txn. Returns False when
|
||||
the task is missing or not in the ``triage`` column — callers should
|
||||
surface that as "nothing to specify" rather than an error.
|
||||
|
||||
``todo`` (not ``ready``) is the correct landing column: ``recompute_ready``
|
||||
promotes parent-free / parent-done todos to ``ready`` on the next
|
||||
dispatcher tick, which keeps the normal parent-gating behaviour intact
|
||||
for specified tasks that happen to have open parents.
|
||||
|
||||
``author`` is recorded on an audit comment only when at least one of
|
||||
``title`` / ``body`` actually changed — avoids noisy comment spam for
|
||||
status-only promotions.
|
||||
"""
|
||||
if title is not None and not title.strip():
|
||||
raise ValueError("title cannot be blank")
|
||||
with write_txn(conn):
|
||||
existing = conn.execute(
|
||||
"SELECT title, body FROM tasks WHERE id = ? AND status = 'triage'",
|
||||
(task_id,),
|
||||
).fetchone()
|
||||
if existing is None:
|
||||
return False
|
||||
sets: list[str] = ["status = 'todo'"]
|
||||
params: list[Any] = []
|
||||
changed_fields: list[str] = []
|
||||
if title is not None and title.strip() != (existing["title"] or ""):
|
||||
sets.append("title = ?")
|
||||
params.append(title.strip())
|
||||
changed_fields.append("title")
|
||||
if body is not None and (body or "") != (existing["body"] or ""):
|
||||
sets.append("body = ?")
|
||||
params.append(body)
|
||||
changed_fields.append("body")
|
||||
params.append(task_id)
|
||||
cur = conn.execute(
|
||||
f"UPDATE tasks SET {', '.join(sets)} "
|
||||
f"WHERE id = ? AND status = 'triage'",
|
||||
tuple(params),
|
||||
)
|
||||
if cur.rowcount != 1:
|
||||
return False
|
||||
if changed_fields and author and author.strip():
|
||||
# Inline INSERT (rather than ``add_comment``) because we're
|
||||
# already inside this function's write_txn — nested BEGIN
|
||||
# IMMEDIATE would raise OperationalError. We also skip the
|
||||
# 'commented' event that ``add_comment`` emits, since the
|
||||
# 'specified' event below already records the change.
|
||||
conn.execute(
|
||||
"INSERT INTO task_comments (task_id, author, body, created_at) "
|
||||
"VALUES (?, ?, ?, ?)",
|
||||
(
|
||||
task_id,
|
||||
author.strip(),
|
||||
"Specified — updated "
|
||||
+ ", ".join(changed_fields)
|
||||
+ " and promoted to todo.",
|
||||
int(time.time()),
|
||||
),
|
||||
)
|
||||
_append_event(
|
||||
conn,
|
||||
task_id,
|
||||
"specified",
|
||||
{"changed_fields": changed_fields} if changed_fields else None,
|
||||
)
|
||||
# Outside the write_txn above, so we don't nest BEGIN IMMEDIATE — the
|
||||
# ready-promotion pass opens its own IMMEDIATE txn. This runs the same
|
||||
# logic the dispatcher would on its next tick, so a specified task
|
||||
# with no open parents flips straight to 'ready' here instead of
|
||||
# idling in 'todo' until the next sweep.
|
||||
recompute_ready(conn)
|
||||
return True
|
||||
|
||||
|
||||
def archive_task(conn: sqlite3.Connection, task_id: str) -> bool:
|
||||
with write_txn(conn):
|
||||
cur = conn.execute(
|
||||
@@ -2548,11 +2692,11 @@ def set_workspace_path(
|
||||
# Dispatcher (one-shot pass)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# After this many consecutive `spawn_failed` events on a task, the dispatcher
|
||||
# stops retrying and parks the task in ``blocked`` with a reason so a human
|
||||
# can investigate. Prevents the dispatcher from thrashing forever on a task
|
||||
# whose profile doesn't exist, whose workspace is unmountable, etc.
|
||||
DEFAULT_FAILURE_LIMIT = 5
|
||||
# After this many consecutive non-success attempts on a task/profile, the
|
||||
# dispatcher stops retrying and parks the task in ``blocked`` with a reason so
|
||||
# a human can investigate. Prevents retry storms when a worker repeatedly times
|
||||
# out, crashes, or cannot spawn.
|
||||
DEFAULT_FAILURE_LIMIT = 2
|
||||
# Legacy alias — callers / tests still reference the old name.
|
||||
DEFAULT_SPAWN_FAILURE_LIMIT = DEFAULT_FAILURE_LIMIT
|
||||
|
||||
@@ -2587,6 +2731,77 @@ class DispatchResult:
|
||||
"""Task ids whose workers exceeded ``max_runtime_seconds``."""
|
||||
|
||||
|
||||
# Bounded registry of recently-reaped worker child exits, populated by the
|
||||
# reap loop at the top of ``dispatch_once`` and consulted by
|
||||
# ``detect_crashed_workers`` to classify a dead-pid task.
|
||||
#
|
||||
# Entry: ``pid -> (raw_wait_status, reaped_at_epoch)``. We keep raw status
|
||||
# so both ``os.WIFEXITED`` / ``os.WEXITSTATUS`` and ``os.WIFSIGNALED`` can
|
||||
# be consulted. Entries are trimmed by age (and total size cap as a
|
||||
# belt-and-braces against unbounded growth on exotic platforms).
|
||||
_RECENT_WORKER_EXIT_TTL_SECONDS = 600
|
||||
_RECENT_WORKER_EXITS_MAX = 4096
|
||||
_recent_worker_exits: "dict[int, tuple[int, float]]" = {}
|
||||
|
||||
|
||||
def _record_worker_exit(pid: int, raw_status: int) -> None:
|
||||
"""Record a reaped child's exit status for later classification.
|
||||
|
||||
Called from the reap loop in ``dispatch_once``. Safe to call many
|
||||
times; duplicate pids overwrite (pids can cycle, latest wins).
|
||||
"""
|
||||
if not pid or pid <= 0:
|
||||
return
|
||||
now = time.time()
|
||||
_recent_worker_exits[int(pid)] = (int(raw_status), now)
|
||||
# Age-based trim: drop entries older than the TTL.
|
||||
if len(_recent_worker_exits) > _RECENT_WORKER_EXITS_MAX // 2:
|
||||
cutoff = now - _RECENT_WORKER_EXIT_TTL_SECONDS
|
||||
for _pid in [p for p, (_s, t) in _recent_worker_exits.items() if t < cutoff]:
|
||||
_recent_worker_exits.pop(_pid, None)
|
||||
# Size cap as a final guard.
|
||||
if len(_recent_worker_exits) > _RECENT_WORKER_EXITS_MAX:
|
||||
# Drop oldest half.
|
||||
ordered = sorted(_recent_worker_exits.items(), key=lambda kv: kv[1][1])
|
||||
for _pid, _ in ordered[: len(ordered) // 2]:
|
||||
_recent_worker_exits.pop(_pid, None)
|
||||
|
||||
|
||||
def _classify_worker_exit(pid: int) -> "tuple[str, Optional[int]]":
|
||||
"""Classify a recently-reaped worker by pid.
|
||||
|
||||
Returns ``(kind, code)`` where ``kind`` is one of:
|
||||
|
||||
* ``"clean_exit"`` — ``WIFEXITED`` with ``WEXITSTATUS == 0``. When the
|
||||
task is still ``running`` in the DB, this is a protocol violation
|
||||
(worker exited without calling ``kanban_complete`` / ``kanban_block``)
|
||||
and should be auto-blocked immediately — retrying will just loop.
|
||||
* ``"nonzero_exit"`` — ``WIFEXITED`` with non-zero status. Real error.
|
||||
* ``"signaled"`` — ``WIFSIGNALED`` (OOM killer, SIGKILL, etc). Real crash.
|
||||
* ``"unknown"`` — pid was not in the reap registry (either reaped by
|
||||
something else, or died between reap tick and liveness check). Fall
|
||||
back to existing crashed-counter behavior.
|
||||
|
||||
``code`` is the exit status (for ``clean_exit`` / ``nonzero_exit``) or
|
||||
the signal number (for ``signaled``), or ``None`` for ``unknown``.
|
||||
"""
|
||||
entry = _recent_worker_exits.get(int(pid))
|
||||
if entry is None:
|
||||
return ("unknown", None)
|
||||
raw, _ = entry
|
||||
try:
|
||||
if os.WIFEXITED(raw):
|
||||
code = os.WEXITSTATUS(raw)
|
||||
if code == 0:
|
||||
return ("clean_exit", 0)
|
||||
return ("nonzero_exit", code)
|
||||
if os.WIFSIGNALED(raw):
|
||||
return ("signaled", os.WTERMSIG(raw))
|
||||
except Exception:
|
||||
pass
|
||||
return ("unknown", None)
|
||||
|
||||
|
||||
def _pid_alive(pid: Optional[int]) -> bool:
|
||||
"""Return True if ``pid`` is still running on this host.
|
||||
|
||||
@@ -2620,7 +2835,7 @@ def _pid_alive(pid: Optional[int]) -> bool:
|
||||
# where we have a cheap, deterministic process-state probe.
|
||||
if sys.platform == "linux":
|
||||
try:
|
||||
with open(f"/proc/{int(pid)}/status", "r") as f:
|
||||
with open(f"/proc/{int(pid)}/status", "r", encoding="utf-8") as f:
|
||||
for line in f:
|
||||
if line.startswith("State:"):
|
||||
# "State:\tZ (zombie)" → dead
|
||||
@@ -2652,6 +2867,62 @@ def _pid_alive(pid: Optional[int]) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def _terminate_reclaimed_worker(
|
||||
pid: Optional[int],
|
||||
claim_lock: Optional[str],
|
||||
*,
|
||||
signal_fn=None,
|
||||
) -> dict[str, Any]:
|
||||
"""Best-effort host-local worker termination for reclaim paths."""
|
||||
import signal
|
||||
|
||||
info: dict[str, Any] = {
|
||||
"prev_pid": int(pid) if pid else None,
|
||||
"host_local": False,
|
||||
"termination_attempted": False,
|
||||
"terminated": False,
|
||||
"sigkill": False,
|
||||
}
|
||||
if not pid or pid <= 0 or not claim_lock:
|
||||
return info
|
||||
|
||||
host_prefix = f"{_claimer_id().split(':', 1)[0]}:"
|
||||
if not str(claim_lock).startswith(host_prefix):
|
||||
return info
|
||||
info["host_local"] = True
|
||||
|
||||
kill = signal_fn if signal_fn is not None else (
|
||||
os.kill if hasattr(os, "kill") else None
|
||||
)
|
||||
if kill is None:
|
||||
return info
|
||||
|
||||
info["termination_attempted"] = True
|
||||
try:
|
||||
kill(int(pid), signal.SIGTERM)
|
||||
except (ProcessLookupError, OSError):
|
||||
return info
|
||||
|
||||
for _ in range(10):
|
||||
if not _pid_alive(pid):
|
||||
info["terminated"] = True
|
||||
return info
|
||||
time.sleep(0.5)
|
||||
|
||||
if _pid_alive(pid):
|
||||
try:
|
||||
# signal.SIGKILL doesn't exist on Windows; fall back to SIGTERM
|
||||
# (which maps to TerminateProcess via the stdlib shim).
|
||||
_sigkill = getattr(signal, "SIGKILL", signal.SIGTERM)
|
||||
kill(int(pid), _sigkill)
|
||||
info["sigkill"] = True
|
||||
except (ProcessLookupError, OSError):
|
||||
return info
|
||||
|
||||
info["terminated"] = not _pid_alive(pid)
|
||||
return info
|
||||
|
||||
|
||||
def heartbeat_worker(
|
||||
conn: sqlite3.Connection,
|
||||
task_id: str,
|
||||
@@ -2767,7 +3038,9 @@ def enforce_max_runtime(
|
||||
time.sleep(0.5)
|
||||
if _pid_alive(pid):
|
||||
try:
|
||||
kill(pid, signal.SIGKILL)
|
||||
# signal.SIGKILL doesn't exist on Windows.
|
||||
_sigkill = getattr(signal, "SIGKILL", signal.SIGTERM)
|
||||
kill(pid, _sigkill)
|
||||
killed = True
|
||||
except (ProcessLookupError, OSError):
|
||||
pass
|
||||
@@ -2840,12 +3113,22 @@ def detect_crashed_workers(conn: sqlite3.Connection) -> list[str]:
|
||||
are meaningless here. The host-local check is enough because
|
||||
``_default_spawn`` always runs the worker on the same host as the
|
||||
dispatcher (the whole design is single-host).
|
||||
|
||||
When the reap registry shows the worker exited cleanly (rc=0) but
|
||||
the task was still ``running`` in the DB, treat it as a protocol
|
||||
violation (worker answered conversationally without calling
|
||||
``kanban_complete`` / ``kanban_block``) and trip the circuit breaker
|
||||
on the first occurrence — retrying a worker whose CLI keeps
|
||||
returning 0 without a terminal transition just loops forever.
|
||||
"""
|
||||
crashed: list[str] = []
|
||||
# Per-crash details collected inside the main txn, used after it
|
||||
# closes to run ``_record_task_failure`` (which needs its own
|
||||
# write_txn so can't nest).
|
||||
crash_details: list[tuple[str, int, str]] = [] # (task_id, pid, claimer)
|
||||
# write_txn so can't nest). ``protocol_violation`` flags the
|
||||
# clean-exit-but-still-running case so we can trip the breaker
|
||||
# immediately instead of incrementing by 1.
|
||||
crash_details: list[tuple[str, int, str, bool, str]] = []
|
||||
# (task_id, pid, claimer, protocol_violation, error_text)
|
||||
with write_txn(conn):
|
||||
rows = conn.execute(
|
||||
"SELECT id, worker_pid, claim_lock FROM tasks "
|
||||
@@ -2859,6 +3142,39 @@ def detect_crashed_workers(conn: sqlite3.Connection) -> list[str]:
|
||||
continue
|
||||
if _pid_alive(row["worker_pid"]):
|
||||
continue
|
||||
|
||||
pid = int(row["worker_pid"])
|
||||
kind, code = _classify_worker_exit(pid)
|
||||
if kind == "clean_exit":
|
||||
# Worker subprocess returned 0 but its task is still
|
||||
# ``running`` in the DB — it exited without calling
|
||||
# ``kanban_complete`` / ``kanban_block``. Retrying won't
|
||||
# help.
|
||||
protocol_violation = True
|
||||
error_text = (
|
||||
"worker exited cleanly (rc=0) without calling "
|
||||
"kanban_complete or kanban_block — protocol violation"
|
||||
)
|
||||
event_kind = "protocol_violation"
|
||||
event_payload = {
|
||||
"pid": pid,
|
||||
"claimer": row["claim_lock"],
|
||||
"exit_code": code,
|
||||
}
|
||||
else:
|
||||
protocol_violation = False
|
||||
if kind == "nonzero_exit":
|
||||
error_text = f"pid {pid} exited with code {code}"
|
||||
elif kind == "signaled":
|
||||
error_text = f"pid {pid} killed by signal {code}"
|
||||
else:
|
||||
error_text = f"pid {pid} not alive"
|
||||
event_kind = "crashed"
|
||||
event_payload = {"pid": pid, "claimer": row["claim_lock"]}
|
||||
if code is not None and kind != "unknown":
|
||||
event_payload["exit_kind"] = kind
|
||||
event_payload["exit_code"] = code
|
||||
|
||||
cur = conn.execute(
|
||||
"UPDATE tasks SET status = 'ready', claim_lock = NULL, "
|
||||
"claim_expires = NULL, worker_pid = NULL "
|
||||
@@ -2869,34 +3185,47 @@ def detect_crashed_workers(conn: sqlite3.Connection) -> list[str]:
|
||||
run_id = _end_run(
|
||||
conn, row["id"],
|
||||
outcome="crashed", status="crashed",
|
||||
error=f"pid {int(row['worker_pid'])} not alive",
|
||||
metadata={
|
||||
"pid": int(row["worker_pid"]),
|
||||
"claimer": row["claim_lock"],
|
||||
},
|
||||
error=error_text,
|
||||
metadata=dict(event_payload),
|
||||
)
|
||||
_append_event(
|
||||
conn, row["id"], "crashed",
|
||||
{"pid": int(row["worker_pid"]), "claimer": row["claim_lock"]},
|
||||
conn, row["id"], event_kind,
|
||||
event_payload,
|
||||
run_id=run_id,
|
||||
)
|
||||
crashed.append(row["id"])
|
||||
crash_details.append(
|
||||
(row["id"], int(row["worker_pid"]), row["claim_lock"])
|
||||
(row["id"], pid, row["claim_lock"],
|
||||
protocol_violation, error_text)
|
||||
)
|
||||
# Outside the main txn: increment the unified failure counter for
|
||||
# each crashed task. If the breaker trips, the task transitions
|
||||
# ready → blocked with a ``gave_up`` event on top of the ``crashed``
|
||||
# event we already emitted.
|
||||
for tid, pid, claimer in crash_details:
|
||||
_record_task_failure(
|
||||
#
|
||||
# Protocol-violation crashes force an immediate trip (failure_limit=1)
|
||||
# because clean-exit-without-transition is deterministic: the next
|
||||
# respawn will do exactly the same thing. Better to surface to a
|
||||
# human with a clear reason than to loop ``DEFAULT_FAILURE_LIMIT``
|
||||
# times first.
|
||||
auto_blocked: list[str] = []
|
||||
for tid, pid, claimer, protocol_violation, error_text in crash_details:
|
||||
tripped = _record_task_failure(
|
||||
conn, tid,
|
||||
error=f"pid {pid} not alive",
|
||||
error=error_text,
|
||||
outcome="crashed",
|
||||
failure_limit=(1 if protocol_violation else None),
|
||||
release_claim=False,
|
||||
end_run=False,
|
||||
event_payload_extra={"pid": pid, "claimer": claimer},
|
||||
)
|
||||
if tripped:
|
||||
auto_blocked.append(tid)
|
||||
# Stash auto-blocked ids on the function for the dispatch loop to pick up.
|
||||
# Keeps the public return type (``list[str]``) stable for direct callers
|
||||
# and tests that destructure the result; ``dispatch_once`` reads this
|
||||
# side-channel attribute to populate ``DispatchResult.auto_blocked``.
|
||||
detect_crashed_workers._last_auto_blocked = auto_blocked # type: ignore[attr-defined]
|
||||
return crashed
|
||||
|
||||
|
||||
@@ -2938,20 +3267,39 @@ def _record_task_failure(
|
||||
``event_payload_extra`` merges into the ``gave_up`` event payload
|
||||
when the breaker trips, so callers can include outcome-specific
|
||||
context (e.g. pid on crash, elapsed on timeout).
|
||||
|
||||
Resolution order for the effective threshold:
|
||||
1. per-task ``max_retries`` if set (nothing else overrides)
|
||||
2. caller-supplied ``failure_limit`` (gateway passes the config
|
||||
value from ``kanban.failure_limit``; tests pass fixed values)
|
||||
3. ``DEFAULT_FAILURE_LIMIT``
|
||||
"""
|
||||
if failure_limit is None:
|
||||
failure_limit = DEFAULT_FAILURE_LIMIT
|
||||
blocked = False
|
||||
with write_txn(conn):
|
||||
row = conn.execute(
|
||||
"SELECT consecutive_failures, status FROM tasks WHERE id = ?", (task_id,),
|
||||
"SELECT consecutive_failures, status, max_retries "
|
||||
"FROM tasks WHERE id = ?", (task_id,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return False
|
||||
failures = int(row["consecutive_failures"]) + 1
|
||||
cur_status = row["status"]
|
||||
|
||||
if failures >= failure_limit:
|
||||
# Per-task override wins over both caller-supplied and default
|
||||
# thresholds. None (the common case) falls through.
|
||||
task_override = (
|
||||
row["max_retries"] if "max_retries" in row.keys() else None
|
||||
)
|
||||
if task_override is not None:
|
||||
effective_limit = int(task_override)
|
||||
limit_source = "task"
|
||||
else:
|
||||
effective_limit = int(failure_limit)
|
||||
limit_source = "dispatcher"
|
||||
|
||||
if failures >= effective_limit:
|
||||
# Trip the breaker.
|
||||
if release_claim:
|
||||
# Spawn path: still running, also clear claim state.
|
||||
@@ -2979,10 +3327,17 @@ def _record_task_failure(
|
||||
conn, task_id,
|
||||
outcome="gave_up", status="gave_up",
|
||||
error=error[:500],
|
||||
metadata={"failures": failures, "trigger_outcome": outcome},
|
||||
metadata={
|
||||
"failures": failures,
|
||||
"trigger_outcome": outcome,
|
||||
"effective_limit": effective_limit,
|
||||
"limit_source": limit_source,
|
||||
},
|
||||
)
|
||||
payload = {
|
||||
"failures": failures,
|
||||
"effective_limit": effective_limit,
|
||||
"limit_source": limit_source,
|
||||
"error": error[:500],
|
||||
"trigger_outcome": outcome,
|
||||
}
|
||||
@@ -3150,9 +3505,50 @@ def dispatch_once(
|
||||
``board`` pins workspace/log/db resolution for this tick to a specific
|
||||
board. When omitted, the current-board resolution chain is used.
|
||||
"""
|
||||
# Reap zombie children from previously spawned workers.
|
||||
# The gateway-embedded dispatcher is the parent of every worker spawned
|
||||
# via _default_spawn (start_new_session=True only detaches the
|
||||
# controlling tty, not the parent). Without an explicit waitpid, each
|
||||
# completed worker becomes a <defunct> entry that lingers until gateway
|
||||
# exit. WNOHANG keeps this non-blocking; ChildProcessError means no
|
||||
# children to reap. Bounded: at most one tick's worth of completions
|
||||
# can be in <defunct> at once.
|
||||
#
|
||||
# We also record the exit status keyed by pid, so
|
||||
# ``detect_crashed_workers`` can distinguish a worker that exited
|
||||
# cleanly without calling ``kanban_complete`` / ``kanban_block``
|
||||
# (protocol violation — auto-block) from a real crash (OOM killer,
|
||||
# SIGKILL, non-zero exit — existing counter behavior).
|
||||
#
|
||||
# Windows has no zombies / no os.WNOHANG — subprocess.Popen handles
|
||||
# are freed when the Python object is garbage-collected or .wait() is
|
||||
# called explicitly. The kanban dispatcher discards the Popen handle
|
||||
# after spawn (``_default_spawn`` → abandon), so on Windows there's
|
||||
# nothing to reap here — skip the whole block.
|
||||
if os.name != "nt":
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
_pid, _status = os.waitpid(-1, os.WNOHANG)
|
||||
except ChildProcessError:
|
||||
break
|
||||
if _pid == 0:
|
||||
break
|
||||
_record_worker_exit(_pid, _status)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
result = DispatchResult()
|
||||
result.reclaimed = release_stale_claims(conn)
|
||||
result.crashed = detect_crashed_workers(conn)
|
||||
# detect_crashed_workers stashes protocol-violation auto-blocks on
|
||||
# itself so the public list-return stays stable. Pull them into the
|
||||
# DispatchResult here so telemetry / tests see the trip.
|
||||
_crash_auto_blocked = getattr(
|
||||
detect_crashed_workers, "_last_auto_blocked", []
|
||||
)
|
||||
if _crash_auto_blocked:
|
||||
result.auto_blocked.extend(_crash_auto_blocked)
|
||||
result.timed_out = enforce_max_runtime(conn)
|
||||
result.promoted = recompute_ready(conn)
|
||||
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
"""Kanban triage specifier — flesh out a one-liner into a real spec.
|
||||
|
||||
Used by ``hermes kanban specify [task_id | --all]``. Takes a task that
|
||||
lives in the Triage column (a rough idea, typically only a title), calls
|
||||
the auxiliary LLM to produce:
|
||||
|
||||
* A tightened title (optional — only replaces if the model proposes a
|
||||
materially different one)
|
||||
* A concrete body: goal, proposed approach, acceptance criteria
|
||||
|
||||
and then flips the task ``triage -> todo`` via
|
||||
``kanban_db.specify_triage_task``. The dispatcher promotes it to
|
||||
``ready`` on its next tick (or immediately if there are no open parents).
|
||||
|
||||
Design notes
|
||||
------------
|
||||
|
||||
* This module intentionally mirrors ``hermes_cli/goals.py`` — same aux
|
||||
client pattern, same "empty config => skip, don't crash" tolerance.
|
||||
Keeps the surface area tiny and the failure modes predictable.
|
||||
|
||||
* The prompt is a short system + user pair. We ask for JSON with
|
||||
``{title, body}``; if parsing fails, we fall back to treating the
|
||||
whole response as the body and leave the title untouched. No
|
||||
retry loop — one shot, keep cost bounded.
|
||||
|
||||
* Structured output / JSON mode is not requested explicitly so the
|
||||
specifier works on providers that don't implement it. The parse
|
||||
is lenient (tolerates markdown code fences around the JSON).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
from hermes_cli import kanban_db as kb
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
_SYSTEM_PROMPT = """You are the Kanban triage specifier for the Hermes Agent board.
|
||||
A user dropped a rough idea into the Triage column. Your job is to turn it
|
||||
into a concrete, actionable task spec that an autonomous worker can pick up
|
||||
and execute without further clarification.
|
||||
|
||||
Output a single JSON object with exactly two keys:
|
||||
|
||||
{
|
||||
"title": "<tightened task title, <= 80 chars, imperative voice>",
|
||||
"body": "<multi-line spec, see structure below>"
|
||||
}
|
||||
|
||||
The body MUST include these sections, each prefixed with a bold markdown
|
||||
heading, in this order:
|
||||
|
||||
**Goal** — one sentence, user-facing outcome.
|
||||
**Approach** — 2-5 bullets on how a worker should tackle it.
|
||||
**Acceptance criteria** — checklist of concrete, verifiable conditions.
|
||||
**Out of scope** — short list of things NOT to touch (omit if nothing
|
||||
obvious; never invent scope creep).
|
||||
|
||||
Rules:
|
||||
- Keep the tightened title close in meaning to the original idea — do
|
||||
NOT invent a different project.
|
||||
- If the original idea is already detailed, preserve its substance and
|
||||
just reformat into the sections above.
|
||||
- Never add invented requirements the user didn't hint at.
|
||||
- No preamble, no closing remarks, no code fences around the JSON.
|
||||
- Output only the JSON object and nothing else.
|
||||
"""
|
||||
|
||||
|
||||
_USER_TEMPLATE = """Task id: {task_id}
|
||||
Current title: {title}
|
||||
Current body:
|
||||
{body}
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class SpecifyOutcome:
|
||||
"""Result of specifying a single triage task."""
|
||||
|
||||
task_id: str
|
||||
ok: bool
|
||||
reason: str = ""
|
||||
new_title: Optional[str] = None
|
||||
|
||||
|
||||
def _truncate(text: str, limit: int) -> str:
|
||||
if len(text) <= limit:
|
||||
return text
|
||||
return text[: limit - 1] + "…"
|
||||
|
||||
|
||||
_FENCE_RE = re.compile(r"^\s*```(?:json)?\s*|\s*```\s*$", re.IGNORECASE)
|
||||
|
||||
|
||||
def _extract_json_blob(raw: str) -> Optional[dict]:
|
||||
"""Lenient JSON extraction — tolerates fenced code blocks and
|
||||
leading/trailing whitespace. Returns None if nothing parses."""
|
||||
if not raw:
|
||||
return None
|
||||
stripped = _FENCE_RE.sub("", raw.strip())
|
||||
# Greedy: find the first `{` and last `}` and try that slice.
|
||||
first = stripped.find("{")
|
||||
last = stripped.rfind("}")
|
||||
if first == -1 or last == -1 or last <= first:
|
||||
return None
|
||||
candidate = stripped[first : last + 1]
|
||||
try:
|
||||
val = json.loads(candidate)
|
||||
except (ValueError, json.JSONDecodeError):
|
||||
return None
|
||||
if not isinstance(val, dict):
|
||||
return None
|
||||
return val
|
||||
|
||||
|
||||
def _profile_author() -> str:
|
||||
"""Mirror of ``hermes_cli.kanban._profile_author``. Kept local to
|
||||
avoid a circular import when kanban.py imports this module."""
|
||||
return (
|
||||
os.environ.get("HERMES_PROFILE")
|
||||
or os.environ.get("USER")
|
||||
or "specifier"
|
||||
)
|
||||
|
||||
|
||||
def specify_task(
|
||||
task_id: str,
|
||||
*,
|
||||
author: Optional[str] = None,
|
||||
timeout: Optional[int] = None,
|
||||
) -> SpecifyOutcome:
|
||||
"""Specify a single triage task and promote it to ``todo``.
|
||||
|
||||
Returns an outcome describing what happened. Never raises for expected
|
||||
failure modes (task not in triage, no aux client configured, API
|
||||
error, malformed response) — those surface via ``ok=False`` so the
|
||||
``--all`` sweep can continue past individual failures.
|
||||
"""
|
||||
with kb.connect() as conn:
|
||||
task = kb.get_task(conn, task_id)
|
||||
if task is None:
|
||||
return SpecifyOutcome(task_id, False, "unknown task id")
|
||||
if task.status != "triage":
|
||||
return SpecifyOutcome(
|
||||
task_id, False, f"task is not in triage (status={task.status!r})"
|
||||
)
|
||||
|
||||
try:
|
||||
from agent.auxiliary_client import get_text_auxiliary_client
|
||||
except Exception as exc: # pragma: no cover — import smoke test
|
||||
logger.debug("specify: auxiliary client import failed: %s", exc)
|
||||
return SpecifyOutcome(task_id, False, "auxiliary client unavailable")
|
||||
|
||||
try:
|
||||
client, model = get_text_auxiliary_client("triage_specifier")
|
||||
except Exception as exc:
|
||||
logger.debug("specify: get_text_auxiliary_client failed: %s", exc)
|
||||
return SpecifyOutcome(task_id, False, "auxiliary client unavailable")
|
||||
|
||||
if client is None or not model:
|
||||
return SpecifyOutcome(
|
||||
task_id, False, "no auxiliary client configured"
|
||||
)
|
||||
|
||||
user_msg = _USER_TEMPLATE.format(
|
||||
task_id=task.id,
|
||||
title=_truncate(task.title or "", 400),
|
||||
body=_truncate(task.body or "(no body)", 4000),
|
||||
)
|
||||
|
||||
try:
|
||||
resp = client.chat.completions.create(
|
||||
model=model,
|
||||
messages=[
|
||||
{"role": "system", "content": _SYSTEM_PROMPT},
|
||||
{"role": "user", "content": user_msg},
|
||||
],
|
||||
temperature=0.3,
|
||||
max_tokens=1500,
|
||||
timeout=timeout or 120,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.info(
|
||||
"specify: API call failed for %s (%s) — skipping",
|
||||
task_id, exc,
|
||||
)
|
||||
return SpecifyOutcome(
|
||||
task_id, False, f"LLM error: {type(exc).__name__}"
|
||||
)
|
||||
|
||||
try:
|
||||
raw = resp.choices[0].message.content or ""
|
||||
except Exception:
|
||||
raw = ""
|
||||
|
||||
parsed = _extract_json_blob(raw)
|
||||
|
||||
new_title: Optional[str]
|
||||
new_body: Optional[str]
|
||||
if parsed is None:
|
||||
# Fall back: treat the whole reply as the body, leave title as-is.
|
||||
# Worst case the user edits afterward — still better than stranding
|
||||
# the task in triage on a malformed LLM reply.
|
||||
stripped_raw = raw.strip()
|
||||
if not stripped_raw:
|
||||
return SpecifyOutcome(
|
||||
task_id, False, "LLM returned an empty response"
|
||||
)
|
||||
new_title = None
|
||||
new_body = stripped_raw
|
||||
else:
|
||||
title_val = parsed.get("title")
|
||||
body_val = parsed.get("body")
|
||||
new_title = (
|
||||
title_val.strip()
|
||||
if isinstance(title_val, str) and title_val.strip()
|
||||
else None
|
||||
)
|
||||
new_body = (
|
||||
body_val if isinstance(body_val, str) and body_val.strip() else None
|
||||
)
|
||||
if new_body is None and new_title is None:
|
||||
return SpecifyOutcome(
|
||||
task_id, False, "LLM response missing title and body"
|
||||
)
|
||||
|
||||
with kb.connect() as conn:
|
||||
ok = kb.specify_triage_task(
|
||||
conn,
|
||||
task_id,
|
||||
title=new_title,
|
||||
body=new_body,
|
||||
author=author or _profile_author(),
|
||||
)
|
||||
if not ok:
|
||||
# Race: someone else promoted / archived the task between our
|
||||
# read above and the write. Report, don't crash.
|
||||
return SpecifyOutcome(
|
||||
task_id, False, "task moved out of triage before promotion"
|
||||
)
|
||||
return SpecifyOutcome(task_id, True, "specified", new_title=new_title)
|
||||
|
||||
|
||||
def list_triage_ids(*, tenant: Optional[str] = None) -> list[str]:
|
||||
"""Return task ids currently in the triage column.
|
||||
|
||||
``tenant`` narrows the sweep; ``None`` returns every triage task.
|
||||
"""
|
||||
with kb.connect() as conn:
|
||||
tasks = kb.list_tasks(
|
||||
conn,
|
||||
status="triage",
|
||||
tenant=tenant,
|
||||
include_archived=False,
|
||||
)
|
||||
return [t.id for t in tasks]
|
||||
+144
-34
@@ -43,6 +43,11 @@ Usage:
|
||||
hermes claw migrate --dry-run # Preview migration without changes
|
||||
"""
|
||||
|
||||
# IMPORTANT: hermes_bootstrap must be the very first import — it sets up
|
||||
# UTF-8 stdio on Windows so print()/subprocess children don't hit
|
||||
# UnicodeEncodeError with non-ASCII characters. No-op on POSIX.
|
||||
import hermes_bootstrap # noqa: F401
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
@@ -230,6 +235,7 @@ except Exception:
|
||||
pass # best-effort — don't crash if config isn't available yet
|
||||
|
||||
import logging
|
||||
import threading
|
||||
import time as _time
|
||||
from datetime import datetime
|
||||
|
||||
@@ -3534,7 +3540,7 @@ def _remove_custom_provider(config):
|
||||
clear_screen=False,
|
||||
title="Select provider to remove:",
|
||||
)
|
||||
idx: int | None = menu.show() # ty:ignore[invalid-assignment] - TerminalMenu.show() is always `int | None` when multi_select is False / not provided.
|
||||
idx = menu.show()
|
||||
from hermes_cli.curses_ui import flush_stdin
|
||||
|
||||
flush_stdin()
|
||||
@@ -3620,7 +3626,7 @@ def _model_flow_named_custom(config, provider_info):
|
||||
clear_screen=False,
|
||||
title=f"Select model from {name}:",
|
||||
)
|
||||
idx: int | None = menu.show() # ty:ignore[invalid-assignment] - TerminalMenu.show() is always `int | None` when multi_select is False / not provided.
|
||||
idx = menu.show()
|
||||
from hermes_cli.curses_ui import flush_stdin
|
||||
|
||||
flush_stdin()
|
||||
@@ -3796,7 +3802,7 @@ def _prompt_reasoning_effort_selection(efforts, current_effort=""):
|
||||
clear_screen=False,
|
||||
title="Select reasoning effort:",
|
||||
)
|
||||
idx: int | None = menu.show() # ty:ignore[invalid-assignment] - TerminalMenu.show() is always `int | None` when multi_select is False / not provided.
|
||||
idx = menu.show()
|
||||
from hermes_cli.curses_ui import flush_stdin
|
||||
|
||||
flush_stdin()
|
||||
@@ -6445,6 +6451,45 @@ def _load_installable_optional_extras() -> list[str]:
|
||||
return referenced
|
||||
|
||||
|
||||
def _run_install_with_heartbeat(
|
||||
cmd: list[str],
|
||||
*,
|
||||
env: dict[str, str] | None = None,
|
||||
heartbeat_interval_seconds: int = 30,
|
||||
) -> None:
|
||||
"""Run dependency install command with periodic heartbeat output.
|
||||
|
||||
Some resolvers/build backends (especially when compiling Rust/C extensions)
|
||||
can stay quiet for minutes. Emit a simple elapsed-time heartbeat so users
|
||||
know ``hermes update`` is still progressing even if pip/uv itself is silent.
|
||||
"""
|
||||
done = threading.Event()
|
||||
start = _time.time()
|
||||
|
||||
def _heartbeat() -> None:
|
||||
# Wait first, then print, so short installs don't emit noise.
|
||||
while not done.wait(heartbeat_interval_seconds):
|
||||
elapsed = int(_time.time() - start)
|
||||
print(
|
||||
f" … still installing dependencies ({elapsed}s elapsed)"
|
||||
" — compiling Rust/C extensions can take several minutes",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
t = threading.Thread(target=_heartbeat, daemon=True)
|
||||
t.start()
|
||||
try:
|
||||
subprocess.run(
|
||||
cmd,
|
||||
cwd=PROJECT_ROOT,
|
||||
check=True,
|
||||
env=env,
|
||||
)
|
||||
finally:
|
||||
done.set()
|
||||
t.join(timeout=0.2)
|
||||
|
||||
|
||||
def _install_python_dependencies_with_optional_fallback(
|
||||
install_cmd_prefix: list[str],
|
||||
*,
|
||||
@@ -6461,12 +6506,13 @@ def _install_python_dependencies_with_optional_fallback(
|
||||
Collecting/Building/Installing step), so keeping it visible costs
|
||||
nothing on fast hardware and prevents the "hermes update hangs" reports
|
||||
on slow hardware.
|
||||
|
||||
We also add periodic heartbeat lines in case the resolver/build backend is
|
||||
itself silent for long stretches.
|
||||
"""
|
||||
try:
|
||||
subprocess.run(
|
||||
_run_install_with_heartbeat(
|
||||
install_cmd_prefix + ["install", "-e", ".[all]"],
|
||||
cwd=PROJECT_ROOT,
|
||||
check=True,
|
||||
env=env,
|
||||
)
|
||||
return
|
||||
@@ -6475,10 +6521,8 @@ def _install_python_dependencies_with_optional_fallback(
|
||||
" ⚠ Optional extras failed, reinstalling base dependencies and retrying extras individually..."
|
||||
)
|
||||
|
||||
subprocess.run(
|
||||
_run_install_with_heartbeat(
|
||||
install_cmd_prefix + ["install", "-e", "."],
|
||||
cwd=PROJECT_ROOT,
|
||||
check=True,
|
||||
env=env,
|
||||
)
|
||||
|
||||
@@ -6486,10 +6530,8 @@ def _install_python_dependencies_with_optional_fallback(
|
||||
installed_extras: list[str] = []
|
||||
for extra in _load_installable_optional_extras():
|
||||
try:
|
||||
subprocess.run(
|
||||
_run_install_with_heartbeat(
|
||||
install_cmd_prefix + ["install", "-e", f".[{extra}]"],
|
||||
cwd=PROJECT_ROOT,
|
||||
check=True,
|
||||
env=env,
|
||||
)
|
||||
installed_extras.append(extra)
|
||||
@@ -7331,7 +7373,9 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
||||
for p in all_profiles:
|
||||
try:
|
||||
r = seed_profile_skills(p.path, quiet=True)
|
||||
if r:
|
||||
if r and r.get("skipped_opt_out"):
|
||||
status = "opted out (--no-skills)"
|
||||
elif r:
|
||||
copied = len(r.get("copied", []))
|
||||
updated = len(r.get("updated", []))
|
||||
modified = len(r.get("user_modified", []))
|
||||
@@ -7402,11 +7446,8 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
||||
.lower()
|
||||
)
|
||||
elif not (sys.stdin.isatty() and sys.stdout.isatty()):
|
||||
print(" ℹ Non-interactive session — skipping config migration prompt.")
|
||||
print(
|
||||
" Run 'hermes config migrate' later to apply any new config/env options."
|
||||
)
|
||||
response = "n"
|
||||
print(" ℹ Non-interactive session — applying safe config migrations.")
|
||||
response = "auto"
|
||||
else:
|
||||
try:
|
||||
response = (
|
||||
@@ -7417,19 +7458,22 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
||||
except EOFError:
|
||||
response = "n"
|
||||
|
||||
if response in ("", "y", "yes"):
|
||||
if response in ("", "y", "yes", "auto"):
|
||||
print()
|
||||
# In gateway mode OR under --yes, run auto-migrations only (no
|
||||
# input() prompts for API keys which would hang the detached
|
||||
# process / defeat the point of --yes).
|
||||
results = migrate_config(
|
||||
interactive=not (gateway_mode or assume_yes), quiet=False
|
||||
# Gateway mode, --yes, and non-interactive update contexts
|
||||
# (dashboard / web server actions) cannot prompt for API keys.
|
||||
# Still run the non-interactive migration pass before restarting
|
||||
# so new default config fields and version bumps are written
|
||||
# before the freshly updated gateway validates config at startup.
|
||||
interactive_migration = not (
|
||||
gateway_mode or assume_yes or response == "auto"
|
||||
)
|
||||
results = migrate_config(interactive=interactive_migration, quiet=False)
|
||||
|
||||
if results["env_added"] or results["config_added"]:
|
||||
print()
|
||||
print("✓ Configuration updated!")
|
||||
if (gateway_mode or assume_yes) and missing_env:
|
||||
if (gateway_mode or assume_yes or response == "auto") and missing_env:
|
||||
print(" ℹ API keys require manual entry: hermes config migrate")
|
||||
else:
|
||||
print()
|
||||
@@ -7582,7 +7626,7 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
||||
# systemd units without SIGUSR1 wiring this wait just times out
|
||||
# and we fall back to ``systemctl restart`` (the old behaviour).
|
||||
try:
|
||||
from gateway.restart import (
|
||||
from hermes_constants import (
|
||||
DEFAULT_GATEWAY_RESTART_DRAIN_TIMEOUT as _DEFAULT_DRAIN,
|
||||
)
|
||||
except Exception:
|
||||
@@ -7733,6 +7777,23 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
||||
# when the graceful path failed (unit missing
|
||||
# SIGUSR1 wiring, drain exceeded the budget,
|
||||
# restart-policy mismatch).
|
||||
#
|
||||
# Always `reset-failed` first. If systemd's own
|
||||
# auto-restart attempts already parked the unit
|
||||
# in a failed state (transient CHDIR / OOM /
|
||||
# filesystem race after our drain + exit-75),
|
||||
# a plain `systemctl restart` can wedge against
|
||||
# the RestartSec backoff and leave the unit
|
||||
# dead. Clearing the failed state first makes
|
||||
# the restart idempotent. Mirrors the recovery
|
||||
# path in `hermes gateway restart`
|
||||
# (`systemd_restart()`) as of PR #20949.
|
||||
subprocess.run(
|
||||
scope_cmd + ["reset-failed", svc_name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
restart = subprocess.run(
|
||||
scope_cmd + ["restart", svc_name],
|
||||
capture_output=True,
|
||||
@@ -7752,10 +7813,19 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
||||
else:
|
||||
# Retry once — transient startup failures
|
||||
# (stale module cache, import race) often
|
||||
# resolve on the second attempt.
|
||||
# resolve on the second attempt. Again
|
||||
# clear any failed state first so the
|
||||
# retry isn't blocked by the previous
|
||||
# crash.
|
||||
print(
|
||||
f" ⚠ {svc_name} died after restart, retrying..."
|
||||
)
|
||||
subprocess.run(
|
||||
scope_cmd + ["reset-failed", svc_name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
subprocess.run(
|
||||
scope_cmd + ["restart", svc_name],
|
||||
capture_output=True,
|
||||
@@ -7770,10 +7840,13 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
||||
restarted_services.append(svc_name)
|
||||
print(f" ✓ {svc_name} recovered on retry")
|
||||
else:
|
||||
_scope_flag = "--user " if scope == "user" else ""
|
||||
print(
|
||||
f" ✗ {svc_name} failed to stay running after restart.\n"
|
||||
f" Check logs: journalctl --user -u {svc_name} --since '2 min ago'\n"
|
||||
f" Restart manually: systemctl {'--user ' if scope == 'user' else ''}restart {svc_name}"
|
||||
f" Check logs: journalctl {_scope_flag}-u {svc_name} --since '2 min ago'\n"
|
||||
f" Recover manually:\n"
|
||||
f" systemctl {_scope_flag}reset-failed {svc_name}\n"
|
||||
f" systemctl {_scope_flag}restart {svc_name}"
|
||||
)
|
||||
else:
|
||||
print(
|
||||
@@ -7897,10 +7970,15 @@ def _cmd_update_impl(args, gateway_mode: bool):
|
||||
print(
|
||||
f" ⚠ {len(_stuck)} gateway process(es) ignored SIGTERM — force-killing"
|
||||
)
|
||||
from gateway.status import terminate_pid as _terminate_pid
|
||||
for pid in _stuck:
|
||||
try:
|
||||
os.kill(pid, _signal.SIGKILL)
|
||||
except (ProcessLookupError, PermissionError):
|
||||
# Routes through taskkill /T /F on Windows,
|
||||
# SIGKILL on POSIX — _signal.SIGKILL doesn't
|
||||
# exist on Windows so the old raw os.kill call
|
||||
# used to crash the entire update path.
|
||||
_terminate_pid(pid, force=True)
|
||||
except (ProcessLookupError, PermissionError, OSError):
|
||||
pass
|
||||
# Give the OS a beat to reap the processes so the
|
||||
# watchers see them exit and respawn.
|
||||
@@ -8124,6 +8202,7 @@ def cmd_profile(args):
|
||||
clone = getattr(args, "clone", False)
|
||||
clone_all = getattr(args, "clone_all", False)
|
||||
no_alias = getattr(args, "no_alias", False)
|
||||
no_skills = getattr(args, "no_skills", False)
|
||||
|
||||
try:
|
||||
clone_from = getattr(args, "clone_from", None)
|
||||
@@ -8134,6 +8213,7 @@ def cmd_profile(args):
|
||||
clone_all=clone_all,
|
||||
clone_config=clone,
|
||||
no_alias=no_alias,
|
||||
no_skills=no_skills,
|
||||
)
|
||||
print(f"\nProfile '{name}' created at {profile_dir}")
|
||||
|
||||
@@ -8158,10 +8238,17 @@ def cmd_profile(args):
|
||||
except Exception:
|
||||
pass # Honcho plugin not installed or not configured
|
||||
|
||||
# Seed bundled skills (skip if --clone-all already copied them)
|
||||
# Seed bundled skills (skip if --clone-all already copied them, or
|
||||
# if --no-skills was passed — in which case seed_profile_skills()
|
||||
# honors the marker file and returns skipped_opt_out=True).
|
||||
if not clone_all:
|
||||
result = seed_profile_skills(profile_dir)
|
||||
if result:
|
||||
if result and result.get("skipped_opt_out"):
|
||||
print(
|
||||
"No bundled skills seeded (--no-skills). "
|
||||
"Delete .no-bundled-skills in the profile to opt back in."
|
||||
)
|
||||
elif result:
|
||||
copied = len(result.get("copied", []))
|
||||
print(f"{copied} bundled skills synced.")
|
||||
else:
|
||||
@@ -8477,6 +8564,13 @@ def _build_provider_choices() -> list[str]:
|
||||
|
||||
def main():
|
||||
"""Main entry point for hermes CLI."""
|
||||
# Force UTF-8 stdio on Windows before anything prints. No-op elsewhere.
|
||||
try:
|
||||
from hermes_cli.stdio import configure_windows_stdio
|
||||
configure_windows_stdio()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
from hermes_cli._parser import build_top_level_parser
|
||||
|
||||
parser, subparsers, chat_parser = build_top_level_parser()
|
||||
@@ -8679,6 +8773,9 @@ def main():
|
||||
help="Target the Linux system-level gateway service",
|
||||
)
|
||||
|
||||
# gateway list
|
||||
gateway_subparsers.add_parser("list", help="List all profiles and their gateway status")
|
||||
|
||||
# gateway setup
|
||||
gateway_subparsers.add_parser("setup", help="Configure messaging platforms")
|
||||
|
||||
@@ -9996,7 +10093,15 @@ Examples:
|
||||
)
|
||||
mcp_add_p.add_argument("name", help="Server name (used as config key)")
|
||||
mcp_add_p.add_argument("--url", help="HTTP/SSE endpoint URL")
|
||||
mcp_add_p.add_argument("--command", help="Stdio command (e.g. npx)")
|
||||
# dest="mcp_command" so this flag does not clobber the top-level
|
||||
# subparser's args.command attribute, which the dispatcher reads to
|
||||
# route to cmd_mcp. Without an explicit dest, argparse derives
|
||||
# dest="command" from the flag name and sets it to None when the
|
||||
# flag is omitted, causing `hermes mcp add ...` to fall through to
|
||||
# interactive chat.
|
||||
mcp_add_p.add_argument(
|
||||
"--command", dest="mcp_command", help="Stdio command (e.g. npx)"
|
||||
)
|
||||
mcp_add_p.add_argument(
|
||||
"--args", nargs="*", default=[], help="Arguments for stdio command"
|
||||
)
|
||||
@@ -10523,6 +10628,11 @@ Examples:
|
||||
profile_create.add_argument(
|
||||
"--no-alias", action="store_true", help="Skip wrapper script creation"
|
||||
)
|
||||
profile_create.add_argument(
|
||||
"--no-skills",
|
||||
action="store_true",
|
||||
help="Create an empty profile with no bundled skills (opts out of `hermes update` skill sync)",
|
||||
)
|
||||
|
||||
profile_delete = profile_subparsers.add_parser("delete", help="Delete a profile")
|
||||
profile_delete.add_argument("profile_name", help="Profile to delete")
|
||||
|
||||
@@ -221,7 +221,10 @@ def cmd_mcp_add(args):
|
||||
"""Add a new MCP server with discovery-first tool selection."""
|
||||
name = args.name
|
||||
url = getattr(args, "url", None)
|
||||
command = getattr(args, "command", None)
|
||||
# Read from `mcp_command` (set by --command via explicit dest) — see
|
||||
# mcp_add_p.add_argument("--command", dest="mcp_command", ...) in
|
||||
# hermes_cli/main.py for why the dest is renamed.
|
||||
command = getattr(args, "mcp_command", None)
|
||||
cmd_args = getattr(args, "args", None) or []
|
||||
auth_type = getattr(args, "auth", None)
|
||||
preset_name = getattr(args, "preset", None)
|
||||
|
||||
@@ -69,7 +69,7 @@ def _install_dependencies(provider_name: str) -> None:
|
||||
|
||||
try:
|
||||
import yaml
|
||||
with open(yaml_path) as f:
|
||||
with open(yaml_path, encoding="utf-8") as f:
|
||||
meta = yaml.safe_load(f) or {}
|
||||
except Exception:
|
||||
return
|
||||
@@ -377,7 +377,7 @@ def _write_env_vars(env_path: Path, env_writes: dict) -> None:
|
||||
if key not in updated_keys:
|
||||
new_lines.append(f"{key}={val}")
|
||||
|
||||
env_path.write_text("\n".join(new_lines) + "\n")
|
||||
env_path.write_text("\n".join(new_lines) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -173,7 +173,7 @@ def _read_disk_cache() -> tuple[dict[str, Any] | None, float]:
|
||||
except (OSError, FileNotFoundError):
|
||||
return (None, 0.0)
|
||||
try:
|
||||
with open(path) as fh:
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
data = json.load(fh)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return (None, 0.0)
|
||||
@@ -187,7 +187,7 @@ def _write_disk_cache(data: dict[str, Any]) -> None:
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||
with open(tmp, "w") as fh:
|
||||
with open(tmp, "w", encoding="utf-8") as fh:
|
||||
json.dump(data, fh, indent=2)
|
||||
fh.write("\n")
|
||||
atomic_replace(tmp, path)
|
||||
|
||||
@@ -1637,7 +1637,8 @@ def list_authenticated_providers(
|
||||
groups[group_key]["models"].append(m)
|
||||
|
||||
_section4_emitted_slugs: set = set()
|
||||
for grp in groups.values():
|
||||
for grp_key, grp in groups.items():
|
||||
api_url, api_key = grp_key
|
||||
slug = grp["slug"]
|
||||
# If the slug is already claimed by a built-in / overlay /
|
||||
# user-provider row (sections 1-3), skip this custom group
|
||||
@@ -1675,6 +1676,18 @@ def list_authenticated_providers(
|
||||
_grp_url_norm = _pair_key[1]
|
||||
if _grp_url_norm and _grp_url_norm in _builtin_endpoints:
|
||||
continue
|
||||
# Live model discovery from custom provider endpoints (matches
|
||||
# Section 3 behavior for user ``providers:`` entries).
|
||||
if api_url and api_key:
|
||||
try:
|
||||
from hermes_cli.models import fetch_api_models
|
||||
|
||||
live_models = fetch_api_models(api_key, api_url)
|
||||
if live_models:
|
||||
grp["models"] = live_models
|
||||
grp["total_models"] = len(live_models)
|
||||
except Exception:
|
||||
pass
|
||||
results.append({
|
||||
"slug": slug,
|
||||
"name": grp["name"],
|
||||
|
||||
@@ -46,6 +46,7 @@ OPENROUTER_MODELS: list[tuple[str, str]] = [
|
||||
("xiaomi/mimo-v2.5-pro", ""),
|
||||
("xiaomi/mimo-v2.5", ""),
|
||||
("tencent/hy3-preview:free", "free"),
|
||||
("tencent/hy3-preview", ""),
|
||||
("openai/gpt-5.3-codex", ""),
|
||||
("google/gemini-3-pro-image-preview", ""),
|
||||
("google/gemini-3-flash-preview", ""),
|
||||
@@ -416,6 +417,18 @@ _PROVIDER_MODELS: dict[str, list[str]] = {
|
||||
"glm-4.7",
|
||||
"MiniMax-M2.5",
|
||||
],
|
||||
# Alibaba Coding Plan — same platform as alibaba (DashScope coding-intl),
|
||||
# separate provider ID with its own base_url_env_var.
|
||||
"alibaba-coding-plan": [
|
||||
"qwen3.6-plus",
|
||||
"qwen3.5-plus",
|
||||
"qwen3-coder-plus",
|
||||
"qwen3-coder-next",
|
||||
"kimi-k2.5",
|
||||
"glm-5",
|
||||
"glm-4.7",
|
||||
"MiniMax-M2.5",
|
||||
],
|
||||
# Curated HF model list — only agentic models that map to OpenRouter defaults.
|
||||
"huggingface": [
|
||||
"moonshotai/Kimi-K2.5",
|
||||
|
||||
@@ -174,7 +174,7 @@ def run_oneshot(
|
||||
# Redirect stderr AND stdout to devnull for the entire call tree.
|
||||
# We'll print the final response to the real stdout at the end.
|
||||
real_stdout = sys.stdout
|
||||
devnull = open(os.devnull, "w")
|
||||
devnull = open(os.devnull, "w", encoding="utf-8")
|
||||
|
||||
try:
|
||||
with redirect_stdout(devnull), redirect_stderr(devnull):
|
||||
|
||||
@@ -73,6 +73,24 @@ def _cmd_approve(store, platform: str, code: str):
|
||||
display = f"{name} ({uid})" if name else uid
|
||||
print(f"\n Approved! User {display} on {platform} can now use the bot~")
|
||||
print(" They'll be recognized automatically on their next message.\n")
|
||||
elif store._is_locked_out(platform):
|
||||
# Disambiguate: approve_code returns None for both invalid codes
|
||||
# and lockout. Tell the operator it's lockout so they don't chase
|
||||
# a "wrong code" rabbit hole (#10195).
|
||||
import time as _time
|
||||
limits = store._load_json(store._rate_limit_path())
|
||||
lockout_until = limits.get(f"_lockout:{platform}", 0)
|
||||
remaining = max(0, int(lockout_until - _time.time()))
|
||||
mins = remaining // 60
|
||||
print(
|
||||
f"\n Platform '{platform}' is locked out after too many failed "
|
||||
f"approval attempts."
|
||||
)
|
||||
print(f" Lockout clears in ~{mins} minute(s).")
|
||||
print(
|
||||
" To reset sooner, delete the '_lockout:{0}' entry from "
|
||||
"~/.hermes/platforms/pairing/_rate_limits.json\n".format(platform)
|
||||
)
|
||||
else:
|
||||
print(f"\n Code '{code}' not found or expired for platform '{platform}'.")
|
||||
print(" Run 'hermes pairing list' to see pending codes.\n")
|
||||
|
||||
@@ -80,6 +80,10 @@ VALID_HOOKS: Set[str] = {
|
||||
"post_tool_call",
|
||||
"transform_terminal_output",
|
||||
"transform_tool_result",
|
||||
# Transform LLM output before it's returned to the user.
|
||||
# Plugins return a string to replace the response text, or None/empty to leave unchanged.
|
||||
# First non-None string wins. Useful for vocabulary/personality transformation.
|
||||
"transform_llm_output",
|
||||
"pre_llm_call",
|
||||
"post_llm_call",
|
||||
"pre_api_request",
|
||||
@@ -866,7 +870,7 @@ class PluginManager:
|
||||
if yaml is None:
|
||||
logger.warning("PyYAML not installed – cannot load %s", manifest_file)
|
||||
return None
|
||||
data = yaml.safe_load(manifest_file.read_text()) or {}
|
||||
data = yaml.safe_load(manifest_file.read_text(encoding="utf-8")) or {}
|
||||
|
||||
name = data.get("name", plugin_dir.name)
|
||||
key = f"{prefix}/{plugin_dir.name}" if prefix else name
|
||||
|
||||
@@ -127,7 +127,7 @@ def _read_manifest(plugin_dir: Path) -> dict:
|
||||
try:
|
||||
import yaml
|
||||
|
||||
with open(manifest_file) as f:
|
||||
with open(manifest_file, encoding="utf-8") as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
except Exception as e:
|
||||
logger.warning("Failed to read plugin.yaml in %s: %s", plugin_dir, e)
|
||||
@@ -703,7 +703,7 @@ def _discover_all_plugins() -> list:
|
||||
description = ""
|
||||
if yaml:
|
||||
try:
|
||||
with open(manifest_file) as f:
|
||||
with open(manifest_file, encoding="utf-8") as f:
|
||||
manifest = yaml.safe_load(f) or {}
|
||||
name = manifest.get("name", d.name)
|
||||
version = manifest.get("version", "")
|
||||
|
||||
+65
-6
@@ -71,6 +71,22 @@ _CLONE_ALL_STRIP = [
|
||||
"processes.json",
|
||||
]
|
||||
|
||||
# Marker file written by `hermes profile create --no-skills`. When present in
|
||||
# a profile's root, callers of seed_profile_skills() (fresh-create, `hermes
|
||||
# update`'s all-profile sync, the web dashboard) skip bundled-skill seeding
|
||||
# for that profile. The user can still install skills manually via
|
||||
# `hermes skills install` or drop SKILL.md files into the profile's skills/.
|
||||
# Delete the marker file to opt back in.
|
||||
NO_BUNDLED_SKILLS_MARKER = ".no-bundled-skills"
|
||||
|
||||
|
||||
def has_bundled_skills_opt_out(profile_dir: Path) -> bool:
|
||||
"""Return True if the profile opted out of bundled-skill seeding."""
|
||||
try:
|
||||
return (profile_dir / NO_BUNDLED_SKILLS_MARKER).exists()
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def _clone_all_copytree_ignore(source_dir: Path):
|
||||
"""Ignore ``profiles/`` at the root of *source_dir* only.
|
||||
@@ -338,7 +354,7 @@ def _read_config_model(profile_dir: Path) -> tuple:
|
||||
return None, None
|
||||
try:
|
||||
import yaml
|
||||
with open(config_path, "r") as f:
|
||||
with open(config_path, "r", encoding="utf-8") as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
model_cfg = cfg.get("model", {})
|
||||
if isinstance(model_cfg, str):
|
||||
@@ -427,6 +443,7 @@ def create_profile(
|
||||
clone_all: bool = False,
|
||||
clone_config: bool = False,
|
||||
no_alias: bool = False,
|
||||
no_skills: bool = False,
|
||||
) -> Path:
|
||||
"""Create a new profile directory.
|
||||
|
||||
@@ -444,12 +461,22 @@ def create_profile(
|
||||
skills, and selected profile identity files from the source profile.
|
||||
no_alias:
|
||||
If True, skip wrapper script creation.
|
||||
no_skills:
|
||||
If True, create an empty profile with no bundled skills, and write
|
||||
a marker file so ``hermes update`` skips re-seeding this profile's
|
||||
skills. Mutually exclusive with ``clone_config``/``clone_all`` (those
|
||||
explicitly copy skills from the source).
|
||||
|
||||
Returns
|
||||
-------
|
||||
Path
|
||||
The newly created profile directory.
|
||||
"""
|
||||
if no_skills and (clone_config or clone_all):
|
||||
raise ValueError(
|
||||
"--no-skills is mutually exclusive with --clone / --clone-all "
|
||||
"(cloning explicitly copies skills from the source profile)."
|
||||
)
|
||||
canon = normalize_profile_name(name)
|
||||
validate_profile_name(canon)
|
||||
|
||||
@@ -527,6 +554,19 @@ def create_profile(
|
||||
except Exception:
|
||||
pass # best-effort — don't fail profile creation over this
|
||||
|
||||
# Write the opt-out marker so seed_profile_skills() and `hermes update`'s
|
||||
# all-profile sync loop both skip this profile for bundled-skill seeding.
|
||||
if no_skills:
|
||||
try:
|
||||
(profile_dir / NO_BUNDLED_SKILLS_MARKER).write_text(
|
||||
"This profile opted out of bundled-skill seeding "
|
||||
"(`hermes profile create --no-skills`).\n"
|
||||
"Delete this file to re-enable sync on the next `hermes update`.\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
except OSError:
|
||||
pass # best-effort — the feature still works via the empty skills/ dir
|
||||
|
||||
return profile_dir
|
||||
|
||||
|
||||
@@ -535,7 +575,19 @@ def seed_profile_skills(profile_dir: Path, quiet: bool = False) -> Optional[dict
|
||||
|
||||
Uses subprocess because sync_skills() caches HERMES_HOME at module level.
|
||||
Returns the sync result dict, or None on failure.
|
||||
|
||||
Profiles that opted out of bundled skills (via ``hermes profile create
|
||||
--no-skills`` — which writes ``.no-bundled-skills`` to the profile root)
|
||||
are skipped and get an empty-result dict so callers can report
|
||||
"opted out" instead of "failed".
|
||||
"""
|
||||
if has_bundled_skills_opt_out(profile_dir):
|
||||
return {
|
||||
"copied": [],
|
||||
"updated": [],
|
||||
"user_modified": [],
|
||||
"skipped_opt_out": True,
|
||||
}
|
||||
project_root = Path(__file__).parent.parent.resolve()
|
||||
try:
|
||||
result = subprocess.run(
|
||||
@@ -706,7 +758,6 @@ def _cleanup_gateway_service(name: str, profile_dir: Path) -> None:
|
||||
|
||||
def _stop_gateway_process(profile_dir: Path) -> None:
|
||||
"""Stop a running gateway process via its PID file."""
|
||||
import signal as _signal
|
||||
import time as _time
|
||||
|
||||
pid_file = profile_dir / "gateway.pid"
|
||||
@@ -717,19 +768,27 @@ def _stop_gateway_process(profile_dir: Path) -> None:
|
||||
raw = pid_file.read_text().strip()
|
||||
data = json.loads(raw) if raw.startswith("{") else {"pid": int(raw)}
|
||||
pid = int(data["pid"])
|
||||
os.kill(pid, _signal.SIGTERM)
|
||||
# Route through terminate_pid so Windows uses the appropriate
|
||||
# primitive (taskkill / TerminateProcess) — raw os.kill with
|
||||
# _signal.SIGKILL raises AttributeError at import time on Windows,
|
||||
# and raw os.kill with SIGTERM doesn't cascade to child processes
|
||||
# the same way taskkill /T does.
|
||||
from gateway.status import terminate_pid as _terminate_pid
|
||||
_terminate_pid(pid) # graceful first
|
||||
# Wait up to 10s for graceful shutdown
|
||||
for _ in range(20):
|
||||
_time.sleep(0.5)
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except ProcessLookupError:
|
||||
except (ProcessLookupError, OSError):
|
||||
# OSError covers Windows' WinError 87 "invalid parameter"
|
||||
# returned for an invalid/gone PID probe.
|
||||
print(f"✓ Gateway stopped (PID {pid})")
|
||||
return
|
||||
# Force kill
|
||||
try:
|
||||
os.kill(pid, _signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
_terminate_pid(pid, force=True)
|
||||
except (ProcessLookupError, OSError):
|
||||
pass
|
||||
print(f"✓ Gateway force-stopped (PID {pid})")
|
||||
except (ProcessLookupError, PermissionError):
|
||||
|
||||
@@ -7,11 +7,14 @@ keystrokes can be fed back in. The only caller today is the
|
||||
|
||||
Design constraints:
|
||||
|
||||
* **POSIX-only.** Hermes Agent supports Windows exclusively via WSL, which
|
||||
exposes a native POSIX PTY via ``openpty(3)``. Native Windows Python
|
||||
has no PTY; :class:`PtyUnavailableError` is raised with a user-readable
|
||||
install/platform message so the dashboard can render a banner instead of
|
||||
crashing.
|
||||
* **POSIX-only.** This module depends on ``fcntl``, ``termios``, and
|
||||
``ptyprocess``, none of which exist on native Windows Python. Native
|
||||
Windows ConPTY is a different API (Windows 10 build 17763+) and would
|
||||
need a separate Windows implementation (``pywinpty``) — that's tracked
|
||||
as a future enhancement. On native Windows, importing this module
|
||||
raises :class:`ImportError` and the dashboard's ``/chat`` tab shows a
|
||||
WSL-recommended banner instead of crashing. Every other feature in the
|
||||
dashboard (sessions, jobs, metrics, config editor) works natively.
|
||||
* **Zero Node dependency on the server side.** We use :mod:`ptyprocess`,
|
||||
which is a pure-Python wrapper around the OS calls. The browser talks
|
||||
to the same ``hermes --tui`` binary it would launch from the CLI, so
|
||||
|
||||
+60
-4
@@ -84,18 +84,34 @@ def resolve_hermes_bin() -> Optional[str]:
|
||||
1. ``sys.argv[0]`` if it resolves to a real executable.
|
||||
2. ``shutil.which("hermes")`` on PATH.
|
||||
3. ``None`` → caller should fall back to ``python -m hermes_cli.main``.
|
||||
|
||||
Windows note: ``os.access(path, os.X_OK)`` returns True for ``.py`` and
|
||||
``.pyc`` files on Windows (the OS treats anything listed in PATHEXT as
|
||||
executable, and Python files are often registered there). But
|
||||
``subprocess.run([script.py, ...])`` can't actually execute a .py
|
||||
directly — CreateProcessW needs a real .exe, not a script associated
|
||||
with the Python launcher. On Windows we therefore skip the argv[0]
|
||||
fast-path when it points at a .py file and fall through to either
|
||||
``hermes.exe`` on PATH or the ``sys.executable -m hermes_cli.main``
|
||||
fallback.
|
||||
"""
|
||||
argv0 = sys.argv[0]
|
||||
_is_windows = sys.platform == "win32"
|
||||
|
||||
def _is_python_script(p: str) -> bool:
|
||||
return p.lower().endswith((".py", ".pyc"))
|
||||
|
||||
# Absolute path to an executable (covers nix store, venv wrappers, etc.)
|
||||
if os.path.isabs(argv0) and os.path.isfile(argv0) and os.access(argv0, os.X_OK):
|
||||
return argv0
|
||||
if not (_is_windows and _is_python_script(argv0)):
|
||||
return argv0
|
||||
|
||||
# Relative path — resolve against CWD
|
||||
if not argv0.startswith("-") and os.path.isfile(argv0):
|
||||
abs_path = os.path.abspath(argv0)
|
||||
if os.access(abs_path, os.X_OK):
|
||||
return abs_path
|
||||
if not (_is_windows and _is_python_script(abs_path)):
|
||||
return abs_path
|
||||
|
||||
# PATH lookup
|
||||
path_bin = shutil.which("hermes")
|
||||
@@ -142,8 +158,48 @@ def relaunch(
|
||||
preserve_inherited: bool = True,
|
||||
original_argv: Optional[Sequence[str]] = None,
|
||||
) -> None:
|
||||
"""Replace the current process with a fresh hermes invocation."""
|
||||
"""Replace the current process with a fresh hermes invocation.
|
||||
|
||||
On POSIX we use ``os.execvp`` which replaces the running process with
|
||||
the new one in place — same PID, no double-fork. That's what the
|
||||
relaunch contract wants: "run hermes again as if the user had typed
|
||||
the new argv".
|
||||
|
||||
Windows has no native exec semantics — ``os.execvp`` on Windows
|
||||
*emulates* exec by spawning the child and exiting the parent, but
|
||||
only works when the target is a real Win32 executable. Our target
|
||||
is usually ``hermes.exe`` (a Python console-script shim that wraps
|
||||
``python -m hermes_cli.main``) or a ``.cmd`` batch file, and both
|
||||
raise ``OSError(8, "Exec format error")`` on Windows' execvp.
|
||||
|
||||
The Windows-correct pattern is: spawn the child with ``subprocess.run``
|
||||
(which routes through ``cmd.exe`` via ``shell=False`` + PATHEXT resolution),
|
||||
wait for it to exit, then propagate its exit code via ``sys.exit``.
|
||||
That's functionally equivalent — the user sees "hermes exited, then
|
||||
new hermes started" — just with two PIDs in play instead of one.
|
||||
"""
|
||||
new_argv = build_relaunch_argv(
|
||||
extra_args, preserve_inherited=preserve_inherited, original_argv=original_argv
|
||||
)
|
||||
os.execvp(new_argv[0], new_argv)
|
||||
if sys.platform == "win32":
|
||||
# Windows: subprocess + exit, because execvp can't swap to .cmd/.exe shims.
|
||||
import subprocess
|
||||
try:
|
||||
result = subprocess.run(new_argv)
|
||||
sys.exit(result.returncode)
|
||||
except KeyboardInterrupt:
|
||||
sys.exit(130)
|
||||
except OSError as exc:
|
||||
# Surface a helpful error rather than the raw OSError — the
|
||||
# caller used to see ``[Errno 8] Exec format error`` which is
|
||||
# cryptic. Common causes: ``hermes`` not on PATH yet (install
|
||||
# hasn't propagated User PATH into this shell) or a stale shim.
|
||||
print(
|
||||
f"\nHermes relaunch failed: {exc}\n"
|
||||
f"Command: {' '.join(new_argv)}\n"
|
||||
f"Fix: open a new terminal so PATH picks up, then re-run hermes.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
else:
|
||||
os.execvp(new_argv[0], new_argv)
|
||||
@@ -319,9 +319,10 @@ def _try_resolve_from_custom_pool(
|
||||
base_url: str,
|
||||
provider_label: str,
|
||||
api_mode_override: Optional[str] = None,
|
||||
provider_name: Optional[str] = None,
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""Check if a credential pool exists for a custom endpoint and return a runtime dict if so."""
|
||||
pool_key = get_custom_provider_pool_key(base_url)
|
||||
pool_key = get_custom_provider_pool_key(base_url, provider_name=provider_name)
|
||||
if not pool_key:
|
||||
return None
|
||||
try:
|
||||
@@ -521,7 +522,7 @@ def _resolve_named_custom_runtime(
|
||||
return None
|
||||
|
||||
# Check if a credential pool exists for this custom endpoint
|
||||
pool_result = _try_resolve_from_custom_pool(base_url, "custom", custom_provider.get("api_mode"))
|
||||
pool_result = _try_resolve_from_custom_pool(base_url, "custom", custom_provider.get("api_mode"), provider_name=custom_provider.get("name"))
|
||||
if pool_result:
|
||||
# Propagate the model name even when using pooled credentials —
|
||||
# the pool doesn't know about the custom_providers model field.
|
||||
@@ -640,8 +641,11 @@ def _resolve_openrouter_runtime(
|
||||
|
||||
# For custom endpoints, check if a credential pool exists
|
||||
if effective_provider == "custom" and base_url:
|
||||
# Pass requested_provider so pool lookup prefers name match over base_url,
|
||||
# fixing credential mix-ups when multiple custom providers share a base_url.
|
||||
pool_result = _try_resolve_from_custom_pool(
|
||||
base_url, effective_provider, _parse_api_mode(model_cfg.get("api_mode")),
|
||||
provider_name=requested_provider if requested_norm != "custom" else None,
|
||||
)
|
||||
if pool_result:
|
||||
return pool_result
|
||||
|
||||
+22
-2
@@ -2462,6 +2462,9 @@ def setup_gateway(config: dict):
|
||||
launchd_start,
|
||||
launchd_restart,
|
||||
UserSystemdUnavailableError,
|
||||
SystemScopeRequiresRootError,
|
||||
_system_scope_wizard_would_need_root,
|
||||
_print_system_scope_remediation,
|
||||
)
|
||||
|
||||
service_installed = _is_service_installed()
|
||||
@@ -2479,7 +2482,9 @@ def setup_gateway(config: dict):
|
||||
print()
|
||||
|
||||
if service_running:
|
||||
if prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
||||
if supports_systemd and _system_scope_wizard_would_need_root():
|
||||
_print_system_scope_remediation("restart")
|
||||
elif prompt_yes_no(" Restart the gateway to pick up changes?", True):
|
||||
try:
|
||||
if supports_systemd:
|
||||
systemd_restart()
|
||||
@@ -2489,10 +2494,19 @@ def setup_gateway(config: dict):
|
||||
print_error(" Restart failed — user systemd not reachable:")
|
||||
for line in str(e).splitlines():
|
||||
print(f" {line}")
|
||||
except SystemScopeRequiresRootError as e:
|
||||
# Defense in depth: the pre-check above should have
|
||||
# caught this, but a race (unit file appearing mid-run)
|
||||
# could still land here. Previously this exited the
|
||||
# whole wizard via sys.exit(1).
|
||||
print_error(f" Restart failed: {e}")
|
||||
_print_system_scope_remediation("restart")
|
||||
except Exception as e:
|
||||
print_error(f" Restart failed: {e}")
|
||||
elif service_installed:
|
||||
if prompt_yes_no(" Start the gateway service?", True):
|
||||
if supports_systemd and _system_scope_wizard_would_need_root():
|
||||
_print_system_scope_remediation("start")
|
||||
elif prompt_yes_no(" Start the gateway service?", True):
|
||||
try:
|
||||
if supports_systemd:
|
||||
systemd_start()
|
||||
@@ -2502,6 +2516,9 @@ def setup_gateway(config: dict):
|
||||
print_error(" Start failed — user systemd not reachable:")
|
||||
for line in str(e).splitlines():
|
||||
print(f" {line}")
|
||||
except SystemScopeRequiresRootError as e:
|
||||
print_error(f" Start failed: {e}")
|
||||
_print_system_scope_remediation("start")
|
||||
except Exception as e:
|
||||
print_error(f" Start failed: {e}")
|
||||
elif supports_service_manager:
|
||||
@@ -2529,6 +2546,9 @@ def setup_gateway(config: dict):
|
||||
print_error(" Start failed — user systemd not reachable:")
|
||||
for line in str(e).splitlines():
|
||||
print(f" {line}")
|
||||
except SystemScopeRequiresRootError as e:
|
||||
print_error(f" Start failed: {e}")
|
||||
_print_system_scope_remediation("start")
|
||||
except Exception as e:
|
||||
print_error(f" Start failed: {e}")
|
||||
except Exception as e:
|
||||
|
||||
@@ -1257,7 +1257,7 @@ def do_snapshot_export(output_path: str, console: Optional[Console] = None) -> N
|
||||
sys.stdout.write(payload)
|
||||
else:
|
||||
out = Path(output_path)
|
||||
out.write_text(payload)
|
||||
out.write_text(payload, encoding="utf-8")
|
||||
c.print(f"[bold green]Snapshot exported:[/] {out}")
|
||||
c.print(f"[dim]{len(installed)} skill(s), {len(tap_list)} tap(s)[/]\n")
|
||||
|
||||
@@ -1274,7 +1274,7 @@ def do_snapshot_import(input_path: str, force: bool = False,
|
||||
return
|
||||
|
||||
try:
|
||||
snapshot = json.loads(inp.read_text())
|
||||
snapshot = json.loads(inp.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError:
|
||||
c.print(f"[bold red]Error:[/] Invalid JSON in {inp}\n")
|
||||
return
|
||||
|
||||
@@ -0,0 +1,252 @@
|
||||
"""Windows-safe stdio configuration.
|
||||
|
||||
On Windows, Python's ``sys.stdout``/``sys.stderr`` default to the console's
|
||||
active code page (often ``cp1252``, sometimes ``cp437``, occasionally ``cp932``
|
||||
on Japanese locales, etc.). Hermes's banners, tool output feed, and slash
|
||||
command listings all contain Unicode: box-drawing characters (``─┌┐└┘├┤``),
|
||||
mathematical and geometric symbols (``◆ ◇ ◎ ▣ ⚔ ⚖ →``), and user-supplied
|
||||
text in any language. Printing those to a cp1252 console raises
|
||||
``UnicodeEncodeError: 'charmap' codec can't encode character…`` and kills the
|
||||
whole CLI before the REPL even opens.
|
||||
|
||||
The fix is to force UTF-8 on the Python side and also flip the console's
|
||||
code page to UTF-8 (65001). Both matter: Python-level only helps when
|
||||
Python's stdout is a real TTY; code-page flipping lets subprocesses and
|
||||
child Python ``print()`` calls agree on encoding.
|
||||
|
||||
This module is a no-op on every non-Windows platform, and idempotent.
|
||||
Entry points (``cli.py`` ``main``, ``hermes_cli/main.py`` CLI dispatch,
|
||||
``gateway/run.py`` startup) call :func:`configure_windows_stdio` exactly
|
||||
once early in startup.
|
||||
|
||||
Patterns cribbed from Claude Code (``src/utils/platform.ts``), OpenCode
|
||||
(``packages/opencode/src/pty/index.ts`` env injection), and OpenAI Codex
|
||||
(``codex-rs/core/src/unified_exec/process_manager.rs``). None of those
|
||||
actually flip the console code page — they rely on their runtime (Node or
|
||||
Rust) writing UTF-16 to the Win32 console API and letting the terminal
|
||||
sort it out. Python doesn't get that luxury.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
__all__ = ["configure_windows_stdio", "is_windows"]
|
||||
|
||||
|
||||
_CONFIGURED = False
|
||||
|
||||
|
||||
def is_windows() -> bool:
|
||||
"""Return True iff running on native Windows (not WSL)."""
|
||||
return sys.platform == "win32"
|
||||
|
||||
|
||||
def _flip_console_code_page_to_utf8() -> None:
|
||||
"""Set the attached console's input and output code pages to UTF-8.
|
||||
|
||||
Uses ``SetConsoleCP`` / ``SetConsoleOutputCP`` via ``ctypes``. Failure
|
||||
is silent — if there's no attached console (e.g. Hermes is running
|
||||
behind a redirected stdout, under a service, or inside a PTY-less CI
|
||||
runner) these calls simply return 0 and we move on.
|
||||
|
||||
CP_UTF8 is 65001.
|
||||
"""
|
||||
try:
|
||||
import ctypes
|
||||
|
||||
kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined]
|
||||
# Best-effort; if there's no console attached these just fail silently.
|
||||
kernel32.SetConsoleCP(65001)
|
||||
kernel32.SetConsoleOutputCP(65001)
|
||||
except Exception:
|
||||
# ctypes import, missing kernel32, or non-Windows — any failure here
|
||||
# is non-fatal. We've still reconfigured Python's own streams below.
|
||||
pass
|
||||
|
||||
|
||||
def _reconfigure_stream(stream, *, encoding: str = "utf-8", errors: str = "replace") -> None:
|
||||
"""Reconfigure a text stream to UTF-8 in place.
|
||||
|
||||
Uses ``TextIOWrapper.reconfigure`` (Python 3.7+). If the stream isn't
|
||||
a ``TextIOWrapper`` (e.g. it's been redirected to an ``io.StringIO``
|
||||
during tests), we skip rather than blow up.
|
||||
"""
|
||||
try:
|
||||
reconfigure = getattr(stream, "reconfigure", None)
|
||||
if reconfigure is None:
|
||||
return
|
||||
reconfigure(encoding=encoding, errors=errors)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def configure_windows_stdio() -> bool:
|
||||
"""Force UTF-8 stdio on Windows. No-op elsewhere.
|
||||
|
||||
Idempotent — safe to call multiple times from different entry points.
|
||||
|
||||
Returns ``True`` if anything was actually changed, ``False`` on
|
||||
non-Windows or on a repeat call.
|
||||
|
||||
Set ``HERMES_DISABLE_WINDOWS_UTF8=1`` in the environment to opt out
|
||||
(for diagnosing encoding-related bugs by forcing the old cp1252 path).
|
||||
|
||||
Also sets a sensible default ``EDITOR`` on Windows if none is already
|
||||
set — see :func:`_default_windows_editor`.
|
||||
"""
|
||||
global _CONFIGURED
|
||||
|
||||
if _CONFIGURED:
|
||||
return False
|
||||
if not is_windows():
|
||||
# Mark configured so repeated calls on POSIX are true no-ops.
|
||||
_CONFIGURED = True
|
||||
return False
|
||||
|
||||
if os.environ.get("HERMES_DISABLE_WINDOWS_UTF8") in ("1", "true", "True", "yes"):
|
||||
_CONFIGURED = True
|
||||
return False
|
||||
|
||||
# Encourage every child Python process spawned by the agent to also use
|
||||
# UTF-8 for its stdio. PYTHONIOENCODING wins over the locale-based
|
||||
# default in subprocesses. Don't override an explicit user setting.
|
||||
os.environ.setdefault("PYTHONIOENCODING", "utf-8")
|
||||
# PYTHONUTF8 = 1 enables UTF-8 Mode globally for any Python subprocess
|
||||
# (PEP 540). Again, don't override an explicit setting.
|
||||
os.environ.setdefault("PYTHONUTF8", "1")
|
||||
|
||||
# Set EDITOR to a working Windows default if neither EDITOR nor VISUAL
|
||||
# is set. prompt_toolkit's ``open_in_editor`` falls back to POSIX-only
|
||||
# paths (``/usr/bin/nano``, ``/usr/bin/vi``) that don't exist on
|
||||
# Windows — Ctrl+X Ctrl+E and ``/edit`` silently do nothing there
|
||||
# otherwise. This happens even with full Git for Windows installed,
|
||||
# so it's not a MinGit-specific issue.
|
||||
_default_editor = _default_windows_editor()
|
||||
if _default_editor and not os.environ.get("EDITOR") and not os.environ.get("VISUAL"):
|
||||
os.environ["EDITOR"] = _default_editor
|
||||
|
||||
# Augment PATH with the Hermes-managed Git install directories so
|
||||
# subprocess calls (bash, rg, grep, etc.) resolve even in sessions
|
||||
# that started before the User PATH broadcast reached them. When
|
||||
# install.ps1 adds these to User PATH via SetEnvironmentVariable,
|
||||
# already-running shells don't see the change — which means hermes
|
||||
# launched from the install session won't find rg / bash / grep
|
||||
# even though they're "installed". Prepending the known paths here
|
||||
# closes that gap. No-op when the paths don't exist (e.g. system-Git
|
||||
# install without Hermes-managed PortableGit).
|
||||
_augment_path_with_known_tools()
|
||||
|
||||
# Flip the console code page first so that any subprocess that
|
||||
# inherits the console (e.g. a launched shell) also sees CP_UTF8.
|
||||
_flip_console_code_page_to_utf8()
|
||||
|
||||
# Reconfigure Python's own stdio wrappers so ``print()`` calls from
|
||||
# this process round-trip emoji / box-drawing / non-Latin text.
|
||||
# ``errors="replace"`` means a genuinely unencodable byte sequence
|
||||
# gets a ``?`` rather than crashing the interpreter — we prefer
|
||||
# degraded output over a stack trace.
|
||||
_reconfigure_stream(sys.stdout)
|
||||
_reconfigure_stream(sys.stderr)
|
||||
# stdin is re-configured for completeness; Hermes's interactive
|
||||
# input path uses prompt_toolkit which manages its own encoding,
|
||||
# but batch/pipe input benefits from UTF-8 decoding on stdin too.
|
||||
_reconfigure_stream(sys.stdin)
|
||||
|
||||
_CONFIGURED = True
|
||||
return True
|
||||
|
||||
|
||||
def _default_windows_editor() -> str:
|
||||
"""Return a Windows-appropriate default for ``$EDITOR``.
|
||||
|
||||
Priority order, first match wins:
|
||||
|
||||
1. ``notepad`` — ships with every Windows install, no deps, works as a
|
||||
blocking editor (``subprocess.call(["notepad", file])`` blocks until
|
||||
the user closes the window). This is the "always-works" default.
|
||||
|
||||
The prompt_toolkit buffer's ``open_in_editor`` and Hermes's
|
||||
``hermes config edit`` both honour ``$EDITOR``. Users who prefer a
|
||||
different editor can override:
|
||||
|
||||
- VSCode: ``$env:EDITOR = "code --wait"`` (``--wait`` is critical;
|
||||
without it the editor returns immediately and any input is lost)
|
||||
- Notepad++: ``$env:EDITOR = "'C:\\Program Files\\Notepad++\\notepad++.exe' -multiInst -nosession"``
|
||||
- Neovim: ``$env:EDITOR = "nvim"`` (if installed)
|
||||
|
||||
Set this before launching Hermes (User env var in Windows Settings, or
|
||||
export in a PowerShell profile) and Hermes picks it up automatically.
|
||||
"""
|
||||
import shutil
|
||||
|
||||
# notepad.exe is always in %SystemRoot%\System32 on Windows, so shutil.which
|
||||
# will reliably find it. Return the bare name so prompt_toolkit's shlex
|
||||
# split doesn't trip over a path containing spaces.
|
||||
if shutil.which("notepad"):
|
||||
return "notepad"
|
||||
# On the extreme off-chance notepad is missing (WinPE, Nano Server), fall
|
||||
# back to nothing and let prompt_toolkit's silent no-op do its thing.
|
||||
return ""
|
||||
|
||||
|
||||
|
||||
def _augment_path_with_known_tools() -> None:
|
||||
"""Prepend well-known Hermes-managed tool directories to os.environ['PATH'].
|
||||
|
||||
Fixes the "User PATH was just updated but my process can't see it" gap on
|
||||
Windows. When install.ps1 runs, it adds entries like
|
||||
``%LOCALAPPDATA%\\hermes\\git\\bin`` to the User PATH via
|
||||
``SetEnvironmentVariable(..., "User")``. That write propagates to newly
|
||||
*spawned* processes only — already-running shells (including the one the
|
||||
user invokes ``hermes`` from right after install) retain their old PATH.
|
||||
|
||||
Any subprocess Hermes spawns — bash, ``rg``, ``grep``, ``npm`` — inherits
|
||||
that stale PATH and reports commands as missing even though they're on
|
||||
disk. Symptom: ``search_files`` reports "rg/find not available" when
|
||||
the user clearly just installed ripgrep.
|
||||
|
||||
Patch-up strategy: add the known Hermes-managed tool directories to our
|
||||
PATH at startup so subprocess calls resolve correctly. No-op on POSIX
|
||||
and when the directories don't exist. The User PATH broadcast still
|
||||
happens in the background for future shells; this just smooths over
|
||||
the first-launch gap.
|
||||
"""
|
||||
if not is_windows():
|
||||
return
|
||||
|
||||
import shutil as _shutil
|
||||
|
||||
local_appdata = os.environ.get("LOCALAPPDATA", "")
|
||||
if not local_appdata:
|
||||
return
|
||||
|
||||
# Known tool dirs installed by scripts/install.ps1. Kept in sync with
|
||||
# the PATH entries that installer adds to User scope — the two lists
|
||||
# should match so this prefill fully mirrors what a fresh shell would
|
||||
# see on next launch.
|
||||
candidate_dirs = [
|
||||
os.path.join(local_appdata, "hermes", "git", "cmd"),
|
||||
os.path.join(local_appdata, "hermes", "git", "bin"),
|
||||
os.path.join(local_appdata, "hermes", "git", "usr", "bin"),
|
||||
# Hermes venv Scripts directory — host of the hermes.exe shim itself,
|
||||
# also where any pip-installed console scripts land. Usually already
|
||||
# on PATH when the user invokes hermes, but harmless to include.
|
||||
os.path.join(local_appdata, "hermes", "hermes-agent", "venv", "Scripts"),
|
||||
# WinGet packages directory — where ``winget install`` drops CLI
|
||||
# shims by default (ripgrep lands here as rg.exe). Covers the case
|
||||
# of a system-Git install + ripgrep-via-winget that isn't yet on
|
||||
# the spawning shell's PATH.
|
||||
os.path.join(local_appdata, "Microsoft", "WinGet", "Links"),
|
||||
]
|
||||
|
||||
existing = os.environ.get("PATH", "")
|
||||
existing_lower = {p.lower() for p in existing.split(os.pathsep) if p}
|
||||
prepend = []
|
||||
for d in candidate_dirs:
|
||||
if os.path.isdir(d) and d.lower() not in existing_lower:
|
||||
prepend.append(d)
|
||||
|
||||
if prepend:
|
||||
os.environ["PATH"] = os.pathsep.join([*prepend, existing])
|
||||
@@ -308,6 +308,23 @@ TOOL_CATEGORIES = {
|
||||
{"key": "SEARXNG_URL", "prompt": "Your SearXNG instance URL (e.g., http://localhost:8080)", "url": "https://searxng.github.io/searxng/"},
|
||||
],
|
||||
},
|
||||
{
|
||||
"name": "Brave Search (Free Tier)",
|
||||
"badge": "free tier · search only",
|
||||
"tag": "2,000 queries/mo free — search only (pair with any extract provider)",
|
||||
"web_backend": "brave-free",
|
||||
"env_vars": [
|
||||
{"key": "BRAVE_SEARCH_API_KEY", "prompt": "Brave Search subscription token", "url": "https://brave.com/search/api/"},
|
||||
],
|
||||
},
|
||||
{
|
||||
"name": "DuckDuckGo (ddgs)",
|
||||
"badge": "free · no key · search only",
|
||||
"tag": "Search via the ddgs Python package — no API key (pair with any extract provider)",
|
||||
"web_backend": "ddgs",
|
||||
"env_vars": [],
|
||||
"post_setup": "ddgs",
|
||||
},
|
||||
],
|
||||
},
|
||||
"image_gen": {
|
||||
@@ -492,8 +509,12 @@ def _run_post_setup(post_setup_key: str):
|
||||
if not node_modules.exists() and npm_bin:
|
||||
_print_info(" Installing Node.js dependencies for browser tools...")
|
||||
import subprocess
|
||||
# Use the resolved npm_bin absolute path so subprocess.Popen can
|
||||
# execute npm.cmd on Windows (CreateProcessW otherwise rejects
|
||||
# batch shims). On POSIX npm_bin is the plain path — same
|
||||
# behaviour as before.
|
||||
result = subprocess.run(
|
||||
["npm", "install", "--silent"],
|
||||
[npm_bin, "install", "--silent"],
|
||||
capture_output=True, text=True, cwd=str(PROJECT_ROOT)
|
||||
)
|
||||
if result.returncode == 0:
|
||||
@@ -592,11 +613,13 @@ def _run_post_setup(post_setup_key: str):
|
||||
|
||||
elif post_setup_key == "camofox":
|
||||
camofox_dir = PROJECT_ROOT / "node_modules" / "@askjo" / "camofox-browser"
|
||||
if not camofox_dir.exists() and shutil.which("npm"):
|
||||
_npm_bin = shutil.which("npm")
|
||||
if not camofox_dir.exists() and _npm_bin:
|
||||
_print_info(" Installing Camofox browser server...")
|
||||
import subprocess
|
||||
# Absolute npm path so .cmd shim executes on Windows.
|
||||
result = subprocess.run(
|
||||
["npm", "install", "--silent"],
|
||||
[_npm_bin, "install", "--silent"],
|
||||
capture_output=True, text=True, cwd=str(PROJECT_ROOT)
|
||||
)
|
||||
if result.returncode == 0:
|
||||
@@ -669,6 +692,32 @@ def _run_post_setup(post_setup_key: str):
|
||||
_print_info(" Full voice list: https://github.com/OHF-Voice/piper1-gpl/blob/main/docs/VOICES.md")
|
||||
_print_info(" Switch voices by setting tts.piper.voice in ~/.hermes/config.yaml")
|
||||
|
||||
elif post_setup_key == "ddgs":
|
||||
try:
|
||||
__import__("ddgs")
|
||||
_print_success(" ddgs is already installed")
|
||||
except ImportError:
|
||||
import subprocess
|
||||
_print_info(" Installing ddgs (DuckDuckGo search package)...")
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "pip", "install", "-U", "ddgs", "--quiet"],
|
||||
capture_output=True, text=True, timeout=300,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
_print_success(" ddgs installed")
|
||||
else:
|
||||
_print_warning(" ddgs install failed:")
|
||||
_print_info(f" {result.stderr.strip()[:300]}")
|
||||
_print_info(" Run manually: python -m pip install -U ddgs")
|
||||
return
|
||||
except subprocess.TimeoutExpired:
|
||||
_print_warning(" ddgs install timed out (>5min)")
|
||||
_print_info(" Run manually: python -m pip install -U ddgs")
|
||||
return
|
||||
_print_info(" No API key required. DuckDuckGo enforces server-side rate limits.")
|
||||
_print_info(" Pair with an extract provider if you also need web_extract.")
|
||||
|
||||
elif post_setup_key == "spotify":
|
||||
# Run the full `hermes auth spotify` flow — if the user has no
|
||||
# client_id yet, this drops them into the interactive wizard
|
||||
|
||||
+148
-36
@@ -281,6 +281,8 @@ _recorder_lock = threading.Lock()
|
||||
# ── Continuous (VAD) state ───────────────────────────────────────────
|
||||
_continuous_lock = threading.Lock()
|
||||
_continuous_active = False
|
||||
_continuous_stopping = False
|
||||
_continuous_auto_restart: bool = True
|
||||
_continuous_recorder: Any = None
|
||||
|
||||
# ── TTS-vs-STT feedback guard ────────────────────────────────────────
|
||||
@@ -370,32 +372,43 @@ def start_continuous(
|
||||
on_silent_limit: Optional[Callable[[], None]] = None,
|
||||
silence_threshold: int = 200,
|
||||
silence_duration: float = 3.0,
|
||||
) -> None:
|
||||
auto_restart: bool = True,
|
||||
) -> bool:
|
||||
"""Start a VAD-driven continuous recording loop.
|
||||
|
||||
The loop calls ``on_transcript(text)`` each time speech is detected and
|
||||
transcribed successfully, then auto-restarts. After
|
||||
``_CONTINUOUS_NO_SPEECH_LIMIT`` consecutive silent cycles (no speech
|
||||
picked up at all) the loop stops itself and calls ``on_silent_limit``
|
||||
so the UI can reflect "voice off". Idempotent — calling while already
|
||||
active is a no-op.
|
||||
transcribed successfully. If ``auto_restart`` is True, it auto-restarts
|
||||
for the next turn and resets the no-speech counter for that loop. If
|
||||
``auto_restart`` is False, the first silence-triggered transcription ends
|
||||
the loop and reports ``"idle"``; no-speech counts are retained across
|
||||
starts so a push-to-talk caller can still enforce the three-strikes guard.
|
||||
After ``_CONTINUOUS_NO_SPEECH_LIMIT`` consecutive silent cycles (no speech
|
||||
picked up at all) the loop stops itself and calls ``on_silent_limit`` so the
|
||||
UI can reflect "voice off". Returns False if a previous stop is still
|
||||
transcribing/cleaning up; otherwise returns True. Idempotent — calling while
|
||||
already active is a successful no-op.
|
||||
|
||||
``on_status`` is called with ``"listening"`` / ``"transcribing"`` /
|
||||
``"idle"`` so the UI can show a live indicator.
|
||||
"""
|
||||
global _continuous_active, _continuous_recorder
|
||||
global _continuous_active, _continuous_recorder, _continuous_auto_restart
|
||||
global _continuous_on_transcript, _continuous_on_status, _continuous_on_silent_limit
|
||||
global _continuous_no_speech_count
|
||||
|
||||
with _continuous_lock:
|
||||
if _continuous_active:
|
||||
_debug("start_continuous: already active — no-op")
|
||||
return
|
||||
return True
|
||||
if _continuous_stopping:
|
||||
_debug("start_continuous: stop/transcribe in progress — busy")
|
||||
return False
|
||||
_continuous_active = True
|
||||
_continuous_auto_restart = auto_restart
|
||||
_continuous_on_transcript = on_transcript
|
||||
_continuous_on_status = on_status
|
||||
_continuous_on_silent_limit = on_silent_limit
|
||||
_continuous_no_speech_count = 0
|
||||
if auto_restart:
|
||||
_continuous_no_speech_count = 0
|
||||
|
||||
if _continuous_recorder is None:
|
||||
_continuous_recorder = create_audio_recorder()
|
||||
@@ -428,15 +441,18 @@ def start_continuous(
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return True
|
||||
|
||||
def stop_continuous() -> None:
|
||||
|
||||
def stop_continuous(force_transcribe: bool = False) -> None:
|
||||
"""Stop the active continuous loop and release the microphone.
|
||||
|
||||
Idempotent — calling while not active is a no-op. Any in-flight
|
||||
transcription completes but its result is discarded (the callback
|
||||
checks ``_continuous_active`` before firing).
|
||||
Idempotent — calling while not active is a no-op. If ``force_transcribe`` is
|
||||
True, the recorder stops synchronously, then transcription/cleanup runs on a
|
||||
background thread before reporting ``"idle"``. Otherwise the buffer is
|
||||
discarded.
|
||||
"""
|
||||
global _continuous_active, _continuous_on_transcript
|
||||
global _continuous_active, _continuous_on_transcript, _continuous_stopping
|
||||
global _continuous_on_status, _continuous_on_silent_limit
|
||||
global _continuous_recorder, _continuous_no_speech_count
|
||||
|
||||
@@ -446,18 +462,98 @@ def stop_continuous() -> None:
|
||||
_continuous_active = False
|
||||
rec = _continuous_recorder
|
||||
on_status = _continuous_on_status
|
||||
on_transcript = _continuous_on_transcript
|
||||
on_silent_limit = _continuous_on_silent_limit
|
||||
auto_restart = _continuous_auto_restart
|
||||
track_no_speech = force_transcribe and not auto_restart
|
||||
_continuous_stopping = rec is not None
|
||||
_continuous_on_transcript = None
|
||||
_continuous_on_status = None
|
||||
_continuous_on_silent_limit = None
|
||||
_continuous_no_speech_count = 0
|
||||
if not track_no_speech:
|
||||
_continuous_no_speech_count = 0
|
||||
|
||||
if rec is not None:
|
||||
try:
|
||||
# cancel() (not stop()) discards buffered frames — the loop
|
||||
# is over, we don't want to transcribe a half-captured turn.
|
||||
rec.cancel()
|
||||
except Exception as e:
|
||||
logger.warning("failed to cancel recorder: %s", e)
|
||||
if force_transcribe and on_transcript:
|
||||
if on_status:
|
||||
try:
|
||||
on_status("transcribing")
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
wav_path = rec.stop()
|
||||
except Exception as e:
|
||||
logger.warning("failed to stop recorder: %s", e)
|
||||
try:
|
||||
rec.cancel()
|
||||
except Exception as cancel_error:
|
||||
logger.warning("failed to cancel recorder: %s", cancel_error)
|
||||
wav_path = None
|
||||
|
||||
def _transcribe_and_cleanup():
|
||||
global _continuous_no_speech_count, _continuous_stopping
|
||||
transcript: Optional[str] = None
|
||||
should_halt = False
|
||||
|
||||
try:
|
||||
if wav_path:
|
||||
try:
|
||||
result = transcribe_recording(wav_path)
|
||||
if result.get("success"):
|
||||
text = (result.get("transcript") or "").strip()
|
||||
if text and not is_whisper_hallucination(text):
|
||||
transcript = text
|
||||
finally:
|
||||
if os.path.isfile(wav_path):
|
||||
os.unlink(wav_path)
|
||||
except Exception as e:
|
||||
logger.warning("failed to stop/transcribe recorder: %s", e)
|
||||
finally:
|
||||
if transcript:
|
||||
try:
|
||||
on_transcript(transcript)
|
||||
except Exception as e:
|
||||
logger.warning("on_transcript callback raised: %s", e)
|
||||
|
||||
if track_no_speech:
|
||||
with _continuous_lock:
|
||||
if transcript:
|
||||
_continuous_no_speech_count = 0
|
||||
else:
|
||||
_continuous_no_speech_count += 1
|
||||
should_halt = (
|
||||
_continuous_no_speech_count
|
||||
>= _CONTINUOUS_NO_SPEECH_LIMIT
|
||||
)
|
||||
if should_halt:
|
||||
_continuous_no_speech_count = 0
|
||||
if should_halt and on_silent_limit:
|
||||
try:
|
||||
on_silent_limit()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
_play_beep(frequency=660, count=2)
|
||||
with _continuous_lock:
|
||||
_continuous_stopping = False
|
||||
if on_status:
|
||||
try:
|
||||
on_status("idle")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
threading.Thread(target=_transcribe_and_cleanup, daemon=True).start()
|
||||
return
|
||||
else:
|
||||
try:
|
||||
# cancel() (not stop()) discards buffered frames — the loop
|
||||
# is over, we don't want to transcribe a half-captured turn.
|
||||
rec.cancel()
|
||||
except Exception as e:
|
||||
logger.warning("failed to cancel recorder: %s", e)
|
||||
|
||||
with _continuous_lock:
|
||||
_continuous_stopping = False
|
||||
|
||||
# Audible "recording stopped" cue (CLI parity: same 660 Hz × 2 the
|
||||
# silence-auto-stop path plays).
|
||||
@@ -603,23 +699,39 @@ def _continuous_on_silence() -> None:
|
||||
_debug("_continuous_on_silence: stopped while waiting for TTS")
|
||||
return
|
||||
|
||||
# Restart for the next turn.
|
||||
_debug(f"_continuous_on_silence: restarting loop (no_speech={no_speech})")
|
||||
_play_beep(frequency=880, count=1)
|
||||
try:
|
||||
rec.start(on_silence_stop=_continuous_on_silence)
|
||||
except Exception as e:
|
||||
logger.error("failed to restart continuous recording: %s", e)
|
||||
_debug(f"_continuous_on_silence: restart raised {type(e).__name__}: {e}")
|
||||
if _continuous_auto_restart:
|
||||
# Restart for the next turn.
|
||||
_debug(f"_continuous_on_silence: restarting loop (no_speech={no_speech})")
|
||||
_play_beep(frequency=880, count=1)
|
||||
try:
|
||||
rec.start(on_silence_stop=_continuous_on_silence)
|
||||
except Exception as e:
|
||||
logger.error("failed to restart continuous recording: %s", e)
|
||||
_debug(f"_continuous_on_silence: restart raised {type(e).__name__}: {e}")
|
||||
with _continuous_lock:
|
||||
_continuous_active = False
|
||||
if on_status:
|
||||
try:
|
||||
on_status("idle")
|
||||
except Exception:
|
||||
pass
|
||||
return
|
||||
|
||||
if on_status:
|
||||
try:
|
||||
on_status("listening")
|
||||
except Exception:
|
||||
pass
|
||||
else:
|
||||
# Do not auto-restart. Clean up state and notify idle.
|
||||
_debug("_continuous_on_silence: auto_restart=False, stopping loop")
|
||||
with _continuous_lock:
|
||||
_continuous_active = False
|
||||
return
|
||||
|
||||
if on_status:
|
||||
try:
|
||||
on_status("listening")
|
||||
except Exception:
|
||||
pass
|
||||
if on_status:
|
||||
try:
|
||||
on_status("idle")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ── TTS API ──────────────────────────────────────────────────────────
|
||||
|
||||
+209
-13
@@ -52,7 +52,7 @@ from gateway.status import get_running_pid, read_runtime_status
|
||||
try:
|
||||
from fastapi import FastAPI, HTTPException, Request, WebSocket, WebSocketDisconnect
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse
|
||||
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel
|
||||
except ImportError:
|
||||
@@ -692,7 +692,7 @@ def _tail_lines(path: Path, n: int) -> List[str]:
|
||||
if not path.exists():
|
||||
return []
|
||||
try:
|
||||
text = path.read_text(errors="replace")
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
except OSError:
|
||||
return []
|
||||
lines = text.splitlines()
|
||||
@@ -1877,8 +1877,8 @@ async def _start_device_code_flow(provider_id: str) -> Dict[str, Any]:
|
||||
name=f"oauth-codex-{sid[:6]}",
|
||||
).start()
|
||||
# Block briefly until the worker has populated the user_code, OR error.
|
||||
deadline = time.time() + 10
|
||||
while time.time() < deadline:
|
||||
deadline = time.monotonic() + 10
|
||||
while time.monotonic() < deadline:
|
||||
with _oauth_sessions_lock:
|
||||
s = _oauth_sessions.get(sid)
|
||||
if s and (s.get("user_code") or s["status"] != "pending"):
|
||||
@@ -2012,10 +2012,10 @@ def _codex_full_login_worker(session_id: str) -> None:
|
||||
sess["expires_at"] = time.time() + sess["expires_in"]
|
||||
|
||||
# Step 2: poll until authorized
|
||||
deadline = time.time() + sess["expires_in"]
|
||||
deadline = time.monotonic() + sess["expires_in"]
|
||||
code_resp = None
|
||||
with httpx.Client(timeout=httpx.Timeout(15.0)) as client:
|
||||
while time.time() < deadline:
|
||||
while time.monotonic() < deadline:
|
||||
time.sleep(poll_interval)
|
||||
poll = client.post(
|
||||
f"{issuer}/api/accounts/deviceauth/token",
|
||||
@@ -2173,6 +2173,83 @@ async def cancel_oauth_session(session_id: str, request: Request):
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
|
||||
def _session_latest_descendant(session_id: str):
|
||||
"""Resolve a session id to the newest child leaf session.
|
||||
|
||||
/model may create child sessions. Dashboard refresh should continue the
|
||||
newest child instead of reopening the old parent.
|
||||
"""
|
||||
from hermes_state import SessionDB
|
||||
|
||||
def row_get(row, key, index):
|
||||
if isinstance(row, dict):
|
||||
return row.get(key)
|
||||
try:
|
||||
return row[key]
|
||||
except Exception:
|
||||
try:
|
||||
return row[index]
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
db = SessionDB()
|
||||
try:
|
||||
sid = db.resolve_session_id(session_id)
|
||||
if not sid or not db.get_session(sid):
|
||||
return None, []
|
||||
|
||||
conn = (
|
||||
getattr(db, "conn", None)
|
||||
or getattr(db, "_conn", None)
|
||||
or getattr(db, "connection", None)
|
||||
or getattr(db, "_connection", None)
|
||||
)
|
||||
|
||||
rows = []
|
||||
if conn is not None:
|
||||
raw_rows = conn.execute(
|
||||
"SELECT id, parent_session_id, started_at FROM sessions"
|
||||
).fetchall()
|
||||
for row in raw_rows:
|
||||
rows.append({
|
||||
"id": row_get(row, "id", 0),
|
||||
"parent_session_id": row_get(row, "parent_session_id", 1),
|
||||
"started_at": row_get(row, "started_at", 2),
|
||||
})
|
||||
else:
|
||||
rows = db.list_sessions_rich(limit=10000, offset=0)
|
||||
|
||||
children = {}
|
||||
for row in rows:
|
||||
rid = row.get("id")
|
||||
parent = row.get("parent_session_id")
|
||||
if rid and parent:
|
||||
children.setdefault(parent, []).append(row)
|
||||
|
||||
def started(row):
|
||||
try:
|
||||
return float(row.get("started_at") or 0)
|
||||
except Exception:
|
||||
return 0.0
|
||||
|
||||
current = sid
|
||||
path = [sid]
|
||||
seen = {sid}
|
||||
|
||||
while children.get(current):
|
||||
candidates = [r for r in children[current] if r.get("id") not in seen]
|
||||
if not candidates:
|
||||
break
|
||||
candidates.sort(key=started, reverse=True)
|
||||
current = candidates[0]["id"]
|
||||
path.append(current)
|
||||
seen.add(current)
|
||||
|
||||
return current, path
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@app.get("/api/sessions/{session_id}")
|
||||
async def get_session_detail(session_id: str):
|
||||
from hermes_state import SessionDB
|
||||
@@ -2187,6 +2264,19 @@ async def get_session_detail(session_id: str):
|
||||
db.close()
|
||||
|
||||
|
||||
|
||||
@app.get("/api/sessions/{session_id}/latest-descendant")
|
||||
async def get_session_latest_descendant(session_id: str):
|
||||
latest, path = _session_latest_descendant(session_id)
|
||||
if not latest:
|
||||
raise HTTPException(status_code=404, detail="Session not found")
|
||||
return {
|
||||
"requested_session_id": path[0] if path else session_id,
|
||||
"session_id": latest,
|
||||
"path": path,
|
||||
"changed": bool(path and latest != path[0]),
|
||||
}
|
||||
|
||||
@app.get("/api/sessions/{session_id}/messages")
|
||||
async def get_session_messages(session_id: str):
|
||||
from hermes_state import SessionDB
|
||||
@@ -2366,6 +2456,7 @@ async def delete_cron_job(job_id: str):
|
||||
class ProfileCreate(BaseModel):
|
||||
name: str
|
||||
clone_from_default: bool = False
|
||||
no_skills: bool = False
|
||||
|
||||
|
||||
class ProfileRename(BaseModel):
|
||||
@@ -2471,11 +2562,13 @@ async def create_profile_endpoint(body: ProfileCreate):
|
||||
name=body.name,
|
||||
clone_from="default" if body.clone_from_default else None,
|
||||
clone_config=body.clone_from_default,
|
||||
no_skills=body.no_skills,
|
||||
)
|
||||
# Match the CLI's profile-create flow: fresh named profiles get the
|
||||
# bundled skills installed. When cloning from default, create_profile()
|
||||
# has already copied the source profile's skills, including any
|
||||
# user-installed skills.
|
||||
# user-installed skills. When no_skills=True, create_profile() wrote
|
||||
# the opt-out marker and seed_profile_skills() will no-op.
|
||||
if not body.clone_from_default:
|
||||
profiles_mod.seed_profile_skills(path, quiet=True)
|
||||
|
||||
@@ -2886,7 +2979,20 @@ async def get_models_analytics(days: int = 30):
|
||||
import re
|
||||
import asyncio
|
||||
|
||||
from hermes_cli.pty_bridge import PtyBridge, PtyUnavailableError
|
||||
# PTY bridge is POSIX-only (depends on fcntl/termios/ptyprocess). On native
|
||||
# Windows the import raises; catch and leave PtyBridge=None so the rest of
|
||||
# the dashboard (sessions, jobs, metrics, config editor) still loads and the
|
||||
# /api/pty endpoint cleanly refuses with a WSL-suggested message.
|
||||
try:
|
||||
from hermes_cli.pty_bridge import PtyBridge, PtyUnavailableError
|
||||
_PTY_BRIDGE_AVAILABLE = True
|
||||
except ImportError as _pty_import_err: # pragma: no cover - Windows-only path
|
||||
PtyBridge = None # type: ignore[assignment]
|
||||
_PTY_BRIDGE_AVAILABLE = False
|
||||
|
||||
class PtyUnavailableError(RuntimeError): # type: ignore[no-redef]
|
||||
"""Stub on platforms where pty_bridge can't be imported."""
|
||||
pass
|
||||
|
||||
_RESIZE_RE = re.compile(rb"\x1b\[RESIZE:(\d+);(\d+)\]")
|
||||
_PTY_READ_CHUNK_TIMEOUT = 0.2
|
||||
@@ -2946,8 +3052,18 @@ def _resolve_chat_argv(
|
||||
argv, cwd = _make_tui_argv(PROJECT_ROOT / "ui-tui", tui_dev=False)
|
||||
env = os.environ.copy()
|
||||
env.setdefault("NODE_ENV", "production")
|
||||
# Browser-embedded chat should prefer stable wheel-based scrollback over
|
||||
# native terminal mouse tracking. When mouse tracking is enabled, wheel
|
||||
# events are consumed by the TUI and forwarded as terminal input, which
|
||||
# makes browser-side transcript scrolling feel broken. Keep the terminal
|
||||
# build unchanged for native CLI usage; only disable mouse tracking for
|
||||
# the dashboard PTY path.
|
||||
env.setdefault("HERMES_TUI_DISABLE_MOUSE", "1")
|
||||
|
||||
if resume:
|
||||
latest_resume, _latest_path = _session_latest_descendant(resume)
|
||||
if latest_resume:
|
||||
resume = latest_resume
|
||||
env["HERMES_TUI_RESUME"] = resume
|
||||
|
||||
if sidecar_url:
|
||||
@@ -3010,6 +3126,18 @@ async def pty_ws(ws: WebSocket) -> None:
|
||||
|
||||
await ws.accept()
|
||||
|
||||
# On native Windows, the POSIX PTY bridge can't be imported. Tell the
|
||||
# client and close cleanly rather than pretending the feature works.
|
||||
if not _PTY_BRIDGE_AVAILABLE:
|
||||
await ws.send_text(
|
||||
"\r\n\x1b[31mChat unavailable: the embedded terminal requires a "
|
||||
"POSIX PTY, which native Windows Python doesn't provide.\x1b[0m\r\n"
|
||||
"\x1b[33mInstall Hermes inside WSL2 to use the dashboard's /chat "
|
||||
"tab — the rest of the dashboard works here.\x1b[0m\r\n"
|
||||
)
|
||||
await ws.close(code=1011)
|
||||
return
|
||||
|
||||
# --- spawn PTY ------------------------------------------------------
|
||||
resume = ws.query_params.get("resume") or None
|
||||
channel = _channel_or_close_code(ws)
|
||||
@@ -3205,12 +3333,42 @@ async def events_ws(ws: WebSocket) -> None:
|
||||
_event_channels.pop(channel, None)
|
||||
|
||||
|
||||
def _normalise_prefix(raw: Optional[str]) -> str:
|
||||
"""Normalise an X-Forwarded-Prefix header value.
|
||||
|
||||
Returns a string like ``"/hermes"`` (no trailing slash) or ``""`` when
|
||||
no prefix is set / the header is malformed. We deliberately reject
|
||||
anything containing ``..`` or non-printable bytes so a hostile proxy
|
||||
can't inject HTML via the prefix.
|
||||
"""
|
||||
if not raw:
|
||||
return ""
|
||||
p = raw.strip()
|
||||
if not p:
|
||||
return ""
|
||||
if not p.startswith("/"):
|
||||
p = "/" + p
|
||||
p = p.rstrip("/")
|
||||
if "//" in p or ".." in p or any(c in p for c in ('"', "'", "<", ">", " ", "\n", "\r", "\t")):
|
||||
return ""
|
||||
if len(p) > 64:
|
||||
return ""
|
||||
return p
|
||||
|
||||
|
||||
def mount_spa(application: FastAPI):
|
||||
"""Mount the built SPA. Falls back to index.html for client-side routing.
|
||||
|
||||
The session token is injected into index.html via a ``<script>`` tag so
|
||||
the SPA can authenticate against protected API endpoints without a
|
||||
separate (unauthenticated) token-dispensing endpoint.
|
||||
|
||||
When served behind a path-prefix reverse proxy (e.g.
|
||||
``mission-control.tilos.com/hermes/*`` -> local Caddy -> :9119), the
|
||||
proxy injects ``X-Forwarded-Prefix: /hermes`` on every request. We
|
||||
rewrite the served ``index.html`` so absolute asset URLs (``/assets/...``)
|
||||
and the SPA's runtime ``__HERMES_BASE_PATH__`` honour that prefix
|
||||
without rebuilding the bundle.
|
||||
"""
|
||||
if not WEB_DIST.exists():
|
||||
@application.get("/{full_path:path}")
|
||||
@@ -3223,24 +3381,62 @@ def mount_spa(application: FastAPI):
|
||||
|
||||
_index_path = WEB_DIST / "index.html"
|
||||
|
||||
def _serve_index():
|
||||
"""Return index.html with the session token injected."""
|
||||
def _serve_index(prefix: str = ""):
|
||||
"""Return index.html with the session token + base-path injected.
|
||||
|
||||
``prefix`` is the normalised ``X-Forwarded-Prefix`` (e.g. ``/hermes``)
|
||||
or empty string when served at root.
|
||||
"""
|
||||
html = _index_path.read_text()
|
||||
chat_js = "true" if _DASHBOARD_EMBEDDED_CHAT_ENABLED else "false"
|
||||
token_script = (
|
||||
f'<script>window.__HERMES_SESSION_TOKEN__="{_SESSION_TOKEN}";'
|
||||
f"window.__HERMES_DASHBOARD_EMBEDDED_CHAT__={chat_js};</script>"
|
||||
f"window.__HERMES_DASHBOARD_EMBEDDED_CHAT__={chat_js};"
|
||||
f'window.__HERMES_BASE_PATH__="{prefix}";</script>'
|
||||
)
|
||||
if prefix:
|
||||
# Rewrite absolute asset URLs baked into the Vite build so the
|
||||
# browser fetches them through the same proxy prefix.
|
||||
html = html.replace('href="/assets/', f'href="{prefix}/assets/')
|
||||
html = html.replace('src="/assets/', f'src="{prefix}/assets/')
|
||||
html = html.replace('href="/favicon.ico"', f'href="{prefix}/favicon.ico"')
|
||||
html = html.replace('href="/fonts/', f'href="{prefix}/fonts/')
|
||||
html = html.replace('href="/ds-assets/', f'href="{prefix}/ds-assets/')
|
||||
html = html.replace('src="/ds-assets/', f'src="{prefix}/ds-assets/')
|
||||
html = html.replace("</head>", f"{token_script}</head>", 1)
|
||||
return HTMLResponse(
|
||||
html,
|
||||
headers={"Cache-Control": "no-store, no-cache, must-revalidate"},
|
||||
)
|
||||
|
||||
# When served behind a path-prefix proxy, the built CSS contains
|
||||
# absolute ``url(/fonts/...)`` and ``url(/ds-assets/...)`` references.
|
||||
# Browsers resolve those against the document origin, which means
|
||||
# under ``/hermes`` they'd hit ``mission-control.tilos.com/fonts/...``
|
||||
# (the MC Pages app), not the Hermes backend. Intercept CSS asset
|
||||
# requests BEFORE the StaticFiles mount and rewrite the absolute paths
|
||||
# when a prefix is in play.
|
||||
@application.get("/assets/{filename}.css")
|
||||
async def serve_css(filename: str, request: Request):
|
||||
css_path = WEB_DIST / "assets" / f"{filename}.css"
|
||||
if not css_path.is_file() or not css_path.resolve().is_relative_to(
|
||||
WEB_DIST.resolve()
|
||||
):
|
||||
return JSONResponse({"error": "not found"}, status_code=404)
|
||||
prefix = _normalise_prefix(request.headers.get("x-forwarded-prefix"))
|
||||
css = css_path.read_text()
|
||||
if prefix:
|
||||
for asset_dir in ("/fonts/", "/fonts-terminal/", "/ds-assets/", "/assets/"):
|
||||
css = css.replace(f"url({asset_dir}", f"url({prefix}{asset_dir}")
|
||||
css = css.replace(f"url(\"{asset_dir}", f"url(\"{prefix}{asset_dir}")
|
||||
css = css.replace(f"url('{asset_dir}", f"url('{prefix}{asset_dir}")
|
||||
return Response(content=css, media_type="text/css")
|
||||
|
||||
application.mount("/assets", StaticFiles(directory=WEB_DIST / "assets"), name="assets")
|
||||
|
||||
@application.get("/{full_path:path}")
|
||||
async def serve_spa(full_path: str):
|
||||
async def serve_spa(full_path: str, request: Request):
|
||||
prefix = _normalise_prefix(request.headers.get("x-forwarded-prefix"))
|
||||
file_path = WEB_DIST / full_path
|
||||
# Prevent path traversal via url-encoded sequences (%2e%2e/)
|
||||
if (
|
||||
@@ -3250,7 +3446,7 @@ def mount_spa(application: FastAPI):
|
||||
and file_path.is_file()
|
||||
):
|
||||
return FileResponse(file_path)
|
||||
return _serve_index()
|
||||
return _serve_index(prefix)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
+2
-2
@@ -233,7 +233,7 @@ def is_wsl() -> bool:
|
||||
if _wsl_detected is not None:
|
||||
return _wsl_detected
|
||||
try:
|
||||
with open("/proc/version", "r") as f:
|
||||
with open("/proc/version", "r", encoding="utf-8") as f:
|
||||
_wsl_detected = "microsoft" in f.read().lower()
|
||||
except Exception:
|
||||
_wsl_detected = False
|
||||
@@ -260,7 +260,7 @@ def is_container() -> bool:
|
||||
_container_detected = True
|
||||
return True
|
||||
try:
|
||||
with open("/proc/1/cgroup", "r") as f:
|
||||
with open("/proc/1/cgroup", "r", encoding="utf-8") as f:
|
||||
cgroup = f.read()
|
||||
if "docker" in cgroup or "podman" in cgroup or "/lxc/" in cgroup:
|
||||
_container_detected = True
|
||||
|
||||
@@ -612,6 +612,11 @@ class SessionDB:
|
||||
the caller already holds cumulative totals (gateway path, where the
|
||||
cached agent accumulates across messages).
|
||||
"""
|
||||
# Ensure the session row exists so the UPDATE doesn't silently affect
|
||||
# 0 rows. Under concurrent load (cron + kanban + delegate_task) the
|
||||
# initial create_session() may have failed due to SQLite locking.
|
||||
# INSERT OR IGNORE is cheap and idempotent.
|
||||
self._insert_session_row(session_id, "unknown", model=model)
|
||||
if absolute:
|
||||
sql = """UPDATE sessions SET
|
||||
input_tokens = ?,
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ def _resolve_timezone_name() -> str:
|
||||
import yaml
|
||||
config_path = get_config_path()
|
||||
if config_path.exists():
|
||||
with open(config_path) as f:
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
tz_cfg = cfg.get("timezone", "")
|
||||
if isinstance(tz_cfg, str) and tz_cfg.strip():
|
||||
|
||||
+32
-2
@@ -115,6 +115,25 @@ def _load_channel_directory() -> dict:
|
||||
return {}
|
||||
|
||||
|
||||
def _coerce_int(
|
||||
value,
|
||||
*,
|
||||
default: int,
|
||||
minimum: int,
|
||||
maximum: int,
|
||||
) -> int:
|
||||
"""Coerce value to int with fallback and clamping.
|
||||
|
||||
Used at MCP tool boundaries to handle invalid types from external clients.
|
||||
Returns default if value cannot be converted to int.
|
||||
"""
|
||||
try:
|
||||
coerced = int(value)
|
||||
except (TypeError, ValueError):
|
||||
coerced = default
|
||||
return max(minimum, min(coerced, maximum))
|
||||
|
||||
|
||||
def _extract_message_content(msg: dict) -> str:
|
||||
"""Extract text content from a message, handling multi-part content."""
|
||||
content = msg.get("content", "")
|
||||
@@ -465,6 +484,7 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
||||
limit: Maximum number of conversations to return (default 50)
|
||||
search: Optional text to filter conversations by name
|
||||
"""
|
||||
limit = _coerce_int(limit, default=50, minimum=1, maximum=200)
|
||||
entries = _load_sessions_index()
|
||||
conversations = []
|
||||
|
||||
@@ -552,6 +572,7 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
||||
session_key: The session key from conversations_list
|
||||
limit: Maximum number of messages to return (default 50, most recent)
|
||||
"""
|
||||
limit = _coerce_int(limit, default=50, minimum=1, maximum=200)
|
||||
entries = _load_sessions_index()
|
||||
entry = entries.get(session_key)
|
||||
if not entry:
|
||||
@@ -664,6 +685,8 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
||||
session_key: Optional filter to one conversation
|
||||
limit: Maximum events to return (default 20)
|
||||
"""
|
||||
after_cursor = _coerce_int(after_cursor, default=0, minimum=0, maximum=10**18)
|
||||
limit = _coerce_int(limit, default=20, minimum=1, maximum=200)
|
||||
result = bridge.poll_events(
|
||||
after_cursor=after_cursor,
|
||||
session_key=session_key,
|
||||
@@ -689,10 +712,17 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
||||
session_key: Optional filter to one conversation
|
||||
timeout_ms: Maximum wait time in milliseconds (default 30000)
|
||||
"""
|
||||
after_cursor = _coerce_int(after_cursor, default=0, minimum=0, maximum=10**18)
|
||||
timeout_ms = _coerce_int(
|
||||
timeout_ms,
|
||||
default=30000,
|
||||
minimum=0,
|
||||
maximum=300000,
|
||||
) # Cap at 5 minutes
|
||||
event = bridge.wait_for_event(
|
||||
after_cursor=after_cursor,
|
||||
session_key=session_key,
|
||||
timeout_ms=min(timeout_ms, 300000), # Cap at 5 minutes
|
||||
timeout_ms=timeout_ms,
|
||||
)
|
||||
if event:
|
||||
return json.dumps({"event": event}, indent=2)
|
||||
@@ -772,7 +802,7 @@ def create_mcp_server(event_bridge: Optional[EventBridge] = None) -> "FastMCP":
|
||||
return json.dumps({"count": len(targets), "channels": targets}, indent=2)
|
||||
|
||||
channels = []
|
||||
for plat, entries_list in directory.items():
|
||||
for plat, entries_list in directory.get("platforms", {}).items():
|
||||
if platform and plat.lower() != platform.lower():
|
||||
continue
|
||||
if isinstance(entries_list, list):
|
||||
|
||||
+6
-6
@@ -730,8 +730,8 @@ def handle_function_call(
|
||||
session_id=session_id or "",
|
||||
tool_call_id=tool_call_id or "",
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _hook_err:
|
||||
logger.debug("pre_tool_call hook error: %s", _hook_err)
|
||||
|
||||
if block_message is not None:
|
||||
return json.dumps({"error": block_message}, ensure_ascii=False)
|
||||
@@ -782,8 +782,8 @@ def handle_function_call(
|
||||
tool_call_id=tool_call_id or "",
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _hook_err:
|
||||
logger.debug("post_tool_call hook error: %s", _hook_err)
|
||||
|
||||
# Generic tool-result canonicalization seam: plugins receive the
|
||||
# final result string (JSON, usually) and may replace it by
|
||||
@@ -807,8 +807,8 @@ def handle_function_call(
|
||||
if isinstance(hook_result, str):
|
||||
result = hook_result
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _hook_err:
|
||||
logger.debug("transform_tool_result hook error: %s", _hook_err)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
@@ -0,0 +1,432 @@
|
||||
---
|
||||
name: 3-statement-model
|
||||
description: Build fully-integrated 3-statement models (IS, BS, CF) in Excel with working capital schedules, D&A roll-forwards, debt schedule, and the plugs that make cash and retained earnings tie. Pairs with excel-author.
|
||||
version: 1.0.0
|
||||
author: Anthropic (adapted by Nous Research)
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [finance, three-statement, income-statement, balance-sheet, cash-flow, excel, openpyxl, modeling]
|
||||
related_skills: [excel-author, pptx-author, dcf-model, lbo-model]
|
||||
---
|
||||
|
||||
## Environment
|
||||
|
||||
This skill assumes **headless openpyxl** — you are producing an .xlsx file on disk.
|
||||
Follow the `excel-author` skill's conventions for cell coloring, formulas, named ranges, and sensitivity tables.
|
||||
Recalculate before delivery: `python /path/to/excel-author/scripts/recalc.py ./out/model.xlsx`.
|
||||
|
||||
# 3-Statement Financial Model Template Completion
|
||||
|
||||
Complete and populate integrated financial model templates with proper linkages between Income Statement, Balance Sheet, and Cash Flow Statement.
|
||||
|
||||
## ⚠️ CRITICAL PRINCIPLES — Read Before Populating Any Template
|
||||
|
||||
**Formulas over hardcodes (non-negotiable):**
|
||||
- Every projection cell, roll-forward, linkage, and subtotal MUST be an Excel formula — never a pre-computed value
|
||||
- When using Python/openpyxl: write formula strings (`ws["D15"] = "=D14*(1+Assumptions!$B$5)"`), NOT computed results (`ws["D15"] = 12500`)
|
||||
- The ONLY cells that should contain hardcoded numbers are: (1) historical actuals, (2) assumption drivers in the Assumptions tab
|
||||
- If you find yourself computing a value in Python and writing the result to a cell — STOP. Write the formula instead.
|
||||
- Why: the model must flex when scenarios toggle or assumptions change. Hardcodes break every downstream integrity check silently.
|
||||
|
||||
**Verify step-by-step with the user:**
|
||||
1. **After mapping the template** → show the user which tabs/sections you've identified and confirm before touching any cells
|
||||
2. **After populating historicals** → show the user the historical block and confirm values/periods match source data
|
||||
3. **After building IS projections** → run the subtotal checks, show the user the projected IS, confirm before moving to BS
|
||||
4. **After building BS** → show the user the balance check (Assets = L+E) for every period, confirm before moving to CF
|
||||
5. **After building CF** → show the user the cash tie-out (CF ending cash = BS cash), confirm before finalizing
|
||||
6. **Do NOT populate the entire model end-to-end and present it complete** — break at each statement, show the work, catch errors early
|
||||
|
||||
## Formatting — Professional Blue/Grey Palette (Default unless template/user specifies otherwise)
|
||||
|
||||
**Keep colors minimal.** Use only blues and greys for cell fills. Do NOT introduce greens, yellows, oranges, or multiple accent colors — a clean model uses restraint.
|
||||
|
||||
| Element | Fill | Font |
|
||||
|---|---|---|
|
||||
| Section headers (IS / BS / CF titles) | Dark blue `#1F4E79` | White bold |
|
||||
| Column headers (FY2024A, FY2025E, etc.) | Light blue `#D9E1F2` | Black bold |
|
||||
| Input cells (historicals, assumption drivers) | Light grey `#F2F2F2` or white | Blue `#0000FF` |
|
||||
| Formula cells | White | Black |
|
||||
| Cross-tab links | White | Green `#008000` |
|
||||
| Check rows / key totals | Medium blue `#BDD7EE` | Black bold |
|
||||
|
||||
**That's 3 blues + 1 grey + white.** If the template has its own color scheme, follow the template instead.
|
||||
|
||||
Font color signals *what* a cell is (input/formula/link). Fill color signals *where* you are (header/data/check).
|
||||
|
||||
## Model Structure
|
||||
|
||||
### Identifying Template Tab Organization
|
||||
|
||||
Templates vary in their tab naming conventions and organization. Before populating, review all tabs to understand the template's structure. Below are common tab names and their typical contents:
|
||||
|
||||
| Common Tab Names | Contents to Look For |
|
||||
|------------------|----------------------|
|
||||
| IS, P&L, Income Statement | Income Statement |
|
||||
| BS, Balance Sheet | Balance Sheet |
|
||||
| CF, CFS, Cash Flow | Cash Flow Statement |
|
||||
| WC, Working Capital | Working Capital Schedule |
|
||||
| DA, D&A, Depreciation, PP&E | Depreciation & Amortization Schedule |
|
||||
| Debt, Debt Schedule | Debt Schedule |
|
||||
| NOL, Tax, DTA | Net Operating Loss Schedule |
|
||||
| Assumptions, Inputs, Drivers | Driver assumptions and inputs |
|
||||
| Checks, Audit, Validation | Error-checking dashboard |
|
||||
|
||||
**Template Review Checklist**
|
||||
- Identify which tabs exist in the template (not all templates include every schedule)
|
||||
- Note any template-specific tabs not listed above
|
||||
- Understand tab dependencies (e.g., which schedules feed into the main statements)
|
||||
- Locate input cells vs. formula cells on each tab
|
||||
|
||||
### Understanding Template Structure
|
||||
|
||||
Before populating a template, familiarize yourself with its existing layout to ensure data is entered in the correct locations and formulas remain intact.
|
||||
|
||||
**Identifying Row Structure**
|
||||
- Locate the model title at top of each tab
|
||||
- Identify section headers and their visual separation
|
||||
- Find the units row indicating $ millions, %, x, etc.
|
||||
- Note column headers distinguishing Actuals vs. Estimates periods
|
||||
- Confirm period labels (e.g., FY2024A, FY2025E)
|
||||
- Identify input cells vs. formula cells (typically distinguished by font color)
|
||||
|
||||
**Identifying Column Structure**
|
||||
- Confirm line item labels in leftmost column
|
||||
- Verify historical years precede projection years
|
||||
- Note the visual border separating historical from projected periods
|
||||
- Check for consistent column order across all tabs
|
||||
|
||||
**Working with Named Ranges**
|
||||
Templates often use named ranges for key inputs and outputs. Before entering data:
|
||||
- Review existing named ranges in the template (Formulas → Name Manager in Excel)
|
||||
- Common named ranges include: Revenue growth rates, cost percentages, key outputs (Net Income, EBITDA, Total Debt, Cash), scenario selector cell
|
||||
- Ensure inputs are entered in cells that feed into these named ranges
|
||||
|
||||
### Projection Period
|
||||
- Templates typically project 5 years forward from last historical year
|
||||
- Verify historical (A) vs. projected (E) columns are clearly separated
|
||||
- Confirm columns use fiscal year notation (e.g., FY2024A, FY2025E)
|
||||
|
||||
## Margin Analysis
|
||||
|
||||
**Note: The following margin analysis should only be performed if prompted by the user or if the template explicitly requires it. If no prompt is given, skip this section.**
|
||||
|
||||
Calculate and display profitability margins on the Income Statement (IS) tab to track operational efficiency and enable peer comparison.
|
||||
|
||||
### Core Margins to Include
|
||||
|
||||
| Margin | Formula | What It Measures |
|
||||
|--------|---------|------------------|
|
||||
| Gross Margin | Gross Profit / Revenue | Pricing power, production efficiency |
|
||||
| EBITDA Margin | EBITDA / Revenue | Core operating profitability |
|
||||
| EBIT Margin | EBIT / Revenue | Operating profitability after D&A |
|
||||
| Net Income Margin | Net Income / Revenue | Bottom-line profitability |
|
||||
|
||||
### Income Statement Layout with Margins
|
||||
|
||||
Display margin percentages directly below each profit line item:
|
||||
- Gross Margin % below Gross Profit
|
||||
- EBIT Margin % below EBIT
|
||||
- EBITDA Margin % below EBITDA
|
||||
- Net Income Margin % below Net Income
|
||||
|
||||
## Credit Metrics
|
||||
|
||||
**Note: The following Credit analysis should only be performed if prompted by the user or if the template explicitly requires it. If no prompt is given, skip this section.**
|
||||
|
||||
Calculate and display credit/leverage metrics on the Balance Sheet (BS) tab to assess financial health, debt capacity, and covenant compliance.
|
||||
|
||||
### Core Credit Metrics to Include
|
||||
|
||||
| Metric | Formula | What It Measures |
|
||||
|--------|---------|------------------|
|
||||
| Total Debt / EBITDA | Total Debt / LTM EBITDA | Leverage multiple |
|
||||
| Net Debt / EBITDA | (Total Debt - Cash) / LTM EBITDA | Leverage net of cash |
|
||||
| Interest Coverage | EBITDA / Interest Expense | Ability to service debt |
|
||||
| Debt / Total Cap | Total Debt / (Total Debt + Equity) | Capital structure |
|
||||
| Debt / Equity | Total Debt / Total Equity | Financial leverage |
|
||||
| Current Ratio | Current Assets / Current Liabilities | Short-term liquidity |
|
||||
| Quick Ratio | (Current Assets - Inventory) / Current Liabilities | Immediate liquidity |
|
||||
|
||||
### Credit Metric Hierarchy Checks
|
||||
|
||||
Validate that Upside shows strongest credit profile:
|
||||
- Leverage: Upside < Base < Downside (lower is better)
|
||||
- Coverage: Upside > Base > Downside (higher is better)
|
||||
- Liquidity: Upside > Base > Downside (higher is better)
|
||||
|
||||
### Covenant Compliance Tracking
|
||||
|
||||
If debt covenants are known, add explicit compliance checks comparing actual metrics to covenant thresholds.
|
||||
|
||||
## Scenario Analysis (Base / Upside / Downside)
|
||||
|
||||
Use a scenario toggle (dropdown) in the Assumptions tab with CHOOSE or INDEX/MATCH formulas.
|
||||
|
||||
| Scenario | Description |
|
||||
|----------|-------------|
|
||||
| Base Case | Management guidance or consensus estimates |
|
||||
| Upside Case | Above-guidance growth, margin expansion |
|
||||
| Downside Case | Below-trend growth, margin compression |
|
||||
|
||||
**Key Drivers to Sensitize**: Revenue growth, Gross margin, SG&A %, DSO/DIO/DPO, CapEx %, Interest rate, Tax rate.
|
||||
|
||||
**Scenario Audit Checks**: Toggle switches all statements, BS balances in all scenarios, Cash ties out, Hierarchy holds (Upside > Base > Downside for NI, EBITDA, FCF, margins).
|
||||
|
||||
## SEC Filings Data Extraction
|
||||
|
||||
If the template specifically requires pulling data from SEC filings (10-K, 10-Q), see [references/sec-filings.md](references/sec-filings.md) for detailed extraction guidance. This reference is only needed when populating templates with public company data from regulatory filings.
|
||||
|
||||
## Completing Model Templates
|
||||
|
||||
This section provides general guidance for completing any 3-statement financial model template while preserving existing formulas and ensuring data integrity.
|
||||
|
||||
### Step 1: Analyze the Template Structure
|
||||
|
||||
Before entering any data, thoroughly review the template to understand its architecture:
|
||||
|
||||
**Identify Input vs. Formula Cells**
|
||||
- Look for visual cues (font color, cell shading) that distinguish input cells from formula cells
|
||||
- Common conventions: Blue font = inputs, Black font = formulas, Green font = links to other sheets
|
||||
- Use Excel's Trace Precedents/Dependents (Formulas → Trace Precedents) to understand cell relationships
|
||||
- Check for named ranges that may control key inputs (Formulas → Name Manager)
|
||||
|
||||
**Map the Template's Flow**
|
||||
- Identify which tabs feed into others (e.g., Assumptions → IS → BS → CF)
|
||||
- Note any supporting schedules and their linkages to main statements
|
||||
- Document the template's specific line items and structure before populating
|
||||
|
||||
### Step 2: Filling in Data Without Breaking Formulas
|
||||
|
||||
**Golden Rules for Data Entry**
|
||||
|
||||
| Rule | Description |
|
||||
|------|-------------|
|
||||
| Only edit input cells | Never overwrite cells containing formulas unless intentionally replacing the formula |
|
||||
| Preserve cell references | When copying data, use Paste Values (Ctrl+Shift+V) to avoid overwriting formulas with source formatting |
|
||||
| Match the template's units | Verify if template uses thousands, millions, or actual values before entering data |
|
||||
| Respect sign conventions | Follow the template's existing sign convention (e.g., expenses as positive or negative) |
|
||||
| Check for circular references | If the template uses iterative calculations, ensure Enable Iterative Calculation is turned on |
|
||||
|
||||
**Safe Data Entry Process**
|
||||
1. Identify the exact cells designated for input (usually highlighted or labeled)
|
||||
2. Enter historical data first, then verify formulas are calculating correctly for those periods
|
||||
3. Enter assumption drivers that feed forecast calculations
|
||||
4. Review calculated outputs to confirm formulas are working as intended
|
||||
5. If a formula cell must be modified, document the original formula before making changes
|
||||
|
||||
**Handling Pre-Built Formulas**
|
||||
- If formulas reference cells you haven't populated yet, expect temporary errors (#REF!, #DIV/0!) until all inputs are complete
|
||||
- When formulas produce unexpected results, trace precedents to identify missing or incorrect inputs
|
||||
- Never delete rows/columns without checking for formula dependencies across all tabs
|
||||
|
||||
### Step 3: Validating Formulas
|
||||
|
||||
**Formula Integrity Checks**
|
||||
|
||||
Before relying on template outputs, validate that formulas are functioning correctly:
|
||||
|
||||
| Check Type | Method |
|
||||
|------------|--------|
|
||||
| Trace precedents | Select a formula cell → Formulas → Trace Precedents to verify it references correct inputs |
|
||||
| Trace dependents | Verify key inputs flow to expected output cells |
|
||||
| Evaluate formula | Use Formulas → Evaluate Formula to step through complex calculations |
|
||||
| Check for hardcodes | Projection formulas should reference assumptions, not contain hardcoded values |
|
||||
| Test with known values | Input simple test values to verify formulas produce expected results |
|
||||
| Cross-tab consistency | Ensure the same formula logic applies across all projection periods |
|
||||
|
||||
**Common Formula Issues to Watch For**
|
||||
- Mixed absolute/relative references causing incorrect results when copied across periods
|
||||
- Broken links to external files or deleted ranges (#REF! errors)
|
||||
- Division by zero in early periods before revenue ramps (#DIV/0! errors)
|
||||
- Circular reference warnings (may be intentional for interest calculations)
|
||||
- Inconsistent formulas across projection columns (use Ctrl+\ to find differences)
|
||||
|
||||
**Validating Cross-Tab Linkages**
|
||||
- Confirm values that appear on multiple tabs are linked (not duplicated)
|
||||
- Verify schedule totals tie to corresponding line items on main statements
|
||||
- Check that period labels align across all tabs
|
||||
|
||||
### Step 4: Quality Checks by Sheet
|
||||
|
||||
Perform these validation checks on each sheet after populating the template:
|
||||
|
||||
**Income Statement (IS) Quality Checks**
|
||||
- Revenue figures match source data for historical periods
|
||||
- All expense line items sum to reported totals
|
||||
- Subtotals (Gross Profit, EBIT, EBT, Net Income) calculate correctly
|
||||
- Tax calculation logic is appropriate (handles losses correctly)
|
||||
- Forecast drivers reference assumptions tab (no hardcodes)
|
||||
- Period-over-period changes are directionally reasonable
|
||||
|
||||
**Balance Sheet (BS) Quality Checks**
|
||||
- Assets = Liabilities + Equity for every period (primary check)
|
||||
- Cash balance matches Cash Flow Statement ending cash
|
||||
- Working capital accounts tie to supporting schedules (if applicable)
|
||||
- Retained Earnings rolls forward correctly: Prior RE + Net Income - Dividends +/- Adjustments = Ending RE
|
||||
- Debt balances tie to debt schedule (if applicable)
|
||||
- All balance sheet items have appropriate signs (assets positive, most liabilities positive)
|
||||
|
||||
**Cash Flow Statement (CF) Quality Checks**
|
||||
- Net Income at top of CFO matches Income Statement Net Income
|
||||
- Non-cash add-backs (D&A, SBC, etc.) tie to their source schedules/statements
|
||||
- Working capital changes have correct signs (increase in asset = use of cash = negative)
|
||||
- CapEx ties to PP&E schedule or fixed asset roll-forward
|
||||
- Financing activities tie to changes in debt and equity accounts on BS
|
||||
- Ending Cash matches Balance Sheet Cash
|
||||
- Beginning Cash equals prior period Ending Cash
|
||||
|
||||
**Supporting Schedule Quality Checks**
|
||||
- Opening balances equal prior period closing balances
|
||||
- Roll-forward logic is complete (Beginning + Additions - Deductions = Ending)
|
||||
- Schedule totals tie to main statement line items
|
||||
- Assumptions used in calculations match Assumptions tab
|
||||
|
||||
### Step 5: Cross-Statement Integrity Checks
|
||||
|
||||
After validating individual sheets, confirm the three statements are properly integrated:
|
||||
|
||||
| Check | Formula | Expected Result |
|
||||
|-------|---------|-----------------|
|
||||
| Balance Sheet Balance | Assets - Liabilities - Equity | = 0 |
|
||||
| Cash Tie-Out | CF Ending Cash - BS Cash | = 0 |
|
||||
| Net Income Link | IS Net Income - CF Starting Net Income | = 0 |
|
||||
| Retained Earnings | Prior RE + NI - Dividends - BS Ending RE | = 0 (adjust for SBC/other items as needed) |
|
||||
|
||||
### Step 6: Final Review
|
||||
|
||||
Before considering the model complete:
|
||||
- Toggle through all scenarios (if applicable) to verify checks pass in each case
|
||||
- Review all #REF!, #DIV/0!, #VALUE!, and #NAME? errors and resolve or document
|
||||
- Confirm all input cells have been populated (search for placeholder values)
|
||||
- Verify units are consistent across all tabs
|
||||
- Save a clean version before making any additional modifications
|
||||
|
||||
## Model Validation and Audit
|
||||
|
||||
This section consolidates all validation checks and audit procedures for completed templates.
|
||||
|
||||
### Core Linkages (Must Always Hold)
|
||||
|
||||
See [references/formulas.md](references/formulas.md) for all formula details.
|
||||
|
||||
| Check | Formula | Expected Result |
|
||||
|-------|---------|-----------------|
|
||||
| Balance Sheet Balance | Assets - Liabilities - Equity | = 0 |
|
||||
| Cash Tie-Out | CF Ending Cash - BS Cash | = 0 |
|
||||
| Cash Monthly vs Annual | Closing Cash (Monthly) - Closing Cash (Annual) | = 0 |
|
||||
| Net Income Link | IS Net Income - CF Starting Net Income | = 0 |
|
||||
| Retained Earnings | Prior RE + NI + SBC - Dividends - BS Ending RE | = 0 |
|
||||
| Equity Financing | ΔCommon Stock/APIC (BS) - Equity Issuance (CFF) | = 0 |
|
||||
| Year 0 Equity | Equity Raised (Year 0) - Beginning Equity Capital (Year 1) | = 0 |
|
||||
|
||||
### Sign Convention Reference
|
||||
|
||||
| Statement | Item | Sign Convention |
|
||||
|-----------|------|-----------------|
|
||||
| CFO | D&A, SBC | Positive (add-back) |
|
||||
| CFO | ΔAR (increase) | Negative (use of cash) |
|
||||
| CFO | ΔAP (increase) | Positive (source of cash) |
|
||||
| CFI | CapEx | Negative |
|
||||
| CFF | Debt issuance | Positive |
|
||||
| CFF | Debt repayments | Negative |
|
||||
| CFF | Dividends | Negative |
|
||||
|
||||
### Circular Reference Handling
|
||||
|
||||
Interest expense creates circularity: Interest → Net Income → Cash → Debt Balance → Interest
|
||||
|
||||
Enable iterative calculation in Excel: File → Options → Formulas → Enable iterative calculation. Set maximum iterations to 100, maximum change to 0.001. Add a circuit breaker toggle in Assumptions tab.
|
||||
|
||||
### Check Categories
|
||||
|
||||
**Section 1: Currency Consistency**
|
||||
- Currency identified and documented in Assumptions
|
||||
- All tabs use consistent currency symbol and scale
|
||||
- Units row matches model currency
|
||||
|
||||
**Section 2: Balance Sheet Integrity**
|
||||
- Assets = Liabilities + Equity (for each period)
|
||||
- Formula: Assets - Liabilities - Equity (must = 0)
|
||||
|
||||
**Section 3: Cash Flow Integrity**
|
||||
- Cash ties to BS (CF Ending Cash = BS Cash)
|
||||
- Cash Monthly vs Annual: Closing Cash (Monthly) = Closing Cash (Annual)
|
||||
- NI ties to IS (CF Net Income = IS Net Income)
|
||||
- D&A ties to schedule
|
||||
- SBC ties to IS
|
||||
- ΔAR, ΔInventory, ΔAP tie to WC schedule
|
||||
- CapEx ties to DA schedule
|
||||
|
||||
**Section 4: Retained Earnings**
|
||||
- RE roll-forward check: Prior RE + NI + SBC - Dividends = Ending RE
|
||||
- Show component breakdown for debugging
|
||||
|
||||
**Section 5: Working Capital**
|
||||
- AR, Inventory, AP tie to BS
|
||||
- DSO, DIO, DPO reasonability checks (flag if outside normal ranges)
|
||||
|
||||
**Section 6: Debt Schedule**
|
||||
- Total Debt ties to BS (Current + LT Debt)
|
||||
- Interest calculation ties to IS
|
||||
|
||||
**Section 6b: Equity Financing**
|
||||
- Equity issuance proceeds tie to BS Common Stock/APIC increase
|
||||
- Cash increase from equity = Equity account increase (must balance)
|
||||
- Equity Raise Tie-Out: ΔCommon Stock/APIC (BS) = Equity Issuance (CFF) (must = 0)
|
||||
- Year 0 Equity Tie-Out: Equity Raised (Year 0) = Beginning Equity Capital (Year 1)
|
||||
|
||||
**Section 6c: NOL Schedule**
|
||||
- Beginning NOL (Year 1 / Formation) = 0 (new business starts with zero NOL)
|
||||
- NOL increases only when EBT < 0 (losses must be realized to generate NOL)
|
||||
- DTA ties to BS (NOL Schedule DTA = BS Deferred Tax Asset)
|
||||
- NOL utilization ≤ 80% of EBT (post-2017 federal limitation)
|
||||
- NOL balance is non-negative (cannot utilize more than available)
|
||||
- NOL generated only when EBT < 0
|
||||
- Tax expense = 0 when taxable income ≤ 0
|
||||
|
||||
**Section 7: Scenario Hierarchy**
|
||||
- Absolute metrics: Upside > Base > Downside (NI, EBITDA, FCF)
|
||||
- Margins: Upside > Base > Downside (GM%, EBITDA%, NI%)
|
||||
- Credit metrics: Upside < Base < Downside for leverage (inverted)
|
||||
|
||||
**Section 8: Formula Integrity**
|
||||
- COGS, S&M, G&A, R&D, SBC driven by % of Revenue (no hardcodes)
|
||||
- Consistent formulas across projection years
|
||||
- No #REF!, #DIV/0!, #VALUE! errors
|
||||
|
||||
**Section 9: Credit Metric Thresholds**
|
||||
- Flag metrics as Green/Yellow/Red based on covenant thresholds
|
||||
- Summary of any red flags
|
||||
|
||||
### Master Check Formula
|
||||
|
||||
Aggregate all section statuses into a single master check:
|
||||
- If all sections pass → "✓ ALL CHECKS PASS"
|
||||
- If any section fails → "✗ ERRORS DETECTED - REVIEW BELOW"
|
||||
|
||||
### Quick Debug Workflow
|
||||
|
||||
When Master Status shows errors:
|
||||
1. Scroll to find red-highlighted sections
|
||||
2. Identify which check category has failures
|
||||
3. Navigate to source tab to investigate
|
||||
4. Fix the underlying issue
|
||||
5. Return to Checks tab to verify resolution
|
||||
|
||||
|
||||
## Data sources — MCP first, web fallback
|
||||
|
||||
Many passages below say "use the S&P Kensho MCP / Daloopa MCP / FactSet MCP". Those are commercial financial-data MCPs from the original Cowork plugin context. In Hermes:
|
||||
|
||||
- **If you have any structured financial-data MCP configured** (Hermes supports MCP — see `native-mcp` skill), prefer it for point-in-time comps, precedent transactions, and filings.
|
||||
- **Otherwise**, fall back to:
|
||||
- `web_search` / `web_extract` against SEC EDGAR (`https://www.sec.gov/cgi-bin/browse-edgar`) for US filings
|
||||
- Company IR pages for press releases, earnings decks
|
||||
- `browser_navigate` for interactive data portals
|
||||
- User-provided data (explicitly ask when the context doesn't have it)
|
||||
- **Never fabricate**. If a multiple, precedent, or filing number can't be sourced, flag the cell as `[UNSOURCED]` and surface it to the user.
|
||||
|
||||
## Attribution
|
||||
|
||||
This skill is adapted from Anthropic's Claude for Financial Services plugin suite (Apache-2.0). The Office-JS / Cowork live-Excel paths have been removed; this version targets headless openpyxl via the `excel-author` skill's conventions. Original: https://github.com/anthropics/financial-services
|
||||
@@ -0,0 +1,118 @@
|
||||
# Formatting Standards Reference
|
||||
|
||||
| Element | Format |
|
||||
|---------|--------|
|
||||
| Hard-coded inputs | Blue font |
|
||||
| Formulas | Black font |
|
||||
| Links to other sheets | Green font |
|
||||
| Check cells | Red if error, green if balanced |
|
||||
| Negative values | Parentheses, not minus signs |
|
||||
| Currency | No decimals for large figures, 2 decimals for per-share |
|
||||
| Percentages | 1 decimal place |
|
||||
| Headers | Bold, bottom border |
|
||||
| Units row | Include units row below headers ($ millions, %, etc.) |
|
||||
|
||||
## Visual Separation Guidelines
|
||||
|
||||
- Thin vertical border between historical and projected columns
|
||||
- Thick bottom border after section totals (e.g., Total Assets)
|
||||
- Single bottom border for subtotals
|
||||
- Double bottom border for grand totals
|
||||
|
||||
## Total and Subtotal Row Formatting
|
||||
|
||||
All total and subtotal rows must use **bold font formatting** for their numerical values to clearly distinguish aggregated figures from individual line items.
|
||||
|
||||
### Income Statement (P&L) Tab
|
||||
| Row | Formatting |
|
||||
|-----|------------|
|
||||
| Gross Revenue | Bold |
|
||||
| Total Cost of Revenue | Bold |
|
||||
| Gross Profit | Bold |
|
||||
| Total SG&A | Bold |
|
||||
| EBITDA | Bold |
|
||||
| EBIT | Bold |
|
||||
| EBT | Bold |
|
||||
| Net Profit After Tax | Bold |
|
||||
|
||||
### Balance Sheet Tab
|
||||
| Row | Formatting |
|
||||
|-----|------------|
|
||||
| Total Current Assets | Bold |
|
||||
| Total Non-Current Assets | Bold |
|
||||
| Total Other Assets | Bold |
|
||||
| Total Assets | Bold |
|
||||
| Total Current Liabilities | Bold |
|
||||
| Total Non-Current Liabilities | Bold |
|
||||
| Total Equity | Bold |
|
||||
| Total Liabilities and Equity | Bold |
|
||||
|
||||
### Cash Flow Statement Tab
|
||||
| Row | Formatting |
|
||||
|-----|------------|
|
||||
| Cash Generated from Operations Before Working Capital Changes | Bold |
|
||||
| Total Working Capital Changes | Bold |
|
||||
| Net Cash Generated from Operations | Bold |
|
||||
| Net Cash Flow from Investing Activities | Bold |
|
||||
| Net Cash Flow from Financing Activities | Bold |
|
||||
| Closing Cash Balance | Bold |
|
||||
|
||||
**Note:** This list is non-exhaustive. Apply bold formatting to any row that represents a total, subtotal, or summary calculation across the model.
|
||||
|
||||
## Balance Sheet Check Row Formatting
|
||||
|
||||
The Balance Sheet check row (below Total Liabilities and Equity) uses conditional number formatting that displays non-zero values in red. When the balance sheet balances correctly (check = 0), the values display in black or standard formatting.
|
||||
|
||||
| Check Value | Font Color |
|
||||
|-------------|------------|
|
||||
| = 0 (balanced) | Black (standard) |
|
||||
| ≠ 0 (error) | Red |
|
||||
|
||||
**Implementation:** Apply custom number format `[Red][<>0]0.00;[Red][<>0](0.00);0.00` or use Excel conditional formatting with the rule "Cell Value ≠ 0" → Red font.
|
||||
|
||||
## Margin Row Formatting
|
||||
|
||||
| Element | Format |
|
||||
|---------|--------|
|
||||
| Margin % rows | Indent, italics, 1 decimal place |
|
||||
| Positive trend | No special formatting (or subtle green) |
|
||||
| Negative trend | Flag for review (subtle yellow) |
|
||||
| Below peer average | Consider highlighting for discussion |
|
||||
|
||||
## Credit Metric Formatting
|
||||
|
||||
| Element | Format |
|
||||
|---------|--------|
|
||||
| Leverage multiples | 1 decimal with "x" suffix (e.g., 2.5x) |
|
||||
| Percentages | 1 decimal with "%" suffix |
|
||||
| Net Debt negative | Parentheses, indicates net cash position |
|
||||
| Section header | Bold, "CREDIT METRICS" |
|
||||
| Separator line | Thin border above credit metrics section |
|
||||
|
||||
## Credit Metric Threshold Colors
|
||||
|
||||
| Metric | Green | Yellow | Red |
|
||||
|--------|-------|--------|-----|
|
||||
| Total Debt / EBITDA | < 2.5x | 2.5x-4.0x | > 4.0x |
|
||||
| Net Debt / EBITDA | < 2.0x | 2.0x-3.5x | > 3.5x |
|
||||
| Interest Coverage | > 4.0x | 2.5x-4.0x | < 2.5x |
|
||||
| Debt / Total Cap | < 40% | 40%-60% | > 60% |
|
||||
| Current Ratio | > 1.5x | 1.0x-1.5x | < 1.0x |
|
||||
| Quick Ratio | > 1.0x | 0.75x-1.0x | < 0.75x |
|
||||
|
||||
## Conditional Formatting for Checks Tab
|
||||
|
||||
- Cell contains pass indicator → Green fill
|
||||
- Cell contains fail indicator → Red fill
|
||||
- Cell contains warning → Yellow fill
|
||||
- Difference cells = 0 → Light green fill
|
||||
- Difference cells ≠ 0 → Light red fill
|
||||
|
||||
## Margin Reasonability Flags
|
||||
|
||||
- Gross Margin < 0% → ERROR: Review COGS
|
||||
- Gross Margin > 80% → WARNING: Verify revenue/COGS
|
||||
- EBITDA Margin < 0% → FLAG: Operating losses
|
||||
- EBITDA Margin > 50% → WARNING: Unusually high
|
||||
- Net Margin < 0% → FLAG: Net losses (may be acceptable in growth phase)
|
||||
- Net Margin > Gross Margin → ERROR: Formula issue
|
||||
@@ -0,0 +1,292 @@
|
||||
# Formula Reference
|
||||
|
||||
**IMPORTANT:** Use the formulas outlined in this reference document unless otherwise specified by the user.
|
||||
|
||||
---
|
||||
|
||||
## Core Linkages
|
||||
|
||||
```
|
||||
Balance Sheet: Assets = Liabilities + Equity
|
||||
Net Income: IS Net Income → CF Operations (starting point)
|
||||
Cash Flow: ΔCash = CFO + CFI + CFF
|
||||
Cash Tie-Out: Ending Cash (CF) = Cash (BS Asset)
|
||||
Cash Monthly/Annual: Closing Cash (Monthly) = Closing Cash (Annual)
|
||||
Retained Earnings: Prior RE + Net Income - Dividends = Ending RE
|
||||
Equity Raise: ΔCommon Stock/APIC (BS) = Equity Issuance (CFF)
|
||||
Year 0 Equity: Equity Raised (Year 0) = Beginning Equity (Year 1)
|
||||
```
|
||||
|
||||
## Gross Profit Calculation
|
||||
|
||||
**IMPORTANT:** Gross Profit must be calculated from Net Revenue, not Gross Revenue.
|
||||
|
||||
```
|
||||
Net Revenue - Cost of Revenue = Gross Profit
|
||||
```
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| Gross Revenue | Total revenue before any deductions |
|
||||
| Net Revenue | Gross Revenue - Returns - Allowances - Discounts |
|
||||
| Cost of Revenue | Direct costs attributable to production of goods/services sold |
|
||||
| Gross Profit | Net Revenue - Cost of Revenue |
|
||||
|
||||
**Note:** Always use Net Revenue (also called "Net Sales" or simply "Revenue" on most financial statements) as the starting point for profitability calculations. Gross Revenue overstates the true top-line performance.
|
||||
|
||||
## Margin Formulas
|
||||
|
||||
```
|
||||
Gross Margin % = Gross Profit / Net Revenue
|
||||
EBITDA = EBIT + D&A (or = Gross Profit - OpEx)
|
||||
EBITDA Margin % = EBITDA / Net Revenue
|
||||
EBIT Margin % = EBIT / Net Revenue
|
||||
Net Income Margin % = Net Income / Net Revenue
|
||||
```
|
||||
|
||||
## Credit Metric Formulas
|
||||
|
||||
```
|
||||
Total Debt = Current Portion of Debt + Long-Term Debt
|
||||
Net Debt = Total Debt - Cash
|
||||
Total Debt / EBITDA = Total Debt / EBITDA (from IS)
|
||||
Net Debt / EBITDA = Net Debt / EBITDA (from IS)
|
||||
Interest Coverage = EBITDA / Interest Expense (from IS)
|
||||
Net Int Exp % Debt = Net Interest Expense / Long-Term Debt
|
||||
Debt / Total Cap = Total Debt / (Total Debt + Total Equity)
|
||||
Debt / Equity = Total Debt / Total Equity
|
||||
Current Ratio = Total Current Assets / Total Current Liabilities
|
||||
Quick Ratio = (Total Current Assets - Inventory) / Total Current Liabilities
|
||||
```
|
||||
|
||||
## Forecast Formulas (% of Net Revenue Method)
|
||||
|
||||
```
|
||||
Cost of Revenue (Forecast) = Net Revenue × Cost of Revenue % Assumption
|
||||
S&M (Forecast) = Net Revenue × S&M % Assumption
|
||||
G&A (Forecast) = Net Revenue × G&A % Assumption
|
||||
R&D (Forecast) = Net Revenue × R&D % Assumption
|
||||
SBC (Forecast) = Net Revenue × SBC % Assumption
|
||||
```
|
||||
|
||||
## Working Capital Formulas
|
||||
|
||||
```
|
||||
Accounts Receivable
|
||||
Prior AR
|
||||
+ Revenue (from IS)
|
||||
- Cash Collections (plug)
|
||||
= Ending AR
|
||||
DSO = (AR / Revenue) × 365
|
||||
|
||||
Inventory
|
||||
Prior Inventory
|
||||
+ Purchases (plug)
|
||||
- COGS (from IS)
|
||||
= Ending Inventory
|
||||
DIO = (Inventory / COGS) × 365
|
||||
|
||||
Accounts Payable
|
||||
Prior AP
|
||||
+ Purchases (from Inventory calc)
|
||||
- Cash Payments (plug)
|
||||
= Ending AP
|
||||
DPO = (AP / COGS) × 365
|
||||
|
||||
Net Working Capital = AR + Inventory - AP
|
||||
ΔWC = Current NWC - Prior NWC
|
||||
```
|
||||
|
||||
## D&A Schedule Formulas
|
||||
|
||||
```
|
||||
Beginning PP&E (Gross)
|
||||
+ CapEx
|
||||
= Ending PP&E (Gross)
|
||||
|
||||
Beginning Accumulated Depreciation
|
||||
+ Depreciation Expense
|
||||
= Ending Accumulated Depreciation
|
||||
|
||||
PP&E (Net) = Gross PP&E - Accumulated Depreciation
|
||||
```
|
||||
|
||||
## Debt Schedule Formulas
|
||||
|
||||
```
|
||||
Beginning Debt Balance
|
||||
+ New Borrowings
|
||||
- Repayments
|
||||
= Ending Debt Balance
|
||||
|
||||
Interest Expense = Avg Debt Balance × Interest Rate
|
||||
(Use beginning balance to avoid circularity, or iterate if circular refs enabled)
|
||||
```
|
||||
|
||||
## Retained Earnings Formula
|
||||
|
||||
```
|
||||
Beginning Retained Earnings
|
||||
+ Net Income (from IS)
|
||||
+ Stock-Based Compensation (SBC) (from IS)
|
||||
- Dividends
|
||||
= Ending Retained Earnings
|
||||
```
|
||||
|
||||
## NOL (Net Operating Loss) Schedule Formulas
|
||||
|
||||
```
|
||||
NOL CARRYFORWARD SCHEDULE
|
||||
|
||||
Beginning NOL Balance (Year 1 / Formation = 0)
|
||||
+ NOL Generated (if EBT < 0, then ABS(EBT), else 0)
|
||||
- NOL Utilized (limited by taxable income and utilization cap)
|
||||
= Ending NOL Balance
|
||||
|
||||
STARTING BALANCE RULE
|
||||
|
||||
For a new business or first modeled period:
|
||||
Beginning NOL Balance = 0
|
||||
NOL can only increase through realized losses (EBT < 0)
|
||||
NOL cannot be created from thin air or assumed
|
||||
|
||||
NOL UTILIZATION CALCULATION
|
||||
|
||||
Pre-Tax Income (EBT)
|
||||
If EBT > 0:
|
||||
NOL Available = Beginning NOL Balance
|
||||
Utilization Limit = EBT × 80% (post-2017 federal limit)
|
||||
NOL Utilized = MIN(NOL Available, Utilization Limit)
|
||||
Taxable Income = EBT - NOL Utilized
|
||||
If EBT ≤ 0:
|
||||
NOL Utilized = 0
|
||||
Taxable Income = 0
|
||||
NOL Generated = ABS(EBT)
|
||||
|
||||
TAX CALCULATION WITH NOL
|
||||
|
||||
Taxes Payable = MAX(0, Taxable Income × Tax Rate)
|
||||
(Taxes cannot be negative; losses create NOL asset instead)
|
||||
|
||||
DEFERRED TAX ASSET (DTA) FOR NOL
|
||||
|
||||
DTA - NOL Carryforward = Ending NOL Balance × Tax Rate
|
||||
ΔDTA = Current DTA - Prior DTA
|
||||
(Increase in DTA = non-cash benefit on CF)
|
||||
(Decrease in DTA = non-cash expense on CF)
|
||||
```
|
||||
|
||||
## Balance Sheet Structure
|
||||
|
||||
```
|
||||
ASSETS
|
||||
Cash (from CF ending cash)
|
||||
Accounts Receivable (from WC)
|
||||
Inventory (from WC)
|
||||
Total Current Assets
|
||||
|
||||
PP&E, Net (from DA)
|
||||
Deferred Tax Asset - NOL (from NOL schedule)
|
||||
Total Non-Current Assets
|
||||
Total Assets
|
||||
|
||||
LIABILITIES
|
||||
Accounts Payable (from WC)
|
||||
Current Portion of Debt (from Debt)
|
||||
Total Current Liabilities
|
||||
|
||||
Long-Term Debt (from Debt)
|
||||
Total Liabilities
|
||||
|
||||
EQUITY
|
||||
Common Stock
|
||||
Retained Earnings (from RE schedule)
|
||||
Total Equity
|
||||
|
||||
CHECK: Assets - Liabilities - Equity = 0
|
||||
```
|
||||
|
||||
## Cash Flow Statement Structure
|
||||
|
||||
```
|
||||
CASH FROM OPERATIONS (CFO)
|
||||
Net Income (LINK: IS)
|
||||
+ D&A (LINK: DA schedule)
|
||||
+ Stock-Based Compensation (SBC) (LINK: IS or Assumptions)
|
||||
- ΔDTA (Deferred Tax Asset) (LINK: NOL schedule; increase in DTA = use of cash)
|
||||
- ΔAR (LINK: WC)
|
||||
- ΔInventory (LINK: WC)
|
||||
+ ΔAP (LINK: WC)
|
||||
= CFO
|
||||
|
||||
CASH FROM INVESTING (CFI)
|
||||
- CapEx (LINK: DA schedule)
|
||||
= CFI
|
||||
|
||||
CASH FROM FINANCING (CFF)
|
||||
+ Debt Issuance (LINK: Debt)
|
||||
- Debt Repayment (LINK: Debt)
|
||||
+ Equity Issuance (LINK: BS Common Stock/APIC)
|
||||
- Dividends (LINK: RE schedule)
|
||||
= CFF
|
||||
|
||||
Net Change in Cash = CFO + CFI + CFF
|
||||
Beginning Cash
|
||||
+ Net Change in Cash
|
||||
= Ending Cash (LINK TO: BS Cash)
|
||||
```
|
||||
|
||||
## Income Statement Structure
|
||||
|
||||
```
|
||||
Net Revenue
|
||||
Growth %
|
||||
(-) Cost of Revenue
|
||||
% of Net Revenue
|
||||
────────────────
|
||||
Gross Profit (= Net Revenue - Cost of Revenue)
|
||||
Gross Margin %
|
||||
|
||||
(-) S&M
|
||||
% of Net Revenue
|
||||
(-) G&A
|
||||
% of Net Revenue
|
||||
(-) R&D
|
||||
% of Net Revenue
|
||||
(-) D&A
|
||||
(-) SBC
|
||||
% of Net Revenue
|
||||
────────────────
|
||||
EBIT
|
||||
EBIT Margin %
|
||||
|
||||
EBITDA
|
||||
EBITDA Margin %
|
||||
|
||||
(-) Interest Expense
|
||||
────────────────
|
||||
EBT (Pre-Tax Income)
|
||||
(-) NOL Utilization (from NOL schedule, reduces taxable income)
|
||||
────────────────
|
||||
Taxable Income
|
||||
(-) Taxes (Taxable Income × Tax Rate)
|
||||
────────────────
|
||||
Net Income
|
||||
Net Income Margin %
|
||||
```
|
||||
|
||||
## Check Formulas
|
||||
|
||||
```
|
||||
BS Balance Check: = Assets - Liabilities - Equity (must = 0)
|
||||
Cash Tie-Out: = BS Cash - CF Ending Cash (must = 0)
|
||||
RE Roll-Forward: = Prior RE + NI + SBC - Div - BS RE (must = 0)
|
||||
DTA Tie-Out: = NOL Schedule DTA - BS DTA (must = 0)
|
||||
Equity Raise Tie-Out: = ΔCommon Stock/APIC (BS) - Equity Issuance (CFF) (must = 0)
|
||||
Year 0 Equity Tie-Out: = Equity Raised (Year 0) - Beginning Equity (Year 1) (must = 0)
|
||||
Cash Monthly vs Annual: = Closing Cash (Monthly) - Closing Cash (Annual) (must = 0)
|
||||
NOL Utilization Cap: = NOL Utilized ≤ EBT × 80% (must be TRUE for post-2017)
|
||||
NOL Non-Negative: = Ending NOL Balance ≥ 0 (must be TRUE)
|
||||
NOL Starting Balance: = Beginning NOL (Year 1) = 0 (must be TRUE for new business)
|
||||
NOL Accumulation: = NOL increases only when EBT < 0 (losses generate NOL)
|
||||
```
|
||||
@@ -0,0 +1,125 @@
|
||||
# SEC Filings Data Extraction Reference
|
||||
|
||||
**When to Use:** Only reference this file when a model template specifically requires pulling data from SEC filings (10-K, 10-Q). For templates that provide data directly or use other data sources, this reference is not needed.
|
||||
|
||||
---
|
||||
|
||||
## Extracting Data from SEC Filings (10-K / 10-Q)
|
||||
|
||||
When populating a model template with public company data, extract financials directly from SEC filings.
|
||||
|
||||
### Step 1: Locate the Filing
|
||||
|
||||
1. Use SEC EDGAR: `https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=[TICKER]&type=10-K`
|
||||
2. For quarterly data, use `type=10-Q`
|
||||
|
||||
### Step 2: Identify Filing Currency
|
||||
|
||||
Before extracting data, identify the reporting currency:
|
||||
- Check the cover page or header for reporting currency
|
||||
- Look at statement headers (e.g., "in thousands of U.S. dollars")
|
||||
- Review Note 1 (Summary of Significant Accounting Policies)
|
||||
|
||||
**Common Currency Indicators**
|
||||
|
||||
| Indicator | Currency |
|
||||
|-----------|----------|
|
||||
| $, USD | US Dollar |
|
||||
| €, EUR | Euro |
|
||||
| £, GBP | British Pound |
|
||||
| ¥, JPY | Japanese Yen |
|
||||
| ¥, CNY, RMB | Chinese Yuan |
|
||||
| CHF | Swiss Franc |
|
||||
| CAD, C$ | Canadian Dollar |
|
||||
|
||||
Set model currency to match filing; document in Assumptions tab.
|
||||
|
||||
### Step 3: Navigate to Financial Statements
|
||||
|
||||
Within the 10-K or 10-Q, locate:
|
||||
- **Item 8** (10-K) or **Item 1** (10-Q): Financial Statements
|
||||
- Key sections to extract:
|
||||
- Consolidated Statements of Operations (Income Statement)
|
||||
- Consolidated Balance Sheets
|
||||
- Consolidated Statements of Cash Flows
|
||||
- Notes to Financial Statements (for schedule details)
|
||||
|
||||
### Step 4: Data Extraction Mapping
|
||||
|
||||
**Income Statement (from Consolidated Statements of Operations)**
|
||||
|
||||
| Filing Line Item | Model Line Item |
|
||||
|------------------|-----------------|
|
||||
| Net revenues / Net sales | Revenue |
|
||||
| Cost of goods sold | COGS |
|
||||
| Selling, general and administrative | SG&A |
|
||||
| Depreciation and amortization | D&A |
|
||||
| Interest expense, net | Interest Expense |
|
||||
| Income tax expense | Taxes |
|
||||
| Net income | Net Income |
|
||||
|
||||
**Balance Sheet (from Consolidated Balance Sheets)**
|
||||
|
||||
| Filing Line Item | Model Line Item |
|
||||
|------------------|-----------------|
|
||||
| Cash and cash equivalents | Cash |
|
||||
| Accounts receivable, net | AR |
|
||||
| Inventories | Inventory |
|
||||
| Property, plant and equipment, net | PP&E (Net) |
|
||||
| Total assets | Total Assets |
|
||||
| Accounts payable | AP |
|
||||
| Short-term debt / Current portion of LT debt | Current Debt |
|
||||
| Long-term debt | LT Debt |
|
||||
| Retained earnings | Retained Earnings |
|
||||
| Total stockholders' equity | Total Equity |
|
||||
|
||||
**Cash Flow Statement (from Consolidated Statements of Cash Flows)**
|
||||
|
||||
| Filing Line Item | Model Line Item |
|
||||
|------------------|-----------------|
|
||||
| Net income | Net Income |
|
||||
| Depreciation and amortization | D&A |
|
||||
| Changes in accounts receivable | ΔAR |
|
||||
| Changes in inventories | ΔInventory |
|
||||
| Changes in accounts payable | ΔAP |
|
||||
| Capital expenditures | CapEx |
|
||||
| Proceeds from issuance of common stock | Equity Issuance |
|
||||
| Proceeds from / Repayments of debt | Debt activity |
|
||||
| Dividends paid | Dividends |
|
||||
|
||||
### Step 5: Extract Supporting Detail from Notes
|
||||
|
||||
For schedules, pull from Notes to Financial Statements:
|
||||
- **Note: Debt** → Maturity schedule, interest rates, covenants
|
||||
- **Note: Property, Plant & Equipment** → Gross PP&E, accumulated depreciation, useful lives
|
||||
- **Note: Revenue** → Segment breakdowns, geographic splits
|
||||
- **Note: Leases** → Operating vs. finance lease obligations
|
||||
|
||||
### Step 6: Historical Data Requirements
|
||||
|
||||
Extract 3 years of historical data minimum:
|
||||
- 10-K provides 3 years of IS/CF, 2 years of BS
|
||||
- For 3rd year BS, pull from prior year's 10-K
|
||||
- Use 10-Qs to fill in quarterly granularity if needed
|
||||
|
||||
### Data Extraction Checklist
|
||||
|
||||
- Identify reporting currency and scale (thousands, millions)
|
||||
- 3 years historical Income Statement
|
||||
- 3 years historical Cash Flow Statement
|
||||
- 3 years historical Balance Sheet
|
||||
- Verify IS Net Income = CF starting Net Income (each year)
|
||||
- Verify BS Cash = CF Ending Cash (each year)
|
||||
- Extract debt maturity schedule from notes
|
||||
- Extract D&A detail or useful life assumptions
|
||||
- Note any non-recurring / one-time items to normalize
|
||||
|
||||
### Handling Common Filing Variations
|
||||
|
||||
| Variation | How to Handle |
|
||||
|-----------|---------------|
|
||||
| D&A embedded in COGS/SG&A | Pull D&A from Cash Flow Statement |
|
||||
| "Other" line items are material | Check notes for breakdown |
|
||||
| Restatements | Use restated figures, note in assumptions |
|
||||
| Fiscal year ≠ calendar year | Label with fiscal year end (e.g., FYE Jan 2025) |
|
||||
| Non-USD reporting currency | Adapt model currency to match filing |
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user