Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0d814cd115 |
@@ -5,7 +5,6 @@
|
||||
|
||||
# Dependencies
|
||||
node_modules
|
||||
.venv
|
||||
|
||||
# CI/CD
|
||||
.github
|
||||
|
||||
@@ -24,15 +24,6 @@
|
||||
# Optional base URL override (default: Google's OpenAI-compatible endpoint)
|
||||
# GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
|
||||
|
||||
# =============================================================================
|
||||
# LLM PROVIDER (Ollama Cloud)
|
||||
# =============================================================================
|
||||
# Cloud-hosted open models via Ollama's OpenAI-compatible endpoint.
|
||||
# Get your key at: https://ollama.com/settings
|
||||
# OLLAMA_API_KEY=your_ollama_key_here
|
||||
# Optional base URL override (default: https://ollama.com/v1)
|
||||
# OLLAMA_BASE_URL=https://ollama.com/v1
|
||||
|
||||
# =============================================================================
|
||||
# LLM PROVIDER (z.ai / GLM)
|
||||
# =============================================================================
|
||||
@@ -52,15 +43,6 @@
|
||||
# KIMI_BASE_URL=https://api.kimi.com/coding/v1 # Default for sk-kimi- keys
|
||||
# KIMI_BASE_URL=https://api.moonshot.ai/v1 # For legacy Moonshot keys
|
||||
# KIMI_BASE_URL=https://api.moonshot.cn/v1 # For Moonshot China keys
|
||||
# KIMI_CN_API_KEY= # Dedicated Moonshot China key
|
||||
|
||||
# =============================================================================
|
||||
# LLM PROVIDER (Arcee AI)
|
||||
# =============================================================================
|
||||
# Arcee AI provides access to Trinity models (trinity-mini, trinity-large-*)
|
||||
# Get an Arcee key at: https://chat.arcee.ai/
|
||||
# ARCEEAI_API_KEY=
|
||||
# ARCEE_BASE_URL= # Override default base URL
|
||||
|
||||
# =============================================================================
|
||||
# LLM PROVIDER (MiniMax)
|
||||
@@ -154,10 +136,6 @@
|
||||
# Only override here if you need to force a backend without touching config.yaml:
|
||||
# TERMINAL_ENV=local
|
||||
|
||||
# Override the container runtime binary (e.g. to use Podman instead of Docker).
|
||||
# Useful on systems where Docker's storage driver is broken or unavailable.
|
||||
# HERMES_DOCKER_BINARY=/usr/local/bin/podman
|
||||
|
||||
# Container images (for singularity/docker/modal backends)
|
||||
# TERMINAL_DOCKER_IMAGE=nikolaik/python-nodejs:python3.11-nodejs20
|
||||
# TERMINAL_SINGULARITY_IMAGE=docker://nikolaik/python-nodejs:python3.11-nodejs20
|
||||
|
||||
@@ -1,5 +1 @@
|
||||
watch_file pyproject.toml uv.lock
|
||||
watch_file ui-tui/package-lock.json ui-tui/package.json
|
||||
watch_file flake.nix flake.lock nix/devShell.nix nix/tui.nix nix/package.nix nix/python.nix
|
||||
|
||||
use flake
|
||||
|
||||
@@ -1,2 +0,0 @@
|
||||
# Auto-generated files — collapse diffs and exclude from language stats
|
||||
web/package-lock.json linguist-generated=true
|
||||
@@ -11,7 +11,6 @@ body:
|
||||
**Before submitting**, please:
|
||||
- [ ] Search [existing issues](https://github.com/NousResearch/hermes-agent/issues) to avoid duplicates
|
||||
- [ ] Update to the latest version (`hermes update`) and confirm the bug still exists
|
||||
- [ ] Run `hermes debug share` and paste the links below (see Debug Report section)
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
@@ -83,25 +82,6 @@ body:
|
||||
- Slack
|
||||
- WhatsApp
|
||||
|
||||
- type: textarea
|
||||
id: debug-report
|
||||
attributes:
|
||||
label: Debug Report
|
||||
description: |
|
||||
Run `hermes debug share` from your terminal and paste the links it prints here.
|
||||
This uploads your system info, config, and recent logs to a paste service automatically.
|
||||
|
||||
If you're in an interactive chat session, you can also use the `/debug` slash command — it does the same thing.
|
||||
|
||||
If the upload fails, run `hermes debug share --local` and paste the output directly.
|
||||
placeholder: |
|
||||
Report https://paste.rs/abc123
|
||||
agent.log https://paste.rs/def456
|
||||
gateway.log https://paste.rs/ghi789
|
||||
render: shell
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: os
|
||||
attributes:
|
||||
@@ -117,6 +97,8 @@ body:
|
||||
label: Python Version
|
||||
description: Output of `python --version`
|
||||
placeholder: "3.11.9"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: hermes-version
|
||||
@@ -124,14 +106,14 @@ body:
|
||||
label: Hermes Version
|
||||
description: Output of `hermes version`
|
||||
placeholder: "2.1.0"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: logs
|
||||
attributes:
|
||||
label: Additional Logs / Traceback (optional)
|
||||
description: |
|
||||
The debug report above covers most logs. Use this field for any extra error output,
|
||||
tracebacks, or screenshots not captured by `hermes debug share`.
|
||||
label: Relevant Logs / Traceback
|
||||
description: Paste any error output, traceback, or log messages. This will be auto-formatted as code.
|
||||
render: shell
|
||||
|
||||
- type: textarea
|
||||
|
||||
@@ -71,15 +71,3 @@ body:
|
||||
label: Contribution
|
||||
options:
|
||||
- label: I'd like to implement this myself and submit a PR
|
||||
|
||||
- type: textarea
|
||||
id: debug-report
|
||||
attributes:
|
||||
label: Debug Report (optional)
|
||||
description: |
|
||||
If this feature request is related to a problem you're experiencing, run `hermes debug share` and paste the links here.
|
||||
In an interactive chat session, you can use `/debug` instead.
|
||||
This helps us understand your environment and any related logs.
|
||||
placeholder: |
|
||||
Report https://paste.rs/abc123
|
||||
render: shell
|
||||
|
||||
@@ -9,8 +9,7 @@ body:
|
||||
Sorry you're having trouble! Please fill out the details below so we can help.
|
||||
|
||||
**Quick checks first:**
|
||||
- Run `hermes debug share` and paste the links in the Debug Report section below
|
||||
- If you're in a chat session, you can use `/debug` instead — it does the same thing
|
||||
- Run `hermes doctor` and include the output below
|
||||
- Try `hermes update` to get the latest version
|
||||
- Check the [README troubleshooting section](https://github.com/NousResearch/hermes-agent#troubleshooting)
|
||||
- For general questions, consider the [Nous Research Discord](https://discord.gg/NousResearch) for faster help
|
||||
@@ -75,21 +74,10 @@ body:
|
||||
placeholder: "2.1.0"
|
||||
|
||||
- type: textarea
|
||||
id: debug-report
|
||||
id: doctor-output
|
||||
attributes:
|
||||
label: Debug Report
|
||||
description: |
|
||||
Run `hermes debug share` from your terminal and paste the links it prints here.
|
||||
This uploads your system info, config, and recent logs to a paste service automatically.
|
||||
|
||||
If you're in an interactive chat session, you can also use the `/debug` slash command — it does the same thing.
|
||||
|
||||
If the upload fails or install didn't get that far, run `hermes debug share --local` and paste the output directly.
|
||||
If even that doesn't work, run `hermes doctor` and paste that output instead.
|
||||
placeholder: |
|
||||
Report https://paste.rs/abc123
|
||||
agent.log https://paste.rs/def456
|
||||
gateway.log https://paste.rs/ghi789
|
||||
label: Output of `hermes doctor`
|
||||
description: Run `hermes doctor` and paste the full output. This will be auto-formatted.
|
||||
render: shell
|
||||
|
||||
- type: textarea
|
||||
|
||||
@@ -1,73 +0,0 @@
|
||||
name: Contributor Attribution Check
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths:
|
||||
# Only run when code files change (not docs-only PRs)
|
||||
- '*.py'
|
||||
- '**/*.py'
|
||||
- '.github/workflows/contributor-check.yml'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
check-attribution:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
fetch-depth: 0 # Full history needed for git log
|
||||
|
||||
- name: Check for unmapped contributor emails
|
||||
run: |
|
||||
# Get the merge base between this PR and main
|
||||
MERGE_BASE=$(git merge-base origin/main HEAD)
|
||||
|
||||
# Find any new author emails in this PR's commits
|
||||
NEW_EMAILS=$(git log ${MERGE_BASE}..HEAD --format='%ae' --no-merges | sort -u)
|
||||
|
||||
if [ -z "$NEW_EMAILS" ]; then
|
||||
echo "No new commits to check."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check each email against AUTHOR_MAP in release.py
|
||||
MISSING=""
|
||||
while IFS= read -r email; do
|
||||
# Skip teknium and bot emails
|
||||
case "$email" in
|
||||
*teknium*|*noreply@github.com*|*dependabot*|*github-actions*|*anthropic.com*|*cursor.com*)
|
||||
continue ;;
|
||||
esac
|
||||
|
||||
# Check if email is in AUTHOR_MAP (either as a key or matches noreply pattern)
|
||||
if echo "$email" | grep -qP '\+.*@users\.noreply\.github\.com'; then
|
||||
continue # GitHub noreply emails auto-resolve
|
||||
fi
|
||||
|
||||
if ! grep -qF "\"${email}\"" scripts/release.py 2>/dev/null; then
|
||||
AUTHOR=$(git log --author="$email" --format='%an' -1)
|
||||
MISSING="${MISSING}\n ${email} (${AUTHOR})"
|
||||
fi
|
||||
done <<< "$NEW_EMAILS"
|
||||
|
||||
if [ -n "$MISSING" ]; then
|
||||
echo ""
|
||||
echo "⚠️ New contributor email(s) not in AUTHOR_MAP:"
|
||||
echo -e "$MISSING"
|
||||
echo ""
|
||||
echo "Please add mappings to scripts/release.py AUTHOR_MAP:"
|
||||
echo -e "$MISSING" | while read -r line; do
|
||||
email=$(echo "$line" | sed 's/^ *//' | cut -d' ' -f1)
|
||||
[ -z "$email" ] && continue
|
||||
echo " \"${email}\": \"<github-username>\","
|
||||
done
|
||||
echo ""
|
||||
echo "To find the GitHub username for an email:"
|
||||
echo " gh api 'search/users?q=EMAIL+in:email' --jq '.items[0].login'"
|
||||
exit 1
|
||||
else
|
||||
echo "✅ All contributor emails are mapped in AUTHOR_MAP."
|
||||
fi
|
||||
@@ -1,12 +1,11 @@
|
||||
name: Deploy Site
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'website/**'
|
||||
- 'landingpage/**'
|
||||
- 'skills/**'
|
||||
- 'optional-skills/**'
|
||||
- '.github/workflows/deploy-site.yml'
|
||||
@@ -21,46 +20,32 @@ concurrency:
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
deploy-vercel:
|
||||
if: github.event_name == 'release'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Trigger Vercel Deploy
|
||||
run: curl -X POST "${{ secrets.VERCEL_DEPLOY_HOOK }}"
|
||||
|
||||
deploy-docs:
|
||||
build-and-deploy:
|
||||
# Only run on the upstream repository, not on forks
|
||||
if: github.repository == 'NousResearch/hermes-agent'
|
||||
runs-on: ubuntu-latest
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deploy.outputs.page_url }}
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: npm
|
||||
cache-dependency-path: website/package-lock.json
|
||||
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Install PyYAML for skill extraction
|
||||
run: pip install pyyaml==6.0.2 httpx==0.28.1
|
||||
run: pip install pyyaml
|
||||
|
||||
- name: Extract skill metadata for dashboard
|
||||
run: python3 website/scripts/extract-skills.py
|
||||
|
||||
- name: Build skills index (if not already present)
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
if [ ! -f website/static/api/skills-index.json ]; then
|
||||
python3 scripts/build_skills_index.py || echo "Skills index build failed (non-fatal)"
|
||||
fi
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
working-directory: website
|
||||
@@ -72,13 +57,18 @@ jobs:
|
||||
- name: Stage deployment
|
||||
run: |
|
||||
mkdir -p _site/docs
|
||||
# Landing page at root
|
||||
cp -r landingpage/* _site/
|
||||
# Docusaurus at /docs/
|
||||
cp -r website/build/* _site/docs/
|
||||
# CNAME so GitHub Pages keeps the custom domain between deploys
|
||||
echo "hermes-agent.nousresearch.com" > _site/CNAME
|
||||
|
||||
- name: Upload artifact
|
||||
uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3
|
||||
uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: _site
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
id: deploy
|
||||
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4
|
||||
uses: actions/deploy-pages@v4
|
||||
|
||||
@@ -23,21 +23,21 @@ jobs:
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
|
||||
uses: docker/setup-qemu-action@v3
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
# Build amd64 only so we can `load` the image for smoke testing.
|
||||
# `load: true` cannot export a multi-arch manifest to the local daemon.
|
||||
# The multi-arch build follows on push to main / release.
|
||||
- name: Build image (amd64, smoke test)
|
||||
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: Dockerfile
|
||||
@@ -56,31 +56,36 @@ jobs:
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main' || github.event_name == 'release'
|
||||
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Push multi-arch image (main branch)
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: Dockerfile
|
||||
push: true
|
||||
platforms: linux/amd64,linux/arm64
|
||||
tags: nousresearch/hermes-agent:latest
|
||||
tags: |
|
||||
nousresearch/hermes-agent:latest
|
||||
nousresearch/hermes-agent:${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Push multi-arch image (release)
|
||||
if: github.event_name == 'release'
|
||||
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: Dockerfile
|
||||
push: true
|
||||
platforms: linux/amd64,linux/arm64
|
||||
tags: nousresearch/hermes-agent:${{ github.event.release.tag_name }}
|
||||
tags: |
|
||||
nousresearch/hermes-agent:latest
|
||||
nousresearch/hermes-agent:${{ github.event.release.tag_name }}
|
||||
nousresearch/hermes-agent:${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
@@ -7,16 +7,13 @@ on:
|
||||
- '.github/workflows/docs-site-checks.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
docs-site-checks:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: npm
|
||||
@@ -26,7 +23,7 @@ jobs:
|
||||
run: npm ci
|
||||
working-directory: website
|
||||
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
|
||||
@@ -14,9 +14,6 @@ on:
|
||||
- 'run_agent.py'
|
||||
- 'acp_adapter/**'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: nix-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
@@ -29,7 +26,7 @@ jobs:
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
- uses: actions/checkout@v4
|
||||
- uses: DeterminateSystems/nix-installer-action@ef8a148080ab6020fd15196c2084a2eea5ff2d25 # v22
|
||||
- uses: DeterminateSystems/magic-nix-cache-action@565684385bcd71bad329742eefe8d12f2e765b39 # v13
|
||||
- name: Check flake
|
||||
|
||||
@@ -1,101 +0,0 @@
|
||||
name: Build Skills Index
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Run twice daily: 6 AM and 6 PM UTC
|
||||
- cron: '0 6,18 * * *'
|
||||
workflow_dispatch: # Manual trigger
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'scripts/build_skills_index.py'
|
||||
- '.github/workflows/skills-index.yml'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
build-index:
|
||||
# Only run on the upstream repository, not on forks
|
||||
if: github.repository == 'NousResearch/hermes-agent'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Install dependencies
|
||||
run: pip install httpx==0.28.1 pyyaml==6.0.2
|
||||
|
||||
- name: Build skills index
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: python scripts/build_skills_index.py
|
||||
|
||||
- name: Upload index artifact
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
|
||||
with:
|
||||
name: skills-index
|
||||
path: website/static/api/skills-index.json
|
||||
retention-days: 7
|
||||
|
||||
deploy-with-index:
|
||||
needs: build-index
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
pages: write
|
||||
id-token: write
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deploy.outputs.page_url }}
|
||||
# Only deploy on schedule or manual trigger (not on every push to the script)
|
||||
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
||||
with:
|
||||
name: skills-index
|
||||
path: website/static/api/
|
||||
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: npm
|
||||
cache-dependency-path: website/package-lock.json
|
||||
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Install PyYAML for skill extraction
|
||||
run: pip install pyyaml==6.0.2
|
||||
|
||||
- name: Extract skill metadata for dashboard
|
||||
run: python3 website/scripts/extract-skills.py
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
working-directory: website
|
||||
|
||||
- name: Build Docusaurus
|
||||
run: npm run build
|
||||
working-directory: website
|
||||
|
||||
- name: Stage deployment
|
||||
run: |
|
||||
mkdir -p _site/docs
|
||||
cp -r landingpage/* _site/
|
||||
cp -r website/build/* _site/docs/
|
||||
echo "hermes-agent.nousresearch.com" > _site/CNAME
|
||||
|
||||
- name: Upload artifact
|
||||
uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3
|
||||
with:
|
||||
path: _site
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
id: deploy
|
||||
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4
|
||||
@@ -14,7 +14,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
@@ -149,62 +149,6 @@ jobs:
|
||||
"
|
||||
fi
|
||||
|
||||
# --- CI/CD workflow files modified ---
|
||||
WORKFLOW_HITS=$(git diff --name-only "$BASE".."$HEAD" | grep -E '\.github/workflows/.*\.ya?ml$' || true)
|
||||
if [ -n "$WORKFLOW_HITS" ]; then
|
||||
FINDINGS="${FINDINGS}
|
||||
### ⚠️ WARNING: CI/CD workflow files modified
|
||||
Changes to workflow files can alter build pipelines, inject steps, or modify permissions. Verify no unauthorized actions or secrets access were added.
|
||||
|
||||
**Files:**
|
||||
\`\`\`
|
||||
${WORKFLOW_HITS}
|
||||
\`\`\`
|
||||
"
|
||||
fi
|
||||
|
||||
# --- Dockerfile / container build files modified ---
|
||||
DOCKER_HITS=$(git diff --name-only "$BASE".."$HEAD" | grep -iE '(Dockerfile|\.dockerignore|docker-compose)' || true)
|
||||
if [ -n "$DOCKER_HITS" ]; then
|
||||
FINDINGS="${FINDINGS}
|
||||
### ⚠️ WARNING: Container build files modified
|
||||
Changes to Dockerfiles or compose files can alter base images, add build steps, or expose ports. Verify base image pins and build commands.
|
||||
|
||||
**Files:**
|
||||
\`\`\`
|
||||
${DOCKER_HITS}
|
||||
\`\`\`
|
||||
"
|
||||
fi
|
||||
|
||||
# --- Dependency manifest files modified ---
|
||||
DEP_HITS=$(git diff --name-only "$BASE".."$HEAD" | grep -E '(pyproject\.toml|requirements.*\.txt|package\.json|Gemfile|go\.mod|Cargo\.toml)$' || true)
|
||||
if [ -n "$DEP_HITS" ]; then
|
||||
FINDINGS="${FINDINGS}
|
||||
### ⚠️ WARNING: Dependency manifest files modified
|
||||
Changes to dependency files can introduce new packages or change version pins. Verify all dependency changes are intentional and from trusted sources.
|
||||
|
||||
**Files:**
|
||||
\`\`\`
|
||||
${DEP_HITS}
|
||||
\`\`\`
|
||||
"
|
||||
fi
|
||||
|
||||
# --- GitHub Actions version unpinning (mutable tags instead of SHAs) ---
|
||||
ACTIONS_UNPIN=$(echo "$DIFF" | grep -n '^\+' | grep 'uses:' | grep -v '#' | grep -E '@v[0-9]' | head -10 || true)
|
||||
if [ -n "$ACTIONS_UNPIN" ]; then
|
||||
FINDINGS="${FINDINGS}
|
||||
### ⚠️ WARNING: GitHub Actions with mutable version tags
|
||||
Actions should be pinned to full commit SHAs (not \`@v4\`, \`@v5\`). Mutable tags can be retargeted silently if a maintainer account is compromised.
|
||||
|
||||
**Matches:**
|
||||
\`\`\`
|
||||
${ACTIONS_UNPIN}
|
||||
\`\`\`
|
||||
"
|
||||
fi
|
||||
|
||||
# --- Output results ---
|
||||
if [ -n "$FINDINGS" ]; then
|
||||
echo "found=true" >> "$GITHUB_OUTPUT"
|
||||
@@ -239,7 +183,7 @@ jobs:
|
||||
---
|
||||
*Automated scan triggered by [supply-chain-audit](/.github/workflows/supply-chain-audit.yml). If this is a false positive, a maintainer can approve after manual review.*"
|
||||
|
||||
gh pr comment "${{ github.event.pull_request.number }}" --body "$BODY" || echo "::warning::Could not post PR comment (expected for fork PRs — GITHUB_TOKEN is read-only)"
|
||||
gh pr comment "${{ github.event.pull_request.number }}" --body "$BODY"
|
||||
|
||||
- name: Fail on critical findings
|
||||
if: steps.scan.outputs.critical == 'true'
|
||||
|
||||
@@ -6,9 +6,6 @@ on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
# Cancel in-progress runs for the same PR/branch
|
||||
concurrency:
|
||||
group: tests-${{ github.ref }}
|
||||
@@ -20,13 +17,13 @@ jobs:
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Install system dependencies
|
||||
run: sudo apt-get update && sudo apt-get install -y ripgrep
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5
|
||||
uses: astral-sh/setup-uv@v5
|
||||
|
||||
- name: Set up Python 3.11
|
||||
run: uv python install 3.11
|
||||
@@ -52,10 +49,10 @@ jobs:
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5
|
||||
uses: astral-sh/setup-uv@v5
|
||||
|
||||
- name: Set up Python 3.11
|
||||
run: uv python install 3.11
|
||||
|
||||
@@ -51,15 +51,10 @@ ignored/
|
||||
.worktrees/
|
||||
environments/benchmarks/evals/
|
||||
|
||||
# Web UI build output
|
||||
hermes_cli/web_dist/
|
||||
|
||||
# Release script temp files
|
||||
.release_notes.md
|
||||
mini-swe-agent/
|
||||
|
||||
# Nix
|
||||
.direnv/
|
||||
.nix-stamps/
|
||||
result
|
||||
website/static/api/skills-index.json
|
||||
|
||||
@@ -1,108 +0,0 @@
|
||||
# .mailmap — canonical author mapping for git shortlog / git log / GitHub
|
||||
# Format: Canonical Name <canonical@email> <commit@email>
|
||||
# See: https://git-scm.com/docs/gitmailmap
|
||||
#
|
||||
# This maps commit emails to GitHub noreply addresses so that:
|
||||
# 1. `git shortlog -sn` shows deduplicated contributor counts
|
||||
# 2. GitHub's contributor graph can attribute commits correctly
|
||||
# 3. Contributors with personal/work emails get proper credit
|
||||
#
|
||||
# When adding entries: use the contributor's GitHub noreply email as canonical
|
||||
# so GitHub can link commits to their profile.
|
||||
|
||||
# === Teknium (multiple emails) ===
|
||||
Teknium <127238744+teknium1@users.noreply.github.com> <teknium1@gmail.com>
|
||||
Teknium <127238744+teknium1@users.noreply.github.com> <teknium@nousresearch.com>
|
||||
|
||||
# === Contributors — personal/work emails mapped to GitHub noreply ===
|
||||
# Format: Canonical Name <GH-noreply> <commit-email>
|
||||
|
||||
# Verified via GH API email search
|
||||
luyao618 <364939526@qq.com> <364939526@qq.com>
|
||||
ethernet8023 <arilotter@gmail.com> <arilotter@gmail.com>
|
||||
nicoloboschi <boschi1997@gmail.com> <boschi1997@gmail.com>
|
||||
cherifya <chef.ya@gmail.com> <chef.ya@gmail.com>
|
||||
BongSuCHOI <chlqhdtn98@gmail.com> <chlqhdtn98@gmail.com>
|
||||
dsocolobsky <dsocolobsky@gmail.com> <dsocolobsky@gmail.com>
|
||||
pefontana <fontana.pedro93@gmail.com> <fontana.pedro93@gmail.com>
|
||||
Helmi <frank@helmschrott.de> <frank@helmschrott.de>
|
||||
hata1234 <hata1234@gmail.com> <hata1234@gmail.com>
|
||||
|
||||
# Verified via PR investigation / salvage PR bodies
|
||||
DeployFaith <agents@kylefrench.dev> <agents@kylefrench.dev>
|
||||
flobo3 <floptopbot33@gmail.com> <floptopbot33@gmail.com>
|
||||
gaixianggeng <gaixg94@gmail.com> <gaixg94@gmail.com>
|
||||
KUSH42 <xush@xush.org> <xush@xush.org>
|
||||
konsisumer <der@konsi.org> <der@konsi.org>
|
||||
WorldInnovationsDepartment <vorvul.danylo@gmail.com> <vorvul.danylo@gmail.com>
|
||||
m0n5t3r <iacobs@m0n5t3r.info> <iacobs@m0n5t3r.info>
|
||||
sprmn24 <oncuevtv@gmail.com> <oncuevtv@gmail.com>
|
||||
fancydirty <fancydirty@gmail.com> <fancydirty@gmail.com>
|
||||
fxfitz <francis.x.fitzpatrick@gmail.com> <francis.x.fitzpatrick@gmail.com>
|
||||
limars874 <limars874@gmail.com> <limars874@gmail.com>
|
||||
AaronWong1999 <aaronwong1999@icloud.com> <aaronwong1999@icloud.com>
|
||||
dippwho <dipp.who@gmail.com> <dipp.who@gmail.com>
|
||||
duerzy <duerzy@gmail.com> <duerzy@gmail.com>
|
||||
geoffwellman <geoff.wellman@gmail.com> <geoff.wellman@gmail.com>
|
||||
hcshen0111 <shenhaocheng19990111@gmail.com> <shenhaocheng19990111@gmail.com>
|
||||
jamesarch <han.shan@live.cn> <han.shan@live.cn>
|
||||
stephenschoettler <stephenschoettler@gmail.com> <stephenschoettler@gmail.com>
|
||||
Tranquil-Flow <tranquil_flow@protonmail.com> <tranquil_flow@protonmail.com>
|
||||
Dusk1e <yusufalweshdemir@gmail.com> <yusufalweshdemir@gmail.com>
|
||||
Awsh1 <ysfalweshcan@gmail.com> <ysfalweshcan@gmail.com>
|
||||
WAXLYY <ysfwaxlycan@gmail.com> <ysfwaxlycan@gmail.com>
|
||||
donrhmexe <don.rhm@gmail.com> <don.rhm@gmail.com>
|
||||
hqhq1025 <1506751656@qq.com> <1506751656@qq.com>
|
||||
BlackishGreen33 <s5460703@gmail.com> <s5460703@gmail.com>
|
||||
tomqiaozc <zqiao@microsoft.com> <zqiao@microsoft.com>
|
||||
MagicRay1217 <mingjwan@microsoft.com> <mingjwan@microsoft.com>
|
||||
aaronagent <1115117931@qq.com> <1115117931@qq.com>
|
||||
YoungYang963 <young@YoungdeMacBook-Pro.local> <young@YoungdeMacBook-Pro.local>
|
||||
LongOddCode <haolong@microsoft.com> <haolong@microsoft.com>
|
||||
Cafexss <coffeemjj@gmail.com> <coffeemjj@gmail.com>
|
||||
Cygra <sjtuwbh@gmail.com> <sjtuwbh@gmail.com>
|
||||
DomGrieco <dgrieco@redhat.com> <dgrieco@redhat.com>
|
||||
|
||||
# Duplicate email mapping (same person, multiple emails)
|
||||
Sertug17 <104278804+Sertug17@users.noreply.github.com> <srhtsrht17@gmail.com>
|
||||
yyovil <birdiegyal@gmail.com> <tanishq231003@gmail.com>
|
||||
DomGrieco <dgrieco@redhat.com> <dgrieco@redhat.com>
|
||||
dsocolobsky <dsocolobsky@gmail.com> <dylan.socolobsky@lambdaclass.com>
|
||||
olafthiele <programming@olafthiele.com> <olafthiele@gmail.com>
|
||||
|
||||
# Verified via git display name matching GH contributor username
|
||||
cokemine <aptx4561@gmail.com> <aptx4561@gmail.com>
|
||||
dalianmao000 <dalianmao0107@gmail.com> <dalianmao0107@gmail.com>
|
||||
emozilla <emozilla@nousresearch.com> <emozilla@nousresearch.com>
|
||||
jjovalle99 <juan.ovalle@mistral.ai> <juan.ovalle@mistral.ai>
|
||||
kagura-agent <kagura.chen28@gmail.com> <kagura.chen28@gmail.com>
|
||||
spniyant <niyant@spicefi.xyz> <niyant@spicefi.xyz>
|
||||
olafthiele <programming@olafthiele.com> <programming@olafthiele.com>
|
||||
r266-tech <r2668940489@gmail.com> <r2668940489@gmail.com>
|
||||
xingkongliang <tianliangjay@gmail.com> <tianliangjay@gmail.com>
|
||||
win4r <win4r@outlook.com> <win4r@outlook.com>
|
||||
zhouboli <zhouboli@gmail.com> <zhouboli@gmail.com>
|
||||
yongtenglei <yongtenglei@gmail.com> <yongtenglei@gmail.com>
|
||||
|
||||
# Nous Research team
|
||||
benbarclay <ben@nousresearch.com> <ben@nousresearch.com>
|
||||
jquesnelle <jonny@nousresearch.com> <jonny@nousresearch.com>
|
||||
|
||||
# GH contributor list verified
|
||||
spideystreet <dhicham.pro@gmail.com> <dhicham.pro@gmail.com>
|
||||
dorukardahan <dorukardahan@hotmail.com> <dorukardahan@hotmail.com>
|
||||
MustafaKara7 <karamusti912@gmail.com> <karamusti912@gmail.com>
|
||||
Hmbown <hmbown@gmail.com> <hmbown@gmail.com>
|
||||
kamil-gwozdz <kamil@gwozdz.me> <kamil@gwozdz.me>
|
||||
kira-ariaki <kira@ariaki.me> <kira@ariaki.me>
|
||||
knopki <knopki@duck.com> <knopki@duck.com>
|
||||
Unayung <unayung@gmail.com> <unayung@gmail.com>
|
||||
SeeYangZhi <yangzhi.see@gmail.com> <yangzhi.see@gmail.com>
|
||||
Julientalbot <julien.talbot@ergonomia.re> <julien.talbot@ergonomia.re>
|
||||
lesterli <lisicheng168@gmail.com> <lisicheng168@gmail.com>
|
||||
JiayuuWang <jiayuw794@gmail.com> <jiayuw794@gmail.com>
|
||||
tesseracttars-creator <tesseracttars@gmail.com> <tesseracttars@gmail.com>
|
||||
xinbenlv <zzn+pa@zzn.im> <zzn+pa@zzn.im>
|
||||
SaulJWu <saul.jj.wu@gmail.com> <saul.jj.wu@gmail.com>
|
||||
angelos <angelos@oikos.lan.home.malaiwah.com> <angelos@oikos.lan.home.malaiwah.com>
|
||||
MestreY0d4-Uninter <241404605+MestreY0d4-Uninter@users.noreply.github.com> <MestreY0d4-Uninter@users.noreply.github.com>
|
||||
@@ -13,7 +13,7 @@ source venv/bin/activate # ALWAYS activate before running Python
|
||||
```
|
||||
hermes-agent/
|
||||
├── run_agent.py # AIAgent class — core conversation loop
|
||||
├── model_tools.py # Tool orchestration, discover_builtin_tools(), handle_function_call()
|
||||
├── model_tools.py # Tool orchestration, _discover_tools(), handle_function_call()
|
||||
├── toolsets.py # Toolset definitions, _HERMES_CORE_TOOLS list
|
||||
├── cli.py # HermesCLI class — interactive CLI orchestrator
|
||||
├── hermes_state.py # SessionDB — SQLite session store (FTS5 search)
|
||||
@@ -55,20 +55,7 @@ hermes-agent/
|
||||
├── gateway/ # Messaging platform gateway
|
||||
│ ├── run.py # Main loop, slash commands, message dispatch
|
||||
│ ├── session.py # SessionStore — conversation persistence
|
||||
│ └── platforms/ # Adapters: telegram, discord, slack, whatsapp, homeassistant, signal, qqbot
|
||||
├── ui-tui/ # Ink (React) terminal UI — `hermes --tui`
|
||||
│ ├── src/entry.tsx # TTY gate + render()
|
||||
│ ├── src/app.tsx # Main state machine and UI
|
||||
│ ├── src/gatewayClient.ts # Child process + JSON-RPC bridge
|
||||
│ ├── src/app/ # Decomposed app logic (event handler, slash handler, stores, hooks)
|
||||
│ ├── src/components/ # Ink components (branding, markdown, prompts, pickers, etc.)
|
||||
│ ├── src/hooks/ # useCompletion, useInputHistory, useQueue, useVirtualHistory
|
||||
│ └── src/lib/ # Pure helpers (history, osc52, text, rpc, messages)
|
||||
├── tui_gateway/ # Python JSON-RPC backend for the TUI
|
||||
│ ├── entry.py # stdio entrypoint
|
||||
│ ├── server.py # RPC handlers and session logic
|
||||
│ ├── render.py # Optional rich/ANSI bridge
|
||||
│ └── slash_worker.py # Persistent HermesCLI subprocess for slash commands
|
||||
│ └── platforms/ # Adapters: telegram, discord, slack, whatsapp, homeassistant, signal
|
||||
├── acp_adapter/ # ACP server (VS Code / Zed / JetBrains integration)
|
||||
├── cron/ # Scheduler (jobs.py, scheduler.py)
|
||||
├── environments/ # RL training environments (Atropos)
|
||||
@@ -192,62 +179,9 @@ if canonical == "mycommand":
|
||||
|
||||
---
|
||||
|
||||
## TUI Architecture (ui-tui + tui_gateway)
|
||||
|
||||
The TUI is a full replacement for the classic (prompt_toolkit) CLI, activated via `hermes --tui` or `HERMES_TUI=1`.
|
||||
|
||||
### Process Model
|
||||
|
||||
```
|
||||
hermes --tui
|
||||
└─ Node (Ink) ──stdio JSON-RPC── Python (tui_gateway)
|
||||
│ └─ AIAgent + tools + sessions
|
||||
└─ renders transcript, composer, prompts, activity
|
||||
```
|
||||
|
||||
TypeScript owns the screen. Python owns sessions, tools, model calls, and slash command logic.
|
||||
|
||||
### Transport
|
||||
|
||||
Newline-delimited JSON-RPC over stdio. Requests from Ink, events from Python. See `tui_gateway/server.py` for the full method/event catalog.
|
||||
|
||||
### Key Surfaces
|
||||
|
||||
| Surface | Ink component | Gateway method |
|
||||
|---------|---------------|----------------|
|
||||
| Chat streaming | `app.tsx` + `messageLine.tsx` | `prompt.submit` → `message.delta/complete` |
|
||||
| Tool activity | `thinking.tsx` | `tool.start/progress/complete` |
|
||||
| Approvals | `prompts.tsx` | `approval.respond` ← `approval.request` |
|
||||
| Clarify/sudo/secret | `prompts.tsx`, `maskedPrompt.tsx` | `clarify/sudo/secret.respond` |
|
||||
| Session picker | `sessionPicker.tsx` | `session.list/resume` |
|
||||
| Slash commands | Local handler + fallthrough | `slash.exec` → `_SlashWorker`, `command.dispatch` |
|
||||
| Completions | `useCompletion` hook | `complete.slash`, `complete.path` |
|
||||
| Theming | `theme.ts` + `branding.tsx` | `gateway.ready` with skin data |
|
||||
|
||||
### Slash Command Flow
|
||||
|
||||
1. Built-in client commands (`/help`, `/quit`, `/clear`, `/resume`, `/copy`, `/paste`, etc.) handled locally in `app.tsx`
|
||||
2. Everything else → `slash.exec` (runs in persistent `_SlashWorker` subprocess) → `command.dispatch` fallback
|
||||
|
||||
### Dev Commands
|
||||
|
||||
```bash
|
||||
cd ui-tui
|
||||
npm install # first time
|
||||
npm run dev # watch mode (rebuilds hermes-ink + tsx --watch)
|
||||
npm start # production
|
||||
npm run build # full build (hermes-ink + tsc)
|
||||
npm run type-check # typecheck only (tsc --noEmit)
|
||||
npm run lint # eslint
|
||||
npm run fmt # prettier
|
||||
npm test # vitest
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding New Tools
|
||||
|
||||
Requires changes in **2 files**:
|
||||
Requires changes in **3 files**:
|
||||
|
||||
**1. Create `tools/your_tool.py`:**
|
||||
```python
|
||||
@@ -270,9 +204,9 @@ registry.register(
|
||||
)
|
||||
```
|
||||
|
||||
**2. Add to `toolsets.py`** — either `_HERMES_CORE_TOOLS` (all platforms) or a new toolset.
|
||||
**2. Add import** in `model_tools.py` `_discover_tools()` list.
|
||||
|
||||
Auto-discovery: any `tools/*.py` file with a top-level `registry.register()` call is imported automatically — no manual import list to maintain.
|
||||
**3. Add to `toolsets.py`** — either `_HERMES_CORE_TOOLS` (all platforms) or a new toolset.
|
||||
|
||||
The registry handles schema collection, dispatch, availability checking, and error wrapping. All handlers MUST return a JSON string.
|
||||
|
||||
@@ -524,45 +458,13 @@ def profile_env(tmp_path, monkeypatch):
|
||||
|
||||
## Testing
|
||||
|
||||
**ALWAYS use `scripts/run_tests.sh`** — do not call `pytest` directly. The script enforces
|
||||
hermetic environment parity with CI (unset credential vars, TZ=UTC, LANG=C.UTF-8,
|
||||
4 xdist workers matching GHA ubuntu-latest). Direct `pytest` on a 16+ core
|
||||
developer machine with API keys set diverges from CI in ways that have caused
|
||||
multiple "works locally, fails in CI" incidents (and the reverse).
|
||||
|
||||
```bash
|
||||
scripts/run_tests.sh # full suite, CI-parity
|
||||
scripts/run_tests.sh tests/gateway/ # one directory
|
||||
scripts/run_tests.sh tests/agent/test_foo.py::test_x # one test
|
||||
scripts/run_tests.sh -v --tb=long # pass-through pytest flags
|
||||
```
|
||||
|
||||
### Why the wrapper (and why the old "just call pytest" doesn't work)
|
||||
|
||||
Five real sources of local-vs-CI drift the script closes:
|
||||
|
||||
| | Without wrapper | With wrapper |
|
||||
|---|---|---|
|
||||
| Provider API keys | Whatever is in your env (auto-detects pool) | All `*_API_KEY`/`*_TOKEN`/etc. unset |
|
||||
| HOME / `~/.hermes/` | Your real config+auth.json | Temp dir per test |
|
||||
| Timezone | Local TZ (PDT etc.) | UTC |
|
||||
| Locale | Whatever is set | C.UTF-8 |
|
||||
| xdist workers | `-n auto` = all cores (20+ on a workstation) | `-n 4` matching CI |
|
||||
|
||||
`tests/conftest.py` also enforces points 1-4 as an autouse fixture so ANY pytest
|
||||
invocation (including IDE integrations) gets hermetic behavior — but the wrapper
|
||||
is belt-and-suspenders.
|
||||
|
||||
### Running without the wrapper (only if you must)
|
||||
|
||||
If you can't use the wrapper (e.g. on Windows or inside an IDE that shells
|
||||
pytest directly), at minimum activate the venv and pass `-n 4`:
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python -m pytest tests/ -q -n 4
|
||||
python -m pytest tests/ -q # Full suite (~3000 tests, ~3 min)
|
||||
python -m pytest tests/test_model_tools.py -q # Toolset resolution
|
||||
python -m pytest tests/test_cli_init.py -q # CLI config loading
|
||||
python -m pytest tests/gateway/ -q # Gateway tests
|
||||
python -m pytest tests/tools/ -q # Tool-level tests
|
||||
```
|
||||
|
||||
Worker count above 4 will surface test-ordering flakes that CI never sees.
|
||||
|
||||
Always run the full suite before pushing changes.
|
||||
|
||||
+1
-1
@@ -12,7 +12,7 @@ ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/.playwright
|
||||
# Install system dependencies in one layer, clear APT cache
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends \
|
||||
build-essential nodejs npm python3 ripgrep ffmpeg gcc python3-dev libffi-dev procps git && \
|
||||
build-essential nodejs npm python3 ripgrep ffmpeg gcc python3-dev libffi-dev procps && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Non-root user for runtime; UID can be overridden via HERMES_UID at runtime
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
|
||||
**The self-improving AI agent built by [Nous Research](https://nousresearch.com).** It's the only agent with a built-in learning loop — it creates skills from experience, improves them during use, nudges itself to persist knowledge, searches its own past conversations, and builds a deepening model of who you are across sessions. Run it on a $5 VPS, a GPU cluster, or serverless infrastructure that costs nearly nothing when idle. It's not tied to your laptop — talk to it from Telegram while it works on a cloud VM.
|
||||
|
||||
Use any model you want — [Nous Portal](https://portal.nousresearch.com), [OpenRouter](https://openrouter.ai) (200+ models), [NVIDIA NIM](https://build.nvidia.com) (Nemotron), [Xiaomi MiMo](https://platform.xiaomimimo.com), [z.ai/GLM](https://z.ai), [Kimi/Moonshot](https://platform.moonshot.ai), [MiniMax](https://www.minimax.io), [Hugging Face](https://huggingface.co), OpenAI, or your own endpoint. Switch with `hermes model` — no code changes, no lock-in.
|
||||
Use any model you want — [Nous Portal](https://portal.nousresearch.com), [OpenRouter](https://openrouter.ai) (200+ models), [z.ai/GLM](https://z.ai), [Kimi/Moonshot](https://platform.moonshot.ai), [MiniMax](https://www.minimax.io), OpenAI, or your own endpoint. Switch with `hermes model` — no code changes, no lock-in.
|
||||
|
||||
<table>
|
||||
<tr><td><b>A real terminal interface</b></td><td>Full TUI with multiline editing, slash-command autocomplete, conversation history, interrupt-and-redirect, and streaming tool output.</td></tr>
|
||||
@@ -141,18 +141,11 @@ See `hermes claw migrate --help` for all options, or use the `openclaw-migration
|
||||
|
||||
We welcome contributions! See the [Contributing Guide](https://hermes-agent.nousresearch.com/docs/developer-guide/contributing) for development setup, code style, and PR process.
|
||||
|
||||
Quick start for contributors — clone and go with `setup-hermes.sh`:
|
||||
Quick start for contributors:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/NousResearch/hermes-agent.git
|
||||
cd hermes-agent
|
||||
./setup-hermes.sh # installs uv, creates venv, installs .[all], symlinks ~/.local/bin/hermes
|
||||
./hermes # auto-detects the venv, no need to `source` first
|
||||
```
|
||||
|
||||
Manual path (equivalent to the above):
|
||||
|
||||
```bash
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
uv venv venv --python 3.11
|
||||
source venv/bin/activate
|
||||
@@ -174,7 +167,6 @@ python -m pytest tests/ -q
|
||||
- 📚 [Skills Hub](https://agentskills.io)
|
||||
- 🐛 [Issues](https://github.com/NousResearch/hermes-agent/issues)
|
||||
- 💡 [Discussions](https://github.com/NousResearch/hermes-agent/discussions)
|
||||
- 🔌 [HermesClaw](https://github.com/AaronWong1999/hermesclaw) — Community WeChat bridge: Run Hermes Agent and OpenClaw on the same WeChat account.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
# Hermes Agent v0.10.0 (v2026.4.16)
|
||||
|
||||
**Release Date:** April 16, 2026
|
||||
|
||||
> The Tool Gateway release — paid Nous Portal subscribers can now use web search, image generation, text-to-speech, and browser automation through their existing subscription with zero additional API keys.
|
||||
|
||||
---
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Nous Tool Gateway** — Paid [Nous Portal](https://portal.nousresearch.com) subscribers now get automatic access to **web search** (Firecrawl), **image generation** (FAL / FLUX 2 Pro), **text-to-speech** (OpenAI TTS), and **browser automation** (Browser Use) through their existing subscription. No separate API keys needed — just run `hermes model`, select Nous Portal, and pick which tools to enable. Per-tool opt-in via `use_gateway` config, full integration with `hermes tools` and `hermes status`, and the runtime correctly prefers the gateway even when direct API keys exist. Replaces the old hidden `HERMES_ENABLE_NOUS_MANAGED_TOOLS` env var with clean subscription-based detection. ([#11206](https://github.com/NousResearch/hermes-agent/pull/11206), based on work by @jquesnelle; docs: [#11208](https://github.com/NousResearch/hermes-agent/pull/11208))
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Bug Fixes & Improvements
|
||||
|
||||
This release includes 180+ commits with numerous bug fixes, platform improvements, and reliability enhancements across the agent core, gateway, CLI, and tool system. Full details will be published in the v0.11.0 changelog.
|
||||
|
||||
---
|
||||
|
||||
## 👥 Contributors
|
||||
|
||||
- **@jquesnelle** (emozilla) — Original Tool Gateway implementation ([#10799](https://github.com/NousResearch/hermes-agent/pull/10799)), salvaged and shipped in this release
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: [v2026.4.13...v2026.4.16](https://github.com/NousResearch/hermes-agent/compare/v2026.4.13...v2026.4.16)
|
||||
@@ -1,329 +0,0 @@
|
||||
# Hermes Agent v0.9.0 (v2026.4.13)
|
||||
|
||||
**Release Date:** April 13, 2026
|
||||
**Since v0.8.0:** 487 commits · 269 merged PRs · 167 resolved issues · 493 files changed · 63,281 insertions · 24 contributors
|
||||
|
||||
> The everywhere release — Hermes goes mobile with Termux/Android, adds iMessage and WeChat, ships Fast Mode for OpenAI and Anthropic, introduces background process monitoring, launches a local web dashboard for managing your agent, and delivers the deepest security hardening pass yet across 16 supported platforms.
|
||||
|
||||
---
|
||||
|
||||
## ✨ Highlights
|
||||
|
||||
- **Local Web Dashboard** — A new browser-based dashboard for managing your Hermes Agent locally. Configure settings, monitor sessions, browse skills, and manage your gateway — all from a clean web interface without touching config files or the terminal. The easiest way to get started with Hermes.
|
||||
|
||||
- **Fast Mode (`/fast`)** — Priority processing for OpenAI and Anthropic models. Toggle `/fast` to route through priority queues for significantly lower latency on supported models (GPT-5.4, Codex, Claude). Expands across all OpenAI Priority Processing models and Anthropic's fast tier. ([#6875](https://github.com/NousResearch/hermes-agent/pull/6875), [#6960](https://github.com/NousResearch/hermes-agent/pull/6960), [#7037](https://github.com/NousResearch/hermes-agent/pull/7037))
|
||||
|
||||
- **iMessage via BlueBubbles** — Full iMessage integration through BlueBubbles, bringing Hermes to Apple's messaging ecosystem. Auto-webhook registration, setup wizard integration, and crash resilience. ([#6437](https://github.com/NousResearch/hermes-agent/pull/6437), [#6460](https://github.com/NousResearch/hermes-agent/pull/6460), [#6494](https://github.com/NousResearch/hermes-agent/pull/6494))
|
||||
|
||||
- **WeChat (Weixin) & WeCom Callback Mode** — Native WeChat support via iLink Bot API and a new WeCom callback-mode adapter for self-built enterprise apps. Streaming cursor, media uploads, markdown link handling, and atomic state persistence. Hermes now covers the Chinese messaging ecosystem end-to-end. ([#7166](https://github.com/NousResearch/hermes-agent/pull/7166), [#7943](https://github.com/NousResearch/hermes-agent/pull/7943))
|
||||
|
||||
- **Termux / Android Support** — Run Hermes natively on Android via Termux. Adapted install paths, TUI optimizations for mobile screens, voice backend support, and the `/image` command work on-device. ([#6834](https://github.com/NousResearch/hermes-agent/pull/6834))
|
||||
|
||||
- **Background Process Monitoring (`watch_patterns`)** — Set patterns to watch for in background process output and get notified in real-time when they match. Monitor for errors, wait for specific events ("listening on port"), or watch build logs — all without polling. ([#7635](https://github.com/NousResearch/hermes-agent/pull/7635))
|
||||
|
||||
- **Native xAI & Xiaomi MiMo Providers** — First-class provider support for xAI (Grok) and Xiaomi MiMo, with direct API access, model catalogs, and setup wizard integration. Plus Qwen OAuth with portal request support. ([#7372](https://github.com/NousResearch/hermes-agent/pull/7372), [#7855](https://github.com/NousResearch/hermes-agent/pull/7855))
|
||||
|
||||
- **Pluggable Context Engine** — Context management is now a pluggable slot via `hermes plugins`. Swap in custom context engines that control what the agent sees each turn — filtering, summarization, or domain-specific context injection. ([#7464](https://github.com/NousResearch/hermes-agent/pull/7464))
|
||||
|
||||
- **Unified Proxy Support** — SOCKS proxy, `DISCORD_PROXY`, and system proxy auto-detection across all gateway platforms. Hermes behind corporate firewalls just works. ([#6814](https://github.com/NousResearch/hermes-agent/pull/6814))
|
||||
|
||||
- **Comprehensive Security Hardening** — Path traversal protection in checkpoint manager, shell injection neutralization in sandbox writes, SSRF redirect guards in Slack image uploads, Twilio webhook signature validation (SMS RCE fix), API server auth enforcement, git argument injection prevention, and approval button authorization. ([#7933](https://github.com/NousResearch/hermes-agent/pull/7933), [#7944](https://github.com/NousResearch/hermes-agent/pull/7944), [#7940](https://github.com/NousResearch/hermes-agent/pull/7940), [#7151](https://github.com/NousResearch/hermes-agent/pull/7151), [#7156](https://github.com/NousResearch/hermes-agent/pull/7156))
|
||||
|
||||
- **`hermes backup` & `hermes import`** — Full backup and restore of your Hermes configuration, sessions, skills, and memory. Migrate between machines or create snapshots before major changes. ([#7997](https://github.com/NousResearch/hermes-agent/pull/7997))
|
||||
|
||||
- **16 Supported Platforms** — With BlueBubbles (iMessage) and WeChat joining Telegram, Discord, Slack, WhatsApp, Signal, Matrix, Email, SMS, DingTalk, Feishu, WeCom, Mattermost, Home Assistant, and Webhooks, Hermes now runs on 16 messaging platforms out of the box.
|
||||
|
||||
- **`/debug` & `hermes debug share`** — New debugging toolkit: `/debug` slash command across all platforms for quick diagnostics, plus `hermes debug share` to upload a full debug report to a pastebin for easy sharing when troubleshooting. ([#8681](https://github.com/NousResearch/hermes-agent/pull/8681))
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Core Agent & Architecture
|
||||
|
||||
### Provider & Model Support
|
||||
- **Native xAI (Grok) provider** with direct API access and model catalog ([#7372](https://github.com/NousResearch/hermes-agent/pull/7372))
|
||||
- **Xiaomi MiMo as first-class provider** — setup wizard, model catalog, empty response recovery ([#7855](https://github.com/NousResearch/hermes-agent/pull/7855))
|
||||
- **Qwen OAuth provider** with portal request support ([#6282](https://github.com/NousResearch/hermes-agent/pull/6282))
|
||||
- **Fast Mode** — `/fast` toggle for OpenAI Priority Processing + Anthropic fast tier ([#6875](https://github.com/NousResearch/hermes-agent/pull/6875), [#6960](https://github.com/NousResearch/hermes-agent/pull/6960), [#7037](https://github.com/NousResearch/hermes-agent/pull/7037))
|
||||
- **Structured API error classification** for smart failover decisions ([#6514](https://github.com/NousResearch/hermes-agent/pull/6514))
|
||||
- **Rate limit header capture** shown in `/usage` ([#6541](https://github.com/NousResearch/hermes-agent/pull/6541))
|
||||
- **API server model name** derived from profile name ([#6857](https://github.com/NousResearch/hermes-agent/pull/6857))
|
||||
- **Custom providers** now included in `/model` listings and resolution ([#7088](https://github.com/NousResearch/hermes-agent/pull/7088))
|
||||
- **Fallback provider activation** on repeated empty responses with user-visible status ([#7505](https://github.com/NousResearch/hermes-agent/pull/7505))
|
||||
- **OpenRouter variant tags** (`:free`, `:extended`, `:fast`) preserved during model switch ([#6383](https://github.com/NousResearch/hermes-agent/pull/6383))
|
||||
- **Credential exhaustion TTL** reduced from 24 hours to 1 hour ([#6504](https://github.com/NousResearch/hermes-agent/pull/6504))
|
||||
- **OAuth credential lifecycle** hardening — stale pool keys, auth.json sync, Codex CLI race fixes ([#6874](https://github.com/NousResearch/hermes-agent/pull/6874))
|
||||
- Empty response recovery for reasoning models (MiMo, Qwen, GLM) ([#8609](https://github.com/NousResearch/hermes-agent/pull/8609))
|
||||
- MiniMax context lengths, thinking guard, endpoint corrections ([#6082](https://github.com/NousResearch/hermes-agent/pull/6082), [#7126](https://github.com/NousResearch/hermes-agent/pull/7126))
|
||||
- Z.AI endpoint auto-detect via probe and cache ([#5763](https://github.com/NousResearch/hermes-agent/pull/5763))
|
||||
|
||||
### Agent Loop & Conversation
|
||||
- **Pluggable context engine slot** via `hermes plugins` ([#7464](https://github.com/NousResearch/hermes-agent/pull/7464))
|
||||
- **Background process monitoring** — `watch_patterns` for real-time output alerts ([#7635](https://github.com/NousResearch/hermes-agent/pull/7635))
|
||||
- **Improved context compression** — higher limits, tool tracking, degradation warnings, token-budget tail protection ([#6395](https://github.com/NousResearch/hermes-agent/pull/6395), [#6453](https://github.com/NousResearch/hermes-agent/pull/6453))
|
||||
- **`/compress <focus>`** — guided compression with a focus topic ([#8017](https://github.com/NousResearch/hermes-agent/pull/8017))
|
||||
- **Tiered context pressure warnings** with gateway dedup ([#6411](https://github.com/NousResearch/hermes-agent/pull/6411))
|
||||
- **Staged inactivity warning** before timeout escalation ([#6387](https://github.com/NousResearch/hermes-agent/pull/6387))
|
||||
- **Prevent agent from stopping mid-task** — compression floor, budget overhaul, activity tracking ([#7983](https://github.com/NousResearch/hermes-agent/pull/7983))
|
||||
- **Propagate child activity to parent** during `delegate_task` ([#7295](https://github.com/NousResearch/hermes-agent/pull/7295))
|
||||
- **Truncated streaming tool call detection** before execution ([#6847](https://github.com/NousResearch/hermes-agent/pull/6847))
|
||||
- Empty response retry (3 attempts with nudge) ([#6488](https://github.com/NousResearch/hermes-agent/pull/6488))
|
||||
- Adaptive streaming backoff + cursor strip to prevent message truncation ([#7683](https://github.com/NousResearch/hermes-agent/pull/7683))
|
||||
- Compression uses live session model instead of stale persisted config ([#8258](https://github.com/NousResearch/hermes-agent/pull/8258))
|
||||
- Strip `<thought>` tags from Gemma 4 responses ([#8562](https://github.com/NousResearch/hermes-agent/pull/8562))
|
||||
- Prevent `<think>` in prose from suppressing response output ([#6968](https://github.com/NousResearch/hermes-agent/pull/6968))
|
||||
- Turn-exit diagnostic logging to agent loop ([#6549](https://github.com/NousResearch/hermes-agent/pull/6549))
|
||||
- Scope tool interrupt signal per-thread to prevent cross-session leaks ([#7930](https://github.com/NousResearch/hermes-agent/pull/7930))
|
||||
|
||||
### Memory & Sessions
|
||||
- **Hindsight memory plugin** — feature parity, setup wizard, config improvements — @nicoloboschi ([#6428](https://github.com/NousResearch/hermes-agent/pull/6428))
|
||||
- **Honcho** — opt-in `initOnSessionStart` for tools mode — @Kathie-yu ([#6995](https://github.com/NousResearch/hermes-agent/pull/6995))
|
||||
- Orphan children instead of cascade-deleting in prune/delete ([#6513](https://github.com/NousResearch/hermes-agent/pull/6513))
|
||||
- Doctor command only checks the active memory provider ([#6285](https://github.com/NousResearch/hermes-agent/pull/6285))
|
||||
|
||||
---
|
||||
|
||||
## 📱 Messaging Platforms (Gateway)
|
||||
|
||||
### New Platforms
|
||||
- **BlueBubbles (iMessage)** — full adapter with auto-webhook registration, setup wizard, and crash resilience ([#6437](https://github.com/NousResearch/hermes-agent/pull/6437), [#6460](https://github.com/NousResearch/hermes-agent/pull/6460), [#6494](https://github.com/NousResearch/hermes-agent/pull/6494), [#7107](https://github.com/NousResearch/hermes-agent/pull/7107))
|
||||
- **Weixin (WeChat)** — native support via iLink Bot API with streaming, media uploads, markdown links ([#7166](https://github.com/NousResearch/hermes-agent/pull/7166), [#8665](https://github.com/NousResearch/hermes-agent/pull/8665))
|
||||
- **WeCom Callback Mode** — self-built enterprise app adapter with atomic state persistence ([#7943](https://github.com/NousResearch/hermes-agent/pull/7943), [#7928](https://github.com/NousResearch/hermes-agent/pull/7928))
|
||||
|
||||
### Discord
|
||||
- **Allowed channels whitelist** config — @jarvis-phw ([#7044](https://github.com/NousResearch/hermes-agent/pull/7044))
|
||||
- **Forum channel topic inheritance** in thread sessions — @hermes-agent-dhabibi ([#6377](https://github.com/NousResearch/hermes-agent/pull/6377))
|
||||
- **DISCORD_REPLY_TO_MODE** setting ([#6333](https://github.com/NousResearch/hermes-agent/pull/6333))
|
||||
- Accept `.log` attachments, raise document size limit — @kira-ariaki ([#6467](https://github.com/NousResearch/hermes-agent/pull/6467))
|
||||
- Decouple readiness from slash sync ([#8016](https://github.com/NousResearch/hermes-agent/pull/8016))
|
||||
|
||||
### Slack
|
||||
- **Consolidated Slack improvements** — 7 community PRs salvaged into one ([#6809](https://github.com/NousResearch/hermes-agent/pull/6809))
|
||||
- Handle assistant thread lifecycle events ([#6433](https://github.com/NousResearch/hermes-agent/pull/6433))
|
||||
|
||||
### Matrix
|
||||
- **Migrated from matrix-nio to mautrix-python** ([#7518](https://github.com/NousResearch/hermes-agent/pull/7518))
|
||||
- SQLite crypto store replacing pickle (fixes E2EE decryption) — @alt-glitch ([#7981](https://github.com/NousResearch/hermes-agent/pull/7981))
|
||||
- Cross-signing recovery key verification for E2EE migration ([#8282](https://github.com/NousResearch/hermes-agent/pull/8282))
|
||||
- DM mention threads + group chat events for Feishu ([#7423](https://github.com/NousResearch/hermes-agent/pull/7423))
|
||||
|
||||
### Gateway Core
|
||||
- **Unified proxy support** — SOCKS, DISCORD_PROXY, multi-platform with macOS auto-detection ([#6814](https://github.com/NousResearch/hermes-agent/pull/6814))
|
||||
- **Inbound text batching** for Discord, Matrix, WeCom + adaptive delay ([#6979](https://github.com/NousResearch/hermes-agent/pull/6979))
|
||||
- **Surface natural mid-turn assistant messages** in chat platforms ([#7978](https://github.com/NousResearch/hermes-agent/pull/7978))
|
||||
- **WSL-aware gateway** with smart systemd detection ([#7510](https://github.com/NousResearch/hermes-agent/pull/7510))
|
||||
- **All missing platforms added to setup wizard** ([#7949](https://github.com/NousResearch/hermes-agent/pull/7949))
|
||||
- **Per-platform `tool_progress` overrides** ([#6348](https://github.com/NousResearch/hermes-agent/pull/6348))
|
||||
- **Configurable 'still working' notification interval** ([#8572](https://github.com/NousResearch/hermes-agent/pull/8572))
|
||||
- `/model` switch persists across messages ([#7081](https://github.com/NousResearch/hermes-agent/pull/7081))
|
||||
- `/usage` shows rate limits, cost, and token details between turns ([#7038](https://github.com/NousResearch/hermes-agent/pull/7038))
|
||||
- Drain in-flight work before restart ([#7503](https://github.com/NousResearch/hermes-agent/pull/7503))
|
||||
- Don't evict cached agent on failed runs — prevents MCP restart loop ([#7539](https://github.com/NousResearch/hermes-agent/pull/7539))
|
||||
- Replace `os.environ` session state with `contextvars` ([#7454](https://github.com/NousResearch/hermes-agent/pull/7454))
|
||||
- Derive channel directory platforms from enum instead of hardcoded list ([#7450](https://github.com/NousResearch/hermes-agent/pull/7450))
|
||||
- Validate image downloads before caching (cross-platform) ([#7125](https://github.com/NousResearch/hermes-agent/pull/7125))
|
||||
- Cross-platform webhook delivery for all platforms ([#7095](https://github.com/NousResearch/hermes-agent/pull/7095))
|
||||
- Cron Discord thread_id delivery support ([#7106](https://github.com/NousResearch/hermes-agent/pull/7106))
|
||||
- Feishu QR-based bot onboarding ([#8570](https://github.com/NousResearch/hermes-agent/pull/8570))
|
||||
- Gateway status scoped to active profile ([#7951](https://github.com/NousResearch/hermes-agent/pull/7951))
|
||||
- Prevent background process notifications from triggering false pairing requests ([#6434](https://github.com/NousResearch/hermes-agent/pull/6434))
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ CLI & User Experience
|
||||
|
||||
### Interactive CLI
|
||||
- **Termux / Android support** — adapted install paths, TUI, voice, `/image` ([#6834](https://github.com/NousResearch/hermes-agent/pull/6834))
|
||||
- **Native `/model` picker modal** for provider → model selection ([#8003](https://github.com/NousResearch/hermes-agent/pull/8003))
|
||||
- **Live per-tool elapsed timer** restored in TUI spinner ([#7359](https://github.com/NousResearch/hermes-agent/pull/7359))
|
||||
- **Stacked tool progress scrollback** in TUI ([#8201](https://github.com/NousResearch/hermes-agent/pull/8201))
|
||||
- **Random tips on new session start** (CLI + gateway, 279 tips) ([#8225](https://github.com/NousResearch/hermes-agent/pull/8225), [#8237](https://github.com/NousResearch/hermes-agent/pull/8237))
|
||||
- **`hermes dump`** — copy-pasteable setup summary for debugging ([#6550](https://github.com/NousResearch/hermes-agent/pull/6550))
|
||||
- **`hermes backup` / `hermes import`** — full config backup and restore ([#7997](https://github.com/NousResearch/hermes-agent/pull/7997))
|
||||
- **WSL environment hint** in system prompt ([#8285](https://github.com/NousResearch/hermes-agent/pull/8285))
|
||||
- **Profile creation UX** — seed SOUL.md + credential warning ([#8553](https://github.com/NousResearch/hermes-agent/pull/8553))
|
||||
- Shell-aware sudo detection, empty password support ([#6517](https://github.com/NousResearch/hermes-agent/pull/6517))
|
||||
- Flush stdin after curses/terminal menus to prevent escape sequence leakage ([#7167](https://github.com/NousResearch/hermes-agent/pull/7167))
|
||||
- Handle broken stdin in prompt_toolkit startup ([#8560](https://github.com/NousResearch/hermes-agent/pull/8560))
|
||||
|
||||
### Setup & Configuration
|
||||
- **Per-platform display verbosity** configuration ([#8006](https://github.com/NousResearch/hermes-agent/pull/8006))
|
||||
- **Component-separated logging** with session context and filtering ([#7991](https://github.com/NousResearch/hermes-agent/pull/7991))
|
||||
- **`network.force_ipv4`** config to fix IPv6 timeout issues ([#8196](https://github.com/NousResearch/hermes-agent/pull/8196))
|
||||
- **Standardize message whitespace and JSON formatting** ([#7988](https://github.com/NousResearch/hermes-agent/pull/7988))
|
||||
- **Rebrand OpenClaw → Hermes** during migration ([#8210](https://github.com/NousResearch/hermes-agent/pull/8210))
|
||||
- Config.yaml takes priority over env vars for auxiliary settings ([#7889](https://github.com/NousResearch/hermes-agent/pull/7889))
|
||||
- Harden setup provider flows + live OpenRouter catalog refresh ([#7078](https://github.com/NousResearch/hermes-agent/pull/7078))
|
||||
- Normalize reasoning effort ordering across all surfaces ([#6804](https://github.com/NousResearch/hermes-agent/pull/6804))
|
||||
- Remove dead `LLM_MODEL` env var + migration to clear stale entries ([#6543](https://github.com/NousResearch/hermes-agent/pull/6543))
|
||||
- Remove `/prompt` slash command — prefix expansion footgun ([#6752](https://github.com/NousResearch/hermes-agent/pull/6752))
|
||||
- `HERMES_HOME_MODE` env var to override permissions — @ygd58 ([#6993](https://github.com/NousResearch/hermes-agent/pull/6993))
|
||||
- Fall back to default model when model config is empty ([#8303](https://github.com/NousResearch/hermes-agent/pull/8303))
|
||||
- Warn when compression model context is too small ([#7894](https://github.com/NousResearch/hermes-agent/pull/7894))
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Tool System
|
||||
|
||||
### Environments & Execution
|
||||
- **Unified spawn-per-call execution layer** for environments ([#6343](https://github.com/NousResearch/hermes-agent/pull/6343))
|
||||
- **Unified file sync** with mtime tracking, deletion, and transactional state ([#7087](https://github.com/NousResearch/hermes-agent/pull/7087))
|
||||
- **Persistent sandbox envs** survive between turns ([#6412](https://github.com/NousResearch/hermes-agent/pull/6412))
|
||||
- **Bulk file sync** via tar pipe for SSH/Modal backends — @alt-glitch ([#8014](https://github.com/NousResearch/hermes-agent/pull/8014))
|
||||
- **Daytona** — bulk upload, config bridge, silent disk cap ([#7538](https://github.com/NousResearch/hermes-agent/pull/7538))
|
||||
- Foreground timeout cap to prevent session deadlocks ([#7082](https://github.com/NousResearch/hermes-agent/pull/7082))
|
||||
- Guard invalid command values ([#6417](https://github.com/NousResearch/hermes-agent/pull/6417))
|
||||
|
||||
### MCP
|
||||
- **`hermes mcp add --env` and `--preset`** support ([#7970](https://github.com/NousResearch/hermes-agent/pull/7970))
|
||||
- Combine `content` and `structuredContent` when both present ([#7118](https://github.com/NousResearch/hermes-agent/pull/7118))
|
||||
- MCP tool name deconfliction fixes ([#7654](https://github.com/NousResearch/hermes-agent/pull/7654))
|
||||
|
||||
### Browser
|
||||
- Browser hardening — dead code removal, caching, scroll perf, security, thread safety ([#7354](https://github.com/NousResearch/hermes-agent/pull/7354))
|
||||
- `/browser connect` auto-launch uses dedicated Chrome profile dir ([#6821](https://github.com/NousResearch/hermes-agent/pull/6821))
|
||||
- Reap orphaned browser sessions on startup ([#7931](https://github.com/NousResearch/hermes-agent/pull/7931))
|
||||
|
||||
### Voice & Vision
|
||||
- **Voxtral TTS provider** (Mistral AI) ([#7653](https://github.com/NousResearch/hermes-agent/pull/7653))
|
||||
- **TTS speed support** for Edge TTS, OpenAI TTS, MiniMax ([#8666](https://github.com/NousResearch/hermes-agent/pull/8666))
|
||||
- **Vision auto-resize** for oversized images, raise limit to 20 MB, retry-on-failure ([#7883](https://github.com/NousResearch/hermes-agent/pull/7883), [#7902](https://github.com/NousResearch/hermes-agent/pull/7902))
|
||||
- STT provider-model mismatch fix (whisper-1 vs faster-whisper) ([#7113](https://github.com/NousResearch/hermes-agent/pull/7113))
|
||||
|
||||
### Other Tools
|
||||
- **`hermes dump`** command for setup summary ([#6550](https://github.com/NousResearch/hermes-agent/pull/6550))
|
||||
- TODO store enforces ID uniqueness during replace operations ([#7986](https://github.com/NousResearch/hermes-agent/pull/7986))
|
||||
- List all available toolsets in `delegate_task` schema description ([#8231](https://github.com/NousResearch/hermes-agent/pull/8231))
|
||||
- API server: tool progress as custom SSE event to prevent model corruption ([#7500](https://github.com/NousResearch/hermes-agent/pull/7500))
|
||||
- API server: share one Docker container across all conversations ([#7127](https://github.com/NousResearch/hermes-agent/pull/7127))
|
||||
|
||||
---
|
||||
|
||||
## 🧩 Skills Ecosystem
|
||||
|
||||
- **Centralized skills index + tree cache** — eliminates rate-limit failures on install ([#8575](https://github.com/NousResearch/hermes-agent/pull/8575))
|
||||
- **More aggressive skill loading instructions** in system prompt (v3) ([#8209](https://github.com/NousResearch/hermes-agent/pull/8209), [#8286](https://github.com/NousResearch/hermes-agent/pull/8286))
|
||||
- **Google Workspace skill** migrated to GWS CLI backend ([#6788](https://github.com/NousResearch/hermes-agent/pull/6788))
|
||||
- **Creative divergence strategies** skill — @SHL0MS ([#6882](https://github.com/NousResearch/hermes-agent/pull/6882))
|
||||
- **Creative ideation** — constraint-driven project generation — @SHL0MS ([#7555](https://github.com/NousResearch/hermes-agent/pull/7555))
|
||||
- Parallelize skills browse/search to prevent hanging ([#7301](https://github.com/NousResearch/hermes-agent/pull/7301))
|
||||
- Read name from SKILL.md frontmatter in skills_sync ([#7623](https://github.com/NousResearch/hermes-agent/pull/7623))
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security & Reliability
|
||||
|
||||
### Security Hardening
|
||||
- **Twilio webhook signature validation** — SMS RCE fix ([#7933](https://github.com/NousResearch/hermes-agent/pull/7933))
|
||||
- **Shell injection neutralization** in `_write_to_sandbox` via path quoting ([#7940](https://github.com/NousResearch/hermes-agent/pull/7940))
|
||||
- **Git argument injection** and path traversal prevention in checkpoint manager ([#7944](https://github.com/NousResearch/hermes-agent/pull/7944))
|
||||
- **SSRF redirect bypass** in Slack image uploads + base.py cache helpers ([#7151](https://github.com/NousResearch/hermes-agent/pull/7151))
|
||||
- **Path traversal, credential gate, DANGEROUS_PATTERNS gaps** ([#7156](https://github.com/NousResearch/hermes-agent/pull/7156))
|
||||
- **API bind guard** — enforce `API_SERVER_KEY` for non-loopback binding ([#7455](https://github.com/NousResearch/hermes-agent/pull/7455))
|
||||
- **Approval button authorization** — require auth for session continuation — @Cafexss ([#6930](https://github.com/NousResearch/hermes-agent/pull/6930))
|
||||
- Path boundary enforcement in skill manager operations ([#7156](https://github.com/NousResearch/hermes-agent/pull/7156))
|
||||
- DingTalk/API webhook URL origin validation, header injection rejection ([#7455](https://github.com/NousResearch/hermes-agent/pull/7455))
|
||||
|
||||
### Reliability
|
||||
- **Contextual error diagnostics** for invalid API responses ([#8565](https://github.com/NousResearch/hermes-agent/pull/8565))
|
||||
- **Prevent 400 format errors** from triggering compression loop on Codex ([#6751](https://github.com/NousResearch/hermes-agent/pull/6751))
|
||||
- **Don't halve context_length** on output-cap-too-large errors — @KUSH42 ([#6664](https://github.com/NousResearch/hermes-agent/pull/6664))
|
||||
- **Recover primary client** on OpenAI transport errors ([#7108](https://github.com/NousResearch/hermes-agent/pull/7108))
|
||||
- **Credential pool rotation** on billing-classified 400s ([#7112](https://github.com/NousResearch/hermes-agent/pull/7112))
|
||||
- **Auto-increase stream read timeout** for local LLM providers ([#6967](https://github.com/NousResearch/hermes-agent/pull/6967))
|
||||
- **Fall back to default certs** when CA bundle path doesn't exist ([#7352](https://github.com/NousResearch/hermes-agent/pull/7352))
|
||||
- **Disambiguate usage-limit patterns** in error classifier — @sprmn24 ([#6836](https://github.com/NousResearch/hermes-agent/pull/6836))
|
||||
- Harden cron script timeout and provider recovery ([#7079](https://github.com/NousResearch/hermes-agent/pull/7079))
|
||||
- Gateway interrupt detection resilient to monitor task failures ([#8208](https://github.com/NousResearch/hermes-agent/pull/8208))
|
||||
- Prevent unwanted session auto-reset after graceful gateway restarts ([#8299](https://github.com/NousResearch/hermes-agent/pull/8299))
|
||||
- Prevent duplicate update prompt spam in gateway watcher ([#8343](https://github.com/NousResearch/hermes-agent/pull/8343))
|
||||
- Deduplicate reasoning items in Responses API input ([#7946](https://github.com/NousResearch/hermes-agent/pull/7946))
|
||||
|
||||
### Infrastructure
|
||||
- **Multi-arch Docker image** — amd64 + arm64 ([#6124](https://github.com/NousResearch/hermes-agent/pull/6124))
|
||||
- **Docker runs as non-root user** with virtualenv — @benbarclay contributing ([#8226](https://github.com/NousResearch/hermes-agent/pull/8226))
|
||||
- **Use `uv`** for Docker dependency resolution to fix resolution-too-deep ([#6965](https://github.com/NousResearch/hermes-agent/pull/6965))
|
||||
- **Container-aware Nix CLI** — auto-route into managed container — @alt-glitch ([#7543](https://github.com/NousResearch/hermes-agent/pull/7543))
|
||||
- **Nix shared-state permission model** for interactive CLI users — @alt-glitch ([#6796](https://github.com/NousResearch/hermes-agent/pull/6796))
|
||||
- **Per-profile subprocess HOME isolation** ([#7357](https://github.com/NousResearch/hermes-agent/pull/7357))
|
||||
- Profile paths fixed in Docker — profiles go to mounted volume ([#7170](https://github.com/NousResearch/hermes-agent/pull/7170))
|
||||
- Docker container gateway pathway hardened ([#8614](https://github.com/NousResearch/hermes-agent/pull/8614))
|
||||
- Enable unbuffered stdout for live Docker logs ([#6749](https://github.com/NousResearch/hermes-agent/pull/6749))
|
||||
- Install procps in Docker image — @HiddenPuppy ([#7032](https://github.com/NousResearch/hermes-agent/pull/7032))
|
||||
- Shallow git clone for faster installation — @sosyz ([#8396](https://github.com/NousResearch/hermes-agent/pull/8396))
|
||||
- `hermes update` always reset on stash conflict ([#7010](https://github.com/NousResearch/hermes-agent/pull/7010))
|
||||
- Write update exit code before gateway restart (cgroup kill race) ([#8288](https://github.com/NousResearch/hermes-agent/pull/8288))
|
||||
- Nix: `setupSecrets` optional, tirith runtime dep — @devorun, @ethernet8023 ([#6261](https://github.com/NousResearch/hermes-agent/pull/6261), [#6721](https://github.com/NousResearch/hermes-agent/pull/6721))
|
||||
- launchd stop uses `bootout` so `KeepAlive` doesn't respawn ([#7119](https://github.com/NousResearch/hermes-agent/pull/7119))
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Notable Bug Fixes
|
||||
|
||||
- Fix: `/model` switch not persisting across gateway messages ([#7081](https://github.com/NousResearch/hermes-agent/pull/7081))
|
||||
- Fix: session-scoped gateway model overrides ignored — @Hygaard ([#7662](https://github.com/NousResearch/hermes-agent/pull/7662))
|
||||
- Fix: compaction model context length ignoring config — 3 related issues ([#8258](https://github.com/NousResearch/hermes-agent/pull/8258), [#8107](https://github.com/NousResearch/hermes-agent/pull/8107))
|
||||
- Fix: OpenCode.ai context window resolved to 128K instead of 1M ([#6472](https://github.com/NousResearch/hermes-agent/pull/6472))
|
||||
- Fix: Codex fallback auth-store lookup — @cherifya ([#6462](https://github.com/NousResearch/hermes-agent/pull/6462))
|
||||
- Fix: duplicate completion notifications when process killed ([#7124](https://github.com/NousResearch/hermes-agent/pull/7124))
|
||||
- Fix: agent daemon thread prevents orphan CLI processes on tab close ([#8557](https://github.com/NousResearch/hermes-agent/pull/8557))
|
||||
- Fix: stale image attachment on text paste and voice input ([#7077](https://github.com/NousResearch/hermes-agent/pull/7077))
|
||||
- Fix: DM thread session seeding causing cross-thread contamination ([#7084](https://github.com/NousResearch/hermes-agent/pull/7084))
|
||||
- Fix: OpenClaw migration shows dry-run preview before executing ([#6769](https://github.com/NousResearch/hermes-agent/pull/6769))
|
||||
- Fix: auth errors misclassified as retryable — @kuishou68 ([#7027](https://github.com/NousResearch/hermes-agent/pull/7027))
|
||||
- Fix: Copilot-Integration-Id header missing ([#7083](https://github.com/NousResearch/hermes-agent/pull/7083))
|
||||
- Fix: ACP session capabilities — @luyao618 ([#6985](https://github.com/NousResearch/hermes-agent/pull/6985))
|
||||
- Fix: ACP PromptResponse usage from top-level fields ([#7086](https://github.com/NousResearch/hermes-agent/pull/7086))
|
||||
- Fix: several failing/flaky tests on main — @dsocolobsky ([#6777](https://github.com/NousResearch/hermes-agent/pull/6777))
|
||||
- Fix: backup marker filenames — @sprmn24 ([#8600](https://github.com/NousResearch/hermes-agent/pull/8600))
|
||||
- Fix: `NoneType` in fast_mode check — @0xbyt4 ([#7350](https://github.com/NousResearch/hermes-agent/pull/7350))
|
||||
- Fix: missing imports in uninstall.py — @JiayuuWang ([#7034](https://github.com/NousResearch/hermes-agent/pull/7034))
|
||||
|
||||
---
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
- Platform adapter developer guide + WeCom Callback docs ([#7969](https://github.com/NousResearch/hermes-agent/pull/7969))
|
||||
- Cron troubleshooting guide ([#7122](https://github.com/NousResearch/hermes-agent/pull/7122))
|
||||
- Streaming timeout auto-detection for local LLMs ([#6990](https://github.com/NousResearch/hermes-agent/pull/6990))
|
||||
- Tool-use enforcement documentation expanded ([#7984](https://github.com/NousResearch/hermes-agent/pull/7984))
|
||||
- BlueBubbles pairing instructions ([#6548](https://github.com/NousResearch/hermes-agent/pull/6548))
|
||||
- Telegram proxy support section ([#6348](https://github.com/NousResearch/hermes-agent/pull/6348))
|
||||
- `hermes dump` and `hermes logs` CLI reference ([#6552](https://github.com/NousResearch/hermes-agent/pull/6552))
|
||||
- `tool_progress_overrides` configuration reference ([#6364](https://github.com/NousResearch/hermes-agent/pull/6364))
|
||||
- Compression model context length warning docs ([#7879](https://github.com/NousResearch/hermes-agent/pull/7879))
|
||||
|
||||
---
|
||||
|
||||
## 👥 Contributors
|
||||
|
||||
**269 merged PRs** from **24 contributors** across **487 commits**.
|
||||
|
||||
### Community Contributors
|
||||
- **@alt-glitch** (6 PRs) — Nix container-aware CLI, shared-state permissions, Matrix SQLite crypto store, bulk SSH/Modal file sync, Matrix mautrix compat
|
||||
- **@SHL0MS** (2 PRs) — Creative divergence strategies skill, creative ideation skill
|
||||
- **@sprmn24** (2 PRs) — Error classifier disambiguation, backup marker fix
|
||||
- **@nicoloboschi** — Hindsight memory plugin feature parity
|
||||
- **@Hygaard** — Session-scoped gateway model override fix
|
||||
- **@jarvis-phw** — Discord allowed_channels whitelist
|
||||
- **@Kathie-yu** — Honcho initOnSessionStart for tools mode
|
||||
- **@hermes-agent-dhabibi** — Discord forum channel topic inheritance
|
||||
- **@kira-ariaki** — Discord .log attachments and size limit
|
||||
- **@cherifya** — Codex fallback auth-store lookup
|
||||
- **@Cafexss** — Security: auth for session continuation
|
||||
- **@KUSH42** — Compaction context_length fix
|
||||
- **@kuishou68** — Auth error retryable classification fix
|
||||
- **@luyao618** — ACP session capabilities
|
||||
- **@ygd58** — HERMES_HOME_MODE env var override
|
||||
- **@0xbyt4** — Fast mode NoneType fix
|
||||
- **@JiayuuWang** — CLI uninstall import fix
|
||||
- **@HiddenPuppy** — Docker procps installation
|
||||
- **@dsocolobsky** — Test suite fixes
|
||||
- **@bobashopcashier** (1 PR) — Graceful gateway drain before restart (salvaged into #7503 from #7290)
|
||||
- **@benbarclay** — Docker image tag simplification
|
||||
- **@sosyz** — Shallow git clone for faster install
|
||||
- **@devorun** — Nix setupSecrets optional
|
||||
- **@ethernet8023** — Nix tirith runtime dep
|
||||
|
||||
---
|
||||
|
||||
**Full Changelog**: [v2026.4.8...v2026.4.13](https://github.com/NousResearch/hermes-agent/compare/v2026.4.8...v2026.4.13)
|
||||
-84
@@ -1,84 +0,0 @@
|
||||
# Hermes Agent Security Policy
|
||||
|
||||
This document outlines the security protocols, trust model, and deployment hardening guidelines for the **Hermes Agent** project.
|
||||
|
||||
## 1. Vulnerability Reporting
|
||||
|
||||
Hermes Agent does **not** operate a bug bounty program. Security issues should be reported via [GitHub Security Advisories (GHSA)](https://github.com/NousResearch/hermes-agent/security/advisories/new) or by emailing **security@nousresearch.com**. Do not open public issues for security vulnerabilities.
|
||||
|
||||
### Required Submission Details
|
||||
- **Title & Severity:** Concise description and CVSS score/rating.
|
||||
- **Affected Component:** Exact file path and line range (e.g., `tools/approval.py:120-145`).
|
||||
- **Environment:** Output of `hermes version`, commit SHA, OS, and Python version.
|
||||
- **Reproduction:** Step-by-step Proof-of-Concept (PoC) against `main` or the latest release.
|
||||
- **Impact:** Explanation of what trust boundary was crossed.
|
||||
|
||||
---
|
||||
|
||||
## 2. Trust Model
|
||||
|
||||
The core assumption is that Hermes is a **personal agent** with one trusted operator.
|
||||
|
||||
### Operator & Session Trust
|
||||
- **Single Tenant:** The system protects the operator from LLM actions, not from malicious co-tenants. Multi-user isolation must happen at the OS/host level.
|
||||
- **Gateway Security:** Authorized callers (Telegram, Discord, Slack, etc.) receive equal trust. Session keys are used for routing, not as authorization boundaries.
|
||||
- **Execution:** Defaults to `terminal.backend: local` (direct host execution). Container isolation (Docker, Modal, Daytona) is opt-in for sandboxing.
|
||||
|
||||
### Dangerous Command Approval
|
||||
The approval system (`tools/approval.py`) is a core security boundary. Terminal commands, file operations, and other potentially destructive actions are gated behind explicit user confirmation before execution. The approval mode is configurable via `approvals.mode` in `config.yaml`:
|
||||
- `"on"` (default) — prompts the user to approve dangerous commands.
|
||||
- `"auto"` — auto-approves after a configurable delay.
|
||||
- `"off"` — disables the gate entirely (break-glass; see Section 3).
|
||||
|
||||
### Output Redaction
|
||||
`agent/redact.py` strips secret-like patterns (API keys, tokens, credentials) from all display output before it reaches the terminal or gateway platform. This prevents accidental credential leakage in chat logs, tool previews, and response text. Redaction operates on the display layer only — underlying values remain intact for internal agent operations.
|
||||
|
||||
### Skills vs. MCP Servers
|
||||
- **Installed Skills:** High trust. Equivalent to local host code; skills can read environment variables and run arbitrary commands.
|
||||
- **MCP Servers:** Lower trust. MCP subprocesses receive a filtered environment (`_build_safe_env()` in `tools/mcp_tool.py`) — only safe baseline variables (`PATH`, `HOME`, `XDG_*`) plus variables explicitly declared in the server's `env` config block are passed through. Host credentials are stripped by default. Additionally, packages invoked via `npx`/`uvx` are checked against the OSV malware database before spawning.
|
||||
|
||||
### Code Execution Sandbox
|
||||
The `execute_code` tool (`tools/code_execution_tool.py`) runs LLM-generated Python scripts in a child process with API keys and tokens stripped from the environment to prevent credential exfiltration. Only environment variables explicitly declared by loaded skills (via `env_passthrough`) or by the user in `config.yaml` (`terminal.env_passthrough`) are passed through. The child accesses Hermes tools via RPC, not direct API calls.
|
||||
|
||||
### Subagents
|
||||
- **No recursive delegation:** The `delegate_task` tool is disabled for child agents.
|
||||
- **Depth limit:** `MAX_DEPTH = 2` — parent (depth 0) can spawn a child (depth 1); grandchildren are rejected.
|
||||
- **Memory isolation:** Subagents run with `skip_memory=True` and do not have access to the parent's persistent memory provider. The parent receives only the task prompt and final response as an observation.
|
||||
|
||||
---
|
||||
|
||||
## 3. Out of Scope (Non-Vulnerabilities)
|
||||
|
||||
The following scenarios are **not** considered security breaches:
|
||||
- **Prompt Injection:** Unless it results in a concrete bypass of the approval system, toolset restrictions, or container sandbox.
|
||||
- **Public Exposure:** Deploying the gateway to the public internet without external authentication or network protection.
|
||||
- **Trusted State Access:** Reports that require pre-existing write access to `~/.hermes/`, `.env`, or `config.yaml` (these are operator-owned files).
|
||||
- **Default Behavior:** Host-level command execution when `terminal.backend` is set to `local` — this is the documented default, not a vulnerability.
|
||||
- **Configuration Trade-offs:** Intentional break-glass settings such as `approvals.mode: "off"` or `terminal.backend: local` in production.
|
||||
- **Tool-level read/access restrictions:** The agent has unrestricted shell access via the `terminal` tool by design. Reports that a specific tool (e.g., `read_file`) can access a resource are not vulnerabilities if the same access is available through `terminal`. Tool-level deny lists only constitute a meaningful security boundary when paired with equivalent restrictions on the terminal side (as with write operations, where `WRITE_DENIED_PATHS` is paired with the dangerous command approval system).
|
||||
|
||||
---
|
||||
|
||||
## 4. Deployment Hardening & Best Practices
|
||||
|
||||
### Filesystem & Network
|
||||
- **Production sandboxing:** Use container backends (`docker`, `modal`, `daytona`) instead of `local` for untrusted workloads.
|
||||
- **File permissions:** Run as non-root (the Docker image uses UID 10000); protect credentials with `chmod 600 ~/.hermes/.env` on local installs.
|
||||
- **Network exposure:** Do not expose the gateway or API server to the public internet without VPN, Tailscale, or firewall protection. SSRF protection is enabled by default across all gateway platform adapters (Telegram, Discord, Slack, Matrix, Mattermost, etc.) with redirect validation. Note: the local terminal backend does not apply SSRF filtering, as it operates within the trusted operator's environment.
|
||||
|
||||
### Skills & Supply Chain
|
||||
- **Skill installation:** Review Skills Guard reports (`tools/skills_guard.py`) before installing third-party skills. The audit log at `~/.hermes/skills/.hub/audit.log` tracks every install and removal.
|
||||
- **MCP safety:** OSV malware checking runs automatically for `npx`/`uvx` packages before MCP server processes are spawned.
|
||||
- **CI/CD:** GitHub Actions are pinned to full commit SHAs. The `supply-chain-audit.yml` workflow blocks PRs containing `.pth` files or suspicious `base64`+`exec` patterns.
|
||||
|
||||
### Credential Storage
|
||||
- API keys and tokens belong exclusively in `~/.hermes/.env` — never in `config.yaml` or checked into version control.
|
||||
- The credential pool system (`agent/credential_pool.py`) handles key rotation and fallback. Credentials are resolved from environment variables, not stored in plaintext databases.
|
||||
|
||||
---
|
||||
|
||||
## 5. Disclosure Process
|
||||
|
||||
- **Coordinated Disclosure:** 90-day window or until a fix is released, whichever comes first.
|
||||
- **Communication:** All updates occur via the GHSA thread or email correspondence with security@nousresearch.com.
|
||||
- **Credits:** Reporters are credited in release notes unless anonymity is requested.
|
||||
+1
-20
@@ -49,7 +49,6 @@ def make_tool_progress_cb(
|
||||
session_id: str,
|
||||
loop: asyncio.AbstractEventLoop,
|
||||
tool_call_ids: Dict[str, Deque[str]],
|
||||
tool_call_meta: Dict[str, Dict[str, Any]],
|
||||
) -> Callable:
|
||||
"""Create a ``tool_progress_callback`` for AIAgent.
|
||||
|
||||
@@ -85,16 +84,6 @@ def make_tool_progress_cb(
|
||||
tool_call_ids[name] = queue
|
||||
queue.append(tc_id)
|
||||
|
||||
snapshot = None
|
||||
if name in {"write_file", "patch", "skill_manage"}:
|
||||
try:
|
||||
from agent.display import capture_local_edit_snapshot
|
||||
|
||||
snapshot = capture_local_edit_snapshot(name, args)
|
||||
except Exception:
|
||||
logger.debug("Failed to capture ACP edit snapshot for %s", name, exc_info=True)
|
||||
tool_call_meta[tc_id] = {"args": args, "snapshot": snapshot}
|
||||
|
||||
update = build_tool_start(tc_id, name, args)
|
||||
_send_update(conn, session_id, loop, update)
|
||||
|
||||
@@ -130,7 +119,6 @@ def make_step_cb(
|
||||
session_id: str,
|
||||
loop: asyncio.AbstractEventLoop,
|
||||
tool_call_ids: Dict[str, Deque[str]],
|
||||
tool_call_meta: Dict[str, Dict[str, Any]],
|
||||
) -> Callable:
|
||||
"""Create a ``step_callback`` for AIAgent.
|
||||
|
||||
@@ -144,12 +132,10 @@ def make_step_cb(
|
||||
for tool_info in prev_tools:
|
||||
tool_name = None
|
||||
result = None
|
||||
function_args = None
|
||||
|
||||
if isinstance(tool_info, dict):
|
||||
tool_name = tool_info.get("name") or tool_info.get("function_name")
|
||||
result = tool_info.get("result") or tool_info.get("output")
|
||||
function_args = tool_info.get("arguments") or tool_info.get("args")
|
||||
elif isinstance(tool_info, str):
|
||||
tool_name = tool_info
|
||||
|
||||
@@ -159,13 +145,8 @@ def make_step_cb(
|
||||
tool_call_ids[tool_name] = queue
|
||||
if tool_name and queue:
|
||||
tc_id = queue.popleft()
|
||||
meta = tool_call_meta.pop(tc_id, {})
|
||||
update = build_tool_complete(
|
||||
tc_id,
|
||||
tool_name,
|
||||
result=str(result) if result is not None else None,
|
||||
function_args=function_args or meta.get("args"),
|
||||
snapshot=meta.get("snapshot"),
|
||||
tc_id, tool_name, result=str(result) if result is not None else None
|
||||
)
|
||||
_send_update(conn, session_id, loop, update)
|
||||
if not queue:
|
||||
|
||||
+30
-148
@@ -26,7 +26,6 @@ from acp.schema import (
|
||||
McpServerHttp,
|
||||
McpServerSse,
|
||||
McpServerStdio,
|
||||
ModelInfo,
|
||||
NewSessionResponse,
|
||||
PromptResponse,
|
||||
ResumeSessionResponse,
|
||||
@@ -37,7 +36,6 @@ from acp.schema import (
|
||||
SessionCapabilities,
|
||||
SessionForkCapabilities,
|
||||
SessionListCapabilities,
|
||||
SessionModelState,
|
||||
SessionResumeCapabilities,
|
||||
SessionInfo,
|
||||
TextContentBlock,
|
||||
@@ -149,98 +147,6 @@ class HermesACPAgent(acp.Agent):
|
||||
self._conn = conn
|
||||
logger.info("ACP client connected")
|
||||
|
||||
@staticmethod
|
||||
def _encode_model_choice(provider: str | None, model: str | None) -> str:
|
||||
"""Encode a model selection so ACP clients can keep provider context."""
|
||||
raw_model = str(model or "").strip()
|
||||
if not raw_model:
|
||||
return ""
|
||||
raw_provider = str(provider or "").strip().lower()
|
||||
if not raw_provider:
|
||||
return raw_model
|
||||
return f"{raw_provider}:{raw_model}"
|
||||
|
||||
def _build_model_state(self, state: SessionState) -> SessionModelState | None:
|
||||
"""Return the ACP model selector payload for editors like Zed."""
|
||||
model = str(state.model or getattr(state.agent, "model", "") or "").strip()
|
||||
provider = getattr(state.agent, "provider", None) or detect_provider() or "openrouter"
|
||||
|
||||
try:
|
||||
from hermes_cli.models import curated_models_for_provider, normalize_provider, provider_label
|
||||
|
||||
normalized_provider = normalize_provider(provider)
|
||||
provider_name = provider_label(normalized_provider)
|
||||
available_models: list[ModelInfo] = []
|
||||
seen_ids: set[str] = set()
|
||||
|
||||
for model_id, description in curated_models_for_provider(normalized_provider):
|
||||
rendered_model = str(model_id or "").strip()
|
||||
if not rendered_model:
|
||||
continue
|
||||
choice_id = self._encode_model_choice(normalized_provider, rendered_model)
|
||||
if choice_id in seen_ids:
|
||||
continue
|
||||
desc_parts = [f"Provider: {provider_name}"]
|
||||
if description:
|
||||
desc_parts.append(str(description).strip())
|
||||
if rendered_model == model:
|
||||
desc_parts.append("current")
|
||||
available_models.append(
|
||||
ModelInfo(
|
||||
model_id=choice_id,
|
||||
name=rendered_model,
|
||||
description=" • ".join(part for part in desc_parts if part),
|
||||
)
|
||||
)
|
||||
seen_ids.add(choice_id)
|
||||
|
||||
current_model_id = self._encode_model_choice(normalized_provider, model)
|
||||
if current_model_id and current_model_id not in seen_ids:
|
||||
available_models.insert(
|
||||
0,
|
||||
ModelInfo(
|
||||
model_id=current_model_id,
|
||||
name=model,
|
||||
description=f"Provider: {provider_name} • current",
|
||||
),
|
||||
)
|
||||
|
||||
if available_models:
|
||||
return SessionModelState(
|
||||
available_models=available_models,
|
||||
current_model_id=current_model_id or available_models[0].model_id,
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Could not build ACP model state", exc_info=True)
|
||||
|
||||
if not model:
|
||||
return None
|
||||
|
||||
fallback_choice = self._encode_model_choice(provider, model)
|
||||
return SessionModelState(
|
||||
available_models=[ModelInfo(model_id=fallback_choice, name=model)],
|
||||
current_model_id=fallback_choice,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _resolve_model_selection(raw_model: str, current_provider: str) -> tuple[str, str]:
|
||||
"""Resolve ``provider:model`` input into the provider and normalized model id."""
|
||||
target_provider = current_provider
|
||||
new_model = raw_model.strip()
|
||||
|
||||
try:
|
||||
from hermes_cli.models import detect_provider_for_model, parse_model_input
|
||||
|
||||
target_provider, new_model = parse_model_input(new_model, current_provider)
|
||||
if target_provider == current_provider:
|
||||
detected = detect_provider_for_model(new_model, current_provider)
|
||||
if detected:
|
||||
target_provider, new_model = detected
|
||||
except Exception:
|
||||
logger.debug("Provider detection failed, using model as-is", exc_info=True)
|
||||
|
||||
return target_provider, new_model
|
||||
|
||||
async def _register_session_mcp_servers(
|
||||
self,
|
||||
state: SessionState,
|
||||
@@ -367,10 +273,7 @@ class HermesACPAgent(acp.Agent):
|
||||
await self._register_session_mcp_servers(state, mcp_servers)
|
||||
logger.info("New session %s (cwd=%s)", state.session_id, cwd)
|
||||
self._schedule_available_commands_update(state.session_id)
|
||||
return NewSessionResponse(
|
||||
session_id=state.session_id,
|
||||
models=self._build_model_state(state),
|
||||
)
|
||||
return NewSessionResponse(session_id=state.session_id)
|
||||
|
||||
async def load_session(
|
||||
self,
|
||||
@@ -386,7 +289,7 @@ class HermesACPAgent(acp.Agent):
|
||||
await self._register_session_mcp_servers(state, mcp_servers)
|
||||
logger.info("Loaded session %s", session_id)
|
||||
self._schedule_available_commands_update(session_id)
|
||||
return LoadSessionResponse(models=self._build_model_state(state))
|
||||
return LoadSessionResponse()
|
||||
|
||||
async def resume_session(
|
||||
self,
|
||||
@@ -402,7 +305,7 @@ class HermesACPAgent(acp.Agent):
|
||||
await self._register_session_mcp_servers(state, mcp_servers)
|
||||
logger.info("Resumed session %s", state.session_id)
|
||||
self._schedule_available_commands_update(state.session_id)
|
||||
return ResumeSessionResponse(models=self._build_model_state(state))
|
||||
return ResumeSessionResponse()
|
||||
|
||||
async def cancel(self, session_id: str, **kwargs: Any) -> None:
|
||||
state = self.session_manager.get_session(session_id)
|
||||
@@ -437,20 +340,11 @@ class HermesACPAgent(acp.Agent):
|
||||
cwd: str | None = None,
|
||||
**kwargs: Any,
|
||||
) -> ListSessionsResponse:
|
||||
infos = self.session_manager.list_sessions(cwd=cwd)
|
||||
sessions = []
|
||||
for s in infos:
|
||||
updated_at = s.get("updated_at")
|
||||
if updated_at is not None and not isinstance(updated_at, str):
|
||||
updated_at = str(updated_at)
|
||||
sessions.append(
|
||||
SessionInfo(
|
||||
session_id=s["session_id"],
|
||||
cwd=s["cwd"],
|
||||
title=s.get("title"),
|
||||
updated_at=updated_at,
|
||||
)
|
||||
)
|
||||
infos = self.session_manager.list_sessions()
|
||||
sessions = [
|
||||
SessionInfo(session_id=s["session_id"], cwd=s["cwd"])
|
||||
for s in infos
|
||||
]
|
||||
return ListSessionsResponse(sessions=sessions)
|
||||
|
||||
# ---- Prompt (core) ------------------------------------------------------
|
||||
@@ -495,13 +389,12 @@ class HermesACPAgent(acp.Agent):
|
||||
state.cancel_event.clear()
|
||||
|
||||
tool_call_ids: dict[str, Deque[str]] = defaultdict(deque)
|
||||
tool_call_meta: dict[str, dict[str, Any]] = {}
|
||||
previous_approval_cb = None
|
||||
|
||||
if conn:
|
||||
tool_progress_cb = make_tool_progress_cb(conn, session_id, loop, tool_call_ids, tool_call_meta)
|
||||
tool_progress_cb = make_tool_progress_cb(conn, session_id, loop, tool_call_ids)
|
||||
thinking_cb = make_thinking_cb(conn, session_id, loop)
|
||||
step_cb = make_step_cb(conn, session_id, loop, tool_call_ids, tool_call_meta)
|
||||
step_cb = make_step_cb(conn, session_id, loop, tool_call_ids)
|
||||
message_cb = make_message_cb(conn, session_id, loop)
|
||||
approval_cb = make_approval_callback(conn.request_permission, loop, session_id)
|
||||
else:
|
||||
@@ -556,19 +449,6 @@ class HermesACPAgent(acp.Agent):
|
||||
self.session_manager.save_session(session_id)
|
||||
|
||||
final_response = result.get("final_response", "")
|
||||
if final_response:
|
||||
try:
|
||||
from agent.title_generator import maybe_auto_title
|
||||
|
||||
maybe_auto_title(
|
||||
self.session_manager._get_db(),
|
||||
session_id,
|
||||
user_text,
|
||||
final_response,
|
||||
state.history,
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Failed to auto-title ACP session %s", session_id, exc_info=True)
|
||||
if final_response and conn:
|
||||
update = acp.update_agent_message_text(final_response)
|
||||
await conn.session_update(session_id, update)
|
||||
@@ -676,15 +556,27 @@ class HermesACPAgent(acp.Agent):
|
||||
provider = getattr(state.agent, "provider", None) or "auto"
|
||||
return f"Current model: {model}\nProvider: {provider}"
|
||||
|
||||
new_model = args.strip()
|
||||
target_provider = None
|
||||
current_provider = getattr(state.agent, "provider", None) or "openrouter"
|
||||
target_provider, new_model = self._resolve_model_selection(args, current_provider)
|
||||
|
||||
# Auto-detect provider for the requested model
|
||||
try:
|
||||
from hermes_cli.models import parse_model_input, detect_provider_for_model
|
||||
target_provider, new_model = parse_model_input(new_model, current_provider)
|
||||
if target_provider == current_provider:
|
||||
detected = detect_provider_for_model(new_model, current_provider)
|
||||
if detected:
|
||||
target_provider, new_model = detected
|
||||
except Exception:
|
||||
logger.debug("Provider detection failed, using model as-is", exc_info=True)
|
||||
|
||||
state.model = new_model
|
||||
state.agent = self.session_manager._make_agent(
|
||||
session_id=state.session_id,
|
||||
cwd=state.cwd,
|
||||
model=new_model,
|
||||
requested_provider=target_provider,
|
||||
requested_provider=target_provider or current_provider,
|
||||
)
|
||||
self.session_manager.save_session(state.session_id)
|
||||
provider_label = getattr(state.agent, "provider", None) or target_provider or current_provider
|
||||
@@ -786,30 +678,20 @@ class HermesACPAgent(acp.Agent):
|
||||
"""Switch the model for a session (called by ACP protocol)."""
|
||||
state = self.session_manager.get_session(session_id)
|
||||
if state:
|
||||
state.model = model_id
|
||||
current_provider = getattr(state.agent, "provider", None)
|
||||
requested_provider, resolved_model = self._resolve_model_selection(
|
||||
model_id,
|
||||
current_provider or "openrouter",
|
||||
)
|
||||
state.model = resolved_model
|
||||
provider_changed = bool(current_provider and requested_provider != current_provider)
|
||||
current_base_url = None if provider_changed else getattr(state.agent, "base_url", None)
|
||||
current_api_mode = None if provider_changed else getattr(state.agent, "api_mode", None)
|
||||
current_base_url = getattr(state.agent, "base_url", None)
|
||||
current_api_mode = getattr(state.agent, "api_mode", None)
|
||||
state.agent = self.session_manager._make_agent(
|
||||
session_id=session_id,
|
||||
cwd=state.cwd,
|
||||
model=resolved_model,
|
||||
requested_provider=requested_provider,
|
||||
model=model_id,
|
||||
requested_provider=current_provider,
|
||||
base_url=current_base_url,
|
||||
api_mode=current_api_mode,
|
||||
)
|
||||
self.session_manager.save_session(session_id)
|
||||
logger.info(
|
||||
"Session %s: model switched to %s via provider %s",
|
||||
session_id,
|
||||
resolved_model,
|
||||
requested_provider,
|
||||
)
|
||||
logger.info("Session %s: model switched to %s", session_id, model_id)
|
||||
return SetSessionModelResponse()
|
||||
logger.warning("Session %s: model switch requested for missing session", session_id)
|
||||
return None
|
||||
|
||||
+34
-127
@@ -13,12 +13,8 @@ from hermes_constants import get_hermes_home
|
||||
import copy
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from dataclasses import dataclass, field
|
||||
from threading import Lock
|
||||
from typing import Any, Dict, List, Optional
|
||||
@@ -26,64 +22,6 @@ from typing import Any, Dict, List, Optional
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _normalize_cwd_for_compare(cwd: str | None) -> str:
|
||||
raw = str(cwd or ".").strip()
|
||||
if not raw:
|
||||
raw = "."
|
||||
expanded = os.path.expanduser(raw)
|
||||
|
||||
# Normalize Windows drive paths into the equivalent WSL mount form so
|
||||
# ACP history filters match the same workspace across Windows and WSL.
|
||||
match = re.match(r"^([A-Za-z]):[\\/](.*)$", expanded)
|
||||
if match:
|
||||
drive = match.group(1).lower()
|
||||
tail = match.group(2).replace("\\", "/")
|
||||
expanded = f"/mnt/{drive}/{tail}"
|
||||
elif re.match(r"^/mnt/[A-Za-z]/", expanded):
|
||||
expanded = f"/mnt/{expanded[5].lower()}/{expanded[7:]}"
|
||||
|
||||
return os.path.normpath(expanded)
|
||||
|
||||
|
||||
def _build_session_title(title: Any, preview: Any, cwd: str | None) -> str:
|
||||
explicit = str(title or "").strip()
|
||||
if explicit:
|
||||
return explicit
|
||||
preview_text = str(preview or "").strip()
|
||||
if preview_text:
|
||||
return preview_text
|
||||
leaf = os.path.basename(str(cwd or "").rstrip("/\\"))
|
||||
return leaf or "New thread"
|
||||
|
||||
|
||||
def _format_updated_at(value: Any) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, str) and value.strip():
|
||||
return value
|
||||
try:
|
||||
return datetime.fromtimestamp(float(value), tz=timezone.utc).isoformat()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _updated_at_sort_key(value: Any) -> float:
|
||||
if value is None:
|
||||
return float("-inf")
|
||||
if isinstance(value, (int, float)):
|
||||
return float(value)
|
||||
raw = str(value).strip()
|
||||
if not raw:
|
||||
return float("-inf")
|
||||
try:
|
||||
return datetime.fromisoformat(raw.replace("Z", "+00:00")).timestamp()
|
||||
except Exception:
|
||||
try:
|
||||
return float(raw)
|
||||
except Exception:
|
||||
return float("-inf")
|
||||
|
||||
|
||||
def _acp_stderr_print(*args, **kwargs) -> None:
|
||||
"""Best-effort human-readable output sink for ACP stdio sessions.
|
||||
|
||||
@@ -224,78 +162,47 @@ class SessionManager:
|
||||
logger.info("Forked ACP session %s -> %s", session_id, new_id)
|
||||
return state
|
||||
|
||||
def list_sessions(self, cwd: str | None = None) -> List[Dict[str, Any]]:
|
||||
def list_sessions(self) -> List[Dict[str, Any]]:
|
||||
"""Return lightweight info dicts for all sessions (memory + database)."""
|
||||
normalized_cwd = _normalize_cwd_for_compare(cwd) if cwd else None
|
||||
db = self._get_db()
|
||||
persisted_rows: dict[str, dict[str, Any]] = {}
|
||||
|
||||
if db is not None:
|
||||
try:
|
||||
for row in db.list_sessions_rich(source="acp", limit=1000):
|
||||
persisted_rows[str(row["id"])] = dict(row)
|
||||
except Exception:
|
||||
logger.debug("Failed to load ACP sessions from DB", exc_info=True)
|
||||
|
||||
# Collect in-memory sessions first.
|
||||
with self._lock:
|
||||
seen_ids = set(self._sessions.keys())
|
||||
results = []
|
||||
for s in self._sessions.values():
|
||||
history_len = len(s.history)
|
||||
if history_len <= 0:
|
||||
continue
|
||||
if normalized_cwd and _normalize_cwd_for_compare(s.cwd) != normalized_cwd:
|
||||
continue
|
||||
persisted = persisted_rows.get(s.session_id, {})
|
||||
preview = next(
|
||||
(
|
||||
str(msg.get("content") or "").strip()
|
||||
for msg in s.history
|
||||
if msg.get("role") == "user" and str(msg.get("content") or "").strip()
|
||||
),
|
||||
persisted.get("preview") or "",
|
||||
)
|
||||
results.append(
|
||||
{
|
||||
"session_id": s.session_id,
|
||||
"cwd": s.cwd,
|
||||
"model": s.model,
|
||||
"history_len": history_len,
|
||||
"title": _build_session_title(persisted.get("title"), preview, s.cwd),
|
||||
"updated_at": _format_updated_at(
|
||||
persisted.get("last_active") or persisted.get("started_at") or time.time()
|
||||
),
|
||||
}
|
||||
)
|
||||
results = [
|
||||
{
|
||||
"session_id": s.session_id,
|
||||
"cwd": s.cwd,
|
||||
"model": s.model,
|
||||
"history_len": len(s.history),
|
||||
}
|
||||
for s in self._sessions.values()
|
||||
]
|
||||
|
||||
# Merge any persisted sessions not currently in memory.
|
||||
for sid, row in persisted_rows.items():
|
||||
if sid in seen_ids:
|
||||
continue
|
||||
message_count = int(row.get("message_count") or 0)
|
||||
if message_count <= 0:
|
||||
continue
|
||||
# Extract cwd from model_config JSON.
|
||||
session_cwd = "."
|
||||
mc = row.get("model_config")
|
||||
if mc:
|
||||
try:
|
||||
session_cwd = json.loads(mc).get("cwd", ".")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
if normalized_cwd and _normalize_cwd_for_compare(session_cwd) != normalized_cwd:
|
||||
continue
|
||||
results.append({
|
||||
"session_id": sid,
|
||||
"cwd": session_cwd,
|
||||
"model": row.get("model") or "",
|
||||
"history_len": message_count,
|
||||
"title": _build_session_title(row.get("title"), row.get("preview"), session_cwd),
|
||||
"updated_at": _format_updated_at(row.get("last_active") or row.get("started_at")),
|
||||
})
|
||||
db = self._get_db()
|
||||
if db is not None:
|
||||
try:
|
||||
rows = db.search_sessions(source="acp", limit=1000)
|
||||
for row in rows:
|
||||
sid = row["id"]
|
||||
if sid in seen_ids:
|
||||
continue
|
||||
# Extract cwd from model_config JSON.
|
||||
cwd = "."
|
||||
mc = row.get("model_config")
|
||||
if mc:
|
||||
try:
|
||||
cwd = json.loads(mc).get("cwd", ".")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
results.append({
|
||||
"session_id": sid,
|
||||
"cwd": cwd,
|
||||
"model": row.get("model") or "",
|
||||
"history_len": row.get("message_count") or 0,
|
||||
})
|
||||
except Exception:
|
||||
logger.debug("Failed to list ACP sessions from DB", exc_info=True)
|
||||
|
||||
results.sort(key=lambda item: _updated_at_sort_key(item.get("updated_at")), reverse=True)
|
||||
return results
|
||||
|
||||
def update_cwd(self, session_id: str, cwd: str) -> Optional[SessionState]:
|
||||
|
||||
+9
-174
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import uuid
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
@@ -97,170 +96,6 @@ def build_tool_title(tool_name: str, args: Dict[str, Any]) -> str:
|
||||
return tool_name
|
||||
|
||||
|
||||
def _build_patch_mode_content(patch_text: str) -> List[Any]:
|
||||
"""Parse V4A patch mode input into ACP diff blocks when possible."""
|
||||
if not patch_text:
|
||||
return [acp.tool_content(acp.text_block(""))]
|
||||
|
||||
try:
|
||||
from tools.patch_parser import OperationType, parse_v4a_patch
|
||||
|
||||
operations, error = parse_v4a_patch(patch_text)
|
||||
if error or not operations:
|
||||
return [acp.tool_content(acp.text_block(patch_text))]
|
||||
|
||||
content: List[Any] = []
|
||||
for op in operations:
|
||||
if op.operation == OperationType.UPDATE:
|
||||
old_chunks: list[str] = []
|
||||
new_chunks: list[str] = []
|
||||
for hunk in op.hunks:
|
||||
old_lines = [line.content for line in hunk.lines if line.prefix in (" ", "-")]
|
||||
new_lines = [line.content for line in hunk.lines if line.prefix in (" ", "+")]
|
||||
if old_lines or new_lines:
|
||||
old_chunks.append("\n".join(old_lines))
|
||||
new_chunks.append("\n".join(new_lines))
|
||||
|
||||
old_text = "\n...\n".join(chunk for chunk in old_chunks if chunk)
|
||||
new_text = "\n...\n".join(chunk for chunk in new_chunks if chunk)
|
||||
if old_text or new_text:
|
||||
content.append(
|
||||
acp.tool_diff_content(
|
||||
path=op.file_path,
|
||||
old_text=old_text or None,
|
||||
new_text=new_text or "",
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
if op.operation == OperationType.ADD:
|
||||
added_lines = [line.content for hunk in op.hunks for line in hunk.lines if line.prefix == "+"]
|
||||
content.append(
|
||||
acp.tool_diff_content(
|
||||
path=op.file_path,
|
||||
new_text="\n".join(added_lines),
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
if op.operation == OperationType.DELETE:
|
||||
content.append(
|
||||
acp.tool_diff_content(
|
||||
path=op.file_path,
|
||||
old_text=f"Delete file: {op.file_path}",
|
||||
new_text="",
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
if op.operation == OperationType.MOVE:
|
||||
content.append(
|
||||
acp.tool_content(acp.text_block(f"Move file: {op.file_path} -> {op.new_path}"))
|
||||
)
|
||||
|
||||
return content or [acp.tool_content(acp.text_block(patch_text))]
|
||||
except Exception:
|
||||
return [acp.tool_content(acp.text_block(patch_text))]
|
||||
|
||||
|
||||
def _strip_diff_prefix(path: str) -> str:
|
||||
raw = str(path or "").strip()
|
||||
if raw.startswith(("a/", "b/")):
|
||||
return raw[2:]
|
||||
return raw
|
||||
|
||||
|
||||
def _parse_unified_diff_content(diff_text: str) -> List[Any]:
|
||||
"""Convert unified diff text into ACP diff content blocks."""
|
||||
if not diff_text:
|
||||
return []
|
||||
|
||||
content: List[Any] = []
|
||||
current_old_path: Optional[str] = None
|
||||
current_new_path: Optional[str] = None
|
||||
old_lines: list[str] = []
|
||||
new_lines: list[str] = []
|
||||
|
||||
def _flush() -> None:
|
||||
nonlocal current_old_path, current_new_path, old_lines, new_lines
|
||||
if current_old_path is None and current_new_path is None:
|
||||
return
|
||||
path = current_new_path if current_new_path and current_new_path != "/dev/null" else current_old_path
|
||||
if not path or path == "/dev/null":
|
||||
current_old_path = None
|
||||
current_new_path = None
|
||||
old_lines = []
|
||||
new_lines = []
|
||||
return
|
||||
content.append(
|
||||
acp.tool_diff_content(
|
||||
path=_strip_diff_prefix(path),
|
||||
old_text="\n".join(old_lines) if old_lines else None,
|
||||
new_text="\n".join(new_lines),
|
||||
)
|
||||
)
|
||||
current_old_path = None
|
||||
current_new_path = None
|
||||
old_lines = []
|
||||
new_lines = []
|
||||
|
||||
for line in diff_text.splitlines():
|
||||
if line.startswith("--- "):
|
||||
_flush()
|
||||
current_old_path = line[4:].strip()
|
||||
continue
|
||||
if line.startswith("+++ "):
|
||||
current_new_path = line[4:].strip()
|
||||
continue
|
||||
if line.startswith("@@"):
|
||||
continue
|
||||
if current_old_path is None and current_new_path is None:
|
||||
continue
|
||||
if line.startswith("+"):
|
||||
new_lines.append(line[1:])
|
||||
elif line.startswith("-"):
|
||||
old_lines.append(line[1:])
|
||||
elif line.startswith(" "):
|
||||
shared = line[1:]
|
||||
old_lines.append(shared)
|
||||
new_lines.append(shared)
|
||||
|
||||
_flush()
|
||||
return content
|
||||
|
||||
|
||||
def _build_tool_complete_content(
|
||||
tool_name: str,
|
||||
result: Optional[str],
|
||||
*,
|
||||
function_args: Optional[Dict[str, Any]] = None,
|
||||
snapshot: Any = None,
|
||||
) -> List[Any]:
|
||||
"""Build structured ACP completion content, falling back to plain text."""
|
||||
display_result = result or ""
|
||||
if len(display_result) > 5000:
|
||||
display_result = display_result[:4900] + f"\n... ({len(result)} chars total, truncated)"
|
||||
|
||||
if tool_name in {"write_file", "patch", "skill_manage"}:
|
||||
try:
|
||||
from agent.display import extract_edit_diff
|
||||
|
||||
diff_text = extract_edit_diff(
|
||||
tool_name,
|
||||
result,
|
||||
function_args=function_args,
|
||||
snapshot=snapshot,
|
||||
)
|
||||
if isinstance(diff_text, str) and diff_text.strip():
|
||||
diff_content = _parse_unified_diff_content(diff_text)
|
||||
if diff_content:
|
||||
return diff_content
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return [acp.tool_content(acp.text_block(display_result))]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Build ACP content objects for tool-call events
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -284,8 +119,9 @@ def build_tool_start(
|
||||
new = arguments.get("new_string", "")
|
||||
content = [acp.tool_diff_content(path=path, new_text=new, old_text=old)]
|
||||
else:
|
||||
# Patch mode — show the patch content as text
|
||||
patch_text = arguments.get("patch", "")
|
||||
content = _build_patch_mode_content(patch_text)
|
||||
content = [acp.tool_content(acp.text_block(patch_text))]
|
||||
return acp.start_tool_call(
|
||||
tool_call_id, title, kind=kind, content=content, locations=locations,
|
||||
raw_input=arguments,
|
||||
@@ -342,17 +178,16 @@ def build_tool_complete(
|
||||
tool_call_id: str,
|
||||
tool_name: str,
|
||||
result: Optional[str] = None,
|
||||
function_args: Optional[Dict[str, Any]] = None,
|
||||
snapshot: Any = None,
|
||||
) -> ToolCallProgress:
|
||||
"""Create a ToolCallUpdate (progress) event for a completed tool call."""
|
||||
kind = get_tool_kind(tool_name)
|
||||
content = _build_tool_complete_content(
|
||||
tool_name,
|
||||
result,
|
||||
function_args=function_args,
|
||||
snapshot=snapshot,
|
||||
)
|
||||
|
||||
# Truncate very large results for the UI
|
||||
display_result = result or ""
|
||||
if len(display_result) > 5000:
|
||||
display_result = display_result[:4900] + f"\n... ({len(result)} chars total, truncated)"
|
||||
|
||||
content = [acp.tool_content(acp.text_block(display_result))]
|
||||
return acp.update_tool_call(
|
||||
tool_call_id,
|
||||
kind=kind,
|
||||
|
||||
+18
-128
@@ -28,45 +28,19 @@ except ImportError:
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
THINKING_BUDGET = {"xhigh": 32000, "high": 16000, "medium": 8000, "low": 4000}
|
||||
# Hermes effort → Anthropic adaptive-thinking effort (output_config.effort).
|
||||
# Anthropic exposes 5 levels on 4.7+: low, medium, high, xhigh, max.
|
||||
# Opus/Sonnet 4.6 only expose 4 levels: low, medium, high, max — no xhigh.
|
||||
# We preserve xhigh as xhigh on 4.7+ (the recommended default for coding/
|
||||
# agentic work) and downgrade it to max on pre-4.7 adaptive models (which
|
||||
# is the strongest level they accept). "minimal" is a legacy alias that
|
||||
# maps to low on every model. See:
|
||||
# https://platform.claude.com/docs/en/about-claude/models/migration-guide
|
||||
ADAPTIVE_EFFORT_MAP = {
|
||||
"max": "max",
|
||||
"xhigh": "xhigh",
|
||||
"high": "high",
|
||||
"medium": "medium",
|
||||
"low": "low",
|
||||
"xhigh": "max",
|
||||
"high": "high",
|
||||
"medium": "medium",
|
||||
"low": "low",
|
||||
"minimal": "low",
|
||||
}
|
||||
|
||||
# Models that accept the "xhigh" output_config.effort level. Opus 4.7 added
|
||||
# xhigh as a distinct level between high and max; older adaptive-thinking
|
||||
# models (4.6) reject it with a 400. Keep this substring list in sync with
|
||||
# the Anthropic migration guide as new model families ship.
|
||||
_XHIGH_EFFORT_SUBSTRINGS = ("4-7", "4.7")
|
||||
|
||||
# Models where extended thinking is deprecated/removed (4.6+ behavior: adaptive
|
||||
# is the only supported mode; 4.7 additionally forbids manual thinking entirely
|
||||
# and drops temperature/top_p/top_k).
|
||||
_ADAPTIVE_THINKING_SUBSTRINGS = ("4-6", "4.6", "4-7", "4.7")
|
||||
|
||||
# Models where temperature/top_p/top_k return 400 if set to non-default values.
|
||||
# This is the Opus 4.7 contract; future 4.x+ models are expected to follow it.
|
||||
_NO_SAMPLING_PARAMS_SUBSTRINGS = ("4-7", "4.7")
|
||||
|
||||
# ── Max output token limits per Anthropic model ───────────────────────
|
||||
# Source: Anthropic docs + Cline model catalog. Anthropic's API requires
|
||||
# max_tokens as a mandatory field. Previously we hardcoded 16384, which
|
||||
# starves thinking-enabled models (thinking tokens count toward the limit).
|
||||
_ANTHROPIC_OUTPUT_LIMITS = {
|
||||
# Claude 4.7
|
||||
"claude-opus-4-7": 128_000,
|
||||
# Claude 4.6
|
||||
"claude-opus-4-6": 128_000,
|
||||
"claude-sonnet-4-6": 64_000,
|
||||
@@ -117,37 +91,11 @@ def _get_anthropic_max_output(model: str) -> int:
|
||||
|
||||
|
||||
def _supports_adaptive_thinking(model: str) -> bool:
|
||||
"""Return True for Claude 4.6+ models that support adaptive thinking."""
|
||||
return any(v in model for v in _ADAPTIVE_THINKING_SUBSTRINGS)
|
||||
"""Return True for Claude 4.6 models that support adaptive thinking."""
|
||||
return any(v in model for v in ("4-6", "4.6"))
|
||||
|
||||
|
||||
def _supports_xhigh_effort(model: str) -> bool:
|
||||
"""Return True for models that accept the 'xhigh' adaptive effort level.
|
||||
|
||||
Opus 4.7 introduced xhigh as a distinct level between high and max.
|
||||
Pre-4.7 adaptive models (Opus/Sonnet 4.6) only accept low/medium/high/max
|
||||
and reject xhigh with an HTTP 400. Callers should downgrade xhigh→max
|
||||
when this returns False.
|
||||
"""
|
||||
return any(v in model for v in _XHIGH_EFFORT_SUBSTRINGS)
|
||||
|
||||
|
||||
def _forbids_sampling_params(model: str) -> bool:
|
||||
"""Return True for models that 400 on any non-default temperature/top_p/top_k.
|
||||
|
||||
Opus 4.7 explicitly rejects sampling parameters; later Claude releases are
|
||||
expected to follow suit. Callers should omit these fields entirely rather
|
||||
than passing zero/default values (the API rejects anything non-null).
|
||||
"""
|
||||
return any(v in model for v in _NO_SAMPLING_PARAMS_SUBSTRINGS)
|
||||
|
||||
|
||||
# Beta headers for enhanced features (sent with ALL auth types).
|
||||
# As of Opus 4.7 (2026-04-16), both of these are GA on Claude 4.6+ — the
|
||||
# beta headers are still accepted (harmless no-op) but not required. Kept
|
||||
# here so older Claude (4.5, 4.1) + third-party Anthropic-compat endpoints
|
||||
# that still gate on the headers continue to get the enhanced features.
|
||||
# Migration guide: remove these if you no longer support ≤4.5 models.
|
||||
# Beta headers for enhanced features (sent with ALL auth types)
|
||||
_COMMON_BETAS = [
|
||||
"interleaved-thinking-2025-05-14",
|
||||
"fine-grained-tool-streaming-2025-05-14",
|
||||
@@ -350,33 +298,6 @@ def build_anthropic_client(api_key: str, base_url: str = None):
|
||||
return _anthropic_sdk.Anthropic(**kwargs)
|
||||
|
||||
|
||||
def build_anthropic_bedrock_client(region: str):
|
||||
"""Create an AnthropicBedrock client for Bedrock Claude models.
|
||||
|
||||
Uses the Anthropic SDK's native Bedrock adapter, which provides full
|
||||
Claude feature parity: prompt caching, thinking budgets, adaptive
|
||||
thinking, fast mode — features not available via the Converse API.
|
||||
|
||||
Auth uses the boto3 default credential chain (IAM roles, SSO, env vars).
|
||||
"""
|
||||
if _anthropic_sdk is None:
|
||||
raise ImportError(
|
||||
"The 'anthropic' package is required for the Bedrock provider. "
|
||||
"Install it with: pip install 'anthropic>=0.39.0'"
|
||||
)
|
||||
if not hasattr(_anthropic_sdk, "AnthropicBedrock"):
|
||||
raise ImportError(
|
||||
"anthropic.AnthropicBedrock not available. "
|
||||
"Upgrade with: pip install 'anthropic>=0.39.0'"
|
||||
)
|
||||
from httpx import Timeout
|
||||
|
||||
return _anthropic_sdk.AnthropicBedrock(
|
||||
aws_region=region,
|
||||
timeout=Timeout(timeout=900.0, connect=10.0),
|
||||
)
|
||||
|
||||
|
||||
def read_claude_code_credentials() -> Optional[Dict[str, Any]]:
|
||||
"""Read refreshable Claude Code OAuth credentials from ~/.claude/.credentials.json.
|
||||
|
||||
@@ -1309,10 +1230,9 @@ def build_anthropic_kwargs(
|
||||
When *base_url* points to a third-party Anthropic-compatible endpoint,
|
||||
thinking block signatures are stripped (they are Anthropic-proprietary).
|
||||
|
||||
When *fast_mode* is True, adds ``extra_body["speed"] = "fast"`` and the
|
||||
fast-mode beta header for ~2.5x faster output throughput on Opus 4.6.
|
||||
Currently only supported on native Anthropic endpoints (not third-party
|
||||
compatible ones).
|
||||
When *fast_mode* is True, adds ``speed: "fast"`` and the fast-mode beta
|
||||
header for ~2.5x faster output throughput on Opus 4.6. Currently only
|
||||
supported on native Anthropic endpoints (not third-party compatible ones).
|
||||
"""
|
||||
system, anthropic_messages = convert_messages_to_anthropic(messages, base_url=base_url)
|
||||
anthropic_tools = convert_tools_to_anthropic(tools) if tools else []
|
||||
@@ -1393,31 +1313,18 @@ def build_anthropic_kwargs(
|
||||
kwargs["tool_choice"] = {"type": "tool", "name": tool_choice}
|
||||
|
||||
# Map reasoning_config to Anthropic's thinking parameter.
|
||||
# Claude 4.6+ models use adaptive thinking + output_config.effort.
|
||||
# Claude 4.6 models use adaptive thinking + output_config.effort.
|
||||
# Older models use manual thinking with budget_tokens.
|
||||
# MiniMax Anthropic-compat endpoints support thinking (manual mode only,
|
||||
# not adaptive). Haiku does NOT support extended thinking — skip entirely.
|
||||
#
|
||||
# On 4.7+ the `thinking.display` field defaults to "omitted", which
|
||||
# silently hides reasoning text that Hermes surfaces in its CLI. We
|
||||
# request "summarized" so the reasoning blocks stay populated — matching
|
||||
# 4.6 behavior and preserving the activity-feed UX during long tool runs.
|
||||
if reasoning_config and isinstance(reasoning_config, dict):
|
||||
if reasoning_config.get("enabled") is not False and "haiku" not in model.lower():
|
||||
effort = str(reasoning_config.get("effort", "medium")).lower()
|
||||
budget = THINKING_BUDGET.get(effort, 8000)
|
||||
if _supports_adaptive_thinking(model):
|
||||
kwargs["thinking"] = {
|
||||
"type": "adaptive",
|
||||
"display": "summarized",
|
||||
}
|
||||
adaptive_effort = ADAPTIVE_EFFORT_MAP.get(effort, "medium")
|
||||
# Downgrade xhigh→max on models that don't list xhigh as a
|
||||
# supported level (Opus/Sonnet 4.6). Opus 4.7+ keeps xhigh.
|
||||
if adaptive_effort == "xhigh" and not _supports_xhigh_effort(model):
|
||||
adaptive_effort = "max"
|
||||
kwargs["thinking"] = {"type": "adaptive"}
|
||||
kwargs["output_config"] = {
|
||||
"effort": adaptive_effort,
|
||||
"effort": ADAPTIVE_EFFORT_MAP.get(effort, "medium")
|
||||
}
|
||||
else:
|
||||
kwargs["thinking"] = {"type": "enabled", "budget_tokens": budget}
|
||||
@@ -1425,21 +1332,12 @@ def build_anthropic_kwargs(
|
||||
kwargs["temperature"] = 1
|
||||
kwargs["max_tokens"] = max(effective_max_tokens, budget + 4096)
|
||||
|
||||
# ── Strip sampling params on 4.7+ ─────────────────────────────────
|
||||
# Opus 4.7 rejects any non-default temperature/top_p/top_k with a 400.
|
||||
# Callers (auxiliary_client, flush_memories, etc.) may set these for
|
||||
# older models; drop them here as a safety net so upstream 4.6 → 4.7
|
||||
# migrations don't require coordinated edits everywhere.
|
||||
if _forbids_sampling_params(model):
|
||||
for _sampling_key in ("temperature", "top_p", "top_k"):
|
||||
kwargs.pop(_sampling_key, None)
|
||||
|
||||
# ── Fast mode (Opus 4.6 only) ────────────────────────────────────
|
||||
# Adds extra_body.speed="fast" + the fast-mode beta header for ~2.5x
|
||||
# output speed. Only for native Anthropic endpoints — third-party
|
||||
# providers would reject the unknown beta header and speed parameter.
|
||||
# Adds speed:"fast" + the fast-mode beta header for ~2.5x output speed.
|
||||
# Only for native Anthropic endpoints — third-party providers would
|
||||
# reject the unknown beta header and speed parameter.
|
||||
if fast_mode and not _is_third_party_anthropic_endpoint(base_url):
|
||||
kwargs.setdefault("extra_body", {})["speed"] = "fast"
|
||||
kwargs["speed"] = "fast"
|
||||
# Build extra_headers with ALL applicable betas (the per-request
|
||||
# extra_headers override the client-level anthropic-beta header).
|
||||
betas = list(_common_betas_for_base_url(base_url))
|
||||
@@ -1491,20 +1389,12 @@ def normalize_anthropic_response(
|
||||
)
|
||||
)
|
||||
|
||||
# Map Anthropic stop_reason to OpenAI finish_reason.
|
||||
# Newer stop reasons added in Claude 4.5+ / 4.7:
|
||||
# - refusal: the model declined to answer (cyber safeguards, CSAM, etc.)
|
||||
# - model_context_window_exceeded: hit context limit (not max_tokens)
|
||||
# Both need distinct handling upstream — a refusal should surface to the
|
||||
# user with a clear message, and a context-window overflow should trigger
|
||||
# compression/truncation rather than be treated as normal end-of-turn.
|
||||
# Map Anthropic stop_reason to OpenAI finish_reason
|
||||
stop_reason_map = {
|
||||
"end_turn": "stop",
|
||||
"tool_use": "tool_calls",
|
||||
"max_tokens": "length",
|
||||
"stop_sequence": "stop",
|
||||
"refusal": "content_filter",
|
||||
"model_context_window_exceeded": "length",
|
||||
}
|
||||
finish_reason = stop_reason_map.get(response.stop_reason, "stop")
|
||||
|
||||
|
||||
+135
-412
@@ -27,6 +27,10 @@ Per-task overrides are configured in config.yaml under the ``auxiliary:`` sectio
|
||||
(e.g. ``auxiliary.vision.provider``, ``auxiliary.compression.model``).
|
||||
Default "auto" follows the chains above.
|
||||
|
||||
Legacy env var overrides (AUXILIARY_{TASK}_PROVIDER, AUXILIARY_{TASK}_MODEL,
|
||||
AUXILIARY_{TASK}_BASE_URL, etc.) are still read as a backward-compat fallback
|
||||
but config.yaml takes priority. New configuration should always use config.yaml.
|
||||
|
||||
Payment / credit exhaustion fallback:
|
||||
When a resolved provider returns HTTP 402 or a credit-related error,
|
||||
call_llm() automatically retries with the next available provider in the
|
||||
@@ -58,17 +62,12 @@ _PROVIDER_ALIASES = {
|
||||
"google": "gemini",
|
||||
"google-gemini": "gemini",
|
||||
"google-ai-studio": "gemini",
|
||||
"x-ai": "xai",
|
||||
"x.ai": "xai",
|
||||
"grok": "xai",
|
||||
"glm": "zai",
|
||||
"z-ai": "zai",
|
||||
"z.ai": "zai",
|
||||
"zhipu": "zai",
|
||||
"kimi": "kimi-coding",
|
||||
"moonshot": "kimi-coding",
|
||||
"kimi-cn": "kimi-coding-cn",
|
||||
"moonshot-cn": "kimi-coding-cn",
|
||||
"minimax-china": "minimax-cn",
|
||||
"minimax_cn": "minimax-cn",
|
||||
"claude": "anthropic",
|
||||
@@ -76,13 +75,13 @@ _PROVIDER_ALIASES = {
|
||||
}
|
||||
|
||||
|
||||
def _normalize_aux_provider(provider: Optional[str]) -> str:
|
||||
def _normalize_aux_provider(provider: Optional[str], *, for_vision: bool = False) -> str:
|
||||
normalized = (provider or "auto").strip().lower()
|
||||
if normalized.startswith("custom:"):
|
||||
suffix = normalized.split(":", 1)[1].strip()
|
||||
if not suffix:
|
||||
return "custom"
|
||||
normalized = suffix
|
||||
normalized = suffix if not for_vision else "custom"
|
||||
if normalized == "codex":
|
||||
return "openai-codex"
|
||||
if normalized == "main":
|
||||
@@ -94,23 +93,11 @@ def _normalize_aux_provider(provider: Optional[str]) -> str:
|
||||
return "custom"
|
||||
return _PROVIDER_ALIASES.get(normalized, normalized)
|
||||
|
||||
|
||||
_FIXED_TEMPERATURE_MODELS: Dict[str, float] = {
|
||||
"kimi-for-coding": 0.6,
|
||||
}
|
||||
|
||||
|
||||
def _fixed_temperature_for_model(model: Optional[str]) -> Optional[float]:
|
||||
"""Return a required temperature override for models with strict contracts."""
|
||||
normalized = (model or "").strip().lower()
|
||||
return _FIXED_TEMPERATURE_MODELS.get(normalized)
|
||||
|
||||
# Default auxiliary models for direct API-key providers (cheap/fast for side tasks)
|
||||
_API_KEY_PROVIDER_AUX_MODELS: Dict[str, str] = {
|
||||
"gemini": "gemini-3-flash-preview",
|
||||
"zai": "glm-4.5-flash",
|
||||
"kimi-coding": "kimi-k2-turbo-preview",
|
||||
"kimi-coding-cn": "kimi-k2-turbo-preview",
|
||||
"minimax": "MiniMax-M2.7",
|
||||
"minimax-cn": "MiniMax-M2.7",
|
||||
"anthropic": "claude-haiku-4-5-20251001",
|
||||
@@ -118,7 +105,6 @@ _API_KEY_PROVIDER_AUX_MODELS: Dict[str, str] = {
|
||||
"opencode-zen": "gemini-3-flash",
|
||||
"opencode-go": "glm-5",
|
||||
"kilocode": "google/gemini-3-flash-preview",
|
||||
"ollama-cloud": "nemotron-3-nano:30b",
|
||||
}
|
||||
|
||||
# Vision-specific model overrides for direct providers.
|
||||
@@ -127,7 +113,6 @@ _API_KEY_PROVIDER_AUX_MODELS: Dict[str, str] = {
|
||||
# "exotic provider" branch checks this before falling back to the main model.
|
||||
_PROVIDER_VISION_MODELS: Dict[str, str] = {
|
||||
"xiaomi": "mimo-v2-omni",
|
||||
"zai": "glm-5v-turbo",
|
||||
}
|
||||
|
||||
# OpenRouter app attribution headers
|
||||
@@ -529,13 +514,8 @@ class _AnthropicCompletionsAdapter:
|
||||
tool_choice=normalized_tool_choice,
|
||||
is_oauth=self._is_oauth,
|
||||
)
|
||||
# Opus 4.7+ rejects any non-default temperature/top_p/top_k; only set
|
||||
# temperature for models that still accept it. build_anthropic_kwargs
|
||||
# additionally strips these keys as a safety net — keep both layers.
|
||||
if temperature is not None:
|
||||
from agent.anthropic_adapter import _forbids_sampling_params
|
||||
if not _forbids_sampling_params(model):
|
||||
anthropic_kwargs["temperature"] = temperature
|
||||
anthropic_kwargs["temperature"] = temperature
|
||||
|
||||
response = self._client.messages.create(**anthropic_kwargs)
|
||||
assistant_message, finish_reason = normalize_anthropic_response(response)
|
||||
@@ -745,15 +725,6 @@ def _resolve_api_key_provider() -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
from hermes_cli.models import copilot_default_headers
|
||||
|
||||
extra["default_headers"] = copilot_default_headers()
|
||||
elif "generativelanguage.googleapis.com" in base_url.lower():
|
||||
# Google's OpenAI-compatible endpoint only accepts x-goog-api-key.
|
||||
# Passing api_key= causes the SDK to inject Authorization: Bearer,
|
||||
# which Google rejects with HTTP 400 "Multiple authentication
|
||||
# credentials received". Use a placeholder for api_key and pass
|
||||
# the real key via x-goog-api-key header instead.
|
||||
# Fixes: https://github.com/NousResearch/hermes-agent/issues/7893
|
||||
extra["default_headers"] = {"x-goog-api-key": api_key}
|
||||
api_key = "not-used"
|
||||
return OpenAI(api_key=api_key, base_url=base_url, **extra), model
|
||||
|
||||
creds = resolve_api_key_provider_credentials(provider_id)
|
||||
@@ -775,15 +746,6 @@ def _resolve_api_key_provider() -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
from hermes_cli.models import copilot_default_headers
|
||||
|
||||
extra["default_headers"] = copilot_default_headers()
|
||||
elif "generativelanguage.googleapis.com" in base_url.lower():
|
||||
# Google's OpenAI-compatible endpoint only accepts x-goog-api-key.
|
||||
# Passing api_key= causes the SDK to inject Authorization: Bearer,
|
||||
# which Google rejects with HTTP 400 "Multiple authentication
|
||||
# credentials received". Use a placeholder for api_key and pass
|
||||
# the real key via x-goog-api-key header instead.
|
||||
# Fixes: https://github.com/NousResearch/hermes-agent/issues/7893
|
||||
extra["default_headers"] = {"x-goog-api-key": api_key}
|
||||
api_key = "not-used"
|
||||
return OpenAI(api_key=api_key, base_url=base_url, **extra), model
|
||||
|
||||
return None, None
|
||||
@@ -791,6 +753,30 @@ def _resolve_api_key_provider() -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
|
||||
# ── Provider resolution helpers ─────────────────────────────────────────────
|
||||
|
||||
def _get_auxiliary_provider(task: str = "") -> str:
|
||||
"""Read the provider override for a specific auxiliary task.
|
||||
|
||||
Checks AUXILIARY_{TASK}_PROVIDER first (e.g. AUXILIARY_VISION_PROVIDER),
|
||||
then CONTEXT_{TASK}_PROVIDER (for the compression section's summary_provider),
|
||||
then falls back to "auto". Returns one of: "auto", "openrouter", "nous", "main".
|
||||
"""
|
||||
if task:
|
||||
for prefix in ("AUXILIARY_", "CONTEXT_"):
|
||||
val = os.getenv(f"{prefix}{task.upper()}_PROVIDER", "").strip().lower()
|
||||
if val and val != "auto":
|
||||
return val
|
||||
return "auto"
|
||||
|
||||
|
||||
def _get_auxiliary_env_override(task: str, suffix: str) -> Optional[str]:
|
||||
"""Read an auxiliary env override from AUXILIARY_* or CONTEXT_* prefixes."""
|
||||
if not task:
|
||||
return None
|
||||
for prefix in ("AUXILIARY_", "CONTEXT_"):
|
||||
val = os.getenv(f"{prefix}{task.upper()}_{suffix}", "").strip()
|
||||
if val:
|
||||
return val
|
||||
return None
|
||||
|
||||
|
||||
def _try_openrouter() -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
@@ -813,21 +799,6 @@ def _try_openrouter() -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
|
||||
|
||||
def _try_nous(vision: bool = False) -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
# Check cross-session rate limit guard before attempting Nous —
|
||||
# if another session already recorded a 429, skip Nous entirely
|
||||
# to avoid piling more requests onto the tapped RPH bucket.
|
||||
try:
|
||||
from agent.nous_rate_guard import nous_rate_limit_remaining
|
||||
_remaining = nous_rate_limit_remaining()
|
||||
if _remaining is not None and _remaining > 0:
|
||||
logger.debug(
|
||||
"Auxiliary: skipping Nous Portal (rate-limited, resets in %.0fs)",
|
||||
_remaining,
|
||||
)
|
||||
return None, None
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
nous = _read_nous_auth()
|
||||
if not nous:
|
||||
return None, None
|
||||
@@ -952,51 +923,6 @@ def _current_custom_base_url() -> str:
|
||||
return custom_base or ""
|
||||
|
||||
|
||||
def _validate_proxy_env_urls() -> None:
|
||||
"""Fail fast with a clear error when proxy env vars have malformed URLs.
|
||||
|
||||
Common cause: shell config (e.g. .zshrc) with a typo like
|
||||
``export HTTP_PROXY=http://127.0.0.1:6153export NEXT_VAR=...``
|
||||
which concatenates 'export' into the port number. Without this
|
||||
check the OpenAI/httpx client raises a cryptic ``Invalid port``
|
||||
error that doesn't name the offending env var.
|
||||
"""
|
||||
from urllib.parse import urlparse
|
||||
|
||||
for key in ("HTTPS_PROXY", "HTTP_PROXY", "ALL_PROXY",
|
||||
"https_proxy", "http_proxy", "all_proxy"):
|
||||
value = str(os.environ.get(key) or "").strip()
|
||||
if not value:
|
||||
continue
|
||||
try:
|
||||
parsed = urlparse(value)
|
||||
if parsed.scheme:
|
||||
_ = parsed.port # raises ValueError for e.g. '6153export'
|
||||
except ValueError as exc:
|
||||
raise RuntimeError(
|
||||
f"Malformed proxy environment variable {key}={value!r}. "
|
||||
"Fix or unset your proxy settings and try again."
|
||||
) from exc
|
||||
|
||||
|
||||
def _validate_base_url(base_url: str) -> None:
|
||||
"""Reject obviously broken custom endpoint URLs before they reach httpx."""
|
||||
from urllib.parse import urlparse
|
||||
|
||||
candidate = str(base_url or "").strip()
|
||||
if not candidate or candidate.startswith("acp://"):
|
||||
return
|
||||
try:
|
||||
parsed = urlparse(candidate)
|
||||
if parsed.scheme in {"http", "https"}:
|
||||
_ = parsed.port # raises ValueError for malformed ports
|
||||
except ValueError as exc:
|
||||
raise RuntimeError(
|
||||
f"Malformed custom endpoint URL: {candidate!r}. "
|
||||
"Run `hermes setup` or `hermes model` and enter a valid http(s) base URL."
|
||||
) from exc
|
||||
|
||||
|
||||
def _try_custom_endpoint() -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
runtime = _resolve_custom_runtime()
|
||||
if len(runtime) == 2:
|
||||
@@ -1093,22 +1019,7 @@ _AUTO_PROVIDER_LABELS = {
|
||||
"_resolve_api_key_provider": "api-key",
|
||||
}
|
||||
|
||||
_MAIN_RUNTIME_FIELDS = ("provider", "model", "base_url", "api_key", "api_mode")
|
||||
|
||||
|
||||
def _normalize_main_runtime(main_runtime: Optional[Dict[str, Any]]) -> Dict[str, str]:
|
||||
"""Return a sanitized copy of a live main-runtime override."""
|
||||
if not isinstance(main_runtime, dict):
|
||||
return {}
|
||||
normalized: Dict[str, str] = {}
|
||||
for field in _MAIN_RUNTIME_FIELDS:
|
||||
value = main_runtime.get(field)
|
||||
if isinstance(value, str) and value.strip():
|
||||
normalized[field] = value.strip()
|
||||
provider = normalized.get("provider")
|
||||
if provider:
|
||||
normalized["provider"] = provider.lower()
|
||||
return normalized
|
||||
_AGGREGATOR_PROVIDERS = frozenset({"openrouter", "nous"})
|
||||
|
||||
|
||||
def _get_provider_chain() -> List[tuple]:
|
||||
@@ -1219,28 +1130,18 @@ def _try_payment_fallback(
|
||||
return None, None, ""
|
||||
|
||||
|
||||
def _resolve_auto(main_runtime: Optional[Dict[str, Any]] = None) -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
def _resolve_auto() -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
"""Full auto-detection chain.
|
||||
|
||||
Priority:
|
||||
1. User's main provider + main model, regardless of provider type.
|
||||
This means auxiliary tasks (compression, vision, web extraction,
|
||||
session search, etc.) use the same model the user configured for
|
||||
chat. Users on OpenRouter/Nous get their chosen chat model; users
|
||||
on DeepSeek/ZAI/Alibaba get theirs; etc. Running aux tasks on the
|
||||
user's picked model keeps behavior predictable — no surprise
|
||||
switches to a cheap fallback model for side tasks.
|
||||
2. OpenRouter → Nous → custom → Codex → API-key providers (fallback
|
||||
chain, only used when the main provider has no working client).
|
||||
1. If the user's main provider is NOT an aggregator (OpenRouter / Nous),
|
||||
use their main provider + main model directly. This ensures users on
|
||||
Alibaba, DeepSeek, ZAI, etc. get auxiliary tasks handled by the same
|
||||
provider they already have credentials for — no OpenRouter key needed.
|
||||
2. OpenRouter → Nous → custom → Codex → API-key providers (original chain).
|
||||
"""
|
||||
global auxiliary_is_nous, _stale_base_url_warned
|
||||
auxiliary_is_nous = False # Reset — _try_nous() will set True if it wins
|
||||
runtime = _normalize_main_runtime(main_runtime)
|
||||
runtime_provider = runtime.get("provider", "")
|
||||
runtime_model = runtime.get("model", "")
|
||||
runtime_base_url = runtime.get("base_url", "")
|
||||
runtime_api_key = runtime.get("api_key", "")
|
||||
runtime_api_mode = runtime.get("api_mode", "")
|
||||
|
||||
# ── Warn once if OPENAI_BASE_URL is set but config.yaml uses a named
|
||||
# provider (not 'custom'). This catches the common "env poisoning"
|
||||
@@ -1248,7 +1149,7 @@ def _resolve_auto(main_runtime: Optional[Dict[str, Any]] = None) -> Tuple[Option
|
||||
# old OPENAI_BASE_URL lingers in ~/.hermes/.env. ──
|
||||
if not _stale_base_url_warned:
|
||||
_env_base = os.getenv("OPENAI_BASE_URL", "").strip()
|
||||
_cfg_provider = runtime_provider or _read_main_provider()
|
||||
_cfg_provider = _read_main_provider()
|
||||
if (_env_base and _cfg_provider
|
||||
and _cfg_provider != "custom"
|
||||
and not _cfg_provider.startswith("custom:")):
|
||||
@@ -1261,31 +1162,13 @@ def _resolve_auto(main_runtime: Optional[Dict[str, Any]] = None) -> Tuple[Option
|
||||
)
|
||||
_stale_base_url_warned = True
|
||||
|
||||
# ── Step 1: main provider + main model → use them directly ──
|
||||
#
|
||||
# This is the primary aux backend for every user. "auto" means
|
||||
# "use my main chat model for side tasks as well" — including users
|
||||
# on aggregators (OpenRouter, Nous) who previously got routed to a
|
||||
# cheap provider-side default. Explicit per-task overrides set via
|
||||
# config.yaml (auxiliary.<task>.provider) still win over this.
|
||||
main_provider = runtime_provider or _read_main_provider()
|
||||
main_model = runtime_model or _read_main_model()
|
||||
# ── Step 1: non-aggregator main provider → use main model directly ──
|
||||
main_provider = _read_main_provider()
|
||||
main_model = _read_main_model()
|
||||
if (main_provider and main_model
|
||||
and main_provider not in _AGGREGATOR_PROVIDERS
|
||||
and main_provider not in ("auto", "")):
|
||||
resolved_provider = main_provider
|
||||
explicit_base_url = None
|
||||
explicit_api_key = None
|
||||
if runtime_base_url and (main_provider == "custom" or main_provider.startswith("custom:")):
|
||||
resolved_provider = "custom"
|
||||
explicit_base_url = runtime_base_url
|
||||
explicit_api_key = runtime_api_key or None
|
||||
client, resolved = resolve_provider_client(
|
||||
resolved_provider,
|
||||
main_model,
|
||||
explicit_base_url=explicit_base_url,
|
||||
explicit_api_key=explicit_api_key,
|
||||
api_mode=runtime_api_mode or None,
|
||||
)
|
||||
client, resolved = resolve_provider_client(main_provider, main_model)
|
||||
if client is not None:
|
||||
logger.info("Auxiliary auto-detect: using main provider %s (%s)",
|
||||
main_provider, resolved or main_model)
|
||||
@@ -1329,12 +1212,6 @@ def _to_async_client(sync_client, model: str):
|
||||
return AsyncCodexAuxiliaryClient(sync_client), model
|
||||
if isinstance(sync_client, AnthropicAuxiliaryClient):
|
||||
return AsyncAnthropicAuxiliaryClient(sync_client), model
|
||||
try:
|
||||
from agent.copilot_acp_client import CopilotACPClient
|
||||
if isinstance(sync_client, CopilotACPClient):
|
||||
return sync_client, model
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
async_kwargs = {
|
||||
"api_key": sync_client.api_key,
|
||||
@@ -1372,7 +1249,6 @@ def resolve_provider_client(
|
||||
explicit_base_url: str = None,
|
||||
explicit_api_key: str = None,
|
||||
api_mode: str = None,
|
||||
main_runtime: Optional[Dict[str, Any]] = None,
|
||||
) -> Tuple[Optional[Any], Optional[str]]:
|
||||
"""Central router: given a provider name and optional model, return a
|
||||
configured client with the correct auth, base URL, and API format.
|
||||
@@ -1404,7 +1280,6 @@ def resolve_provider_client(
|
||||
Returns:
|
||||
(client, resolved_model) or (None, None) if auth is unavailable.
|
||||
"""
|
||||
_validate_proxy_env_urls()
|
||||
# Normalise aliases
|
||||
provider = _normalize_aux_provider(provider)
|
||||
|
||||
@@ -1444,7 +1319,7 @@ def resolve_provider_client(
|
||||
|
||||
# ── Auto: try all providers in priority order ────────────────────
|
||||
if provider == "auto":
|
||||
client, resolved = _resolve_auto(main_runtime=main_runtime)
|
||||
client, resolved = _resolve_auto()
|
||||
if client is None:
|
||||
return None, None
|
||||
# When auto-detection lands on a non-OpenRouter provider (e.g. a
|
||||
@@ -1554,14 +1429,10 @@ def resolve_provider_client(
|
||||
custom_entry = _get_named_custom_provider(provider)
|
||||
if custom_entry:
|
||||
custom_base = custom_entry.get("base_url", "").strip()
|
||||
custom_key = custom_entry.get("api_key", "").strip()
|
||||
custom_key_env = custom_entry.get("key_env", "").strip()
|
||||
if not custom_key and custom_key_env:
|
||||
custom_key = os.getenv(custom_key_env, "").strip()
|
||||
custom_key = custom_key or "no-key-required"
|
||||
custom_key = custom_entry.get("api_key", "").strip() or "no-key-required"
|
||||
if custom_base:
|
||||
final_model = _normalize_resolved_model(
|
||||
model or custom_entry.get("model") or _read_main_model() or "gpt-4o-mini",
|
||||
model or _read_main_model() or "gpt-4o-mini",
|
||||
provider,
|
||||
)
|
||||
client = OpenAI(api_key=custom_key, base_url=custom_base)
|
||||
@@ -1580,11 +1451,7 @@ def resolve_provider_client(
|
||||
|
||||
# ── API-key providers from PROVIDER_REGISTRY ─────────────────────
|
||||
try:
|
||||
from hermes_cli.auth import (
|
||||
PROVIDER_REGISTRY,
|
||||
resolve_api_key_provider_credentials,
|
||||
resolve_external_process_provider_credentials,
|
||||
)
|
||||
from hermes_cli.auth import PROVIDER_REGISTRY, resolve_api_key_provider_credentials
|
||||
except ImportError:
|
||||
logger.debug("hermes_cli.auth not available for provider %s", provider)
|
||||
return None, None
|
||||
@@ -1629,15 +1496,6 @@ def resolve_provider_client(
|
||||
from hermes_cli.models import copilot_default_headers
|
||||
|
||||
headers.update(copilot_default_headers())
|
||||
elif "generativelanguage.googleapis.com" in base_url.lower():
|
||||
# Google's OpenAI-compatible endpoint only accepts x-goog-api-key.
|
||||
# Passing api_key= causes the OpenAI SDK to inject Authorization: Bearer,
|
||||
# which Google rejects with HTTP 400 "Multiple authentication credentials
|
||||
# received". Use a placeholder for api_key and pass the real key via
|
||||
# x-goog-api-key header instead.
|
||||
# Fixes: https://github.com/NousResearch/hermes-agent/issues/7893
|
||||
headers["x-goog-api-key"] = api_key
|
||||
api_key = "not-used"
|
||||
|
||||
client = OpenAI(api_key=api_key, base_url=base_url,
|
||||
**({"default_headers": headers} if headers else {}))
|
||||
@@ -1667,41 +1525,6 @@ def resolve_provider_client(
|
||||
return (_to_async_client(client, final_model) if async_mode
|
||||
else (client, final_model))
|
||||
|
||||
if pconfig.auth_type == "external_process":
|
||||
creds = resolve_external_process_provider_credentials(provider)
|
||||
final_model = _normalize_resolved_model(model or _read_main_model(), provider)
|
||||
if provider == "copilot-acp":
|
||||
api_key = str(creds.get("api_key", "")).strip()
|
||||
base_url = str(creds.get("base_url", "")).strip()
|
||||
command = str(creds.get("command", "")).strip() or None
|
||||
args = list(creds.get("args") or [])
|
||||
if not final_model:
|
||||
logger.warning(
|
||||
"resolve_provider_client: copilot-acp requested but no model "
|
||||
"was provided or configured"
|
||||
)
|
||||
return None, None
|
||||
if not api_key or not base_url:
|
||||
logger.warning(
|
||||
"resolve_provider_client: copilot-acp requested but external "
|
||||
"process credentials are incomplete"
|
||||
)
|
||||
return None, None
|
||||
from agent.copilot_acp_client import CopilotACPClient
|
||||
|
||||
client = CopilotACPClient(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
command=command,
|
||||
args=args,
|
||||
)
|
||||
logger.debug("resolve_provider_client: %s (%s)", provider, final_model)
|
||||
return (_to_async_client(client, final_model) if async_mode
|
||||
else (client, final_model))
|
||||
logger.warning("resolve_provider_client: external-process provider %s not "
|
||||
"directly supported", provider)
|
||||
return None, None
|
||||
|
||||
elif pconfig.auth_type in ("oauth_device_code", "oauth_external"):
|
||||
# OAuth providers — route through their specific try functions
|
||||
if provider == "nous":
|
||||
@@ -1720,19 +1543,15 @@ def resolve_provider_client(
|
||||
|
||||
# ── Public API ──────────────────────────────────────────────────────────────
|
||||
|
||||
def get_text_auxiliary_client(
|
||||
task: str = "",
|
||||
*,
|
||||
main_runtime: Optional[Dict[str, Any]] = None,
|
||||
) -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
def get_text_auxiliary_client(task: str = "") -> Tuple[Optional[OpenAI], Optional[str]]:
|
||||
"""Return (client, default_model_slug) for text-only auxiliary tasks.
|
||||
|
||||
Args:
|
||||
task: Optional task name ("compression", "web_extract") to check
|
||||
for a task-specific provider override.
|
||||
|
||||
Callers may override the returned model via config.yaml
|
||||
(e.g. auxiliary.compression.model, auxiliary.web_extract.model).
|
||||
Callers may override the returned model with a per-task env var
|
||||
(e.g. CONTEXT_COMPRESSION_MODEL, AUXILIARY_WEB_EXTRACT_MODEL).
|
||||
"""
|
||||
provider, model, base_url, api_key, api_mode = _resolve_task_provider_model(task or None)
|
||||
return resolve_provider_client(
|
||||
@@ -1741,11 +1560,10 @@ def get_text_auxiliary_client(
|
||||
explicit_base_url=base_url,
|
||||
explicit_api_key=api_key,
|
||||
api_mode=api_mode,
|
||||
main_runtime=main_runtime,
|
||||
)
|
||||
|
||||
|
||||
def get_async_text_auxiliary_client(task: str = "", *, main_runtime: Optional[Dict[str, Any]] = None):
|
||||
def get_async_text_auxiliary_client(task: str = ""):
|
||||
"""Return (async_client, model_slug) for async consumers.
|
||||
|
||||
For standard providers returns (AsyncOpenAI, model). For Codex returns
|
||||
@@ -1760,7 +1578,6 @@ def get_async_text_auxiliary_client(task: str = "", *, main_runtime: Optional[Di
|
||||
explicit_base_url=base_url,
|
||||
explicit_api_key=api_key,
|
||||
api_mode=api_mode,
|
||||
main_runtime=main_runtime,
|
||||
)
|
||||
|
||||
|
||||
@@ -1771,7 +1588,7 @@ _VISION_AUTO_PROVIDER_ORDER = (
|
||||
|
||||
|
||||
def _normalize_vision_provider(provider: Optional[str]) -> str:
|
||||
return _normalize_aux_provider(provider)
|
||||
return _normalize_aux_provider(provider, for_vision=True)
|
||||
|
||||
|
||||
def _resolve_strict_vision_backend(provider: str) -> Tuple[Optional[Any], Optional[str]]:
|
||||
@@ -1854,7 +1671,6 @@ def resolve_vision_provider_client(
|
||||
async_mode=async_mode,
|
||||
explicit_base_url=resolved_base_url,
|
||||
explicit_api_key=resolved_api_key,
|
||||
api_mode=resolved_api_mode,
|
||||
)
|
||||
if client is None:
|
||||
return "custom", None, None
|
||||
@@ -1862,31 +1678,33 @@ def resolve_vision_provider_client(
|
||||
|
||||
if requested == "auto":
|
||||
# Vision auto-detection order:
|
||||
# 1. User's main provider + main model (including aggregators).
|
||||
# _PROVIDER_VISION_MODELS provides per-provider vision model
|
||||
# overrides when the provider has a dedicated multimodal model
|
||||
# that differs from the chat model (e.g. xiaomi → mimo-v2-omni,
|
||||
# zai → glm-5v-turbo).
|
||||
# 2. OpenRouter (vision-capable aggregator fallback)
|
||||
# 3. Nous Portal (vision-capable aggregator fallback)
|
||||
# 1. Active provider + model (user's main chat config)
|
||||
# 2. OpenRouter (known vision-capable default model)
|
||||
# 3. Nous Portal (known vision-capable default model)
|
||||
# 4. Stop
|
||||
main_provider = _read_main_provider()
|
||||
main_model = _read_main_model()
|
||||
if main_provider and main_provider not in ("auto", ""):
|
||||
vision_model = _PROVIDER_VISION_MODELS.get(main_provider, main_model)
|
||||
rpc_client, rpc_model = resolve_provider_client(
|
||||
main_provider, vision_model,
|
||||
api_mode=resolved_api_mode)
|
||||
if rpc_client is not None:
|
||||
logger.info(
|
||||
"Vision auto-detect: using main provider %s (%s)",
|
||||
main_provider, rpc_model or vision_model,
|
||||
)
|
||||
return _finalize(
|
||||
main_provider, rpc_client, rpc_model or vision_model)
|
||||
if main_provider in _VISION_AUTO_PROVIDER_ORDER:
|
||||
# Known strict backend — use its defaults.
|
||||
sync_client, default_model = _resolve_strict_vision_backend(main_provider)
|
||||
if sync_client is not None:
|
||||
return _finalize(main_provider, sync_client, default_model)
|
||||
else:
|
||||
# Exotic provider (DeepSeek, Alibaba, Xiaomi, named custom, etc.)
|
||||
# Use provider-specific vision model if available, otherwise main model.
|
||||
vision_model = _PROVIDER_VISION_MODELS.get(main_provider, main_model)
|
||||
rpc_client, rpc_model = resolve_provider_client(
|
||||
main_provider, vision_model)
|
||||
if rpc_client is not None:
|
||||
logger.info(
|
||||
"Vision auto-detect: using active provider %s (%s)",
|
||||
main_provider, rpc_model or vision_model,
|
||||
)
|
||||
return _finalize(
|
||||
main_provider, rpc_client, rpc_model or vision_model)
|
||||
|
||||
# Fall back through aggregators (uses their dedicated vision model,
|
||||
# not the user's main model) when main provider has no client.
|
||||
# Fall back through aggregators.
|
||||
for candidate in _VISION_AUTO_PROVIDER_ORDER:
|
||||
if candidate == main_provider:
|
||||
continue # already tried above
|
||||
@@ -1901,8 +1719,7 @@ def resolve_vision_provider_client(
|
||||
sync_client, default_model = _resolve_strict_vision_backend(requested)
|
||||
return _finalize(requested, sync_client, default_model)
|
||||
|
||||
client, final_model = _get_cached_client(requested, resolved_model, async_mode,
|
||||
api_mode=resolved_api_mode)
|
||||
client, final_model = _get_cached_client(requested, resolved_model, async_mode)
|
||||
if client is None:
|
||||
return requested, None, None
|
||||
return requested, client, final_model
|
||||
@@ -1947,15 +1764,9 @@ def auxiliary_max_tokens_param(value: int) -> dict:
|
||||
# Every auxiliary LLM consumer should use these instead of manually
|
||||
# constructing clients and calling .chat.completions.create().
|
||||
|
||||
# Client cache: (provider, async_mode, base_url, api_key, api_mode, runtime_key) -> (client, default_model, loop)
|
||||
# NOTE: loop identity is NOT part of the key. On async cache hits we check
|
||||
# whether the cached loop is the *current* loop; if not, the stale entry is
|
||||
# replaced in-place. This bounds cache growth to one entry per unique
|
||||
# provider config rather than one per (config × event-loop), which previously
|
||||
# caused unbounded fd accumulation in long-running gateway processes (#10200).
|
||||
# Client cache: (provider, async_mode, base_url, api_key) -> (client, default_model)
|
||||
_client_cache: Dict[tuple, tuple] = {}
|
||||
_client_cache_lock = threading.Lock()
|
||||
_CLIENT_CACHE_MAX_SIZE = 64 # safety belt — evict oldest when exceeded
|
||||
|
||||
|
||||
def neuter_async_httpx_del() -> None:
|
||||
@@ -2081,56 +1892,43 @@ def _get_cached_client(
|
||||
base_url: str = None,
|
||||
api_key: str = None,
|
||||
api_mode: str = None,
|
||||
main_runtime: Optional[Dict[str, Any]] = None,
|
||||
) -> Tuple[Optional[Any], Optional[str]]:
|
||||
"""Get or create a cached client for the given provider.
|
||||
|
||||
Async clients (AsyncOpenAI) use httpx.AsyncClient internally, which
|
||||
binds to the event loop that was current when the client was created.
|
||||
Using such a client on a *different* loop causes deadlocks or
|
||||
RuntimeError. To prevent cross-loop issues, the cache validates on
|
||||
every async hit that the cached loop is the *current, open* loop.
|
||||
If the loop changed (e.g. a new gateway worker-thread loop), the stale
|
||||
entry is replaced in-place rather than creating an additional entry.
|
||||
|
||||
This keeps cache size bounded to one entry per unique provider config,
|
||||
preventing the fd-exhaustion that previously occurred in long-running
|
||||
gateways where recycled worker threads created unbounded entries (#10200).
|
||||
RuntimeError. To prevent cross-loop issues (especially in gateway
|
||||
mode where _run_async() may spawn fresh loops in worker threads), the
|
||||
cache key for async clients includes the current event loop's identity
|
||||
so each loop gets its own client instance.
|
||||
"""
|
||||
# Resolve the current event loop for async clients so we can validate
|
||||
# cached entries. Loop identity is NOT in the cache key — instead we
|
||||
# check at hit time whether the cached loop is still current and open.
|
||||
# This prevents unbounded cache growth from recycled worker-thread loops
|
||||
# while still guaranteeing we never reuse a client on the wrong loop
|
||||
# (which causes deadlocks, see #2681).
|
||||
# Include loop identity for async clients to prevent cross-loop reuse.
|
||||
# httpx.AsyncClient (inside AsyncOpenAI) is bound to the loop where it
|
||||
# was created — reusing it on a different loop causes deadlocks (#2681).
|
||||
loop_id = 0
|
||||
current_loop = None
|
||||
if async_mode:
|
||||
try:
|
||||
import asyncio as _aio
|
||||
current_loop = _aio.get_event_loop()
|
||||
loop_id = id(current_loop)
|
||||
except RuntimeError:
|
||||
pass
|
||||
runtime = _normalize_main_runtime(main_runtime)
|
||||
runtime_key = tuple(runtime.get(field, "") for field in _MAIN_RUNTIME_FIELDS) if provider == "auto" else ()
|
||||
cache_key = (provider, async_mode, base_url or "", api_key or "", api_mode or "", runtime_key)
|
||||
cache_key = (provider, async_mode, base_url or "", api_key or "", api_mode or "", loop_id)
|
||||
with _client_cache_lock:
|
||||
if cache_key in _client_cache:
|
||||
cached_client, cached_default, cached_loop = _client_cache[cache_key]
|
||||
if async_mode:
|
||||
# Validate: the cached client must be bound to the CURRENT,
|
||||
# OPEN loop. If the loop changed or was closed, the httpx
|
||||
# transport inside is dead — force-close and replace.
|
||||
loop_ok = (
|
||||
cached_loop is not None
|
||||
and cached_loop is current_loop
|
||||
and not cached_loop.is_closed()
|
||||
)
|
||||
if loop_ok:
|
||||
# A cached async client whose loop has been closed will raise
|
||||
# "Event loop is closed" when httpx tries to clean up its
|
||||
# transport. Discard the stale client and create a fresh one.
|
||||
if cached_loop is not None and cached_loop.is_closed():
|
||||
_force_close_async_httpx(cached_client)
|
||||
del _client_cache[cache_key]
|
||||
else:
|
||||
effective = _compat_model(cached_client, model, cached_default)
|
||||
return cached_client, effective
|
||||
# Stale — evict and fall through to create a new client.
|
||||
_force_close_async_httpx(cached_client)
|
||||
del _client_cache[cache_key]
|
||||
else:
|
||||
effective = _compat_model(cached_client, model, cached_default)
|
||||
return cached_client, effective
|
||||
@@ -2142,7 +1940,6 @@ def _get_cached_client(
|
||||
explicit_base_url=base_url,
|
||||
explicit_api_key=api_key,
|
||||
api_mode=api_mode,
|
||||
main_runtime=runtime,
|
||||
)
|
||||
if client is not None:
|
||||
# For async clients, remember which loop they were created on so we
|
||||
@@ -2150,12 +1947,6 @@ def _get_cached_client(
|
||||
bound_loop = current_loop
|
||||
with _client_cache_lock:
|
||||
if cache_key not in _client_cache:
|
||||
# Safety belt: if the cache has grown beyond the max, evict
|
||||
# the oldest entries (FIFO — dict preserves insertion order).
|
||||
while len(_client_cache) >= _CLIENT_CACHE_MAX_SIZE:
|
||||
evict_key, evict_entry = next(iter(_client_cache.items()))
|
||||
_force_close_async_httpx(evict_entry[0])
|
||||
del _client_cache[evict_key]
|
||||
_client_cache[cache_key] = (client, default_model, bound_loop)
|
||||
else:
|
||||
client, default_model, _ = _client_cache[cache_key]
|
||||
@@ -2173,8 +1964,9 @@ def _resolve_task_provider_model(
|
||||
|
||||
Priority:
|
||||
1. Explicit provider/model/base_url/api_key args (always win)
|
||||
2. Config file (auxiliary.{task}.provider/model/base_url)
|
||||
3. "auto" (full auto-detection chain)
|
||||
2. Config file (auxiliary.{task}.* or compression.*)
|
||||
3. Env var overrides (backward-compat: AUXILIARY_{TASK}_*, CONTEXT_{TASK}_*)
|
||||
4. "auto" (full auto-detection chain)
|
||||
|
||||
Returns (provider, model, base_url, api_key, api_mode) where model may
|
||||
be None (use provider default). When base_url is set, provider is forced
|
||||
@@ -2205,8 +1997,22 @@ def _resolve_task_provider_model(
|
||||
cfg_api_key = str(task_config.get("api_key", "")).strip() or None
|
||||
cfg_api_mode = str(task_config.get("api_mode", "")).strip() or None
|
||||
|
||||
resolved_model = model or cfg_model
|
||||
resolved_api_mode = cfg_api_mode
|
||||
# Backwards compat: compression section has its own keys.
|
||||
# The auxiliary.compression defaults to provider="auto", so treat
|
||||
# both None and "auto" as "not explicitly configured".
|
||||
if task == "compression" and (not cfg_provider or cfg_provider == "auto"):
|
||||
comp = config.get("compression", {}) if isinstance(config, dict) else {}
|
||||
if isinstance(comp, dict):
|
||||
cfg_provider = comp.get("summary_provider", "").strip() or None
|
||||
cfg_model = cfg_model or comp.get("summary_model", "").strip() or None
|
||||
_sbu = comp.get("summary_base_url") or ""
|
||||
cfg_base_url = cfg_base_url or _sbu.strip() or None
|
||||
|
||||
# Env vars are backward-compat fallback only — config.yaml is primary.
|
||||
env_model = _get_auxiliary_env_override(task, "MODEL") if task else None
|
||||
env_api_mode = _get_auxiliary_env_override(task, "API_MODE") if task else None
|
||||
resolved_model = model or cfg_model or env_model
|
||||
resolved_api_mode = cfg_api_mode or env_api_mode
|
||||
|
||||
if base_url:
|
||||
return "custom", resolved_model, base_url, api_key, resolved_api_mode
|
||||
@@ -2220,6 +2026,17 @@ def _resolve_task_provider_model(
|
||||
if cfg_provider and cfg_provider != "auto":
|
||||
return cfg_provider, resolved_model, None, None, resolved_api_mode
|
||||
|
||||
# Env vars are backward-compat fallback for users who haven't
|
||||
# migrated to config.yaml yet.
|
||||
env_base_url = _get_auxiliary_env_override(task, "BASE_URL")
|
||||
env_api_key = _get_auxiliary_env_override(task, "API_KEY")
|
||||
if env_base_url:
|
||||
return "custom", resolved_model, env_base_url, env_api_key, resolved_api_mode
|
||||
|
||||
env_provider = _get_auxiliary_provider(task)
|
||||
if env_provider != "auto":
|
||||
return env_provider, resolved_model, None, None, resolved_api_mode
|
||||
|
||||
return "auto", resolved_model, None, None, resolved_api_mode
|
||||
|
||||
return "auto", resolved_model, None, None, resolved_api_mode
|
||||
@@ -2248,75 +2065,6 @@ def _get_task_timeout(task: str, default: float = _DEFAULT_AUX_TIMEOUT) -> float
|
||||
return default
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Anthropic-compatible endpoint detection + image block conversion
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Providers that use Anthropic-compatible endpoints (via OpenAI SDK wrapper).
|
||||
# Their image content blocks must use Anthropic format, not OpenAI format.
|
||||
_ANTHROPIC_COMPAT_PROVIDERS = frozenset({"minimax", "minimax-cn"})
|
||||
|
||||
|
||||
def _is_anthropic_compat_endpoint(provider: str, base_url: str) -> bool:
|
||||
"""Detect if an endpoint expects Anthropic-format content blocks.
|
||||
|
||||
Returns True for known Anthropic-compatible providers (MiniMax) and
|
||||
any endpoint whose URL contains ``/anthropic`` in the path.
|
||||
"""
|
||||
if provider in _ANTHROPIC_COMPAT_PROVIDERS:
|
||||
return True
|
||||
url_lower = (base_url or "").lower()
|
||||
return "/anthropic" in url_lower
|
||||
|
||||
|
||||
def _convert_openai_images_to_anthropic(messages: list) -> list:
|
||||
"""Convert OpenAI ``image_url`` content blocks to Anthropic ``image`` blocks.
|
||||
|
||||
Only touches messages that have list-type content with ``image_url`` blocks;
|
||||
plain text messages pass through unchanged.
|
||||
"""
|
||||
converted = []
|
||||
for msg in messages:
|
||||
content = msg.get("content")
|
||||
if not isinstance(content, list):
|
||||
converted.append(msg)
|
||||
continue
|
||||
new_content = []
|
||||
changed = False
|
||||
for block in content:
|
||||
if block.get("type") == "image_url":
|
||||
image_url_val = (block.get("image_url") or {}).get("url", "")
|
||||
if image_url_val.startswith("data:"):
|
||||
# Parse data URI: data:<media_type>;base64,<data>
|
||||
header, _, b64data = image_url_val.partition(",")
|
||||
media_type = "image/png"
|
||||
if ":" in header and ";" in header:
|
||||
media_type = header.split(":", 1)[1].split(";", 1)[0]
|
||||
new_content.append({
|
||||
"type": "image",
|
||||
"source": {
|
||||
"type": "base64",
|
||||
"media_type": media_type,
|
||||
"data": b64data,
|
||||
},
|
||||
})
|
||||
else:
|
||||
# URL-based image
|
||||
new_content.append({
|
||||
"type": "image",
|
||||
"source": {
|
||||
"type": "url",
|
||||
"url": image_url_val,
|
||||
},
|
||||
})
|
||||
changed = True
|
||||
else:
|
||||
new_content.append(block)
|
||||
converted.append({**msg, "content": new_content} if changed else msg)
|
||||
return converted
|
||||
|
||||
|
||||
|
||||
def _build_call_kwargs(
|
||||
provider: str,
|
||||
model: str,
|
||||
@@ -2335,19 +2083,6 @@ def _build_call_kwargs(
|
||||
"timeout": timeout,
|
||||
}
|
||||
|
||||
fixed_temperature = _fixed_temperature_for_model(model)
|
||||
if fixed_temperature is not None:
|
||||
temperature = fixed_temperature
|
||||
|
||||
# Opus 4.7+ rejects any non-default temperature/top_p/top_k — silently
|
||||
# drop here so auxiliary callers that hardcode temperature (e.g. 0.3 on
|
||||
# flush_memories, 0 on structured-JSON extraction) don't 400 the moment
|
||||
# the aux model is flipped to 4.7.
|
||||
if temperature is not None:
|
||||
from agent.anthropic_adapter import _forbids_sampling_params
|
||||
if _forbids_sampling_params(model):
|
||||
temperature = None
|
||||
|
||||
if temperature is not None:
|
||||
kwargs["temperature"] = temperature
|
||||
|
||||
@@ -2414,7 +2149,6 @@ def call_llm(
|
||||
model: str = None,
|
||||
base_url: str = None,
|
||||
api_key: str = None,
|
||||
main_runtime: Optional[Dict[str, Any]] = None,
|
||||
messages: list,
|
||||
temperature: float = None,
|
||||
max_tokens: int = None,
|
||||
@@ -2451,10 +2185,10 @@ def call_llm(
|
||||
|
||||
if task == "vision":
|
||||
effective_provider, client, final_model = resolve_vision_provider_client(
|
||||
provider=resolved_provider if resolved_provider != "auto" else provider,
|
||||
model=resolved_model or model,
|
||||
base_url=resolved_base_url or base_url,
|
||||
api_key=resolved_api_key or api_key,
|
||||
provider=provider,
|
||||
model=model,
|
||||
base_url=base_url,
|
||||
api_key=api_key,
|
||||
async_mode=False,
|
||||
)
|
||||
if client is None and resolved_provider != "auto" and not resolved_base_url:
|
||||
@@ -2480,7 +2214,6 @@ def call_llm(
|
||||
base_url=resolved_base_url,
|
||||
api_key=resolved_api_key,
|
||||
api_mode=resolved_api_mode,
|
||||
main_runtime=main_runtime,
|
||||
)
|
||||
if client is None:
|
||||
# When the user explicitly chose a non-OpenRouter provider but no
|
||||
@@ -2501,7 +2234,7 @@ def call_llm(
|
||||
if not resolved_base_url:
|
||||
logger.info("Auxiliary %s: provider %s unavailable, trying auto-detection chain",
|
||||
task or "call", resolved_provider)
|
||||
client, final_model = _get_cached_client("auto", main_runtime=main_runtime)
|
||||
client, final_model = _get_cached_client("auto")
|
||||
if client is None:
|
||||
raise RuntimeError(
|
||||
f"No LLM provider configured for task={task} provider={resolved_provider}. "
|
||||
@@ -2522,11 +2255,6 @@ def call_llm(
|
||||
tools=tools, timeout=effective_timeout, extra_body=extra_body,
|
||||
base_url=resolved_base_url)
|
||||
|
||||
# Convert image blocks for Anthropic-compatible endpoints (e.g. MiniMax)
|
||||
_client_base = str(getattr(client, "base_url", "") or "")
|
||||
if _is_anthropic_compat_endpoint(resolved_provider, _client_base):
|
||||
kwargs["messages"] = _convert_openai_images_to_anthropic(kwargs["messages"])
|
||||
|
||||
# Handle max_tokens vs max_completion_tokens retry, then payment fallback.
|
||||
try:
|
||||
return _validate_llm_response(
|
||||
@@ -2603,9 +2331,9 @@ def extract_content_or_reasoning(response) -> str:
|
||||
if content:
|
||||
# Strip inline think/reasoning blocks (mirrors _strip_think_blocks)
|
||||
cleaned = re.sub(
|
||||
r"<(?:think|thinking|reasoning|thought|REASONING_SCRATCHPAD)>"
|
||||
r"<(?:think|thinking|reasoning|REASONING_SCRATCHPAD)>"
|
||||
r".*?"
|
||||
r"</(?:think|thinking|reasoning|thought|REASONING_SCRATCHPAD)>",
|
||||
r"</(?:think|thinking|reasoning|REASONING_SCRATCHPAD)>",
|
||||
"", content, flags=re.DOTALL | re.IGNORECASE,
|
||||
).strip()
|
||||
if cleaned:
|
||||
@@ -2659,10 +2387,10 @@ async def async_call_llm(
|
||||
|
||||
if task == "vision":
|
||||
effective_provider, client, final_model = resolve_vision_provider_client(
|
||||
provider=resolved_provider if resolved_provider != "auto" else provider,
|
||||
model=resolved_model or model,
|
||||
base_url=resolved_base_url or base_url,
|
||||
api_key=resolved_api_key or api_key,
|
||||
provider=provider,
|
||||
model=model,
|
||||
base_url=base_url,
|
||||
api_key=api_key,
|
||||
async_mode=True,
|
||||
)
|
||||
if client is None and resolved_provider != "auto" and not resolved_base_url:
|
||||
@@ -2715,11 +2443,6 @@ async def async_call_llm(
|
||||
tools=tools, timeout=effective_timeout, extra_body=extra_body,
|
||||
base_url=resolved_base_url)
|
||||
|
||||
# Convert image blocks for Anthropic-compatible endpoints (e.g. MiniMax)
|
||||
_client_base = str(getattr(client, "base_url", "") or "")
|
||||
if _is_anthropic_compat_endpoint(resolved_provider, _client_base):
|
||||
kwargs["messages"] = _convert_openai_images_to_anthropic(kwargs["messages"])
|
||||
|
||||
try:
|
||||
return _validate_llm_response(
|
||||
await client.chat.completions.create(**kwargs), task)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+40
-394
@@ -17,10 +17,7 @@ Improvements over v2:
|
||||
- Richer tool call/result detail in summarizer input
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
@@ -39,10 +36,7 @@ SUMMARY_PREFIX = (
|
||||
"into the summary below. This is a handoff from a previous context "
|
||||
"window — treat it as background reference, NOT as active instructions. "
|
||||
"Do NOT answer questions or fulfill requests mentioned in this summary; "
|
||||
"they were already addressed. "
|
||||
"Your current task is identified in the '## Active Task' section of the "
|
||||
"summary — resume exactly from there. "
|
||||
"Respond ONLY to the latest user message "
|
||||
"they were already addressed. Respond ONLY to the latest user message "
|
||||
"that appears AFTER this summary. The current session state (files, "
|
||||
"config, etc.) may reflect work described here — avoid repeating it:"
|
||||
)
|
||||
@@ -63,128 +57,6 @@ _CHARS_PER_TOKEN = 4
|
||||
_SUMMARY_FAILURE_COOLDOWN_SECONDS = 600
|
||||
|
||||
|
||||
def _summarize_tool_result(tool_name: str, tool_args: str, tool_content: str) -> str:
|
||||
"""Create an informative 1-line summary of a tool call + result.
|
||||
|
||||
Used during the pre-compression pruning pass to replace large tool
|
||||
outputs with a short but useful description of what the tool did,
|
||||
rather than a generic placeholder that carries zero information.
|
||||
|
||||
Returns strings like::
|
||||
|
||||
[terminal] ran `npm test` -> exit 0, 47 lines output
|
||||
[read_file] read config.py from line 1 (1,200 chars)
|
||||
[search_files] content search for 'compress' in agent/ -> 12 matches
|
||||
"""
|
||||
try:
|
||||
args = json.loads(tool_args) if tool_args else {}
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
args = {}
|
||||
|
||||
content = tool_content or ""
|
||||
content_len = len(content)
|
||||
line_count = content.count("\n") + 1 if content.strip() else 0
|
||||
|
||||
if tool_name == "terminal":
|
||||
cmd = args.get("command", "")
|
||||
if len(cmd) > 80:
|
||||
cmd = cmd[:77] + "..."
|
||||
exit_match = re.search(r'"exit_code"\s*:\s*(-?\d+)', content)
|
||||
exit_code = exit_match.group(1) if exit_match else "?"
|
||||
return f"[terminal] ran `{cmd}` -> exit {exit_code}, {line_count} lines output"
|
||||
|
||||
if tool_name == "read_file":
|
||||
path = args.get("path", "?")
|
||||
offset = args.get("offset", 1)
|
||||
return f"[read_file] read {path} from line {offset} ({content_len:,} chars)"
|
||||
|
||||
if tool_name == "write_file":
|
||||
path = args.get("path", "?")
|
||||
written_lines = args.get("content", "").count("\n") + 1 if args.get("content") else "?"
|
||||
return f"[write_file] wrote to {path} ({written_lines} lines)"
|
||||
|
||||
if tool_name == "search_files":
|
||||
pattern = args.get("pattern", "?")
|
||||
path = args.get("path", ".")
|
||||
target = args.get("target", "content")
|
||||
match_count = re.search(r'"total_count"\s*:\s*(\d+)', content)
|
||||
count = match_count.group(1) if match_count else "?"
|
||||
return f"[search_files] {target} search for '{pattern}' in {path} -> {count} matches"
|
||||
|
||||
if tool_name == "patch":
|
||||
path = args.get("path", "?")
|
||||
mode = args.get("mode", "replace")
|
||||
return f"[patch] {mode} in {path} ({content_len:,} chars result)"
|
||||
|
||||
if tool_name in ("browser_navigate", "browser_click", "browser_snapshot",
|
||||
"browser_type", "browser_scroll", "browser_vision"):
|
||||
url = args.get("url", "")
|
||||
ref = args.get("ref", "")
|
||||
detail = f" {url}" if url else (f" ref={ref}" if ref else "")
|
||||
return f"[{tool_name}]{detail} ({content_len:,} chars)"
|
||||
|
||||
if tool_name == "web_search":
|
||||
query = args.get("query", "?")
|
||||
return f"[web_search] query='{query}' ({content_len:,} chars result)"
|
||||
|
||||
if tool_name == "web_extract":
|
||||
urls = args.get("urls", [])
|
||||
url_desc = urls[0] if isinstance(urls, list) and urls else "?"
|
||||
if isinstance(urls, list) and len(urls) > 1:
|
||||
url_desc += f" (+{len(urls) - 1} more)"
|
||||
return f"[web_extract] {url_desc} ({content_len:,} chars)"
|
||||
|
||||
if tool_name == "delegate_task":
|
||||
goal = args.get("goal", "")
|
||||
if len(goal) > 60:
|
||||
goal = goal[:57] + "..."
|
||||
return f"[delegate_task] '{goal}' ({content_len:,} chars result)"
|
||||
|
||||
if tool_name == "execute_code":
|
||||
code_preview = (args.get("code") or "")[:60].replace("\n", " ")
|
||||
if len(args.get("code", "")) > 60:
|
||||
code_preview += "..."
|
||||
return f"[execute_code] `{code_preview}` ({line_count} lines output)"
|
||||
|
||||
if tool_name in ("skill_view", "skills_list", "skill_manage"):
|
||||
name = args.get("name", "?")
|
||||
return f"[{tool_name}] name={name} ({content_len:,} chars)"
|
||||
|
||||
if tool_name == "vision_analyze":
|
||||
question = args.get("question", "")[:50]
|
||||
return f"[vision_analyze] '{question}' ({content_len:,} chars)"
|
||||
|
||||
if tool_name == "memory":
|
||||
action = args.get("action", "?")
|
||||
target = args.get("target", "?")
|
||||
return f"[memory] {action} on {target}"
|
||||
|
||||
if tool_name == "todo":
|
||||
return "[todo] updated task list"
|
||||
|
||||
if tool_name == "clarify":
|
||||
return "[clarify] asked user a question"
|
||||
|
||||
if tool_name == "text_to_speech":
|
||||
return f"[text_to_speech] generated audio ({content_len:,} chars)"
|
||||
|
||||
if tool_name == "cronjob":
|
||||
action = args.get("action", "?")
|
||||
return f"[cronjob] {action}"
|
||||
|
||||
if tool_name == "process":
|
||||
action = args.get("action", "?")
|
||||
sid = args.get("session_id", "?")
|
||||
return f"[process] {action} session={sid}"
|
||||
|
||||
# Generic fallback
|
||||
first_arg = ""
|
||||
for k, v in list(args.items())[:2]:
|
||||
sv = str(v)[:40]
|
||||
first_arg += f" {k}={sv}"
|
||||
return f"[{tool_name}]{first_arg} ({content_len:,} chars result)"
|
||||
|
||||
|
||||
class ContextCompressor(ContextEngine):
|
||||
"""Default context engine — compresses conversation context via lossy summarization.
|
||||
|
||||
@@ -206,8 +78,6 @@ class ContextCompressor(ContextEngine):
|
||||
self._context_probed = False
|
||||
self._context_probe_persistable = False
|
||||
self._previous_summary = None
|
||||
self._last_compression_savings_pct = 100.0
|
||||
self._ineffective_compression_count = 0
|
||||
|
||||
def update_model(
|
||||
self,
|
||||
@@ -216,14 +86,12 @@ class ContextCompressor(ContextEngine):
|
||||
base_url: str = "",
|
||||
api_key: str = "",
|
||||
provider: str = "",
|
||||
api_mode: str = "",
|
||||
) -> None:
|
||||
"""Update model info after a model switch or fallback activation."""
|
||||
self.model = model
|
||||
self.base_url = base_url
|
||||
self.api_key = api_key
|
||||
self.provider = provider
|
||||
self.api_mode = api_mode
|
||||
self.context_length = context_length
|
||||
self.threshold_tokens = max(
|
||||
int(context_length * self.threshold_percent),
|
||||
@@ -243,13 +111,11 @@ class ContextCompressor(ContextEngine):
|
||||
api_key: str = "",
|
||||
config_context_length: int | None = None,
|
||||
provider: str = "",
|
||||
api_mode: str = "",
|
||||
):
|
||||
self.model = model
|
||||
self.base_url = base_url
|
||||
self.api_key = api_key
|
||||
self.provider = provider
|
||||
self.api_mode = api_mode
|
||||
self.threshold_percent = threshold_percent
|
||||
self.protect_first_n = protect_first_n
|
||||
self.protect_last_n = protect_last_n
|
||||
@@ -297,9 +163,6 @@ class ContextCompressor(ContextEngine):
|
||||
|
||||
# Stores the previous compaction summary for iterative updates
|
||||
self._previous_summary: Optional[str] = None
|
||||
# Anti-thrashing: track whether last compression was effective
|
||||
self._last_compression_savings_pct: float = 100.0
|
||||
self._ineffective_compression_count: int = 0
|
||||
self._summary_failure_cooldown_until: float = 0.0
|
||||
|
||||
def update_from_response(self, usage: Dict[str, Any]):
|
||||
@@ -308,26 +171,9 @@ class ContextCompressor(ContextEngine):
|
||||
self.last_completion_tokens = usage.get("completion_tokens", 0)
|
||||
|
||||
def should_compress(self, prompt_tokens: int = None) -> bool:
|
||||
"""Check if context exceeds the compression threshold.
|
||||
|
||||
Includes anti-thrashing protection: if the last two compressions
|
||||
each saved less than 10%, skip compression to avoid infinite loops
|
||||
where each pass removes only 1-2 messages.
|
||||
"""
|
||||
"""Check if context exceeds the compression threshold."""
|
||||
tokens = prompt_tokens if prompt_tokens is not None else self.last_prompt_tokens
|
||||
if tokens < self.threshold_tokens:
|
||||
return False
|
||||
# Anti-thrashing: back off if recent compressions were ineffective
|
||||
if self._ineffective_compression_count >= 2:
|
||||
if not self.quiet_mode:
|
||||
logger.warning(
|
||||
"Compression skipped — last %d compressions saved <10%% each. "
|
||||
"Consider /new to start a fresh session, or /compress <topic> "
|
||||
"for focused compression.",
|
||||
self._ineffective_compression_count,
|
||||
)
|
||||
return False
|
||||
return True
|
||||
return tokens >= self.threshold_tokens
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Tool output pruning (cheap pre-pass, no LLM call)
|
||||
@@ -337,16 +183,7 @@ class ContextCompressor(ContextEngine):
|
||||
self, messages: List[Dict[str, Any]], protect_tail_count: int,
|
||||
protect_tail_tokens: int | None = None,
|
||||
) -> tuple[List[Dict[str, Any]], int]:
|
||||
"""Replace old tool result contents with informative 1-line summaries.
|
||||
|
||||
Instead of a generic placeholder, generates a summary like::
|
||||
|
||||
[terminal] ran `npm test` -> exit 0, 47 lines output
|
||||
[read_file] read config.py from line 1 (3,400 chars)
|
||||
|
||||
Also deduplicates identical tool results (e.g. reading the same file
|
||||
5x keeps only the newest full copy) and truncates large tool_call
|
||||
arguments in assistant messages outside the protected tail.
|
||||
"""Replace old tool result contents with a short placeholder.
|
||||
|
||||
Walks backward from the end, protecting the most recent messages that
|
||||
fall within ``protect_tail_tokens`` (when provided) OR the last
|
||||
@@ -362,22 +199,6 @@ class ContextCompressor(ContextEngine):
|
||||
result = [m.copy() for m in messages]
|
||||
pruned = 0
|
||||
|
||||
# Build index: tool_call_id -> (tool_name, arguments_json)
|
||||
call_id_to_tool: Dict[str, tuple] = {}
|
||||
for msg in result:
|
||||
if msg.get("role") == "assistant":
|
||||
for tc in msg.get("tool_calls") or []:
|
||||
if isinstance(tc, dict):
|
||||
cid = tc.get("id", "")
|
||||
fn = tc.get("function", {})
|
||||
call_id_to_tool[cid] = (fn.get("name", "unknown"), fn.get("arguments", ""))
|
||||
else:
|
||||
cid = getattr(tc, "id", "") or ""
|
||||
fn = getattr(tc, "function", None)
|
||||
name = getattr(fn, "name", "unknown") if fn else "unknown"
|
||||
args_str = getattr(fn, "arguments", "") if fn else ""
|
||||
call_id_to_tool[cid] = (name, args_str)
|
||||
|
||||
# Determine the prune boundary
|
||||
if protect_tail_tokens is not None and protect_tail_tokens > 0:
|
||||
# Token-budget approach: walk backward accumulating tokens
|
||||
@@ -386,8 +207,7 @@ class ContextCompressor(ContextEngine):
|
||||
min_protect = min(protect_tail_count, len(result) - 1)
|
||||
for i in range(len(result) - 1, -1, -1):
|
||||
msg = result[i]
|
||||
raw_content = msg.get("content") or ""
|
||||
content_len = sum(len(p.get("text", "")) for p in raw_content) if isinstance(raw_content, list) else len(raw_content)
|
||||
content_len = len(msg.get("content") or "")
|
||||
msg_tokens = content_len // _CHARS_PER_TOKEN + 10
|
||||
for tc in msg.get("tool_calls") or []:
|
||||
if isinstance(tc, dict):
|
||||
@@ -402,69 +222,18 @@ class ContextCompressor(ContextEngine):
|
||||
else:
|
||||
prune_boundary = len(result) - protect_tail_count
|
||||
|
||||
# Pass 1: Deduplicate identical tool results.
|
||||
# When the same file is read multiple times, keep only the most recent
|
||||
# full copy and replace older duplicates with a back-reference.
|
||||
content_hashes: dict = {} # hash -> (index, tool_call_id)
|
||||
for i in range(len(result) - 1, -1, -1):
|
||||
msg = result[i]
|
||||
if msg.get("role") != "tool":
|
||||
continue
|
||||
content = msg.get("content") or ""
|
||||
# Skip multimodal content (list of content blocks)
|
||||
if isinstance(content, list):
|
||||
continue
|
||||
if len(content) < 200:
|
||||
continue
|
||||
h = hashlib.md5(content.encode("utf-8", errors="replace")).hexdigest()[:12]
|
||||
if h in content_hashes:
|
||||
# This is an older duplicate — replace with back-reference
|
||||
result[i] = {**msg, "content": "[Duplicate tool output — same content as a more recent call]"}
|
||||
pruned += 1
|
||||
else:
|
||||
content_hashes[h] = (i, msg.get("tool_call_id", "?"))
|
||||
|
||||
# Pass 2: Replace old tool results with informative summaries
|
||||
for i in range(prune_boundary):
|
||||
msg = result[i]
|
||||
if msg.get("role") != "tool":
|
||||
continue
|
||||
content = msg.get("content", "")
|
||||
# Skip multimodal content (list of content blocks)
|
||||
if isinstance(content, list):
|
||||
continue
|
||||
if not content or content == _PRUNED_TOOL_PLACEHOLDER:
|
||||
continue
|
||||
# Skip already-deduplicated or previously-summarized results
|
||||
if content.startswith("[Duplicate tool output"):
|
||||
continue
|
||||
# Only prune if the content is substantial (>200 chars)
|
||||
if len(content) > 200:
|
||||
call_id = msg.get("tool_call_id", "")
|
||||
tool_name, tool_args = call_id_to_tool.get(call_id, ("unknown", ""))
|
||||
summary = _summarize_tool_result(tool_name, tool_args, content)
|
||||
result[i] = {**msg, "content": summary}
|
||||
result[i] = {**msg, "content": _PRUNED_TOOL_PLACEHOLDER}
|
||||
pruned += 1
|
||||
|
||||
# Pass 3: Truncate large tool_call arguments in assistant messages
|
||||
# outside the protected tail. write_file with 50KB content, for
|
||||
# example, survives pruning entirely without this.
|
||||
for i in range(prune_boundary):
|
||||
msg = result[i]
|
||||
if msg.get("role") != "assistant" or not msg.get("tool_calls"):
|
||||
continue
|
||||
new_tcs = []
|
||||
modified = False
|
||||
for tc in msg["tool_calls"]:
|
||||
if isinstance(tc, dict):
|
||||
args = tc.get("function", {}).get("arguments", "")
|
||||
if len(args) > 500:
|
||||
tc = {**tc, "function": {**tc["function"], "arguments": args[:200] + "...[truncated]"}}
|
||||
modified = True
|
||||
new_tcs.append(tc)
|
||||
if modified:
|
||||
result[i] = {**msg, "tool_calls": new_tcs}
|
||||
|
||||
return result, pruned
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -584,45 +353,29 @@ class ContextCompressor(ContextEngine):
|
||||
)
|
||||
|
||||
# Shared structured template (used by both paths).
|
||||
_template_sections = f"""## Active Task
|
||||
[THE SINGLE MOST IMPORTANT FIELD. Copy the user's most recent request or
|
||||
task assignment verbatim — the exact words they used. If multiple tasks
|
||||
were requested and only some are done, list only the ones NOT yet completed.
|
||||
The next assistant must pick up exactly here. Example:
|
||||
"User asked: 'Now refactor the auth module to use JWT instead of sessions'"
|
||||
If no outstanding task exists, write "None."]
|
||||
|
||||
## Goal
|
||||
[What the user is trying to accomplish overall]
|
||||
# Key changes vs v1:
|
||||
# - "Pending User Asks" section (from Claude Code) explicitly tracks
|
||||
# unanswered questions so the model knows what's resolved vs open
|
||||
# - "Remaining Work" replaces "Next Steps" to avoid reading as active
|
||||
# instructions
|
||||
# - "Resolved Questions" makes it clear which questions were already
|
||||
# answered (prevents model from re-answering them)
|
||||
_template_sections = f"""## Goal
|
||||
[What the user is trying to accomplish]
|
||||
|
||||
## Constraints & Preferences
|
||||
[User preferences, coding style, constraints, important decisions]
|
||||
|
||||
## Completed Actions
|
||||
[Numbered list of concrete actions taken — include tool used, target, and outcome.
|
||||
Format each as: N. ACTION target — outcome [tool: name]
|
||||
Example:
|
||||
1. READ config.py:45 — found `==` should be `!=` [tool: read_file]
|
||||
2. PATCH config.py:45 — changed `==` to `!=` [tool: patch]
|
||||
3. TEST `pytest tests/` — 3/50 failed: test_parse, test_validate, test_edge [tool: terminal]
|
||||
Be specific with file paths, commands, line numbers, and results.]
|
||||
|
||||
## Active State
|
||||
[Current working state — include:
|
||||
- Working directory and branch (if applicable)
|
||||
- Modified/created files with brief note on each
|
||||
- Test status (X/Y passing)
|
||||
- Any running processes or servers
|
||||
- Environment details that matter]
|
||||
|
||||
## In Progress
|
||||
[Work currently underway — what was being done when compaction fired]
|
||||
|
||||
## Blocked
|
||||
[Any blockers, errors, or issues not yet resolved. Include exact error messages.]
|
||||
## Progress
|
||||
### Done
|
||||
[Completed work — include specific file paths, commands run, results obtained]
|
||||
### In Progress
|
||||
[Work currently underway]
|
||||
### Blocked
|
||||
[Any blockers or issues encountered]
|
||||
|
||||
## Key Decisions
|
||||
[Important technical decisions and WHY they were made]
|
||||
[Important technical decisions and why they were made]
|
||||
|
||||
## Resolved Questions
|
||||
[Questions the user asked that were ALREADY answered — include the answer so the next assistant does not re-answer them]
|
||||
@@ -639,7 +392,10 @@ Be specific with file paths, commands, line numbers, and results.]
|
||||
## Critical Context
|
||||
[Any specific values, error messages, configuration details, or data that would be lost without explicit preservation]
|
||||
|
||||
Target ~{summary_budget} tokens. Be CONCRETE — include file paths, command outputs, error messages, line numbers, and specific values. Avoid vague descriptions like "made some changes" — say exactly what changed.
|
||||
## Tools & Patterns
|
||||
[Which tools were used, how they were used effectively, and any tool-specific discoveries]
|
||||
|
||||
Target ~{summary_budget} tokens. Be specific — include file paths, command outputs, error messages, and concrete values rather than vague descriptions.
|
||||
|
||||
Write only the summary body. Do not include any preamble or prefix."""
|
||||
|
||||
@@ -655,7 +411,7 @@ PREVIOUS SUMMARY:
|
||||
NEW TURNS TO INCORPORATE:
|
||||
{content_to_summarize}
|
||||
|
||||
Update the summary using this exact structure. PRESERVE all existing information that is still relevant. ADD new completed actions to the numbered list (continue numbering). Move items from "In Progress" to "Completed Actions" when done. Move answered questions to "Resolved Questions". Update "Active State" to reflect current state. Remove information only if it is clearly obsolete. CRITICAL: Update "## Active Task" to reflect the user's most recent unfulfilled request — this is the most important field for task continuity.
|
||||
Update the summary using this exact structure. PRESERVE all existing information that is still relevant. ADD new progress. Move items from "In Progress" to "Done" when completed. Move answered questions to "Resolved Questions". Remove information only if it is clearly obsolete.
|
||||
|
||||
{_template_sections}"""
|
||||
else:
|
||||
@@ -682,15 +438,8 @@ The user has requested that this compaction PRIORITISE preserving all informatio
|
||||
try:
|
||||
call_kwargs = {
|
||||
"task": "compression",
|
||||
"main_runtime": {
|
||||
"model": self.model,
|
||||
"provider": self.provider,
|
||||
"base_url": self.base_url,
|
||||
"api_key": self.api_key,
|
||||
"api_mode": self.api_mode,
|
||||
},
|
||||
"messages": [{"role": "user", "content": prompt}],
|
||||
"max_tokens": int(summary_budget * 1.3),
|
||||
"max_tokens": summary_budget * 2,
|
||||
# timeout resolved from auxiliary.compression.timeout config by call_llm
|
||||
}
|
||||
if self.summary_model:
|
||||
@@ -704,10 +453,8 @@ The user has requested that this compaction PRIORITISE preserving all informatio
|
||||
# Store for iterative updates on next compaction
|
||||
self._previous_summary = summary
|
||||
self._summary_failure_cooldown_until = 0.0
|
||||
self._summary_model_fallen_back = False
|
||||
return self._with_summary_prefix(summary)
|
||||
except RuntimeError:
|
||||
# No provider configured — long cooldown, unlikely to self-resolve
|
||||
self._summary_failure_cooldown_until = time.monotonic() + _SUMMARY_FAILURE_COOLDOWN_SECONDS
|
||||
logging.warning("Context compression: no provider available for "
|
||||
"summary. Middle turns will be dropped without summary "
|
||||
@@ -715,42 +462,12 @@ The user has requested that this compaction PRIORITISE preserving all informatio
|
||||
_SUMMARY_FAILURE_COOLDOWN_SECONDS)
|
||||
return None
|
||||
except Exception as e:
|
||||
# If the summary model is different from the main model and the
|
||||
# error looks permanent (model not found, 503, 404), fall back to
|
||||
# using the main model instead of entering cooldown that leaves
|
||||
# context growing unbounded. (#8620 sub-issue 4)
|
||||
_status = getattr(e, "status_code", None) or getattr(getattr(e, "response", None), "status_code", None)
|
||||
_err_str = str(e).lower()
|
||||
_is_model_not_found = (
|
||||
_status in (404, 503)
|
||||
or "model_not_found" in _err_str
|
||||
or "does not exist" in _err_str
|
||||
or "no available channel" in _err_str
|
||||
)
|
||||
if (
|
||||
_is_model_not_found
|
||||
and self.summary_model
|
||||
and self.summary_model != self.model
|
||||
and not getattr(self, "_summary_model_fallen_back", False)
|
||||
):
|
||||
self._summary_model_fallen_back = True
|
||||
logging.warning(
|
||||
"Summary model '%s' not available (%s). "
|
||||
"Falling back to main model '%s' for compression.",
|
||||
self.summary_model, e, self.model,
|
||||
)
|
||||
self.summary_model = "" # empty = use main model
|
||||
self._summary_failure_cooldown_until = 0.0 # no cooldown
|
||||
return self._generate_summary(messages, summary_budget) # retry immediately
|
||||
|
||||
# Transient errors (timeout, rate limit, network) — shorter cooldown
|
||||
_transient_cooldown = 60
|
||||
self._summary_failure_cooldown_until = time.monotonic() + _transient_cooldown
|
||||
self._summary_failure_cooldown_until = time.monotonic() + _SUMMARY_FAILURE_COOLDOWN_SECONDS
|
||||
logging.warning(
|
||||
"Failed to generate context summary: %s. "
|
||||
"Further summary attempts paused for %d seconds.",
|
||||
e,
|
||||
_transient_cooldown,
|
||||
_SUMMARY_FAILURE_COOLDOWN_SECONDS,
|
||||
)
|
||||
return None
|
||||
|
||||
@@ -873,62 +590,6 @@ The user has requested that this compaction PRIORITISE preserving all informatio
|
||||
# Tail protection by token budget
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _find_last_user_message_idx(
|
||||
self, messages: List[Dict[str, Any]], head_end: int
|
||||
) -> int:
|
||||
"""Return the index of the last user-role message at or after *head_end*, or -1."""
|
||||
for i in range(len(messages) - 1, head_end - 1, -1):
|
||||
if messages[i].get("role") == "user":
|
||||
return i
|
||||
return -1
|
||||
|
||||
def _ensure_last_user_message_in_tail(
|
||||
self,
|
||||
messages: List[Dict[str, Any]],
|
||||
cut_idx: int,
|
||||
head_end: int,
|
||||
) -> int:
|
||||
"""Guarantee the most recent user message is in the protected tail.
|
||||
|
||||
Context compressor bug (#10896): ``_align_boundary_backward`` can pull
|
||||
``cut_idx`` past a user message when it tries to keep tool_call/result
|
||||
groups together. If the last user message ends up in the *compressed*
|
||||
middle region the LLM summariser writes it into "Pending User Asks",
|
||||
but ``SUMMARY_PREFIX`` tells the next model to respond only to user
|
||||
messages *after* the summary — so the task effectively disappears from
|
||||
the active context, causing the agent to stall, repeat completed work,
|
||||
or silently drop the user's latest request.
|
||||
|
||||
Fix: if the last user-role message is not already in the tail
|
||||
(``messages[cut_idx:]``), walk ``cut_idx`` back to include it. We
|
||||
then re-align backward one more time to avoid splitting any
|
||||
tool_call/result group that immediately precedes the user message.
|
||||
"""
|
||||
last_user_idx = self._find_last_user_message_idx(messages, head_end)
|
||||
if last_user_idx < 0:
|
||||
# No user message found beyond head — nothing to anchor.
|
||||
return cut_idx
|
||||
|
||||
if last_user_idx >= cut_idx:
|
||||
# Already in the tail; nothing to do.
|
||||
return cut_idx
|
||||
|
||||
# The last user message is in the middle (compressed) region.
|
||||
# Pull cut_idx back to it directly — a user message is already a
|
||||
# clean boundary (no tool_call/result splitting risk), so there is no
|
||||
# need to call _align_boundary_backward here; doing so would
|
||||
# unnecessarily pull the cut further back into the preceding
|
||||
# assistant + tool_calls group.
|
||||
if not self.quiet_mode:
|
||||
logger.debug(
|
||||
"Anchoring tail cut to last user message at index %d "
|
||||
"(was %d) to prevent active-task loss after compression",
|
||||
last_user_idx,
|
||||
cut_idx,
|
||||
)
|
||||
# Safety: never go back into the head region.
|
||||
return max(last_user_idx, head_end + 1)
|
||||
|
||||
def _find_tail_cut_by_tokens(
|
||||
self, messages: List[Dict[str, Any]], head_end: int,
|
||||
token_budget: int | None = None,
|
||||
@@ -946,8 +607,7 @@ The user has requested that this compaction PRIORITISE preserving all informatio
|
||||
read, etc.). If even the minimum 3 messages exceed 1.5x the budget
|
||||
the cut is placed right after the head so compression still runs.
|
||||
|
||||
Never cuts inside a tool_call/result group. Always ensures the most
|
||||
recent user message is in the tail (see ``_ensure_last_user_message_in_tail``).
|
||||
Never cuts inside a tool_call/result group.
|
||||
"""
|
||||
if token_budget is None:
|
||||
token_budget = self.tail_token_budget
|
||||
@@ -986,10 +646,6 @@ The user has requested that this compaction PRIORITISE preserving all informatio
|
||||
# Align to avoid splitting tool groups
|
||||
cut_idx = self._align_boundary_backward(messages, cut_idx)
|
||||
|
||||
# Ensure the most recent user message is always in the tail so the
|
||||
# active task is never lost to compression (fixes #10896).
|
||||
cut_idx = self._ensure_last_user_message_in_tail(messages, cut_idx, head_end)
|
||||
|
||||
return max(cut_idx, head_end + 1)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -1077,11 +733,11 @@ The user has requested that this compaction PRIORITISE preserving all informatio
|
||||
compressed = []
|
||||
for i in range(compress_start):
|
||||
msg = messages[i].copy()
|
||||
if i == 0 and msg.get("role") == "system":
|
||||
existing = msg.get("content") or ""
|
||||
_compression_note = "[Note: Some earlier conversation turns have been compacted into a handoff summary to preserve context space. The current session state may still reflect earlier work, so build on that summary and state rather than re-doing work.]"
|
||||
if _compression_note not in existing:
|
||||
msg["content"] = existing + "\n\n" + _compression_note
|
||||
if i == 0 and msg.get("role") == "system" and self.compression_count == 0:
|
||||
msg["content"] = (
|
||||
(msg.get("content") or "")
|
||||
+ "\n\n[Note: Some earlier conversation turns have been compacted into a handoff summary to preserve context space. The current session state may still reflect earlier work, so build on that summary and state rather than re-doing work.]"
|
||||
)
|
||||
compressed.append(msg)
|
||||
|
||||
# If LLM summary failed, insert a static fallback so the model
|
||||
@@ -1139,24 +795,14 @@ The user has requested that this compaction PRIORITISE preserving all informatio
|
||||
|
||||
compressed = self._sanitize_tool_pairs(compressed)
|
||||
|
||||
new_estimate = estimate_messages_tokens_rough(compressed)
|
||||
saved_estimate = display_tokens - new_estimate
|
||||
|
||||
# Anti-thrashing: track compression effectiveness
|
||||
savings_pct = (saved_estimate / display_tokens * 100) if display_tokens > 0 else 0
|
||||
self._last_compression_savings_pct = savings_pct
|
||||
if savings_pct < 10:
|
||||
self._ineffective_compression_count += 1
|
||||
else:
|
||||
self._ineffective_compression_count = 0
|
||||
|
||||
if not self.quiet_mode:
|
||||
new_estimate = estimate_messages_tokens_rough(compressed)
|
||||
saved_estimate = display_tokens - new_estimate
|
||||
logger.info(
|
||||
"Compressed: %d -> %d messages (~%d tokens saved, %.0f%%)",
|
||||
"Compressed: %d -> %d messages (~%d tokens saved)",
|
||||
n_messages,
|
||||
len(compressed),
|
||||
saved_estimate,
|
||||
savings_pct,
|
||||
)
|
||||
logger.info("Compression #%d complete", self.compression_count)
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ Lifecycle:
|
||||
"""
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Any, Dict, List
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
|
||||
class ContextEngine(ABC):
|
||||
|
||||
@@ -313,25 +313,9 @@ class CopilotACPClient:
|
||||
tools=tools,
|
||||
tool_choice=tool_choice,
|
||||
)
|
||||
# Normalise timeout: run_agent.py may pass an httpx.Timeout object
|
||||
# (used natively by the OpenAI SDK) rather than a plain float.
|
||||
if timeout is None:
|
||||
_effective_timeout = _DEFAULT_TIMEOUT_SECONDS
|
||||
elif isinstance(timeout, (int, float)):
|
||||
_effective_timeout = float(timeout)
|
||||
else:
|
||||
# httpx.Timeout or similar — pick the largest component so the
|
||||
# subprocess has enough wall-clock time for the full response.
|
||||
_candidates = [
|
||||
getattr(timeout, attr, None)
|
||||
for attr in ("read", "write", "connect", "pool", "timeout")
|
||||
]
|
||||
_numeric = [float(v) for v in _candidates if isinstance(v, (int, float))]
|
||||
_effective_timeout = max(_numeric) if _numeric else _DEFAULT_TIMEOUT_SECONDS
|
||||
|
||||
response_text, reasoning_text = self._run_prompt(
|
||||
prompt_text,
|
||||
timeout_seconds=_effective_timeout,
|
||||
timeout_seconds=float(timeout or _DEFAULT_TIMEOUT_SECONDS),
|
||||
)
|
||||
|
||||
tool_calls, cleaned_text = _extract_tool_calls_from_text(response_text)
|
||||
|
||||
+2
-105
@@ -18,12 +18,12 @@ import hermes_cli.auth as auth_mod
|
||||
from hermes_cli.auth import (
|
||||
CODEX_ACCESS_TOKEN_REFRESH_SKEW_SECONDS,
|
||||
DEFAULT_AGENT_KEY_MIN_TTL_SECONDS,
|
||||
KIMI_CODE_BASE_URL,
|
||||
PROVIDER_REGISTRY,
|
||||
_auth_store_lock,
|
||||
_codex_access_token_is_expiring,
|
||||
_decode_jwt_claims,
|
||||
_import_codex_cli_tokens,
|
||||
_write_codex_cli_tokens,
|
||||
_load_auth_store,
|
||||
_load_provider_state,
|
||||
_resolve_kimi_base_url,
|
||||
@@ -288,14 +288,6 @@ def _iter_custom_providers(config: Optional[dict] = None):
|
||||
return
|
||||
custom_providers = config.get("custom_providers")
|
||||
if not isinstance(custom_providers, list):
|
||||
# Fall back to the v12+ providers dict via the compatibility layer
|
||||
try:
|
||||
from hermes_cli.config import get_compatible_custom_providers
|
||||
|
||||
custom_providers = get_compatible_custom_providers(config)
|
||||
except Exception:
|
||||
return
|
||||
if not custom_providers:
|
||||
return
|
||||
for entry in custom_providers:
|
||||
if not isinstance(entry, dict):
|
||||
@@ -701,14 +693,6 @@ class CredentialPool:
|
||||
self._replace_entry(synced, updated)
|
||||
self._persist()
|
||||
self._sync_device_code_entry_to_auth_store(updated)
|
||||
try:
|
||||
_write_codex_cli_tokens(
|
||||
updated.access_token,
|
||||
updated.refresh_token,
|
||||
last_refresh=updated.last_refresh,
|
||||
)
|
||||
except Exception as wexc:
|
||||
logger.debug("Failed to write refreshed Codex tokens to CLI file (retry): %s", wexc)
|
||||
return updated
|
||||
except Exception as retry_exc:
|
||||
logger.debug("Codex retry refresh also failed: %s", retry_exc)
|
||||
@@ -734,17 +718,6 @@ class CredentialPool:
|
||||
# _seed_from_singletons() on the next load_pool() sees fresh state
|
||||
# instead of re-seeding stale/consumed tokens.
|
||||
self._sync_device_code_entry_to_auth_store(updated)
|
||||
# Write refreshed tokens back to ~/.codex/auth.json so Codex CLI
|
||||
# and VS Code don't hit "refresh_token_reused" on their next refresh.
|
||||
if self.provider == "openai-codex":
|
||||
try:
|
||||
_write_codex_cli_tokens(
|
||||
updated.access_token,
|
||||
updated.refresh_token,
|
||||
last_refresh=updated.last_refresh,
|
||||
)
|
||||
except Exception as wexc:
|
||||
logger.debug("Failed to write refreshed Codex tokens to CLI file: %s", wexc)
|
||||
return updated
|
||||
|
||||
def _entry_needs_refresh(self, entry: PooledCredential) -> bool:
|
||||
@@ -1130,14 +1103,6 @@ def _seed_from_singletons(provider: str, entries: List[PooledCredential]) -> Tup
|
||||
state = _load_provider_state(auth_store, "nous")
|
||||
if state:
|
||||
active_sources.add("device_code")
|
||||
# Prefer a user-supplied label embedded in the singleton state
|
||||
# (set by persist_nous_credentials(label=...) when the user ran
|
||||
# `hermes auth add nous --label <name>`). Fall back to the
|
||||
# auto-derived token fingerprint for logins that didn't supply one.
|
||||
custom_label = str(state.get("label") or "").strip()
|
||||
seeded_label = custom_label or label_from_token(
|
||||
state.get("access_token", ""), "device_code"
|
||||
)
|
||||
changed |= _upsert_entry(
|
||||
entries,
|
||||
provider,
|
||||
@@ -1156,79 +1121,11 @@ def _seed_from_singletons(provider: str, entries: List[PooledCredential]) -> Tup
|
||||
"agent_key": state.get("agent_key"),
|
||||
"agent_key_expires_at": state.get("agent_key_expires_at"),
|
||||
"tls": state.get("tls") if isinstance(state.get("tls"), dict) else None,
|
||||
"label": seeded_label,
|
||||
"label": label_from_token(state.get("access_token", ""), "device_code"),
|
||||
},
|
||||
)
|
||||
|
||||
elif provider == "copilot":
|
||||
# Copilot tokens are resolved dynamically via `gh auth token` or
|
||||
# env vars (COPILOT_GITHUB_TOKEN / GH_TOKEN). They don't live in
|
||||
# the auth store or credential pool, so we resolve them here.
|
||||
try:
|
||||
from hermes_cli.copilot_auth import resolve_copilot_token
|
||||
token, source = resolve_copilot_token()
|
||||
if token:
|
||||
source_name = "gh_cli" if "gh" in source.lower() else f"env:{source}"
|
||||
active_sources.add(source_name)
|
||||
pconfig = PROVIDER_REGISTRY.get(provider)
|
||||
changed |= _upsert_entry(
|
||||
entries,
|
||||
provider,
|
||||
source_name,
|
||||
{
|
||||
"source": source_name,
|
||||
"auth_type": AUTH_TYPE_API_KEY,
|
||||
"access_token": token,
|
||||
"base_url": pconfig.inference_base_url if pconfig else "",
|
||||
"label": source,
|
||||
},
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.debug("Copilot token seed failed: %s", exc)
|
||||
|
||||
elif provider == "qwen-oauth":
|
||||
# Qwen OAuth tokens live in ~/.qwen/oauth_creds.json, written by
|
||||
# the Qwen CLI (`qwen auth qwen-oauth`). They aren't in the
|
||||
# Hermes auth store or env vars, so resolve them here.
|
||||
# Use refresh_if_expiring=False to avoid network calls during
|
||||
# pool loading / provider discovery.
|
||||
try:
|
||||
from hermes_cli.auth import resolve_qwen_runtime_credentials
|
||||
creds = resolve_qwen_runtime_credentials(refresh_if_expiring=False)
|
||||
token = creds.get("api_key", "")
|
||||
if token:
|
||||
source_name = creds.get("source", "qwen-cli")
|
||||
active_sources.add(source_name)
|
||||
changed |= _upsert_entry(
|
||||
entries,
|
||||
provider,
|
||||
source_name,
|
||||
{
|
||||
"source": source_name,
|
||||
"auth_type": AUTH_TYPE_OAUTH,
|
||||
"access_token": token,
|
||||
"expires_at_ms": creds.get("expires_at_ms"),
|
||||
"base_url": creds.get("base_url", ""),
|
||||
"label": creds.get("auth_file", source_name),
|
||||
},
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.debug("Qwen OAuth token seed failed: %s", exc)
|
||||
|
||||
elif provider == "openai-codex":
|
||||
# Respect user suppression — `hermes auth remove openai-codex` marks
|
||||
# the device_code source as suppressed so it won't be re-seeded from
|
||||
# either the Hermes auth store or ~/.codex/auth.json. Without this
|
||||
# gate the removal is instantly undone on the next load_pool() call.
|
||||
codex_suppressed = False
|
||||
try:
|
||||
from hermes_cli.auth import is_source_suppressed
|
||||
codex_suppressed = is_source_suppressed(provider, "device_code")
|
||||
except ImportError:
|
||||
pass
|
||||
if codex_suppressed:
|
||||
return changed, active_sources
|
||||
|
||||
state = _load_provider_state(auth_store, "openai-codex")
|
||||
tokens = state.get("tokens") if isinstance(state, dict) else None
|
||||
# Fallback: import from Codex CLI (~/.codex/auth.json) if Hermes auth
|
||||
|
||||
+86
-39
@@ -77,6 +77,12 @@ def _diff_ansi() -> dict[str, str]:
|
||||
return _diff_colors_cached
|
||||
|
||||
|
||||
def reset_diff_colors() -> None:
|
||||
"""Reset cached diff colors (call after /skin switch)."""
|
||||
global _diff_colors_cached
|
||||
_diff_colors_cached = None
|
||||
|
||||
|
||||
# Module-level helpers — each call resolves from the active skin lazily.
|
||||
def _diff_dim(): return _diff_ansi()["dim"]
|
||||
def _diff_file(): return _diff_ansi()["file"]
|
||||
@@ -600,45 +606,6 @@ class KawaiiSpinner:
|
||||
"analyzing", "computing", "synthesizing", "formulating", "brainstorming",
|
||||
]
|
||||
|
||||
@classmethod
|
||||
def get_waiting_faces(cls) -> list:
|
||||
"""Return waiting faces from the active skin, falling back to KAWAII_WAITING."""
|
||||
try:
|
||||
skin = _get_skin()
|
||||
if skin:
|
||||
faces = skin.spinner.get("waiting_faces", [])
|
||||
if faces:
|
||||
return faces
|
||||
except Exception:
|
||||
pass
|
||||
return cls.KAWAII_WAITING
|
||||
|
||||
@classmethod
|
||||
def get_thinking_faces(cls) -> list:
|
||||
"""Return thinking faces from the active skin, falling back to KAWAII_THINKING."""
|
||||
try:
|
||||
skin = _get_skin()
|
||||
if skin:
|
||||
faces = skin.spinner.get("thinking_faces", [])
|
||||
if faces:
|
||||
return faces
|
||||
except Exception:
|
||||
pass
|
||||
return cls.KAWAII_THINKING
|
||||
|
||||
@classmethod
|
||||
def get_thinking_verbs(cls) -> list:
|
||||
"""Return thinking verbs from the active skin, falling back to THINKING_VERBS."""
|
||||
try:
|
||||
skin = _get_skin()
|
||||
if skin:
|
||||
verbs = skin.spinner.get("thinking_verbs", [])
|
||||
if verbs:
|
||||
return verbs
|
||||
except Exception:
|
||||
pass
|
||||
return cls.THINKING_VERBS
|
||||
|
||||
def __init__(self, message: str = "", spinner_type: str = 'dots', print_fn=None):
|
||||
self.message = message
|
||||
self.spinner_frames = self.SPINNERS.get(spinner_type, self.SPINNERS['dots'])
|
||||
@@ -993,4 +960,84 @@ def get_cute_tool_message(
|
||||
# Honcho session line (one-liner with clickable OSC 8 hyperlink)
|
||||
# =========================================================================
|
||||
|
||||
_DIM = "\033[2m"
|
||||
_SKY_BLUE = "\033[38;5;117m"
|
||||
_ANSI_RESET = "\033[0m"
|
||||
|
||||
|
||||
# =========================================================================
|
||||
# Context pressure display (CLI user-facing warnings)
|
||||
# =========================================================================
|
||||
|
||||
# ANSI color codes for context pressure tiers
|
||||
_CYAN = "\033[36m"
|
||||
_YELLOW = "\033[33m"
|
||||
_BOLD = "\033[1m"
|
||||
_DIM_ANSI = "\033[2m"
|
||||
|
||||
# Bar characters
|
||||
_BAR_FILLED = "▰"
|
||||
_BAR_EMPTY = "▱"
|
||||
_BAR_WIDTH = 20
|
||||
|
||||
|
||||
def format_context_pressure(
|
||||
compaction_progress: float,
|
||||
threshold_tokens: int,
|
||||
threshold_percent: float,
|
||||
compression_enabled: bool = True,
|
||||
) -> str:
|
||||
"""Build a formatted context pressure line for CLI display.
|
||||
|
||||
The bar and percentage show progress toward the compaction threshold,
|
||||
NOT the raw context window. 100% = compaction fires.
|
||||
|
||||
Args:
|
||||
compaction_progress: How close to compaction (0.0–1.0, 1.0 = fires).
|
||||
threshold_tokens: Compaction threshold in tokens.
|
||||
threshold_percent: Compaction threshold as a fraction of context window.
|
||||
compression_enabled: Whether auto-compression is active.
|
||||
"""
|
||||
pct_int = min(int(compaction_progress * 100), 100)
|
||||
filled = min(int(compaction_progress * _BAR_WIDTH), _BAR_WIDTH)
|
||||
bar = _BAR_FILLED * filled + _BAR_EMPTY * (_BAR_WIDTH - filled)
|
||||
|
||||
threshold_k = f"{threshold_tokens // 1000}k" if threshold_tokens >= 1000 else str(threshold_tokens)
|
||||
threshold_pct_int = int(threshold_percent * 100)
|
||||
|
||||
color = f"{_BOLD}{_YELLOW}"
|
||||
icon = "⚠"
|
||||
if compression_enabled:
|
||||
hint = "compaction approaching"
|
||||
else:
|
||||
hint = "no auto-compaction"
|
||||
|
||||
return (
|
||||
f" {color}{icon} context {bar} {pct_int}% to compaction{_ANSI_RESET}"
|
||||
f" {_DIM_ANSI}{threshold_k} threshold ({threshold_pct_int}%) · {hint}{_ANSI_RESET}"
|
||||
)
|
||||
|
||||
|
||||
def format_context_pressure_gateway(
|
||||
compaction_progress: float,
|
||||
threshold_percent: float,
|
||||
compression_enabled: bool = True,
|
||||
) -> str:
|
||||
"""Build a plain-text context pressure notification for messaging platforms.
|
||||
|
||||
No ANSI — just Unicode and plain text suitable for Telegram/Discord/etc.
|
||||
The percentage shows progress toward the compaction threshold.
|
||||
"""
|
||||
pct_int = min(int(compaction_progress * 100), 100)
|
||||
filled = min(int(compaction_progress * _BAR_WIDTH), _BAR_WIDTH)
|
||||
bar = _BAR_FILLED * filled + _BAR_EMPTY * (_BAR_WIDTH - filled)
|
||||
|
||||
threshold_pct_int = int(threshold_percent * 100)
|
||||
|
||||
icon = "⚠️"
|
||||
if compression_enabled:
|
||||
hint = f"Context compaction approaching (threshold: {threshold_pct_int}% of window)."
|
||||
else:
|
||||
hint = "Auto-compaction is disabled — context may be truncated."
|
||||
|
||||
return f"{icon} Context: {bar} {pct_int}% to compaction\n{hint}"
|
||||
|
||||
@@ -13,6 +13,7 @@ from __future__ import annotations
|
||||
|
||||
import enum
|
||||
import logging
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
@@ -112,10 +113,6 @@ _RATE_LIMIT_PATTERNS = [
|
||||
"please retry after",
|
||||
"resource_exhausted",
|
||||
"rate increased too quickly", # Alibaba/DashScope throttling
|
||||
# AWS Bedrock throttling
|
||||
"throttlingexception",
|
||||
"too many concurrent requests",
|
||||
"servicequotaexceededexception",
|
||||
]
|
||||
|
||||
# Usage-limit patterns that need disambiguation (could be billing OR rate_limit)
|
||||
@@ -160,26 +157,9 @@ _CONTEXT_OVERFLOW_PATTERNS = [
|
||||
"prompt exceeds max length",
|
||||
"max_tokens",
|
||||
"maximum number of tokens",
|
||||
# vLLM / local inference server patterns
|
||||
"exceeds the max_model_len",
|
||||
"max_model_len",
|
||||
"prompt length", # "engine prompt length X exceeds"
|
||||
"input is too long",
|
||||
"maximum model length",
|
||||
# Ollama patterns
|
||||
"context length exceeded",
|
||||
"truncating input",
|
||||
# llama.cpp / llama-server patterns
|
||||
"slot context", # "slot context: N tokens, prompt N tokens"
|
||||
"n_ctx_slot",
|
||||
# Chinese error messages (some providers return these)
|
||||
"超过最大长度",
|
||||
"上下文长度",
|
||||
# AWS Bedrock Converse API error patterns
|
||||
"input is too long",
|
||||
"max input token",
|
||||
"input token",
|
||||
"exceeds the maximum number of input tokens",
|
||||
]
|
||||
|
||||
# Model not found patterns
|
||||
|
||||
@@ -1,895 +0,0 @@
|
||||
"""OpenAI-compatible facade that talks to Google's Cloud Code Assist backend.
|
||||
|
||||
This adapter lets Hermes use the ``google-gemini-cli`` provider as if it were
|
||||
a standard OpenAI-shaped chat completion endpoint, while the underlying HTTP
|
||||
traffic goes to ``cloudcode-pa.googleapis.com/v1internal:{generateContent,
|
||||
streamGenerateContent}`` with a Bearer access token obtained via OAuth PKCE.
|
||||
|
||||
Architecture
|
||||
------------
|
||||
- ``GeminiCloudCodeClient`` exposes ``.chat.completions.create(**kwargs)``
|
||||
mirroring the subset of the OpenAI SDK that ``run_agent.py`` uses.
|
||||
- Incoming OpenAI ``messages[]`` / ``tools[]`` / ``tool_choice`` are translated
|
||||
to Gemini's native ``contents[]`` / ``tools[].functionDeclarations`` /
|
||||
``toolConfig`` / ``systemInstruction`` shape.
|
||||
- The request body is wrapped ``{project, model, user_prompt_id, request}``
|
||||
per Code Assist API expectations.
|
||||
- Responses (``candidates[].content.parts[]``) are converted back to
|
||||
OpenAI ``choices[0].message`` shape with ``content`` + ``tool_calls``.
|
||||
- Streaming uses SSE (``?alt=sse``) and yields OpenAI-shaped delta chunks.
|
||||
|
||||
Attribution
|
||||
-----------
|
||||
Translation semantics follow jenslys/opencode-gemini-auth (MIT) and the public
|
||||
Gemini API docs. Request envelope shape
|
||||
(``{project, model, user_prompt_id, request}``) is documented nowhere; it is
|
||||
reverse-engineered from the opencode-gemini-auth and clawdbot implementations.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
import uuid
|
||||
from types import SimpleNamespace
|
||||
from typing import Any, Dict, Iterator, List, Optional
|
||||
|
||||
import httpx
|
||||
|
||||
from agent import google_oauth
|
||||
from agent.google_code_assist import (
|
||||
CODE_ASSIST_ENDPOINT,
|
||||
FREE_TIER_ID,
|
||||
CodeAssistError,
|
||||
ProjectContext,
|
||||
resolve_project_context,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Request translation: OpenAI → Gemini
|
||||
# =============================================================================
|
||||
|
||||
_ROLE_MAP_OPENAI_TO_GEMINI = {
|
||||
"user": "user",
|
||||
"assistant": "model",
|
||||
"system": "user", # handled separately via systemInstruction
|
||||
"tool": "user", # functionResponse is wrapped in a user-role turn
|
||||
"function": "user",
|
||||
}
|
||||
|
||||
|
||||
def _coerce_content_to_text(content: Any) -> str:
|
||||
"""OpenAI content may be str or a list of parts; reduce to plain text."""
|
||||
if content is None:
|
||||
return ""
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
pieces: List[str] = []
|
||||
for p in content:
|
||||
if isinstance(p, str):
|
||||
pieces.append(p)
|
||||
elif isinstance(p, dict):
|
||||
if p.get("type") == "text" and isinstance(p.get("text"), str):
|
||||
pieces.append(p["text"])
|
||||
# Multimodal (image_url, etc.) — stub for now; log and skip
|
||||
elif p.get("type") in ("image_url", "input_audio"):
|
||||
logger.debug("Dropping multimodal part (not yet supported): %s", p.get("type"))
|
||||
return "\n".join(pieces)
|
||||
return str(content)
|
||||
|
||||
|
||||
def _translate_tool_call_to_gemini(tool_call: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""OpenAI tool_call -> Gemini functionCall part."""
|
||||
fn = tool_call.get("function") or {}
|
||||
args_raw = fn.get("arguments", "")
|
||||
try:
|
||||
args = json.loads(args_raw) if isinstance(args_raw, str) and args_raw else {}
|
||||
except json.JSONDecodeError:
|
||||
args = {"_raw": args_raw}
|
||||
if not isinstance(args, dict):
|
||||
args = {"_value": args}
|
||||
return {
|
||||
"functionCall": {
|
||||
"name": fn.get("name") or "",
|
||||
"args": args,
|
||||
},
|
||||
# Sentinel signature — matches opencode-gemini-auth's approach.
|
||||
# Without this, Code Assist rejects function calls that originated
|
||||
# outside its own chain.
|
||||
"thoughtSignature": "skip_thought_signature_validator",
|
||||
}
|
||||
|
||||
|
||||
def _translate_tool_result_to_gemini(message: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""OpenAI tool-role message -> Gemini functionResponse part.
|
||||
|
||||
The function name isn't in the OpenAI tool message directly; it must be
|
||||
passed via the assistant message that issued the call. For simplicity we
|
||||
look up ``name`` on the message (OpenAI SDK copies it there) or on the
|
||||
``tool_call_id`` cross-reference.
|
||||
"""
|
||||
name = str(message.get("name") or message.get("tool_call_id") or "tool")
|
||||
content = _coerce_content_to_text(message.get("content"))
|
||||
# Gemini expects the response as a dict under `response`. We wrap plain
|
||||
# text in {"output": "..."}.
|
||||
try:
|
||||
parsed = json.loads(content) if content.strip().startswith(("{", "[")) else None
|
||||
except json.JSONDecodeError:
|
||||
parsed = None
|
||||
response = parsed if isinstance(parsed, dict) else {"output": content}
|
||||
return {
|
||||
"functionResponse": {
|
||||
"name": name,
|
||||
"response": response,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _build_gemini_contents(
|
||||
messages: List[Dict[str, Any]],
|
||||
) -> tuple[List[Dict[str, Any]], Optional[Dict[str, Any]]]:
|
||||
"""Convert OpenAI messages[] to Gemini contents[] + systemInstruction."""
|
||||
system_text_parts: List[str] = []
|
||||
contents: List[Dict[str, Any]] = []
|
||||
|
||||
for msg in messages:
|
||||
if not isinstance(msg, dict):
|
||||
continue
|
||||
role = str(msg.get("role") or "user")
|
||||
|
||||
if role == "system":
|
||||
system_text_parts.append(_coerce_content_to_text(msg.get("content")))
|
||||
continue
|
||||
|
||||
# Tool result message — emit a user-role turn with functionResponse
|
||||
if role == "tool" or role == "function":
|
||||
contents.append({
|
||||
"role": "user",
|
||||
"parts": [_translate_tool_result_to_gemini(msg)],
|
||||
})
|
||||
continue
|
||||
|
||||
gemini_role = _ROLE_MAP_OPENAI_TO_GEMINI.get(role, "user")
|
||||
parts: List[Dict[str, Any]] = []
|
||||
|
||||
text = _coerce_content_to_text(msg.get("content"))
|
||||
if text:
|
||||
parts.append({"text": text})
|
||||
|
||||
# Assistant messages can carry tool_calls
|
||||
tool_calls = msg.get("tool_calls") or []
|
||||
if isinstance(tool_calls, list):
|
||||
for tc in tool_calls:
|
||||
if isinstance(tc, dict):
|
||||
parts.append(_translate_tool_call_to_gemini(tc))
|
||||
|
||||
if not parts:
|
||||
# Gemini rejects empty parts; skip the turn entirely
|
||||
continue
|
||||
|
||||
contents.append({"role": gemini_role, "parts": parts})
|
||||
|
||||
system_instruction: Optional[Dict[str, Any]] = None
|
||||
joined_system = "\n".join(p for p in system_text_parts if p).strip()
|
||||
if joined_system:
|
||||
system_instruction = {
|
||||
"role": "system",
|
||||
"parts": [{"text": joined_system}],
|
||||
}
|
||||
|
||||
return contents, system_instruction
|
||||
|
||||
|
||||
def _translate_tools_to_gemini(tools: Any) -> List[Dict[str, Any]]:
|
||||
"""OpenAI tools[] -> Gemini tools[].functionDeclarations[]."""
|
||||
if not isinstance(tools, list) or not tools:
|
||||
return []
|
||||
declarations: List[Dict[str, Any]] = []
|
||||
for t in tools:
|
||||
if not isinstance(t, dict):
|
||||
continue
|
||||
fn = t.get("function") or {}
|
||||
if not isinstance(fn, dict):
|
||||
continue
|
||||
name = fn.get("name")
|
||||
if not name:
|
||||
continue
|
||||
decl = {"name": str(name)}
|
||||
if fn.get("description"):
|
||||
decl["description"] = str(fn["description"])
|
||||
params = fn.get("parameters")
|
||||
if isinstance(params, dict):
|
||||
decl["parameters"] = params
|
||||
declarations.append(decl)
|
||||
if not declarations:
|
||||
return []
|
||||
return [{"functionDeclarations": declarations}]
|
||||
|
||||
|
||||
def _translate_tool_choice_to_gemini(tool_choice: Any) -> Optional[Dict[str, Any]]:
|
||||
"""OpenAI tool_choice -> Gemini toolConfig.functionCallingConfig."""
|
||||
if tool_choice is None:
|
||||
return None
|
||||
if isinstance(tool_choice, str):
|
||||
if tool_choice == "auto":
|
||||
return {"functionCallingConfig": {"mode": "AUTO"}}
|
||||
if tool_choice == "required":
|
||||
return {"functionCallingConfig": {"mode": "ANY"}}
|
||||
if tool_choice == "none":
|
||||
return {"functionCallingConfig": {"mode": "NONE"}}
|
||||
if isinstance(tool_choice, dict):
|
||||
fn = tool_choice.get("function") or {}
|
||||
name = fn.get("name")
|
||||
if name:
|
||||
return {
|
||||
"functionCallingConfig": {
|
||||
"mode": "ANY",
|
||||
"allowedFunctionNames": [str(name)],
|
||||
},
|
||||
}
|
||||
return None
|
||||
|
||||
|
||||
def _normalize_thinking_config(config: Any) -> Optional[Dict[str, Any]]:
|
||||
"""Accept thinkingBudget / thinkingLevel / includeThoughts (+ snake_case)."""
|
||||
if not isinstance(config, dict) or not config:
|
||||
return None
|
||||
budget = config.get("thinkingBudget", config.get("thinking_budget"))
|
||||
level = config.get("thinkingLevel", config.get("thinking_level"))
|
||||
include = config.get("includeThoughts", config.get("include_thoughts"))
|
||||
normalized: Dict[str, Any] = {}
|
||||
if isinstance(budget, (int, float)):
|
||||
normalized["thinkingBudget"] = int(budget)
|
||||
if isinstance(level, str) and level.strip():
|
||||
normalized["thinkingLevel"] = level.strip().lower()
|
||||
if isinstance(include, bool):
|
||||
normalized["includeThoughts"] = include
|
||||
return normalized or None
|
||||
|
||||
|
||||
def build_gemini_request(
|
||||
*,
|
||||
messages: List[Dict[str, Any]],
|
||||
tools: Any = None,
|
||||
tool_choice: Any = None,
|
||||
temperature: Optional[float] = None,
|
||||
max_tokens: Optional[int] = None,
|
||||
top_p: Optional[float] = None,
|
||||
stop: Any = None,
|
||||
thinking_config: Any = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Build the inner Gemini request body (goes inside ``request`` wrapper)."""
|
||||
contents, system_instruction = _build_gemini_contents(messages)
|
||||
|
||||
body: Dict[str, Any] = {"contents": contents}
|
||||
if system_instruction is not None:
|
||||
body["systemInstruction"] = system_instruction
|
||||
|
||||
gemini_tools = _translate_tools_to_gemini(tools)
|
||||
if gemini_tools:
|
||||
body["tools"] = gemini_tools
|
||||
tool_cfg = _translate_tool_choice_to_gemini(tool_choice)
|
||||
if tool_cfg is not None:
|
||||
body["toolConfig"] = tool_cfg
|
||||
|
||||
generation_config: Dict[str, Any] = {}
|
||||
if isinstance(temperature, (int, float)):
|
||||
generation_config["temperature"] = float(temperature)
|
||||
if isinstance(max_tokens, int) and max_tokens > 0:
|
||||
generation_config["maxOutputTokens"] = max_tokens
|
||||
if isinstance(top_p, (int, float)):
|
||||
generation_config["topP"] = float(top_p)
|
||||
if isinstance(stop, str) and stop:
|
||||
generation_config["stopSequences"] = [stop]
|
||||
elif isinstance(stop, list) and stop:
|
||||
generation_config["stopSequences"] = [str(s) for s in stop if s]
|
||||
normalized_thinking = _normalize_thinking_config(thinking_config)
|
||||
if normalized_thinking:
|
||||
generation_config["thinkingConfig"] = normalized_thinking
|
||||
if generation_config:
|
||||
body["generationConfig"] = generation_config
|
||||
|
||||
return body
|
||||
|
||||
|
||||
def wrap_code_assist_request(
|
||||
*,
|
||||
project_id: str,
|
||||
model: str,
|
||||
inner_request: Dict[str, Any],
|
||||
user_prompt_id: Optional[str] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Wrap the inner Gemini request in the Code Assist envelope."""
|
||||
return {
|
||||
"project": project_id,
|
||||
"model": model,
|
||||
"user_prompt_id": user_prompt_id or str(uuid.uuid4()),
|
||||
"request": inner_request,
|
||||
}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Response translation: Gemini → OpenAI
|
||||
# =============================================================================
|
||||
|
||||
def _translate_gemini_response(
|
||||
resp: Dict[str, Any],
|
||||
model: str,
|
||||
) -> SimpleNamespace:
|
||||
"""Non-streaming Gemini response -> OpenAI-shaped SimpleNamespace.
|
||||
|
||||
Code Assist wraps the actual Gemini response inside ``response``, so we
|
||||
unwrap it first if present.
|
||||
"""
|
||||
inner = resp.get("response") if isinstance(resp.get("response"), dict) else resp
|
||||
|
||||
candidates = inner.get("candidates") or []
|
||||
if not isinstance(candidates, list) or not candidates:
|
||||
return _empty_response(model)
|
||||
|
||||
cand = candidates[0]
|
||||
content_obj = cand.get("content") if isinstance(cand, dict) else {}
|
||||
parts = content_obj.get("parts") if isinstance(content_obj, dict) else []
|
||||
|
||||
text_pieces: List[str] = []
|
||||
reasoning_pieces: List[str] = []
|
||||
tool_calls: List[SimpleNamespace] = []
|
||||
|
||||
for i, part in enumerate(parts or []):
|
||||
if not isinstance(part, dict):
|
||||
continue
|
||||
# Thought parts are model's internal reasoning — surface as reasoning,
|
||||
# don't mix into content.
|
||||
if part.get("thought") is True:
|
||||
if isinstance(part.get("text"), str):
|
||||
reasoning_pieces.append(part["text"])
|
||||
continue
|
||||
if isinstance(part.get("text"), str):
|
||||
text_pieces.append(part["text"])
|
||||
continue
|
||||
fc = part.get("functionCall")
|
||||
if isinstance(fc, dict) and fc.get("name"):
|
||||
try:
|
||||
args_str = json.dumps(fc.get("args") or {}, ensure_ascii=False)
|
||||
except (TypeError, ValueError):
|
||||
args_str = "{}"
|
||||
tool_calls.append(SimpleNamespace(
|
||||
id=f"call_{uuid.uuid4().hex[:12]}",
|
||||
type="function",
|
||||
index=i,
|
||||
function=SimpleNamespace(name=str(fc["name"]), arguments=args_str),
|
||||
))
|
||||
|
||||
finish_reason = "tool_calls" if tool_calls else _map_gemini_finish_reason(
|
||||
str(cand.get("finishReason") or "")
|
||||
)
|
||||
|
||||
usage_meta = inner.get("usageMetadata") or {}
|
||||
usage = SimpleNamespace(
|
||||
prompt_tokens=int(usage_meta.get("promptTokenCount") or 0),
|
||||
completion_tokens=int(usage_meta.get("candidatesTokenCount") or 0),
|
||||
total_tokens=int(usage_meta.get("totalTokenCount") or 0),
|
||||
prompt_tokens_details=SimpleNamespace(
|
||||
cached_tokens=int(usage_meta.get("cachedContentTokenCount") or 0),
|
||||
),
|
||||
)
|
||||
|
||||
message = SimpleNamespace(
|
||||
role="assistant",
|
||||
content="".join(text_pieces) if text_pieces else None,
|
||||
tool_calls=tool_calls or None,
|
||||
reasoning="".join(reasoning_pieces) or None,
|
||||
reasoning_content="".join(reasoning_pieces) or None,
|
||||
reasoning_details=None,
|
||||
)
|
||||
choice = SimpleNamespace(
|
||||
index=0,
|
||||
message=message,
|
||||
finish_reason=finish_reason,
|
||||
)
|
||||
return SimpleNamespace(
|
||||
id=f"chatcmpl-{uuid.uuid4().hex[:12]}",
|
||||
object="chat.completion",
|
||||
created=int(time.time()),
|
||||
model=model,
|
||||
choices=[choice],
|
||||
usage=usage,
|
||||
)
|
||||
|
||||
|
||||
def _empty_response(model: str) -> SimpleNamespace:
|
||||
message = SimpleNamespace(
|
||||
role="assistant", content="", tool_calls=None,
|
||||
reasoning=None, reasoning_content=None, reasoning_details=None,
|
||||
)
|
||||
choice = SimpleNamespace(index=0, message=message, finish_reason="stop")
|
||||
usage = SimpleNamespace(
|
||||
prompt_tokens=0, completion_tokens=0, total_tokens=0,
|
||||
prompt_tokens_details=SimpleNamespace(cached_tokens=0),
|
||||
)
|
||||
return SimpleNamespace(
|
||||
id=f"chatcmpl-{uuid.uuid4().hex[:12]}",
|
||||
object="chat.completion",
|
||||
created=int(time.time()),
|
||||
model=model,
|
||||
choices=[choice],
|
||||
usage=usage,
|
||||
)
|
||||
|
||||
|
||||
def _map_gemini_finish_reason(reason: str) -> str:
|
||||
mapping = {
|
||||
"STOP": "stop",
|
||||
"MAX_TOKENS": "length",
|
||||
"SAFETY": "content_filter",
|
||||
"RECITATION": "content_filter",
|
||||
"OTHER": "stop",
|
||||
}
|
||||
return mapping.get(reason.upper(), "stop")
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Streaming SSE iterator
|
||||
# =============================================================================
|
||||
|
||||
class _GeminiStreamChunk(SimpleNamespace):
|
||||
"""Mimics an OpenAI ChatCompletionChunk with .choices[0].delta."""
|
||||
pass
|
||||
|
||||
|
||||
def _make_stream_chunk(
|
||||
*,
|
||||
model: str,
|
||||
content: str = "",
|
||||
tool_call_delta: Optional[Dict[str, Any]] = None,
|
||||
finish_reason: Optional[str] = None,
|
||||
reasoning: str = "",
|
||||
) -> _GeminiStreamChunk:
|
||||
delta_kwargs: Dict[str, Any] = {"role": "assistant"}
|
||||
if content:
|
||||
delta_kwargs["content"] = content
|
||||
if tool_call_delta is not None:
|
||||
delta_kwargs["tool_calls"] = [SimpleNamespace(
|
||||
index=tool_call_delta.get("index", 0),
|
||||
id=tool_call_delta.get("id") or f"call_{uuid.uuid4().hex[:12]}",
|
||||
type="function",
|
||||
function=SimpleNamespace(
|
||||
name=tool_call_delta.get("name") or "",
|
||||
arguments=tool_call_delta.get("arguments") or "",
|
||||
),
|
||||
)]
|
||||
if reasoning:
|
||||
delta_kwargs["reasoning"] = reasoning
|
||||
delta_kwargs["reasoning_content"] = reasoning
|
||||
delta = SimpleNamespace(**delta_kwargs)
|
||||
choice = SimpleNamespace(index=0, delta=delta, finish_reason=finish_reason)
|
||||
return _GeminiStreamChunk(
|
||||
id=f"chatcmpl-{uuid.uuid4().hex[:12]}",
|
||||
object="chat.completion.chunk",
|
||||
created=int(time.time()),
|
||||
model=model,
|
||||
choices=[choice],
|
||||
usage=None,
|
||||
)
|
||||
|
||||
|
||||
def _iter_sse_events(response: httpx.Response) -> Iterator[Dict[str, Any]]:
|
||||
"""Parse Server-Sent Events from an httpx streaming response."""
|
||||
buffer = ""
|
||||
for chunk in response.iter_text():
|
||||
if not chunk:
|
||||
continue
|
||||
buffer += chunk
|
||||
while "\n" in buffer:
|
||||
line, buffer = buffer.split("\n", 1)
|
||||
line = line.rstrip("\r")
|
||||
if not line:
|
||||
continue
|
||||
if line.startswith("data: "):
|
||||
data = line[6:]
|
||||
if data == "[DONE]":
|
||||
return
|
||||
try:
|
||||
yield json.loads(data)
|
||||
except json.JSONDecodeError:
|
||||
logger.debug("Non-JSON SSE line: %s", data[:200])
|
||||
|
||||
|
||||
def _translate_stream_event(
|
||||
event: Dict[str, Any],
|
||||
model: str,
|
||||
tool_call_indices: Dict[str, int],
|
||||
) -> List[_GeminiStreamChunk]:
|
||||
"""Unwrap Code Assist envelope and emit OpenAI-shaped chunk(s)."""
|
||||
inner = event.get("response") if isinstance(event.get("response"), dict) else event
|
||||
candidates = inner.get("candidates") or []
|
||||
if not candidates:
|
||||
return []
|
||||
cand = candidates[0]
|
||||
if not isinstance(cand, dict):
|
||||
return []
|
||||
|
||||
chunks: List[_GeminiStreamChunk] = []
|
||||
|
||||
content = cand.get("content") or {}
|
||||
parts = content.get("parts") if isinstance(content, dict) else []
|
||||
for part in parts or []:
|
||||
if not isinstance(part, dict):
|
||||
continue
|
||||
if part.get("thought") is True and isinstance(part.get("text"), str):
|
||||
chunks.append(_make_stream_chunk(
|
||||
model=model, reasoning=part["text"],
|
||||
))
|
||||
continue
|
||||
if isinstance(part.get("text"), str) and part["text"]:
|
||||
chunks.append(_make_stream_chunk(model=model, content=part["text"]))
|
||||
fc = part.get("functionCall")
|
||||
if isinstance(fc, dict) and fc.get("name"):
|
||||
name = str(fc["name"])
|
||||
idx = tool_call_indices.setdefault(name, len(tool_call_indices))
|
||||
try:
|
||||
args_str = json.dumps(fc.get("args") or {}, ensure_ascii=False)
|
||||
except (TypeError, ValueError):
|
||||
args_str = "{}"
|
||||
chunks.append(_make_stream_chunk(
|
||||
model=model,
|
||||
tool_call_delta={
|
||||
"index": idx,
|
||||
"name": name,
|
||||
"arguments": args_str,
|
||||
},
|
||||
))
|
||||
|
||||
finish_reason_raw = str(cand.get("finishReason") or "")
|
||||
if finish_reason_raw:
|
||||
mapped = _map_gemini_finish_reason(finish_reason_raw)
|
||||
if tool_call_indices:
|
||||
mapped = "tool_calls"
|
||||
chunks.append(_make_stream_chunk(model=model, finish_reason=mapped))
|
||||
return chunks
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# GeminiCloudCodeClient — OpenAI-compatible facade
|
||||
# =============================================================================
|
||||
|
||||
MARKER_BASE_URL = "cloudcode-pa://google"
|
||||
|
||||
|
||||
class _GeminiChatCompletions:
|
||||
def __init__(self, client: "GeminiCloudCodeClient"):
|
||||
self._client = client
|
||||
|
||||
def create(self, **kwargs: Any) -> Any:
|
||||
return self._client._create_chat_completion(**kwargs)
|
||||
|
||||
|
||||
class _GeminiChatNamespace:
|
||||
def __init__(self, client: "GeminiCloudCodeClient"):
|
||||
self.completions = _GeminiChatCompletions(client)
|
||||
|
||||
|
||||
class GeminiCloudCodeClient:
|
||||
"""Minimal OpenAI-SDK-compatible facade over Code Assist v1internal."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
api_key: Optional[str] = None,
|
||||
base_url: Optional[str] = None,
|
||||
default_headers: Optional[Dict[str, str]] = None,
|
||||
project_id: str = "",
|
||||
**_: Any,
|
||||
):
|
||||
# `api_key` here is a dummy — real auth is the OAuth access token
|
||||
# fetched on every call via agent.google_oauth.get_valid_access_token().
|
||||
# We accept the kwarg for openai.OpenAI interface parity.
|
||||
self.api_key = api_key or "google-oauth"
|
||||
self.base_url = base_url or MARKER_BASE_URL
|
||||
self._default_headers = dict(default_headers or {})
|
||||
self._configured_project_id = project_id
|
||||
self._project_context: Optional[ProjectContext] = None
|
||||
self._project_context_lock = False # simple single-thread guard
|
||||
self.chat = _GeminiChatNamespace(self)
|
||||
self.is_closed = False
|
||||
self._http = httpx.Client(timeout=httpx.Timeout(connect=15.0, read=600.0, write=30.0, pool=30.0))
|
||||
|
||||
def close(self) -> None:
|
||||
self.is_closed = True
|
||||
try:
|
||||
self._http.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Implement the OpenAI SDK's context-manager-ish closure check
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
self.close()
|
||||
|
||||
def _ensure_project_context(self, access_token: str, model: str) -> ProjectContext:
|
||||
"""Lazily resolve and cache the project context for this client."""
|
||||
if self._project_context is not None:
|
||||
return self._project_context
|
||||
|
||||
env_project = google_oauth.resolve_project_id_from_env()
|
||||
creds = google_oauth.load_credentials()
|
||||
stored_project = creds.project_id if creds else ""
|
||||
|
||||
# Prefer what's already baked into the creds
|
||||
if stored_project:
|
||||
self._project_context = ProjectContext(
|
||||
project_id=stored_project,
|
||||
managed_project_id=creds.managed_project_id if creds else "",
|
||||
tier_id="",
|
||||
source="stored",
|
||||
)
|
||||
return self._project_context
|
||||
|
||||
ctx = resolve_project_context(
|
||||
access_token,
|
||||
configured_project_id=self._configured_project_id,
|
||||
env_project_id=env_project,
|
||||
user_agent_model=model,
|
||||
)
|
||||
# Persist discovered project back to the creds file so the next
|
||||
# session doesn't re-run the discovery.
|
||||
if ctx.project_id or ctx.managed_project_id:
|
||||
google_oauth.update_project_ids(
|
||||
project_id=ctx.project_id,
|
||||
managed_project_id=ctx.managed_project_id,
|
||||
)
|
||||
self._project_context = ctx
|
||||
return ctx
|
||||
|
||||
def _create_chat_completion(
|
||||
self,
|
||||
*,
|
||||
model: str = "gemini-2.5-flash",
|
||||
messages: Optional[List[Dict[str, Any]]] = None,
|
||||
stream: bool = False,
|
||||
tools: Any = None,
|
||||
tool_choice: Any = None,
|
||||
temperature: Optional[float] = None,
|
||||
max_tokens: Optional[int] = None,
|
||||
top_p: Optional[float] = None,
|
||||
stop: Any = None,
|
||||
extra_body: Optional[Dict[str, Any]] = None,
|
||||
timeout: Any = None,
|
||||
**_: Any,
|
||||
) -> Any:
|
||||
access_token = google_oauth.get_valid_access_token()
|
||||
ctx = self._ensure_project_context(access_token, model)
|
||||
|
||||
thinking_config = None
|
||||
if isinstance(extra_body, dict):
|
||||
thinking_config = extra_body.get("thinking_config") or extra_body.get("thinkingConfig")
|
||||
|
||||
inner = build_gemini_request(
|
||||
messages=messages or [],
|
||||
tools=tools,
|
||||
tool_choice=tool_choice,
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens,
|
||||
top_p=top_p,
|
||||
stop=stop,
|
||||
thinking_config=thinking_config,
|
||||
)
|
||||
wrapped = wrap_code_assist_request(
|
||||
project_id=ctx.project_id,
|
||||
model=model,
|
||||
inner_request=inner,
|
||||
)
|
||||
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json",
|
||||
"Authorization": f"Bearer {access_token}",
|
||||
"User-Agent": "hermes-agent (gemini-cli-compat)",
|
||||
"X-Goog-Api-Client": "gl-python/hermes",
|
||||
"x-activity-request-id": str(uuid.uuid4()),
|
||||
}
|
||||
headers.update(self._default_headers)
|
||||
|
||||
if stream:
|
||||
return self._stream_completion(model=model, wrapped=wrapped, headers=headers)
|
||||
|
||||
url = f"{CODE_ASSIST_ENDPOINT}/v1internal:generateContent"
|
||||
response = self._http.post(url, json=wrapped, headers=headers)
|
||||
if response.status_code != 200:
|
||||
raise _gemini_http_error(response)
|
||||
try:
|
||||
payload = response.json()
|
||||
except ValueError as exc:
|
||||
raise CodeAssistError(
|
||||
f"Invalid JSON from Code Assist: {exc}",
|
||||
code="code_assist_invalid_json",
|
||||
) from exc
|
||||
return _translate_gemini_response(payload, model=model)
|
||||
|
||||
def _stream_completion(
|
||||
self,
|
||||
*,
|
||||
model: str,
|
||||
wrapped: Dict[str, Any],
|
||||
headers: Dict[str, str],
|
||||
) -> Iterator[_GeminiStreamChunk]:
|
||||
"""Generator that yields OpenAI-shaped streaming chunks."""
|
||||
url = f"{CODE_ASSIST_ENDPOINT}/v1internal:streamGenerateContent?alt=sse"
|
||||
stream_headers = dict(headers)
|
||||
stream_headers["Accept"] = "text/event-stream"
|
||||
|
||||
def _generator() -> Iterator[_GeminiStreamChunk]:
|
||||
try:
|
||||
with self._http.stream("POST", url, json=wrapped, headers=stream_headers) as response:
|
||||
if response.status_code != 200:
|
||||
# Materialize error body for better diagnostics
|
||||
response.read()
|
||||
raise _gemini_http_error(response)
|
||||
tool_call_indices: Dict[str, int] = {}
|
||||
for event in _iter_sse_events(response):
|
||||
for chunk in _translate_stream_event(event, model, tool_call_indices):
|
||||
yield chunk
|
||||
except httpx.HTTPError as exc:
|
||||
raise CodeAssistError(
|
||||
f"Streaming request failed: {exc}",
|
||||
code="code_assist_stream_error",
|
||||
) from exc
|
||||
|
||||
return _generator()
|
||||
|
||||
|
||||
def _gemini_http_error(response: httpx.Response) -> CodeAssistError:
|
||||
"""Translate an httpx response into a CodeAssistError with rich metadata.
|
||||
|
||||
Parses Google's error envelope (``{"error": {"code", "message", "status",
|
||||
"details": [...]}}``) so the agent's error classifier can reason about
|
||||
the failure — ``status_code`` enables the rate_limit / auth classification
|
||||
paths, and ``response`` lets the main loop honor ``Retry-After`` just
|
||||
like it does for OpenAI SDK exceptions.
|
||||
|
||||
Also lifts a few recognizable Google conditions into human-readable
|
||||
messages so the user sees something better than a 500-char JSON dump:
|
||||
|
||||
MODEL_CAPACITY_EXHAUSTED → "Gemini model capacity exhausted for
|
||||
<model>. This is a Google-side throttle..."
|
||||
RESOURCE_EXHAUSTED w/o reason → quota-style message
|
||||
404 → "Model <name> not found at cloudcode-pa..."
|
||||
"""
|
||||
status = response.status_code
|
||||
|
||||
# Parse the body once, surviving any weird encodings.
|
||||
body_text = ""
|
||||
body_json: Dict[str, Any] = {}
|
||||
try:
|
||||
body_text = response.text
|
||||
except Exception:
|
||||
body_text = ""
|
||||
if body_text:
|
||||
try:
|
||||
parsed = json.loads(body_text)
|
||||
if isinstance(parsed, dict):
|
||||
body_json = parsed
|
||||
except (ValueError, TypeError):
|
||||
body_json = {}
|
||||
|
||||
# Dig into Google's error envelope. Shape is:
|
||||
# {"error": {"code": 429, "message": "...", "status": "RESOURCE_EXHAUSTED",
|
||||
# "details": [{"@type": ".../ErrorInfo", "reason": "MODEL_CAPACITY_EXHAUSTED",
|
||||
# "metadata": {...}},
|
||||
# {"@type": ".../RetryInfo", "retryDelay": "30s"}]}}
|
||||
err_obj = body_json.get("error") if isinstance(body_json, dict) else None
|
||||
if not isinstance(err_obj, dict):
|
||||
err_obj = {}
|
||||
err_status = str(err_obj.get("status") or "").strip()
|
||||
err_message = str(err_obj.get("message") or "").strip()
|
||||
err_details_list = err_obj.get("details") if isinstance(err_obj.get("details"), list) else []
|
||||
|
||||
# Extract google.rpc.ErrorInfo reason + metadata. There may be more
|
||||
# than one ErrorInfo (rare), so we pick the first one with a reason.
|
||||
error_reason = ""
|
||||
error_metadata: Dict[str, Any] = {}
|
||||
retry_delay_seconds: Optional[float] = None
|
||||
for detail in err_details_list:
|
||||
if not isinstance(detail, dict):
|
||||
continue
|
||||
type_url = str(detail.get("@type") or "")
|
||||
if not error_reason and type_url.endswith("/google.rpc.ErrorInfo"):
|
||||
reason = detail.get("reason")
|
||||
if isinstance(reason, str) and reason:
|
||||
error_reason = reason
|
||||
md = detail.get("metadata")
|
||||
if isinstance(md, dict):
|
||||
error_metadata = md
|
||||
elif retry_delay_seconds is None and type_url.endswith("/google.rpc.RetryInfo"):
|
||||
# retryDelay is a google.protobuf.Duration string like "30s" or "1.5s".
|
||||
delay_raw = detail.get("retryDelay")
|
||||
if isinstance(delay_raw, str) and delay_raw.endswith("s"):
|
||||
try:
|
||||
retry_delay_seconds = float(delay_raw[:-1])
|
||||
except ValueError:
|
||||
pass
|
||||
elif isinstance(delay_raw, (int, float)):
|
||||
retry_delay_seconds = float(delay_raw)
|
||||
|
||||
# Fall back to the Retry-After header if the body didn't include RetryInfo.
|
||||
if retry_delay_seconds is None:
|
||||
try:
|
||||
header_val = response.headers.get("Retry-After") or response.headers.get("retry-after")
|
||||
except Exception:
|
||||
header_val = None
|
||||
if header_val:
|
||||
try:
|
||||
retry_delay_seconds = float(header_val)
|
||||
except (TypeError, ValueError):
|
||||
retry_delay_seconds = None
|
||||
|
||||
# Classify the error code. ``code_assist_rate_limited`` stays the default
|
||||
# for 429s; a more specific reason tag helps downstream callers (e.g. tests,
|
||||
# logs) without changing the rate_limit classification path.
|
||||
code = f"code_assist_http_{status}"
|
||||
if status == 401:
|
||||
code = "code_assist_unauthorized"
|
||||
elif status == 429:
|
||||
code = "code_assist_rate_limited"
|
||||
if error_reason == "MODEL_CAPACITY_EXHAUSTED":
|
||||
code = "code_assist_capacity_exhausted"
|
||||
|
||||
# Build a human-readable message. Keep the status + a raw-body tail for
|
||||
# debugging, but lead with a friendlier summary when we recognize the
|
||||
# Google signal.
|
||||
model_hint = ""
|
||||
if isinstance(error_metadata, dict):
|
||||
model_hint = str(error_metadata.get("model") or error_metadata.get("modelId") or "").strip()
|
||||
|
||||
if status == 429 and error_reason == "MODEL_CAPACITY_EXHAUSTED":
|
||||
target = model_hint or "this Gemini model"
|
||||
message = (
|
||||
f"Gemini capacity exhausted for {target} (Google-side throttle, "
|
||||
f"not a Hermes issue). Try a different Gemini model or set a "
|
||||
f"fallback_providers entry to a non-Gemini provider."
|
||||
)
|
||||
if retry_delay_seconds is not None:
|
||||
message += f" Google suggests retrying in {retry_delay_seconds:g}s."
|
||||
elif status == 429 and err_status == "RESOURCE_EXHAUSTED":
|
||||
message = (
|
||||
f"Gemini quota exhausted ({err_message or 'RESOURCE_EXHAUSTED'}). "
|
||||
f"Check /gquota for remaining daily requests."
|
||||
)
|
||||
if retry_delay_seconds is not None:
|
||||
message += f" Retry suggested in {retry_delay_seconds:g}s."
|
||||
elif status == 404:
|
||||
# Google returns 404 when a model has been retired or renamed.
|
||||
target = model_hint or (err_message or "model")
|
||||
message = (
|
||||
f"Code Assist 404: {target} is not available at "
|
||||
f"cloudcode-pa.googleapis.com. It may have been renamed or "
|
||||
f"retired. Check hermes_cli/models.py for the current list."
|
||||
)
|
||||
elif err_message:
|
||||
# Generic fallback with the parsed message.
|
||||
message = f"Code Assist HTTP {status} ({err_status or 'error'}): {err_message}"
|
||||
else:
|
||||
# Last-ditch fallback — raw body snippet.
|
||||
message = f"Code Assist returned HTTP {status}: {body_text[:500]}"
|
||||
|
||||
return CodeAssistError(
|
||||
message,
|
||||
code=code,
|
||||
status_code=status,
|
||||
response=response,
|
||||
retry_after=retry_delay_seconds,
|
||||
details={
|
||||
"status": err_status,
|
||||
"reason": error_reason,
|
||||
"metadata": error_metadata,
|
||||
"message": err_message,
|
||||
},
|
||||
)
|
||||
@@ -1,453 +0,0 @@
|
||||
"""Google Code Assist API client — project discovery, onboarding, quota.
|
||||
|
||||
The Code Assist API powers Google's official gemini-cli. It sits at
|
||||
``cloudcode-pa.googleapis.com`` and provides:
|
||||
|
||||
- Free tier access (generous daily quota) for personal Google accounts
|
||||
- Paid tier access via GCP projects with billing / Workspace / Standard / Enterprise
|
||||
|
||||
This module handles the control-plane dance needed before inference:
|
||||
|
||||
1. ``load_code_assist()`` — probe the user's account to learn what tier they're on
|
||||
and whether a ``cloudaicompanionProject`` is already assigned.
|
||||
2. ``onboard_user()`` — if the user hasn't been onboarded yet (new account, fresh
|
||||
free tier, etc.), call this with the chosen tier + project id. Supports LRO
|
||||
polling for slow provisioning.
|
||||
3. ``retrieve_user_quota()`` — fetch the ``buckets[]`` array showing remaining
|
||||
quota per model, used by the ``/gquota`` slash command.
|
||||
|
||||
VPC-SC handling: enterprise accounts under a VPC Service Controls perimeter
|
||||
will get ``SECURITY_POLICY_VIOLATED`` on ``load_code_assist``. We catch this
|
||||
and force the account to ``standard-tier`` so the call chain still succeeds.
|
||||
|
||||
Derived from opencode-gemini-auth (MIT) and clawdbot/extensions/google. The
|
||||
request/response shapes are specific to Google's internal Code Assist API,
|
||||
documented nowhere public — we copy them from the reference implementations.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Constants
|
||||
# =============================================================================
|
||||
|
||||
CODE_ASSIST_ENDPOINT = "https://cloudcode-pa.googleapis.com"
|
||||
|
||||
# Fallback endpoints tried when prod returns an error during project discovery
|
||||
FALLBACK_ENDPOINTS = [
|
||||
"https://daily-cloudcode-pa.sandbox.googleapis.com",
|
||||
"https://autopush-cloudcode-pa.sandbox.googleapis.com",
|
||||
]
|
||||
|
||||
# Tier identifiers that Google's API uses
|
||||
FREE_TIER_ID = "free-tier"
|
||||
LEGACY_TIER_ID = "legacy-tier"
|
||||
STANDARD_TIER_ID = "standard-tier"
|
||||
|
||||
# Default HTTP headers matching gemini-cli's fingerprint.
|
||||
# Google may reject unrecognized User-Agents on these internal endpoints.
|
||||
_GEMINI_CLI_USER_AGENT = "google-api-nodejs-client/9.15.1 (gzip)"
|
||||
_X_GOOG_API_CLIENT = "gl-node/24.0.0"
|
||||
_DEFAULT_REQUEST_TIMEOUT = 30.0
|
||||
_ONBOARDING_POLL_ATTEMPTS = 12
|
||||
_ONBOARDING_POLL_INTERVAL_SECONDS = 5.0
|
||||
|
||||
|
||||
class CodeAssistError(RuntimeError):
|
||||
"""Exception raised by the Code Assist (``cloudcode-pa``) integration.
|
||||
|
||||
Carries HTTP status / response / retry-after metadata so the agent's
|
||||
``error_classifier._extract_status_code`` and the main loop's Retry-After
|
||||
handling (which walks ``error.response.headers``) pick up the right
|
||||
signals. Without these, 429s from the OAuth path look like opaque
|
||||
``RuntimeError`` and skip the rate-limit path.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
*,
|
||||
code: str = "code_assist_error",
|
||||
status_code: Optional[int] = None,
|
||||
response: Any = None,
|
||||
retry_after: Optional[float] = None,
|
||||
details: Optional[Dict[str, Any]] = None,
|
||||
) -> None:
|
||||
super().__init__(message)
|
||||
self.code = code
|
||||
# ``status_code`` is picked up by ``agent.error_classifier._extract_status_code``
|
||||
# so a 429 from Code Assist classifies as FailoverReason.rate_limit and
|
||||
# triggers the main loop's fallback_providers chain the same way SDK
|
||||
# errors do.
|
||||
self.status_code = status_code
|
||||
# ``response`` is the underlying ``httpx.Response`` (or a shim with a
|
||||
# ``.headers`` mapping and ``.json()`` method). The main loop reads
|
||||
# ``error.response.headers["Retry-After"]`` to honor Google's retry
|
||||
# hints when the backend throttles us.
|
||||
self.response = response
|
||||
# Parsed ``Retry-After`` seconds (kept separately for convenience —
|
||||
# Google returns retry hints in both the header and the error body's
|
||||
# ``google.rpc.RetryInfo`` details, and we pick whichever we found).
|
||||
self.retry_after = retry_after
|
||||
# Parsed structured error details from the Google error envelope
|
||||
# (e.g. ``{"reason": "MODEL_CAPACITY_EXHAUSTED", "status": "RESOURCE_EXHAUSTED"}``).
|
||||
# Useful for logging and for tests that want to assert on specifics.
|
||||
self.details = details or {}
|
||||
|
||||
|
||||
class ProjectIdRequiredError(CodeAssistError):
|
||||
def __init__(self, message: str = "GCP project id required for this tier") -> None:
|
||||
super().__init__(message, code="code_assist_project_id_required")
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# HTTP primitive (auth via Bearer token passed per-call)
|
||||
# =============================================================================
|
||||
|
||||
def _build_headers(access_token: str, *, user_agent_model: str = "") -> Dict[str, str]:
|
||||
ua = _GEMINI_CLI_USER_AGENT
|
||||
if user_agent_model:
|
||||
ua = f"{ua} model/{user_agent_model}"
|
||||
return {
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json",
|
||||
"Authorization": f"Bearer {access_token}",
|
||||
"User-Agent": ua,
|
||||
"X-Goog-Api-Client": _X_GOOG_API_CLIENT,
|
||||
"x-activity-request-id": str(uuid.uuid4()),
|
||||
}
|
||||
|
||||
|
||||
def _client_metadata() -> Dict[str, str]:
|
||||
"""Match Google's gemini-cli exactly — unrecognized metadata may be rejected."""
|
||||
return {
|
||||
"ideType": "IDE_UNSPECIFIED",
|
||||
"platform": "PLATFORM_UNSPECIFIED",
|
||||
"pluginType": "GEMINI",
|
||||
}
|
||||
|
||||
|
||||
def _post_json(
|
||||
url: str,
|
||||
body: Dict[str, Any],
|
||||
access_token: str,
|
||||
*,
|
||||
timeout: float = _DEFAULT_REQUEST_TIMEOUT,
|
||||
user_agent_model: str = "",
|
||||
) -> Dict[str, Any]:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
request = urllib.request.Request(
|
||||
url, data=data, method="POST",
|
||||
headers=_build_headers(access_token, user_agent_model=user_agent_model),
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||
raw = response.read().decode("utf-8", errors="replace")
|
||||
return json.loads(raw) if raw else {}
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = ""
|
||||
try:
|
||||
detail = exc.read().decode("utf-8", errors="replace")
|
||||
except Exception:
|
||||
pass
|
||||
# Special case: VPC-SC violation should be distinguishable
|
||||
if _is_vpc_sc_violation(detail):
|
||||
raise CodeAssistError(
|
||||
f"VPC-SC policy violation: {detail}",
|
||||
code="code_assist_vpc_sc",
|
||||
) from exc
|
||||
raise CodeAssistError(
|
||||
f"Code Assist HTTP {exc.code}: {detail or exc.reason}",
|
||||
code=f"code_assist_http_{exc.code}",
|
||||
) from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise CodeAssistError(
|
||||
f"Code Assist request failed: {exc}",
|
||||
code="code_assist_network_error",
|
||||
) from exc
|
||||
|
||||
|
||||
def _is_vpc_sc_violation(body: str) -> bool:
|
||||
"""Detect a VPC Service Controls violation from a response body."""
|
||||
if not body:
|
||||
return False
|
||||
try:
|
||||
parsed = json.loads(body)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return "SECURITY_POLICY_VIOLATED" in body
|
||||
# Walk the nested error structure Google uses
|
||||
error = parsed.get("error") if isinstance(parsed, dict) else None
|
||||
if not isinstance(error, dict):
|
||||
return False
|
||||
details = error.get("details") or []
|
||||
if isinstance(details, list):
|
||||
for item in details:
|
||||
if isinstance(item, dict):
|
||||
reason = item.get("reason") or ""
|
||||
if reason == "SECURITY_POLICY_VIOLATED":
|
||||
return True
|
||||
msg = str(error.get("message", ""))
|
||||
return "SECURITY_POLICY_VIOLATED" in msg
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# load_code_assist — discovers current tier + assigned project
|
||||
# =============================================================================
|
||||
|
||||
@dataclass
|
||||
class CodeAssistProjectInfo:
|
||||
"""Result from ``load_code_assist``."""
|
||||
current_tier_id: str = ""
|
||||
cloudaicompanion_project: str = "" # Google-managed project (free tier)
|
||||
allowed_tiers: List[str] = field(default_factory=list)
|
||||
raw: Dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
def load_code_assist(
|
||||
access_token: str,
|
||||
*,
|
||||
project_id: str = "",
|
||||
user_agent_model: str = "",
|
||||
) -> CodeAssistProjectInfo:
|
||||
"""Call ``POST /v1internal:loadCodeAssist`` with prod → sandbox fallback.
|
||||
|
||||
Returns whatever tier + project info Google reports. On VPC-SC violations,
|
||||
returns a synthetic ``standard-tier`` result so the chain can continue.
|
||||
"""
|
||||
body: Dict[str, Any] = {
|
||||
"metadata": {
|
||||
"duetProject": project_id,
|
||||
**_client_metadata(),
|
||||
},
|
||||
}
|
||||
if project_id:
|
||||
body["cloudaicompanionProject"] = project_id
|
||||
|
||||
endpoints = [CODE_ASSIST_ENDPOINT] + FALLBACK_ENDPOINTS
|
||||
last_err: Optional[Exception] = None
|
||||
for endpoint in endpoints:
|
||||
url = f"{endpoint}/v1internal:loadCodeAssist"
|
||||
try:
|
||||
resp = _post_json(url, body, access_token, user_agent_model=user_agent_model)
|
||||
return _parse_load_response(resp)
|
||||
except CodeAssistError as exc:
|
||||
if exc.code == "code_assist_vpc_sc":
|
||||
logger.info("VPC-SC violation on %s — defaulting to standard-tier", endpoint)
|
||||
return CodeAssistProjectInfo(
|
||||
current_tier_id=STANDARD_TIER_ID,
|
||||
cloudaicompanion_project=project_id,
|
||||
)
|
||||
last_err = exc
|
||||
logger.warning("loadCodeAssist failed on %s: %s", endpoint, exc)
|
||||
continue
|
||||
if last_err:
|
||||
raise last_err
|
||||
return CodeAssistProjectInfo()
|
||||
|
||||
|
||||
def _parse_load_response(resp: Dict[str, Any]) -> CodeAssistProjectInfo:
|
||||
current_tier = resp.get("currentTier") or {}
|
||||
tier_id = str(current_tier.get("id") or "") if isinstance(current_tier, dict) else ""
|
||||
project = str(resp.get("cloudaicompanionProject") or "")
|
||||
allowed = resp.get("allowedTiers") or []
|
||||
allowed_ids: List[str] = []
|
||||
if isinstance(allowed, list):
|
||||
for t in allowed:
|
||||
if isinstance(t, dict):
|
||||
tid = str(t.get("id") or "")
|
||||
if tid:
|
||||
allowed_ids.append(tid)
|
||||
return CodeAssistProjectInfo(
|
||||
current_tier_id=tier_id,
|
||||
cloudaicompanion_project=project,
|
||||
allowed_tiers=allowed_ids,
|
||||
raw=resp,
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# onboard_user — provisions a new user on a tier (with LRO polling)
|
||||
# =============================================================================
|
||||
|
||||
def onboard_user(
|
||||
access_token: str,
|
||||
*,
|
||||
tier_id: str,
|
||||
project_id: str = "",
|
||||
user_agent_model: str = "",
|
||||
) -> Dict[str, Any]:
|
||||
"""Call ``POST /v1internal:onboardUser`` to provision the user.
|
||||
|
||||
For paid tiers, ``project_id`` is REQUIRED (raises ProjectIdRequiredError).
|
||||
For free tiers, ``project_id`` is optional — Google will assign one.
|
||||
|
||||
Returns the final operation response. Polls ``/v1internal/<name>`` for up
|
||||
to ``_ONBOARDING_POLL_ATTEMPTS`` × ``_ONBOARDING_POLL_INTERVAL_SECONDS``
|
||||
(default: 12 × 5s = 1 min).
|
||||
"""
|
||||
if tier_id != FREE_TIER_ID and tier_id != LEGACY_TIER_ID and not project_id:
|
||||
raise ProjectIdRequiredError(
|
||||
f"Tier {tier_id!r} requires a GCP project id. "
|
||||
"Set HERMES_GEMINI_PROJECT_ID or GOOGLE_CLOUD_PROJECT."
|
||||
)
|
||||
|
||||
body: Dict[str, Any] = {
|
||||
"tierId": tier_id,
|
||||
"metadata": _client_metadata(),
|
||||
}
|
||||
if project_id:
|
||||
body["cloudaicompanionProject"] = project_id
|
||||
|
||||
endpoint = CODE_ASSIST_ENDPOINT
|
||||
url = f"{endpoint}/v1internal:onboardUser"
|
||||
resp = _post_json(url, body, access_token, user_agent_model=user_agent_model)
|
||||
|
||||
# Poll if LRO (long-running operation)
|
||||
if not resp.get("done"):
|
||||
op_name = resp.get("name", "")
|
||||
if not op_name:
|
||||
return resp
|
||||
for attempt in range(_ONBOARDING_POLL_ATTEMPTS):
|
||||
time.sleep(_ONBOARDING_POLL_INTERVAL_SECONDS)
|
||||
poll_url = f"{endpoint}/v1internal/{op_name}"
|
||||
try:
|
||||
poll_resp = _post_json(poll_url, {}, access_token, user_agent_model=user_agent_model)
|
||||
except CodeAssistError as exc:
|
||||
logger.warning("Onboarding poll attempt %d failed: %s", attempt + 1, exc)
|
||||
continue
|
||||
if poll_resp.get("done"):
|
||||
return poll_resp
|
||||
logger.warning("Onboarding did not complete within %d attempts", _ONBOARDING_POLL_ATTEMPTS)
|
||||
return resp
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# retrieve_user_quota — for /gquota
|
||||
# =============================================================================
|
||||
|
||||
@dataclass
|
||||
class QuotaBucket:
|
||||
model_id: str
|
||||
token_type: str = ""
|
||||
remaining_fraction: float = 0.0
|
||||
reset_time_iso: str = ""
|
||||
raw: Dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
def retrieve_user_quota(
|
||||
access_token: str,
|
||||
*,
|
||||
project_id: str = "",
|
||||
user_agent_model: str = "",
|
||||
) -> List[QuotaBucket]:
|
||||
"""Call ``POST /v1internal:retrieveUserQuota`` and parse ``buckets[]``."""
|
||||
body: Dict[str, Any] = {}
|
||||
if project_id:
|
||||
body["project"] = project_id
|
||||
url = f"{CODE_ASSIST_ENDPOINT}/v1internal:retrieveUserQuota"
|
||||
resp = _post_json(url, body, access_token, user_agent_model=user_agent_model)
|
||||
raw_buckets = resp.get("buckets") or []
|
||||
buckets: List[QuotaBucket] = []
|
||||
if not isinstance(raw_buckets, list):
|
||||
return buckets
|
||||
for b in raw_buckets:
|
||||
if not isinstance(b, dict):
|
||||
continue
|
||||
buckets.append(QuotaBucket(
|
||||
model_id=str(b.get("modelId") or ""),
|
||||
token_type=str(b.get("tokenType") or ""),
|
||||
remaining_fraction=float(b.get("remainingFraction") or 0.0),
|
||||
reset_time_iso=str(b.get("resetTime") or ""),
|
||||
raw=b,
|
||||
))
|
||||
return buckets
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Project context resolution
|
||||
# =============================================================================
|
||||
|
||||
@dataclass
|
||||
class ProjectContext:
|
||||
"""Resolved state for a given OAuth session."""
|
||||
project_id: str = "" # effective project id sent on requests
|
||||
managed_project_id: str = "" # Google-assigned project (free tier)
|
||||
tier_id: str = ""
|
||||
source: str = "" # "env", "config", "discovered", "onboarded"
|
||||
|
||||
|
||||
def resolve_project_context(
|
||||
access_token: str,
|
||||
*,
|
||||
configured_project_id: str = "",
|
||||
env_project_id: str = "",
|
||||
user_agent_model: str = "",
|
||||
) -> ProjectContext:
|
||||
"""Figure out what project id + tier to use for requests.
|
||||
|
||||
Priority:
|
||||
1. If configured_project_id or env_project_id is set, use that directly
|
||||
and short-circuit (no discovery needed).
|
||||
2. Otherwise call loadCodeAssist to see what Google says.
|
||||
3. If no tier assigned yet, onboard the user (free tier default).
|
||||
"""
|
||||
# Short-circuit: caller provided a project id
|
||||
if configured_project_id:
|
||||
return ProjectContext(
|
||||
project_id=configured_project_id,
|
||||
tier_id=STANDARD_TIER_ID, # assume paid since they specified one
|
||||
source="config",
|
||||
)
|
||||
if env_project_id:
|
||||
return ProjectContext(
|
||||
project_id=env_project_id,
|
||||
tier_id=STANDARD_TIER_ID,
|
||||
source="env",
|
||||
)
|
||||
|
||||
# Discover via loadCodeAssist
|
||||
info = load_code_assist(access_token, user_agent_model=user_agent_model)
|
||||
|
||||
effective_project = info.cloudaicompanion_project
|
||||
tier = info.current_tier_id
|
||||
|
||||
if not tier:
|
||||
# User hasn't been onboarded — provision them on free tier
|
||||
onboard_resp = onboard_user(
|
||||
access_token,
|
||||
tier_id=FREE_TIER_ID,
|
||||
project_id="",
|
||||
user_agent_model=user_agent_model,
|
||||
)
|
||||
# Re-parse from the onboard response
|
||||
response_body = onboard_resp.get("response") or {}
|
||||
if isinstance(response_body, dict):
|
||||
effective_project = (
|
||||
effective_project
|
||||
or str(response_body.get("cloudaicompanionProject") or "")
|
||||
)
|
||||
tier = FREE_TIER_ID
|
||||
source = "onboarded"
|
||||
else:
|
||||
source = "discovered"
|
||||
|
||||
return ProjectContext(
|
||||
project_id=effective_project,
|
||||
managed_project_id=effective_project if tier == FREE_TIER_ID else "",
|
||||
tier_id=tier,
|
||||
source=source,
|
||||
)
|
||||
File diff suppressed because it is too large
Load Diff
+27
-5
@@ -27,6 +27,7 @@ from agent.usage_pricing import (
|
||||
DEFAULT_PRICING,
|
||||
estimate_usage_cost,
|
||||
format_duration_compact,
|
||||
get_pricing,
|
||||
has_known_pricing,
|
||||
)
|
||||
|
||||
@@ -634,7 +635,13 @@ class InsightsEngine:
|
||||
lines.append(f" Sessions: {o['total_sessions']:<12} Messages: {o['total_messages']:,}")
|
||||
lines.append(f" Tool calls: {o['total_tool_calls']:<12,} User messages: {o['user_messages']:,}")
|
||||
lines.append(f" Input tokens: {o['total_input_tokens']:<12,} Output tokens: {o['total_output_tokens']:,}")
|
||||
lines.append(f" Total tokens: {o['total_tokens']:,}")
|
||||
cache_total = o.get("total_cache_read_tokens", 0) + o.get("total_cache_write_tokens", 0)
|
||||
if cache_total > 0:
|
||||
lines.append(f" Cache read: {o['total_cache_read_tokens']:<12,} Cache write: {o['total_cache_write_tokens']:,}")
|
||||
cost_str = f"${o['estimated_cost']:.2f}"
|
||||
if o.get("models_without_pricing"):
|
||||
cost_str += " *"
|
||||
lines.append(f" Total tokens: {o['total_tokens']:<12,} Est. cost: {cost_str}")
|
||||
if o["total_hours"] > 0:
|
||||
lines.append(f" Active time: ~{_format_duration(o['total_hours'] * 3600):<11} Avg session: ~{_format_duration(o['avg_session_duration'])}")
|
||||
lines.append(f" Avg msgs/session: {o['avg_messages_per_session']:.1f}")
|
||||
@@ -644,10 +651,16 @@ class InsightsEngine:
|
||||
if report["models"]:
|
||||
lines.append(" 🤖 Models Used")
|
||||
lines.append(" " + "─" * 56)
|
||||
lines.append(f" {'Model':<30} {'Sessions':>8} {'Tokens':>12}")
|
||||
lines.append(f" {'Model':<30} {'Sessions':>8} {'Tokens':>12} {'Cost':>8}")
|
||||
for m in report["models"]:
|
||||
model_name = m["model"][:28]
|
||||
lines.append(f" {model_name:<30} {m['sessions']:>8} {m['total_tokens']:>12,}")
|
||||
if m.get("has_pricing"):
|
||||
cost_cell = f"${m['cost']:>6.2f}"
|
||||
else:
|
||||
cost_cell = " N/A"
|
||||
lines.append(f" {model_name:<30} {m['sessions']:>8} {m['total_tokens']:>12,} {cost_cell}")
|
||||
if o.get("models_without_pricing"):
|
||||
lines.append(" * Cost N/A for custom/self-hosted models")
|
||||
lines.append("")
|
||||
|
||||
# Platform breakdown
|
||||
@@ -727,7 +740,15 @@ class InsightsEngine:
|
||||
|
||||
# Overview
|
||||
lines.append(f"**Sessions:** {o['total_sessions']} | **Messages:** {o['total_messages']:,} | **Tool calls:** {o['total_tool_calls']:,}")
|
||||
lines.append(f"**Tokens:** {o['total_tokens']:,} (in: {o['total_input_tokens']:,} / out: {o['total_output_tokens']:,})")
|
||||
cache_total = o.get("total_cache_read_tokens", 0) + o.get("total_cache_write_tokens", 0)
|
||||
if cache_total > 0:
|
||||
lines.append(f"**Tokens:** {o['total_tokens']:,} (in: {o['total_input_tokens']:,} / out: {o['total_output_tokens']:,} / cache: {cache_total:,})")
|
||||
else:
|
||||
lines.append(f"**Tokens:** {o['total_tokens']:,} (in: {o['total_input_tokens']:,} / out: {o['total_output_tokens']:,})")
|
||||
cost_note = ""
|
||||
if o.get("models_without_pricing"):
|
||||
cost_note = " _(excludes custom/self-hosted models)_"
|
||||
lines.append(f"**Est. cost:** ${o['estimated_cost']:.2f}{cost_note}")
|
||||
if o["total_hours"] > 0:
|
||||
lines.append(f"**Active time:** ~{_format_duration(o['total_hours'] * 3600)} | **Avg session:** ~{_format_duration(o['avg_session_duration'])}")
|
||||
lines.append("")
|
||||
@@ -736,7 +757,8 @@ class InsightsEngine:
|
||||
if report["models"]:
|
||||
lines.append("**🤖 Models:**")
|
||||
for m in report["models"][:5]:
|
||||
lines.append(f" {m['model'][:25]} — {m['sessions']} sessions, {m['total_tokens']:,} tokens")
|
||||
cost_str = f"${m['cost']:.2f}" if m.get("has_pricing") else "N/A"
|
||||
lines.append(f" {m['model'][:25]} — {m['sessions']} sessions, {m['total_tokens']:,} tokens, {cost_str}")
|
||||
lines.append("")
|
||||
|
||||
# Platforms (if multi-platform)
|
||||
|
||||
+2
-13
@@ -44,22 +44,11 @@ logger = logging.getLogger(__name__)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_FENCE_TAG_RE = re.compile(r'</?\s*memory-context\s*>', re.IGNORECASE)
|
||||
_INTERNAL_CONTEXT_RE = re.compile(
|
||||
r'<\s*memory-context\s*>[\s\S]*?</\s*memory-context\s*>',
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_INTERNAL_NOTE_RE = re.compile(
|
||||
r'\[System note:\s*The following is recalled memory context,\s*NOT new user input\.\s*Treat as informational background data\.\]\s*',
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
|
||||
def sanitize_context(text: str) -> str:
|
||||
"""Strip fence tags, injected context blocks, and system notes from provider output."""
|
||||
text = _INTERNAL_CONTEXT_RE.sub('', text)
|
||||
text = _INTERNAL_NOTE_RE.sub('', text)
|
||||
text = _FENCE_TAG_RE.sub('', text)
|
||||
return text
|
||||
"""Strip fence-escape sequences from provider output."""
|
||||
return _FENCE_TAG_RE.sub('', text)
|
||||
|
||||
|
||||
def build_memory_context_block(raw_context: str) -> str:
|
||||
|
||||
+12
-46
@@ -5,6 +5,7 @@ and run_agent.py for pre-flight context checks.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
@@ -23,22 +24,17 @@ logger = logging.getLogger(__name__)
|
||||
# are preserved so the full model name reaches cache lookups and server queries.
|
||||
_PROVIDER_PREFIXES: frozenset[str] = frozenset({
|
||||
"openrouter", "nous", "openai-codex", "copilot", "copilot-acp",
|
||||
"gemini", "ollama-cloud", "zai", "kimi-coding", "kimi-coding-cn", "minimax", "minimax-cn", "anthropic", "deepseek",
|
||||
"gemini", "zai", "kimi-coding", "minimax", "minimax-cn", "anthropic", "deepseek",
|
||||
"opencode-zen", "opencode-go", "ai-gateway", "kilocode", "alibaba",
|
||||
"qwen-oauth",
|
||||
"xiaomi",
|
||||
"arcee",
|
||||
"custom", "local",
|
||||
# Common aliases
|
||||
"google", "google-gemini", "google-ai-studio",
|
||||
"glm", "z-ai", "z.ai", "zhipu", "github", "github-copilot",
|
||||
"github-models", "kimi", "moonshot", "kimi-cn", "moonshot-cn", "claude", "deep-seek",
|
||||
"ollama",
|
||||
"github-models", "kimi", "moonshot", "claude", "deep-seek",
|
||||
"opencode", "zen", "go", "vercel", "kilo", "dashscope", "aliyun", "qwen",
|
||||
"mimo", "xiaomi-mimo",
|
||||
"arcee-ai", "arceeai",
|
||||
"xai", "x-ai", "x.ai", "grok",
|
||||
"nvidia", "nim", "nvidia-nim", "nemotron",
|
||||
"qwen-portal",
|
||||
})
|
||||
|
||||
@@ -103,28 +99,21 @@ DEFAULT_CONTEXT_LENGTHS = {
|
||||
# fuzzy-match collisions (e.g. "anthropic/claude-sonnet-4" is a
|
||||
# substring of "anthropic/claude-sonnet-4.6").
|
||||
# OpenRouter-prefixed models resolve via OpenRouter live API or models.dev.
|
||||
"claude-opus-4-7": 1000000,
|
||||
"claude-opus-4.7": 1000000,
|
||||
"claude-opus-4-6": 1000000,
|
||||
"claude-sonnet-4-6": 1000000,
|
||||
"claude-opus-4.6": 1000000,
|
||||
"claude-sonnet-4.6": 1000000,
|
||||
# Catch-all for older Claude models (must sort after specific entries)
|
||||
"claude": 200000,
|
||||
# OpenAI — GPT-5 family (most have 400k; specific overrides first)
|
||||
# Source: https://developers.openai.com/api/docs/models
|
||||
"gpt-5.4-nano": 400000, # 400k (not 1.05M like full 5.4)
|
||||
"gpt-5.4-mini": 400000, # 400k (not 1.05M like full 5.4)
|
||||
"gpt-5.4": 1050000, # GPT-5.4, GPT-5.4 Pro (1.05M context)
|
||||
"gpt-5.3-codex-spark": 128000, # Spark variant has reduced 128k context
|
||||
"gpt-5.1-chat": 128000, # Chat variant has 128k context
|
||||
"gpt-5": 400000, # GPT-5.x base, mini, codex variants (400k)
|
||||
# OpenAI
|
||||
"gpt-4.1": 1047576,
|
||||
"gpt-5": 128000,
|
||||
"gpt-4": 128000,
|
||||
# Google
|
||||
"gemini": 1048576,
|
||||
# Gemma (open models served via AI Studio)
|
||||
"gemma-4-31b": 256000,
|
||||
"gemma-4-26b": 256000,
|
||||
"gemma-3": 131072,
|
||||
"gemma": 8192, # fallback for older gemma models
|
||||
# DeepSeek
|
||||
@@ -158,12 +147,8 @@ DEFAULT_CONTEXT_LENGTHS = {
|
||||
"grok": 131072, # catch-all (grok-beta, unknown grok-*)
|
||||
# Kimi
|
||||
"kimi": 262144,
|
||||
# Nemotron — NVIDIA's open-weights series (128K context across all sizes)
|
||||
"nemotron": 131072,
|
||||
# Arcee
|
||||
"trinity": 262144,
|
||||
# OpenRouter
|
||||
"elephant": 262144,
|
||||
# Hugging Face Inference Providers — model IDs use org/name format
|
||||
"Qwen/Qwen3.5-397B-A17B": 131072,
|
||||
"Qwen/Qwen3.5-35B-A3B": 131072,
|
||||
@@ -226,9 +211,7 @@ _URL_TO_PROVIDER: Dict[str, str] = {
|
||||
"api.anthropic.com": "anthropic",
|
||||
"api.z.ai": "zai",
|
||||
"api.moonshot.ai": "kimi-coding",
|
||||
"api.moonshot.cn": "kimi-coding-cn",
|
||||
"api.kimi.com": "kimi-coding",
|
||||
"api.arcee.ai": "arcee",
|
||||
"api.minimax": "minimax",
|
||||
"dashscope.aliyuncs.com": "alibaba",
|
||||
"dashscope-intl.aliyuncs.com": "alibaba",
|
||||
@@ -242,10 +225,8 @@ _URL_TO_PROVIDER: Dict[str, str] = {
|
||||
"api.fireworks.ai": "fireworks",
|
||||
"opencode.ai": "opencode-go",
|
||||
"api.x.ai": "xai",
|
||||
"integrate.api.nvidia.com": "nvidia",
|
||||
"api.xiaomimimo.com": "xiaomi",
|
||||
"xiaomimimo.com": "xiaomi",
|
||||
"ollama.com": "ollama-cloud",
|
||||
}
|
||||
|
||||
|
||||
@@ -794,12 +775,12 @@ def _query_local_context_length(model: str, base_url: str) -> Optional[int]:
|
||||
resp = client.post(f"{server_url}/api/show", json={"name": model})
|
||||
if resp.status_code == 200:
|
||||
data = resp.json()
|
||||
# Prefer explicit num_ctx from Modelfile parameters: this is
|
||||
# the *runtime* context Ollama will actually allocate KV cache
|
||||
# for. The GGUF model_info.context_length is the training max,
|
||||
# which can be larger than num_ctx — using it here would let
|
||||
# Hermes grow conversations past the runtime limit and Ollama
|
||||
# would silently truncate. Matches query_ollama_num_ctx().
|
||||
# Check model_info for context length
|
||||
model_info = data.get("model_info", {})
|
||||
for key, value in model_info.items():
|
||||
if "context_length" in key and isinstance(value, (int, float)):
|
||||
return int(value)
|
||||
# Check parameters string for num_ctx
|
||||
params = data.get("parameters", "")
|
||||
if "num_ctx" in params:
|
||||
for line in params.split("\n"):
|
||||
@@ -810,11 +791,6 @@ def _query_local_context_length(model: str, base_url: str) -> Optional[int]:
|
||||
return int(parts[-1])
|
||||
except ValueError:
|
||||
pass
|
||||
# Fall back to GGUF model_info context_length (training max)
|
||||
model_info = data.get("model_info", {})
|
||||
for key, value in model_info.items():
|
||||
if "context_length" in key and isinstance(value, (int, float)):
|
||||
return int(value)
|
||||
|
||||
# LM Studio native API: /api/v1/models returns max_context_length.
|
||||
# This is more reliable than the OpenAI-compat /v1/models which
|
||||
@@ -1019,16 +995,6 @@ def get_model_context_length(
|
||||
if ctx:
|
||||
return ctx
|
||||
|
||||
# 4b. AWS Bedrock — use static context length table.
|
||||
# Bedrock's ListFoundationModels doesn't expose context window sizes,
|
||||
# so we maintain a curated table in bedrock_adapter.py.
|
||||
if provider == "bedrock" or (base_url and "bedrock-runtime" in base_url):
|
||||
try:
|
||||
from agent.bedrock_adapter import get_bedrock_context_length
|
||||
return get_bedrock_context_length(model)
|
||||
except ImportError:
|
||||
pass # boto3 not installed — fall through to generic resolution
|
||||
|
||||
# 5. Provider-aware lookups (before generic OpenRouter cache)
|
||||
# These are provider-specific and take priority over the generic OR cache,
|
||||
# since the same model can have different context limits per provider
|
||||
|
||||
+96
-4
@@ -18,8 +18,10 @@ Other modules should import the dataclasses and query functions from here
|
||||
rather than parsing the raw JSON themselves.
|
||||
"""
|
||||
|
||||
import difflib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
@@ -142,11 +144,8 @@ class ProviderInfo:
|
||||
PROVIDER_TO_MODELS_DEV: Dict[str, str] = {
|
||||
"openrouter": "openrouter",
|
||||
"anthropic": "anthropic",
|
||||
"openai": "openai",
|
||||
"openai-codex": "openai",
|
||||
"zai": "zai",
|
||||
"kimi-coding": "kimi-for-coding",
|
||||
"kimi-coding-cn": "kimi-for-coding",
|
||||
"minimax": "minimax",
|
||||
"minimax-cn": "minimax-cn",
|
||||
"deepseek": "deepseek",
|
||||
@@ -169,13 +168,19 @@ PROVIDER_TO_MODELS_DEV: Dict[str, str] = {
|
||||
"togetherai": "togetherai",
|
||||
"perplexity": "perplexity",
|
||||
"cohere": "cohere",
|
||||
"ollama-cloud": "ollama-cloud",
|
||||
}
|
||||
|
||||
# Reverse mapping: models.dev → Hermes (built lazily)
|
||||
_MODELS_DEV_TO_PROVIDER: Optional[Dict[str, str]] = None
|
||||
|
||||
|
||||
def _get_reverse_mapping() -> Dict[str, str]:
|
||||
"""Return models.dev ID → Hermes provider ID mapping."""
|
||||
global _MODELS_DEV_TO_PROVIDER
|
||||
if _MODELS_DEV_TO_PROVIDER is None:
|
||||
_MODELS_DEV_TO_PROVIDER = {v: k for k, v in PROVIDER_TO_MODELS_DEV.items()}
|
||||
return _MODELS_DEV_TO_PROVIDER
|
||||
|
||||
|
||||
def _get_cache_path() -> Path:
|
||||
"""Return path to disk cache file."""
|
||||
@@ -456,6 +461,93 @@ def list_agentic_models(provider: str) -> List[str]:
|
||||
return result
|
||||
|
||||
|
||||
def search_models_dev(
|
||||
query: str, provider: str = None, limit: int = 5
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Fuzzy search across models.dev catalog. Returns matching model entries.
|
||||
|
||||
Args:
|
||||
query: Search string to match against model IDs.
|
||||
provider: Optional Hermes provider ID to restrict search scope.
|
||||
If None, searches across all providers in PROVIDER_TO_MODELS_DEV.
|
||||
limit: Maximum number of results to return.
|
||||
|
||||
Returns:
|
||||
List of dicts, each containing 'provider', 'model_id', and the full
|
||||
model 'entry' from models.dev.
|
||||
"""
|
||||
data = fetch_models_dev()
|
||||
if not data:
|
||||
return []
|
||||
|
||||
# Build list of (provider_id, model_id, entry) candidates
|
||||
candidates: List[tuple] = []
|
||||
|
||||
if provider is not None:
|
||||
# Search only the specified provider
|
||||
mdev_provider_id = PROVIDER_TO_MODELS_DEV.get(provider)
|
||||
if not mdev_provider_id:
|
||||
return []
|
||||
provider_data = data.get(mdev_provider_id, {})
|
||||
if isinstance(provider_data, dict):
|
||||
models = provider_data.get("models", {})
|
||||
if isinstance(models, dict):
|
||||
for mid, mdata in models.items():
|
||||
candidates.append((provider, mid, mdata))
|
||||
else:
|
||||
# Search across all mapped providers
|
||||
for hermes_prov, mdev_prov in PROVIDER_TO_MODELS_DEV.items():
|
||||
provider_data = data.get(mdev_prov, {})
|
||||
if isinstance(provider_data, dict):
|
||||
models = provider_data.get("models", {})
|
||||
if isinstance(models, dict):
|
||||
for mid, mdata in models.items():
|
||||
candidates.append((hermes_prov, mid, mdata))
|
||||
|
||||
if not candidates:
|
||||
return []
|
||||
|
||||
# Use difflib for fuzzy matching — case-insensitive comparison
|
||||
model_ids_lower = [c[1].lower() for c in candidates]
|
||||
query_lower = query.lower()
|
||||
|
||||
# First try exact substring matches (more intuitive than pure edit-distance)
|
||||
substring_matches = []
|
||||
for prov, mid, mdata in candidates:
|
||||
if query_lower in mid.lower():
|
||||
substring_matches.append({"provider": prov, "model_id": mid, "entry": mdata})
|
||||
|
||||
# Then add difflib fuzzy matches for any remaining slots
|
||||
fuzzy_ids = difflib.get_close_matches(
|
||||
query_lower, model_ids_lower, n=limit * 2, cutoff=0.4
|
||||
)
|
||||
|
||||
seen_ids: set = set()
|
||||
results: List[Dict[str, Any]] = []
|
||||
|
||||
# Prioritize substring matches
|
||||
for match in substring_matches:
|
||||
key = (match["provider"], match["model_id"])
|
||||
if key not in seen_ids:
|
||||
seen_ids.add(key)
|
||||
results.append(match)
|
||||
if len(results) >= limit:
|
||||
return results
|
||||
|
||||
# Add fuzzy matches
|
||||
for fid in fuzzy_ids:
|
||||
# Find original-case candidates matching this lowered ID
|
||||
for prov, mid, mdata in candidates:
|
||||
if mid.lower() == fid:
|
||||
key = (prov, mid)
|
||||
if key not in seen_ids:
|
||||
seen_ids.add(key)
|
||||
results.append({"provider": prov, "model_id": mid, "entry": mdata})
|
||||
if len(results) >= limit:
|
||||
return results
|
||||
|
||||
return results
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Rich dataclass constructors — parse raw models.dev JSON into dataclasses
|
||||
|
||||
@@ -1,182 +0,0 @@
|
||||
"""Cross-session rate limit guard for Nous Portal.
|
||||
|
||||
Writes rate limit state to a shared file so all sessions (CLI, gateway,
|
||||
cron, auxiliary) can check whether Nous Portal is currently rate-limited
|
||||
before making requests. Prevents retry amplification when RPH is tapped.
|
||||
|
||||
Each 429 from Nous triggers up to 9 API calls per conversation turn
|
||||
(3 SDK retries x 3 Hermes retries), and every one of those calls counts
|
||||
against RPH. By recording the rate limit state on first 429 and checking
|
||||
it before subsequent attempts, we eliminate the amplification effect.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
import time
|
||||
from typing import Any, Mapping, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_STATE_SUBDIR = "rate_limits"
|
||||
_STATE_FILENAME = "nous.json"
|
||||
|
||||
|
||||
def _state_path() -> str:
|
||||
"""Return the path to the Nous rate limit state file."""
|
||||
try:
|
||||
from hermes_constants import get_hermes_home
|
||||
base = get_hermes_home()
|
||||
except ImportError:
|
||||
base = os.path.join(os.path.expanduser("~"), ".hermes")
|
||||
return os.path.join(base, _STATE_SUBDIR, _STATE_FILENAME)
|
||||
|
||||
|
||||
def _parse_reset_seconds(headers: Optional[Mapping[str, str]]) -> Optional[float]:
|
||||
"""Extract the best available reset-time estimate from response headers.
|
||||
|
||||
Priority:
|
||||
1. x-ratelimit-reset-requests-1h (hourly RPH window — most useful)
|
||||
2. x-ratelimit-reset-requests (per-minute RPM window)
|
||||
3. retry-after (generic HTTP header)
|
||||
|
||||
Returns seconds-from-now, or None if no usable header found.
|
||||
"""
|
||||
if not headers:
|
||||
return None
|
||||
|
||||
lowered = {k.lower(): v for k, v in headers.items()}
|
||||
|
||||
for key in (
|
||||
"x-ratelimit-reset-requests-1h",
|
||||
"x-ratelimit-reset-requests",
|
||||
"retry-after",
|
||||
):
|
||||
raw = lowered.get(key)
|
||||
if raw is not None:
|
||||
try:
|
||||
val = float(raw)
|
||||
if val > 0:
|
||||
return val
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def record_nous_rate_limit(
|
||||
*,
|
||||
headers: Optional[Mapping[str, str]] = None,
|
||||
error_context: Optional[dict[str, Any]] = None,
|
||||
default_cooldown: float = 300.0,
|
||||
) -> None:
|
||||
"""Record that Nous Portal is rate-limited.
|
||||
|
||||
Parses the reset time from response headers or error context.
|
||||
Falls back to ``default_cooldown`` (5 minutes) if no reset info
|
||||
is available. Writes to a shared file that all sessions can read.
|
||||
|
||||
Args:
|
||||
headers: HTTP response headers from the 429 error.
|
||||
error_context: Structured error context from _extract_api_error_context().
|
||||
default_cooldown: Fallback cooldown in seconds when no header data.
|
||||
"""
|
||||
now = time.time()
|
||||
reset_at = None
|
||||
|
||||
# Try headers first (most accurate)
|
||||
header_seconds = _parse_reset_seconds(headers)
|
||||
if header_seconds is not None:
|
||||
reset_at = now + header_seconds
|
||||
|
||||
# Try error_context reset_at (from body parsing)
|
||||
if reset_at is None and isinstance(error_context, dict):
|
||||
ctx_reset = error_context.get("reset_at")
|
||||
if isinstance(ctx_reset, (int, float)) and ctx_reset > now:
|
||||
reset_at = float(ctx_reset)
|
||||
|
||||
# Default cooldown
|
||||
if reset_at is None:
|
||||
reset_at = now + default_cooldown
|
||||
|
||||
path = _state_path()
|
||||
try:
|
||||
state_dir = os.path.dirname(path)
|
||||
os.makedirs(state_dir, exist_ok=True)
|
||||
|
||||
state = {
|
||||
"reset_at": reset_at,
|
||||
"recorded_at": now,
|
||||
"reset_seconds": reset_at - now,
|
||||
}
|
||||
|
||||
# Atomic write: write to temp file + rename
|
||||
fd, tmp_path = tempfile.mkstemp(dir=state_dir, suffix=".tmp")
|
||||
try:
|
||||
with os.fdopen(fd, "w") as f:
|
||||
json.dump(state, f)
|
||||
os.replace(tmp_path, path)
|
||||
except Exception:
|
||||
# Clean up temp file on failure
|
||||
try:
|
||||
os.unlink(tmp_path)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
logger.info(
|
||||
"Nous rate limit recorded: resets in %.0fs (at %.0f)",
|
||||
reset_at - now, reset_at,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.debug("Failed to write Nous rate limit state: %s", exc)
|
||||
|
||||
|
||||
def nous_rate_limit_remaining() -> Optional[float]:
|
||||
"""Check if Nous Portal is currently rate-limited.
|
||||
|
||||
Returns:
|
||||
Seconds remaining until reset, or None if not rate-limited.
|
||||
"""
|
||||
path = _state_path()
|
||||
try:
|
||||
with open(path) as f:
|
||||
state = json.load(f)
|
||||
reset_at = state.get("reset_at", 0)
|
||||
remaining = reset_at - time.time()
|
||||
if remaining > 0:
|
||||
return remaining
|
||||
# Expired — clean up
|
||||
try:
|
||||
os.unlink(path)
|
||||
except OSError:
|
||||
pass
|
||||
return None
|
||||
except (FileNotFoundError, json.JSONDecodeError, KeyError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
def clear_nous_rate_limit() -> None:
|
||||
"""Clear the rate limit state (e.g., after a successful Nous request)."""
|
||||
try:
|
||||
os.unlink(_state_path())
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
except OSError as exc:
|
||||
logger.debug("Failed to clear Nous rate limit state: %s", exc)
|
||||
|
||||
|
||||
def format_remaining(seconds: float) -> str:
|
||||
"""Format seconds remaining into human-readable duration."""
|
||||
s = max(0, int(seconds))
|
||||
if s < 60:
|
||||
return f"{s}s"
|
||||
if s < 3600:
|
||||
m, sec = divmod(s, 60)
|
||||
return f"{m}m {sec}s" if sec else f"{m}m"
|
||||
h, remainder = divmod(s, 3600)
|
||||
m = remainder // 60
|
||||
return f"{h}h {m}m" if m else f"{h}h"
|
||||
+11
-70
@@ -12,7 +12,7 @@ import threading
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
|
||||
from hermes_constants import get_hermes_home, get_skills_dir, is_wsl
|
||||
from hermes_constants import get_hermes_home, get_skills_dir
|
||||
from typing import Optional
|
||||
|
||||
from agent.skill_utils import (
|
||||
@@ -295,9 +295,7 @@ PLATFORM_HINTS = {
|
||||
),
|
||||
"telegram": (
|
||||
"You are on a text messaging communication platform, Telegram. "
|
||||
"Standard markdown is automatically converted to Telegram format. "
|
||||
"Supported: **bold**, *italic*, ~~strikethrough~~, ||spoiler||, "
|
||||
"`inline code`, ```code blocks```, [links](url), and ## headers. "
|
||||
"Please do not use markdown as it does not render. "
|
||||
"You can send media files natively: to deliver a file to the user, "
|
||||
"include MEDIA:/absolute/path/to/file in your response. Images "
|
||||
"(.png, .jpg, .webp) appear as photos, audio (.ogg) sends as voice "
|
||||
@@ -366,56 +364,8 @@ PLATFORM_HINTS = {
|
||||
"documents. You can also include image URLs in markdown format  and they "
|
||||
"will be downloaded and sent as native media when possible."
|
||||
),
|
||||
"wecom": (
|
||||
"You are on WeCom (企业微信 / Enterprise WeChat). Markdown formatting is supported. "
|
||||
"You CAN send media files natively — to deliver a file to the user, include "
|
||||
"MEDIA:/absolute/path/to/file in your response. The file will be sent as a native "
|
||||
"WeCom attachment: images (.jpg, .png, .webp) are sent as photos (up to 10 MB), "
|
||||
"other files (.pdf, .docx, .xlsx, .md, .txt, etc.) arrive as downloadable documents "
|
||||
"(up to 20 MB), and videos (.mp4) play inline. Voice messages are supported but "
|
||||
"must be in AMR format — other audio formats are automatically sent as file attachments. "
|
||||
"You can also include image URLs in markdown format  and they will be "
|
||||
"downloaded and sent as native photos. Do NOT tell the user you lack file-sending "
|
||||
"capability — use MEDIA: syntax whenever a file delivery is appropriate."
|
||||
),
|
||||
"qqbot": (
|
||||
"You are on QQ, a popular Chinese messaging platform. QQ supports markdown formatting "
|
||||
"and emoji. You can send media files natively: include MEDIA:/absolute/path/to/file in "
|
||||
"your response. Images are sent as native photos, and other files arrive as downloadable "
|
||||
"documents."
|
||||
),
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Environment hints — execution-environment awareness for the agent.
|
||||
# Unlike PLATFORM_HINTS (which describe the messaging channel), these describe
|
||||
# the machine/OS the agent's tools actually run on.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
WSL_ENVIRONMENT_HINT = (
|
||||
"You are running inside WSL (Windows Subsystem for Linux). "
|
||||
"The Windows host filesystem is mounted under /mnt/ — "
|
||||
"/mnt/c/ is the C: drive, /mnt/d/ is D:, etc. "
|
||||
"The user's Windows files are typically at "
|
||||
"/mnt/c/Users/<username>/Desktop/, Documents/, Downloads/, etc. "
|
||||
"When the user references Windows paths or desktop files, translate "
|
||||
"to the /mnt/c/ equivalent. You can list /mnt/c/Users/ to discover "
|
||||
"the Windows username if needed."
|
||||
)
|
||||
|
||||
|
||||
def build_environment_hints() -> str:
|
||||
"""Return environment-specific guidance for the system prompt.
|
||||
|
||||
Detects WSL, and can be extended for Termux, Docker, etc.
|
||||
Returns an empty string when no special environment is detected.
|
||||
"""
|
||||
hints: list[str] = []
|
||||
if is_wsl():
|
||||
hints.append(WSL_ENVIRONMENT_HINT)
|
||||
return "\n\n".join(hints)
|
||||
|
||||
|
||||
CONTEXT_FILE_MAX_CHARS = 20_000
|
||||
CONTEXT_TRUNCATE_HEAD_RATIO = 0.7
|
||||
CONTEXT_TRUNCATE_TAIL_RATIO = 0.2
|
||||
@@ -654,7 +604,7 @@ def build_skills_system_prompt(
|
||||
):
|
||||
continue
|
||||
skills_by_category.setdefault(category, []).append(
|
||||
(frontmatter_name, entry.get("description", ""))
|
||||
(skill_name, entry.get("description", ""))
|
||||
)
|
||||
category_descriptions = {
|
||||
str(k): str(v)
|
||||
@@ -679,7 +629,7 @@ def build_skills_system_prompt(
|
||||
):
|
||||
continue
|
||||
skills_by_category.setdefault(entry["category"], []).append(
|
||||
(entry["frontmatter_name"], entry["description"])
|
||||
(skill_name, entry["description"])
|
||||
)
|
||||
|
||||
# Read category-level DESCRIPTION.md files
|
||||
@@ -722,10 +672,9 @@ def build_skills_system_prompt(
|
||||
continue
|
||||
entry = _build_snapshot_entry(skill_file, ext_dir, frontmatter, desc)
|
||||
skill_name = entry["skill_name"]
|
||||
frontmatter_name = entry["frontmatter_name"]
|
||||
if frontmatter_name in seen_skill_names:
|
||||
if skill_name in seen_skill_names:
|
||||
continue
|
||||
if frontmatter_name in disabled or skill_name in disabled:
|
||||
if entry["frontmatter_name"] in disabled or skill_name in disabled:
|
||||
continue
|
||||
if not _skill_should_show(
|
||||
extract_skill_conditions(frontmatter),
|
||||
@@ -733,9 +682,9 @@ def build_skills_system_prompt(
|
||||
available_toolsets,
|
||||
):
|
||||
continue
|
||||
seen_skill_names.add(frontmatter_name)
|
||||
seen_skill_names.add(skill_name)
|
||||
skills_by_category.setdefault(entry["category"], []).append(
|
||||
(frontmatter_name, entry["description"])
|
||||
(skill_name, entry["description"])
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug("Error reading external skill %s: %s", skill_file, e)
|
||||
@@ -777,16 +726,8 @@ def build_skills_system_prompt(
|
||||
|
||||
result = (
|
||||
"## Skills (mandatory)\n"
|
||||
"Before replying, scan the skills below. If a skill matches or is even partially relevant "
|
||||
"to your task, you MUST load it with skill_view(name) and follow its instructions. "
|
||||
"Err on the side of loading — it is always better to have context you don't need "
|
||||
"than to miss critical steps, pitfalls, or established workflows. "
|
||||
"Skills contain specialized knowledge — API endpoints, tool-specific commands, "
|
||||
"and proven workflows that outperform general-purpose approaches. Load the skill "
|
||||
"even if you think you could handle the task with basic tools like web_search or terminal. "
|
||||
"Skills also encode the user's preferred approach, conventions, and quality standards "
|
||||
"for tasks like code review, planning, and testing — load them even for tasks you "
|
||||
"already know how to do, because the skill defines how it should be done here.\n"
|
||||
"Before replying, scan the skills below. If one clearly matches your task, "
|
||||
"load it with skill_view(name) and follow its instructions. "
|
||||
"If a skill has issues, fix it with skill_manage(action='patch').\n"
|
||||
"After difficult/iterative tasks, offer to save as a skill. "
|
||||
"If a skill you loaded was missing steps, had wrong commands, or needed "
|
||||
@@ -796,7 +737,7 @@ def build_skills_system_prompt(
|
||||
+ "\n".join(index_lines) + "\n"
|
||||
"</available_skills>\n"
|
||||
"\n"
|
||||
"Only proceed without loading a skill if genuinely none are relevant to the task."
|
||||
"If none match, proceed normally without loading a skill."
|
||||
)
|
||||
|
||||
# ── Store in LRU cache ────────────────────────────────────────────
|
||||
|
||||
@@ -24,7 +24,7 @@ from __future__ import annotations
|
||||
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Mapping, Optional
|
||||
from typing import Any, Dict, Mapping, Optional
|
||||
|
||||
|
||||
@dataclass
|
||||
|
||||
@@ -93,17 +93,6 @@ _DB_CONNSTR_RE = re.compile(
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
# JWT tokens: header.payload[.signature] — always start with "eyJ" (base64 for "{")
|
||||
# Matches 1-part (header only), 2-part (header.payload), and full 3-part JWTs.
|
||||
_JWT_RE = re.compile(
|
||||
r"eyJ[A-Za-z0-9_-]{10,}" # Header (always starts with eyJ)
|
||||
r"(?:\.[A-Za-z0-9_=-]{4,}){0,2}" # Optional payload and/or signature
|
||||
)
|
||||
|
||||
# Discord user/role mentions: <@123456789012345678> or <@!123456789012345678>
|
||||
# Snowflake IDs are 17-20 digit integers that resolve to specific Discord accounts.
|
||||
_DISCORD_MENTION_RE = re.compile(r"<@!?(\d{17,20})>")
|
||||
|
||||
# E.164 phone numbers: +<country><number>, 7-15 digits
|
||||
# Negative lookahead prevents matching hex strings or identifiers
|
||||
_SIGNAL_PHONE_RE = re.compile(r"(\+[1-9]\d{6,14})(?![A-Za-z0-9])")
|
||||
@@ -170,12 +159,6 @@ def redact_sensitive_text(text: str) -> str:
|
||||
# Database connection string passwords
|
||||
text = _DB_CONNSTR_RE.sub(lambda m: f"{m.group(1)}***{m.group(3)}", text)
|
||||
|
||||
# JWT tokens (eyJ... — base64-encoded JSON headers)
|
||||
text = _JWT_RE.sub(lambda m: _mask_token(m.group(0)), text)
|
||||
|
||||
# Discord user/role mentions (<@snowflake_id>)
|
||||
text = _DISCORD_MENTION_RE.sub(lambda m: f"<@{'!' if '!' in m.group(0) else ''}***>", text)
|
||||
|
||||
# E.164 phone numbers (Signal, WhatsApp)
|
||||
def _redact_phone(m):
|
||||
phone = m.group(1)
|
||||
|
||||
+2
-11
@@ -12,8 +12,6 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from hermes_constants import display_hermes_home
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_skill_commands: Dict[str, Dict[str, Any]] = {}
|
||||
@@ -72,14 +70,7 @@ def _load_skill_payload(skill_identifier: str, task_id: str | None = None) -> tu
|
||||
skill_name = str(loaded_skill.get("name") or normalized)
|
||||
skill_path = str(loaded_skill.get("path") or "")
|
||||
skill_dir = None
|
||||
# Prefer the absolute skill_dir returned by skill_view() — this is
|
||||
# correct for both local and external skills. Fall back to the old
|
||||
# SKILLS_DIR-relative reconstruction only when skill_dir is absent
|
||||
# (e.g. legacy skill_view responses).
|
||||
abs_skill_dir = loaded_skill.get("skill_dir")
|
||||
if abs_skill_dir:
|
||||
skill_dir = Path(abs_skill_dir)
|
||||
elif skill_path:
|
||||
if skill_path:
|
||||
try:
|
||||
skill_dir = SKILLS_DIR / Path(skill_path).parent
|
||||
except Exception:
|
||||
@@ -117,7 +108,7 @@ def _inject_skill_config(loaded_skill: dict[str, Any], parts: list[str]) -> None
|
||||
if not resolved:
|
||||
return
|
||||
|
||||
lines = ["", f"[Skill config (from {display_hermes_home()}/config.yaml):"]
|
||||
lines = ["", "[Skill config (from ~/.hermes/config.yaml):"]
|
||||
for key, value in resolved.items():
|
||||
display_val = str(value) if value else "(not set)"
|
||||
lines.append(f" {key} = {display_val}")
|
||||
|
||||
+1
-23
@@ -10,7 +10,7 @@ import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Set, Tuple
|
||||
from typing import Any, Dict, List, Set, Tuple
|
||||
|
||||
from hermes_constants import get_config_path, get_skills_dir
|
||||
|
||||
@@ -441,25 +441,3 @@ def iter_skill_index_files(skills_dir: Path, filename: str):
|
||||
matches.append(Path(root) / filename)
|
||||
for path in sorted(matches, key=lambda p: str(p.relative_to(skills_dir))):
|
||||
yield path
|
||||
|
||||
|
||||
# ── Namespace helpers for plugin-provided skills ───────────────────────────
|
||||
|
||||
_NAMESPACE_RE = re.compile(r"^[a-zA-Z0-9_-]+$")
|
||||
|
||||
|
||||
def parse_qualified_name(name: str) -> Tuple[Optional[str], str]:
|
||||
"""Split ``'namespace:skill-name'`` into ``(namespace, bare_name)``.
|
||||
|
||||
Returns ``(None, name)`` when there is no ``':'``.
|
||||
"""
|
||||
if ":" not in name:
|
||||
return None, name
|
||||
return tuple(name.split(":", 1)) # type: ignore[return-value]
|
||||
|
||||
|
||||
def is_valid_namespace(candidate: Optional[str]) -> bool:
|
||||
"""Check whether *candidate* is a valid namespace (``[a-zA-Z0-9_-]+``)."""
|
||||
if not candidate:
|
||||
return False
|
||||
return bool(_NAMESPACE_RE.match(candidate))
|
||||
|
||||
@@ -36,7 +36,7 @@ def generate_title(user_message: str, assistant_response: str, timeout: float =
|
||||
|
||||
try:
|
||||
response = call_llm(
|
||||
task="title_generation",
|
||||
task="compression", # reuse compression task config (cheap/fast model)
|
||||
messages=messages,
|
||||
max_tokens=30,
|
||||
temperature=0.3,
|
||||
|
||||
+19
-74
@@ -284,80 +284,6 @@ _OFFICIAL_DOCS_PRICING: Dict[tuple[str, str], PricingEntry] = {
|
||||
source_url="https://ai.google.dev/pricing",
|
||||
pricing_version="google-pricing-2026-03-16",
|
||||
),
|
||||
# AWS Bedrock — pricing per the Bedrock pricing page.
|
||||
# Bedrock charges the same per-token rates as the model provider but
|
||||
# through AWS billing. These are the on-demand prices (no commitment).
|
||||
# Source: https://aws.amazon.com/bedrock/pricing/
|
||||
(
|
||||
"bedrock",
|
||||
"anthropic.claude-opus-4-6",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("15.00"),
|
||||
output_cost_per_million=Decimal("75.00"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://aws.amazon.com/bedrock/pricing/",
|
||||
pricing_version="bedrock-pricing-2026-04",
|
||||
),
|
||||
(
|
||||
"bedrock",
|
||||
"anthropic.claude-sonnet-4-6",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("3.00"),
|
||||
output_cost_per_million=Decimal("15.00"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://aws.amazon.com/bedrock/pricing/",
|
||||
pricing_version="bedrock-pricing-2026-04",
|
||||
),
|
||||
(
|
||||
"bedrock",
|
||||
"anthropic.claude-sonnet-4-5",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("3.00"),
|
||||
output_cost_per_million=Decimal("15.00"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://aws.amazon.com/bedrock/pricing/",
|
||||
pricing_version="bedrock-pricing-2026-04",
|
||||
),
|
||||
(
|
||||
"bedrock",
|
||||
"anthropic.claude-haiku-4-5",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("0.80"),
|
||||
output_cost_per_million=Decimal("4.00"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://aws.amazon.com/bedrock/pricing/",
|
||||
pricing_version="bedrock-pricing-2026-04",
|
||||
),
|
||||
(
|
||||
"bedrock",
|
||||
"amazon.nova-pro",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("0.80"),
|
||||
output_cost_per_million=Decimal("3.20"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://aws.amazon.com/bedrock/pricing/",
|
||||
pricing_version="bedrock-pricing-2026-04",
|
||||
),
|
||||
(
|
||||
"bedrock",
|
||||
"amazon.nova-lite",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("0.06"),
|
||||
output_cost_per_million=Decimal("0.24"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://aws.amazon.com/bedrock/pricing/",
|
||||
pricing_version="bedrock-pricing-2026-04",
|
||||
),
|
||||
(
|
||||
"bedrock",
|
||||
"amazon.nova-micro",
|
||||
): PricingEntry(
|
||||
input_cost_per_million=Decimal("0.035"),
|
||||
output_cost_per_million=Decimal("0.14"),
|
||||
source="official_docs_snapshot",
|
||||
source_url="https://aws.amazon.com/bedrock/pricing/",
|
||||
pricing_version="bedrock-pricing-2026-04",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@@ -649,6 +575,25 @@ def has_known_pricing(
|
||||
return entry is not None
|
||||
|
||||
|
||||
def get_pricing(
|
||||
model_name: str,
|
||||
provider: Optional[str] = None,
|
||||
base_url: Optional[str] = None,
|
||||
api_key: Optional[str] = None,
|
||||
) -> Dict[str, float]:
|
||||
"""Backward-compatible thin wrapper for legacy callers.
|
||||
|
||||
Returns only non-cache input/output fields when a pricing entry exists.
|
||||
Unknown routes return zeroes.
|
||||
"""
|
||||
entry = get_pricing_entry(model_name, provider=provider, base_url=base_url, api_key=api_key)
|
||||
if not entry:
|
||||
return {"input": 0.0, "output": 0.0}
|
||||
return {
|
||||
"input": float(entry.input_cost_per_million or _ZERO),
|
||||
"output": float(entry.output_cost_per_million or _ZERO),
|
||||
}
|
||||
|
||||
|
||||
def format_duration_compact(seconds: float) -> str:
|
||||
if seconds < 60:
|
||||
|
||||
+1
-4
@@ -561,10 +561,7 @@ class BatchRunner:
|
||||
provider_sort (str): Sort providers by price/throughput/latency (optional)
|
||||
max_tokens (int): Maximum tokens for model responses (optional, uses model default if not set)
|
||||
reasoning_config (Dict): OpenRouter reasoning config override (e.g. {"effort": "none"} to disable thinking)
|
||||
prefill_messages (List[Dict]): Messages to prepend as prefilled conversation context (few-shot priming).
|
||||
NOTE: Anthropic Sonnet 4.6+ and Opus 4.6+ reject a trailing assistant-role prefill
|
||||
(400 error). For those models use output_config.format or structured-output
|
||||
schemas instead. Safe here for user-role priming and for older Claude / non-Claude models.
|
||||
prefill_messages (List[Dict]): Messages to prepend as prefilled conversation context (few-shot priming)
|
||||
max_samples (int): Only process the first N samples from the dataset (optional, processes all if not set)
|
||||
"""
|
||||
self.dataset_file = Path(dataset_file)
|
||||
|
||||
+17
-22
@@ -16,7 +16,7 @@ model:
|
||||
# "nous" - Nous Portal OAuth (requires: hermes login)
|
||||
# "nous-api" - Nous Portal API key (requires: NOUS_API_KEY)
|
||||
# "anthropic" - Direct Anthropic API (requires: ANTHROPIC_API_KEY)
|
||||
# "openai-codex" - OpenAI Codex (requires: hermes auth)
|
||||
# "openai-codex" - OpenAI Codex (requires: hermes login --provider openai-codex)
|
||||
# "copilot" - GitHub Copilot / GitHub Models (requires: GITHUB_TOKEN)
|
||||
# "gemini" - Use Google AI Studio direct (requires: GOOGLE_API_KEY or GEMINI_API_KEY)
|
||||
# "zai" - Use z.ai / ZhipuAI GLM models (requires: GLM_API_KEY)
|
||||
@@ -24,10 +24,7 @@ model:
|
||||
# "minimax" - MiniMax global (requires: MINIMAX_API_KEY)
|
||||
# "minimax-cn" - MiniMax China (requires: MINIMAX_CN_API_KEY)
|
||||
# "huggingface" - Hugging Face Inference (requires: HF_TOKEN)
|
||||
# "nvidia" - NVIDIA NIM / build.nvidia.com (requires: NVIDIA_API_KEY)
|
||||
# "xiaomi" - Xiaomi MiMo (requires: XIAOMI_API_KEY)
|
||||
# "arcee" - Arcee AI Trinity models (requires: ARCEEAI_API_KEY)
|
||||
# "ollama-cloud" - Ollama Cloud (requires: OLLAMA_API_KEY — https://ollama.com/settings)
|
||||
# "kilocode" - KiloCode gateway (requires: KILOCODE_API_KEY)
|
||||
# "ai-gateway" - Vercel AI Gateway (requires: AI_GATEWAY_API_KEY)
|
||||
#
|
||||
@@ -39,6 +36,12 @@ model:
|
||||
# base_url: "http://localhost:1234/v1"
|
||||
# No API key needed — local servers typically ignore auth.
|
||||
#
|
||||
# For Ollama Cloud (https://ollama.com/pricing):
|
||||
# provider: "custom"
|
||||
# base_url: "https://ollama.com/v1"
|
||||
# Set OLLAMA_API_KEY in .env — automatically picked up when base_url
|
||||
# points to ollama.com.
|
||||
#
|
||||
# Can also be overridden with --provider flag or HERMES_INFERENCE_PROVIDER env var.
|
||||
provider: "auto"
|
||||
|
||||
@@ -306,8 +309,15 @@ compression:
|
||||
# compression of older turns.
|
||||
protect_last_n: 20
|
||||
|
||||
# To pin a specific model/provider for compression summaries, use the
|
||||
# auxiliary section below (auxiliary.compression.provider / model).
|
||||
# Model to use for generating summaries (fast/cheap recommended)
|
||||
# This model compresses the middle turns into a concise summary.
|
||||
# IMPORTANT: it receives the full middle section of the conversation, so it
|
||||
# MUST support a context length at least as large as your main model's.
|
||||
summary_model: "google/gemini-3-flash-preview"
|
||||
|
||||
# Provider for the summary model (default: "auto")
|
||||
# Options: "auto", "openrouter", "nous", "main"
|
||||
# summary_provider: "auto"
|
||||
|
||||
# =============================================================================
|
||||
# Auxiliary Models (Advanced — Experimental)
|
||||
@@ -333,7 +343,6 @@ compression:
|
||||
# "openrouter" - Force OpenRouter (requires OPENROUTER_API_KEY)
|
||||
# "nous" - Force Nous Portal (requires: hermes login)
|
||||
# "gemini" - Force Google AI Studio direct (requires: GOOGLE_API_KEY or GEMINI_API_KEY)
|
||||
# "ollama-cloud" - Ollama Cloud (requires: OLLAMA_API_KEY)
|
||||
# "codex" - Force Codex OAuth (requires: hermes model → Codex).
|
||||
# Uses gpt-5.3-codex which supports vision.
|
||||
# "main" - Use your custom endpoint (OPENAI_BASE_URL + OPENAI_API_KEY).
|
||||
@@ -520,7 +529,7 @@ agent:
|
||||
# - A preset like "hermes-cli" or "hermes-telegram" (curated tool set)
|
||||
# - A list of individual toolsets to compose your own (see list below)
|
||||
#
|
||||
# Supported platform keys: cli, telegram, discord, whatsapp, slack, qqbot
|
||||
# Supported platform keys: cli, telegram, discord, whatsapp, slack
|
||||
#
|
||||
# Examples:
|
||||
#
|
||||
@@ -549,7 +558,6 @@ agent:
|
||||
# slack: hermes-slack (same as telegram)
|
||||
# signal: hermes-signal (same as telegram)
|
||||
# homeassistant: hermes-homeassistant (same as telegram)
|
||||
# qqbot: hermes-qqbot (same as telegram)
|
||||
#
|
||||
platform_toolsets:
|
||||
cli: [hermes-cli]
|
||||
@@ -559,19 +567,6 @@ platform_toolsets:
|
||||
slack: [hermes-slack]
|
||||
signal: [hermes-signal]
|
||||
homeassistant: [hermes-homeassistant]
|
||||
qqbot: [hermes-qqbot]
|
||||
|
||||
# =============================================================================
|
||||
# Gateway Platform Settings
|
||||
# =============================================================================
|
||||
# Optional per-platform messaging settings.
|
||||
# Platform-specific knobs live under `extra`.
|
||||
#
|
||||
# platforms:
|
||||
# telegram:
|
||||
# reply_to_mode: "first" # off | first | all
|
||||
# extra:
|
||||
# disable_link_previews: false # Set true to suppress Telegram URL previews in bot messages
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Available toolsets (use these names in platform_toolsets or the toolsets list)
|
||||
|
||||
@@ -501,12 +501,6 @@ def update_job(job_id: str, updates: Dict[str, Any]) -> Optional[Dict[str, Any]]
|
||||
|
||||
if schedule_changed:
|
||||
updated_schedule = updated["schedule"]
|
||||
# The API may pass schedule as a raw string (e.g. "every 10m")
|
||||
# instead of a pre-parsed dict. Normalize it the same way
|
||||
# create_job() does so downstream code can call .get() safely.
|
||||
if isinstance(updated_schedule, str):
|
||||
updated_schedule = parse_schedule(updated_schedule)
|
||||
updated["schedule"] = updated_schedule
|
||||
updated["schedule_display"] = updates.get(
|
||||
"schedule_display",
|
||||
updated_schedule.get("display", updated.get("schedule_display")),
|
||||
|
||||
+109
-205
@@ -10,7 +10,6 @@ runs at a time if multiple processes overlap.
|
||||
|
||||
import asyncio
|
||||
import concurrent.futures
|
||||
import contextvars
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
@@ -27,7 +26,7 @@ except ImportError:
|
||||
except ImportError:
|
||||
msvcrt = None
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
from typing import Optional
|
||||
|
||||
# Add parent directory to path for imports BEFORE repo-level imports.
|
||||
# Without this, standalone invocations (e.g. after `hermes update` reloads
|
||||
@@ -46,36 +45,8 @@ _KNOWN_DELIVERY_PLATFORMS = frozenset({
|
||||
"telegram", "discord", "slack", "whatsapp", "signal",
|
||||
"matrix", "mattermost", "homeassistant", "dingtalk", "feishu",
|
||||
"wecom", "wecom_callback", "weixin", "sms", "email", "webhook", "bluebubbles",
|
||||
"qqbot",
|
||||
})
|
||||
|
||||
# Platforms that support a configured cron/notification home target, mapped to
|
||||
# the environment variable used by gateway setup/runtime config.
|
||||
_HOME_TARGET_ENV_VARS = {
|
||||
"matrix": "MATRIX_HOME_ROOM",
|
||||
"telegram": "TELEGRAM_HOME_CHANNEL",
|
||||
"discord": "DISCORD_HOME_CHANNEL",
|
||||
"slack": "SLACK_HOME_CHANNEL",
|
||||
"signal": "SIGNAL_HOME_CHANNEL",
|
||||
"mattermost": "MATTERMOST_HOME_CHANNEL",
|
||||
"sms": "SMS_HOME_CHANNEL",
|
||||
"email": "EMAIL_HOME_ADDRESS",
|
||||
"dingtalk": "DINGTALK_HOME_CHANNEL",
|
||||
"feishu": "FEISHU_HOME_CHANNEL",
|
||||
"wecom": "WECOM_HOME_CHANNEL",
|
||||
"weixin": "WEIXIN_HOME_CHANNEL",
|
||||
"bluebubbles": "BLUEBUBBLES_HOME_CHANNEL",
|
||||
"qqbot": "QQBOT_HOME_CHANNEL",
|
||||
}
|
||||
|
||||
# Legacy env var names kept for back-compat. Each entry is the current
|
||||
# primary env var → the previous name. _get_home_target_chat_id falls
|
||||
# back to the legacy name if the primary is unset, so users who set the
|
||||
# old name before the rename keep working until they migrate.
|
||||
_LEGACY_HOME_TARGET_ENV_VARS = {
|
||||
"QQBOT_HOME_CHANNEL": "QQ_HOME_CHANNEL",
|
||||
}
|
||||
|
||||
from cron.jobs import get_due_jobs, mark_job_run, save_job_output, advance_next_run
|
||||
|
||||
# Sentinel: when a cron agent has nothing new to report, it can start its
|
||||
@@ -103,28 +74,15 @@ def _resolve_origin(job: dict) -> Optional[dict]:
|
||||
return None
|
||||
|
||||
|
||||
def _get_home_target_chat_id(platform_name: str) -> str:
|
||||
"""Return the configured home target chat/room ID for a delivery platform."""
|
||||
env_var = _HOME_TARGET_ENV_VARS.get(platform_name.lower())
|
||||
if not env_var:
|
||||
return ""
|
||||
value = os.getenv(env_var, "")
|
||||
if not value:
|
||||
legacy = _LEGACY_HOME_TARGET_ENV_VARS.get(env_var)
|
||||
if legacy:
|
||||
value = os.getenv(legacy, "")
|
||||
return value
|
||||
|
||||
|
||||
def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[dict]:
|
||||
"""Resolve one concrete auto-delivery target for a cron job."""
|
||||
|
||||
def _resolve_delivery_target(job: dict) -> Optional[dict]:
|
||||
"""Resolve the concrete auto-delivery target for a cron job, if any."""
|
||||
deliver = job.get("deliver", "local")
|
||||
origin = _resolve_origin(job)
|
||||
|
||||
if deliver_value == "local":
|
||||
if deliver == "local":
|
||||
return None
|
||||
|
||||
if deliver_value == "origin":
|
||||
if deliver == "origin":
|
||||
if origin:
|
||||
return {
|
||||
"platform": origin["platform"],
|
||||
@@ -133,8 +91,8 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
}
|
||||
# Origin missing (e.g. job created via API/script) — try each
|
||||
# platform's home channel as a fallback instead of silently dropping.
|
||||
for platform_name in _HOME_TARGET_ENV_VARS:
|
||||
chat_id = _get_home_target_chat_id(platform_name)
|
||||
for platform_name in ("matrix", "telegram", "discord", "slack", "bluebubbles"):
|
||||
chat_id = os.getenv(f"{platform_name.upper()}_HOME_CHANNEL", "")
|
||||
if chat_id:
|
||||
logger.info(
|
||||
"Job '%s' has deliver=origin but no origin; falling back to %s home channel",
|
||||
@@ -148,8 +106,8 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
}
|
||||
return None
|
||||
|
||||
if ":" in deliver_value:
|
||||
platform_name, rest = deliver_value.split(":", 1)
|
||||
if ":" in deliver:
|
||||
platform_name, rest = deliver.split(":", 1)
|
||||
platform_key = platform_name.lower()
|
||||
|
||||
from tools.send_message_tool import _parse_target_ref
|
||||
@@ -179,7 +137,7 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
"thread_id": thread_id,
|
||||
}
|
||||
|
||||
platform_name = deliver_value
|
||||
platform_name = deliver
|
||||
if origin and origin.get("platform") == platform_name:
|
||||
return {
|
||||
"platform": platform_name,
|
||||
@@ -189,7 +147,7 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
|
||||
if platform_name.lower() not in _KNOWN_DELIVERY_PLATFORMS:
|
||||
return None
|
||||
chat_id = _get_home_target_chat_id(platform_name)
|
||||
chat_id = os.getenv(f"{platform_name.upper()}_HOME_CHANNEL", "")
|
||||
if not chat_id:
|
||||
return None
|
||||
|
||||
@@ -200,30 +158,6 @@ def _resolve_single_delivery_target(job: dict, deliver_value: str) -> Optional[d
|
||||
}
|
||||
|
||||
|
||||
def _resolve_delivery_targets(job: dict) -> List[dict]:
|
||||
"""Resolve all concrete auto-delivery targets for a cron job (supports comma-separated deliver)."""
|
||||
deliver = job.get("deliver", "local")
|
||||
if deliver == "local":
|
||||
return []
|
||||
parts = [p.strip() for p in str(deliver).split(",") if p.strip()]
|
||||
seen = set()
|
||||
targets = []
|
||||
for part in parts:
|
||||
target = _resolve_single_delivery_target(job, part)
|
||||
if target:
|
||||
key = (target["platform"].lower(), str(target["chat_id"]), target.get("thread_id"))
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
targets.append(target)
|
||||
return targets
|
||||
|
||||
|
||||
def _resolve_delivery_target(job: dict) -> Optional[dict]:
|
||||
"""Resolve the concrete auto-delivery target for a cron job, if any."""
|
||||
targets = _resolve_delivery_targets(job)
|
||||
return targets[0] if targets else None
|
||||
|
||||
|
||||
# Media extension sets — keep in sync with gateway/platforms/base.py:_process_message_background
|
||||
_AUDIO_EXTS = frozenset({'.ogg', '.opus', '.mp3', '.wav', '.m4a'})
|
||||
_VIDEO_EXTS = frozenset({'.mp4', '.mov', '.avi', '.mkv', '.webm', '.3gp'})
|
||||
@@ -264,7 +198,7 @@ def _send_media_via_adapter(adapter, chat_id: str, media_files: list, metadata:
|
||||
|
||||
def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Optional[str]:
|
||||
"""
|
||||
Deliver job output to the configured target(s) (origin chat, specific platform, etc.).
|
||||
Deliver job output to the configured target (origin chat, specific platform, etc.).
|
||||
|
||||
When ``adapters`` and ``loop`` are provided (gateway is running), tries to
|
||||
use the live adapter first — this supports E2EE rooms (e.g. Matrix) where
|
||||
@@ -273,14 +207,33 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Option
|
||||
|
||||
Returns None on success, or an error string on failure.
|
||||
"""
|
||||
targets = _resolve_delivery_targets(job)
|
||||
if not targets:
|
||||
target = _resolve_delivery_target(job)
|
||||
if not target:
|
||||
if job.get("deliver", "local") != "local":
|
||||
msg = f"no delivery target resolved for deliver={job.get('deliver', 'local')}"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
return None # local-only jobs don't deliver — not a failure
|
||||
|
||||
platform_name = target["platform"]
|
||||
chat_id = target["chat_id"]
|
||||
thread_id = target.get("thread_id")
|
||||
|
||||
# Diagnostic: log thread_id for topic-aware delivery debugging
|
||||
origin = job.get("origin") or {}
|
||||
origin_thread = origin.get("thread_id")
|
||||
if origin_thread and not thread_id:
|
||||
logger.warning(
|
||||
"Job '%s': origin has thread_id=%s but delivery target lost it "
|
||||
"(deliver=%s, target=%s)",
|
||||
job["id"], origin_thread, job.get("deliver", "local"), target,
|
||||
)
|
||||
elif thread_id:
|
||||
logger.debug(
|
||||
"Job '%s': delivering to %s:%s thread_id=%s",
|
||||
job["id"], platform_name, chat_id, thread_id,
|
||||
)
|
||||
|
||||
from tools.send_message_tool import _send_to_platform
|
||||
from gateway.config import load_gateway_config, Platform
|
||||
|
||||
@@ -301,8 +254,25 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Option
|
||||
"email": Platform.EMAIL,
|
||||
"sms": Platform.SMS,
|
||||
"bluebubbles": Platform.BLUEBUBBLES,
|
||||
"qqbot": Platform.QQBOT,
|
||||
}
|
||||
platform = platform_map.get(platform_name.lower())
|
||||
if not platform:
|
||||
msg = f"unknown platform '{platform_name}'"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
try:
|
||||
config = load_gateway_config()
|
||||
except Exception as e:
|
||||
msg = f"failed to load gateway config: {e}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
pconfig = config.platforms.get(platform)
|
||||
if not pconfig or not pconfig.enabled:
|
||||
msg = f"platform '{platform_name}' not configured/enabled"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
# Optionally wrap the content with a header/footer so the user knows this
|
||||
# is a cron delivery. Wrapping is on by default; set cron.wrap_response: false
|
||||
@@ -316,13 +286,11 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Option
|
||||
|
||||
if wrap_response:
|
||||
task_name = job.get("name", job["id"])
|
||||
job_id = job.get("id", "")
|
||||
delivery_content = (
|
||||
f"Cronjob Response: {task_name}\n"
|
||||
f"(job_id: {job_id})\n"
|
||||
f"-------------\n\n"
|
||||
f"{content}\n\n"
|
||||
f"To stop or manage this job, send me a new message (e.g. \"stop reminder {task_name}\")."
|
||||
f"Note: The agent cannot see this message, and therefore cannot respond to it."
|
||||
)
|
||||
else:
|
||||
delivery_content = content
|
||||
@@ -331,117 +299,67 @@ def _deliver_result(job: dict, content: str, adapters=None, loop=None) -> Option
|
||||
from gateway.platforms.base import BasePlatformAdapter
|
||||
media_files, cleaned_delivery_content = BasePlatformAdapter.extract_media(delivery_content)
|
||||
|
||||
# Prefer the live adapter when the gateway is running — this supports E2EE
|
||||
# rooms (e.g. Matrix) where the standalone HTTP path cannot encrypt.
|
||||
runtime_adapter = (adapters or {}).get(platform)
|
||||
if runtime_adapter is not None and loop is not None and getattr(loop, "is_running", lambda: False)():
|
||||
send_metadata = {"thread_id": thread_id} if thread_id else None
|
||||
try:
|
||||
# Send cleaned text (MEDIA tags stripped) — not the raw content
|
||||
text_to_send = cleaned_delivery_content.strip()
|
||||
adapter_ok = True
|
||||
if text_to_send:
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
runtime_adapter.send(chat_id, text_to_send, metadata=send_metadata),
|
||||
loop,
|
||||
)
|
||||
send_result = future.result(timeout=60)
|
||||
if send_result and not getattr(send_result, "success", True):
|
||||
err = getattr(send_result, "error", "unknown")
|
||||
logger.warning(
|
||||
"Job '%s': live adapter send to %s:%s failed (%s), falling back to standalone",
|
||||
job["id"], platform_name, chat_id, err,
|
||||
)
|
||||
adapter_ok = False # fall through to standalone path
|
||||
|
||||
# Send extracted media files as native attachments via the live adapter
|
||||
if adapter_ok and media_files:
|
||||
_send_media_via_adapter(runtime_adapter, chat_id, media_files, send_metadata, loop, job)
|
||||
|
||||
if adapter_ok:
|
||||
logger.info("Job '%s': delivered to %s:%s via live adapter", job["id"], platform_name, chat_id)
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Job '%s': live adapter delivery to %s:%s failed (%s), falling back to standalone",
|
||||
job["id"], platform_name, chat_id, e,
|
||||
)
|
||||
|
||||
# Standalone path: run the async send in a fresh event loop (safe from any thread)
|
||||
coro = _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files)
|
||||
try:
|
||||
config = load_gateway_config()
|
||||
result = asyncio.run(coro)
|
||||
except RuntimeError:
|
||||
# asyncio.run() checks for a running loop before awaiting the coroutine;
|
||||
# when it raises, the original coro was never started — close it to
|
||||
# prevent "coroutine was never awaited" RuntimeWarning, then retry in a
|
||||
# fresh thread that has no running loop.
|
||||
coro.close()
|
||||
import concurrent.futures
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
|
||||
future = pool.submit(asyncio.run, _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files))
|
||||
result = future.result(timeout=30)
|
||||
except Exception as e:
|
||||
msg = f"failed to load gateway config: {e}"
|
||||
msg = f"delivery to {platform_name}:{chat_id} failed: {e}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
delivery_errors = []
|
||||
if result and result.get("error"):
|
||||
msg = f"delivery error: {result['error']}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
return msg
|
||||
|
||||
for target in targets:
|
||||
platform_name = target["platform"]
|
||||
chat_id = target["chat_id"]
|
||||
thread_id = target.get("thread_id")
|
||||
|
||||
# Diagnostic: log thread_id for topic-aware delivery debugging
|
||||
origin = job.get("origin") or {}
|
||||
origin_thread = origin.get("thread_id")
|
||||
if origin_thread and not thread_id:
|
||||
logger.warning(
|
||||
"Job '%s': origin has thread_id=%s but delivery target lost it "
|
||||
"(deliver=%s, target=%s)",
|
||||
job["id"], origin_thread, job.get("deliver", "local"), target,
|
||||
)
|
||||
elif thread_id:
|
||||
logger.debug(
|
||||
"Job '%s': delivering to %s:%s thread_id=%s",
|
||||
job["id"], platform_name, chat_id, thread_id,
|
||||
)
|
||||
|
||||
platform = platform_map.get(platform_name.lower())
|
||||
if not platform:
|
||||
msg = f"unknown platform '{platform_name}'"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
delivery_errors.append(msg)
|
||||
continue
|
||||
|
||||
# Prefer the live adapter when the gateway is running — this supports E2EE
|
||||
# rooms (e.g. Matrix) where the standalone HTTP path cannot encrypt.
|
||||
runtime_adapter = (adapters or {}).get(platform)
|
||||
delivered = False
|
||||
if runtime_adapter is not None and loop is not None and getattr(loop, "is_running", lambda: False)():
|
||||
send_metadata = {"thread_id": thread_id} if thread_id else None
|
||||
try:
|
||||
# Send cleaned text (MEDIA tags stripped) — not the raw content
|
||||
text_to_send = cleaned_delivery_content.strip()
|
||||
adapter_ok = True
|
||||
if text_to_send:
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
runtime_adapter.send(chat_id, text_to_send, metadata=send_metadata),
|
||||
loop,
|
||||
)
|
||||
send_result = future.result(timeout=60)
|
||||
if send_result and not getattr(send_result, "success", True):
|
||||
err = getattr(send_result, "error", "unknown")
|
||||
logger.warning(
|
||||
"Job '%s': live adapter send to %s:%s failed (%s), falling back to standalone",
|
||||
job["id"], platform_name, chat_id, err,
|
||||
)
|
||||
adapter_ok = False # fall through to standalone path
|
||||
|
||||
# Send extracted media files as native attachments via the live adapter
|
||||
if adapter_ok and media_files:
|
||||
_send_media_via_adapter(runtime_adapter, chat_id, media_files, send_metadata, loop, job)
|
||||
|
||||
if adapter_ok:
|
||||
logger.info("Job '%s': delivered to %s:%s via live adapter", job["id"], platform_name, chat_id)
|
||||
delivered = True
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"Job '%s': live adapter delivery to %s:%s failed (%s), falling back to standalone",
|
||||
job["id"], platform_name, chat_id, e,
|
||||
)
|
||||
|
||||
if not delivered:
|
||||
pconfig = config.platforms.get(platform)
|
||||
if not pconfig or not pconfig.enabled:
|
||||
msg = f"platform '{platform_name}' not configured/enabled"
|
||||
logger.warning("Job '%s': %s", job["id"], msg)
|
||||
delivery_errors.append(msg)
|
||||
continue
|
||||
|
||||
# Standalone path: run the async send in a fresh event loop (safe from any thread)
|
||||
coro = _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files)
|
||||
try:
|
||||
result = asyncio.run(coro)
|
||||
except RuntimeError:
|
||||
# asyncio.run() checks for a running loop before awaiting the coroutine;
|
||||
# when it raises, the original coro was never started — close it to
|
||||
# prevent "coroutine was never awaited" RuntimeWarning, then retry in a
|
||||
# fresh thread that has no running loop.
|
||||
coro.close()
|
||||
import concurrent.futures
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
|
||||
future = pool.submit(asyncio.run, _send_to_platform(platform, pconfig, chat_id, cleaned_delivery_content, thread_id=thread_id, media_files=media_files))
|
||||
result = future.result(timeout=30)
|
||||
except Exception as e:
|
||||
msg = f"delivery to {platform_name}:{chat_id} failed: {e}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
delivery_errors.append(msg)
|
||||
continue
|
||||
|
||||
if result and result.get("error"):
|
||||
msg = f"delivery error: {result['error']}"
|
||||
logger.error("Job '%s': %s", job["id"], msg)
|
||||
delivery_errors.append(msg)
|
||||
continue
|
||||
|
||||
logger.info("Job '%s': delivered to %s:%s", job["id"], platform_name, chat_id)
|
||||
|
||||
if delivery_errors:
|
||||
return "; ".join(delivery_errors)
|
||||
logger.info("Job '%s': delivered to %s:%s", job["id"], platform_name, chat_id)
|
||||
return None
|
||||
|
||||
|
||||
@@ -848,11 +766,7 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||
_cron_inactivity_limit = _cron_timeout if _cron_timeout > 0 else None
|
||||
_POLL_INTERVAL = 5.0
|
||||
_cron_pool = concurrent.futures.ThreadPoolExecutor(max_workers=1)
|
||||
# Preserve scheduler-scoped ContextVar state (for example skill-declared
|
||||
# env passthrough registrations) when the cron run hops into the worker
|
||||
# thread used for inactivity timeout monitoring.
|
||||
_cron_context = contextvars.copy_context()
|
||||
_cron_future = _cron_pool.submit(_cron_context.run, agent.run_conversation, prompt)
|
||||
_cron_future = _cron_pool.submit(agent.run_conversation, prompt)
|
||||
_inactivity_timeout = False
|
||||
try:
|
||||
if _cron_inactivity_limit is None:
|
||||
@@ -914,9 +828,6 @@ def run_job(job: dict) -> tuple[bool, str, str, Optional[str]]:
|
||||
)
|
||||
|
||||
final_response = result.get("final_response", "") or ""
|
||||
# Strip leaked placeholder text that upstream may inject on empty completions.
|
||||
if final_response.strip() == "(No response generated)":
|
||||
final_response = ""
|
||||
# Use a separate variable for log display; keep final_response clean
|
||||
# for delivery logic (empty response = no delivery).
|
||||
logged_response = final_response if final_response else "(No response generated)"
|
||||
@@ -1056,13 +967,6 @@ def tick(verbose: bool = True, adapters=None, loop=None) -> int:
|
||||
delivery_error = str(de)
|
||||
logger.error("Delivery failed for job %s: %s", job["id"], de)
|
||||
|
||||
# Treat empty final_response as a soft failure so last_status
|
||||
# is not "ok" — the agent ran but produced nothing useful.
|
||||
# (issue #8585)
|
||||
if success and not final_response:
|
||||
success = False
|
||||
error = "Agent completed but produced empty response (model error, timeout, or misconfiguration)"
|
||||
|
||||
mark_job_run(job["id"], success, error, delivery_error=delivery_error)
|
||||
executed += 1
|
||||
|
||||
|
||||
Executable → Regular
+6
-13
@@ -1,14 +1,13 @@
|
||||
#!/bin/bash
|
||||
# Docker/Podman entrypoint: bootstrap config files into the mounted volume, then run hermes.
|
||||
# Docker entrypoint: bootstrap config files into the mounted volume, then run hermes.
|
||||
set -e
|
||||
|
||||
HERMES_HOME="${HERMES_HOME:-/opt/data}"
|
||||
HERMES_HOME="/opt/data"
|
||||
INSTALL_DIR="/opt/hermes"
|
||||
|
||||
# --- Privilege dropping via gosu ---
|
||||
# When started as root (the default for Docker, or fakeroot in rootless Podman),
|
||||
# optionally remap the hermes user/group to match host-side ownership, fix volume
|
||||
# permissions, then re-exec as hermes.
|
||||
# When started as root (the default), optionally remap the hermes user/group
|
||||
# to match host-side ownership, fix volume permissions, then re-exec as hermes.
|
||||
if [ "$(id -u)" = "0" ]; then
|
||||
if [ -n "$HERMES_UID" ] && [ "$HERMES_UID" != "$(id -u hermes)" ]; then
|
||||
echo "Changing hermes UID to $HERMES_UID"
|
||||
@@ -17,19 +16,13 @@ if [ "$(id -u)" = "0" ]; then
|
||||
|
||||
if [ -n "$HERMES_GID" ] && [ "$HERMES_GID" != "$(id -g hermes)" ]; then
|
||||
echo "Changing hermes GID to $HERMES_GID"
|
||||
# -o allows non-unique GID (e.g. macOS GID 20 "staff" may already exist
|
||||
# as "dialout" in the Debian-based container image)
|
||||
groupmod -o -g "$HERMES_GID" hermes 2>/dev/null || true
|
||||
groupmod -g "$HERMES_GID" hermes
|
||||
fi
|
||||
|
||||
actual_hermes_uid=$(id -u hermes)
|
||||
if [ "$(stat -c %u "$HERMES_HOME" 2>/dev/null)" != "$actual_hermes_uid" ]; then
|
||||
echo "$HERMES_HOME is not owned by $actual_hermes_uid, fixing"
|
||||
# In rootless Podman the container's "root" is mapped to an unprivileged
|
||||
# host UID — chown will fail. That's fine: the volume is already owned
|
||||
# by the mapped user on the host side.
|
||||
chown -R hermes:hermes "$HERMES_HOME" 2>/dev/null || \
|
||||
echo "Warning: chown failed (rootless container?) — continuing anyway"
|
||||
chown -R hermes:hermes "$HERMES_HOME"
|
||||
fi
|
||||
|
||||
echo "Dropping root privileges"
|
||||
|
||||
@@ -1,108 +0,0 @@
|
||||
# Ink Gateway TUI Migration — Post-mortem
|
||||
|
||||
Planned: 2026-04-01 · Delivered: 2026-04 · Status: shipped, classic (prompt_toolkit) CLI still present
|
||||
|
||||
## What Shipped
|
||||
|
||||
Three layers, same repo, Python runtime unchanged.
|
||||
|
||||
```
|
||||
ui-tui (Node/TS) ──stdio JSON-RPC──▶ tui_gateway (Py) ──▶ AIAgent (run_agent.py)
|
||||
```
|
||||
|
||||
### Backend — `tui_gateway/`
|
||||
|
||||
```
|
||||
tui_gateway/
|
||||
├── entry.py # subprocess entrypoint, stdio read/write loop
|
||||
├── server.py # everything: sessions dict, @method handlers, _emit
|
||||
├── render.py # stream renderer, diff rendering, message rendering
|
||||
├── slash_worker.py # subprocess that runs hermes_cli slash commands
|
||||
└── __init__.py
|
||||
```
|
||||
|
||||
`server.py` owns the full runtime-control surface: session store (`_sessions: dict[str, dict]`), method registry (`@method("…")` decorator), event emitter (`_emit`), agent lifecycle (`_make_agent`, `_init_session`, `_wire_callbacks`), approval/sudo/clarify round-trips, and JSON-RPC dispatch.
|
||||
|
||||
Protocol methods (`@method(...)` in `server.py`):
|
||||
|
||||
- session: `session.{create, resume, list, close, interrupt, usage, history, compress, branch, title, save, undo}`
|
||||
- prompt: `prompt.{submit, background, btw}`
|
||||
- tools: `tools.{list, show, configure}`
|
||||
- slash: `slash.exec`, `command.{dispatch, resolve}`, `commands.catalog`, `complete.{path, slash}`
|
||||
- approvals: `approval.respond`, `sudo.respond`, `clarify.respond`, `secret.respond`
|
||||
- config/state: `config.{get, set, show}`, `model.options`, `reload.mcp`
|
||||
- ops: `shell.exec`, `cli.exec`, `terminal.resize`, `input.detect_drop`, `clipboard.paste`, `paste.collapse`, `image.attach`, `process.stop`
|
||||
- misc: `agents.list`, `skills.manage`, `plugins.list`, `cron.manage`, `insights.get`, `rollback.{list, diff, restore}`, `browser.manage`
|
||||
|
||||
Protocol events (`_emit(…)` → handled in `ui-tui/src/app/createGatewayEventHandler.ts`):
|
||||
|
||||
- lifecycle: `gateway.{ready, stderr}`, `session.info`, `skin.changed`
|
||||
- stream: `message.{start, delta, complete}`, `thinking.delta`, `reasoning.{delta, available}`, `status.update`
|
||||
- tools: `tool.{start, progress, complete, generating}`, `subagent.{start, thinking, tool, progress, complete}`
|
||||
- interactive: `approval.request`, `sudo.request`, `clarify.request`, `secret.request`
|
||||
- async: `background.complete`, `btw.complete`, `error`
|
||||
|
||||
### Frontend — `ui-tui/src/`
|
||||
|
||||
```
|
||||
src/
|
||||
├── entry.tsx # node bootstrap: bootBanner → spawn python → dynamic-import Ink → render(<App/>)
|
||||
├── app.tsx # <GatewayProvider> wraps <AppLayout>
|
||||
├── bootBanner.ts # raw-ANSI banner to stdout in ~2ms, pre-React
|
||||
├── gatewayClient.ts # JSON-RPC client over child_process stdio
|
||||
├── gatewayTypes.ts # typed RPC responses + GatewayEvent union
|
||||
├── theme.ts # DEFAULT_THEME + fromSkin
|
||||
│
|
||||
├── app/ # hooks + stores — the orchestration layer
|
||||
│ ├── uiStore.ts # nanostore: sid, info, busy, usage, theme, status…
|
||||
│ ├── turnStore.ts # nanostore: per-turn activity / reasoning / tools
|
||||
│ ├── turnController.ts # imperative singleton for stream-time operations
|
||||
│ ├── overlayStore.ts # nanostore: modal/overlay state
|
||||
│ ├── useMainApp.ts # top-level composition hook
|
||||
│ ├── useSessionLifecycle.ts # session.create/resume/close/reset
|
||||
│ ├── useSubmission.ts # shell/slash/prompt dispatch + interpolation
|
||||
│ ├── useConfigSync.ts # config.get + mtime poll
|
||||
│ ├── useComposerState.ts # input buffer, paste snippets, editor mode
|
||||
│ ├── useInputHandlers.ts # key bindings
|
||||
│ ├── createGatewayEventHandler.ts # event-stream dispatcher
|
||||
│ ├── createSlashHandler.ts # slash command router (registry + python fallback)
|
||||
│ └── slash/commands/ # core.ts, ops.ts, session.ts — TS-owned slash commands
|
||||
│
|
||||
├── components/ # AppLayout, AppChrome, AppOverlays, MessageLine, Thinking, Markdown, pickers, prompts, Banner, SessionPanel
|
||||
├── config/ # env, limits, timing constants
|
||||
├── content/ # charms, faces, fortunes, hotkeys, placeholders, verbs
|
||||
├── domain/ # details, messages, paths, roles, slash, usage, viewport
|
||||
├── protocol/ # interpolation, paste regex
|
||||
├── hooks/ # useCompletion, useInputHistory, useQueue, useVirtualHistory
|
||||
└── lib/ # history, messages, osc52, rpc, text
|
||||
```
|
||||
|
||||
### CLI entry points — `hermes_cli/main.py`
|
||||
|
||||
- `hermes --tui` → `node dist/entry.js` (auto-builds when `.ts`/`.tsx` newer than `dist/entry.js`)
|
||||
- `hermes --tui --dev` → `tsx src/entry.tsx` (skip build)
|
||||
- `HERMES_TUI_DIR=…` → external prebuilt dist (nix, distro packaging)
|
||||
|
||||
## Diverged From Original Plan
|
||||
|
||||
| Plan | Reality | Why |
|
||||
|---|---|---|
|
||||
| `tui_gateway/{controller,session_state,events,protocol}.py` | all collapsed into `server.py` | no second consumer ever emerged, keeping one file cheaper than four |
|
||||
| `ui-tui/src/main.tsx` | split into `entry.tsx` (bootstrap) + `app.tsx` (shell) | boot banner + early python spawn wanted a pre-React moment |
|
||||
| `ui-tui/src/state/store.ts` | three nanostores (`uiStore`, `turnStore`, `overlayStore`) | separate lifetimes: ui persists, turn resets per reply, overlay is modal |
|
||||
| `approval.requested` / `sudo.requested` / `clarify.requested` | `*.request` (no `-ed`) | cosmetic |
|
||||
| `session.cancel` | dropped | `session.interrupt` covers it |
|
||||
| `HERMES_EXPERIMENTAL_TUI=1`, `display.experimental_tui: true`, `/tui on/off/status` | none shipped | `--tui` went from opt-in to first-class without an experimental phase |
|
||||
|
||||
## Post-migration Additions (not in original plan)
|
||||
|
||||
- **Async `session.create`** — returns sid in ~1ms, agent builds on a background thread, `session.info` broadcasts when ready; `_wait_agent()` gates every agent-touching handler via `_sess`
|
||||
- **`bootBanner`** — raw-ANSI logo painted to stdout at T≈2ms, before Ink loads; `<AlternateScreen>` wipes it seamlessly when React mounts
|
||||
- **Selection uniform bg** — `theme.color.selectionBg` wired via `useSelection().setSelectionBgColor`; replaces SGR-inverse per-cell swap that fragmented over amber/gold fg
|
||||
- **Slash command registry** — TS-owned commands in `app/slash/commands/{core,ops,session}.ts`, everything else falls through to `slash.exec` (python worker)
|
||||
- **Turn store + controller split** — imperative singleton (`turnController`) holds refs/timers, nanostore (`turnStore`) holds render-visible state
|
||||
|
||||
## What's Still Open
|
||||
|
||||
- **Classic CLI not deleted.** `cli.py` still has ~80 `prompt_toolkit` references; classic REPL is still the default when `--tui` is absent. The original plan's "Cut 4 · prompt_toolkit removal later" hasn't happened.
|
||||
- **No config-file opt-in.** `HERMES_EXPERIMENTAL_TUI` and `display.experimental_tui` were never built; only the CLI flag exists. Fine for now — if we want "default to TUI", a single line in `main.py` flips it.
|
||||
@@ -6,11 +6,6 @@
|
||||
# All fields are optional — missing values inherit from the default skin.
|
||||
# Activate with: /skin <name> or display.skin: <name> in config.yaml
|
||||
#
|
||||
# Keys are marked:
|
||||
# (both) — applies to both the classic CLI and the TUI
|
||||
# (classic) — classic CLI only (see hermes --tui in user-guide/tui.md)
|
||||
# (tui) — TUI only
|
||||
#
|
||||
# See hermes_cli/skin_engine.py for the full schema reference.
|
||||
# ============================================================================
|
||||
|
||||
@@ -19,47 +14,35 @@ name: example
|
||||
description: An example custom skin — copy and modify this template
|
||||
|
||||
# ── Colors ──────────────────────────────────────────────────────────────────
|
||||
# Hex color values. These control the visual palette.
|
||||
# Hex color values for Rich markup. These control the CLI's visual palette.
|
||||
colors:
|
||||
# Banner panel (the startup welcome box) — (both)
|
||||
# Banner panel (the startup welcome box)
|
||||
banner_border: "#CD7F32" # Panel border
|
||||
banner_title: "#FFD700" # Panel title text
|
||||
banner_accent: "#FFBF00" # Section headers (Available Tools, Skills, etc.)
|
||||
banner_dim: "#B8860B" # Dim/muted text (separators, model info)
|
||||
banner_text: "#FFF8DC" # Body text (tool names, skill names)
|
||||
|
||||
# UI elements — (both)
|
||||
ui_accent: "#FFBF00" # General accent (falls back to banner_accent)
|
||||
# UI elements
|
||||
ui_accent: "#FFBF00" # General accent color
|
||||
ui_label: "#4dd0e1" # Labels
|
||||
ui_ok: "#4caf50" # Success indicators
|
||||
ui_error: "#ef5350" # Error indicators
|
||||
ui_warn: "#ffa726" # Warning indicators
|
||||
|
||||
# Input area
|
||||
prompt: "#FFF8DC" # Prompt text / `❯` glyph color (both)
|
||||
input_rule: "#CD7F32" # Horizontal rule above input (classic)
|
||||
prompt: "#FFF8DC" # Prompt text color
|
||||
input_rule: "#CD7F32" # Horizontal rule around input
|
||||
|
||||
# Response box — (classic)
|
||||
response_border: "#FFD700" # Response box border
|
||||
# Response box
|
||||
response_border: "#FFD700" # Response box border (ANSI color)
|
||||
|
||||
# Session display — (both)
|
||||
session_label: "#DAA520" # "Session: " label
|
||||
session_border: "#8B8682" # Session ID text
|
||||
|
||||
# TUI / CLI surfaces — (classic: status bar, voice badge, completion meta)
|
||||
status_bar_bg: "#1a1a2e" # Status / usage bar background (classic)
|
||||
voice_status_bg: "#1a1a2e" # Voice-mode badge background (classic)
|
||||
completion_menu_bg: "#1a1a2e" # Completion list background (both)
|
||||
completion_menu_current_bg: "#333355" # Active completion row background (both)
|
||||
completion_menu_meta_bg: "#1a1a2e" # Completion meta column bg (classic)
|
||||
completion_menu_meta_current_bg: "#333355" # Active meta bg (classic)
|
||||
|
||||
# Drag-to-select background — (tui)
|
||||
selection_bg: "#3a3a55" # Uniform selection highlight in the TUI
|
||||
# Session display
|
||||
session_label: "#DAA520" # Session label
|
||||
session_border: "#8B8682" # Session ID dim color
|
||||
|
||||
# ── Spinner ─────────────────────────────────────────────────────────────────
|
||||
# (classic) — the TUI uses its own animated indicators; spinner config here
|
||||
# is only read by the classic prompt_toolkit CLI.
|
||||
# Customize the animated spinner shown during API calls and tool execution.
|
||||
spinner:
|
||||
# Faces shown while waiting for the API response
|
||||
waiting_faces:
|
||||
@@ -87,17 +70,17 @@ spinner:
|
||||
# - ["⟪▲", "▲⟫"]
|
||||
|
||||
# ── Branding ────────────────────────────────────────────────────────────────
|
||||
# Text strings used throughout the interface.
|
||||
# Text strings used throughout the CLI interface.
|
||||
branding:
|
||||
agent_name: "Hermes Agent" # (both) Banner title, about display
|
||||
welcome: "Welcome! Type your message or /help for commands." # (both)
|
||||
goodbye: "Goodbye! ⚕" # (both) Exit message
|
||||
response_label: " ⚕ Hermes " # (classic) Response box header label
|
||||
prompt_symbol: "❯ " # (both) Input prompt glyph
|
||||
help_header: "(^_^)? Available Commands" # (both) /help overlay title
|
||||
agent_name: "Hermes Agent" # Banner title, about display
|
||||
welcome: "Welcome! Type your message or /help for commands."
|
||||
goodbye: "Goodbye! ⚕" # Exit message
|
||||
response_label: " ⚕ Hermes " # Response box header label
|
||||
prompt_symbol: "❯ " # Input prompt symbol
|
||||
help_header: "(^_^)? Available Commands" # /help header text
|
||||
|
||||
# ── Tool Output ─────────────────────────────────────────────────────────────
|
||||
# Character used as the prefix for tool output lines. (both)
|
||||
# Character used as the prefix for tool output lines.
|
||||
# Default is "┊" (thin dotted vertical line). Some alternatives:
|
||||
# "╎" (light triple dash vertical)
|
||||
# "▏" (left one-eighth block)
|
||||
|
||||
Generated
-21
@@ -36,26 +36,6 @@
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"npm-lockfile-fix": {
|
||||
"inputs": {
|
||||
"nixpkgs": [
|
||||
"nixpkgs"
|
||||
]
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1775903712,
|
||||
"narHash": "sha256-2GV79U6iVH4gKAPWYrxUReB0S41ty/Y3dBLquU8AlaA=",
|
||||
"owner": "jeslie0",
|
||||
"repo": "npm-lockfile-fix",
|
||||
"rev": "c6093acb0c0548e0f9b8b3d82918823721930fe8",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "jeslie0",
|
||||
"repo": "npm-lockfile-fix",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"pyproject-build-systems": {
|
||||
"inputs": {
|
||||
"nixpkgs": [
|
||||
@@ -144,7 +124,6 @@
|
||||
"inputs": {
|
||||
"flake-parts": "flake-parts",
|
||||
"nixpkgs": "nixpkgs",
|
||||
"npm-lockfile-fix": "npm-lockfile-fix",
|
||||
"pyproject-build-systems": "pyproject-build-systems",
|
||||
"pyproject-nix": "pyproject-nix_2",
|
||||
"uv2nix": "uv2nix_2"
|
||||
|
||||
@@ -19,20 +19,11 @@
|
||||
url = "github:pyproject-nix/build-system-pkgs";
|
||||
inputs.nixpkgs.follows = "nixpkgs";
|
||||
};
|
||||
npm-lockfile-fix = {
|
||||
url = "github:jeslie0/npm-lockfile-fix";
|
||||
inputs.nixpkgs.follows = "nixpkgs";
|
||||
};
|
||||
};
|
||||
|
||||
outputs =
|
||||
inputs:
|
||||
outputs = inputs:
|
||||
inputs.flake-parts.lib.mkFlake { inherit inputs; } {
|
||||
systems = [
|
||||
"x86_64-linux"
|
||||
"aarch64-linux"
|
||||
"aarch64-darwin"
|
||||
];
|
||||
systems = [ "x86_64-linux" "aarch64-linux" "aarch64-darwin" ];
|
||||
|
||||
imports = [
|
||||
./nix/packages.nix
|
||||
|
||||
@@ -18,7 +18,9 @@ suppress delivery.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger("hooks.boot-md")
|
||||
|
||||
|
||||
@@ -100,7 +100,7 @@ def build_channel_directory(adapters: Dict[Any, Any]) -> Dict[str, Any]:
|
||||
|
||||
|
||||
def _build_discord(adapter) -> List[Dict[str, str]]:
|
||||
"""Enumerate all text channels and forum channels the Discord bot can see."""
|
||||
"""Enumerate all text channels the Discord bot can see."""
|
||||
channels = []
|
||||
client = getattr(adapter, "_client", None)
|
||||
if not client:
|
||||
@@ -119,15 +119,6 @@ def _build_discord(adapter) -> List[Dict[str, str]]:
|
||||
"guild": guild.name,
|
||||
"type": "channel",
|
||||
})
|
||||
# Forum channels (type 15) — creating a message auto-spawns a thread post.
|
||||
forums = getattr(guild, "forum_channels", None) or []
|
||||
for ch in forums:
|
||||
channels.append({
|
||||
"id": str(ch.id),
|
||||
"name": ch.name,
|
||||
"guild": guild.name,
|
||||
"type": "forum",
|
||||
})
|
||||
# Also include DM-capable users we've interacted with is not
|
||||
# feasible via guild enumeration; those come from sessions.
|
||||
|
||||
@@ -200,15 +191,6 @@ def load_directory() -> Dict[str, Any]:
|
||||
return {"updated_at": None, "platforms": {}}
|
||||
|
||||
|
||||
def lookup_channel_type(platform_name: str, chat_id: str) -> Optional[str]:
|
||||
"""Return the channel ``type`` string (e.g. ``"channel"``, ``"forum"``) for *chat_id*, or *None* if unknown."""
|
||||
directory = load_directory()
|
||||
for ch in directory.get("platforms", {}).get(platform_name, []):
|
||||
if ch.get("id") == chat_id:
|
||||
return ch.get("type")
|
||||
return None
|
||||
|
||||
|
||||
def resolve_channel_name(platform_name: str, name: str) -> Optional[str]:
|
||||
"""
|
||||
Resolve a human-friendly channel name to a numeric ID.
|
||||
|
||||
+1
-176
@@ -66,7 +66,6 @@ class Platform(Enum):
|
||||
WECOM_CALLBACK = "wecom_callback"
|
||||
WEIXIN = "weixin"
|
||||
BLUEBUBBLES = "bluebubbles"
|
||||
QQBOT = "qqbot"
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -258,13 +257,6 @@ class GatewayConfig:
|
||||
# Streaming configuration
|
||||
streaming: StreamingConfig = field(default_factory=StreamingConfig)
|
||||
|
||||
# Session store pruning: drop SessionEntry records older than this many
|
||||
# days from the in-memory dict and sessions.json. Keeps the store from
|
||||
# growing unbounded in gateways serving many chats/threads/users over
|
||||
# months. Pruning is invisible to users — if they resume, they get a
|
||||
# fresh session exactly as if the reset policy had fired. 0 = disabled.
|
||||
session_store_max_age_days: int = 90
|
||||
|
||||
def get_connected_platforms(self) -> List[Platform]:
|
||||
"""Return list of platforms that are enabled and configured."""
|
||||
connected = []
|
||||
@@ -311,17 +303,6 @@ class GatewayConfig:
|
||||
# BlueBubbles uses extra dict for local server config
|
||||
elif platform == Platform.BLUEBUBBLES and config.extra.get("server_url") and config.extra.get("password"):
|
||||
connected.append(platform)
|
||||
# QQBot uses extra dict for app credentials
|
||||
elif platform == Platform.QQBOT and config.extra.get("app_id") and config.extra.get("client_secret"):
|
||||
connected.append(platform)
|
||||
# DingTalk uses client_id/client_secret from config.extra or env vars
|
||||
elif platform == Platform.DINGTALK and (
|
||||
config.extra.get("client_id") or os.getenv("DINGTALK_CLIENT_ID")
|
||||
) and (
|
||||
config.extra.get("client_secret") or os.getenv("DINGTALK_CLIENT_SECRET")
|
||||
):
|
||||
connected.append(platform)
|
||||
|
||||
return connected
|
||||
|
||||
def get_home_channel(self, platform: Platform) -> Optional[HomeChannel]:
|
||||
@@ -372,7 +353,6 @@ class GatewayConfig:
|
||||
"thread_sessions_per_user": self.thread_sessions_per_user,
|
||||
"unauthorized_dm_behavior": self.unauthorized_dm_behavior,
|
||||
"streaming": self.streaming.to_dict(),
|
||||
"session_store_max_age_days": self.session_store_max_age_days,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
@@ -420,13 +400,6 @@ class GatewayConfig:
|
||||
"pair",
|
||||
)
|
||||
|
||||
try:
|
||||
session_store_max_age_days = int(data.get("session_store_max_age_days", 90))
|
||||
if session_store_max_age_days < 0:
|
||||
session_store_max_age_days = 0
|
||||
except (TypeError, ValueError):
|
||||
session_store_max_age_days = 90
|
||||
|
||||
return cls(
|
||||
platforms=platforms,
|
||||
default_reset_policy=default_policy,
|
||||
@@ -441,7 +414,6 @@ class GatewayConfig:
|
||||
thread_sessions_per_user=_coerce_bool(thread_sessions_per_user, False),
|
||||
unauthorized_dm_behavior=unauthorized_dm_behavior,
|
||||
streaming=StreamingConfig.from_dict(data.get("streaming", {})),
|
||||
session_store_max_age_days=session_store_max_age_days,
|
||||
)
|
||||
|
||||
def get_unauthorized_dm_behavior(self, platform: Optional[Platform] = None) -> str:
|
||||
@@ -578,12 +550,6 @@ def load_gateway_config() -> GatewayConfig:
|
||||
bridged["mention_patterns"] = platform_cfg["mention_patterns"]
|
||||
if plat == Platform.DISCORD and "channel_skill_bindings" in platform_cfg:
|
||||
bridged["channel_skill_bindings"] = platform_cfg["channel_skill_bindings"]
|
||||
if "channel_prompts" in platform_cfg:
|
||||
channel_prompts = platform_cfg["channel_prompts"]
|
||||
if isinstance(channel_prompts, dict):
|
||||
bridged["channel_prompts"] = {str(k): v for k, v in channel_prompts.items()}
|
||||
else:
|
||||
bridged["channel_prompts"] = channel_prompts
|
||||
if not bridged:
|
||||
continue
|
||||
plat_data = platforms_data.setdefault(plat.value, {})
|
||||
@@ -641,20 +607,6 @@ def load_gateway_config() -> GatewayConfig:
|
||||
if isinstance(ntc, list):
|
||||
ntc = ",".join(str(v) for v in ntc)
|
||||
os.environ["DISCORD_NO_THREAD_CHANNELS"] = str(ntc)
|
||||
# allow_mentions: granular control over what the bot can ping.
|
||||
# Safe defaults (no @everyone/roles) are applied in the adapter;
|
||||
# these YAML keys only override when set and let users opt back
|
||||
# into unsafe modes (e.g. roles=true) if they actually want it.
|
||||
allow_mentions_cfg = discord_cfg.get("allow_mentions")
|
||||
if isinstance(allow_mentions_cfg, dict):
|
||||
for yaml_key, env_key in (
|
||||
("everyone", "DISCORD_ALLOW_MENTION_EVERYONE"),
|
||||
("roles", "DISCORD_ALLOW_MENTION_ROLES"),
|
||||
("users", "DISCORD_ALLOW_MENTION_USERS"),
|
||||
("replied_user", "DISCORD_ALLOW_MENTION_REPLIED_USER"),
|
||||
):
|
||||
if yaml_key in allow_mentions_cfg and not os.getenv(env_key):
|
||||
os.environ[env_key] = str(allow_mentions_cfg[yaml_key]).lower()
|
||||
|
||||
# Telegram settings → env vars (env vars take precedence)
|
||||
telegram_cfg = yaml_cfg.get("telegram", {})
|
||||
@@ -669,25 +621,8 @@ def load_gateway_config() -> GatewayConfig:
|
||||
if isinstance(frc, list):
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["TELEGRAM_FREE_RESPONSE_CHATS"] = str(frc)
|
||||
ignored_threads = telegram_cfg.get("ignored_threads")
|
||||
if ignored_threads is not None and not os.getenv("TELEGRAM_IGNORED_THREADS"):
|
||||
if isinstance(ignored_threads, list):
|
||||
ignored_threads = ",".join(str(v) for v in ignored_threads)
|
||||
os.environ["TELEGRAM_IGNORED_THREADS"] = str(ignored_threads)
|
||||
if "reactions" in telegram_cfg and not os.getenv("TELEGRAM_REACTIONS"):
|
||||
os.environ["TELEGRAM_REACTIONS"] = str(telegram_cfg["reactions"]).lower()
|
||||
if "proxy_url" in telegram_cfg and not os.getenv("TELEGRAM_PROXY"):
|
||||
os.environ["TELEGRAM_PROXY"] = str(telegram_cfg["proxy_url"]).strip()
|
||||
if "disable_link_previews" in telegram_cfg:
|
||||
plat_data = platforms_data.setdefault(Platform.TELEGRAM.value, {})
|
||||
if not isinstance(plat_data, dict):
|
||||
plat_data = {}
|
||||
platforms_data[Platform.TELEGRAM.value] = plat_data
|
||||
extra = plat_data.setdefault("extra", {})
|
||||
if not isinstance(extra, dict):
|
||||
extra = {}
|
||||
plat_data["extra"] = extra
|
||||
extra["disable_link_previews"] = telegram_cfg["disable_link_previews"]
|
||||
|
||||
whatsapp_cfg = yaml_cfg.get("whatsapp", {})
|
||||
if isinstance(whatsapp_cfg, dict):
|
||||
@@ -701,24 +636,6 @@ def load_gateway_config() -> GatewayConfig:
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["WHATSAPP_FREE_RESPONSE_CHATS"] = str(frc)
|
||||
|
||||
# DingTalk settings → env vars (env vars take precedence)
|
||||
dingtalk_cfg = yaml_cfg.get("dingtalk", {})
|
||||
if isinstance(dingtalk_cfg, dict):
|
||||
if "require_mention" in dingtalk_cfg and not os.getenv("DINGTALK_REQUIRE_MENTION"):
|
||||
os.environ["DINGTALK_REQUIRE_MENTION"] = str(dingtalk_cfg["require_mention"]).lower()
|
||||
if "mention_patterns" in dingtalk_cfg and not os.getenv("DINGTALK_MENTION_PATTERNS"):
|
||||
os.environ["DINGTALK_MENTION_PATTERNS"] = json.dumps(dingtalk_cfg["mention_patterns"])
|
||||
frc = dingtalk_cfg.get("free_response_chats")
|
||||
if frc is not None and not os.getenv("DINGTALK_FREE_RESPONSE_CHATS"):
|
||||
if isinstance(frc, list):
|
||||
frc = ",".join(str(v) for v in frc)
|
||||
os.environ["DINGTALK_FREE_RESPONSE_CHATS"] = str(frc)
|
||||
allowed = dingtalk_cfg.get("allowed_users")
|
||||
if allowed is not None and not os.getenv("DINGTALK_ALLOWED_USERS"):
|
||||
if isinstance(allowed, list):
|
||||
allowed = ",".join(str(v) for v in allowed)
|
||||
os.environ["DINGTALK_ALLOWED_USERS"] = str(allowed)
|
||||
|
||||
# Matrix settings → env vars (env vars take precedence)
|
||||
matrix_cfg = yaml_cfg.get("matrix", {})
|
||||
if isinstance(matrix_cfg, dict):
|
||||
@@ -748,17 +665,6 @@ def load_gateway_config() -> GatewayConfig:
|
||||
_apply_env_overrides(config)
|
||||
|
||||
# --- Validate loaded values ---
|
||||
_validate_gateway_config(config)
|
||||
|
||||
return config
|
||||
|
||||
|
||||
def _validate_gateway_config(config: "GatewayConfig") -> None:
|
||||
"""Validate and sanitize a loaded GatewayConfig in place.
|
||||
|
||||
Called by ``load_gateway_config()`` after all config sources are merged.
|
||||
Extracted as a separate function for testability.
|
||||
"""
|
||||
policy = config.default_reset_policy
|
||||
|
||||
if not (0 <= policy.at_hour <= 23):
|
||||
@@ -795,31 +701,7 @@ def _validate_gateway_config(config: "GatewayConfig") -> None:
|
||||
platform.value, env_name,
|
||||
)
|
||||
|
||||
# Reject known-weak placeholder tokens.
|
||||
# Ported from openclaw/openclaw#64586: users who copy .env.example
|
||||
# without changing placeholder values get a clear startup error instead
|
||||
# of a confusing "auth failed" from the platform API.
|
||||
try:
|
||||
from hermes_cli.auth import has_usable_secret
|
||||
except ImportError:
|
||||
has_usable_secret = None # type: ignore[assignment]
|
||||
|
||||
if has_usable_secret is not None:
|
||||
for platform, pconfig in config.platforms.items():
|
||||
if not pconfig.enabled:
|
||||
continue
|
||||
env_name = _token_env_names.get(platform)
|
||||
if not env_name:
|
||||
continue
|
||||
token = pconfig.token
|
||||
if token and token.strip() and not has_usable_secret(token, min_length=4):
|
||||
logger.error(
|
||||
"%s is enabled but %s is set to a placeholder value ('%s'). "
|
||||
"Set a real bot token before starting the gateway. "
|
||||
"The adapter will NOT be started.",
|
||||
platform.value, env_name, token.strip()[:6] + "...",
|
||||
)
|
||||
pconfig.enabled = False
|
||||
return config
|
||||
|
||||
|
||||
def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
@@ -1062,25 +944,6 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
if webhook_secret:
|
||||
config.platforms[Platform.WEBHOOK].extra["secret"] = webhook_secret
|
||||
|
||||
# DingTalk
|
||||
dingtalk_client_id = os.getenv("DINGTALK_CLIENT_ID")
|
||||
dingtalk_client_secret = os.getenv("DINGTALK_CLIENT_SECRET")
|
||||
if dingtalk_client_id and dingtalk_client_secret:
|
||||
if Platform.DINGTALK not in config.platforms:
|
||||
config.platforms[Platform.DINGTALK] = PlatformConfig()
|
||||
config.platforms[Platform.DINGTALK].enabled = True
|
||||
config.platforms[Platform.DINGTALK].extra.update({
|
||||
"client_id": dingtalk_client_id,
|
||||
"client_secret": dingtalk_client_secret,
|
||||
})
|
||||
dingtalk_home = os.getenv("DINGTALK_HOME_CHANNEL")
|
||||
if dingtalk_home:
|
||||
config.platforms[Platform.DINGTALK].home_channel = HomeChannel(
|
||||
platform=Platform.DINGTALK,
|
||||
chat_id=dingtalk_home,
|
||||
name=os.getenv("DINGTALK_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# Feishu / Lark
|
||||
feishu_app_id = os.getenv("FEISHU_APP_ID")
|
||||
feishu_app_secret = os.getenv("FEISHU_APP_SECRET")
|
||||
@@ -1211,44 +1074,6 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
|
||||
name=os.getenv("BLUEBUBBLES_HOME_CHANNEL_NAME", "Home"),
|
||||
)
|
||||
|
||||
# QQ (Official Bot API v2)
|
||||
qq_app_id = os.getenv("QQ_APP_ID")
|
||||
qq_client_secret = os.getenv("QQ_CLIENT_SECRET")
|
||||
if qq_app_id or qq_client_secret:
|
||||
if Platform.QQBOT not in config.platforms:
|
||||
config.platforms[Platform.QQBOT] = PlatformConfig()
|
||||
config.platforms[Platform.QQBOT].enabled = True
|
||||
extra = config.platforms[Platform.QQBOT].extra
|
||||
if qq_app_id:
|
||||
extra["app_id"] = qq_app_id
|
||||
if qq_client_secret:
|
||||
extra["client_secret"] = qq_client_secret
|
||||
qq_allowed_users = os.getenv("QQ_ALLOWED_USERS", "").strip()
|
||||
if qq_allowed_users:
|
||||
extra["allow_from"] = qq_allowed_users
|
||||
qq_group_allowed = os.getenv("QQ_GROUP_ALLOWED_USERS", "").strip()
|
||||
if qq_group_allowed:
|
||||
extra["group_allow_from"] = qq_group_allowed
|
||||
qq_home = os.getenv("QQBOT_HOME_CHANNEL", "").strip()
|
||||
qq_home_name_env = "QQBOT_HOME_CHANNEL_NAME"
|
||||
if not qq_home:
|
||||
# Back-compat: accept the pre-rename name and log a one-time warning.
|
||||
legacy_home = os.getenv("QQ_HOME_CHANNEL", "").strip()
|
||||
if legacy_home:
|
||||
qq_home = legacy_home
|
||||
qq_home_name_env = "QQ_HOME_CHANNEL_NAME"
|
||||
import logging
|
||||
logging.getLogger(__name__).warning(
|
||||
"QQ_HOME_CHANNEL is deprecated; rename to QQBOT_HOME_CHANNEL "
|
||||
"in your .env for consistency with the platform key."
|
||||
)
|
||||
if qq_home:
|
||||
config.platforms[Platform.QQBOT].home_channel = HomeChannel(
|
||||
platform=Platform.QQBOT,
|
||||
chat_id=qq_home,
|
||||
name=os.getenv("QQBOT_HOME_CHANNEL_NAME") or os.getenv(qq_home_name_env, "Home"),
|
||||
)
|
||||
|
||||
# Session settings
|
||||
idle_minutes = os.getenv("SESSION_IDLE_MINUTES")
|
||||
if idle_minutes:
|
||||
|
||||
+1
-1
@@ -12,7 +12,7 @@ import logging
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from dataclasses import dataclass
|
||||
from typing import Dict, List, Optional, Any
|
||||
from typing import Dict, List, Optional, Any, Union
|
||||
|
||||
from hermes_cli.config import get_hermes_home
|
||||
|
||||
|
||||
+24
-12
@@ -9,10 +9,6 @@ Resolution order (first non-None wins):
|
||||
3. ``_PLATFORM_DEFAULTS[<platform>][<key>]`` — built-in sensible default
|
||||
4. ``_GLOBAL_DEFAULTS[<key>]`` — built-in global default
|
||||
|
||||
Exception: ``display.streaming`` is CLI-only. Gateway streaming follows the
|
||||
top-level ``streaming`` config unless ``display.platforms.<platform>.streaming``
|
||||
sets an explicit per-platform override.
|
||||
|
||||
Backward compatibility: ``display.tool_progress_overrides`` is still read as a
|
||||
fallback for ``tool_progress`` when no ``display.platforms`` entry exists. A
|
||||
config migration (version bump) automatically moves the old format into the new
|
||||
@@ -86,7 +82,7 @@ _PLATFORM_DEFAULTS: dict[str, dict[str, Any]] = {
|
||||
|
||||
# Tier 3 — no edit support, progress messages are permanent
|
||||
"signal": _TIER_LOW,
|
||||
"whatsapp": _TIER_MEDIUM, # Baileys bridge supports /edit
|
||||
"whatsapp": _TIER_LOW,
|
||||
"bluebubbles": _TIER_LOW,
|
||||
"weixin": _TIER_LOW,
|
||||
"wecom": _TIER_LOW,
|
||||
@@ -147,13 +143,10 @@ def resolve_display_setting(
|
||||
if val is not None:
|
||||
return _normalise(setting, val)
|
||||
|
||||
# 2. Global user setting (display.<key>). Skip display.streaming because
|
||||
# that key controls only CLI terminal streaming; gateway token streaming is
|
||||
# governed by the top-level streaming config plus per-platform overrides.
|
||||
if setting != "streaming":
|
||||
val = display_cfg.get(setting)
|
||||
if val is not None:
|
||||
return _normalise(setting, val)
|
||||
# 2. Global user setting (display.<key>)
|
||||
val = display_cfg.get(setting)
|
||||
if val is not None:
|
||||
return _normalise(setting, val)
|
||||
|
||||
# 3. Built-in platform default
|
||||
plat_defaults = _PLATFORM_DEFAULTS.get(platform_key)
|
||||
@@ -170,6 +163,25 @@ def resolve_display_setting(
|
||||
return fallback
|
||||
|
||||
|
||||
def get_platform_defaults(platform_key: str) -> dict[str, Any]:
|
||||
"""Return the built-in default display settings for a platform.
|
||||
|
||||
Falls back to ``_GLOBAL_DEFAULTS`` for unknown platforms.
|
||||
"""
|
||||
return dict(_PLATFORM_DEFAULTS.get(platform_key, _GLOBAL_DEFAULTS))
|
||||
|
||||
|
||||
def get_effective_display(user_config: dict, platform_key: str) -> dict[str, Any]:
|
||||
"""Return the fully-resolved display settings for a platform.
|
||||
|
||||
Useful for status commands that want to show all effective settings.
|
||||
"""
|
||||
return {
|
||||
key: resolve_display_setting(user_config, platform_key, key)
|
||||
for key in OVERRIDEABLE_KEYS
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -9,11 +9,9 @@ Each adapter handles:
|
||||
"""
|
||||
|
||||
from .base import BasePlatformAdapter, MessageEvent, SendResult
|
||||
from .qqbot import QQAdapter
|
||||
|
||||
__all__ = [
|
||||
"BasePlatformAdapter",
|
||||
"MessageEvent",
|
||||
"SendResult",
|
||||
"QQAdapter",
|
||||
]
|
||||
|
||||
+18
-616
@@ -10,7 +10,6 @@ Exposes an HTTP server with endpoints:
|
||||
- POST /v1/runs — start a run, returns run_id immediately (202)
|
||||
- GET /v1/runs/{run_id}/events — SSE stream of structured lifecycle events
|
||||
- GET /health — health check
|
||||
- GET /health/detailed — rich status for cross-container dashboard probing
|
||||
|
||||
Any OpenAI-compatible frontend (Open WebUI, LobeChat, LibreChat,
|
||||
AnythingLLM, NextChat, ChatBox, etc.) can connect to hermes-agent
|
||||
@@ -55,66 +54,6 @@ DEFAULT_PORT = 8642
|
||||
MAX_STORED_RESPONSES = 100
|
||||
MAX_REQUEST_BYTES = 1_000_000 # 1 MB default limit for POST bodies
|
||||
CHAT_COMPLETIONS_SSE_KEEPALIVE_SECONDS = 30.0
|
||||
MAX_NORMALIZED_TEXT_LENGTH = 65_536 # 64 KB cap for normalized content parts
|
||||
MAX_CONTENT_LIST_SIZE = 1_000 # Max items when content is an array
|
||||
|
||||
|
||||
def _normalize_chat_content(
|
||||
content: Any, *, _max_depth: int = 10, _depth: int = 0,
|
||||
) -> str:
|
||||
"""Normalize OpenAI chat message content into a plain text string.
|
||||
|
||||
Some clients (Open WebUI, LobeChat, etc.) send content as an array of
|
||||
typed parts instead of a plain string::
|
||||
|
||||
[{"type": "text", "text": "hello"}, {"type": "input_text", "text": "..."}]
|
||||
|
||||
This function flattens those into a single string so the agent pipeline
|
||||
(which expects strings) doesn't choke.
|
||||
|
||||
Defensive limits prevent abuse: recursion depth, list size, and output
|
||||
length are all bounded.
|
||||
"""
|
||||
if _depth > _max_depth:
|
||||
return ""
|
||||
if content is None:
|
||||
return ""
|
||||
if isinstance(content, str):
|
||||
return content[:MAX_NORMALIZED_TEXT_LENGTH] if len(content) > MAX_NORMALIZED_TEXT_LENGTH else content
|
||||
|
||||
if isinstance(content, list):
|
||||
parts: List[str] = []
|
||||
items = content[:MAX_CONTENT_LIST_SIZE] if len(content) > MAX_CONTENT_LIST_SIZE else content
|
||||
for item in items:
|
||||
if isinstance(item, str):
|
||||
if item:
|
||||
parts.append(item[:MAX_NORMALIZED_TEXT_LENGTH])
|
||||
elif isinstance(item, dict):
|
||||
item_type = str(item.get("type") or "").strip().lower()
|
||||
if item_type in {"text", "input_text", "output_text"}:
|
||||
text = item.get("text", "")
|
||||
if text:
|
||||
try:
|
||||
parts.append(str(text)[:MAX_NORMALIZED_TEXT_LENGTH])
|
||||
except Exception:
|
||||
pass
|
||||
# Silently skip image_url / other non-text parts
|
||||
elif isinstance(item, list):
|
||||
nested = _normalize_chat_content(item, _max_depth=_max_depth, _depth=_depth + 1)
|
||||
if nested:
|
||||
parts.append(nested)
|
||||
# Check accumulated size
|
||||
if sum(len(p) for p in parts) >= MAX_NORMALIZED_TEXT_LENGTH:
|
||||
break
|
||||
result = "\n".join(parts)
|
||||
return result[:MAX_NORMALIZED_TEXT_LENGTH] if len(result) > MAX_NORMALIZED_TEXT_LENGTH else result
|
||||
|
||||
# Fallback for unexpected types (int, float, bool, etc.)
|
||||
try:
|
||||
result = str(content)
|
||||
return result[:MAX_NORMALIZED_TEXT_LENGTH] if len(result) > MAX_NORMALIZED_TEXT_LENGTH else result
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def check_api_server_requirements() -> bool:
|
||||
@@ -515,8 +454,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
session_id: Optional[str] = None,
|
||||
stream_delta_callback=None,
|
||||
tool_progress_callback=None,
|
||||
tool_start_callback=None,
|
||||
tool_complete_callback=None,
|
||||
) -> Any:
|
||||
"""
|
||||
Create an AIAgent instance using the gateway's runtime config.
|
||||
@@ -555,8 +492,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
platform="api_server",
|
||||
stream_delta_callback=stream_delta_callback,
|
||||
tool_progress_callback=tool_progress_callback,
|
||||
tool_start_callback=tool_start_callback,
|
||||
tool_complete_callback=tool_complete_callback,
|
||||
session_db=self._ensure_session_db(),
|
||||
fallback_model=fallback_model,
|
||||
)
|
||||
@@ -570,27 +505,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
"""GET /health — simple health check."""
|
||||
return web.json_response({"status": "ok", "platform": "hermes-agent"})
|
||||
|
||||
async def _handle_health_detailed(self, request: "web.Request") -> "web.Response":
|
||||
"""GET /health/detailed — rich status for cross-container dashboard probing.
|
||||
|
||||
Returns gateway state, connected platforms, PID, and uptime so the
|
||||
dashboard can display full status without needing a shared PID file or
|
||||
/proc access. No authentication required.
|
||||
"""
|
||||
from gateway.status import read_runtime_status
|
||||
|
||||
runtime = read_runtime_status() or {}
|
||||
return web.json_response({
|
||||
"status": "ok",
|
||||
"platform": "hermes-agent",
|
||||
"gateway_state": runtime.get("gateway_state"),
|
||||
"platforms": runtime.get("platforms", {}),
|
||||
"active_agents": runtime.get("active_agents", 0),
|
||||
"exit_reason": runtime.get("exit_reason"),
|
||||
"updated_at": runtime.get("updated_at"),
|
||||
"pid": os.getpid(),
|
||||
})
|
||||
|
||||
async def _handle_models(self, request: "web.Request") -> "web.Response":
|
||||
"""GET /v1/models — return hermes-agent as an available model."""
|
||||
auth_err = self._check_auth(request)
|
||||
@@ -639,7 +553,7 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
|
||||
for msg in messages:
|
||||
role = msg.get("role", "")
|
||||
content = _normalize_chat_content(msg.get("content", ""))
|
||||
content = msg.get("content", "")
|
||||
if role == "system":
|
||||
# Accumulate system messages
|
||||
if system_prompt is None:
|
||||
@@ -902,7 +816,7 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
return time.monotonic()
|
||||
|
||||
# Stream content chunks as they arrive from the agent
|
||||
loop = asyncio.get_running_loop()
|
||||
loop = asyncio.get_event_loop()
|
||||
while True:
|
||||
try:
|
||||
delta = await loop.run_in_executor(None, lambda: stream_q.get(timeout=0.5))
|
||||
@@ -969,427 +883,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
|
||||
return response
|
||||
|
||||
async def _write_sse_responses(
|
||||
self,
|
||||
request: "web.Request",
|
||||
response_id: str,
|
||||
model: str,
|
||||
created_at: int,
|
||||
stream_q,
|
||||
agent_task,
|
||||
agent_ref,
|
||||
conversation_history: List[Dict[str, str]],
|
||||
user_message: str,
|
||||
instructions: Optional[str],
|
||||
conversation: Optional[str],
|
||||
store: bool,
|
||||
session_id: str,
|
||||
) -> "web.StreamResponse":
|
||||
"""Write an SSE stream for POST /v1/responses (OpenAI Responses API).
|
||||
|
||||
Emits spec-compliant event types as the agent runs:
|
||||
|
||||
- ``response.created`` — initial envelope (status=in_progress)
|
||||
- ``response.output_text.delta`` / ``response.output_text.done`` —
|
||||
streamed assistant text
|
||||
- ``response.output_item.added`` / ``response.output_item.done``
|
||||
with ``item.type == "function_call"`` — when the agent invokes a
|
||||
tool (both events fire; the ``done`` event carries the finalized
|
||||
``arguments`` string)
|
||||
- ``response.output_item.added`` with
|
||||
``item.type == "function_call_output"`` — tool result with
|
||||
``{call_id, output, status}``
|
||||
- ``response.completed`` — terminal event carrying the full
|
||||
response object with all output items + usage (same payload
|
||||
shape as the non-streaming path for parity)
|
||||
- ``response.failed`` — terminal event on agent error
|
||||
|
||||
If the client disconnects mid-stream, ``agent.interrupt()`` is
|
||||
called so the agent stops issuing upstream LLM calls, then the
|
||||
asyncio task is cancelled. When ``store=True`` the full response
|
||||
is persisted to the ResponseStore in a ``finally`` block so GET
|
||||
/v1/responses/{id} and ``previous_response_id`` chaining work the
|
||||
same as the batch path.
|
||||
"""
|
||||
import queue as _q
|
||||
|
||||
sse_headers = {
|
||||
"Content-Type": "text/event-stream",
|
||||
"Cache-Control": "no-cache",
|
||||
"X-Accel-Buffering": "no",
|
||||
}
|
||||
origin = request.headers.get("Origin", "")
|
||||
cors = self._cors_headers_for_origin(origin) if origin else None
|
||||
if cors:
|
||||
sse_headers.update(cors)
|
||||
if session_id:
|
||||
sse_headers["X-Hermes-Session-Id"] = session_id
|
||||
response = web.StreamResponse(status=200, headers=sse_headers)
|
||||
await response.prepare(request)
|
||||
|
||||
# State accumulated during the stream
|
||||
final_text_parts: List[str] = []
|
||||
# Track open function_call items by name so we can emit a matching
|
||||
# ``done`` event when the tool completes. Order preserved.
|
||||
pending_tool_calls: List[Dict[str, Any]] = []
|
||||
# Output items we've emitted so far (used to build the terminal
|
||||
# response.completed payload). Kept in the order they appeared.
|
||||
emitted_items: List[Dict[str, Any]] = []
|
||||
# Monotonic counter for output_index (spec requires it).
|
||||
output_index = 0
|
||||
# Monotonic counter for call_id generation if the agent doesn't
|
||||
# provide one (it doesn't, from tool_progress_callback).
|
||||
call_counter = 0
|
||||
# Canonical Responses SSE events include a monotonically increasing
|
||||
# sequence_number. Add it server-side for every emitted event so
|
||||
# clients that validate the OpenAI event schema can parse our stream.
|
||||
sequence_number = 0
|
||||
# Track the assistant message item id + content index for text
|
||||
# delta events — the spec ties deltas to a specific item.
|
||||
message_item_id = f"msg_{uuid.uuid4().hex[:24]}"
|
||||
message_output_index: Optional[int] = None
|
||||
message_opened = False
|
||||
|
||||
async def _write_event(event_type: str, data: Dict[str, Any]) -> None:
|
||||
nonlocal sequence_number
|
||||
if "sequence_number" not in data:
|
||||
data["sequence_number"] = sequence_number
|
||||
sequence_number += 1
|
||||
payload = f"event: {event_type}\ndata: {json.dumps(data)}\n\n"
|
||||
await response.write(payload.encode())
|
||||
|
||||
def _envelope(status: str) -> Dict[str, Any]:
|
||||
env: Dict[str, Any] = {
|
||||
"id": response_id,
|
||||
"object": "response",
|
||||
"status": status,
|
||||
"created_at": created_at,
|
||||
"model": model,
|
||||
}
|
||||
return env
|
||||
|
||||
final_response_text = ""
|
||||
agent_error: Optional[str] = None
|
||||
usage: Dict[str, int] = {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0}
|
||||
|
||||
try:
|
||||
# response.created — initial envelope, status=in_progress
|
||||
created_env = _envelope("in_progress")
|
||||
created_env["output"] = []
|
||||
await _write_event("response.created", {
|
||||
"type": "response.created",
|
||||
"response": created_env,
|
||||
})
|
||||
last_activity = time.monotonic()
|
||||
|
||||
async def _open_message_item() -> None:
|
||||
"""Emit response.output_item.added for the assistant message
|
||||
the first time any text delta arrives."""
|
||||
nonlocal message_opened, message_output_index, output_index
|
||||
if message_opened:
|
||||
return
|
||||
message_opened = True
|
||||
message_output_index = output_index
|
||||
output_index += 1
|
||||
item = {
|
||||
"id": message_item_id,
|
||||
"type": "message",
|
||||
"status": "in_progress",
|
||||
"role": "assistant",
|
||||
"content": [],
|
||||
}
|
||||
await _write_event("response.output_item.added", {
|
||||
"type": "response.output_item.added",
|
||||
"output_index": message_output_index,
|
||||
"item": item,
|
||||
})
|
||||
|
||||
async def _emit_text_delta(delta_text: str) -> None:
|
||||
await _open_message_item()
|
||||
final_text_parts.append(delta_text)
|
||||
await _write_event("response.output_text.delta", {
|
||||
"type": "response.output_text.delta",
|
||||
"item_id": message_item_id,
|
||||
"output_index": message_output_index,
|
||||
"content_index": 0,
|
||||
"delta": delta_text,
|
||||
"logprobs": [],
|
||||
})
|
||||
|
||||
async def _emit_tool_started(payload: Dict[str, Any]) -> str:
|
||||
"""Emit response.output_item.added for a function_call.
|
||||
|
||||
Returns the call_id so the matching completion event can
|
||||
reference it. Prefer the real ``tool_call_id`` from the
|
||||
agent when available; fall back to a generated call id for
|
||||
safety in tests or older code paths.
|
||||
"""
|
||||
nonlocal output_index, call_counter
|
||||
call_counter += 1
|
||||
call_id = payload.get("tool_call_id") or f"call_{response_id[5:]}_{call_counter}"
|
||||
args = payload.get("arguments", {})
|
||||
if isinstance(args, dict):
|
||||
arguments_str = json.dumps(args)
|
||||
else:
|
||||
arguments_str = str(args)
|
||||
item = {
|
||||
"id": f"fc_{uuid.uuid4().hex[:24]}",
|
||||
"type": "function_call",
|
||||
"status": "in_progress",
|
||||
"name": payload.get("name", ""),
|
||||
"call_id": call_id,
|
||||
"arguments": arguments_str,
|
||||
}
|
||||
idx = output_index
|
||||
output_index += 1
|
||||
pending_tool_calls.append({
|
||||
"call_id": call_id,
|
||||
"name": payload.get("name", ""),
|
||||
"arguments": arguments_str,
|
||||
"item_id": item["id"],
|
||||
"output_index": idx,
|
||||
})
|
||||
emitted_items.append({
|
||||
"type": "function_call",
|
||||
"name": payload.get("name", ""),
|
||||
"arguments": arguments_str,
|
||||
"call_id": call_id,
|
||||
})
|
||||
await _write_event("response.output_item.added", {
|
||||
"type": "response.output_item.added",
|
||||
"output_index": idx,
|
||||
"item": item,
|
||||
})
|
||||
return call_id
|
||||
|
||||
async def _emit_tool_completed(payload: Dict[str, Any]) -> None:
|
||||
"""Emit response.output_item.done (function_call) followed
|
||||
by response.output_item.added (function_call_output)."""
|
||||
nonlocal output_index
|
||||
call_id = payload.get("tool_call_id")
|
||||
result = payload.get("result", "")
|
||||
pending = None
|
||||
if call_id:
|
||||
for i, p in enumerate(pending_tool_calls):
|
||||
if p["call_id"] == call_id:
|
||||
pending = pending_tool_calls.pop(i)
|
||||
break
|
||||
if pending is None:
|
||||
# Completion without a matching start — skip to avoid
|
||||
# emitting orphaned done events.
|
||||
return
|
||||
|
||||
# function_call done
|
||||
done_item = {
|
||||
"id": pending["item_id"],
|
||||
"type": "function_call",
|
||||
"status": "completed",
|
||||
"name": pending["name"],
|
||||
"call_id": pending["call_id"],
|
||||
"arguments": pending["arguments"],
|
||||
}
|
||||
await _write_event("response.output_item.done", {
|
||||
"type": "response.output_item.done",
|
||||
"output_index": pending["output_index"],
|
||||
"item": done_item,
|
||||
})
|
||||
|
||||
# function_call_output added (result)
|
||||
result_str = result if isinstance(result, str) else json.dumps(result)
|
||||
output_parts = [{"type": "input_text", "text": result_str}]
|
||||
output_item = {
|
||||
"id": f"fco_{uuid.uuid4().hex[:24]}",
|
||||
"type": "function_call_output",
|
||||
"call_id": pending["call_id"],
|
||||
"output": output_parts,
|
||||
"status": "completed",
|
||||
}
|
||||
idx = output_index
|
||||
output_index += 1
|
||||
emitted_items.append({
|
||||
"type": "function_call_output",
|
||||
"call_id": pending["call_id"],
|
||||
"output": output_parts,
|
||||
})
|
||||
await _write_event("response.output_item.added", {
|
||||
"type": "response.output_item.added",
|
||||
"output_index": idx,
|
||||
"item": output_item,
|
||||
})
|
||||
await _write_event("response.output_item.done", {
|
||||
"type": "response.output_item.done",
|
||||
"output_index": idx,
|
||||
"item": output_item,
|
||||
})
|
||||
|
||||
# Main drain loop — thread-safe queue fed by agent callbacks.
|
||||
async def _dispatch(it) -> None:
|
||||
"""Route a queue item to the correct SSE emitter.
|
||||
|
||||
Plain strings are text deltas. Tagged tuples with
|
||||
``__tool_started__`` / ``__tool_completed__`` prefixes
|
||||
are tool lifecycle events.
|
||||
"""
|
||||
if isinstance(it, tuple) and len(it) == 2 and isinstance(it[0], str):
|
||||
tag, payload = it
|
||||
if tag == "__tool_started__":
|
||||
await _emit_tool_started(payload)
|
||||
elif tag == "__tool_completed__":
|
||||
await _emit_tool_completed(payload)
|
||||
# Unknown tags are silently ignored (forward-compat).
|
||||
elif isinstance(it, str):
|
||||
await _emit_text_delta(it)
|
||||
# Other types (non-string, non-tuple) are silently dropped.
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
while True:
|
||||
try:
|
||||
item = await loop.run_in_executor(None, lambda: stream_q.get(timeout=0.5))
|
||||
except _q.Empty:
|
||||
if agent_task.done():
|
||||
# Drain remaining
|
||||
while True:
|
||||
try:
|
||||
item = stream_q.get_nowait()
|
||||
if item is None:
|
||||
break
|
||||
await _dispatch(item)
|
||||
last_activity = time.monotonic()
|
||||
except _q.Empty:
|
||||
break
|
||||
break
|
||||
if time.monotonic() - last_activity >= CHAT_COMPLETIONS_SSE_KEEPALIVE_SECONDS:
|
||||
await response.write(b": keepalive\n\n")
|
||||
last_activity = time.monotonic()
|
||||
continue
|
||||
|
||||
if item is None: # EOS sentinel
|
||||
break
|
||||
|
||||
await _dispatch(item)
|
||||
last_activity = time.monotonic()
|
||||
|
||||
# Pick up agent result + usage from the completed task
|
||||
try:
|
||||
result, agent_usage = await agent_task
|
||||
usage = agent_usage or usage
|
||||
# If the agent produced a final_response but no text
|
||||
# deltas were streamed (e.g. some providers only emit
|
||||
# the full response at the end), emit a single fallback
|
||||
# delta so Responses clients still receive a live text part.
|
||||
agent_final = result.get("final_response", "") if isinstance(result, dict) else ""
|
||||
if agent_final and not final_text_parts:
|
||||
await _emit_text_delta(agent_final)
|
||||
if agent_final and not final_response_text:
|
||||
final_response_text = agent_final
|
||||
if isinstance(result, dict) and result.get("error") and not final_response_text:
|
||||
agent_error = result["error"]
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.error("Error running agent for streaming responses: %s", e, exc_info=True)
|
||||
agent_error = str(e)
|
||||
|
||||
# Close the message item if it was opened
|
||||
final_response_text = "".join(final_text_parts) or final_response_text
|
||||
if message_opened:
|
||||
await _write_event("response.output_text.done", {
|
||||
"type": "response.output_text.done",
|
||||
"item_id": message_item_id,
|
||||
"output_index": message_output_index,
|
||||
"content_index": 0,
|
||||
"text": final_response_text,
|
||||
"logprobs": [],
|
||||
})
|
||||
msg_done_item = {
|
||||
"id": message_item_id,
|
||||
"type": "message",
|
||||
"status": "completed",
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "output_text", "text": final_response_text}
|
||||
],
|
||||
}
|
||||
await _write_event("response.output_item.done", {
|
||||
"type": "response.output_item.done",
|
||||
"output_index": message_output_index,
|
||||
"item": msg_done_item,
|
||||
})
|
||||
|
||||
# Always append a final message item in the completed
|
||||
# response envelope so clients that only parse the terminal
|
||||
# payload still see the assistant text. This mirrors the
|
||||
# shape produced by _extract_output_items in the batch path.
|
||||
final_items: List[Dict[str, Any]] = list(emitted_items)
|
||||
final_items.append({
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "output_text", "text": final_response_text or (agent_error or "")}
|
||||
],
|
||||
})
|
||||
|
||||
if agent_error:
|
||||
failed_env = _envelope("failed")
|
||||
failed_env["output"] = final_items
|
||||
failed_env["error"] = {"message": agent_error, "type": "server_error"}
|
||||
failed_env["usage"] = {
|
||||
"input_tokens": usage.get("input_tokens", 0),
|
||||
"output_tokens": usage.get("output_tokens", 0),
|
||||
"total_tokens": usage.get("total_tokens", 0),
|
||||
}
|
||||
await _write_event("response.failed", {
|
||||
"type": "response.failed",
|
||||
"response": failed_env,
|
||||
})
|
||||
else:
|
||||
completed_env = _envelope("completed")
|
||||
completed_env["output"] = final_items
|
||||
completed_env["usage"] = {
|
||||
"input_tokens": usage.get("input_tokens", 0),
|
||||
"output_tokens": usage.get("output_tokens", 0),
|
||||
"total_tokens": usage.get("total_tokens", 0),
|
||||
}
|
||||
await _write_event("response.completed", {
|
||||
"type": "response.completed",
|
||||
"response": completed_env,
|
||||
})
|
||||
|
||||
# Persist for future chaining / GET retrieval, mirroring
|
||||
# the batch path behavior.
|
||||
if store:
|
||||
full_history = list(conversation_history)
|
||||
full_history.append({"role": "user", "content": user_message})
|
||||
if isinstance(result, dict) and result.get("messages"):
|
||||
full_history.extend(result["messages"])
|
||||
else:
|
||||
full_history.append({"role": "assistant", "content": final_response_text})
|
||||
self._response_store.put(response_id, {
|
||||
"response": completed_env,
|
||||
"conversation_history": full_history,
|
||||
"instructions": instructions,
|
||||
"session_id": session_id,
|
||||
})
|
||||
if conversation:
|
||||
self._response_store.set_conversation(conversation, response_id)
|
||||
|
||||
except (ConnectionResetError, ConnectionAbortedError, BrokenPipeError, OSError):
|
||||
# Client disconnected — interrupt the agent so it stops
|
||||
# making upstream LLM calls, then cancel the task.
|
||||
agent = agent_ref[0] if agent_ref else None
|
||||
if agent is not None:
|
||||
try:
|
||||
agent.interrupt("SSE client disconnected")
|
||||
except Exception:
|
||||
pass
|
||||
if not agent_task.done():
|
||||
agent_task.cancel()
|
||||
try:
|
||||
await agent_task
|
||||
except (asyncio.CancelledError, Exception):
|
||||
pass
|
||||
logger.info("SSE client disconnected; interrupted agent task %s", response_id)
|
||||
|
||||
return response
|
||||
|
||||
async def _handle_responses(self, request: "web.Request") -> "web.Response":
|
||||
"""POST /v1/responses — OpenAI Responses API format."""
|
||||
auth_err = self._check_auth(request)
|
||||
@@ -1433,7 +926,18 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
input_messages.append({"role": "user", "content": item})
|
||||
elif isinstance(item, dict):
|
||||
role = item.get("role", "user")
|
||||
content = _normalize_chat_content(item.get("content", ""))
|
||||
content = item.get("content", "")
|
||||
# Handle content that may be a list of content parts
|
||||
if isinstance(content, list):
|
||||
text_parts = []
|
||||
for part in content:
|
||||
if isinstance(part, dict) and part.get("type") == "input_text":
|
||||
text_parts.append(part.get("text", ""))
|
||||
elif isinstance(part, dict) and part.get("type") == "output_text":
|
||||
text_parts.append(part.get("text", ""))
|
||||
elif isinstance(part, str):
|
||||
text_parts.append(part)
|
||||
content = "\n".join(text_parts)
|
||||
input_messages.append({"role": role, "content": content})
|
||||
else:
|
||||
return web.json_response(_openai_error("'input' must be a string or array"), status=400)
|
||||
@@ -1460,13 +964,11 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
if previous_response_id:
|
||||
logger.debug("Both conversation_history and previous_response_id provided; using conversation_history")
|
||||
|
||||
stored_session_id = None
|
||||
if not conversation_history and previous_response_id:
|
||||
stored = self._response_store.get(previous_response_id)
|
||||
if stored is None:
|
||||
return web.json_response(_openai_error(f"Previous response not found: {previous_response_id}"), status=404)
|
||||
conversation_history = list(stored.get("conversation_history", []))
|
||||
stored_session_id = stored.get("session_id")
|
||||
# If no instructions provided, carry forward from previous
|
||||
if instructions is None:
|
||||
instructions = stored.get("instructions")
|
||||
@@ -1484,83 +986,8 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
if body.get("truncation") == "auto" and len(conversation_history) > 100:
|
||||
conversation_history = conversation_history[-100:]
|
||||
|
||||
# Reuse session from previous_response_id chain so the dashboard
|
||||
# groups the entire conversation under one session entry.
|
||||
session_id = stored_session_id or str(uuid.uuid4())
|
||||
|
||||
stream = bool(body.get("stream", False))
|
||||
if stream:
|
||||
# Streaming branch — emit OpenAI Responses SSE events as the
|
||||
# agent runs so frontends can render text deltas and tool
|
||||
# calls in real time. See _write_sse_responses for details.
|
||||
import queue as _q
|
||||
_stream_q: _q.Queue = _q.Queue()
|
||||
|
||||
def _on_delta(delta):
|
||||
# None from the agent is a CLI box-close signal, not EOS.
|
||||
# Forwarding would kill the SSE stream prematurely; the
|
||||
# SSE writer detects completion via agent_task.done().
|
||||
if delta is not None:
|
||||
_stream_q.put(delta)
|
||||
|
||||
def _on_tool_progress(event_type, name, preview, args, **kwargs):
|
||||
"""Queue non-start tool progress events if needed in future.
|
||||
|
||||
The structured Responses stream uses ``tool_start_callback``
|
||||
and ``tool_complete_callback`` for exact call-id correlation,
|
||||
so progress events are currently ignored here.
|
||||
"""
|
||||
return
|
||||
|
||||
def _on_tool_start(tool_call_id, function_name, function_args):
|
||||
"""Queue a started tool for live function_call streaming."""
|
||||
_stream_q.put(("__tool_started__", {
|
||||
"tool_call_id": tool_call_id,
|
||||
"name": function_name,
|
||||
"arguments": function_args or {},
|
||||
}))
|
||||
|
||||
def _on_tool_complete(tool_call_id, function_name, function_args, function_result):
|
||||
"""Queue a completed tool result for live function_call_output streaming."""
|
||||
_stream_q.put(("__tool_completed__", {
|
||||
"tool_call_id": tool_call_id,
|
||||
"name": function_name,
|
||||
"arguments": function_args or {},
|
||||
"result": function_result,
|
||||
}))
|
||||
|
||||
agent_ref = [None]
|
||||
agent_task = asyncio.ensure_future(self._run_agent(
|
||||
user_message=user_message,
|
||||
conversation_history=conversation_history,
|
||||
ephemeral_system_prompt=instructions,
|
||||
session_id=session_id,
|
||||
stream_delta_callback=_on_delta,
|
||||
tool_progress_callback=_on_tool_progress,
|
||||
tool_start_callback=_on_tool_start,
|
||||
tool_complete_callback=_on_tool_complete,
|
||||
agent_ref=agent_ref,
|
||||
))
|
||||
|
||||
response_id = f"resp_{uuid.uuid4().hex[:28]}"
|
||||
model_name = body.get("model", self._model_name)
|
||||
created_at = int(time.time())
|
||||
|
||||
return await self._write_sse_responses(
|
||||
request=request,
|
||||
response_id=response_id,
|
||||
model=model_name,
|
||||
created_at=created_at,
|
||||
stream_q=_stream_q,
|
||||
agent_task=agent_task,
|
||||
agent_ref=agent_ref,
|
||||
conversation_history=conversation_history,
|
||||
user_message=user_message,
|
||||
instructions=instructions,
|
||||
conversation=conversation,
|
||||
store=store,
|
||||
session_id=session_id,
|
||||
)
|
||||
# Run the agent (with Idempotency-Key support)
|
||||
session_id = str(uuid.uuid4())
|
||||
|
||||
async def _compute_response():
|
||||
return await self._run_agent(
|
||||
@@ -1635,7 +1062,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
"response": response_data,
|
||||
"conversation_history": full_history,
|
||||
"instructions": instructions,
|
||||
"session_id": session_id,
|
||||
})
|
||||
# Update conversation mapping so the next request with the same
|
||||
# conversation name automatically chains to this response
|
||||
@@ -1989,8 +1415,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
session_id: Optional[str] = None,
|
||||
stream_delta_callback=None,
|
||||
tool_progress_callback=None,
|
||||
tool_start_callback=None,
|
||||
tool_complete_callback=None,
|
||||
agent_ref: Optional[list] = None,
|
||||
) -> tuple:
|
||||
"""
|
||||
@@ -2004,7 +1428,7 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
callers (e.g. the SSE writer) to call ``agent.interrupt()`` from
|
||||
another thread to stop in-progress LLM calls.
|
||||
"""
|
||||
loop = asyncio.get_running_loop()
|
||||
loop = asyncio.get_event_loop()
|
||||
|
||||
def _run():
|
||||
agent = self._create_agent(
|
||||
@@ -2012,8 +1436,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
session_id=session_id,
|
||||
stream_delta_callback=stream_delta_callback,
|
||||
tool_progress_callback=tool_progress_callback,
|
||||
tool_start_callback=tool_start_callback,
|
||||
tool_complete_callback=tool_complete_callback,
|
||||
)
|
||||
if agent_ref is not None:
|
||||
agent_ref[0] = agent
|
||||
@@ -2150,12 +1572,10 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
if previous_response_id:
|
||||
logger.debug("Both conversation_history and previous_response_id provided; using conversation_history")
|
||||
|
||||
stored_session_id = None
|
||||
if not conversation_history and previous_response_id:
|
||||
stored = self._response_store.get(previous_response_id)
|
||||
if stored:
|
||||
conversation_history = list(stored.get("conversation_history", []))
|
||||
stored_session_id = stored.get("session_id")
|
||||
if instructions is None:
|
||||
instructions = stored.get("instructions")
|
||||
|
||||
@@ -2174,7 +1594,7 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
)
|
||||
conversation_history.append({"role": msg["role"], "content": str(content)})
|
||||
|
||||
session_id = body.get("session_id") or stored_session_id or run_id
|
||||
session_id = body.get("session_id") or run_id
|
||||
ephemeral_system_prompt = instructions
|
||||
|
||||
async def _run_and_close():
|
||||
@@ -2314,7 +1734,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
self._app = web.Application(middlewares=mws)
|
||||
self._app["api_server_adapter"] = self
|
||||
self._app.router.add_get("/health", self._handle_health)
|
||||
self._app.router.add_get("/health/detailed", self._handle_health_detailed)
|
||||
self._app.router.add_get("/v1/health", self._handle_health)
|
||||
self._app.router.add_get("/v1/models", self._handle_models)
|
||||
self._app.router.add_post("/v1/chat/completions", self._handle_chat_completions)
|
||||
@@ -2351,23 +1770,6 @@ class APIServerAdapter(BasePlatformAdapter):
|
||||
)
|
||||
return False
|
||||
|
||||
# Refuse to start network-accessible with a placeholder key.
|
||||
# Ported from openclaw/openclaw#64586.
|
||||
if is_network_accessible(self._host) and self._api_key:
|
||||
try:
|
||||
from hermes_cli.auth import has_usable_secret
|
||||
if not has_usable_secret(self._api_key, min_length=8):
|
||||
logger.error(
|
||||
"[%s] Refusing to start: API_SERVER_KEY is set to a "
|
||||
"placeholder value. Generate a real secret "
|
||||
"(e.g. `openssl rand -hex 32`) and set API_SERVER_KEY "
|
||||
"before exposing the API server on %s.",
|
||||
self.name, self._host,
|
||||
)
|
||||
return False
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# Port conflict detection — fail fast if port is already in use
|
||||
try:
|
||||
with _socket.socket(_socket.AF_INET, _socket.SOCK_STREAM) as _s:
|
||||
|
||||
+23
-226
@@ -21,59 +21,6 @@ from urllib.parse import urlsplit
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def utf16_len(s: str) -> int:
|
||||
"""Count UTF-16 code units in *s*.
|
||||
|
||||
Telegram's message-length limit (4 096) is measured in UTF-16 code units,
|
||||
**not** Unicode code-points. Characters outside the Basic Multilingual
|
||||
Plane (emoji like 😀, CJK Extension B, musical symbols, …) are encoded as
|
||||
surrogate pairs and therefore consume **two** UTF-16 code units each, even
|
||||
though Python's ``len()`` counts them as one.
|
||||
|
||||
Ported from nearai/ironclaw#2304 which discovered the same discrepancy in
|
||||
Rust's ``chars().count()``.
|
||||
"""
|
||||
return len(s.encode("utf-16-le")) // 2
|
||||
|
||||
|
||||
def _prefix_within_utf16_limit(s: str, limit: int) -> str:
|
||||
"""Return the longest prefix of *s* whose UTF-16 length ≤ *limit*.
|
||||
|
||||
Unlike a plain ``s[:limit]``, this respects surrogate-pair boundaries so
|
||||
we never slice a multi-code-unit character in half.
|
||||
"""
|
||||
if utf16_len(s) <= limit:
|
||||
return s
|
||||
# Binary search for the longest safe prefix
|
||||
lo, hi = 0, len(s)
|
||||
while lo < hi:
|
||||
mid = (lo + hi + 1) // 2
|
||||
if utf16_len(s[:mid]) <= limit:
|
||||
lo = mid
|
||||
else:
|
||||
hi = mid - 1
|
||||
return s[:lo]
|
||||
|
||||
|
||||
def _custom_unit_to_cp(s: str, budget: int, len_fn) -> int:
|
||||
"""Return the largest codepoint offset *n* such that ``len_fn(s[:n]) <= budget``.
|
||||
|
||||
Used by :meth:`BasePlatformAdapter.truncate_message` when *len_fn* measures
|
||||
length in units different from Python codepoints (e.g. UTF-16 code units).
|
||||
Falls back to binary search which is O(log n) calls to *len_fn*.
|
||||
"""
|
||||
if len_fn(s) <= budget:
|
||||
return len(s)
|
||||
lo, hi = 0, len(s)
|
||||
while lo < hi:
|
||||
mid = (lo + hi + 1) // 2
|
||||
if len_fn(s[:mid]) <= budget:
|
||||
lo = mid
|
||||
else:
|
||||
hi = mid - 1
|
||||
return lo
|
||||
|
||||
|
||||
def is_network_accessible(host: str) -> bool:
|
||||
"""Return True if *host* would expose the server beyond loopback.
|
||||
|
||||
@@ -669,15 +616,6 @@ class MessageEvent:
|
||||
# Original platform data
|
||||
raw_message: Any = None
|
||||
message_id: Optional[str] = None
|
||||
|
||||
# Platform-specific update identifier. For Telegram this is the
|
||||
# ``update_id`` from the PTB Update wrapper; other platforms currently
|
||||
# ignore it. Used by ``/restart`` to record the triggering update so the
|
||||
# new gateway can advance the Telegram offset past it and avoid processing
|
||||
# the same ``/restart`` twice if PTB's graceful-shutdown ACK times out
|
||||
# ("Error while calling `get_updates` one more time to mark all fetched
|
||||
# updates" in gateway.log).
|
||||
platform_update_id: Optional[int] = None
|
||||
|
||||
# Media attachments
|
||||
# media_urls: local file paths (for vision tool access)
|
||||
@@ -691,10 +629,6 @@ class MessageEvent:
|
||||
# Auto-loaded skill(s) for topic/channel bindings (e.g., Telegram DM Topics,
|
||||
# Discord channel_skill_bindings). A single name or ordered list.
|
||||
auto_skill: Optional[str | list[str]] = None
|
||||
|
||||
# Per-channel ephemeral system prompt (e.g. Discord channel_prompts).
|
||||
# Applied at API call time and never persisted to transcript history.
|
||||
channel_prompt: Optional[str] = None
|
||||
|
||||
# Internal flag — set for synthetic events (e.g. background process
|
||||
# completion notifications) that must bypass user authorization checks.
|
||||
@@ -743,56 +677,25 @@ def merge_pending_message_event(
|
||||
pending_messages: Dict[str, MessageEvent],
|
||||
session_key: str,
|
||||
event: MessageEvent,
|
||||
*,
|
||||
merge_text: bool = False,
|
||||
) -> None:
|
||||
"""Store or merge a pending event for a session.
|
||||
|
||||
Photo bursts/albums often arrive as multiple near-simultaneous PHOTO
|
||||
events. Merge those into the existing queued event so the next turn sees
|
||||
the whole burst.
|
||||
|
||||
When ``merge_text`` is enabled, rapid follow-up TEXT events are appended
|
||||
instead of replacing the pending turn. This is used for Telegram bursty
|
||||
follow-ups so a multi-part user thought is not silently truncated to only
|
||||
the last queued fragment.
|
||||
the whole burst, while non-photo follow-ups still replace the pending
|
||||
event normally.
|
||||
"""
|
||||
existing = pending_messages.get(session_key)
|
||||
if existing:
|
||||
existing_is_photo = getattr(existing, "message_type", None) == MessageType.PHOTO
|
||||
incoming_is_photo = event.message_type == MessageType.PHOTO
|
||||
existing_has_media = bool(existing.media_urls)
|
||||
incoming_has_media = bool(event.media_urls)
|
||||
|
||||
if existing_is_photo and incoming_is_photo:
|
||||
existing.media_urls.extend(event.media_urls)
|
||||
existing.media_types.extend(event.media_types)
|
||||
if event.text:
|
||||
existing.text = BasePlatformAdapter._merge_caption(existing.text, event.text)
|
||||
return
|
||||
|
||||
if existing_has_media or incoming_has_media:
|
||||
if incoming_has_media:
|
||||
existing.media_urls.extend(event.media_urls)
|
||||
existing.media_types.extend(event.media_types)
|
||||
if event.text:
|
||||
if existing.text:
|
||||
existing.text = BasePlatformAdapter._merge_caption(existing.text, event.text)
|
||||
else:
|
||||
existing.text = event.text
|
||||
if existing_is_photo or incoming_is_photo:
|
||||
existing.message_type = MessageType.PHOTO
|
||||
return
|
||||
|
||||
if (
|
||||
merge_text
|
||||
and getattr(existing, "message_type", None) == MessageType.TEXT
|
||||
and event.message_type == MessageType.TEXT
|
||||
):
|
||||
if event.text:
|
||||
existing.text = f"{existing.text}\n{event.text}" if existing.text else event.text
|
||||
return
|
||||
|
||||
if (
|
||||
existing
|
||||
and getattr(existing, "message_type", None) == MessageType.PHOTO
|
||||
and event.message_type == MessageType.PHOTO
|
||||
):
|
||||
existing.media_urls.extend(event.media_urls)
|
||||
existing.media_types.extend(event.media_types)
|
||||
if event.text:
|
||||
existing.text = BasePlatformAdapter._merge_caption(existing.text, event.text)
|
||||
return
|
||||
pending_messages[session_key] = event
|
||||
|
||||
|
||||
@@ -820,36 +723,6 @@ _RETRYABLE_ERROR_PATTERNS = (
|
||||
MessageHandler = Callable[[MessageEvent], Awaitable[Optional[str]]]
|
||||
|
||||
|
||||
def resolve_channel_prompt(
|
||||
config_extra: dict,
|
||||
channel_id: str,
|
||||
parent_id: str | None = None,
|
||||
) -> str | None:
|
||||
"""Resolve a per-channel ephemeral prompt from platform config.
|
||||
|
||||
Looks up ``channel_prompts`` in the adapter's ``config.extra`` dict.
|
||||
Prefers an exact match on *channel_id*; falls back to *parent_id*
|
||||
(useful for forum threads / child channels inheriting a parent prompt).
|
||||
|
||||
Returns the prompt string, or None if no match is found. Blank/whitespace-
|
||||
only prompts are treated as absent.
|
||||
"""
|
||||
prompts = config_extra.get("channel_prompts") or {}
|
||||
if not isinstance(prompts, dict):
|
||||
return None
|
||||
|
||||
for key in (channel_id, parent_id):
|
||||
if not key:
|
||||
continue
|
||||
prompt = prompts.get(key)
|
||||
if prompt is None:
|
||||
continue
|
||||
prompt = str(prompt).strip()
|
||||
if prompt:
|
||||
return prompt
|
||||
return None
|
||||
|
||||
|
||||
class BasePlatformAdapter(ABC):
|
||||
"""
|
||||
Base class for platform adapters.
|
||||
@@ -879,11 +752,6 @@ class BasePlatformAdapter(ABC):
|
||||
# Gateway shutdown cancels these so an old gateway instance doesn't keep
|
||||
# working on a task after --replace or manual restarts.
|
||||
self._background_tasks: set[asyncio.Task] = set()
|
||||
# One-shot callbacks to fire after the main response is delivered.
|
||||
# Keyed by session_key. GatewayRunner uses this to defer
|
||||
# background-review notifications ("💾 Skill created") until the
|
||||
# primary reply has been sent.
|
||||
self._post_delivery_callbacks: Dict[str, Callable] = {}
|
||||
self._expected_cancelled_tasks: set[asyncio.Task] = set()
|
||||
self._busy_session_handler: Optional[Callable[[MessageEvent, str], Awaitable[bool]]] = None
|
||||
# Chats where auto-TTS on voice input is disabled (set by /voice off)
|
||||
@@ -1054,40 +922,16 @@ class BasePlatformAdapter(ABC):
|
||||
"""
|
||||
pass
|
||||
|
||||
# Default: the adapter treats ``finalize=True`` on edit_message as a
|
||||
# no-op and is happy to have the stream consumer skip redundant final
|
||||
# edits. Subclasses that *require* an explicit finalize call to close
|
||||
# out the message lifecycle (e.g. rich card / AI assistant surfaces
|
||||
# such as DingTalk AI Cards) override this to True (class attribute or
|
||||
# property) so the stream consumer knows not to short-circuit.
|
||||
REQUIRES_EDIT_FINALIZE: bool = False
|
||||
|
||||
async def edit_message(
|
||||
self,
|
||||
chat_id: str,
|
||||
message_id: str,
|
||||
content: str,
|
||||
*,
|
||||
finalize: bool = False,
|
||||
) -> SendResult:
|
||||
"""
|
||||
Edit a previously sent message. Optional — platforms that don't
|
||||
support editing return success=False and callers fall back to
|
||||
sending a new message.
|
||||
|
||||
``finalize`` signals that this is the last edit in a streaming
|
||||
sequence. Most platforms (Telegram, Slack, Discord, Matrix,
|
||||
etc.) treat it as a no-op because their edit APIs have no notion
|
||||
of message lifecycle state — an edit is an edit. Platforms that
|
||||
render streaming updates with a distinct "in progress" state and
|
||||
require explicit closure (e.g. rich card / AI assistant surfaces
|
||||
such as DingTalk AI Cards) use it to finalize the message and
|
||||
transition the UI out of the streaming indicator — those should
|
||||
also set ``REQUIRES_EDIT_FINALIZE = True`` so callers route a
|
||||
final edit through even when content is unchanged. Callers
|
||||
should set ``finalize=True`` on the final edit of a streamed
|
||||
response (typically when ``got_done`` fires in the stream
|
||||
consumer) and leave it ``False`` on intermediate edits.
|
||||
"""
|
||||
return SendResult(success=False, error="Not supported")
|
||||
|
||||
@@ -1324,7 +1168,7 @@ class BasePlatformAdapter(ABC):
|
||||
path = path[1:-1].strip()
|
||||
path = path.lstrip("`\"'").rstrip("`\"',.;:)}]")
|
||||
if path:
|
||||
media.append((os.path.expanduser(path), has_voice_tag))
|
||||
media.append((path, has_voice_tag))
|
||||
|
||||
# Remove MEDIA tags from content (including surrounding quote/backtick wrappers)
|
||||
if media:
|
||||
@@ -1612,9 +1456,7 @@ class BasePlatformAdapter(ABC):
|
||||
# session lifecycle and its cleanup races with the running task
|
||||
# (see PR #4926).
|
||||
cmd = event.get_command()
|
||||
from hermes_cli.commands import should_bypass_active_session
|
||||
|
||||
if should_bypass_active_session(cmd):
|
||||
if cmd in ("approve", "deny", "status", "stop", "new", "reset", "background", "restart"):
|
||||
logger.debug(
|
||||
"[%s] Command '/%s' bypassing active-session guard for %s",
|
||||
self.name, cmd, session_key,
|
||||
@@ -1729,21 +1571,6 @@ class BasePlatformAdapter(ABC):
|
||||
# streaming already delivered the text (already_sent=True) or
|
||||
# when the message was queued behind an active agent. Log at
|
||||
# DEBUG to avoid noisy warnings for expected behavior.
|
||||
#
|
||||
# Suppress stale response when the session was interrupted by a
|
||||
# new message that hasn't been consumed yet. The pending message
|
||||
# is processed by the pending-message handler below (#8221/#2483).
|
||||
if (
|
||||
response
|
||||
and interrupt_event.is_set()
|
||||
and session_key in self._pending_messages
|
||||
):
|
||||
logger.info(
|
||||
"[%s] Suppressing stale response for interrupted session %s",
|
||||
self.name,
|
||||
session_key,
|
||||
)
|
||||
response = None
|
||||
if not response:
|
||||
logger.debug("[%s] Handler returned empty/None response for %s", self.name, event.source.chat_id)
|
||||
if response:
|
||||
@@ -1965,14 +1792,6 @@ class BasePlatformAdapter(ABC):
|
||||
except Exception:
|
||||
pass # Last resort — don't let error reporting crash the handler
|
||||
finally:
|
||||
# Fire any one-shot post-delivery callback registered for this
|
||||
# session (e.g. deferred background-review notifications).
|
||||
_post_cb = getattr(self, "_post_delivery_callbacks", {}).pop(session_key, None)
|
||||
if callable(_post_cb):
|
||||
try:
|
||||
_post_cb()
|
||||
except Exception:
|
||||
pass
|
||||
# Stop typing indicator
|
||||
typing_task.cancel()
|
||||
try:
|
||||
@@ -2026,7 +1845,6 @@ class BasePlatformAdapter(ABC):
|
||||
chat_topic: Optional[str] = None,
|
||||
user_id_alt: Optional[str] = None,
|
||||
chat_id_alt: Optional[str] = None,
|
||||
is_bot: bool = False,
|
||||
) -> SessionSource:
|
||||
"""Helper to build a SessionSource for this platform."""
|
||||
# Normalize empty topic to None
|
||||
@@ -2043,7 +1861,6 @@ class BasePlatformAdapter(ABC):
|
||||
chat_topic=chat_topic.strip() if chat_topic else None,
|
||||
user_id_alt=user_id_alt,
|
||||
chat_id_alt=chat_id_alt,
|
||||
is_bot=is_bot,
|
||||
)
|
||||
|
||||
@abstractmethod
|
||||
@@ -2069,11 +1886,7 @@ class BasePlatformAdapter(ABC):
|
||||
return content
|
||||
|
||||
@staticmethod
|
||||
def truncate_message(
|
||||
content: str,
|
||||
max_length: int = 4096,
|
||||
len_fn: Optional["Callable[[str], int]"] = None,
|
||||
) -> List[str]:
|
||||
def truncate_message(content: str, max_length: int = 4096) -> List[str]:
|
||||
"""
|
||||
Split a long message into chunks, preserving code block boundaries.
|
||||
|
||||
@@ -2085,16 +1898,11 @@ class BasePlatformAdapter(ABC):
|
||||
Args:
|
||||
content: The full message content
|
||||
max_length: Maximum length per chunk (platform-specific)
|
||||
len_fn: Optional length function for measuring string length.
|
||||
Defaults to ``len`` (Unicode code-points). Pass
|
||||
``utf16_len`` for platforms that measure message
|
||||
length in UTF-16 code units (e.g. Telegram).
|
||||
|
||||
Returns:
|
||||
List of message chunks
|
||||
"""
|
||||
_len = len_fn or len
|
||||
if _len(content) <= max_length:
|
||||
if len(content) <= max_length:
|
||||
return [content]
|
||||
|
||||
INDICATOR_RESERVE = 10 # room for " (XX/XX)"
|
||||
@@ -2113,33 +1921,22 @@ class BasePlatformAdapter(ABC):
|
||||
|
||||
# How much body text we can fit after accounting for the prefix,
|
||||
# a potential closing fence, and the chunk indicator.
|
||||
headroom = max_length - INDICATOR_RESERVE - _len(prefix) - _len(FENCE_CLOSE)
|
||||
headroom = max_length - INDICATOR_RESERVE - len(prefix) - len(FENCE_CLOSE)
|
||||
if headroom < 1:
|
||||
headroom = max_length // 2
|
||||
|
||||
# Everything remaining fits in one final chunk
|
||||
if _len(prefix) + _len(remaining) <= max_length - INDICATOR_RESERVE:
|
||||
if len(prefix) + len(remaining) <= max_length - INDICATOR_RESERVE:
|
||||
chunks.append(prefix + remaining)
|
||||
break
|
||||
|
||||
# Find a natural split point (prefer newlines, then spaces).
|
||||
# When _len != len (e.g. utf16_len for Telegram), headroom is
|
||||
# measured in the custom unit. We need codepoint-based slice
|
||||
# positions that stay within the custom-unit budget.
|
||||
#
|
||||
# _safe_slice_pos() maps a custom-unit budget to the largest
|
||||
# codepoint offset whose custom length ≤ budget.
|
||||
if _len is not len:
|
||||
# Map headroom (custom units) → codepoint slice length
|
||||
_cp_limit = _custom_unit_to_cp(remaining, headroom, _len)
|
||||
else:
|
||||
_cp_limit = headroom
|
||||
region = remaining[:_cp_limit]
|
||||
# Find a natural split point (prefer newlines, then spaces)
|
||||
region = remaining[:headroom]
|
||||
split_at = region.rfind("\n")
|
||||
if split_at < _cp_limit // 2:
|
||||
if split_at < headroom // 2:
|
||||
split_at = region.rfind(" ")
|
||||
if split_at < 1:
|
||||
split_at = _cp_limit
|
||||
split_at = headroom
|
||||
|
||||
# Avoid splitting inside an inline code span (`...`).
|
||||
# If the text before split_at has an odd number of unescaped
|
||||
@@ -2159,7 +1956,7 @@ class BasePlatformAdapter(ABC):
|
||||
safe_split = candidate.rfind(" ", 0, last_bt)
|
||||
nl_split = candidate.rfind("\n", 0, last_bt)
|
||||
safe_split = max(safe_split, nl_split)
|
||||
if safe_split > _cp_limit // 4:
|
||||
if safe_split > headroom // 4:
|
||||
split_at = safe_split
|
||||
|
||||
chunk_body = remaining[:split_at]
|
||||
|
||||
@@ -224,21 +224,6 @@ class BlueBubblesAdapter(BasePlatformAdapter):
|
||||
host = "localhost"
|
||||
return f"http://{host}:{self.webhook_port}{self.webhook_path}"
|
||||
|
||||
@property
|
||||
def _webhook_register_url(self) -> str:
|
||||
"""Webhook URL registered with BlueBubbles, including the password as
|
||||
a query param so inbound webhook POSTs carry credentials.
|
||||
|
||||
BlueBubbles posts events to the exact URL registered via
|
||||
``/api/v1/webhook``. Its webhook registration API does not support
|
||||
custom headers, so embedding the password in the URL is the only
|
||||
way to authenticate inbound webhooks without disabling auth.
|
||||
"""
|
||||
base = self._webhook_url
|
||||
if self.password:
|
||||
return f"{base}?password={quote(self.password, safe='')}"
|
||||
return base
|
||||
|
||||
async def _find_registered_webhooks(self, url: str) -> list:
|
||||
"""Return list of BB webhook entries matching *url*."""
|
||||
try:
|
||||
@@ -260,7 +245,7 @@ class BlueBubblesAdapter(BasePlatformAdapter):
|
||||
if not self.client:
|
||||
return False
|
||||
|
||||
webhook_url = self._webhook_register_url
|
||||
webhook_url = self._webhook_url
|
||||
|
||||
# Crash resilience — reuse an existing registration if present
|
||||
existing = await self._find_registered_webhooks(webhook_url)
|
||||
@@ -272,7 +257,7 @@ class BlueBubblesAdapter(BasePlatformAdapter):
|
||||
|
||||
payload = {
|
||||
"url": webhook_url,
|
||||
"events": ["new-message", "updated-message"],
|
||||
"events": ["new-message", "updated-message", "message"],
|
||||
}
|
||||
|
||||
try:
|
||||
@@ -307,7 +292,7 @@ class BlueBubblesAdapter(BasePlatformAdapter):
|
||||
if not self.client:
|
||||
return False
|
||||
|
||||
webhook_url = self._webhook_register_url
|
||||
webhook_url = self._webhook_url
|
||||
removed = False
|
||||
|
||||
try:
|
||||
@@ -619,6 +604,35 @@ class BlueBubblesAdapter(BasePlatformAdapter):
|
||||
# Tapback reactions
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def send_reaction(
|
||||
self,
|
||||
chat_id: str,
|
||||
message_guid: str,
|
||||
reaction: str,
|
||||
part_index: int = 0,
|
||||
) -> SendResult:
|
||||
"""Send a tapback reaction (requires Private API helper)."""
|
||||
if not self._private_api_enabled or not self._helper_connected:
|
||||
return SendResult(
|
||||
success=False, error="Private API helper not connected"
|
||||
)
|
||||
guid = await self._resolve_chat_guid(chat_id)
|
||||
if not guid:
|
||||
return SendResult(success=False, error=f"Chat not found: {chat_id}")
|
||||
try:
|
||||
res = await self._api_post(
|
||||
"/api/v1/message/react",
|
||||
{
|
||||
"chatGuid": guid,
|
||||
"selectedMessageGuid": message_guid,
|
||||
"reaction": reaction,
|
||||
"partIndex": part_index,
|
||||
},
|
||||
)
|
||||
return SendResult(success=True, raw_response=res)
|
||||
except Exception as exc:
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Chat info
|
||||
# ------------------------------------------------------------------
|
||||
@@ -850,12 +864,6 @@ class BlueBubblesAdapter(BasePlatformAdapter):
|
||||
payload.get("chat_guid"),
|
||||
payload.get("guid"),
|
||||
)
|
||||
# Fallback: BlueBubbles v1.9+ webhook payloads omit top-level chatGuid;
|
||||
# the chat GUID is nested under data.chats[0].guid instead.
|
||||
if not chat_guid:
|
||||
_chats = record.get("chats") or []
|
||||
if _chats and isinstance(_chats[0], dict):
|
||||
chat_guid = _chats[0].get("guid") or _chats[0].get("chatGuid")
|
||||
chat_identifier = self._value(
|
||||
record.get("chatIdentifier"),
|
||||
record.get("identifier"),
|
||||
|
||||
+64
-1092
File diff suppressed because it is too large
Load Diff
+73
-764
File diff suppressed because it is too large
Load Diff
+90
-623
@@ -34,9 +34,6 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import Any, Dict, List, Optional
|
||||
from urllib.error import HTTPError, URLError
|
||||
from urllib.parse import urlencode
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
# aiohttp/websockets are independent optional deps — import outside lark_oapi
|
||||
# so they remain available for tests and webhook mode even if lark_oapi is missing.
|
||||
@@ -72,10 +69,7 @@ try:
|
||||
UpdateMessageRequestBody,
|
||||
)
|
||||
from lark_oapi.core.const import FEISHU_DOMAIN, LARK_DOMAIN
|
||||
from lark_oapi.event.callback.model.p2_card_action_trigger import (
|
||||
CallBackCard,
|
||||
P2CardActionTriggerResponse,
|
||||
)
|
||||
from lark_oapi.event.callback.model.p2_card_action_trigger import P2CardActionTriggerResponse
|
||||
from lark_oapi.event.dispatcher_handler import EventDispatcherHandler
|
||||
from lark_oapi.ws import Client as FeishuWSClient
|
||||
|
||||
@@ -83,7 +77,6 @@ try:
|
||||
except ImportError:
|
||||
FEISHU_AVAILABLE = False
|
||||
lark = None # type: ignore[assignment]
|
||||
CallBackCard = None # type: ignore[assignment]
|
||||
P2CardActionTriggerResponse = None # type: ignore[assignment]
|
||||
EventDispatcherHandler = None # type: ignore[assignment]
|
||||
FeishuWSClient = None # type: ignore[assignment]
|
||||
@@ -173,35 +166,9 @@ _FEISHU_WEBHOOK_BODY_TIMEOUT_SECONDS = 30 # max seconds to read request
|
||||
_FEISHU_WEBHOOK_ANOMALY_THRESHOLD = 25 # consecutive error responses before WARNING log
|
||||
_FEISHU_WEBHOOK_ANOMALY_TTL_SECONDS = 6 * 60 * 60 # anomaly tracker TTL (6 hours) — matches openclaw
|
||||
_FEISHU_CARD_ACTION_DEDUP_TTL_SECONDS = 15 * 60 # card action token dedup window (15 min)
|
||||
|
||||
_APPROVAL_CHOICE_MAP: Dict[str, str] = {
|
||||
"approve_once": "once",
|
||||
"approve_session": "session",
|
||||
"approve_always": "always",
|
||||
"deny": "deny",
|
||||
}
|
||||
_APPROVAL_LABEL_MAP: Dict[str, str] = {
|
||||
"once": "Approved once",
|
||||
"session": "Approved for session",
|
||||
"always": "Approved permanently",
|
||||
"deny": "Denied",
|
||||
}
|
||||
_FEISHU_BOT_MSG_TRACK_SIZE = 512 # LRU size for tracking sent message IDs
|
||||
_FEISHU_REPLY_FALLBACK_CODES = frozenset({230011, 231003}) # reply target withdrawn/missing → create fallback
|
||||
_FEISHU_ACK_EMOJI = "OK"
|
||||
|
||||
# QR onboarding constants
|
||||
_ONBOARD_ACCOUNTS_URLS = {
|
||||
"feishu": "https://accounts.feishu.cn",
|
||||
"lark": "https://accounts.larksuite.com",
|
||||
}
|
||||
_ONBOARD_OPEN_URLS = {
|
||||
"feishu": "https://open.feishu.cn",
|
||||
"lark": "https://open.larksuite.com",
|
||||
}
|
||||
_REGISTRATION_PATH = "/oauth/v1/app/registration"
|
||||
_ONBOARD_REQUEST_TIMEOUT_S = 10
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fallback display strings
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -447,6 +414,14 @@ def _build_markdown_post_payload(content: str) -> str:
|
||||
)
|
||||
|
||||
|
||||
def parse_feishu_post_content(raw_content: str) -> FeishuPostParseResult:
|
||||
try:
|
||||
parsed = json.loads(raw_content) if raw_content else {}
|
||||
except json.JSONDecodeError:
|
||||
return FeishuPostParseResult(text_content=FALLBACK_POST_TEXT)
|
||||
return parse_feishu_post_payload(parsed)
|
||||
|
||||
|
||||
def parse_feishu_post_payload(payload: Any) -> FeishuPostParseResult:
|
||||
resolved = _resolve_post_payload(payload)
|
||||
if not resolved:
|
||||
@@ -1073,13 +1048,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
self._webhook_rate_counts: Dict[str, tuple[int, float]] = {} # rate_key → (count, window_start)
|
||||
self._webhook_anomaly_counts: Dict[str, tuple[int, str, float]] = {} # ip → (count, last_status, first_seen)
|
||||
self._card_action_tokens: Dict[str, float] = {} # token → first_seen_time
|
||||
# Inbound events that arrived before the adapter loop was ready
|
||||
# (e.g. during startup/restart or network-flap reconnect). A single
|
||||
# drainer thread replays them as soon as the loop becomes available.
|
||||
self._pending_inbound_events: List[Any] = []
|
||||
self._pending_inbound_lock = threading.Lock()
|
||||
self._pending_drain_scheduled = False
|
||||
self._pending_inbound_max_depth = 1000 # cap queue; drop oldest beyond
|
||||
self._chat_locks: Dict[str, asyncio.Lock] = {} # chat_id → lock (per-chat serial processing)
|
||||
self._sent_message_ids_to_chat: Dict[str, str] = {} # message_id → chat_id (for reaction routing)
|
||||
self._sent_message_id_order: List[str] = [] # LRU order for _sent_message_ids_to_chat
|
||||
@@ -1226,12 +1194,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
.register_p2_card_action_trigger(self._on_card_action_trigger)
|
||||
.register_p2_im_chat_member_bot_added_v1(self._on_bot_added_to_chat)
|
||||
.register_p2_im_chat_member_bot_deleted_v1(self._on_bot_removed_from_chat)
|
||||
.register_p2_im_chat_access_event_bot_p2p_chat_entered_v1(self._on_p2p_chat_entered)
|
||||
.register_p2_im_message_recalled_v1(self._on_message_recalled)
|
||||
.register_p2_customized_event(
|
||||
"drive.notice.comment_add_v1",
|
||||
self._on_drive_comment_event,
|
||||
)
|
||||
.build()
|
||||
)
|
||||
|
||||
@@ -1520,12 +1482,14 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
logger.warning("[Feishu] send_exec_approval failed: %s", exc)
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
@staticmethod
|
||||
def _build_resolved_approval_card(*, choice: str, user_name: str) -> Dict[str, Any]:
|
||||
"""Build raw card JSON for a resolved approval action."""
|
||||
async def _update_approval_card(
|
||||
self, message_id: str, label: str, user_name: str, choice: str,
|
||||
) -> None:
|
||||
"""Replace the approval card with a resolved status card."""
|
||||
if not self._client or not message_id:
|
||||
return
|
||||
icon = "❌" if choice == "deny" else "✅"
|
||||
label = _APPROVAL_LABEL_MAP.get(choice, "Resolved")
|
||||
return {
|
||||
card = {
|
||||
"config": {"wide_screen_mode": True},
|
||||
"header": {
|
||||
"title": {"content": f"{icon} {label}", "tag": "plain_text"},
|
||||
@@ -1538,6 +1502,13 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
},
|
||||
],
|
||||
}
|
||||
try:
|
||||
payload = json.dumps(card, ensure_ascii=False)
|
||||
body = self._build_update_message_body(msg_type="interactive", content=payload)
|
||||
request = self._build_update_message_request(message_id=message_id, request_body=body)
|
||||
await asyncio.to_thread(self._client.im.v1.message.update, request)
|
||||
except Exception as exc:
|
||||
logger.warning("[Feishu] Failed to update approval card %s: %s", message_id, exc)
|
||||
|
||||
async def send_voice(
|
||||
self,
|
||||
@@ -1770,22 +1741,10 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
# =========================================================================
|
||||
|
||||
def _on_message_event(self, data: Any) -> None:
|
||||
"""Normalize Feishu inbound events into MessageEvent.
|
||||
|
||||
Called by the lark_oapi SDK's event dispatcher on a background thread.
|
||||
If the adapter loop is not currently accepting callbacks (brief window
|
||||
during startup/restart or network-flap reconnect), the event is queued
|
||||
for replay instead of dropped.
|
||||
"""
|
||||
"""Normalize Feishu inbound events into MessageEvent."""
|
||||
loop = self._loop
|
||||
if not self._loop_accepts_callbacks(loop):
|
||||
start_drainer = self._enqueue_pending_inbound_event(data)
|
||||
if start_drainer:
|
||||
threading.Thread(
|
||||
target=self._drain_pending_inbound_events,
|
||||
name="feishu-pending-inbound-drainer",
|
||||
daemon=True,
|
||||
).start()
|
||||
if loop is None or bool(getattr(loop, "is_closed", lambda: False)()):
|
||||
logger.warning("[Feishu] Dropping inbound message before adapter loop is ready")
|
||||
return
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
self._handle_message_event_data(data),
|
||||
@@ -1793,124 +1752,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
)
|
||||
future.add_done_callback(self._log_background_failure)
|
||||
|
||||
def _enqueue_pending_inbound_event(self, data: Any) -> bool:
|
||||
"""Append an event to the pending-inbound queue.
|
||||
|
||||
Returns True if the caller should spawn a drainer thread (no drainer
|
||||
currently scheduled), False if a drainer is already running and will
|
||||
pick up the new event on its next pass.
|
||||
"""
|
||||
with self._pending_inbound_lock:
|
||||
if len(self._pending_inbound_events) >= self._pending_inbound_max_depth:
|
||||
# Queue full — drop the oldest to make room. This happens only
|
||||
# if the loop stays unavailable for an extended period AND the
|
||||
# WS keeps firing callbacks. Still better than silent drops.
|
||||
dropped = self._pending_inbound_events.pop(0)
|
||||
try:
|
||||
event = getattr(dropped, "event", None)
|
||||
message = getattr(event, "message", None)
|
||||
message_id = str(getattr(message, "message_id", "") or "unknown")
|
||||
except Exception:
|
||||
message_id = "unknown"
|
||||
logger.error(
|
||||
"[Feishu] Pending-inbound queue full (%d); dropped oldest event %s",
|
||||
self._pending_inbound_max_depth,
|
||||
message_id,
|
||||
)
|
||||
self._pending_inbound_events.append(data)
|
||||
depth = len(self._pending_inbound_events)
|
||||
should_start = not self._pending_drain_scheduled
|
||||
if should_start:
|
||||
self._pending_drain_scheduled = True
|
||||
logger.warning(
|
||||
"[Feishu] Queued inbound event for replay (loop not ready, queue depth=%d)",
|
||||
depth,
|
||||
)
|
||||
return should_start
|
||||
|
||||
def _drain_pending_inbound_events(self) -> None:
|
||||
"""Replay queued inbound events once the adapter loop is ready.
|
||||
|
||||
Runs in a dedicated daemon thread. Polls ``_running`` and
|
||||
``_loop_accepts_callbacks`` until events can be dispatched or the
|
||||
adapter shuts down. A single drainer handles the entire queue;
|
||||
concurrent ``_on_message_event`` calls just append.
|
||||
"""
|
||||
poll_interval = 0.25
|
||||
max_wait_seconds = 120.0 # safety cap: drop queue after 2 minutes
|
||||
waited = 0.0
|
||||
try:
|
||||
while True:
|
||||
if not getattr(self, "_running", True):
|
||||
# Adapter shutting down — drop queued events rather than
|
||||
# holding them against a closed loop.
|
||||
with self._pending_inbound_lock:
|
||||
dropped = len(self._pending_inbound_events)
|
||||
self._pending_inbound_events.clear()
|
||||
if dropped:
|
||||
logger.warning(
|
||||
"[Feishu] Dropped %d queued inbound event(s) during shutdown",
|
||||
dropped,
|
||||
)
|
||||
return
|
||||
loop = self._loop
|
||||
if self._loop_accepts_callbacks(loop):
|
||||
with self._pending_inbound_lock:
|
||||
batch = self._pending_inbound_events[:]
|
||||
self._pending_inbound_events.clear()
|
||||
if not batch:
|
||||
# Queue emptied between check and grab; done.
|
||||
with self._pending_inbound_lock:
|
||||
if not self._pending_inbound_events:
|
||||
return
|
||||
continue
|
||||
dispatched = 0
|
||||
requeue: List[Any] = []
|
||||
for event in batch:
|
||||
try:
|
||||
fut = asyncio.run_coroutine_threadsafe(
|
||||
self._handle_message_event_data(event),
|
||||
loop,
|
||||
)
|
||||
fut.add_done_callback(self._log_background_failure)
|
||||
dispatched += 1
|
||||
except RuntimeError:
|
||||
# Loop closed between check and submit — requeue
|
||||
# and poll again.
|
||||
requeue.append(event)
|
||||
if requeue:
|
||||
with self._pending_inbound_lock:
|
||||
self._pending_inbound_events[:0] = requeue
|
||||
if dispatched:
|
||||
logger.info(
|
||||
"[Feishu] Replayed %d queued inbound event(s)",
|
||||
dispatched,
|
||||
)
|
||||
if not requeue:
|
||||
# Successfully drained; check if more arrived while
|
||||
# we were dispatching and exit if not.
|
||||
with self._pending_inbound_lock:
|
||||
if not self._pending_inbound_events:
|
||||
return
|
||||
# More events queued or requeue pending — loop again.
|
||||
continue
|
||||
if waited >= max_wait_seconds:
|
||||
with self._pending_inbound_lock:
|
||||
dropped = len(self._pending_inbound_events)
|
||||
self._pending_inbound_events.clear()
|
||||
logger.error(
|
||||
"[Feishu] Adapter loop unavailable for %.0fs; "
|
||||
"dropped %d queued inbound event(s)",
|
||||
max_wait_seconds,
|
||||
dropped,
|
||||
)
|
||||
return
|
||||
time.sleep(poll_interval)
|
||||
waited += poll_interval
|
||||
finally:
|
||||
with self._pending_inbound_lock:
|
||||
self._pending_drain_scheduled = False
|
||||
|
||||
async def _handle_message_event_data(self, data: Any) -> None:
|
||||
"""Shared inbound message handling for websocket and webhook transports."""
|
||||
event = getattr(data, "event", None)
|
||||
@@ -1963,31 +1804,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
logger.info("[Feishu] Bot removed from chat: %s", chat_id)
|
||||
self._chat_info_cache.pop(chat_id, None)
|
||||
|
||||
def _on_p2p_chat_entered(self, data: Any) -> None:
|
||||
logger.debug("[Feishu] User entered P2P chat with bot")
|
||||
|
||||
def _on_message_recalled(self, data: Any) -> None:
|
||||
logger.debug("[Feishu] Message recalled by user")
|
||||
|
||||
def _on_drive_comment_event(self, data: Any) -> None:
|
||||
"""Handle drive document comment notification (drive.notice.comment_add_v1).
|
||||
|
||||
Delegates to :mod:`gateway.platforms.feishu_comment` for parsing,
|
||||
logging, and reaction. Scheduling follows the same
|
||||
``run_coroutine_threadsafe`` pattern used by ``_on_message_event``.
|
||||
"""
|
||||
from gateway.platforms.feishu_comment import handle_drive_comment_event
|
||||
|
||||
loop = self._loop
|
||||
if not self._loop_accepts_callbacks(loop):
|
||||
logger.warning("[Feishu] Dropping drive comment event before adapter loop is ready")
|
||||
return
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
handle_drive_comment_event(self._client, data, self_open_id=self._bot_open_id),
|
||||
loop,
|
||||
)
|
||||
future.add_done_callback(self._log_background_failure)
|
||||
|
||||
def _on_reaction_event(self, event_type: str, data: Any) -> None:
|
||||
"""Route user reactions on bot messages as synthetic text events."""
|
||||
event = getattr(data, "event", None)
|
||||
@@ -2021,82 +1837,20 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
future.add_done_callback(self._log_background_failure)
|
||||
|
||||
def _on_card_action_trigger(self, data: Any) -> Any:
|
||||
"""Handle card-action callback from the Feishu SDK (synchronous).
|
||||
|
||||
For approval actions: parses the event once, returns the resolved card
|
||||
inline (the only reliable way to sync all clients), and schedules a
|
||||
lightweight async method to actually unblock the agent.
|
||||
|
||||
For other card actions: delegates to ``_handle_card_action_event``.
|
||||
"""
|
||||
"""Schedule Feishu card actions on the adapter loop and acknowledge immediately."""
|
||||
loop = self._loop
|
||||
if not self._loop_accepts_callbacks(loop):
|
||||
if loop is None or bool(getattr(loop, "is_closed", lambda: False)()):
|
||||
logger.warning("[Feishu] Dropping card action before adapter loop is ready")
|
||||
return P2CardActionTriggerResponse() if P2CardActionTriggerResponse else None
|
||||
|
||||
event = getattr(data, "event", None)
|
||||
action = getattr(event, "action", None)
|
||||
action_value = getattr(action, "value", {}) or {}
|
||||
hermes_action = action_value.get("hermes_action") if isinstance(action_value, dict) else None
|
||||
|
||||
if hermes_action:
|
||||
return self._handle_approval_card_action(event=event, action_value=action_value, loop=loop)
|
||||
|
||||
self._submit_on_loop(loop, self._handle_card_action_event(data))
|
||||
else:
|
||||
future = asyncio.run_coroutine_threadsafe(
|
||||
self._handle_card_action_event(data),
|
||||
loop,
|
||||
)
|
||||
future.add_done_callback(self._log_background_failure)
|
||||
if P2CardActionTriggerResponse is None:
|
||||
return None
|
||||
return P2CardActionTriggerResponse()
|
||||
|
||||
@staticmethod
|
||||
def _loop_accepts_callbacks(loop: Any) -> bool:
|
||||
"""Return True when the adapter loop can accept thread-safe submissions."""
|
||||
return loop is not None and not bool(getattr(loop, "is_closed", lambda: False)())
|
||||
|
||||
def _submit_on_loop(self, loop: Any, coro: Any) -> None:
|
||||
"""Schedule background work on the adapter loop with shared failure logging."""
|
||||
future = asyncio.run_coroutine_threadsafe(coro, loop)
|
||||
future.add_done_callback(self._log_background_failure)
|
||||
|
||||
def _handle_approval_card_action(self, *, event: Any, action_value: Dict[str, Any], loop: Any) -> Any:
|
||||
"""Schedule approval resolution and build the synchronous callback response."""
|
||||
approval_id = action_value.get("approval_id")
|
||||
if approval_id is None:
|
||||
logger.debug("[Feishu] Card action missing approval_id, ignoring")
|
||||
return P2CardActionTriggerResponse() if P2CardActionTriggerResponse else None
|
||||
choice = _APPROVAL_CHOICE_MAP.get(action_value.get("hermes_action"), "deny")
|
||||
|
||||
operator = getattr(event, "operator", None)
|
||||
open_id = str(getattr(operator, "open_id", "") or "")
|
||||
user_name = self._get_cached_sender_name(open_id) or open_id
|
||||
|
||||
self._submit_on_loop(loop, self._resolve_approval(approval_id, choice, user_name))
|
||||
|
||||
if P2CardActionTriggerResponse is None:
|
||||
return None
|
||||
response = P2CardActionTriggerResponse()
|
||||
if CallBackCard is not None:
|
||||
card = CallBackCard()
|
||||
card.type = "raw"
|
||||
card.data = self._build_resolved_approval_card(choice=choice, user_name=user_name)
|
||||
response.card = card
|
||||
return response
|
||||
|
||||
async def _resolve_approval(self, approval_id: Any, choice: str, user_name: str) -> None:
|
||||
"""Pop approval state and unblock the waiting agent thread."""
|
||||
state = self._approval_state.pop(approval_id, None)
|
||||
if not state:
|
||||
logger.debug("[Feishu] Approval %s already resolved or unknown", approval_id)
|
||||
return
|
||||
try:
|
||||
from tools.approval import resolve_gateway_approval
|
||||
count = resolve_gateway_approval(state["session_key"], choice)
|
||||
logger.info(
|
||||
"Feishu button resolved %d approval(s) for session %s (choice=%s, user=%s)",
|
||||
count, state["session_key"], choice, user_name,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error("Failed to resolve gateway approval from Feishu button: %s", exc)
|
||||
|
||||
async def _handle_reaction_event(self, event_type: str, data: Any) -> None:
|
||||
"""Fetch the reacted-to message; if it was sent by this bot, emit a synthetic text event."""
|
||||
if not self._client:
|
||||
@@ -2188,6 +1942,51 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
action_tag = str(getattr(action, "tag", "") or "button")
|
||||
action_value = getattr(action, "value", {}) or {}
|
||||
|
||||
# --- Exec approval button intercept ---
|
||||
hermes_action = action_value.get("hermes_action") if isinstance(action_value, dict) else None
|
||||
if hermes_action:
|
||||
approval_id = action_value.get("approval_id")
|
||||
state = self._approval_state.pop(approval_id, None)
|
||||
if not state:
|
||||
logger.debug("[Feishu] Approval %s already resolved or unknown", approval_id)
|
||||
return
|
||||
|
||||
choice_map = {
|
||||
"approve_once": "once",
|
||||
"approve_session": "session",
|
||||
"approve_always": "always",
|
||||
"deny": "deny",
|
||||
}
|
||||
choice = choice_map.get(hermes_action, "deny")
|
||||
|
||||
label_map = {
|
||||
"once": "Approved once",
|
||||
"session": "Approved for session",
|
||||
"always": "Approved permanently",
|
||||
"deny": "Denied",
|
||||
}
|
||||
label = label_map.get(choice, "Resolved")
|
||||
|
||||
# Resolve sender name for the status card
|
||||
sender_id = SimpleNamespace(open_id=open_id, user_id=None, union_id=None)
|
||||
sender_profile = await self._resolve_sender_profile(sender_id)
|
||||
user_name = sender_profile.get("user_name") or open_id
|
||||
|
||||
# Resolve the approval — unblocks the agent thread
|
||||
try:
|
||||
from tools.approval import resolve_gateway_approval
|
||||
count = resolve_gateway_approval(state["session_key"], choice)
|
||||
logger.info(
|
||||
"Feishu button resolved %d approval(s) for session %s (choice=%s, user=%s)",
|
||||
count, state["session_key"], choice, user_name,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error("Failed to resolve gateway approval from Feishu button: %s", exc)
|
||||
|
||||
# Update the card to show the decision
|
||||
await self._update_approval_card(state.get("message_id", ""), label, user_name, choice)
|
||||
return
|
||||
|
||||
synthetic_text = f"/card {action_tag}"
|
||||
if action_value:
|
||||
try:
|
||||
@@ -2613,8 +2412,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
self._on_reaction_event(event_type, data)
|
||||
elif event_type == "card.action.trigger":
|
||||
self._on_card_action_trigger(data)
|
||||
elif event_type == "drive.notice.comment_add_v1":
|
||||
self._on_drive_comment_event(data)
|
||||
else:
|
||||
logger.debug("[Feishu] Ignoring webhook event type: %s", event_type or "unknown")
|
||||
return web.json_response({"code": 0, "msg": "ok"})
|
||||
@@ -2875,6 +2672,12 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
return self._resolve_media_message_type(media_types[0] if media_types else "", default=MessageType.DOCUMENT)
|
||||
return MessageType.TEXT
|
||||
|
||||
def _normalize_inbound_text(self, text: str) -> str:
|
||||
"""Strip Feishu mention placeholders from inbound text."""
|
||||
text = _MENTION_RE.sub(" ", text or "")
|
||||
text = _MULTISPACE_RE.sub(" ", text)
|
||||
return text.strip()
|
||||
|
||||
async def _maybe_extract_text_document(self, cached_path: str, media_type: str) -> str:
|
||||
if not cached_path or not media_type.startswith("text/"):
|
||||
return ""
|
||||
@@ -3092,19 +2895,6 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
"user_id_alt": union_id,
|
||||
}
|
||||
|
||||
def _get_cached_sender_name(self, sender_id: Optional[str]) -> Optional[str]:
|
||||
"""Return a cached sender name only while its TTL is still valid."""
|
||||
if not sender_id:
|
||||
return None
|
||||
cached = self._sender_name_cache.get(sender_id)
|
||||
if cached is None:
|
||||
return None
|
||||
name, expire_at = cached
|
||||
if time.time() < expire_at:
|
||||
return name
|
||||
self._sender_name_cache.pop(sender_id, None)
|
||||
return None
|
||||
|
||||
async def _resolve_sender_name_from_api(self, sender_id: Optional[str]) -> Optional[str]:
|
||||
"""Fetch the sender's display name from the Feishu contact API with a 10-minute cache.
|
||||
|
||||
@@ -3117,9 +2907,11 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
if not trimmed:
|
||||
return None
|
||||
now = time.time()
|
||||
cached_name = self._get_cached_sender_name(trimmed)
|
||||
if cached_name is not None:
|
||||
return cached_name
|
||||
cached = self._sender_name_cache.get(trimmed)
|
||||
if cached is not None:
|
||||
name, expire_at = cached
|
||||
if now < expire_at:
|
||||
return name
|
||||
try:
|
||||
from lark_oapi.api.contact.v3 import GetUserRequest # lazy import
|
||||
if trimmed.startswith("ou_"):
|
||||
@@ -3829,328 +3621,3 @@ class FeishuAdapter(BasePlatformAdapter):
|
||||
return _FEISHU_FILE_UPLOAD_TYPE, "file"
|
||||
|
||||
return _FEISHU_FILE_UPLOAD_TYPE, "file"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# QR scan-to-create onboarding
|
||||
#
|
||||
# Device-code flow: user scans a QR code with Feishu/Lark mobile app and the
|
||||
# platform creates a fully configured bot application automatically.
|
||||
# Called by `hermes gateway setup` via _setup_feishu() in hermes_cli/gateway.py.
|
||||
# =============================================================================
|
||||
|
||||
|
||||
def _accounts_base_url(domain: str) -> str:
|
||||
return _ONBOARD_ACCOUNTS_URLS.get(domain, _ONBOARD_ACCOUNTS_URLS["feishu"])
|
||||
|
||||
|
||||
def _onboard_open_base_url(domain: str) -> str:
|
||||
return _ONBOARD_OPEN_URLS.get(domain, _ONBOARD_OPEN_URLS["feishu"])
|
||||
|
||||
|
||||
def _post_registration(base_url: str, body: Dict[str, str]) -> dict:
|
||||
"""POST form-encoded data to the registration endpoint, return parsed JSON.
|
||||
|
||||
The registration endpoint returns JSON even on 4xx (e.g. poll returns
|
||||
authorization_pending as a 400). We always parse the body regardless of
|
||||
HTTP status.
|
||||
"""
|
||||
url = f"{base_url}{_REGISTRATION_PATH}"
|
||||
data = urlencode(body).encode("utf-8")
|
||||
req = Request(url, data=data, headers={"Content-Type": "application/x-www-form-urlencoded"})
|
||||
try:
|
||||
with urlopen(req, timeout=_ONBOARD_REQUEST_TIMEOUT_S) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
except HTTPError as exc:
|
||||
body_bytes = exc.read()
|
||||
if body_bytes:
|
||||
try:
|
||||
return json.loads(body_bytes.decode("utf-8"))
|
||||
except (ValueError, json.JSONDecodeError):
|
||||
raise exc from None
|
||||
raise
|
||||
|
||||
|
||||
def _init_registration(domain: str = "feishu") -> None:
|
||||
"""Verify the environment supports client_secret auth.
|
||||
|
||||
Raises RuntimeError if not supported.
|
||||
"""
|
||||
base_url = _accounts_base_url(domain)
|
||||
res = _post_registration(base_url, {"action": "init"})
|
||||
methods = res.get("supported_auth_methods") or []
|
||||
if "client_secret" not in methods:
|
||||
raise RuntimeError(
|
||||
f"Feishu / Lark registration environment does not support client_secret auth. "
|
||||
f"Supported: {methods}"
|
||||
)
|
||||
|
||||
|
||||
def _begin_registration(domain: str = "feishu") -> dict:
|
||||
"""Start the device-code flow. Returns device_code, qr_url, user_code, interval, expire_in."""
|
||||
base_url = _accounts_base_url(domain)
|
||||
res = _post_registration(base_url, {
|
||||
"action": "begin",
|
||||
"archetype": "PersonalAgent",
|
||||
"auth_method": "client_secret",
|
||||
"request_user_info": "open_id",
|
||||
})
|
||||
device_code = res.get("device_code")
|
||||
if not device_code:
|
||||
raise RuntimeError("Feishu / Lark registration did not return a device_code")
|
||||
qr_url = res.get("verification_uri_complete", "")
|
||||
if "?" in qr_url:
|
||||
qr_url += "&from=hermes&tp=hermes"
|
||||
else:
|
||||
qr_url += "?from=hermes&tp=hermes"
|
||||
return {
|
||||
"device_code": device_code,
|
||||
"qr_url": qr_url,
|
||||
"user_code": res.get("user_code", ""),
|
||||
"interval": res.get("interval") or 5,
|
||||
"expire_in": res.get("expire_in") or 600,
|
||||
}
|
||||
|
||||
|
||||
def _poll_registration(
|
||||
*,
|
||||
device_code: str,
|
||||
interval: int,
|
||||
expire_in: int,
|
||||
domain: str = "feishu",
|
||||
) -> Optional[dict]:
|
||||
"""Poll until the user scans the QR code, or timeout/denial.
|
||||
|
||||
Returns dict with app_id, app_secret, domain, open_id on success.
|
||||
Returns None on failure.
|
||||
"""
|
||||
deadline = time.time() + expire_in
|
||||
current_domain = domain
|
||||
domain_switched = False
|
||||
poll_count = 0
|
||||
|
||||
while time.time() < deadline:
|
||||
base_url = _accounts_base_url(current_domain)
|
||||
try:
|
||||
res = _post_registration(base_url, {
|
||||
"action": "poll",
|
||||
"device_code": device_code,
|
||||
"tp": "ob_app",
|
||||
})
|
||||
except (URLError, OSError, json.JSONDecodeError):
|
||||
time.sleep(interval)
|
||||
continue
|
||||
|
||||
poll_count += 1
|
||||
if poll_count == 1:
|
||||
print(" Fetching configuration results...", end="", flush=True)
|
||||
elif poll_count % 6 == 0:
|
||||
print(".", end="", flush=True)
|
||||
|
||||
# Domain auto-detection
|
||||
user_info = res.get("user_info") or {}
|
||||
tenant_brand = user_info.get("tenant_brand")
|
||||
if tenant_brand == "lark" and not domain_switched:
|
||||
current_domain = "lark"
|
||||
domain_switched = True
|
||||
# Fall through — server may return credentials in this same response.
|
||||
|
||||
# Success
|
||||
if res.get("client_id") and res.get("client_secret"):
|
||||
if poll_count > 0:
|
||||
print() # newline after "Fetching configuration results..." dots
|
||||
return {
|
||||
"app_id": res["client_id"],
|
||||
"app_secret": res["client_secret"],
|
||||
"domain": current_domain,
|
||||
"open_id": user_info.get("open_id"),
|
||||
}
|
||||
|
||||
# Terminal errors
|
||||
error = res.get("error", "")
|
||||
if error in ("access_denied", "expired_token"):
|
||||
if poll_count > 0:
|
||||
print()
|
||||
logger.warning("[Feishu onboard] Registration %s", error)
|
||||
return None
|
||||
|
||||
# authorization_pending or unknown — keep polling
|
||||
time.sleep(interval)
|
||||
|
||||
if poll_count > 0:
|
||||
print()
|
||||
logger.warning("[Feishu onboard] Poll timed out after %ds", expire_in)
|
||||
return None
|
||||
|
||||
|
||||
try:
|
||||
import qrcode as _qrcode_mod
|
||||
except (ImportError, TypeError):
|
||||
_qrcode_mod = None # type: ignore[assignment]
|
||||
|
||||
|
||||
def _render_qr(url: str) -> bool:
|
||||
"""Try to render a QR code in the terminal. Returns True if successful."""
|
||||
if _qrcode_mod is None:
|
||||
return False
|
||||
try:
|
||||
qr = _qrcode_mod.QRCode()
|
||||
qr.add_data(url)
|
||||
qr.make(fit=True)
|
||||
qr.print_ascii(invert=True)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def probe_bot(app_id: str, app_secret: str, domain: str) -> Optional[dict]:
|
||||
"""Verify bot connectivity via /open-apis/bot/v3/info.
|
||||
|
||||
Uses lark_oapi SDK when available, falls back to raw HTTP otherwise.
|
||||
Returns {"bot_name": ..., "bot_open_id": ...} on success, None on failure.
|
||||
"""
|
||||
if FEISHU_AVAILABLE:
|
||||
return _probe_bot_sdk(app_id, app_secret, domain)
|
||||
return _probe_bot_http(app_id, app_secret, domain)
|
||||
|
||||
|
||||
def _build_onboard_client(app_id: str, app_secret: str, domain: str) -> Any:
|
||||
"""Build a lark Client for the given credentials and domain."""
|
||||
sdk_domain = LARK_DOMAIN if domain == "lark" else FEISHU_DOMAIN
|
||||
return (
|
||||
lark.Client.builder()
|
||||
.app_id(app_id)
|
||||
.app_secret(app_secret)
|
||||
.domain(sdk_domain)
|
||||
.log_level(lark.LogLevel.WARNING)
|
||||
.build()
|
||||
)
|
||||
|
||||
|
||||
def _parse_bot_response(data: dict) -> Optional[dict]:
|
||||
"""Extract bot_name and bot_open_id from a /bot/v3/info response."""
|
||||
if data.get("code") != 0:
|
||||
return None
|
||||
bot = data.get("bot") or data.get("data", {}).get("bot") or {}
|
||||
return {
|
||||
"bot_name": bot.get("bot_name"),
|
||||
"bot_open_id": bot.get("open_id"),
|
||||
}
|
||||
|
||||
|
||||
def _probe_bot_sdk(app_id: str, app_secret: str, domain: str) -> Optional[dict]:
|
||||
"""Probe bot info using lark_oapi SDK."""
|
||||
try:
|
||||
client = _build_onboard_client(app_id, app_secret, domain)
|
||||
resp = client.request(
|
||||
method="GET",
|
||||
url="/open-apis/bot/v3/info",
|
||||
body=None,
|
||||
raw_response=True,
|
||||
)
|
||||
return _parse_bot_response(json.loads(resp.content))
|
||||
except Exception as exc:
|
||||
logger.debug("[Feishu onboard] SDK probe failed: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _probe_bot_http(app_id: str, app_secret: str, domain: str) -> Optional[dict]:
|
||||
"""Fallback probe using raw HTTP (when lark_oapi is not installed)."""
|
||||
base_url = _onboard_open_base_url(domain)
|
||||
try:
|
||||
token_data = json.dumps({"app_id": app_id, "app_secret": app_secret}).encode("utf-8")
|
||||
token_req = Request(
|
||||
f"{base_url}/open-apis/auth/v3/tenant_access_token/internal",
|
||||
data=token_data,
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
with urlopen(token_req, timeout=_ONBOARD_REQUEST_TIMEOUT_S) as resp:
|
||||
token_res = json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
access_token = token_res.get("tenant_access_token")
|
||||
if not access_token:
|
||||
return None
|
||||
|
||||
bot_req = Request(
|
||||
f"{base_url}/open-apis/bot/v3/info",
|
||||
headers={
|
||||
"Authorization": f"Bearer {access_token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
with urlopen(bot_req, timeout=_ONBOARD_REQUEST_TIMEOUT_S) as resp:
|
||||
bot_res = json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
return _parse_bot_response(bot_res)
|
||||
except (URLError, OSError, KeyError, json.JSONDecodeError) as exc:
|
||||
logger.debug("[Feishu onboard] HTTP probe failed: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def qr_register(
|
||||
*,
|
||||
initial_domain: str = "feishu",
|
||||
timeout_seconds: int = 600,
|
||||
) -> Optional[dict]:
|
||||
"""Run the Feishu / Lark scan-to-create QR registration flow.
|
||||
|
||||
Returns on success::
|
||||
|
||||
{
|
||||
"app_id": str,
|
||||
"app_secret": str,
|
||||
"domain": "feishu" | "lark",
|
||||
"open_id": str | None,
|
||||
"bot_name": str | None,
|
||||
"bot_open_id": str | None,
|
||||
}
|
||||
|
||||
Returns None on expected failures (network, auth denied, timeout).
|
||||
Unexpected errors (bugs, protocol regressions) propagate to the caller.
|
||||
"""
|
||||
try:
|
||||
return _qr_register_inner(initial_domain=initial_domain, timeout_seconds=timeout_seconds)
|
||||
except (RuntimeError, URLError, OSError, json.JSONDecodeError) as exc:
|
||||
logger.warning("[Feishu onboard] Registration failed: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _qr_register_inner(
|
||||
*,
|
||||
initial_domain: str,
|
||||
timeout_seconds: int,
|
||||
) -> Optional[dict]:
|
||||
"""Run init → begin → poll → probe. Raises on network/protocol errors."""
|
||||
print(" Connecting to Feishu / Lark...", end="", flush=True)
|
||||
_init_registration(initial_domain)
|
||||
begin = _begin_registration(initial_domain)
|
||||
print(" done.")
|
||||
|
||||
print()
|
||||
qr_url = begin["qr_url"]
|
||||
if _render_qr(qr_url):
|
||||
print(f"\n Scan the QR code above, or open this URL directly:\n {qr_url}")
|
||||
else:
|
||||
print(f" Open this URL in Feishu / Lark on your phone:\n\n {qr_url}\n")
|
||||
print(" Tip: pip install qrcode to display a scannable QR code here next time")
|
||||
print()
|
||||
|
||||
result = _poll_registration(
|
||||
device_code=begin["device_code"],
|
||||
interval=begin["interval"],
|
||||
expire_in=min(begin["expire_in"], timeout_seconds),
|
||||
domain=initial_domain,
|
||||
)
|
||||
if not result:
|
||||
return None
|
||||
|
||||
# Probe bot — best-effort, don't fail the registration
|
||||
bot_info = probe_bot(result["app_id"], result["app_secret"], result["domain"])
|
||||
if bot_info:
|
||||
result["bot_name"] = bot_info.get("bot_name")
|
||||
result["bot_open_id"] = bot_info.get("bot_open_id")
|
||||
else:
|
||||
result["bot_name"] = None
|
||||
result["bot_open_id"] = None
|
||||
|
||||
return result
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,429 +0,0 @@
|
||||
"""
|
||||
Feishu document comment access-control rules.
|
||||
|
||||
3-tier rule resolution: exact doc > wildcard "*" > top-level > code defaults.
|
||||
Each field (enabled/policy/allow_from) falls back independently.
|
||||
Config: ~/.hermes/feishu_comment_rules.json (mtime-cached, hot-reload).
|
||||
Pairing store: ~/.hermes/feishu_comment_pairing.json.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Paths
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# Uses the canonical ``get_hermes_home()`` helper (HERMES_HOME-aware and
|
||||
# profile-safe). Resolved at import time; this module is lazy-imported by
|
||||
# the Feishu comment event handler, which runs long after profile overrides
|
||||
# have been applied, so freezing paths here is safe.
|
||||
|
||||
RULES_FILE = get_hermes_home() / "feishu_comment_rules.json"
|
||||
PAIRING_FILE = get_hermes_home() / "feishu_comment_pairing.json"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Data models
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_VALID_POLICIES = ("allowlist", "pairing")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CommentDocumentRule:
|
||||
"""Per-document rule. ``None`` means 'inherit from lower tier'."""
|
||||
enabled: Optional[bool] = None
|
||||
policy: Optional[str] = None
|
||||
allow_from: Optional[frozenset] = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CommentsConfig:
|
||||
"""Top-level comment access config."""
|
||||
enabled: bool = True
|
||||
policy: str = "pairing"
|
||||
allow_from: frozenset = field(default_factory=frozenset)
|
||||
documents: Dict[str, CommentDocumentRule] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ResolvedCommentRule:
|
||||
"""Fully resolved rule after field-by-field fallback."""
|
||||
enabled: bool
|
||||
policy: str
|
||||
allow_from: frozenset
|
||||
match_source: str # e.g. "exact:docx:xxx" | "wildcard" | "top" | "default"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Mtime-cached file loading
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _MtimeCache:
|
||||
"""Generic mtime-based file cache. ``stat()`` per access, re-read only on change."""
|
||||
|
||||
def __init__(self, path: Path):
|
||||
self._path = path
|
||||
self._mtime: float = 0.0
|
||||
self._data: Optional[dict] = None
|
||||
|
||||
def load(self) -> dict:
|
||||
try:
|
||||
st = self._path.stat()
|
||||
mtime = st.st_mtime
|
||||
except FileNotFoundError:
|
||||
self._mtime = 0.0
|
||||
self._data = {}
|
||||
return {}
|
||||
|
||||
if mtime == self._mtime and self._data is not None:
|
||||
return self._data
|
||||
|
||||
try:
|
||||
with open(self._path, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, dict):
|
||||
data = {}
|
||||
except (json.JSONDecodeError, OSError):
|
||||
logger.warning("[Feishu-Rules] Failed to read %s, using empty config", self._path)
|
||||
data = {}
|
||||
|
||||
self._mtime = mtime
|
||||
self._data = data
|
||||
return data
|
||||
|
||||
|
||||
_rules_cache = _MtimeCache(RULES_FILE)
|
||||
_pairing_cache = _MtimeCache(PAIRING_FILE)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _parse_frozenset(raw: Any) -> Optional[frozenset]:
|
||||
"""Parse a list of strings into a frozenset; return None if key absent."""
|
||||
if raw is None:
|
||||
return None
|
||||
if isinstance(raw, (list, tuple)):
|
||||
return frozenset(str(u).strip() for u in raw if str(u).strip())
|
||||
return None
|
||||
|
||||
|
||||
def _parse_document_rule(raw: dict) -> CommentDocumentRule:
|
||||
enabled = raw.get("enabled")
|
||||
if enabled is not None:
|
||||
enabled = bool(enabled)
|
||||
policy = raw.get("policy")
|
||||
if policy is not None:
|
||||
policy = str(policy).strip().lower()
|
||||
if policy not in _VALID_POLICIES:
|
||||
policy = None
|
||||
allow_from = _parse_frozenset(raw.get("allow_from"))
|
||||
return CommentDocumentRule(enabled=enabled, policy=policy, allow_from=allow_from)
|
||||
|
||||
|
||||
def load_config() -> CommentsConfig:
|
||||
"""Load comment rules from disk (mtime-cached)."""
|
||||
raw = _rules_cache.load()
|
||||
if not raw:
|
||||
return CommentsConfig()
|
||||
|
||||
documents: Dict[str, CommentDocumentRule] = {}
|
||||
raw_docs = raw.get("documents", {})
|
||||
if isinstance(raw_docs, dict):
|
||||
for key, rule_raw in raw_docs.items():
|
||||
if isinstance(rule_raw, dict):
|
||||
documents[str(key)] = _parse_document_rule(rule_raw)
|
||||
|
||||
policy = str(raw.get("policy", "pairing")).strip().lower()
|
||||
if policy not in _VALID_POLICIES:
|
||||
policy = "pairing"
|
||||
|
||||
return CommentsConfig(
|
||||
enabled=raw.get("enabled", True),
|
||||
policy=policy,
|
||||
allow_from=_parse_frozenset(raw.get("allow_from")) or frozenset(),
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Rule resolution (§8.4 field-by-field fallback)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def has_wiki_keys(cfg: CommentsConfig) -> bool:
|
||||
"""Check if any document rule key starts with 'wiki:'."""
|
||||
return any(k.startswith("wiki:") for k in cfg.documents)
|
||||
|
||||
|
||||
def resolve_rule(
|
||||
cfg: CommentsConfig,
|
||||
file_type: str,
|
||||
file_token: str,
|
||||
wiki_token: str = "",
|
||||
) -> ResolvedCommentRule:
|
||||
"""Resolve effective rule: exact doc → wiki key → wildcard → top-level → defaults."""
|
||||
exact_key = f"{file_type}:{file_token}"
|
||||
|
||||
exact = cfg.documents.get(exact_key)
|
||||
exact_src = f"exact:{exact_key}"
|
||||
if exact is None and wiki_token:
|
||||
wiki_key = f"wiki:{wiki_token}"
|
||||
exact = cfg.documents.get(wiki_key)
|
||||
exact_src = f"exact:{wiki_key}"
|
||||
|
||||
wildcard = cfg.documents.get("*")
|
||||
|
||||
layers = []
|
||||
if exact is not None:
|
||||
layers.append((exact, exact_src))
|
||||
if wildcard is not None:
|
||||
layers.append((wildcard, "wildcard"))
|
||||
|
||||
def _pick(field_name: str):
|
||||
for layer, source in layers:
|
||||
val = getattr(layer, field_name)
|
||||
if val is not None:
|
||||
return val, source
|
||||
return getattr(cfg, field_name), "top"
|
||||
|
||||
enabled, en_src = _pick("enabled")
|
||||
policy, pol_src = _pick("policy")
|
||||
allow_from, _ = _pick("allow_from")
|
||||
|
||||
# match_source = highest-priority tier that contributed any field
|
||||
priority_order = {"exact": 0, "wildcard": 1, "top": 2}
|
||||
best_src = min(
|
||||
[en_src, pol_src],
|
||||
key=lambda s: priority_order.get(s.split(":")[0], 3),
|
||||
)
|
||||
|
||||
return ResolvedCommentRule(
|
||||
enabled=enabled,
|
||||
policy=policy,
|
||||
allow_from=allow_from,
|
||||
match_source=best_src,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pairing store
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _load_pairing_approved() -> set:
|
||||
"""Return set of approved user open_ids (mtime-cached)."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if isinstance(approved, dict):
|
||||
return set(approved.keys())
|
||||
if isinstance(approved, list):
|
||||
return set(str(u) for u in approved if u)
|
||||
return set()
|
||||
|
||||
|
||||
def _save_pairing(data: dict) -> None:
|
||||
PAIRING_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = PAIRING_FILE.with_suffix(".tmp")
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
tmp.replace(PAIRING_FILE)
|
||||
# Invalidate cache so next load picks up change
|
||||
_pairing_cache._mtime = 0.0
|
||||
_pairing_cache._data = None
|
||||
|
||||
|
||||
def pairing_add(user_open_id: str) -> bool:
|
||||
"""Add a user to the pairing-approved list. Returns True if newly added."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if not isinstance(approved, dict):
|
||||
approved = {}
|
||||
if user_open_id in approved:
|
||||
return False
|
||||
approved[user_open_id] = {"approved_at": time.time()}
|
||||
data["approved"] = approved
|
||||
_save_pairing(data)
|
||||
return True
|
||||
|
||||
|
||||
def pairing_remove(user_open_id: str) -> bool:
|
||||
"""Remove a user from the pairing-approved list. Returns True if removed."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if not isinstance(approved, dict):
|
||||
return False
|
||||
if user_open_id not in approved:
|
||||
return False
|
||||
del approved[user_open_id]
|
||||
data["approved"] = approved
|
||||
_save_pairing(data)
|
||||
return True
|
||||
|
||||
|
||||
def pairing_list() -> Dict[str, Any]:
|
||||
"""Return the approved dict {user_open_id: {approved_at: ...}}."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
return dict(approved) if isinstance(approved, dict) else {}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Access check (public API for feishu_comment.py)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def is_user_allowed(rule: ResolvedCommentRule, user_open_id: str) -> bool:
|
||||
"""Check if user passes the resolved rule's policy gate."""
|
||||
if user_open_id in rule.allow_from:
|
||||
return True
|
||||
if rule.policy == "pairing":
|
||||
return user_open_id in _load_pairing_approved()
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _print_status() -> None:
|
||||
cfg = load_config()
|
||||
print(f"Rules file: {RULES_FILE}")
|
||||
print(f" exists: {RULES_FILE.exists()}")
|
||||
print(f"Pairing file: {PAIRING_FILE}")
|
||||
print(f" exists: {PAIRING_FILE.exists()}")
|
||||
print()
|
||||
print(f"Top-level:")
|
||||
print(f" enabled: {cfg.enabled}")
|
||||
print(f" policy: {cfg.policy}")
|
||||
print(f" allow_from: {sorted(cfg.allow_from) if cfg.allow_from else '[]'}")
|
||||
print()
|
||||
if cfg.documents:
|
||||
print(f"Document rules ({len(cfg.documents)}):")
|
||||
for key, rule in sorted(cfg.documents.items()):
|
||||
parts = []
|
||||
if rule.enabled is not None:
|
||||
parts.append(f"enabled={rule.enabled}")
|
||||
if rule.policy is not None:
|
||||
parts.append(f"policy={rule.policy}")
|
||||
if rule.allow_from is not None:
|
||||
parts.append(f"allow_from={sorted(rule.allow_from)}")
|
||||
print(f" [{key}] {', '.join(parts) if parts else '(empty — inherits all)'}")
|
||||
else:
|
||||
print("Document rules: (none)")
|
||||
print()
|
||||
approved = pairing_list()
|
||||
print(f"Pairing approved ({len(approved)}):")
|
||||
for uid, meta in sorted(approved.items()):
|
||||
ts = meta.get("approved_at", 0)
|
||||
print(f" {uid} (approved_at={ts})")
|
||||
|
||||
|
||||
def _do_check(doc_key: str, user_open_id: str) -> None:
|
||||
cfg = load_config()
|
||||
parts = doc_key.split(":", 1)
|
||||
if len(parts) != 2:
|
||||
print(f"Error: doc_key must be 'fileType:fileToken', got '{doc_key}'")
|
||||
return
|
||||
file_type, file_token = parts
|
||||
rule = resolve_rule(cfg, file_type, file_token)
|
||||
allowed = is_user_allowed(rule, user_open_id)
|
||||
print(f"Document: {doc_key}")
|
||||
print(f"User: {user_open_id}")
|
||||
print(f"Resolved rule:")
|
||||
print(f" enabled: {rule.enabled}")
|
||||
print(f" policy: {rule.policy}")
|
||||
print(f" allow_from: {sorted(rule.allow_from) if rule.allow_from else '[]'}")
|
||||
print(f" match_source: {rule.match_source}")
|
||||
print(f"Result: {'ALLOWED' if allowed else 'DENIED'}")
|
||||
|
||||
|
||||
def _main() -> int:
|
||||
import sys
|
||||
|
||||
try:
|
||||
from hermes_cli.env_loader import load_hermes_dotenv
|
||||
load_hermes_dotenv()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
usage = (
|
||||
"Usage: python -m gateway.platforms.feishu_comment_rules <command> [args]\n"
|
||||
"\n"
|
||||
"Commands:\n"
|
||||
" status Show rules config and pairing state\n"
|
||||
" check <fileType:token> <user> Simulate access check\n"
|
||||
" pairing add <user_open_id> Add user to pairing-approved list\n"
|
||||
" pairing remove <user_open_id> Remove user from pairing-approved list\n"
|
||||
" pairing list List pairing-approved users\n"
|
||||
"\n"
|
||||
f"Rules config file: {RULES_FILE}\n"
|
||||
" Edit this JSON file directly to configure policies and document rules.\n"
|
||||
" Changes take effect on the next comment event (no restart needed).\n"
|
||||
)
|
||||
|
||||
args = sys.argv[1:]
|
||||
if not args:
|
||||
print(usage)
|
||||
return 1
|
||||
|
||||
cmd = args[0]
|
||||
|
||||
if cmd == "status":
|
||||
_print_status()
|
||||
|
||||
elif cmd == "check":
|
||||
if len(args) < 3:
|
||||
print("Usage: check <fileType:fileToken> <user_open_id>")
|
||||
return 1
|
||||
_do_check(args[1], args[2])
|
||||
|
||||
elif cmd == "pairing":
|
||||
if len(args) < 2:
|
||||
print("Usage: pairing <add|remove|list> [args]")
|
||||
return 1
|
||||
sub = args[1]
|
||||
if sub == "add":
|
||||
if len(args) < 3:
|
||||
print("Usage: pairing add <user_open_id>")
|
||||
return 1
|
||||
if pairing_add(args[2]):
|
||||
print(f"Added: {args[2]}")
|
||||
else:
|
||||
print(f"Already approved: {args[2]}")
|
||||
elif sub == "remove":
|
||||
if len(args) < 3:
|
||||
print("Usage: pairing remove <user_open_id>")
|
||||
return 1
|
||||
if pairing_remove(args[2]):
|
||||
print(f"Removed: {args[2]}")
|
||||
else:
|
||||
print(f"Not in approved list: {args[2]}")
|
||||
elif sub == "list":
|
||||
approved = pairing_list()
|
||||
if not approved:
|
||||
print("(no approved users)")
|
||||
for uid, meta in sorted(approved.items()):
|
||||
print(f" {uid} approved_at={meta.get('approved_at', '?')}")
|
||||
else:
|
||||
print(f"Unknown pairing subcommand: {sub}")
|
||||
return 1
|
||||
else:
|
||||
print(f"Unknown command: {cmd}\n")
|
||||
print(usage)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
sys.exit(_main())
|
||||
@@ -49,10 +49,7 @@ class MessageDeduplicator:
|
||||
return False
|
||||
now = time.time()
|
||||
if msg_id in self._seen:
|
||||
if now - self._seen[msg_id] < self._ttl:
|
||||
return True
|
||||
# Entry has expired — remove it and treat as new
|
||||
del self._seen[msg_id]
|
||||
return True
|
||||
self._seen[msg_id] = now
|
||||
if len(self._seen) > self._max_size:
|
||||
cutoff = now - self._ttl
|
||||
|
||||
+280
-465
File diff suppressed because it is too large
Load Diff
@@ -718,12 +718,6 @@ class MattermostAdapter(BasePlatformAdapter):
|
||||
thread_id=thread_id,
|
||||
)
|
||||
|
||||
# Per-channel ephemeral prompt
|
||||
from gateway.platforms.base import resolve_channel_prompt
|
||||
_channel_prompt = resolve_channel_prompt(
|
||||
self.config.extra, channel_id, None,
|
||||
)
|
||||
|
||||
msg_event = MessageEvent(
|
||||
text=message_text,
|
||||
message_type=msg_type,
|
||||
@@ -732,7 +726,6 @@ class MattermostAdapter(BasePlatformAdapter):
|
||||
message_id=post_id,
|
||||
media_urls=media_urls if media_urls else None,
|
||||
media_types=media_types if media_types else None,
|
||||
channel_prompt=_channel_prompt,
|
||||
)
|
||||
|
||||
await self.handle_message(msg_event)
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
"""
|
||||
QQBot platform package.
|
||||
|
||||
Re-exports the main adapter symbols from ``adapter.py`` (the original
|
||||
``qqbot.py``) so that **all existing import paths remain unchanged**::
|
||||
|
||||
from gateway.platforms.qqbot import QQAdapter # works
|
||||
from gateway.platforms.qqbot import check_qq_requirements # works
|
||||
|
||||
New modules:
|
||||
- ``constants`` — shared constants (API URLs, timeouts, message types)
|
||||
- ``utils`` — User-Agent builder, config helpers
|
||||
- ``crypto`` — AES-256-GCM key generation and decryption
|
||||
- ``onboard`` — QR-code scan-to-configure flow
|
||||
"""
|
||||
|
||||
# -- Adapter (original qqbot.py) ------------------------------------------
|
||||
from .adapter import ( # noqa: F401
|
||||
QQAdapter,
|
||||
QQCloseError,
|
||||
check_qq_requirements,
|
||||
_coerce_list,
|
||||
_ssrf_redirect_guard,
|
||||
)
|
||||
|
||||
# -- Onboard (QR-code scan-to-configure) -----------------------------------
|
||||
from .onboard import ( # noqa: F401
|
||||
BindStatus,
|
||||
create_bind_task,
|
||||
poll_bind_result,
|
||||
build_connect_url,
|
||||
)
|
||||
from .crypto import decrypt_secret, generate_bind_key # noqa: F401
|
||||
|
||||
# -- Utils -----------------------------------------------------------------
|
||||
from .utils import build_user_agent, get_api_headers, coerce_list # noqa: F401
|
||||
|
||||
__all__ = [
|
||||
# adapter
|
||||
"QQAdapter",
|
||||
"QQCloseError",
|
||||
"check_qq_requirements",
|
||||
"_coerce_list",
|
||||
"_ssrf_redirect_guard",
|
||||
# onboard
|
||||
"BindStatus",
|
||||
"create_bind_task",
|
||||
"poll_bind_result",
|
||||
"build_connect_url",
|
||||
# crypto
|
||||
"decrypt_secret",
|
||||
"generate_bind_key",
|
||||
# utils
|
||||
"build_user_agent",
|
||||
"get_api_headers",
|
||||
"coerce_list",
|
||||
]
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,74 +0,0 @@
|
||||
"""QQBot package-level constants shared across adapter, onboard, and other modules."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQBot adapter version — bump on functional changes to the adapter package.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
QQBOT_VERSION = "1.1.0"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# API endpoints
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# The portal domain is configurable via QQ_API_HOST for corporate proxies
|
||||
# or test environments. Default: q.qq.com (production).
|
||||
PORTAL_HOST = os.getenv("QQ_PORTAL_HOST", "q.qq.com")
|
||||
|
||||
API_BASE = "https://api.sgroup.qq.com"
|
||||
TOKEN_URL = "https://bots.qq.com/app/getAppAccessToken"
|
||||
GATEWAY_URL_PATH = "/gateway"
|
||||
|
||||
# QR-code onboard endpoints (on the portal host)
|
||||
ONBOARD_CREATE_PATH = "/lite/create_bind_task"
|
||||
ONBOARD_POLL_PATH = "/lite/poll_bind_result"
|
||||
QR_URL_TEMPLATE = (
|
||||
"https://q.qq.com/qqbot/openclaw/connect.html"
|
||||
"?task_id={task_id}&_wv=2&source=hermes"
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Timeouts & retry
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
DEFAULT_API_TIMEOUT = 30.0
|
||||
FILE_UPLOAD_TIMEOUT = 120.0
|
||||
CONNECT_TIMEOUT_SECONDS = 20.0
|
||||
|
||||
RECONNECT_BACKOFF = [2, 5, 10, 30, 60]
|
||||
MAX_RECONNECT_ATTEMPTS = 100
|
||||
RATE_LIMIT_DELAY = 60 # seconds
|
||||
QUICK_DISCONNECT_THRESHOLD = 5.0 # seconds
|
||||
MAX_QUICK_DISCONNECT_COUNT = 3
|
||||
|
||||
ONBOARD_POLL_INTERVAL = 2.0 # seconds between poll_bind_result calls
|
||||
ONBOARD_API_TIMEOUT = 10.0
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Message limits
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MAX_MESSAGE_LENGTH = 4000
|
||||
DEDUP_WINDOW_SECONDS = 300
|
||||
DEDUP_MAX_SIZE = 1000
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQ Bot message types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MSG_TYPE_TEXT = 0
|
||||
MSG_TYPE_MARKDOWN = 2
|
||||
MSG_TYPE_MEDIA = 7
|
||||
MSG_TYPE_INPUT_NOTIFY = 6
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQ Bot file media types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MEDIA_TYPE_IMAGE = 1
|
||||
MEDIA_TYPE_VIDEO = 2
|
||||
MEDIA_TYPE_VOICE = 3
|
||||
MEDIA_TYPE_FILE = 4
|
||||
@@ -1,45 +0,0 @@
|
||||
"""AES-256-GCM utilities for QQBot scan-to-configure credential decryption."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import os
|
||||
|
||||
|
||||
def generate_bind_key() -> str:
|
||||
"""Generate a 256-bit random AES key and return it as base64.
|
||||
|
||||
The key is passed to ``create_bind_task`` so the server can encrypt
|
||||
the bot's *client_secret* before returning it. Only this CLI holds
|
||||
the key, ensuring the secret never travels in plaintext.
|
||||
"""
|
||||
return base64.b64encode(os.urandom(32)).decode()
|
||||
|
||||
|
||||
def decrypt_secret(encrypted_base64: str, key_base64: str) -> str:
|
||||
"""Decrypt a base64-encoded AES-256-GCM ciphertext.
|
||||
|
||||
Ciphertext layout (after base64-decoding)::
|
||||
|
||||
IV (12 bytes) ‖ ciphertext (N bytes) ‖ AuthTag (16 bytes)
|
||||
|
||||
Args:
|
||||
encrypted_base64: The ``bot_encrypt_secret`` value from
|
||||
``poll_bind_result``.
|
||||
key_base64: The base64 AES key generated by
|
||||
:func:`generate_bind_key`.
|
||||
|
||||
Returns:
|
||||
The decrypted *client_secret* as a UTF-8 string.
|
||||
"""
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
|
||||
key = base64.b64decode(key_base64)
|
||||
raw = base64.b64decode(encrypted_base64)
|
||||
|
||||
iv = raw[:12]
|
||||
ciphertext_with_tag = raw[12:] # AESGCM expects ciphertext + tag concatenated
|
||||
|
||||
aesgcm = AESGCM(key)
|
||||
plaintext = aesgcm.decrypt(iv, ciphertext_with_tag, None)
|
||||
return plaintext.decode("utf-8")
|
||||
@@ -1,124 +0,0 @@
|
||||
"""
|
||||
QQBot scan-to-configure (QR code onboard) module.
|
||||
|
||||
Calls the ``q.qq.com`` ``create_bind_task`` / ``poll_bind_result`` APIs to
|
||||
generate a QR-code URL and poll for scan completion. On success the caller
|
||||
receives the bot's *app_id*, *client_secret* (decrypted locally), and the
|
||||
scanner's *user_openid* — enough to fully configure the QQBot gateway.
|
||||
|
||||
Reference: https://bot.q.qq.com/wiki/develop/api-v2/
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from enum import IntEnum
|
||||
from typing import Tuple
|
||||
from urllib.parse import quote
|
||||
|
||||
from .constants import (
|
||||
ONBOARD_API_TIMEOUT,
|
||||
ONBOARD_CREATE_PATH,
|
||||
ONBOARD_POLL_PATH,
|
||||
PORTAL_HOST,
|
||||
QR_URL_TEMPLATE,
|
||||
)
|
||||
from .crypto import generate_bind_key
|
||||
from .utils import get_api_headers
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bind status
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class BindStatus(IntEnum):
|
||||
"""Status codes returned by ``poll_bind_result``."""
|
||||
|
||||
NONE = 0
|
||||
PENDING = 1
|
||||
COMPLETED = 2
|
||||
EXPIRED = 3
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def create_bind_task(
|
||||
timeout: float = ONBOARD_API_TIMEOUT,
|
||||
) -> Tuple[str, str]:
|
||||
"""Create a bind task and return *(task_id, aes_key_base64)*.
|
||||
|
||||
The AES key is generated locally and sent to the server so it can
|
||||
encrypt the bot credentials before returning them.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the API returns a non-zero ``retcode``.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
url = f"https://{PORTAL_HOST}{ONBOARD_CREATE_PATH}"
|
||||
key = generate_bind_key()
|
||||
|
||||
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
|
||||
resp = await client.post(url, json={"key": key}, headers=get_api_headers())
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
if data.get("retcode") != 0:
|
||||
raise RuntimeError(data.get("msg", "create_bind_task failed"))
|
||||
|
||||
task_id = data.get("data", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise RuntimeError("create_bind_task: missing task_id in response")
|
||||
|
||||
logger.debug("create_bind_task ok: task_id=%s", task_id)
|
||||
return task_id, key
|
||||
|
||||
|
||||
async def poll_bind_result(
|
||||
task_id: str,
|
||||
timeout: float = ONBOARD_API_TIMEOUT,
|
||||
) -> Tuple[BindStatus, str, str, str]:
|
||||
"""Poll the bind result for *task_id*.
|
||||
|
||||
Returns:
|
||||
A 4-tuple of ``(status, bot_appid, bot_encrypt_secret, user_openid)``.
|
||||
|
||||
* ``bot_encrypt_secret`` is AES-256-GCM encrypted — decrypt it with
|
||||
:func:`~gateway.platforms.qqbot.crypto.decrypt_secret` using the
|
||||
key from :func:`create_bind_task`.
|
||||
* ``user_openid`` is the OpenID of the person who scanned the code
|
||||
(available when ``status == COMPLETED``).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the API returns a non-zero ``retcode``.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
url = f"https://{PORTAL_HOST}{ONBOARD_POLL_PATH}"
|
||||
|
||||
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
|
||||
resp = await client.post(url, json={"task_id": task_id}, headers=get_api_headers())
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
if data.get("retcode") != 0:
|
||||
raise RuntimeError(data.get("msg", "poll_bind_result failed"))
|
||||
|
||||
d = data.get("data", {})
|
||||
return (
|
||||
BindStatus(d.get("status", 0)),
|
||||
str(d.get("bot_appid", "")),
|
||||
d.get("bot_encrypt_secret", ""),
|
||||
d.get("user_openid", ""),
|
||||
)
|
||||
|
||||
|
||||
def build_connect_url(task_id: str) -> str:
|
||||
"""Build the QR-code target URL for a given *task_id*."""
|
||||
return QR_URL_TEMPLATE.format(task_id=quote(task_id))
|
||||
@@ -1,71 +0,0 @@
|
||||
"""QQBot shared utilities — User-Agent, HTTP helpers, config coercion."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import platform
|
||||
import sys
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from .constants import QQBOT_VERSION
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# User-Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _get_hermes_version() -> str:
|
||||
"""Return the hermes-agent package version, or 'dev' if unavailable."""
|
||||
try:
|
||||
from importlib.metadata import version
|
||||
return version("hermes-agent")
|
||||
except Exception:
|
||||
return "dev"
|
||||
|
||||
|
||||
def build_user_agent() -> str:
|
||||
"""Build a descriptive User-Agent string.
|
||||
|
||||
Format::
|
||||
|
||||
QQBotAdapter/<qqbot_version> (Python/<py_version>; <os>; Hermes/<hermes_version>)
|
||||
|
||||
Example::
|
||||
|
||||
QQBotAdapter/1.0.0 (Python/3.11.15; darwin; Hermes/0.9.0)
|
||||
"""
|
||||
py_version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
|
||||
os_name = platform.system().lower()
|
||||
hermes_version = _get_hermes_version()
|
||||
return f"QQBotAdapter/{QQBOT_VERSION} (Python/{py_version}; {os_name}; Hermes/{hermes_version})"
|
||||
|
||||
|
||||
def get_api_headers() -> Dict[str, str]:
|
||||
"""Return standard HTTP headers for QQBot API requests.
|
||||
|
||||
Includes ``Content-Type``, ``Accept``, and a dynamic ``User-Agent``.
|
||||
``q.qq.com`` requires ``Accept: application/json`` — without it,
|
||||
the server returns a JavaScript anti-bot challenge page.
|
||||
"""
|
||||
return {
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json",
|
||||
"User-Agent": build_user_agent(),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def coerce_list(value: Any) -> List[str]:
|
||||
"""Coerce config values into a trimmed string list.
|
||||
|
||||
Accepts comma-separated strings, lists, tuples, sets, or single values.
|
||||
"""
|
||||
if value is None:
|
||||
return []
|
||||
if isinstance(value, str):
|
||||
return [item.strip() for item in value.split(",") if item.strip()]
|
||||
if isinstance(value, (list, tuple, set)):
|
||||
return [str(item).strip() for item in value if str(item).strip()]
|
||||
return [str(value).strip()] if str(value).strip() else []
|
||||
+22
-78
@@ -17,6 +17,7 @@ import json
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
@@ -160,14 +161,6 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
self._sse_task: Optional[asyncio.Task] = None
|
||||
self._health_monitor_task: Optional[asyncio.Task] = None
|
||||
self._typing_tasks: Dict[str, asyncio.Task] = {}
|
||||
# Per-chat typing-indicator backoff. When signal-cli reports
|
||||
# NETWORK_FAILURE (recipient offline / unroutable), base.py's
|
||||
# _keep_typing refresh loop would otherwise hammer sendTyping every
|
||||
# ~2s indefinitely, producing WARNING-level log spam and pointless
|
||||
# RPC traffic. We track consecutive failures per chat and skip the
|
||||
# RPC during a cooldown window instead.
|
||||
self._typing_failures: Dict[str, int] = {}
|
||||
self._typing_skip_until: Dict[str, float] = {}
|
||||
self._running = False
|
||||
self._last_sse_activity = 0.0
|
||||
self._sse_response: Optional[httpx.Response] = None
|
||||
@@ -556,22 +549,8 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
# JSON-RPC Communication
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _rpc(
|
||||
self,
|
||||
method: str,
|
||||
params: dict,
|
||||
rpc_id: str = None,
|
||||
*,
|
||||
log_failures: bool = True,
|
||||
) -> Any:
|
||||
"""Send a JSON-RPC 2.0 request to signal-cli daemon.
|
||||
|
||||
When ``log_failures=False``, error and exception paths log at DEBUG
|
||||
instead of WARNING — used by the typing-indicator path to silence
|
||||
repeated NETWORK_FAILURE spam for unreachable recipients while
|
||||
still preserving visibility for the first occurrence and for
|
||||
unrelated RPCs.
|
||||
"""
|
||||
async def _rpc(self, method: str, params: dict, rpc_id: str = None) -> Any:
|
||||
"""Send a JSON-RPC 2.0 request to signal-cli daemon."""
|
||||
if not self.client:
|
||||
logger.warning("Signal: RPC called but client not connected")
|
||||
return None
|
||||
@@ -596,19 +575,13 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
data = resp.json()
|
||||
|
||||
if "error" in data:
|
||||
if log_failures:
|
||||
logger.warning("Signal RPC error (%s): %s", method, data["error"])
|
||||
else:
|
||||
logger.debug("Signal RPC error (%s): %s", method, data["error"])
|
||||
logger.warning("Signal RPC error (%s): %s", method, data["error"])
|
||||
return None
|
||||
|
||||
return data.get("result")
|
||||
|
||||
except Exception as e:
|
||||
if log_failures:
|
||||
logger.warning("Signal RPC %s failed: %s", method, e)
|
||||
else:
|
||||
logger.debug("Signal RPC %s failed: %s", method, e)
|
||||
logger.warning("Signal RPC %s failed: %s", method, e)
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -655,28 +628,7 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
self._recent_sent_timestamps.pop()
|
||||
|
||||
async def send_typing(self, chat_id: str, metadata=None) -> None:
|
||||
"""Send a typing indicator.
|
||||
|
||||
base.py's ``_keep_typing`` refresh loop calls this every ~2s while
|
||||
the agent is processing. If signal-cli returns NETWORK_FAILURE for
|
||||
this recipient (offline, unroutable, group membership lost, etc.)
|
||||
the unmitigated behaviour is: a WARNING log every 2 seconds for as
|
||||
long as the agent keeps running. Instead we:
|
||||
|
||||
- silence the WARNING after the first consecutive failure (subsequent
|
||||
attempts log at DEBUG) so transport issues are still visible once
|
||||
but don't flood the log,
|
||||
- skip the RPC entirely during an exponential cooldown window once
|
||||
three consecutive failures have happened, so we stop hammering
|
||||
signal-cli with requests it can't deliver.
|
||||
|
||||
A successful sendTyping clears the counters.
|
||||
"""
|
||||
now = time.monotonic()
|
||||
skip_until = self._typing_skip_until.get(chat_id, 0.0)
|
||||
if now < skip_until:
|
||||
return
|
||||
|
||||
"""Send a typing indicator."""
|
||||
params: Dict[str, Any] = {
|
||||
"account": self.account,
|
||||
}
|
||||
@@ -686,26 +638,7 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
else:
|
||||
params["recipient"] = [chat_id]
|
||||
|
||||
fails = self._typing_failures.get(chat_id, 0)
|
||||
result = await self._rpc(
|
||||
"sendTyping",
|
||||
params,
|
||||
rpc_id="typing",
|
||||
log_failures=(fails == 0),
|
||||
)
|
||||
|
||||
if result is None:
|
||||
fails += 1
|
||||
self._typing_failures[chat_id] = fails
|
||||
# After 3 consecutive failures, back off exponentially (16s,
|
||||
# 32s, 60s cap) to stop spamming signal-cli for a recipient
|
||||
# that clearly isn't reachable right now.
|
||||
if fails >= 3:
|
||||
backoff = min(60.0, 16.0 * (2 ** (fails - 3)))
|
||||
self._typing_skip_until[chat_id] = now + backoff
|
||||
else:
|
||||
self._typing_failures.pop(chat_id, None)
|
||||
self._typing_skip_until.pop(chat_id, None)
|
||||
await self._rpc("sendTyping", params, rpc_id="typing")
|
||||
|
||||
async def send_image(
|
||||
self,
|
||||
@@ -848,6 +781,21 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
# Typing Indicators
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _start_typing_indicator(self, chat_id: str) -> None:
|
||||
"""Start a typing indicator loop for a chat."""
|
||||
if chat_id in self._typing_tasks:
|
||||
return # Already running
|
||||
|
||||
async def _typing_loop():
|
||||
try:
|
||||
while True:
|
||||
await self.send_typing(chat_id)
|
||||
await asyncio.sleep(TYPING_INTERVAL)
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
self._typing_tasks[chat_id] = asyncio.create_task(_typing_loop())
|
||||
|
||||
async def _stop_typing_indicator(self, chat_id: str) -> None:
|
||||
"""Stop a typing indicator loop for a chat."""
|
||||
task = self._typing_tasks.pop(chat_id, None)
|
||||
@@ -857,10 +805,6 @@ class SignalAdapter(BasePlatformAdapter):
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
# Reset per-chat typing backoff state so the next agent turn starts
|
||||
# fresh rather than inheriting a cooldown from a prior conversation.
|
||||
self._typing_failures.pop(chat_id, None)
|
||||
self._typing_skip_until.pop(chat_id, None)
|
||||
|
||||
async def stop_typing(self, chat_id: str) -> None:
|
||||
"""Public interface for stopping typing — called by base adapter's
|
||||
|
||||
@@ -366,20 +366,6 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
# in an assistant-enabled context. Falls back to reactions.
|
||||
logger.debug("[Slack] assistant.threads.setStatus failed: %s", e)
|
||||
|
||||
def _dm_top_level_threads_as_sessions(self) -> bool:
|
||||
"""Whether top-level Slack DMs get per-message session threads.
|
||||
|
||||
Defaults to ``True`` so each visible DM reply thread is isolated as its
|
||||
own Hermes session — matching the per-thread behavior channels already
|
||||
have. Set ``platforms.slack.extra.dm_top_level_threads_as_sessions``
|
||||
to ``false`` in config.yaml to revert to the legacy behavior where all
|
||||
top-level DMs share one continuous session.
|
||||
"""
|
||||
raw = self.config.extra.get("dm_top_level_threads_as_sessions")
|
||||
if raw is None:
|
||||
return True # default: each DM thread is its own session
|
||||
return str(raw).strip().lower() in ("1", "true", "yes", "on")
|
||||
|
||||
def _resolve_thread_ts(
|
||||
self,
|
||||
reply_to: Optional[str] = None,
|
||||
@@ -1010,14 +996,10 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
# Build thread_ts for session keying.
|
||||
# In channels: fall back to ts so each top-level @mention starts a
|
||||
# new thread/session (the bot always replies in a thread).
|
||||
# In DMs: fall back to ts so each top-level DM reply thread gets
|
||||
# its own session key (matching channel behavior). Set
|
||||
# dm_top_level_threads_as_sessions: false in config to revert to
|
||||
# legacy single-session-per-DM-channel behavior.
|
||||
# In DMs: only use the real thread_ts — top-level DMs should share
|
||||
# one continuous session, threaded DMs get their own session.
|
||||
if is_dm:
|
||||
thread_ts = event.get("thread_ts") or assistant_meta.get("thread_ts")
|
||||
if not thread_ts and self._dm_top_level_threads_as_sessions():
|
||||
thread_ts = ts
|
||||
thread_ts = event.get("thread_ts") or assistant_meta.get("thread_ts") # None for top-level DMs
|
||||
else:
|
||||
thread_ts = event.get("thread_ts") or ts # ts fallback for channels
|
||||
|
||||
@@ -1185,12 +1167,6 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
thread_id=thread_ts,
|
||||
)
|
||||
|
||||
# Per-channel ephemeral prompt
|
||||
from gateway.platforms.base import resolve_channel_prompt
|
||||
_channel_prompt = resolve_channel_prompt(
|
||||
self.config.extra, channel_id, None,
|
||||
)
|
||||
|
||||
msg_event = MessageEvent(
|
||||
text=text,
|
||||
message_type=msg_type,
|
||||
@@ -1200,7 +1176,6 @@ class SlackAdapter(BasePlatformAdapter):
|
||||
media_urls=media_urls,
|
||||
media_types=media_types,
|
||||
reply_to_message_id=thread_ts if thread_ts != ts else None,
|
||||
channel_prompt=_channel_prompt,
|
||||
)
|
||||
|
||||
# Only react when bot is directly addressed (DM or @mention).
|
||||
|
||||
+60
-290
@@ -11,7 +11,6 @@ import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import html as _html
|
||||
import re
|
||||
from typing import Dict, List, Optional, Any
|
||||
|
||||
@@ -19,10 +18,6 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
try:
|
||||
from telegram import Update, Bot, Message, InlineKeyboardButton, InlineKeyboardMarkup
|
||||
try:
|
||||
from telegram import LinkPreviewOptions
|
||||
except ImportError:
|
||||
LinkPreviewOptions = None
|
||||
from telegram.ext import (
|
||||
Application,
|
||||
CommandHandler,
|
||||
@@ -41,7 +36,6 @@ except ImportError:
|
||||
Message = Any
|
||||
InlineKeyboardButton = Any
|
||||
InlineKeyboardMarkup = Any
|
||||
LinkPreviewOptions = None
|
||||
Application = Any
|
||||
CommandHandler = Any
|
||||
CallbackQueryHandler = Any
|
||||
@@ -71,10 +65,7 @@ from gateway.platforms.base import (
|
||||
cache_image_from_bytes,
|
||||
cache_audio_from_bytes,
|
||||
cache_document_from_bytes,
|
||||
resolve_proxy_url,
|
||||
SUPPORTED_DOCUMENT_TYPES,
|
||||
utf16_len,
|
||||
_prefix_within_utf16_limit,
|
||||
)
|
||||
from gateway.platforms.telegram_network import (
|
||||
TelegramFallbackTransport,
|
||||
@@ -118,84 +109,6 @@ def _strip_mdv2(text: str) -> str:
|
||||
return cleaned
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Markdown table → code block conversion
|
||||
# ---------------------------------------------------------------------------
|
||||
# Telegram's MarkdownV2 has no table syntax — '|' is just an escaped literal,
|
||||
# so pipe tables render as noisy backslash-pipe text with no alignment.
|
||||
# Wrapping the table in a fenced code block makes Telegram render it as
|
||||
# monospace preformatted text with columns intact.
|
||||
|
||||
# Matches a GFM table delimiter row: optional outer pipes, cells containing
|
||||
# only dashes (with optional leading/trailing colons for alignment) separated
|
||||
# by '|'. Requires at least one internal '|' so lone '---' horizontal rules
|
||||
# are NOT matched.
|
||||
_TABLE_SEPARATOR_RE = re.compile(
|
||||
r'^\s*\|?\s*:?-+:?\s*(?:\|\s*:?-+:?\s*){1,}\|?\s*$'
|
||||
)
|
||||
|
||||
|
||||
def _is_table_row(line: str) -> bool:
|
||||
"""Return True if *line* could plausibly be a table data row."""
|
||||
stripped = line.strip()
|
||||
return bool(stripped) and '|' in stripped
|
||||
|
||||
|
||||
def _wrap_markdown_tables(text: str) -> str:
|
||||
"""Wrap GFM-style pipe tables in ``` fences so Telegram renders them.
|
||||
|
||||
Detected by a row containing '|' immediately followed by a delimiter
|
||||
row matching :data:`_TABLE_SEPARATOR_RE`. Subsequent pipe-containing
|
||||
non-blank lines are consumed as the table body and included in the
|
||||
wrapped block. Tables inside existing fenced code blocks are left
|
||||
alone.
|
||||
"""
|
||||
if '|' not in text or '-' not in text:
|
||||
return text
|
||||
|
||||
lines = text.split('\n')
|
||||
out: list[str] = []
|
||||
in_fence = False
|
||||
i = 0
|
||||
while i < len(lines):
|
||||
line = lines[i]
|
||||
stripped = line.lstrip()
|
||||
|
||||
# Track existing fenced code blocks — never touch content inside.
|
||||
if stripped.startswith('```'):
|
||||
in_fence = not in_fence
|
||||
out.append(line)
|
||||
i += 1
|
||||
continue
|
||||
if in_fence:
|
||||
out.append(line)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# Look for a header row (contains '|') immediately followed by a
|
||||
# delimiter row.
|
||||
if (
|
||||
'|' in line
|
||||
and i + 1 < len(lines)
|
||||
and _TABLE_SEPARATOR_RE.match(lines[i + 1])
|
||||
):
|
||||
table_block = [line, lines[i + 1]]
|
||||
j = i + 2
|
||||
while j < len(lines) and _is_table_row(lines[j]):
|
||||
table_block.append(lines[j])
|
||||
j += 1
|
||||
out.append('```')
|
||||
out.extend(table_block)
|
||||
out.append('```')
|
||||
i = j
|
||||
continue
|
||||
|
||||
out.append(line)
|
||||
i += 1
|
||||
|
||||
return '\n'.join(out)
|
||||
|
||||
|
||||
class TelegramAdapter(BasePlatformAdapter):
|
||||
"""
|
||||
Telegram bot adapter.
|
||||
@@ -213,7 +126,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
# When a chunk is near this limit, a continuation is almost certain.
|
||||
_SPLIT_THRESHOLD = 4000
|
||||
MEDIA_GROUP_WAIT_SECONDS = 0.8
|
||||
_GENERAL_TOPIC_THREAD_ID = "1"
|
||||
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.TELEGRAM)
|
||||
@@ -222,7 +134,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
self._webhook_mode: bool = False
|
||||
self._mention_patterns = self._compile_mention_patterns()
|
||||
self._reply_to_mode: str = getattr(config, 'reply_to_mode', 'first') or 'first'
|
||||
self._disable_link_previews: bool = self._coerce_bool_extra("disable_link_previews", False)
|
||||
# Buffer rapid/album photo updates so Telegram image bursts are handled
|
||||
# as a single MessageEvent instead of self-interrupting multiple turns.
|
||||
self._media_batch_delay_seconds = float(os.getenv("HERMES_TELEGRAM_MEDIA_BATCH_DELAY_SECONDS", "0.8"))
|
||||
@@ -249,38 +160,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
# Approval button state: message_id → session_key
|
||||
self._approval_state: Dict[int, str] = {}
|
||||
|
||||
@staticmethod
|
||||
def _is_callback_user_authorized(user_id: str) -> bool:
|
||||
"""Return whether a Telegram inline-button caller may perform gated actions."""
|
||||
allowed_csv = os.getenv("TELEGRAM_ALLOWED_USERS", "").strip()
|
||||
if not allowed_csv:
|
||||
return True
|
||||
allowed_ids = {uid.strip() for uid in allowed_csv.split(",") if uid.strip()}
|
||||
return "*" in allowed_ids or user_id in allowed_ids
|
||||
|
||||
@classmethod
|
||||
def _metadata_thread_id(cls, metadata: Optional[Dict[str, Any]]) -> Optional[str]:
|
||||
if not metadata:
|
||||
return None
|
||||
thread_id = metadata.get("thread_id") or metadata.get("message_thread_id")
|
||||
return str(thread_id) if thread_id is not None else None
|
||||
|
||||
@classmethod
|
||||
def _message_thread_id_for_send(cls, thread_id: Optional[str]) -> Optional[int]:
|
||||
if not thread_id or str(thread_id) == cls._GENERAL_TOPIC_THREAD_ID:
|
||||
return None
|
||||
return int(thread_id)
|
||||
|
||||
@classmethod
|
||||
def _message_thread_id_for_typing(cls, thread_id: Optional[str]) -> Optional[int]:
|
||||
if not thread_id:
|
||||
return None
|
||||
return int(thread_id)
|
||||
|
||||
@staticmethod
|
||||
def _is_thread_not_found_error(error: Exception) -> bool:
|
||||
return "thread not found" in str(error).lower()
|
||||
|
||||
def _fallback_ips(self) -> list[str]:
|
||||
"""Return validated fallback IPs from config (populated by _apply_env_overrides)."""
|
||||
configured = self.config.extra.get("fallback_ips", []) if getattr(self.config, "extra", None) else []
|
||||
@@ -311,26 +190,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
pass
|
||||
return isinstance(error, OSError)
|
||||
|
||||
def _coerce_bool_extra(self, key: str, default: bool = False) -> bool:
|
||||
value = self.config.extra.get(key) if getattr(self.config, "extra", None) else None
|
||||
if value is None:
|
||||
return default
|
||||
if isinstance(value, str):
|
||||
lowered = value.strip().lower()
|
||||
if lowered in ("true", "1", "yes", "on"):
|
||||
return True
|
||||
if lowered in ("false", "0", "no", "off"):
|
||||
return False
|
||||
return default
|
||||
return bool(value)
|
||||
|
||||
def _link_preview_kwargs(self) -> Dict[str, Any]:
|
||||
if not getattr(self, "_disable_link_previews", False):
|
||||
return {}
|
||||
if LinkPreviewOptions is not None:
|
||||
return {"link_preview_options": LinkPreviewOptions(is_disabled=True)}
|
||||
return {"disable_web_page_preview": True}
|
||||
|
||||
async def _handle_polling_network_error(self, error: Exception) -> None:
|
||||
"""Reconnect polling after a transient network interruption.
|
||||
|
||||
@@ -678,7 +537,10 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
"write_timeout": _env_float("HERMES_TELEGRAM_HTTP_WRITE_TIMEOUT", 20.0),
|
||||
}
|
||||
|
||||
proxy_url = resolve_proxy_url("TELEGRAM_PROXY")
|
||||
proxy_configured = any(
|
||||
(os.getenv(k) or "").strip()
|
||||
for k in ("HTTPS_PROXY", "HTTP_PROXY", "ALL_PROXY", "https_proxy", "http_proxy", "all_proxy")
|
||||
)
|
||||
disable_fallback = (os.getenv("HERMES_TELEGRAM_DISABLE_FALLBACK_IPS", "").strip().lower() in ("1", "true", "yes", "on"))
|
||||
fallback_ips = self._fallback_ips()
|
||||
if not fallback_ips:
|
||||
@@ -689,7 +551,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
", ".join(fallback_ips),
|
||||
)
|
||||
|
||||
if fallback_ips and not proxy_url and not disable_fallback:
|
||||
if fallback_ips and not proxy_configured and not disable_fallback:
|
||||
logger.info(
|
||||
"[%s] Telegram fallback IPs active: %s",
|
||||
self.name,
|
||||
@@ -705,12 +567,10 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
**request_kwargs,
|
||||
httpx_kwargs={"transport": TelegramFallbackTransport(fallback_ips)},
|
||||
)
|
||||
elif proxy_url:
|
||||
logger.info("[%s] Proxy detected; passing explicitly to HTTPXRequest: %s", self.name, proxy_url)
|
||||
request = HTTPXRequest(**request_kwargs, proxy=proxy_url)
|
||||
get_updates_request = HTTPXRequest(**request_kwargs, proxy=proxy_url)
|
||||
else:
|
||||
if disable_fallback:
|
||||
if proxy_configured:
|
||||
logger.info("[%s] Proxy configured; skipping Telegram fallback-IP transport", self.name)
|
||||
elif disable_fallback:
|
||||
logger.info("[%s] Telegram fallback-IP transport disabled via env", self.name)
|
||||
request = HTTPXRequest(**request_kwargs)
|
||||
get_updates_request = HTTPXRequest(**request_kwargs)
|
||||
@@ -744,14 +604,14 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
from telegram.error import NetworkError, TimedOut
|
||||
except ImportError:
|
||||
NetworkError = TimedOut = OSError # type: ignore[misc,assignment]
|
||||
_max_connect = 8
|
||||
_max_connect = 3
|
||||
for _attempt in range(_max_connect):
|
||||
try:
|
||||
await self._app.initialize()
|
||||
break
|
||||
except (NetworkError, TimedOut, OSError) as init_err:
|
||||
if _attempt < _max_connect - 1:
|
||||
wait = min(2 ** _attempt, 15)
|
||||
wait = 2 ** _attempt
|
||||
logger.warning(
|
||||
"[%s] Connect attempt %d/%d failed: %s — retrying in %ds",
|
||||
self.name, _attempt + 1, _max_connect, init_err, wait,
|
||||
@@ -939,9 +799,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
try:
|
||||
# Format and split message if needed
|
||||
formatted = self.format_message(content)
|
||||
chunks = self.truncate_message(
|
||||
formatted, self.MAX_MESSAGE_LENGTH, len_fn=utf16_len,
|
||||
)
|
||||
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
|
||||
if len(chunks) > 1:
|
||||
# truncate_message appends a raw " (1/2)" suffix. Escape the
|
||||
# MarkdownV2-special parentheses so Telegram doesn't reject the
|
||||
@@ -952,7 +810,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
]
|
||||
|
||||
message_ids = []
|
||||
thread_id = self._metadata_thread_id(metadata)
|
||||
thread_id = metadata.get("thread_id") if metadata else None
|
||||
|
||||
try:
|
||||
from telegram.error import NetworkError as _NetErr
|
||||
@@ -972,7 +830,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
for i, chunk in enumerate(chunks):
|
||||
should_thread = self._should_thread_reply(reply_to, i)
|
||||
reply_to_id = int(reply_to) if should_thread else None
|
||||
effective_thread_id = self._message_thread_id_for_send(thread_id)
|
||||
effective_thread_id = int(thread_id) if thread_id else None
|
||||
|
||||
msg = None
|
||||
for _send_attempt in range(3):
|
||||
@@ -985,7 +843,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
parse_mode=ParseMode.MARKDOWN_V2,
|
||||
reply_to_message_id=reply_to_id,
|
||||
message_thread_id=effective_thread_id,
|
||||
**self._link_preview_kwargs(),
|
||||
)
|
||||
except Exception as md_error:
|
||||
# Markdown parsing failed, try plain text
|
||||
@@ -998,7 +855,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
parse_mode=None,
|
||||
reply_to_message_id=reply_to_id,
|
||||
message_thread_id=effective_thread_id,
|
||||
**self._link_preview_kwargs(),
|
||||
)
|
||||
else:
|
||||
raise
|
||||
@@ -1009,7 +865,8 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
# (not transient network issues). Detect and handle
|
||||
# specific cases instead of blindly retrying.
|
||||
if _BadReq and isinstance(send_err, _BadReq):
|
||||
if self._is_thread_not_found_error(send_err) and effective_thread_id is not None:
|
||||
err_lower = str(send_err).lower()
|
||||
if "thread not found" in err_lower and effective_thread_id is not None:
|
||||
# Thread doesn't exist — retry without
|
||||
# message_thread_id so the message still
|
||||
# reaches the chat.
|
||||
@@ -1019,7 +876,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
)
|
||||
effective_thread_id = None
|
||||
continue
|
||||
err_lower = str(send_err).lower()
|
||||
if "message to be replied not found" in err_lower and reply_to_id is not None:
|
||||
# Original message was deleted before we
|
||||
# could reply — clear reply target and retry
|
||||
@@ -1114,9 +970,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
# streaming). Truncate and succeed so the stream consumer can
|
||||
# split the overflow into a new message instead of dying.
|
||||
if "message_too_long" in err_str or "too long" in err_str:
|
||||
truncated = _prefix_within_utf16_limit(
|
||||
content, self.MAX_MESSAGE_LENGTH - 20
|
||||
) + "…"
|
||||
truncated = content[: self.MAX_MESSAGE_LENGTH - 20] + "…"
|
||||
try:
|
||||
await self._bot.edit_message_text(
|
||||
chat_id=int(chat_id),
|
||||
@@ -1186,7 +1040,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
text=text,
|
||||
parse_mode=ParseMode.MARKDOWN,
|
||||
reply_markup=keyboard,
|
||||
**self._link_preview_kwargs(),
|
||||
)
|
||||
return SendResult(success=True, message_id=str(msg.message_id))
|
||||
except Exception as e:
|
||||
@@ -1209,13 +1062,15 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
try:
|
||||
cmd_preview = command[:3800] + "..." if len(command) > 3800 else command
|
||||
text = (
|
||||
f"⚠️ <b>Command Approval Required</b>\n\n"
|
||||
f"<pre>{_html.escape(cmd_preview)}</pre>\n\n"
|
||||
f"Reason: {_html.escape(description)}"
|
||||
f"⚠️ *Command Approval Required*\n\n"
|
||||
f"`{cmd_preview}`\n\n"
|
||||
f"Reason: {description}"
|
||||
)
|
||||
|
||||
# Resolve thread context for thread replies
|
||||
thread_id = self._metadata_thread_id(metadata)
|
||||
thread_id = None
|
||||
if metadata:
|
||||
thread_id = metadata.get("thread_id") or metadata.get("message_thread_id")
|
||||
|
||||
# We'll use the message_id as part of callback_data to look up session_key
|
||||
# Send a placeholder first, then update — or use a counter.
|
||||
@@ -1239,13 +1094,11 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
kwargs: Dict[str, Any] = {
|
||||
"chat_id": int(chat_id),
|
||||
"text": text,
|
||||
"parse_mode": ParseMode.HTML,
|
||||
"parse_mode": ParseMode.MARKDOWN,
|
||||
"reply_markup": keyboard,
|
||||
**self._link_preview_kwargs(),
|
||||
}
|
||||
message_thread_id = self._message_thread_id_for_send(thread_id)
|
||||
if message_thread_id is not None:
|
||||
kwargs["message_thread_id"] = message_thread_id
|
||||
if thread_id:
|
||||
kwargs["message_thread_id"] = int(thread_id)
|
||||
|
||||
msg = await self._bot.send_message(**kwargs)
|
||||
|
||||
@@ -1313,7 +1166,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
parse_mode=ParseMode.MARKDOWN,
|
||||
reply_markup=keyboard,
|
||||
message_thread_id=int(thread_id) if thread_id else None,
|
||||
**self._link_preview_kwargs(),
|
||||
)
|
||||
|
||||
# Store picker state keyed by chat_id
|
||||
@@ -1582,9 +1434,12 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
|
||||
# Only authorized users may click approval buttons.
|
||||
caller_id = str(getattr(query.from_user, "id", ""))
|
||||
if not self._is_callback_user_authorized(caller_id):
|
||||
await query.answer(text="⛔ You are not authorized to approve commands.")
|
||||
return
|
||||
allowed_csv = os.getenv("TELEGRAM_ALLOWED_USERS", "").strip()
|
||||
if allowed_csv:
|
||||
allowed_ids = {uid.strip() for uid in allowed_csv.split(",") if uid.strip()}
|
||||
if "*" not in allowed_ids and caller_id not in allowed_ids:
|
||||
await query.answer(text="⛔ You are not authorized to approve commands.")
|
||||
return
|
||||
|
||||
session_key = self._approval_state.pop(approval_id, None)
|
||||
if not session_key:
|
||||
@@ -1629,10 +1484,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
if not data.startswith("update_prompt:"):
|
||||
return
|
||||
answer = data.split(":", 1)[1] # "y" or "n"
|
||||
caller_id = str(getattr(query.from_user, "id", ""))
|
||||
if not self._is_callback_user_authorized(caller_id):
|
||||
await query.answer(text="⛔ You are not authorized to answer update prompts.")
|
||||
return
|
||||
await query.answer(text=f"Sent '{answer}' to the update process.")
|
||||
# Edit the message to show the choice and remove buttons
|
||||
label = "Yes" if answer == "y" else "No"
|
||||
@@ -1678,23 +1529,23 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
with open(audio_path, "rb") as audio_file:
|
||||
# .ogg files -> send as voice (round playable bubble)
|
||||
if audio_path.endswith((".ogg", ".opus")):
|
||||
_voice_thread = self._metadata_thread_id(metadata)
|
||||
_voice_thread = metadata.get("thread_id") if metadata else None
|
||||
msg = await self._bot.send_voice(
|
||||
chat_id=int(chat_id),
|
||||
voice=audio_file,
|
||||
caption=caption[:1024] if caption else None,
|
||||
reply_to_message_id=int(reply_to) if reply_to else None,
|
||||
message_thread_id=self._message_thread_id_for_send(_voice_thread),
|
||||
message_thread_id=int(_voice_thread) if _voice_thread else None,
|
||||
)
|
||||
else:
|
||||
# .mp3 and others -> send as audio file
|
||||
_audio_thread = self._metadata_thread_id(metadata)
|
||||
_audio_thread = metadata.get("thread_id") if metadata else None
|
||||
msg = await self._bot.send_audio(
|
||||
chat_id=int(chat_id),
|
||||
audio=audio_file,
|
||||
caption=caption[:1024] if caption else None,
|
||||
reply_to_message_id=int(reply_to) if reply_to else None,
|
||||
message_thread_id=self._message_thread_id_for_send(_audio_thread),
|
||||
message_thread_id=int(_audio_thread) if _audio_thread else None,
|
||||
)
|
||||
return SendResult(success=True, message_id=str(msg.message_id))
|
||||
except Exception as e:
|
||||
@@ -1724,14 +1575,14 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
if not os.path.exists(image_path):
|
||||
return SendResult(success=False, error=f"Image file not found: {image_path}")
|
||||
|
||||
_thread = self._metadata_thread_id(metadata)
|
||||
_thread = metadata.get("thread_id") if metadata else None
|
||||
with open(image_path, "rb") as image_file:
|
||||
msg = await self._bot.send_photo(
|
||||
chat_id=int(chat_id),
|
||||
photo=image_file,
|
||||
caption=caption[:1024] if caption else None,
|
||||
reply_to_message_id=int(reply_to) if reply_to else None,
|
||||
message_thread_id=self._message_thread_id_for_send(_thread),
|
||||
message_thread_id=int(_thread) if _thread else None,
|
||||
)
|
||||
return SendResult(success=True, message_id=str(msg.message_id))
|
||||
except Exception as e:
|
||||
@@ -1762,7 +1613,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error=f"File not found: {file_path}")
|
||||
|
||||
display_name = file_name or os.path.basename(file_path)
|
||||
_thread = self._metadata_thread_id(metadata)
|
||||
_thread = metadata.get("thread_id") if metadata else None
|
||||
|
||||
with open(file_path, "rb") as f:
|
||||
msg = await self._bot.send_document(
|
||||
@@ -1771,7 +1622,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
filename=display_name,
|
||||
caption=caption[:1024] if caption else None,
|
||||
reply_to_message_id=int(reply_to) if reply_to else None,
|
||||
message_thread_id=self._message_thread_id_for_send(_thread),
|
||||
message_thread_id=int(_thread) if _thread else None,
|
||||
)
|
||||
return SendResult(success=True, message_id=str(msg.message_id))
|
||||
except Exception as e:
|
||||
@@ -1795,14 +1646,14 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
if not os.path.exists(video_path):
|
||||
return SendResult(success=False, error=f"Video file not found: {video_path}")
|
||||
|
||||
_thread = self._metadata_thread_id(metadata)
|
||||
_thread = metadata.get("thread_id") if metadata else None
|
||||
with open(video_path, "rb") as f:
|
||||
msg = await self._bot.send_video(
|
||||
chat_id=int(chat_id),
|
||||
video=f,
|
||||
caption=caption[:1024] if caption else None,
|
||||
reply_to_message_id=int(reply_to) if reply_to else None,
|
||||
message_thread_id=self._message_thread_id_for_send(_thread),
|
||||
message_thread_id=int(_thread) if _thread else None,
|
||||
)
|
||||
return SendResult(success=True, message_id=str(msg.message_id))
|
||||
except Exception as e:
|
||||
@@ -1832,13 +1683,13 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
|
||||
try:
|
||||
# Telegram can send photos directly from URLs (up to ~5MB)
|
||||
_photo_thread = self._metadata_thread_id(metadata)
|
||||
_photo_thread = metadata.get("thread_id") if metadata else None
|
||||
msg = await self._bot.send_photo(
|
||||
chat_id=int(chat_id),
|
||||
photo=image_url,
|
||||
caption=caption[:1024] if caption else None, # Telegram caption limit
|
||||
reply_to_message_id=int(reply_to) if reply_to else None,
|
||||
message_thread_id=self._message_thread_id_for_send(_photo_thread),
|
||||
message_thread_id=int(_photo_thread) if _photo_thread else None,
|
||||
)
|
||||
return SendResult(success=True, message_id=str(msg.message_id))
|
||||
except Exception as e:
|
||||
@@ -1861,7 +1712,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
photo=image_data,
|
||||
caption=caption[:1024] if caption else None,
|
||||
reply_to_message_id=int(reply_to) if reply_to else None,
|
||||
message_thread_id=self._message_thread_id_for_send(_photo_thread),
|
||||
)
|
||||
return SendResult(success=True, message_id=str(msg.message_id))
|
||||
except Exception as e2:
|
||||
@@ -1887,13 +1737,13 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error="Not connected")
|
||||
|
||||
try:
|
||||
_anim_thread = self._metadata_thread_id(metadata)
|
||||
_anim_thread = metadata.get("thread_id") if metadata else None
|
||||
msg = await self._bot.send_animation(
|
||||
chat_id=int(chat_id),
|
||||
animation=animation_url,
|
||||
caption=caption[:1024] if caption else None,
|
||||
reply_to_message_id=int(reply_to) if reply_to else None,
|
||||
message_thread_id=self._message_thread_id_for_send(_anim_thread),
|
||||
message_thread_id=int(_anim_thread) if _anim_thread else None,
|
||||
)
|
||||
return SendResult(success=True, message_id=str(msg.message_id))
|
||||
except Exception as e:
|
||||
@@ -1910,23 +1760,12 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
"""Send typing indicator."""
|
||||
if self._bot:
|
||||
try:
|
||||
_typing_thread = self._metadata_thread_id(metadata)
|
||||
message_thread_id = self._message_thread_id_for_typing(_typing_thread)
|
||||
try:
|
||||
await self._bot.send_chat_action(
|
||||
chat_id=int(chat_id),
|
||||
action="typing",
|
||||
message_thread_id=message_thread_id,
|
||||
)
|
||||
except Exception as e:
|
||||
if message_thread_id is not None and self._is_thread_not_found_error(e):
|
||||
await self._bot.send_chat_action(
|
||||
chat_id=int(chat_id),
|
||||
action="typing",
|
||||
message_thread_id=None,
|
||||
)
|
||||
else:
|
||||
raise
|
||||
_typing_thread = metadata.get("thread_id") if metadata else None
|
||||
await self._bot.send_chat_action(
|
||||
chat_id=int(chat_id),
|
||||
action="typing",
|
||||
message_thread_id=int(_typing_thread) if _typing_thread else None,
|
||||
)
|
||||
except Exception as e:
|
||||
# Typing failures are non-fatal; log at debug level only.
|
||||
logger.debug(
|
||||
@@ -1994,12 +1833,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
|
||||
text = content
|
||||
|
||||
# 0) Pre-wrap GFM-style pipe tables in ``` fences. Telegram can't
|
||||
# render tables natively, but fenced code blocks render as
|
||||
# monospace preformatted text with columns intact. The wrapped
|
||||
# tables then flow through step (1) below as protected regions.
|
||||
text = _wrap_markdown_tables(text)
|
||||
|
||||
# 1) Protect fenced code blocks (``` ... ```)
|
||||
# Per MarkdownV2 spec, \ and ` inside pre/code must be escaped.
|
||||
def _protect_fenced(m):
|
||||
@@ -2077,20 +1910,9 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
)
|
||||
|
||||
# 9) Convert blockquotes: > at line start → protect > from escaping
|
||||
# Handle both regular blockquotes (> text) and expandable blockquotes
|
||||
# (Telegram MarkdownV2: **> for expandable start, || to end the quote)
|
||||
def _convert_blockquote(m):
|
||||
prefix = m.group(1) # >, >>, >>>, **>, or **>> etc.
|
||||
content = m.group(2)
|
||||
# Check if content ends with || (expandable blockquote end marker)
|
||||
# In this case, preserve the trailing || unescaped for Telegram
|
||||
if prefix.startswith('**') and content.endswith('||'):
|
||||
return _ph(f'{prefix} {_escape_mdv2(content[:-2])}||')
|
||||
return _ph(f'{prefix} {_escape_mdv2(content)}')
|
||||
|
||||
text = re.sub(
|
||||
r'^((?:\*\*)?>{1,3}) (.+)$',
|
||||
_convert_blockquote,
|
||||
r'^(>{1,3}) (.+)$',
|
||||
lambda m: _ph(m.group(1) + ' ' + _escape_mdv2(m.group(2))),
|
||||
text,
|
||||
flags=re.MULTILINE,
|
||||
)
|
||||
@@ -2163,27 +1985,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
return {str(part).strip() for part in raw if str(part).strip()}
|
||||
return {part.strip() for part in str(raw).split(",") if part.strip()}
|
||||
|
||||
def _telegram_ignored_threads(self) -> set[int]:
|
||||
raw = self.config.extra.get("ignored_threads")
|
||||
if raw is None:
|
||||
raw = os.getenv("TELEGRAM_IGNORED_THREADS", "")
|
||||
|
||||
if isinstance(raw, list):
|
||||
values = raw
|
||||
else:
|
||||
values = str(raw).split(",")
|
||||
|
||||
ignored: set[int] = set()
|
||||
for value in values:
|
||||
text = str(value).strip()
|
||||
if not text:
|
||||
continue
|
||||
try:
|
||||
ignored.add(int(text))
|
||||
except (TypeError, ValueError):
|
||||
logger.warning("[%s] Ignoring invalid Telegram thread id: %r", self.name, value)
|
||||
return ignored
|
||||
|
||||
def _compile_mention_patterns(self) -> List[re.Pattern]:
|
||||
"""Compile optional regex wake-word patterns for group triggers."""
|
||||
patterns = self.config.extra.get("mention_patterns")
|
||||
@@ -2295,13 +2096,6 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
"""
|
||||
if not self._is_group_chat(message):
|
||||
return True
|
||||
thread_id = getattr(message, "message_thread_id", None)
|
||||
if thread_id is not None:
|
||||
try:
|
||||
if int(thread_id) in self._telegram_ignored_threads():
|
||||
return False
|
||||
except (TypeError, ValueError):
|
||||
logger.warning("[%s] Ignoring non-numeric Telegram message_thread_id: %r", self.name, thread_id)
|
||||
if str(getattr(getattr(message, "chat", None), "id", "")) in self._telegram_free_response_chats():
|
||||
return True
|
||||
if not self._telegram_require_mention():
|
||||
@@ -2326,7 +2120,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
if not self._should_process_message(update.message):
|
||||
return
|
||||
|
||||
event = self._build_message_event(update.message, MessageType.TEXT, update_id=update.update_id)
|
||||
event = self._build_message_event(update.message, MessageType.TEXT)
|
||||
event.text = self._clean_bot_trigger_text(event.text)
|
||||
self._enqueue_text_event(event)
|
||||
|
||||
@@ -2337,7 +2131,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
if not self._should_process_message(update.message, is_command=True):
|
||||
return
|
||||
|
||||
event = self._build_message_event(update.message, MessageType.COMMAND, update_id=update.update_id)
|
||||
event = self._build_message_event(update.message, MessageType.COMMAND)
|
||||
await self.handle_message(event)
|
||||
|
||||
async def _handle_location_message(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
||||
@@ -2373,7 +2167,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
parts.append(f"Map: https://www.google.com/maps/search/?api=1&query={lat},{lon}")
|
||||
parts.append("Ask what they'd like to find nearby (restaurants, cafes, etc.) and any preferences.")
|
||||
|
||||
event = self._build_message_event(msg, MessageType.LOCATION, update_id=update.update_id)
|
||||
event = self._build_message_event(msg, MessageType.LOCATION)
|
||||
event.text = "\n".join(parts)
|
||||
await self.handle_message(event)
|
||||
|
||||
@@ -2524,7 +2318,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
else:
|
||||
msg_type = MessageType.DOCUMENT
|
||||
|
||||
event = self._build_message_event(msg, msg_type, update_id=update.update_id)
|
||||
event = self._build_message_event(msg, msg_type)
|
||||
|
||||
# Add caption as text
|
||||
if msg.caption:
|
||||
@@ -2863,19 +2657,8 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
self.name, cache_key, thread_id,
|
||||
)
|
||||
|
||||
def _build_message_event(
|
||||
self,
|
||||
message: Message,
|
||||
msg_type: MessageType,
|
||||
update_id: Optional[int] = None,
|
||||
) -> MessageEvent:
|
||||
"""Build a MessageEvent from a Telegram message.
|
||||
|
||||
``update_id`` is the ``Update.update_id`` from PTB; passing it through
|
||||
lets ``/restart`` record the triggering offset so the new gateway
|
||||
process can advance past it (prevents ``/restart`` being re-delivered
|
||||
when PTB's graceful-shutdown ACK fails).
|
||||
"""
|
||||
def _build_message_event(self, message: Message, msg_type: MessageType) -> MessageEvent:
|
||||
"""Build a MessageEvent from a Telegram message."""
|
||||
chat = message.chat
|
||||
user = message.from_user
|
||||
|
||||
@@ -2888,9 +2671,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
|
||||
# Resolve DM topic name and skill binding
|
||||
thread_id_raw = message.message_thread_id
|
||||
thread_id_str = str(thread_id_raw) if thread_id_raw is not None else None
|
||||
if chat_type == "group" and thread_id_str is None and getattr(chat, "is_forum", False):
|
||||
thread_id_str = self._GENERAL_TOPIC_THREAD_ID
|
||||
thread_id_str = str(thread_id_raw) if thread_id_raw else None
|
||||
chat_topic = None
|
||||
topic_skill = None
|
||||
|
||||
@@ -2939,26 +2720,15 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
reply_to_id = str(message.reply_to_message.message_id)
|
||||
reply_to_text = message.reply_to_message.text or message.reply_to_message.caption or None
|
||||
|
||||
# Per-channel/topic ephemeral prompt
|
||||
from gateway.platforms.base import resolve_channel_prompt
|
||||
_chat_id_str = str(chat.id)
|
||||
_channel_prompt = resolve_channel_prompt(
|
||||
self.config.extra,
|
||||
thread_id_str or _chat_id_str,
|
||||
_chat_id_str if thread_id_str else None,
|
||||
)
|
||||
|
||||
return MessageEvent(
|
||||
text=message.text or "",
|
||||
message_type=msg_type,
|
||||
source=source,
|
||||
raw_message=message,
|
||||
message_id=str(message.message_id),
|
||||
platform_update_id=update_id,
|
||||
reply_to_message_id=reply_to_id,
|
||||
reply_to_text=reply_to_text,
|
||||
auto_skill=topic_skill,
|
||||
channel_prompt=_channel_prompt,
|
||||
timestamp=message.date,
|
||||
)
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import ipaddress
|
||||
import logging
|
||||
import os
|
||||
import socket
|
||||
from typing import Iterable, Optional
|
||||
|
||||
@@ -46,7 +47,7 @@ _SEED_FALLBACK_IPS: list[str] = ["149.154.167.220"]
|
||||
def _resolve_proxy_url() -> str | None:
|
||||
# Delegate to shared implementation (env vars + macOS system proxy detection)
|
||||
from gateway.platforms.base import resolve_proxy_url
|
||||
return resolve_proxy_url("TELEGRAM_PROXY")
|
||||
return resolve_proxy_url()
|
||||
|
||||
|
||||
class TelegramFallbackTransport(httpx.AsyncBaseTransport):
|
||||
|
||||
@@ -27,6 +27,7 @@ import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
@@ -203,7 +204,6 @@ class WebhookAdapter(BasePlatformAdapter):
|
||||
"wecom_callback",
|
||||
"weixin",
|
||||
"bluebubbles",
|
||||
"qqbot",
|
||||
):
|
||||
return await self._deliver_cross_platform(
|
||||
deliver_type, content, delivery
|
||||
|
||||
+12
-42
@@ -37,6 +37,7 @@ import logging
|
||||
import mimetypes
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
@@ -180,8 +181,6 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
self._text_batch_split_delay_seconds = float(os.getenv("HERMES_WECOM_TEXT_BATCH_SPLIT_DELAY_SECONDS", "2.0"))
|
||||
self._pending_text_batches: Dict[str, MessageEvent] = {}
|
||||
self._pending_text_batch_tasks: Dict[str, asyncio.Task] = {}
|
||||
self._device_id = uuid.uuid4().hex
|
||||
self._last_chat_req_ids: Dict[str, str] = {}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Connection lifecycle
|
||||
@@ -267,7 +266,7 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
async def _open_connection(self) -> None:
|
||||
"""Open and authenticate a websocket connection."""
|
||||
await self._cleanup_ws()
|
||||
self._session = aiohttp.ClientSession(trust_env=True)
|
||||
self._session = aiohttp.ClientSession()
|
||||
self._ws = await self._session.ws_connect(
|
||||
self._ws_url,
|
||||
heartbeat=HEARTBEAT_INTERVAL_SECONDS * 2,
|
||||
@@ -279,11 +278,7 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
{
|
||||
"cmd": APP_CMD_SUBSCRIBE,
|
||||
"headers": {"req_id": req_id},
|
||||
"body": {
|
||||
"bot_id": self._bot_id,
|
||||
"secret": self._secret,
|
||||
"device_id": self._device_id,
|
||||
},
|
||||
"body": {"bot_id": self._bot_id, "secret": self._secret},
|
||||
}
|
||||
)
|
||||
|
||||
@@ -502,11 +497,6 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
logger.debug("[%s] DM sender %s blocked by policy", self.name, sender_id)
|
||||
return
|
||||
|
||||
# Cache the inbound req_id after policy checks so proactive sends to
|
||||
# this chat can fall back to APP_CMD_RESPONSE (required for groups —
|
||||
# WeCom AI Bots cannot initiate APP_CMD_SEND in group chats).
|
||||
self._remember_chat_req_id(chat_id, self._payload_req_id(payload))
|
||||
|
||||
text, reply_text = self._extract_text(body)
|
||||
media_urls, media_types = await self._extract_media(body)
|
||||
message_type = self._derive_message_type(body, text, media_types)
|
||||
@@ -858,23 +848,6 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
while len(self._reply_req_ids) > DEDUP_MAX_SIZE:
|
||||
self._reply_req_ids.pop(next(iter(self._reply_req_ids)))
|
||||
|
||||
def _remember_chat_req_id(self, chat_id: str, req_id: str) -> None:
|
||||
"""Cache the most recent inbound req_id per chat.
|
||||
|
||||
Used as a fallback reply target when we need to send into a group
|
||||
without an explicit ``reply_to`` — WeCom AI Bots are blocked from
|
||||
APP_CMD_SEND in groups and must use APP_CMD_RESPONSE bound to some
|
||||
prior req_id. Bounded like _reply_req_ids so long-running gateways
|
||||
don't leak memory across many chats.
|
||||
"""
|
||||
normalized_chat_id = str(chat_id or "").strip()
|
||||
normalized_req_id = str(req_id or "").strip()
|
||||
if not normalized_chat_id or not normalized_req_id:
|
||||
return
|
||||
self._last_chat_req_ids[normalized_chat_id] = normalized_req_id
|
||||
while len(self._last_chat_req_ids) > DEDUP_MAX_SIZE:
|
||||
self._last_chat_req_ids.pop(next(iter(self._last_chat_req_ids)))
|
||||
|
||||
def _reply_req_id_for_message(self, reply_to: Optional[str]) -> Optional[str]:
|
||||
normalized = str(reply_to or "").strip()
|
||||
if not normalized or normalized.startswith("quote:"):
|
||||
@@ -1191,15 +1164,19 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
self._raise_for_wecom_error(response, "send media message")
|
||||
return response
|
||||
|
||||
async def _send_reply_markdown(self, reply_req_id: str, content: str) -> Dict[str, Any]:
|
||||
async def _send_reply_stream(self, reply_req_id: str, content: str) -> Dict[str, Any]:
|
||||
response = await self._send_reply_request(
|
||||
reply_req_id,
|
||||
{
|
||||
"msgtype": "markdown",
|
||||
"markdown": {"content": content[:self.MAX_MESSAGE_LENGTH]},
|
||||
"msgtype": "stream",
|
||||
"stream": {
|
||||
"id": self._new_req_id("stream"),
|
||||
"finish": True,
|
||||
"content": content[:self.MAX_MESSAGE_LENGTH],
|
||||
},
|
||||
},
|
||||
)
|
||||
self._raise_for_wecom_error(response, "send reply markdown")
|
||||
self._raise_for_wecom_error(response, "send reply stream")
|
||||
return response
|
||||
|
||||
async def _send_reply_media_message(
|
||||
@@ -1259,9 +1236,6 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error=prepared["reject_reason"])
|
||||
|
||||
reply_req_id = self._reply_req_id_for_message(reply_to)
|
||||
if not reply_req_id and chat_id in self._last_chat_req_ids:
|
||||
reply_req_id = self._last_chat_req_ids[chat_id]
|
||||
|
||||
try:
|
||||
upload_result = await self._upload_media_bytes(
|
||||
prepared["data"],
|
||||
@@ -1329,12 +1303,8 @@ class WeComAdapter(BasePlatformAdapter):
|
||||
|
||||
try:
|
||||
reply_req_id = self._reply_req_id_for_message(reply_to)
|
||||
|
||||
if not reply_req_id and chat_id in self._last_chat_req_ids:
|
||||
reply_req_id = self._last_chat_req_ids[chat_id]
|
||||
|
||||
if reply_req_id:
|
||||
response = await self._send_reply_markdown(reply_req_id, content)
|
||||
response = await self._send_reply_stream(reply_req_id, content)
|
||||
else:
|
||||
response = await self._send_request(
|
||||
APP_CMD_SEND,
|
||||
|
||||
@@ -258,20 +258,6 @@ class WecomCallbackAdapter(BasePlatformAdapter):
|
||||
)
|
||||
event = self._build_event(app, decrypted)
|
||||
if event is not None:
|
||||
# Deduplicate: WeCom retries callbacks on timeout,
|
||||
# producing duplicate inbound messages (#10305).
|
||||
if event.message_id:
|
||||
now = time.time()
|
||||
if event.message_id in self._seen_messages:
|
||||
if now - self._seen_messages[event.message_id] < MESSAGE_DEDUP_TTL_SECONDS:
|
||||
logger.debug("[WecomCallback] Duplicate MsgId %s, skipping", event.message_id)
|
||||
return web.Response(text="success", content_type="text/plain")
|
||||
del self._seen_messages[event.message_id]
|
||||
self._seen_messages[event.message_id] = now
|
||||
# Prune expired entries when cache grows large
|
||||
if len(self._seen_messages) > 2000:
|
||||
cutoff = now - MESSAGE_DEDUP_TTL_SECONDS
|
||||
self._seen_messages = {k: v for k, v in self._seen_messages.items() if v > cutoff}
|
||||
# Record which app this user belongs to.
|
||||
if event.source and event.source.user_id:
|
||||
map_key = self._user_app_key(
|
||||
|
||||
+122
-399
@@ -28,7 +28,7 @@ import uuid
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
from urllib.parse import quote, urlparse
|
||||
from urllib.parse import quote
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -96,28 +96,6 @@ MEDIA_VIDEO = 2
|
||||
MEDIA_FILE = 3
|
||||
MEDIA_VOICE = 4
|
||||
|
||||
_LIVE_ADAPTERS: Dict[str, Any] = {}
|
||||
|
||||
|
||||
def _make_ssl_connector() -> Optional["aiohttp.TCPConnector"]:
|
||||
"""Return a TCPConnector with a certifi CA bundle, or None if certifi is unavailable.
|
||||
|
||||
Tencent's iLink server (``ilinkai.weixin.qq.com``) is not verifiable against
|
||||
some system CA stores (notably Homebrew's OpenSSL on macOS Apple Silicon).
|
||||
When ``certifi`` is installed, use its Mozilla CA bundle to guarantee
|
||||
verification. Otherwise fall back to aiohttp's default (which honors
|
||||
``SSL_CERT_FILE`` env var via ``trust_env=True``).
|
||||
"""
|
||||
try:
|
||||
import ssl
|
||||
import certifi
|
||||
except ImportError:
|
||||
return None
|
||||
if not AIOHTTP_AVAILABLE:
|
||||
return None
|
||||
ssl_ctx = ssl.create_default_context(cafile=certifi.where())
|
||||
return aiohttp.TCPConnector(ssl=ssl_ctx)
|
||||
|
||||
ITEM_TEXT = 1
|
||||
ITEM_IMAGE = 2
|
||||
ITEM_VOICE = 3
|
||||
@@ -134,7 +112,6 @@ TYPING_STOP = 2
|
||||
_HEADER_RE = re.compile(r"^(#{1,6})\s+(.+?)\s*$")
|
||||
_TABLE_RULE_RE = re.compile(r"^\s*\|?(?:\s*:?-{3,}:?\s*\|)+\s*:?-{3,}:?\s*\|?\s*$")
|
||||
_FENCE_RE = re.compile(r"^```([^\n`]*)\s*$")
|
||||
_MARKDOWN_LINK_RE = re.compile(r"\[([^\]]+)\]\(([^)]+)\)")
|
||||
|
||||
|
||||
def check_weixin_requirements() -> bool:
|
||||
@@ -420,25 +397,19 @@ async def _send_message(
|
||||
text: str,
|
||||
context_token: Optional[str],
|
||||
client_id: str,
|
||||
) -> Dict[str, Any]:
|
||||
"""Send a text message via iLink sendmessage API.
|
||||
|
||||
Returns the raw API response dict (may contain error codes like
|
||||
``errcode: -14`` for session expiry that the caller can inspect).
|
||||
"""
|
||||
if not text or not text.strip():
|
||||
raise ValueError("_send_message: text must not be empty")
|
||||
) -> None:
|
||||
message: Dict[str, Any] = {
|
||||
"from_user_id": "",
|
||||
"to_user_id": to,
|
||||
"client_id": client_id,
|
||||
"message_type": MSG_TYPE_BOT,
|
||||
"message_state": MSG_STATE_FINISH,
|
||||
"item_list": [{"type": ITEM_TEXT, "text_item": {"text": text}}],
|
||||
}
|
||||
if text:
|
||||
message["item_list"] = [{"type": ITEM_TEXT, "text_item": {"text": text}}]
|
||||
if context_token:
|
||||
message["context_token"] = context_token
|
||||
return await _api_post(
|
||||
await _api_post(
|
||||
session,
|
||||
base_url=base_url,
|
||||
endpoint=EP_SEND_MESSAGE,
|
||||
@@ -528,15 +499,13 @@ async def _upload_ciphertext(
|
||||
session: "aiohttp.ClientSession",
|
||||
*,
|
||||
ciphertext: bytes,
|
||||
upload_url: str,
|
||||
cdn_base_url: str,
|
||||
upload_param: str,
|
||||
filekey: str,
|
||||
) -> str:
|
||||
"""Upload encrypted media to the CDN.
|
||||
|
||||
Accepts either a constructed CDN URL (from upload_param) or a direct
|
||||
upload_full_url — both use POST with the raw ciphertext as the body.
|
||||
"""
|
||||
url = _cdn_upload_url(cdn_base_url, upload_param, filekey)
|
||||
timeout = aiohttp.ClientTimeout(total=120)
|
||||
async with session.post(upload_url, data=ciphertext, headers={"Content-Type": "application/octet-stream"}, timeout=timeout) as response:
|
||||
async with session.post(url, data=ciphertext, headers={"Content-Type": "application/octet-stream"}, timeout=timeout) as response:
|
||||
if response.status == 200:
|
||||
encrypted_param = response.headers.get("x-encrypted-param")
|
||||
if encrypted_param:
|
||||
@@ -560,39 +529,6 @@ async def _download_bytes(
|
||||
return await response.read()
|
||||
|
||||
|
||||
_WEIXIN_CDN_ALLOWLIST: frozenset[str] = frozenset(
|
||||
{
|
||||
"novac2c.cdn.weixin.qq.com",
|
||||
"ilinkai.weixin.qq.com",
|
||||
"wx.qlogo.cn",
|
||||
"thirdwx.qlogo.cn",
|
||||
"res.wx.qq.com",
|
||||
"mmbiz.qpic.cn",
|
||||
"mmbiz.qlogo.cn",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _assert_weixin_cdn_url(url: str) -> None:
|
||||
"""Raise ValueError if *url* does not point at a known WeChat CDN host."""
|
||||
try:
|
||||
parsed = urlparse(url)
|
||||
scheme = parsed.scheme.lower()
|
||||
host = parsed.hostname or ""
|
||||
except Exception as exc: # noqa: BLE001
|
||||
raise ValueError(f"Unparseable media URL: {url!r}") from exc
|
||||
|
||||
if scheme not in ("http", "https"):
|
||||
raise ValueError(
|
||||
f"Media URL has disallowed scheme {scheme!r}; only http/https are permitted."
|
||||
)
|
||||
if host not in _WEIXIN_CDN_ALLOWLIST:
|
||||
raise ValueError(
|
||||
f"Media URL host {host!r} is not in the WeChat CDN allowlist. "
|
||||
"Refusing to fetch to prevent SSRF."
|
||||
)
|
||||
|
||||
|
||||
def _media_reference(item: Dict[str, Any], key: str) -> Dict[str, Any]:
|
||||
return (item.get(key) or {}).get("media") or {}
|
||||
|
||||
@@ -613,7 +549,6 @@ async def _download_and_decrypt_media(
|
||||
timeout_seconds=timeout_seconds,
|
||||
)
|
||||
elif full_url:
|
||||
_assert_weixin_cdn_url(full_url)
|
||||
raw = await _download_bytes(session, url=full_url, timeout_seconds=timeout_seconds)
|
||||
else:
|
||||
raise RuntimeError("media item had neither encrypt_query_param nor full_url")
|
||||
@@ -684,31 +619,42 @@ def _rewrite_table_block_for_weixin(lines: List[str]) -> str:
|
||||
def _normalize_markdown_blocks(content: str) -> str:
|
||||
lines = content.splitlines()
|
||||
result: List[str] = []
|
||||
i = 0
|
||||
in_code_block = False
|
||||
blank_run = 0
|
||||
|
||||
for raw_line in lines:
|
||||
line = raw_line.rstrip()
|
||||
if _FENCE_RE.match(line.strip()):
|
||||
while i < len(lines):
|
||||
line = lines[i].rstrip()
|
||||
fence_match = _FENCE_RE.match(line.strip())
|
||||
if fence_match:
|
||||
in_code_block = not in_code_block
|
||||
result.append(line)
|
||||
blank_run = 0
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if in_code_block:
|
||||
result.append(line)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if not line.strip():
|
||||
blank_run += 1
|
||||
if blank_run <= 1:
|
||||
result.append("")
|
||||
if (
|
||||
i + 1 < len(lines)
|
||||
and "|" in lines[i]
|
||||
and _TABLE_RULE_RE.match(lines[i + 1].rstrip())
|
||||
):
|
||||
table_lines = [lines[i].rstrip(), lines[i + 1].rstrip()]
|
||||
i += 2
|
||||
while i < len(lines) and "|" in lines[i]:
|
||||
table_lines.append(lines[i].rstrip())
|
||||
i += 1
|
||||
result.append(_rewrite_table_block_for_weixin(table_lines))
|
||||
continue
|
||||
|
||||
blank_run = 0
|
||||
result.append(line)
|
||||
result.append(_rewrite_headers_for_weixin(line))
|
||||
i += 1
|
||||
|
||||
return "\n".join(result).strip()
|
||||
normalized = "\n".join(item.rstrip() for item in result)
|
||||
normalized = re.sub(r"\n{3,}", "\n\n", normalized)
|
||||
return normalized.strip()
|
||||
|
||||
|
||||
def _split_markdown_blocks(content: str) -> List[str]:
|
||||
@@ -754,8 +700,8 @@ def _split_delivery_units_for_weixin(content: str) -> List[str]:
|
||||
|
||||
Weixin can render Markdown, but chat readability is better when top-level
|
||||
line breaks become separate messages. Keep fenced code blocks intact and
|
||||
attach indented continuation lines to the previous top-level line so nested
|
||||
list items do not get torn apart.
|
||||
attach indented continuation lines to the previous top-level line so
|
||||
transformed tables/lists do not get torn apart.
|
||||
"""
|
||||
units: List[str] = []
|
||||
|
||||
@@ -797,9 +743,7 @@ def _looks_like_chatty_line_for_weixin(line: str) -> bool:
|
||||
return False
|
||||
if line.startswith((" ", "\t")):
|
||||
return False
|
||||
if stripped.startswith((">", "-", "*", "【", "#", "|")):
|
||||
return False
|
||||
if _TABLE_RULE_RE.match(stripped):
|
||||
if stripped.startswith((">", "-", "*", "【")):
|
||||
return False
|
||||
if re.match(r"^\*\*[^*]+\*\*$", stripped):
|
||||
return False
|
||||
@@ -809,12 +753,10 @@ def _looks_like_chatty_line_for_weixin(line: str) -> bool:
|
||||
|
||||
|
||||
def _looks_like_heading_line_for_weixin(line: str) -> bool:
|
||||
"""Return True when a short line behaves like a heading."""
|
||||
"""Return True when a short line behaves like a plain-text heading."""
|
||||
stripped = line.strip()
|
||||
if not stripped:
|
||||
return False
|
||||
if _HEADER_RE.match(stripped):
|
||||
return True
|
||||
return len(stripped) <= 24 and stripped.endswith((":", ":"))
|
||||
|
||||
|
||||
@@ -869,8 +811,6 @@ def _split_text_for_weixin_delivery(
|
||||
``platforms.weixin.extra.split_multiline_messages`` (``true`` / ``false``)
|
||||
or the env var ``WEIXIN_SPLIT_MULTILINE_MESSAGES``.
|
||||
"""
|
||||
if not content:
|
||||
return []
|
||||
if split_per_line:
|
||||
# Legacy: one message per top-level delivery unit.
|
||||
if len(content) <= max_length and "\n" not in content:
|
||||
@@ -881,14 +821,14 @@ def _split_text_for_weixin_delivery(
|
||||
chunks.append(unit)
|
||||
continue
|
||||
chunks.extend(_pack_markdown_blocks_for_weixin(unit, max_length))
|
||||
return [c for c in chunks if c] or [content]
|
||||
return chunks or [content]
|
||||
|
||||
# Compact (default): single message when under the limit — unless the
|
||||
# content looks like a short chatty exchange, in which case split into
|
||||
# separate bubbles for a more natural chat feel.
|
||||
if len(content) <= max_length:
|
||||
return (
|
||||
[u for u in _split_delivery_units_for_weixin(content) if u]
|
||||
_split_delivery_units_for_weixin(content)
|
||||
if _should_split_short_chat_block_for_weixin(content)
|
||||
else [content]
|
||||
)
|
||||
@@ -989,7 +929,7 @@ async def qr_login(
|
||||
if not AIOHTTP_AVAILABLE:
|
||||
raise RuntimeError("aiohttp is required for Weixin QR login")
|
||||
|
||||
async with aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector()) as session:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
try:
|
||||
qr_resp = await _api_get(
|
||||
session,
|
||||
@@ -1007,10 +947,6 @@ async def qr_login(
|
||||
logger.error("weixin: QR response missing qrcode")
|
||||
return None
|
||||
|
||||
# qrcode_url is the full scannable liteapp URL; qrcode_value is just the hex token
|
||||
# WeChat needs to scan the full URL, not the raw hex string
|
||||
qr_scan_data = qrcode_url if qrcode_url else qrcode_value
|
||||
|
||||
print("\n请使用微信扫描以下二维码:")
|
||||
if qrcode_url:
|
||||
print(qrcode_url)
|
||||
@@ -1018,11 +954,11 @@ async def qr_login(
|
||||
import qrcode
|
||||
|
||||
qr = qrcode.QRCode()
|
||||
qr.add_data(qr_scan_data)
|
||||
qr.add_data(qrcode_url or qrcode_value)
|
||||
qr.make(fit=True)
|
||||
qr.print_ascii(invert=True)
|
||||
except Exception as _qr_exc:
|
||||
print(f"(终端二维码渲染失败: {_qr_exc},请直接打开上面的二维码链接)")
|
||||
except Exception:
|
||||
print("(终端二维码渲染失败,请直接打开上面的二维码链接)")
|
||||
|
||||
deadline = time.time() + timeout_seconds
|
||||
current_base_url = ILINK_BASE_URL
|
||||
@@ -1068,17 +1004,8 @@ async def qr_login(
|
||||
)
|
||||
qrcode_value = str(qr_resp.get("qrcode") or "")
|
||||
qrcode_url = str(qr_resp.get("qrcode_img_content") or "")
|
||||
qr_scan_data = qrcode_url if qrcode_url else qrcode_value
|
||||
if qrcode_url:
|
||||
print(qrcode_url)
|
||||
try:
|
||||
import qrcode as _qrcode
|
||||
qr = _qrcode.QRCode()
|
||||
qr.add_data(qr_scan_data)
|
||||
qr.make(fit=True)
|
||||
qr.print_ascii(invert=True)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as exc:
|
||||
logger.error("weixin: QR refresh failed: %s", exc)
|
||||
return None
|
||||
@@ -1115,10 +1042,6 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
|
||||
MAX_MESSAGE_LENGTH = 4000
|
||||
|
||||
# WeChat does not support editing sent messages — streaming must use the
|
||||
# fallback "send-final-only" path so the cursor (▉) is never left visible.
|
||||
SUPPORTS_MESSAGE_EDITING = False
|
||||
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.WEIXIN)
|
||||
extra = config.extra or {}
|
||||
@@ -1126,8 +1049,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
self._hermes_home = hermes_home
|
||||
self._token_store = ContextTokenStore(hermes_home)
|
||||
self._typing_cache = TypingTicketCache()
|
||||
self._poll_session: Optional[aiohttp.ClientSession] = None
|
||||
self._send_session: Optional[aiohttp.ClientSession] = None
|
||||
self._session: Optional[aiohttp.ClientSession] = None
|
||||
self._poll_task: Optional[asyncio.Task] = None
|
||||
self._dedup = MessageDeduplicator(ttl_seconds=MESSAGE_DEDUP_TTL_SECONDS)
|
||||
|
||||
@@ -1202,17 +1124,14 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
except Exception as exc:
|
||||
logger.debug("[%s] Token lock unavailable (non-fatal): %s", self.name, exc)
|
||||
|
||||
self._poll_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
||||
self._send_session = aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector())
|
||||
self._session = aiohttp.ClientSession()
|
||||
self._token_store.restore(self._account_id)
|
||||
self._poll_task = asyncio.create_task(self._poll_loop(), name="weixin-poll")
|
||||
self._mark_connected()
|
||||
_LIVE_ADAPTERS[self._token] = self
|
||||
logger.info("[%s] Connected account=%s base=%s", self.name, _safe_id(self._account_id), self._base_url)
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
_LIVE_ADAPTERS.pop(self._token, None)
|
||||
self._running = False
|
||||
if self._poll_task and not self._poll_task.done():
|
||||
self._poll_task.cancel()
|
||||
@@ -1221,18 +1140,15 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._poll_task = None
|
||||
if self._poll_session and not self._poll_session.closed:
|
||||
await self._poll_session.close()
|
||||
self._poll_session = None
|
||||
if self._send_session and not self._send_session.closed:
|
||||
await self._send_session.close()
|
||||
self._send_session = None
|
||||
if self._session and not self._session.closed:
|
||||
await self._session.close()
|
||||
self._session = None
|
||||
self._release_platform_lock()
|
||||
self._mark_disconnected()
|
||||
logger.info("[%s] Disconnected", self.name)
|
||||
|
||||
async def _poll_loop(self) -> None:
|
||||
assert self._poll_session is not None
|
||||
assert self._session is not None
|
||||
sync_buf = _load_sync_buf(self._hermes_home, self._account_id)
|
||||
timeout_ms = LONG_POLL_TIMEOUT_MS
|
||||
consecutive_failures = 0
|
||||
@@ -1240,7 +1156,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
while self._running:
|
||||
try:
|
||||
response = await _get_updates(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
sync_buf=sync_buf,
|
||||
@@ -1297,7 +1213,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
logger.error("[%s] unhandled inbound error from=%s: %s", self.name, _safe_id(message.get("from_user_id")), exc, exc_info=True)
|
||||
|
||||
async def _process_message(self, message: Dict[str, Any]) -> None:
|
||||
assert self._poll_session is not None
|
||||
assert self._session is not None
|
||||
sender_id = str(message.get("from_user_id") or "").strip()
|
||||
if not sender_id:
|
||||
return
|
||||
@@ -1390,7 +1306,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
media = _media_reference(item, "image_item")
|
||||
try:
|
||||
data = await _download_and_decrypt_media(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
encrypted_query_param=media.get("encrypt_query_param"),
|
||||
aes_key_b64=(item.get("image_item") or {}).get("aeskey")
|
||||
@@ -1408,7 +1324,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
media = _media_reference(item, "video_item")
|
||||
try:
|
||||
data = await _download_and_decrypt_media(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
encrypted_query_param=media.get("encrypt_query_param"),
|
||||
aes_key_b64=media.get("aes_key"),
|
||||
@@ -1427,7 +1343,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
mime = _mime_from_filename(filename)
|
||||
try:
|
||||
data = await _download_and_decrypt_media(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
encrypted_query_param=media.get("encrypt_query_param"),
|
||||
aes_key_b64=media.get("aes_key"),
|
||||
@@ -1446,7 +1362,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
return None
|
||||
try:
|
||||
data = await _download_and_decrypt_media(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
encrypted_query_param=media.get("encrypt_query_param"),
|
||||
aes_key_b64=media.get("aes_key"),
|
||||
@@ -1459,13 +1375,13 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
return None
|
||||
|
||||
async def _maybe_fetch_typing_ticket(self, user_id: str, context_token: Optional[str]) -> None:
|
||||
if not self._poll_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return
|
||||
if self._typing_cache.get(user_id):
|
||||
return
|
||||
try:
|
||||
response = await _get_config(
|
||||
self._poll_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
user_id=user_id,
|
||||
@@ -1490,19 +1406,12 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
context_token: Optional[str],
|
||||
client_id: str,
|
||||
) -> None:
|
||||
"""Send a single text chunk with per-chunk retry and backoff.
|
||||
|
||||
On session-expired errors (errcode -14), automatically retries
|
||||
*without* ``context_token`` — iLink accepts tokenless sends as a
|
||||
degraded fallback, which keeps cron-initiated push messages working
|
||||
even when no user message has refreshed the session recently.
|
||||
"""
|
||||
"""Send a single text chunk with per-chunk retry and backoff."""
|
||||
last_error: Optional[Exception] = None
|
||||
retried_without_token = False
|
||||
for attempt in range(self._send_chunk_retries + 1):
|
||||
try:
|
||||
resp = await _send_message(
|
||||
self._send_session,
|
||||
await _send_message(
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to=chat_id,
|
||||
@@ -1510,31 +1419,6 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
context_token=context_token,
|
||||
client_id=client_id,
|
||||
)
|
||||
# Check iLink response for session-expired error
|
||||
if resp and isinstance(resp, dict):
|
||||
ret = resp.get("ret")
|
||||
errcode = resp.get("errcode")
|
||||
if (ret is not None and ret not in (0,)) or (errcode is not None and errcode not in (0,)):
|
||||
is_session_expired = (
|
||||
ret == SESSION_EXPIRED_ERRCODE
|
||||
or errcode == SESSION_EXPIRED_ERRCODE
|
||||
)
|
||||
# Session expired — strip token and retry once
|
||||
if is_session_expired and not retried_without_token and context_token:
|
||||
retried_without_token = True
|
||||
context_token = None
|
||||
self._token_store._cache.pop(
|
||||
self._token_store._key(self._account_id, chat_id), None
|
||||
)
|
||||
logger.warning(
|
||||
"[%s] session expired for %s; retrying without context_token",
|
||||
self.name, _safe_id(chat_id),
|
||||
)
|
||||
continue
|
||||
errmsg = resp.get("errmsg") or resp.get("msg") or "unknown error"
|
||||
raise RuntimeError(
|
||||
f"iLink sendmessage error: ret={ret} errcode={errcode} errmsg={errmsg}"
|
||||
)
|
||||
return
|
||||
except Exception as exc:
|
||||
last_error = exc
|
||||
@@ -1562,48 +1446,12 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
context_token = self._token_store.get(self._account_id, chat_id)
|
||||
last_message_id: Optional[str] = None
|
||||
|
||||
# Extract MEDIA: tags and bare local file paths before text delivery.
|
||||
media_files, cleaned_content = self.extract_media(content)
|
||||
_, image_cleaned = self.extract_images(cleaned_content)
|
||||
local_files, final_content = self.extract_local_files(image_cleaned)
|
||||
|
||||
_AUDIO_EXTS = {".ogg", ".opus", ".mp3", ".wav", ".m4a"}
|
||||
_VIDEO_EXTS = {".mp4", ".mov", ".avi", ".mkv", ".webm", ".3gp"}
|
||||
_IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".webp", ".gif"}
|
||||
|
||||
async def _deliver_media(path: str, is_voice: bool = False) -> None:
|
||||
ext = Path(path).suffix.lower()
|
||||
if is_voice or ext in _AUDIO_EXTS:
|
||||
await self.send_voice(chat_id=chat_id, audio_path=path, metadata=metadata)
|
||||
elif ext in _VIDEO_EXTS:
|
||||
await self.send_video(chat_id=chat_id, video_path=path, metadata=metadata)
|
||||
elif ext in _IMAGE_EXTS:
|
||||
await self.send_image_file(chat_id=chat_id, image_path=path, metadata=metadata)
|
||||
else:
|
||||
await self.send_document(chat_id=chat_id, file_path=path, metadata=metadata)
|
||||
|
||||
try:
|
||||
# Deliver extracted MEDIA: attachments first.
|
||||
for media_path, is_voice in media_files:
|
||||
try:
|
||||
await _deliver_media(media_path, is_voice)
|
||||
except Exception as exc:
|
||||
logger.warning("[%s] media delivery failed for %s: %s", self.name, media_path, exc)
|
||||
|
||||
# Deliver bare local file paths.
|
||||
for file_path in local_files:
|
||||
try:
|
||||
await _deliver_media(file_path, is_voice=False)
|
||||
except Exception as exc:
|
||||
logger.warning("[%s] local file delivery failed for %s: %s", self.name, file_path, exc)
|
||||
|
||||
# Deliver text content.
|
||||
chunks = [c for c in self._split_text(self.format_message(final_content)) if c and c.strip()]
|
||||
chunks = self._split_text(self.format_message(content))
|
||||
for idx, chunk in enumerate(chunks):
|
||||
client_id = f"hermes-weixin-{uuid.uuid4().hex}"
|
||||
await self._send_text_chunk(
|
||||
@@ -1621,14 +1469,14 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
async def send_typing(self, chat_id: str, metadata: Optional[Dict[str, Any]] = None) -> None:
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return
|
||||
typing_ticket = self._typing_cache.get(chat_id)
|
||||
if not typing_ticket:
|
||||
return
|
||||
try:
|
||||
await _send_typing(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to_user_id=chat_id,
|
||||
@@ -1639,14 +1487,14 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
logger.debug("[%s] typing start failed for %s: %s", self.name, _safe_id(chat_id), exc)
|
||||
|
||||
async def stop_typing(self, chat_id: str) -> None:
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return
|
||||
typing_ticket = self._typing_cache.get(chat_id)
|
||||
if not typing_ticket:
|
||||
return
|
||||
try:
|
||||
await _send_typing(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to_user_id=chat_id,
|
||||
@@ -1684,92 +1532,37 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
async def send_image_file(
|
||||
self,
|
||||
chat_id: str,
|
||||
image_path: str,
|
||||
caption: Optional[str] = None,
|
||||
path: str,
|
||||
caption: str = "",
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
del reply_to, kwargs
|
||||
return await self.send_document(
|
||||
chat_id=chat_id,
|
||||
file_path=image_path,
|
||||
caption=caption,
|
||||
metadata=metadata,
|
||||
)
|
||||
return await self.send_document(chat_id, path, caption=caption, metadata=metadata)
|
||||
|
||||
async def send_document(
|
||||
self,
|
||||
chat_id: str,
|
||||
file_path: str,
|
||||
caption: Optional[str] = None,
|
||||
file_name: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
path: str,
|
||||
caption: str = "",
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
del file_name, reply_to, metadata, kwargs
|
||||
if not self._send_session or not self._token:
|
||||
if not self._session or not self._token:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
try:
|
||||
message_id = await self._send_file(chat_id, file_path, caption or "")
|
||||
message_id = await self._send_file(chat_id, path, caption)
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
except Exception as exc:
|
||||
logger.error("[%s] send_document failed to=%s: %s", self.name, _safe_id(chat_id), exc)
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
async def send_video(
|
||||
self,
|
||||
chat_id: str,
|
||||
video_path: str,
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
if not self._send_session or not self._token:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
try:
|
||||
message_id = await self._send_file(chat_id, video_path, caption or "")
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
except Exception as exc:
|
||||
logger.error("[%s] send_video failed to=%s: %s", self.name, _safe_id(chat_id), exc)
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
async def send_voice(
|
||||
self,
|
||||
chat_id: str,
|
||||
audio_path: str,
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
if not self._send_session or not self._token:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
|
||||
# Native outbound Weixin voice bubbles are not proven-working in the
|
||||
# upstream reference implementation. Prefer a reliable file attachment
|
||||
# fallback so users at least receive playable audio, even for .silk.
|
||||
fallback_caption = caption or "[voice message as attachment]"
|
||||
try:
|
||||
message_id = await self._send_file(
|
||||
chat_id,
|
||||
audio_path,
|
||||
fallback_caption,
|
||||
force_file_attachment=True,
|
||||
)
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
except Exception as exc:
|
||||
logger.error("[%s] send_voice failed to=%s: %s", self.name, _safe_id(chat_id), exc)
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
async def _download_remote_media(self, url: str) -> str:
|
||||
from tools.url_safety import is_safe_url
|
||||
|
||||
if not is_safe_url(url):
|
||||
raise ValueError(f"Blocked unsafe URL (SSRF protection): {url}")
|
||||
|
||||
assert self._send_session is not None
|
||||
async with self._send_session.get(url, timeout=aiohttp.ClientTimeout(total=30)) as response:
|
||||
assert self._session is not None
|
||||
async with self._session.get(url, timeout=aiohttp.ClientTimeout(total=30)) as response:
|
||||
response.raise_for_status()
|
||||
data = await response.read()
|
||||
suffix = Path(url.split("?", 1)[0]).suffix or ".bin"
|
||||
@@ -1777,75 +1570,63 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
handle.write(data)
|
||||
return handle.name
|
||||
|
||||
async def _send_file(
|
||||
self,
|
||||
chat_id: str,
|
||||
path: str,
|
||||
caption: str,
|
||||
force_file_attachment: bool = False,
|
||||
) -> str:
|
||||
assert self._send_session is not None and self._token is not None
|
||||
async def _send_file(self, chat_id: str, path: str, caption: str) -> str:
|
||||
assert self._session is not None and self._token is not None
|
||||
plaintext = Path(path).read_bytes()
|
||||
media_type, item_builder = self._outbound_media_builder(path, force_file_attachment=force_file_attachment)
|
||||
media_type, item_builder = self._outbound_media_builder(path)
|
||||
filekey = secrets.token_hex(16)
|
||||
aes_key = secrets.token_bytes(16)
|
||||
rawsize = len(plaintext)
|
||||
rawfilemd5 = hashlib.md5(plaintext).hexdigest()
|
||||
upload_response = await _get_upload_url(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to_user_id=chat_id,
|
||||
media_type=media_type,
|
||||
filekey=filekey,
|
||||
rawsize=rawsize,
|
||||
rawfilemd5=rawfilemd5,
|
||||
rawfilemd5=hashlib.md5(plaintext).hexdigest(),
|
||||
filesize=_aes_padded_size(rawsize),
|
||||
aeskey_hex=aes_key.hex(),
|
||||
)
|
||||
upload_param = str(upload_response.get("upload_param") or "")
|
||||
upload_full_url = str(upload_response.get("upload_full_url") or "")
|
||||
ciphertext = _aes128_ecb_encrypt(plaintext, aes_key)
|
||||
|
||||
# Prefer upload_full_url (direct CDN), fall back to constructed CDN URL
|
||||
# from upload_param. Both paths use POST — the old PUT for
|
||||
# upload_full_url caused 404s on the WeChat CDN.
|
||||
if upload_full_url:
|
||||
upload_url = upload_full_url
|
||||
elif upload_param:
|
||||
upload_url = _cdn_upload_url(self._cdn_base_url, upload_param, filekey)
|
||||
if upload_param:
|
||||
encrypted_query_param = await _upload_ciphertext(
|
||||
self._session,
|
||||
ciphertext=ciphertext,
|
||||
cdn_base_url=self._cdn_base_url,
|
||||
upload_param=upload_param,
|
||||
filekey=filekey,
|
||||
)
|
||||
elif upload_full_url:
|
||||
timeout = aiohttp.ClientTimeout(total=120)
|
||||
async with self._session.put(
|
||||
upload_full_url,
|
||||
data=ciphertext,
|
||||
headers={"Content-Type": "application/octet-stream"},
|
||||
timeout=timeout,
|
||||
) as response:
|
||||
response.raise_for_status()
|
||||
encrypted_query_param = response.headers.get("x-encrypted-param") or filekey
|
||||
else:
|
||||
raise RuntimeError(f"getUploadUrl returned neither upload_param nor upload_full_url: {upload_response}")
|
||||
|
||||
encrypted_query_param = await _upload_ciphertext(
|
||||
self._send_session,
|
||||
ciphertext=ciphertext,
|
||||
upload_url=upload_url,
|
||||
)
|
||||
context_token = self._token_store.get(self._account_id, chat_id)
|
||||
# The iLink API expects aes_key as base64(hex_string), not base64(raw_bytes).
|
||||
# Sending base64(raw_bytes) causes images to show as grey boxes on the
|
||||
# receiver side because the decryption key doesn't match.
|
||||
aes_key_for_api = base64.b64encode(aes_key.hex().encode("ascii")).decode("ascii")
|
||||
item_kwargs = {
|
||||
"encrypt_query_param": encrypted_query_param,
|
||||
"aes_key_for_api": aes_key_for_api,
|
||||
"ciphertext_size": len(ciphertext),
|
||||
"plaintext_size": rawsize,
|
||||
"filename": Path(path).name,
|
||||
"rawfilemd5": rawfilemd5,
|
||||
}
|
||||
if media_type == MEDIA_VOICE and path.endswith(".silk"):
|
||||
item_kwargs["encode_type"] = 6
|
||||
item_kwargs["sample_rate"] = 24000
|
||||
item_kwargs["bits_per_sample"] = 16
|
||||
media_item = item_builder(**item_kwargs)
|
||||
media_item = item_builder(
|
||||
encrypt_query_param=encrypted_query_param,
|
||||
aes_key_b64=base64.b64encode(aes_key).decode("ascii"),
|
||||
ciphertext_size=len(ciphertext),
|
||||
plaintext_size=rawsize,
|
||||
filename=Path(path).name,
|
||||
)
|
||||
|
||||
last_message_id = None
|
||||
if caption:
|
||||
last_message_id = f"hermes-weixin-{uuid.uuid4().hex}"
|
||||
await _send_message(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
token=self._token,
|
||||
to=chat_id,
|
||||
@@ -1856,7 +1637,7 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
|
||||
last_message_id = f"hermes-weixin-{uuid.uuid4().hex}"
|
||||
await _api_post(
|
||||
self._send_session,
|
||||
self._session,
|
||||
base_url=self._base_url,
|
||||
endpoint=EP_SEND_MESSAGE,
|
||||
payload={
|
||||
@@ -1875,72 +1656,42 @@ class WeixinAdapter(BasePlatformAdapter):
|
||||
)
|
||||
return last_message_id
|
||||
|
||||
def _outbound_media_builder(self, path: str, force_file_attachment: bool = False):
|
||||
def _outbound_media_builder(self, path: str):
|
||||
mime = mimetypes.guess_type(path)[0] or "application/octet-stream"
|
||||
if mime.startswith("image/"):
|
||||
return MEDIA_IMAGE, lambda **kw: {
|
||||
return MEDIA_IMAGE, lambda **kwargs: {
|
||||
"type": ITEM_IMAGE,
|
||||
"image_item": {
|
||||
"media": {
|
||||
"encrypt_query_param": kw["encrypt_query_param"],
|
||||
"aes_key": kw["aes_key_for_api"],
|
||||
"encrypt_query_param": kwargs["encrypt_query_param"],
|
||||
"aes_key": kwargs["aes_key_b64"],
|
||||
"encrypt_type": 1,
|
||||
},
|
||||
"mid_size": kw["ciphertext_size"],
|
||||
"mid_size": kwargs["ciphertext_size"],
|
||||
},
|
||||
}
|
||||
if mime.startswith("video/"):
|
||||
return MEDIA_VIDEO, lambda **kw: {
|
||||
return MEDIA_VIDEO, lambda **kwargs: {
|
||||
"type": ITEM_VIDEO,
|
||||
"video_item": {
|
||||
"media": {
|
||||
"encrypt_query_param": kw["encrypt_query_param"],
|
||||
"aes_key": kw["aes_key_for_api"],
|
||||
"encrypt_query_param": kwargs["encrypt_query_param"],
|
||||
"aes_key": kwargs["aes_key_b64"],
|
||||
"encrypt_type": 1,
|
||||
},
|
||||
"video_size": kw["ciphertext_size"],
|
||||
"play_length": kw.get("play_length", 0),
|
||||
"video_md5": kw.get("rawfilemd5", ""),
|
||||
"video_size": kwargs["ciphertext_size"],
|
||||
},
|
||||
}
|
||||
if path.endswith(".silk") and not force_file_attachment:
|
||||
return MEDIA_VOICE, lambda **kw: {
|
||||
"type": ITEM_VOICE,
|
||||
"voice_item": {
|
||||
"media": {
|
||||
"encrypt_query_param": kw["encrypt_query_param"],
|
||||
"aes_key": kw["aes_key_for_api"],
|
||||
"encrypt_type": 1,
|
||||
},
|
||||
"encode_type": kw.get("encode_type"),
|
||||
"bits_per_sample": kw.get("bits_per_sample"),
|
||||
"sample_rate": kw.get("sample_rate"),
|
||||
"playtime": kw.get("playtime", 0),
|
||||
},
|
||||
}
|
||||
if mime.startswith("audio/"):
|
||||
return MEDIA_FILE, lambda **kw: {
|
||||
"type": ITEM_FILE,
|
||||
"file_item": {
|
||||
"media": {
|
||||
"encrypt_query_param": kw["encrypt_query_param"],
|
||||
"aes_key": kw["aes_key_for_api"],
|
||||
"encrypt_type": 1,
|
||||
},
|
||||
"file_name": kw["filename"],
|
||||
"len": str(kw["plaintext_size"]),
|
||||
},
|
||||
}
|
||||
return MEDIA_FILE, lambda **kw: {
|
||||
return MEDIA_FILE, lambda **kwargs: {
|
||||
"type": ITEM_FILE,
|
||||
"file_item": {
|
||||
"media": {
|
||||
"encrypt_query_param": kw["encrypt_query_param"],
|
||||
"aes_key": kw["aes_key_for_api"],
|
||||
"encrypt_query_param": kwargs["encrypt_query_param"],
|
||||
"aes_key": kwargs["aes_key_b64"],
|
||||
"encrypt_type": 1,
|
||||
},
|
||||
"file_name": kw["filename"],
|
||||
"len": str(kw["plaintext_size"]),
|
||||
"file_name": kwargs["filename"],
|
||||
"len": str(kwargs["plaintext_size"]),
|
||||
},
|
||||
}
|
||||
|
||||
@@ -1980,34 +1731,7 @@ async def send_weixin_direct(
|
||||
token_store.restore(account_id)
|
||||
context_token = token_store.get(account_id, chat_id)
|
||||
|
||||
live_adapter = _LIVE_ADAPTERS.get(resolved_token)
|
||||
send_session = getattr(live_adapter, '_send_session', None)
|
||||
if live_adapter is not None and send_session is not None and not send_session.closed:
|
||||
last_result: Optional[SendResult] = None
|
||||
cleaned = live_adapter.format_message(message)
|
||||
if cleaned:
|
||||
last_result = await live_adapter.send(chat_id, cleaned)
|
||||
if not last_result.success:
|
||||
return {"error": f"Weixin send failed: {last_result.error}"}
|
||||
|
||||
for media_path, _is_voice in media_files or []:
|
||||
ext = Path(media_path).suffix.lower()
|
||||
if ext in {".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp"}:
|
||||
last_result = await live_adapter.send_image_file(chat_id, media_path)
|
||||
else:
|
||||
last_result = await live_adapter.send_document(chat_id, media_path)
|
||||
if not last_result.success:
|
||||
return {"error": f"Weixin media send failed: {last_result.error}"}
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"platform": "weixin",
|
||||
"chat_id": chat_id,
|
||||
"message_id": last_result.message_id if last_result else None,
|
||||
"context_token_used": bool(context_token),
|
||||
}
|
||||
|
||||
async with aiohttp.ClientSession(trust_env=True, connector=_make_ssl_connector()) as session:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
adapter = WeixinAdapter(
|
||||
PlatformConfig(
|
||||
enabled=True,
|
||||
@@ -2020,7 +1744,6 @@ async def send_weixin_direct(
|
||||
},
|
||||
)
|
||||
)
|
||||
adapter._send_session = session
|
||||
adapter._session = session
|
||||
adapter._token = resolved_token
|
||||
adapter._account_id = account_id
|
||||
|
||||
+26
-103
@@ -120,9 +120,8 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
- session_path: Path to store WhatsApp session data
|
||||
"""
|
||||
|
||||
# WhatsApp message limits — practical UX limit, not protocol max.
|
||||
# WhatsApp allows ~65K but long messages are unreadable on mobile.
|
||||
MAX_MESSAGE_LENGTH = 4096
|
||||
# WhatsApp message limits
|
||||
MAX_MESSAGE_LENGTH = 65536 # WhatsApp allows longer messages
|
||||
|
||||
# Default bridge location relative to the hermes-agent install
|
||||
_DEFAULT_BRIDGE_DIR = Path(__file__).resolve().parents[2] / "scripts" / "whatsapp-bridge"
|
||||
@@ -532,63 +531,6 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
self._close_bridge_log()
|
||||
print(f"[{self.name}] Disconnected")
|
||||
|
||||
def format_message(self, content: str) -> str:
|
||||
"""Convert standard markdown to WhatsApp-compatible formatting.
|
||||
|
||||
WhatsApp supports: *bold*, _italic_, ~strikethrough~, ```code```,
|
||||
and monospaced `inline`. Standard markdown uses different syntax
|
||||
for bold/italic/strikethrough, so we convert here.
|
||||
|
||||
Code blocks (``` fenced) and inline code (`) are protected from
|
||||
conversion via placeholder substitution.
|
||||
"""
|
||||
if not content:
|
||||
return content
|
||||
|
||||
# --- 1. Protect fenced code blocks from formatting changes ---
|
||||
_FENCE_PH = "\x00FENCE"
|
||||
fences: list[str] = []
|
||||
|
||||
def _save_fence(m: re.Match) -> str:
|
||||
fences.append(m.group(0))
|
||||
return f"{_FENCE_PH}{len(fences) - 1}\x00"
|
||||
|
||||
result = re.sub(r"```[\s\S]*?```", _save_fence, content)
|
||||
|
||||
# --- 2. Protect inline code ---
|
||||
_CODE_PH = "\x00CODE"
|
||||
codes: list[str] = []
|
||||
|
||||
def _save_code(m: re.Match) -> str:
|
||||
codes.append(m.group(0))
|
||||
return f"{_CODE_PH}{len(codes) - 1}\x00"
|
||||
|
||||
result = re.sub(r"`[^`\n]+`", _save_code, result)
|
||||
|
||||
# --- 3. Convert markdown formatting to WhatsApp syntax ---
|
||||
# Bold: **text** or __text__ → *text*
|
||||
result = re.sub(r"\*\*(.+?)\*\*", r"*\1*", result)
|
||||
result = re.sub(r"__(.+?)__", r"*\1*", result)
|
||||
# Strikethrough: ~~text~~ → ~text~
|
||||
result = re.sub(r"~~(.+?)~~", r"~\1~", result)
|
||||
# Italic: *text* is already WhatsApp italic — leave as-is
|
||||
# _text_ is already WhatsApp italic — leave as-is
|
||||
|
||||
# --- 4. Convert markdown headers to bold text ---
|
||||
# # Header → *Header*
|
||||
result = re.sub(r"^#{1,6}\s+(.+)$", r"*\1*", result, flags=re.MULTILINE)
|
||||
|
||||
# --- 5. Convert markdown links: [text](url) → text (url) ---
|
||||
result = re.sub(r"\[([^\]]+)\]\(([^)]+)\)", r"\1 (\2)", result)
|
||||
|
||||
# --- 6. Restore protected sections ---
|
||||
for i, fence in enumerate(fences):
|
||||
result = result.replace(f"{_FENCE_PH}{i}\x00", fence)
|
||||
for i, code in enumerate(codes):
|
||||
result = result.replace(f"{_CODE_PH}{i}\x00", code)
|
||||
|
||||
return result
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
@@ -596,57 +538,38 @@ class WhatsAppAdapter(BasePlatformAdapter):
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None
|
||||
) -> SendResult:
|
||||
"""Send a message via the WhatsApp bridge.
|
||||
|
||||
Formats markdown for WhatsApp, splits long messages into chunks
|
||||
that preserve code block boundaries, and sends each chunk sequentially.
|
||||
"""
|
||||
"""Send a message via the WhatsApp bridge."""
|
||||
if not self._running or not self._http_session:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
bridge_exit = await self._check_managed_bridge_exit()
|
||||
if bridge_exit:
|
||||
return SendResult(success=False, error=bridge_exit)
|
||||
|
||||
if not content or not content.strip():
|
||||
return SendResult(success=True, message_id=None)
|
||||
|
||||
|
||||
try:
|
||||
import aiohttp
|
||||
|
||||
# Format and chunk the message
|
||||
formatted = self.format_message(content)
|
||||
chunks = self.truncate_message(formatted, self.MAX_MESSAGE_LENGTH)
|
||||
|
||||
last_message_id = None
|
||||
for chunk in chunks:
|
||||
payload: Dict[str, Any] = {
|
||||
"chatId": chat_id,
|
||||
"message": chunk,
|
||||
}
|
||||
if reply_to and last_message_id is None:
|
||||
# Only reply-to on the first chunk
|
||||
payload["replyTo"] = reply_to
|
||||
|
||||
async with self._http_session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/send",
|
||||
json=payload,
|
||||
timeout=aiohttp.ClientTimeout(total=30)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
data = await resp.json()
|
||||
last_message_id = data.get("messageId")
|
||||
else:
|
||||
error = await resp.text()
|
||||
return SendResult(success=False, error=error)
|
||||
|
||||
# Small delay between chunks to avoid rate limiting
|
||||
if len(chunks) > 1:
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=last_message_id,
|
||||
)
|
||||
payload = {
|
||||
"chatId": chat_id,
|
||||
"message": content,
|
||||
}
|
||||
if reply_to:
|
||||
payload["replyTo"] = reply_to
|
||||
|
||||
async with self._http_session.post(
|
||||
f"http://127.0.0.1:{self._bridge_port}/send",
|
||||
json=payload,
|
||||
timeout=aiohttp.ClientTimeout(total=30)
|
||||
) as resp:
|
||||
if resp.status == 200:
|
||||
data = await resp.json()
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=data.get("messageId"),
|
||||
raw_response=data
|
||||
)
|
||||
else:
|
||||
error = await resp.text()
|
||||
return SendResult(success=False, error=error)
|
||||
except Exception as e:
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
|
||||
+366
-2101
File diff suppressed because it is too large
Load Diff
+4
-66
@@ -12,6 +12,7 @@ import hashlib
|
||||
import logging
|
||||
import os
|
||||
import json
|
||||
import re
|
||||
import threading
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
@@ -82,7 +83,6 @@ class SessionSource:
|
||||
chat_topic: Optional[str] = None # Channel topic/description (Discord, Slack)
|
||||
user_id_alt: Optional[str] = None # Signal UUID (alternative to phone number)
|
||||
chat_id_alt: Optional[str] = None # Signal group internal ID
|
||||
is_bot: bool = False # True when the message author is a bot/webhook (Discord)
|
||||
|
||||
@property
|
||||
def description(self) -> str:
|
||||
@@ -302,8 +302,6 @@ def build_session_context_prompt(
|
||||
lines.append("")
|
||||
lines.append("**Delivery options for scheduled tasks:**")
|
||||
|
||||
from hermes_constants import display_hermes_home
|
||||
|
||||
# Origin delivery
|
||||
if context.source.platform == Platform.LOCAL:
|
||||
lines.append("- `\"origin\"` → Local output (saved to files)")
|
||||
@@ -312,11 +310,9 @@ def build_session_context_prompt(
|
||||
_hash_chat_id(context.source.chat_id) if redact_pii else context.source.chat_id
|
||||
)
|
||||
lines.append(f"- `\"origin\"` → Back to this chat ({_origin_label})")
|
||||
|
||||
|
||||
# Local always available
|
||||
lines.append(
|
||||
f"- `\"local\"` → Save to local files only ({display_hermes_home()}/cron/output/)"
|
||||
)
|
||||
lines.append("- `\"local\"` → Save to local files only (~/.hermes/cron/output/)")
|
||||
|
||||
# Platform home channels
|
||||
for platform, home in context.home_channels.items():
|
||||
@@ -802,57 +798,6 @@ class SessionStore:
|
||||
return True
|
||||
return False
|
||||
|
||||
def prune_old_entries(self, max_age_days: int) -> int:
|
||||
"""Drop SessionEntry records older than max_age_days.
|
||||
|
||||
Pruning is based on ``updated_at`` (last activity), not ``created_at``.
|
||||
A session that's been active within the window is kept regardless of
|
||||
how old it is. Entries marked ``suspended`` are kept — the user
|
||||
explicitly paused them for later resume. Entries held by an active
|
||||
process (via has_active_processes_fn) are also kept so long-running
|
||||
background work isn't orphaned.
|
||||
|
||||
Pruning is functionally identical to a natural reset-policy expiry:
|
||||
the transcript in SQLite stays, but the session_key → session_id
|
||||
mapping is dropped and the user starts a fresh session on return.
|
||||
|
||||
``max_age_days <= 0`` disables pruning; returns 0 immediately.
|
||||
Returns the number of entries removed.
|
||||
"""
|
||||
if max_age_days is None or max_age_days <= 0:
|
||||
return 0
|
||||
from datetime import timedelta
|
||||
|
||||
cutoff = _now() - timedelta(days=max_age_days)
|
||||
removed_keys: list[str] = []
|
||||
|
||||
with self._lock:
|
||||
self._ensure_loaded_locked()
|
||||
for key, entry in list(self._entries.items()):
|
||||
if entry.suspended:
|
||||
continue
|
||||
# Never prune sessions with an active background process
|
||||
# attached — the user may still be waiting on output.
|
||||
if self._has_active_processes_fn is not None:
|
||||
try:
|
||||
if self._has_active_processes_fn(entry.session_id):
|
||||
continue
|
||||
except Exception:
|
||||
pass
|
||||
if entry.updated_at < cutoff:
|
||||
removed_keys.append(key)
|
||||
for key in removed_keys:
|
||||
self._entries.pop(key, None)
|
||||
if removed_keys:
|
||||
self._save()
|
||||
|
||||
if removed_keys:
|
||||
logger.info(
|
||||
"SessionStore pruned %d entries older than %d days",
|
||||
len(removed_keys), max_age_days,
|
||||
)
|
||||
return len(removed_keys)
|
||||
|
||||
def suspend_recently_active(self, max_age_seconds: int = 120) -> int:
|
||||
"""Mark recently-active sessions as suspended.
|
||||
|
||||
@@ -933,8 +878,7 @@ class SessionStore:
|
||||
Used by ``/resume`` to restore a previously-named session.
|
||||
Ends the current session in SQLite (like reset), but instead of
|
||||
generating a fresh session ID, re-uses ``target_session_id`` so the
|
||||
old transcript is loaded on the next message. If the target session was
|
||||
previously ended, re-open it so gateway resume semantics match the CLI.
|
||||
old transcript is loaded on the next message.
|
||||
"""
|
||||
db_end_session_id = None
|
||||
new_entry = None
|
||||
@@ -974,12 +918,6 @@ class SessionStore:
|
||||
except Exception as e:
|
||||
logger.debug("Session DB end_session failed: %s", e)
|
||||
|
||||
if self._db:
|
||||
try:
|
||||
self._db.reopen_session(target_session_id)
|
||||
except Exception as e:
|
||||
logger.debug("Session DB reopen_session failed: %s", e)
|
||||
|
||||
return new_entry
|
||||
|
||||
def list_sessions(self, active_minutes: Optional[int] = None) -> List[SessionEntry]:
|
||||
|
||||
+17
-34
@@ -37,24 +37,18 @@ needs to replace the import + call site:
|
||||
"""
|
||||
|
||||
from contextvars import ContextVar
|
||||
from typing import Any
|
||||
|
||||
# Sentinel to distinguish "never set in this context" from "explicitly set to empty".
|
||||
# When a contextvar holds _UNSET, we fall back to os.environ (CLI/cron compat).
|
||||
# When it holds "" (after clear_session_vars resets it), we return "" — no fallback.
|
||||
_UNSET: Any = object()
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Per-task session variables
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_SESSION_PLATFORM: ContextVar = ContextVar("HERMES_SESSION_PLATFORM", default=_UNSET)
|
||||
_SESSION_CHAT_ID: ContextVar = ContextVar("HERMES_SESSION_CHAT_ID", default=_UNSET)
|
||||
_SESSION_CHAT_NAME: ContextVar = ContextVar("HERMES_SESSION_CHAT_NAME", default=_UNSET)
|
||||
_SESSION_THREAD_ID: ContextVar = ContextVar("HERMES_SESSION_THREAD_ID", default=_UNSET)
|
||||
_SESSION_USER_ID: ContextVar = ContextVar("HERMES_SESSION_USER_ID", default=_UNSET)
|
||||
_SESSION_USER_NAME: ContextVar = ContextVar("HERMES_SESSION_USER_NAME", default=_UNSET)
|
||||
_SESSION_KEY: ContextVar = ContextVar("HERMES_SESSION_KEY", default=_UNSET)
|
||||
_SESSION_PLATFORM: ContextVar[str] = ContextVar("HERMES_SESSION_PLATFORM", default="")
|
||||
_SESSION_CHAT_ID: ContextVar[str] = ContextVar("HERMES_SESSION_CHAT_ID", default="")
|
||||
_SESSION_CHAT_NAME: ContextVar[str] = ContextVar("HERMES_SESSION_CHAT_NAME", default="")
|
||||
_SESSION_THREAD_ID: ContextVar[str] = ContextVar("HERMES_SESSION_THREAD_ID", default="")
|
||||
_SESSION_USER_ID: ContextVar[str] = ContextVar("HERMES_SESSION_USER_ID", default="")
|
||||
_SESSION_USER_NAME: ContextVar[str] = ContextVar("HERMES_SESSION_USER_NAME", default="")
|
||||
_SESSION_KEY: ContextVar[str] = ContextVar("HERMES_SESSION_KEY", default="")
|
||||
|
||||
_VAR_MAP = {
|
||||
"HERMES_SESSION_PLATFORM": _SESSION_PLATFORM,
|
||||
@@ -97,17 +91,10 @@ def set_session_vars(
|
||||
|
||||
|
||||
def clear_session_vars(tokens: list) -> None:
|
||||
"""Mark session context variables as explicitly cleared.
|
||||
|
||||
Sets all variables to ``""`` so that ``get_session_env`` returns an empty
|
||||
string instead of falling back to (potentially stale) ``os.environ``
|
||||
values. The *tokens* argument is accepted for API compatibility with
|
||||
callers that saved the return value of ``set_session_vars``, but the
|
||||
actual clearing uses ``var.set("")`` rather than ``var.reset(token)``
|
||||
to ensure the "explicitly cleared" state is distinguishable from
|
||||
"never set" (which holds the ``_UNSET`` sentinel).
|
||||
"""
|
||||
for var in (
|
||||
"""Restore session context variables to their pre-handler values."""
|
||||
if not tokens:
|
||||
return
|
||||
vars_in_order = [
|
||||
_SESSION_PLATFORM,
|
||||
_SESSION_CHAT_ID,
|
||||
_SESSION_CHAT_NAME,
|
||||
@@ -115,8 +102,9 @@ def clear_session_vars(tokens: list) -> None:
|
||||
_SESSION_USER_ID,
|
||||
_SESSION_USER_NAME,
|
||||
_SESSION_KEY,
|
||||
):
|
||||
var.set("")
|
||||
]
|
||||
for var, token in zip(vars_in_order, tokens):
|
||||
var.reset(token)
|
||||
|
||||
|
||||
def get_session_env(name: str, default: str = "") -> str:
|
||||
@@ -125,13 +113,8 @@ def get_session_env(name: str, default: str = "") -> str:
|
||||
Drop-in replacement for ``os.getenv("HERMES_SESSION_*", default)``.
|
||||
|
||||
Resolution order:
|
||||
1. Context variable (set by the gateway for concurrency-safe access).
|
||||
If the variable was explicitly set (even to ``""``) via
|
||||
``set_session_vars`` or ``clear_session_vars``, that value is
|
||||
returned — **no fallback to os.environ**.
|
||||
2. ``os.environ`` (only when the context variable was never set in
|
||||
this context — i.e. CLI, cron scheduler, and test processes that
|
||||
don't use ``set_session_vars`` at all).
|
||||
1. Context variable (set by the gateway for concurrency-safe access)
|
||||
2. ``os.environ`` (used by CLI, cron scheduler, and tests)
|
||||
3. *default*
|
||||
"""
|
||||
import os
|
||||
@@ -139,7 +122,7 @@ def get_session_env(name: str, default: str = "") -> str:
|
||||
var = _VAR_MAP.get(name)
|
||||
if var is not None:
|
||||
value = var.get()
|
||||
if value is not _UNSET:
|
||||
if value:
|
||||
return value
|
||||
# Fall back to os.environ for CLI, cron, and test compatibility
|
||||
return os.getenv(name, default)
|
||||
|
||||
+13
-186
@@ -188,8 +188,8 @@ def _write_json_file(path: Path, payload: dict[str, Any]) -> None:
|
||||
path.write_text(json.dumps(payload))
|
||||
|
||||
|
||||
def _read_pid_record(pid_path: Optional[Path] = None) -> Optional[dict]:
|
||||
pid_path = pid_path or _get_pid_path()
|
||||
def _read_pid_record() -> Optional[dict]:
|
||||
pid_path = _get_pid_path()
|
||||
if not pid_path.exists():
|
||||
return None
|
||||
|
||||
@@ -212,18 +212,6 @@ def _read_pid_record(pid_path: Optional[Path] = None) -> Optional[dict]:
|
||||
return None
|
||||
|
||||
|
||||
def _cleanup_invalid_pid_path(pid_path: Path, *, cleanup_stale: bool) -> None:
|
||||
if not cleanup_stale:
|
||||
return
|
||||
try:
|
||||
if pid_path == _get_pid_path():
|
||||
remove_pid_file()
|
||||
else:
|
||||
pid_path.unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def write_pid_file() -> None:
|
||||
"""Write the current process PID and metadata to the gateway PID file."""
|
||||
_write_json_file(_get_pid_path(), _build_pid_record())
|
||||
@@ -278,25 +266,9 @@ def read_runtime_status() -> Optional[dict[str, Any]]:
|
||||
|
||||
|
||||
def remove_pid_file() -> None:
|
||||
"""Remove the gateway PID file, but only if it belongs to this process.
|
||||
|
||||
During --replace handoffs, the old process's atexit handler can fire AFTER
|
||||
the new process has written its own PID file. Blindly removing the file
|
||||
would delete the new process's record, leaving the gateway running with no
|
||||
PID file (invisible to ``get_running_pid()``).
|
||||
"""
|
||||
"""Remove the gateway PID file if it exists."""
|
||||
try:
|
||||
path = _get_pid_path()
|
||||
record = _read_json_file(path)
|
||||
if record is not None:
|
||||
try:
|
||||
file_pid = int(record["pid"])
|
||||
except (KeyError, TypeError, ValueError):
|
||||
file_pid = None
|
||||
if file_pid is not None and file_pid != os.getpid():
|
||||
# PID file belongs to a different process — leave it alone.
|
||||
return
|
||||
path.unlink(missing_ok=True)
|
||||
_get_pid_path().unlink(missing_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -318,15 +290,6 @@ def acquire_scoped_lock(scope: str, identity: str, metadata: Optional[dict[str,
|
||||
}
|
||||
|
||||
existing = _read_json_file(lock_path)
|
||||
if existing is None and lock_path.exists():
|
||||
# Lock file exists but is empty or contains invalid JSON — treat as
|
||||
# stale. This happens when a previous process was killed between
|
||||
# O_CREAT|O_EXCL and the subsequent json.dump() (e.g. DNS failure
|
||||
# during rapid Slack reconnect retries).
|
||||
try:
|
||||
lock_path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
if existing:
|
||||
try:
|
||||
existing_pid = int(existing["pid"])
|
||||
@@ -425,179 +388,43 @@ def release_all_scoped_locks() -> int:
|
||||
return removed
|
||||
|
||||
|
||||
# ── --replace takeover marker ─────────────────────────────────────────
|
||||
#
|
||||
# When a new gateway starts with ``--replace``, it SIGTERMs the existing
|
||||
# gateway so it can take over the bot token. PR #5646 made SIGTERM exit
|
||||
# the gateway with code 1 so ``Restart=on-failure`` can revive it after
|
||||
# unexpected kills — but that also means a --replace takeover target
|
||||
# exits 1, which tricks systemd into reviving it 30 seconds later,
|
||||
# starting a flap loop against the replacer when both services are
|
||||
# enabled in the user's systemd (e.g. ``hermes.service`` + ``hermes-
|
||||
# gateway.service``).
|
||||
#
|
||||
# The takeover marker breaks the loop: the replacer writes a short-lived
|
||||
# file naming the target PID + start_time BEFORE sending SIGTERM.
|
||||
# The target's shutdown handler reads the marker and, if it names
|
||||
# this process, treats the SIGTERM as a planned takeover and exits 0.
|
||||
# The marker is unlinked after the target has consumed it, so a stale
|
||||
# marker left by a crashed replacer can grief at most one future
|
||||
# shutdown on the same PID — and only within _TAKEOVER_MARKER_TTL_S.
|
||||
|
||||
_TAKEOVER_MARKER_FILENAME = ".gateway-takeover.json"
|
||||
_TAKEOVER_MARKER_TTL_S = 60 # Marker older than this is treated as stale
|
||||
|
||||
|
||||
def _get_takeover_marker_path() -> Path:
|
||||
"""Return the path to the --replace takeover marker file."""
|
||||
home = get_hermes_home()
|
||||
return home / _TAKEOVER_MARKER_FILENAME
|
||||
|
||||
|
||||
def write_takeover_marker(target_pid: int) -> bool:
|
||||
"""Record that ``target_pid`` is being replaced by the current process.
|
||||
|
||||
Captures the target's ``start_time`` so that PID reuse after the
|
||||
target exits cannot later match the marker. Also records the
|
||||
replacer's PID and a UTC timestamp for TTL-based staleness checks.
|
||||
|
||||
Returns True on successful write, False on any failure. The caller
|
||||
should proceed with the SIGTERM even if the write fails (the marker
|
||||
is a best-effort signal, not a correctness requirement).
|
||||
"""
|
||||
try:
|
||||
target_start_time = _get_process_start_time(target_pid)
|
||||
record = {
|
||||
"target_pid": target_pid,
|
||||
"target_start_time": target_start_time,
|
||||
"replacer_pid": os.getpid(),
|
||||
"written_at": _utc_now_iso(),
|
||||
}
|
||||
_write_json_file(_get_takeover_marker_path(), record)
|
||||
return True
|
||||
except (OSError, PermissionError):
|
||||
return False
|
||||
|
||||
|
||||
def consume_takeover_marker_for_self() -> bool:
|
||||
"""Check & unlink the takeover marker if it names the current process.
|
||||
|
||||
Returns True only when a valid (non-stale) marker names this PID +
|
||||
start_time. A returning True indicates the current SIGTERM is a
|
||||
planned --replace takeover; the caller should exit 0 instead of
|
||||
signalling ``_signal_initiated_shutdown``.
|
||||
|
||||
Always unlinks the marker on match (and on detected staleness) so
|
||||
subsequent unrelated signals don't re-trigger.
|
||||
"""
|
||||
path = _get_takeover_marker_path()
|
||||
record = _read_json_file(path)
|
||||
if not record:
|
||||
return False
|
||||
|
||||
# Any malformed or stale marker → drop it and return False
|
||||
try:
|
||||
target_pid = int(record["target_pid"])
|
||||
target_start_time = record.get("target_start_time")
|
||||
written_at = record.get("written_at") or ""
|
||||
except (KeyError, TypeError, ValueError):
|
||||
try:
|
||||
path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
return False
|
||||
|
||||
# TTL guard: a stale marker older than _TAKEOVER_MARKER_TTL_S is ignored.
|
||||
stale = False
|
||||
try:
|
||||
written_dt = datetime.fromisoformat(written_at)
|
||||
age = (datetime.now(timezone.utc) - written_dt).total_seconds()
|
||||
if age > _TAKEOVER_MARKER_TTL_S:
|
||||
stale = True
|
||||
except (TypeError, ValueError):
|
||||
stale = True # Unparseable timestamp — treat as stale
|
||||
|
||||
if stale:
|
||||
try:
|
||||
path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
return False
|
||||
|
||||
# Does the marker name THIS process?
|
||||
our_pid = os.getpid()
|
||||
our_start_time = _get_process_start_time(our_pid)
|
||||
matches = (
|
||||
target_pid == our_pid
|
||||
and target_start_time is not None
|
||||
and our_start_time is not None
|
||||
and target_start_time == our_start_time
|
||||
)
|
||||
|
||||
# Consume the marker whether it matched or not — a marker that doesn't
|
||||
# match our identity is stale-for-us anyway.
|
||||
try:
|
||||
path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
return matches
|
||||
|
||||
|
||||
def clear_takeover_marker() -> None:
|
||||
"""Remove the takeover marker unconditionally. Safe to call repeatedly."""
|
||||
try:
|
||||
_get_takeover_marker_path().unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def get_running_pid(
|
||||
pid_path: Optional[Path] = None,
|
||||
*,
|
||||
cleanup_stale: bool = True,
|
||||
) -> Optional[int]:
|
||||
def get_running_pid() -> Optional[int]:
|
||||
"""Return the PID of a running gateway instance, or ``None``.
|
||||
|
||||
Checks the PID file and verifies the process is actually alive.
|
||||
Cleans up stale PID files automatically.
|
||||
"""
|
||||
resolved_pid_path = pid_path or _get_pid_path()
|
||||
record = _read_pid_record(resolved_pid_path)
|
||||
record = _read_pid_record()
|
||||
if not record:
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
try:
|
||||
pid = int(record["pid"])
|
||||
except (KeyError, TypeError, ValueError):
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
try:
|
||||
os.kill(pid, 0) # signal 0 = existence check, no actual signal sent
|
||||
except (ProcessLookupError, PermissionError):
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
recorded_start = record.get("start_time")
|
||||
current_start = _get_process_start_time(pid)
|
||||
if recorded_start is not None and current_start is not None and current_start != recorded_start:
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
if not _looks_like_gateway_process(pid):
|
||||
if not _record_looks_like_gateway(record):
|
||||
_cleanup_invalid_pid_path(resolved_pid_path, cleanup_stale=cleanup_stale)
|
||||
remove_pid_file()
|
||||
return None
|
||||
|
||||
return pid
|
||||
|
||||
|
||||
def is_gateway_running(
|
||||
pid_path: Optional[Path] = None,
|
||||
*,
|
||||
cleanup_stale: bool = True,
|
||||
) -> bool:
|
||||
def is_gateway_running() -> bool:
|
||||
"""Check if the gateway daemon is currently running."""
|
||||
return get_running_pid(pid_path, cleanup_stale=cleanup_stale) is not None
|
||||
return get_running_pid() is not None
|
||||
|
||||
+18
-234
@@ -43,7 +43,6 @@ class StreamConsumerConfig:
|
||||
edit_interval: float = 1.0
|
||||
buffer_threshold: int = 40
|
||||
cursor: str = " ▉"
|
||||
buffer_only: bool = False
|
||||
|
||||
|
||||
class GatewayStreamConsumer:
|
||||
@@ -65,18 +64,6 @@ class GatewayStreamConsumer:
|
||||
# progressive edits for the remainder of the stream.
|
||||
_MAX_FLOOD_STRIKES = 3
|
||||
|
||||
# Reasoning/thinking tags that models emit inline in content.
|
||||
# Must stay in sync with cli.py _OPEN_TAGS/_CLOSE_TAGS and
|
||||
# run_agent.py _strip_think_blocks() tag variants.
|
||||
_OPEN_THINK_TAGS = (
|
||||
"<REASONING_SCRATCHPAD>", "<think>", "<reasoning>",
|
||||
"<THINKING>", "<thinking>", "<thought>",
|
||||
)
|
||||
_CLOSE_THINK_TAGS = (
|
||||
"</REASONING_SCRATCHPAD>", "</think>", "</reasoning>",
|
||||
"</THINKING>", "</thinking>", "</thought>",
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
adapter: Any,
|
||||
@@ -100,18 +87,6 @@ class GatewayStreamConsumer:
|
||||
self._flood_strikes = 0 # Consecutive flood-control edit failures
|
||||
self._current_edit_interval = self.cfg.edit_interval # Adaptive backoff
|
||||
self._final_response_sent = False
|
||||
# Cache adapter lifecycle capability: only platforms that need an
|
||||
# explicit finalize call (e.g. DingTalk AI Cards) force us to make
|
||||
# a redundant final edit. Everyone else keeps the fast path.
|
||||
# Use ``is True`` (not ``bool(...)``) so MagicMock attribute access
|
||||
# in tests doesn't incorrectly enable this path.
|
||||
self._adapter_requires_finalize: bool = (
|
||||
getattr(adapter, "REQUIRES_EDIT_FINALIZE", False) is True
|
||||
)
|
||||
|
||||
# Think-block filter state (mirrors CLI's _stream_delta tag suppression)
|
||||
self._in_think_block = False
|
||||
self._think_buffer = ""
|
||||
|
||||
@property
|
||||
def already_sent(self) -> bool:
|
||||
@@ -157,112 +132,6 @@ class GatewayStreamConsumer:
|
||||
"""Signal that the stream is complete."""
|
||||
self._queue.put(_DONE)
|
||||
|
||||
# ── Think-block filtering ────────────────────────────────────────
|
||||
# Models like MiniMax emit inline <think>...</think> blocks in their
|
||||
# content. The CLI's _stream_delta suppresses these via a state
|
||||
# machine; we do the same here so gateway users never see raw
|
||||
# reasoning tags. The agent also strips them from the final
|
||||
# response (run_agent.py _strip_think_blocks), but the stream
|
||||
# consumer sends intermediate edits before that stripping happens.
|
||||
|
||||
def _filter_and_accumulate(self, text: str) -> None:
|
||||
"""Add a text delta to the accumulated buffer, suppressing think blocks.
|
||||
|
||||
Uses a state machine that tracks whether we are inside a
|
||||
reasoning/thinking block. Text inside such blocks is silently
|
||||
discarded. Partial tags at buffer boundaries are held back in
|
||||
``_think_buffer`` until enough characters arrive to decide.
|
||||
"""
|
||||
buf = self._think_buffer + text
|
||||
self._think_buffer = ""
|
||||
|
||||
while buf:
|
||||
if self._in_think_block:
|
||||
# Look for the earliest closing tag
|
||||
best_idx = -1
|
||||
best_len = 0
|
||||
for tag in self._CLOSE_THINK_TAGS:
|
||||
idx = buf.find(tag)
|
||||
if idx != -1 and (best_idx == -1 or idx < best_idx):
|
||||
best_idx = idx
|
||||
best_len = len(tag)
|
||||
|
||||
if best_len:
|
||||
# Found closing tag — discard block, process remainder
|
||||
self._in_think_block = False
|
||||
buf = buf[best_idx + best_len:]
|
||||
else:
|
||||
# No closing tag yet — hold tail that could be a
|
||||
# partial closing tag prefix, discard the rest.
|
||||
max_tag = max(len(t) for t in self._CLOSE_THINK_TAGS)
|
||||
self._think_buffer = buf[-max_tag:] if len(buf) > max_tag else buf
|
||||
return
|
||||
else:
|
||||
# Look for earliest opening tag at a block boundary
|
||||
# (start of text / preceded by newline + optional whitespace).
|
||||
# This prevents false positives when models *mention* tags
|
||||
# in prose (e.g. "the <think> tag is used for…").
|
||||
best_idx = -1
|
||||
best_len = 0
|
||||
for tag in self._OPEN_THINK_TAGS:
|
||||
search_start = 0
|
||||
while True:
|
||||
idx = buf.find(tag, search_start)
|
||||
if idx == -1:
|
||||
break
|
||||
# Block-boundary check (mirrors cli.py logic)
|
||||
if idx == 0:
|
||||
is_boundary = (
|
||||
not self._accumulated
|
||||
or self._accumulated.endswith("\n")
|
||||
)
|
||||
else:
|
||||
preceding = buf[:idx]
|
||||
last_nl = preceding.rfind("\n")
|
||||
if last_nl == -1:
|
||||
is_boundary = (
|
||||
(not self._accumulated
|
||||
or self._accumulated.endswith("\n"))
|
||||
and preceding.strip() == ""
|
||||
)
|
||||
else:
|
||||
is_boundary = preceding[last_nl + 1:].strip() == ""
|
||||
|
||||
if is_boundary and (best_idx == -1 or idx < best_idx):
|
||||
best_idx = idx
|
||||
best_len = len(tag)
|
||||
break # first boundary hit for this tag is enough
|
||||
search_start = idx + 1
|
||||
|
||||
if best_len:
|
||||
# Emit text before the tag, enter think block
|
||||
self._accumulated += buf[:best_idx]
|
||||
self._in_think_block = True
|
||||
buf = buf[best_idx + best_len:]
|
||||
else:
|
||||
# No opening tag — check for a partial tag at the tail
|
||||
held_back = 0
|
||||
for tag in self._OPEN_THINK_TAGS:
|
||||
for i in range(1, len(tag)):
|
||||
if buf.endswith(tag[:i]) and i > held_back:
|
||||
held_back = i
|
||||
if held_back:
|
||||
self._accumulated += buf[:-held_back]
|
||||
self._think_buffer = buf[-held_back:]
|
||||
else:
|
||||
self._accumulated += buf
|
||||
return
|
||||
|
||||
def _flush_think_buffer(self) -> None:
|
||||
"""Flush any held-back partial-tag buffer into accumulated text.
|
||||
|
||||
Called when the stream ends (got_done) so that partial text that
|
||||
was held back waiting for a possible opening tag is not lost.
|
||||
"""
|
||||
if self._think_buffer and not self._in_think_block:
|
||||
self._accumulated += self._think_buffer
|
||||
self._think_buffer = ""
|
||||
|
||||
async def run(self) -> None:
|
||||
"""Async task that drains the queue and edits the platform message."""
|
||||
# Platform message length limit — leave room for cursor + formatting
|
||||
@@ -287,16 +156,10 @@ class GatewayStreamConsumer:
|
||||
if isinstance(item, tuple) and len(item) == 2 and item[0] is _COMMENTARY:
|
||||
commentary_text = item[1]
|
||||
break
|
||||
self._filter_and_accumulate(item)
|
||||
self._accumulated += item
|
||||
except queue.Empty:
|
||||
break
|
||||
|
||||
# Flush any held-back partial-tag buffer on stream end
|
||||
# so trailing text that was waiting for a potential open
|
||||
# tag is not lost.
|
||||
if got_done:
|
||||
self._flush_think_buffer()
|
||||
|
||||
# Decide whether to flush an edit
|
||||
now = time.monotonic()
|
||||
elapsed = now - self._last_edit_time
|
||||
@@ -304,13 +167,10 @@ class GatewayStreamConsumer:
|
||||
got_done
|
||||
or got_segment_break
|
||||
or commentary_text is not None
|
||||
or (elapsed >= self._current_edit_interval
|
||||
and self._accumulated)
|
||||
or len(self._accumulated) >= self.cfg.buffer_threshold
|
||||
)
|
||||
if not self.cfg.buffer_only:
|
||||
should_edit = should_edit or (
|
||||
(elapsed >= self._current_edit_interval
|
||||
and self._accumulated)
|
||||
or len(self._accumulated) >= self.cfg.buffer_threshold
|
||||
)
|
||||
|
||||
current_update_visible = False
|
||||
if should_edit and self._accumulated:
|
||||
@@ -369,16 +229,7 @@ class GatewayStreamConsumer:
|
||||
if not got_done and not got_segment_break and commentary_text is None:
|
||||
display_text += self.cfg.cursor
|
||||
|
||||
# Segment break: finalize the current message so platforms
|
||||
# that need explicit closure (e.g. DingTalk AI Cards) don't
|
||||
# leave the previous segment stuck in a loading state when
|
||||
# the next segment (tool progress, next chunk) creates a
|
||||
# new message below it. got_done has its own finalize
|
||||
# path below so we don't finalize here for it.
|
||||
current_update_visible = await self._send_or_edit(
|
||||
display_text,
|
||||
finalize=got_segment_break,
|
||||
)
|
||||
current_update_visible = await self._send_or_edit(display_text)
|
||||
self._last_edit_time = time.monotonic()
|
||||
|
||||
if got_done:
|
||||
@@ -389,22 +240,10 @@ class GatewayStreamConsumer:
|
||||
if self._accumulated:
|
||||
if self._fallback_final_send:
|
||||
await self._send_fallback_final(self._accumulated)
|
||||
elif (
|
||||
current_update_visible
|
||||
and not self._adapter_requires_finalize
|
||||
):
|
||||
# Mid-stream edit above already delivered the
|
||||
# final accumulated content. Skip the redundant
|
||||
# final edit — but only for adapters that don't
|
||||
# need an explicit finalize signal.
|
||||
elif current_update_visible:
|
||||
self._final_response_sent = True
|
||||
elif self._message_id:
|
||||
# Either the mid-stream edit didn't run (no
|
||||
# visible update this tick) OR the adapter needs
|
||||
# explicit finalize=True to close the stream.
|
||||
self._final_response_sent = await self._send_or_edit(
|
||||
self._accumulated, finalize=True,
|
||||
)
|
||||
self._final_response_sent = await self._send_or_edit(self._accumulated)
|
||||
elif not self._already_sent:
|
||||
self._final_response_sent = await self._send_or_edit(self._accumulated)
|
||||
return
|
||||
@@ -436,21 +275,11 @@ class GatewayStreamConsumer:
|
||||
|
||||
except asyncio.CancelledError:
|
||||
# Best-effort final edit on cancellation
|
||||
_best_effort_ok = False
|
||||
if self._accumulated and self._message_id:
|
||||
try:
|
||||
_best_effort_ok = bool(await self._send_or_edit(self._accumulated))
|
||||
await self._send_or_edit(self._accumulated)
|
||||
except Exception:
|
||||
pass
|
||||
# Only confirm final delivery if the best-effort send above
|
||||
# actually succeeded OR if the final response was already
|
||||
# confirmed before we were cancelled. Previously this
|
||||
# promoted any partial send (already_sent=True) to
|
||||
# final_response_sent — which suppressed the gateway's
|
||||
# fallback send even when only intermediate text (e.g.
|
||||
# "Let me search…") had been delivered, not the real answer.
|
||||
if _best_effort_ok and not self._final_response_sent:
|
||||
self._final_response_sent = True
|
||||
except Exception as e:
|
||||
logger.error("Stream consumer error: %s", e)
|
||||
|
||||
@@ -548,17 +377,9 @@ class GatewayStreamConsumer:
|
||||
self._fallback_final_send = False
|
||||
if not continuation.strip():
|
||||
# Nothing new to send — the visible partial already matches final text.
|
||||
# BUT: if final_text itself has meaningful content (e.g. a timeout
|
||||
# message after a long tool call), the prefix-based continuation
|
||||
# calculation may wrongly conclude "already shown" because the
|
||||
# streamed prefix was from a *previous* segment (before the tool
|
||||
# boundary). In that case, send the full final_text as-is (#10807).
|
||||
if final_text.strip() and final_text != self._visible_prefix():
|
||||
continuation = final_text
|
||||
else:
|
||||
self._already_sent = True
|
||||
self._final_response_sent = True
|
||||
return
|
||||
self._already_sent = True
|
||||
self._final_response_sent = True
|
||||
return
|
||||
|
||||
raw_limit = getattr(self.adapter, "MAX_MESSAGE_LENGTH", 4096)
|
||||
safe_limit = max(500, raw_limit - 100)
|
||||
@@ -652,74 +473,37 @@ class GatewayStreamConsumer:
|
||||
content=text,
|
||||
metadata=self.metadata,
|
||||
)
|
||||
# Note: do NOT set _already_sent = True here.
|
||||
# Commentary messages are interim status updates (e.g. "Using browser
|
||||
# tool..."), not the final response. Setting already_sent would cause
|
||||
# the final response to be incorrectly suppressed when there are
|
||||
# multiple tool calls. See: https://github.com/NousResearch/hermes-agent/issues/10454
|
||||
return result.success
|
||||
if result.success:
|
||||
self._already_sent = True
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error("Commentary send error: %s", e)
|
||||
return False
|
||||
return False
|
||||
|
||||
async def _send_or_edit(self, text: str, *, finalize: bool = False) -> bool:
|
||||
async def _send_or_edit(self, text: str) -> bool:
|
||||
"""Send or edit the streaming message.
|
||||
|
||||
Returns True if the text was successfully delivered (sent or edited),
|
||||
False otherwise. Callers like the overflow split loop use this to
|
||||
decide whether to advance past the delivered chunk.
|
||||
|
||||
``finalize`` is True when this is the last edit in a streaming
|
||||
sequence.
|
||||
"""
|
||||
# Strip MEDIA: directives so they don't appear as visible text.
|
||||
# Media files are delivered as native attachments after the stream
|
||||
# finishes (via _deliver_media_from_response in gateway/run.py).
|
||||
text = self._clean_for_display(text)
|
||||
# A bare streaming cursor is not meaningful user-visible content and
|
||||
# can render as a stray tofu/white-box message on some clients.
|
||||
visible_without_cursor = text
|
||||
if self.cfg.cursor:
|
||||
visible_without_cursor = visible_without_cursor.replace(self.cfg.cursor, "")
|
||||
_visible_stripped = visible_without_cursor.strip()
|
||||
if not _visible_stripped:
|
||||
return True # cursor-only / whitespace-only update
|
||||
if not text.strip():
|
||||
return True # nothing to send is "success"
|
||||
# Guard: do not create a brand-new standalone message when the only
|
||||
# visible content is a handful of characters alongside the streaming
|
||||
# cursor. During rapid tool-calling the model often emits 1-2 tokens
|
||||
# before switching to tool calls; the resulting "X ▉" message risks
|
||||
# leaving the cursor permanently visible if the follow-up edit (to
|
||||
# strip the cursor on segment break) is rate-limited by the platform.
|
||||
# This was reported on Telegram, Matrix, and other clients where the
|
||||
# ▉ block character renders as a visible white box ("tofu").
|
||||
# Existing messages (edits) are unaffected — only first sends gated.
|
||||
_MIN_NEW_MSG_CHARS = 4
|
||||
if (self._message_id is None
|
||||
and self.cfg.cursor
|
||||
and self.cfg.cursor in text
|
||||
and len(_visible_stripped) < _MIN_NEW_MSG_CHARS):
|
||||
return True # too short for a standalone message — accumulate more
|
||||
try:
|
||||
if self._message_id is not None:
|
||||
if self._edit_supported:
|
||||
# Skip if text is identical to what we last sent.
|
||||
# Exception: adapters that require an explicit finalize
|
||||
# call (REQUIRES_EDIT_FINALIZE) must still receive the
|
||||
# finalize=True edit even when content is unchanged, so
|
||||
# their streaming UI can transition out of the in-
|
||||
# progress state. Everyone else short-circuits.
|
||||
if text == self._last_sent_text and not (
|
||||
finalize and self._adapter_requires_finalize
|
||||
):
|
||||
# Skip if text is identical to what we last sent
|
||||
if text == self._last_sent_text:
|
||||
return True
|
||||
# Edit existing message
|
||||
result = await self.adapter.edit_message(
|
||||
chat_id=self.chat_id,
|
||||
message_id=self._message_id,
|
||||
content=text,
|
||||
finalize=finalize,
|
||||
)
|
||||
if result.success:
|
||||
self._already_sent = True
|
||||
|
||||
@@ -1,160 +0,0 @@
|
||||
# Hermes Agent Has Had "Routines" Since March
|
||||
|
||||
Anthropic just announced [Claude Code Routines](https://claude.com/blog/introducing-routines-in-claude-code) — scheduled tasks, GitHub event triggers, and API-triggered agent runs. Bundled prompt + repo + connectors, running on their infrastructure.
|
||||
|
||||
It's a good feature. We shipped it two months ago.
|
||||
|
||||
---
|
||||
|
||||
## The Three Trigger Types — Side by Side
|
||||
|
||||
Claude Code Routines offers three ways to trigger an automation:
|
||||
|
||||
**1. Scheduled (cron)**
|
||||
> "Every night at 2am: pull the top bug from Linear, attempt a fix, and open a draft PR."
|
||||
|
||||
Hermes equivalent — works today:
|
||||
```bash
|
||||
hermes cron create "0 2 * * *" \
|
||||
"Pull the top bug from the issue tracker, attempt a fix, and open a draft PR." \
|
||||
--name "Nightly bug fix" \
|
||||
--deliver telegram
|
||||
```
|
||||
|
||||
**2. GitHub Events (webhook)**
|
||||
> "Flag PRs that touch the /auth-provider module and post to #auth-changes."
|
||||
|
||||
Hermes equivalent — works today:
|
||||
```bash
|
||||
hermes webhook subscribe auth-watch \
|
||||
--events "pull_request" \
|
||||
--prompt "PR #{pull_request.number}: {pull_request.title} by {pull_request.user.login}. Check if it touches the auth-provider module. If yes, summarize the changes." \
|
||||
--deliver slack
|
||||
```
|
||||
|
||||
**3. API Triggers**
|
||||
> "Read the alert payload, find the owning service, post a triage summary to #oncall."
|
||||
|
||||
Hermes equivalent — works today:
|
||||
```bash
|
||||
hermes webhook subscribe alert-triage \
|
||||
--prompt "Alert: {alert.name} — Severity: {alert.severity}. Find the owning service, investigate, and post a triage summary with proposed first steps." \
|
||||
--deliver slack
|
||||
```
|
||||
|
||||
Every use case in their blog post — backlog triage, docs drift, deploy verification, alert correlation, library porting, bespoke PR review — has a working Hermes implementation. No new features needed. It's been shipping since March 2026.
|
||||
|
||||
---
|
||||
|
||||
## What's Different
|
||||
|
||||
| | Claude Code Routines | Hermes Agent |
|
||||
|---|---|---|
|
||||
| **Scheduled tasks** | ✅ Schedule-based | ✅ Any cron expression + human-readable intervals |
|
||||
| **GitHub triggers** | ✅ PR, issue, push events | ✅ Any GitHub event via webhook subscriptions |
|
||||
| **API triggers** | ✅ POST to unique endpoint | ✅ POST to webhook routes with HMAC auth |
|
||||
| **MCP connectors** | ✅ Native connectors | ✅ Full MCP client support |
|
||||
| **Script pre-processing** | ❌ | ✅ Python scripts run before agent, inject context |
|
||||
| **Skill chaining** | ❌ | ✅ Load multiple skills per automation |
|
||||
| **Daily limit** | 5-25 runs/day | **Unlimited** |
|
||||
| **Model choice** | Claude only | **Any model** — Claude, GPT, Gemini, DeepSeek, Qwen, local |
|
||||
| **Delivery targets** | GitHub comments | Telegram, Discord, Slack, SMS, email, GitHub comments, webhooks, local files |
|
||||
| **Infrastructure** | Anthropic's servers | **Your infrastructure** — VPS, home server, laptop |
|
||||
| **Data residency** | Anthropic's cloud | **Your machines** |
|
||||
| **Cost** | Pro/Max/Team/Enterprise subscription | Your API key, your rates |
|
||||
| **Open source** | No | **Yes** — MIT license |
|
||||
|
||||
---
|
||||
|
||||
## Things Hermes Does That Routines Can't
|
||||
|
||||
### Script Injection
|
||||
|
||||
Run a Python script *before* the agent. The script's stdout becomes context. The script handles mechanical work (fetching, diffing, computing); the agent handles reasoning.
|
||||
|
||||
```bash
|
||||
hermes cron create "every 1h" \
|
||||
"If CHANGE DETECTED, summarize what changed. If NO_CHANGE, respond with [SILENT]." \
|
||||
--script ~/.hermes/scripts/watch-site.py \
|
||||
--name "Pricing monitor" \
|
||||
--deliver telegram
|
||||
```
|
||||
|
||||
The `[SILENT]` pattern means you only get notified when something actually happens. No spam.
|
||||
|
||||
### Multi-Skill Workflows
|
||||
|
||||
Chain specialized skills together. Each skill teaches the agent a specific capability, and the prompt ties them together.
|
||||
|
||||
```bash
|
||||
hermes cron create "0 8 * * *" \
|
||||
"Search arXiv for papers on language model reasoning. Save the top 3 as Obsidian notes." \
|
||||
--skills "arxiv,obsidian" \
|
||||
--name "Paper digest"
|
||||
```
|
||||
|
||||
### Deliver Anywhere
|
||||
|
||||
One automation, any destination:
|
||||
|
||||
```bash
|
||||
--deliver telegram # Telegram home channel
|
||||
--deliver discord # Discord home channel
|
||||
--deliver slack # Slack channel
|
||||
--deliver sms:+15551234567 # Text message
|
||||
--deliver telegram:-1001234567890:42 # Specific Telegram forum topic
|
||||
--deliver local # Save to file, no notification
|
||||
```
|
||||
|
||||
### Model-Agnostic
|
||||
|
||||
Your nightly triage can run on Claude. Your deploy verification can run on GPT. Your cost-sensitive monitors can run on DeepSeek or a local model. Same automation system, any backend.
|
||||
|
||||
---
|
||||
|
||||
## The Limits Tell the Story
|
||||
|
||||
Claude Code Routines: **5 routines per day** on Pro. **25 on Enterprise.** That's their ceiling.
|
||||
|
||||
Hermes has no daily limit. Run 500 automations a day if you want. The only constraint is your API budget, and you choose which models to use for which tasks.
|
||||
|
||||
A nightly backlog triage on Sonnet costs roughly $0.02-0.05. A monitoring check on DeepSeek costs fractions of a cent. You control the economics.
|
||||
|
||||
---
|
||||
|
||||
## Get Started
|
||||
|
||||
Hermes Agent is open source and free. The automation infrastructure — cron scheduler, webhook platform, skill system, multi-platform delivery — is built in.
|
||||
|
||||
```bash
|
||||
pip install hermes-agent
|
||||
hermes setup
|
||||
```
|
||||
|
||||
Set up a scheduled task in 30 seconds:
|
||||
```bash
|
||||
hermes cron create "0 9 * * 1" \
|
||||
"Generate a weekly AI news digest. Search the web for major announcements, trending repos, and notable papers. Keep it under 500 words with links." \
|
||||
--name "Weekly digest" \
|
||||
--deliver telegram
|
||||
```
|
||||
|
||||
Set up a GitHub webhook in 60 seconds:
|
||||
```bash
|
||||
hermes gateway setup # enable webhooks
|
||||
hermes webhook subscribe pr-review \
|
||||
--events "pull_request" \
|
||||
--prompt "Review PR #{pull_request.number}: {pull_request.title}" \
|
||||
--skills "github-code-review" \
|
||||
--deliver github_comment
|
||||
```
|
||||
|
||||
Full automation templates gallery: [hermes-agent.nousresearch.com/docs/guides/automation-templates](https://hermes-agent.nousresearch.com/docs/guides/automation-templates)
|
||||
|
||||
Documentation: [hermes-agent.nousresearch.com](https://hermes-agent.nousresearch.com)
|
||||
|
||||
GitHub: [github.com/NousResearch/hermes-agent](https://github.com/NousResearch/hermes-agent)
|
||||
|
||||
---
|
||||
|
||||
*Hermes Agent is built by [Nous Research](https://nousresearch.com). Open source, model-agnostic, runs on your infrastructure.*
|
||||
@@ -11,5 +11,5 @@ Provides subcommands for:
|
||||
- hermes cron - Manage cron jobs
|
||||
"""
|
||||
|
||||
__version__ = "0.10.0"
|
||||
__release_date__ = "2026.4.16"
|
||||
__version__ = "0.8.0"
|
||||
__release_date__ = "2026.4.8"
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user