Adding finalized endless terminal

- pyproject.toml: Added tinker SDK, torch, wandb, math-verify to [atropos] extras
- README.md: Added comprehensive RL Training with Tinker section including:
  - Architecture diagram (3-process pipeline)
  - Quick start guide for GSM8k agent training
  - Configuration documentation
  - RL CLI usage
  - Sandbox backend options (Nomad, Singularity, Modal)

New files in tinker-atropos submodule (committed there):
- tinker_atropos/environments/gsm8k_agent.py: Agent GSM8k env with Python REPL tool
- configs/gsm8k_agent.yaml: Config for Qwen3-4B training

.clinerules

@@ -0,0 +1,115 @@
+# Cline's Memory Bank
+
+I am Cline, an expert software engineer with a unique characteristic: my memory resets completely between sessions. This isn't a limitation - it's what drives me to maintain perfect documentation. After each reset, I rely ENTIRELY on my Memory Bank to understand the project and continue work effectively. I MUST read ALL memory bank files at the start of EVERY task - this is not optional.
+
+## Memory Bank Structure
+
+The Memory Bank consists of core files and optional context files, all in Markdown format. Files build upon each other in a clear hierarchy:
+
+flowchart TD
+    PB[projectbrief.md] --> PC[productContext.md]
+    PB --> SP[systemPatterns.md]
+    PB --> TC[techContext.md]
+
+    PC --> AC[activeContext.md]
+    SP --> AC
+    TC --> AC
+
+    AC --> P[progress.md]
+
+### Core Files (Required)
+1. `projectbrief.md`
+   - Foundation document that shapes all other files
+   - Created at project start if it doesn't exist
+   - Defines core requirements and goals
+   - Source of truth for project scope
+
+2. `productContext.md`
+   - Why this project exists
+   - Problems it solves
+   - How it should work
+   - User experience goals
+
+3. `activeContext.md`
+   - Current work focus
+   - Recent changes
+   - Next steps
+   - Active decisions and considerations
+   - Important patterns and preferences
+   - Learnings and project insights
+
+4. `systemPatterns.md`
+   - System architecture
+   - Key technical decisions
+   - Design patterns in use
+   - Component relationships
+   - Critical implementation paths
+
+5. `techContext.md`
+   - Technologies used
+   - Development setup
+   - Technical constraints
+   - Dependencies
+   - Tool usage patterns
+
+6. `progress.md`
+   - What works
+   - What's left to build
+   - Current status
+   - Known issues
+   - Evolution of project decisions
+
+### Additional Context
+Create additional files/folders within memory-bank/ when they help organize:
+- Complex feature documentation
+- Integration specifications
+- API documentation
+- Testing strategies
+- Deployment procedures
+
+## Core Workflows
+
+### Plan Mode
+flowchart TD
+    Start[Start] --> ReadFiles[Read Memory Bank]
+    ReadFiles --> CheckFiles{Files Complete?}
+
+    CheckFiles -->|No| Plan[Create Plan]
+    Plan --> Document[Document in Chat]
+
+    CheckFiles -->|Yes| Verify[Verify Context]
+    Verify --> Strategy[Develop Strategy]
+    Strategy --> Present[Present Approach]
+
+### Act Mode
+flowchart TD
+    Start[Start] --> Context[Check Memory Bank]
+    Context --> Update[Update Documentation]
+    Update --> Execute[Execute Task]
+    Execute --> Document[Document Changes]
+
+## Documentation Updates
+
+Memory Bank updates occur when:
+1. Discovering new project patterns
+2. After implementing significant changes
+3. When user requests with **update memory bank** (MUST review ALL files)
+4. When context needs clarification
+
+flowchart TD
+    Start[Update Process]
+
+    subgraph Process
+        P1[Review ALL Files]
+        P2[Document Current State]
+        P3[Clarify Next Steps]
+        P4[Document Insights & Patterns]
+
+        P1 --> P2 --> P3 --> P4
+    end
+
+    Start --> Process
+
+Note: When triggered by **update memory bank**, I MUST review every memory bank file, even if some don't require updates. Focus particularly on activeContext.md and progress.md as they track current state.
+
+REMEMBER: After every memory reset, I begin completely fresh. The Memory Bank is my only link to previous work. It must be maintained with precision and clarity, as my effectiveness depends entirely on its accuracy.

.cursorrules

@@ -1,23 +0,0 @@
-Hermes-Agent is an agent harness for LLMs.
-
-When building, the tool functionality is in the tools/ directory, where each specific tool (or in some cases, tools that are built for the same execution category or api) are placed in a script each their own.
-
-Each tool is then consolidated in the model_tools.py file in the repo root.
-
-There is also a way to consolidate sets of tools in toolsets.py for the agent to use.
-
-The primary agent runner code is in run_agent, but other runners could be developed using the tools and framework.
-
-Always ensure consistency between tools, the model_tools.py and toolsets.py when changing any of them, otherwise they could become desynced in a way that is detrimental to functionality.
-
-The expected pathway for using API keys is to setup and place them in a .env file in the repo root.
-
-Test scripts will be placed in tests/
-
-The run_agent loop is setup to:
-- Process the enabled toolsets to provide to the model,
-- Pipe in a prompt or problem from the input to the agent,
-- Loop the LLM each time it calls a tool, until the model decides no more tools are needed and provides a natural language response,
-- Return that response.
-
-There are additional caveats for logging, where we restructure the "tools" as a system prompt for storage later into a format that can be used and handled properly later.

.env.example

@@ -1,14 +1,77 @@
 # Hermes Agent Environment Configuration
 # Copy this file to .env and fill in your API keys
-# Get API keys from the URLs listed below
 
 # =============================================================================
-# REQUIRED API KEYS
+# CORE SETTINGS
 # =============================================================================
+# Agent backend:
+# - openai  : default Hermes-Agent loop (OpenAI function-calling via OpenAI SDK)
+# - atropos : Atroposlib ServerManager/ManagedServer-backed loop (training/env integration)
+HERMES_BACKEND=openai
 
-# Anthropic API Key - Main agent model
-# Get at: https://console.anthropic.com/
-ANTHROPIC_API_KEY=
+
+# =============================================================================
+# LOCAL / SELF-HOSTED OPENAI-COMPATIBLE ENDPOINTS (vLLM, SGLang, llama.cpp, etc.)
+# =============================================================================
+# For local development (matches the Atropos test env defaults):
+# ATROPOS_SERVER_BASE_URL=http://127.0.0.1:8080
+# ATROPOS_SERVER_MODEL=hermes-4-36b
+# For hosted inference (Nous Research inference API):
+ATROPOS_SERVER_BASE_URL=
+ATROPOS_SERVER_MODEL=
+ATROPOS_TOKENIZER_NAME=
+# Set this to your Nous API key (Bearer token).
+ATROPOS_SERVER_API_KEY=
+
+# Debugging (prints to stdout; use with care)
+# HERMES_DEBUG_ATROPOS_REQUEST=1
+# HERMES_DEBUG_ATROPOS_RESPONSE=1
+# HERMES_DEBUG_OPENAI_REQUEST=1
+# HERMES_DEBUG_OPENAI_RESPONSE=1
+
+# =============================================================================
+# LOCAL / SELF-HOSTED OPENAI-COMPATIBLE ENDPOINTS (vLLM, SGLang, llama.cpp, etc.)
+# =============================================================================
+# If you set ATROPOS_SERVER_BASE_URL or OPENAI_BASE_URL, Hermes will use it instead
+# of OpenRouter.
+#
+# Local server convenience (base URL without /v1):
+# llama.cpp example (see `Hermes-Agent/scripts/launch_llama_cpp_hermes_4_36b.sh`):
+# ATROPOS_SERVER_BASE_URL=http://127.0.0.1:8080
+# ATROPOS_SERVER_MODEL=hermes-4-36b
+# ATROPOS_TOKENIZER_NAME=NousResearch/Hermes-4.3-36B
+# ATROPOS_SERVER_API_KEY=local
+#
+# Hosted Nous inference API:
+# ATROPOS_SERVER_BASE_URL=https://inference-api.nousresearch.com
+# ATROPOS_SERVER_MODEL=Hermes-4.3-36B
+# ATROPOS_TOKENIZER_NAME=NousResearch/Hermes-4.3-36B
+# ATROPOS_SERVER_API_KEY=sk-... (Bearer token)
+#
+# If you plan to run GRPO-style group sampling (e.g. `--env.group_size 4`) against
+# llama.cpp, start the server with at least that many slots, e.g.:
+#   LLAMA_CPP_PARALLEL=4 Hermes-Agent/scripts/launch_llama_cpp_hermes_4_36b.sh
+#
+# Generic OpenAI-compatible (base URL should include /v1):
+# OPENAI_BASE_URL=http://127.0.0.1:8080/v1
+# OPENAI_API_KEY=local
+
+# =============================================================================
+# LLM PROVIDER (OpenRouter)
+# =============================================================================
+# OpenRouter provides access to many models through one API
+# All LLM calls go through OpenRouter - no direct provider keys needed
+# Get your key at: https://openrouter.ai/keys
+OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
+OPENROUTER_API_KEY=
+
+# Default model to use (OpenRouter format: provider/model)
+# Examples: anthropic/claude-opus-4.6, openai/gpt-4o, google/gemini-2.0-flash, zhipuai/glm-4-plus
+LLM_MODEL=anthropic/claude-opus-4.6
+
+# =============================================================================
+# TOOL API KEYS
+# =============================================================================
 
 # Firecrawl API Key - Web search, extract, and crawl
 # Get at: https://firecrawl.dev/
@@ -18,32 +81,239 @@ FIRECRAWL_API_KEY=
 # Get at: https://inference-api.nousresearch.com/
 NOUS_API_KEY=
 
-# Morph API Key - Terminal/command execution tools
-# Get at: https://morph.so/
-MORPH_API_KEY=
-
 # FAL.ai API Key - Image generation
 # Get at: https://fal.ai/
 FAL_KEY=
 
 # =============================================================================
-# OPTIONAL API KEYS
+# TERMINAL TOOL CONFIGURATION (mini-swe-agent backend)
 # =============================================================================
+# Backend type: "local", "singularity", "docker", "modal", or "ssh"
+# - local: Runs directly on your machine (fastest, no isolation)
+# - ssh: Runs on remote server via SSH (great for sandboxing - agent can't touch its own code)
+# - singularity: Runs in Apptainer/Singularity containers (HPC clusters, no root needed)
+# - docker: Runs in Docker containers (isolated, requires Docker + docker group)
+# - modal: Runs in Modal cloud sandboxes (scalable, requires Modal account)
+TERMINAL_ENV=local
 
-# OpenAI API Key - Optional, for enhanced Hecate features
-# Get at: https://platform.openai.com/
-OPENAI_API_KEY=
+
+# Container images (for singularity/docker/modal backends)
+TERMINAL_DOCKER_IMAGE=python:3.11
+TERMINAL_SINGULARITY_IMAGE=docker://python:3.11
+TERMINAL_MODAL_IMAGE=python:3.11
+
+# Working directory for terminal commands
+# For CLI: "." means current directory (resolved automatically from config.yaml)
+# For containers (docker/singularity/modal): absolute path inside the container
+# Usually managed by config.yaml (terminal.cwd) — uncomment to override
+# TERMINAL_CWD=.
+
+# Default command timeout in seconds
+TERMINAL_TIMEOUT=60
+
+# Cleanup inactive environments after this many seconds
+TERMINAL_LIFETIME_SECONDS=300
 
 # =============================================================================
-# OPTIONAL CONFIGURATION
+# SSH REMOTE EXECUTION (for TERMINAL_ENV=ssh)
+# =============================================================================
+# Run terminal commands on a remote server via SSH.
+# Agent code stays on your machine, commands execute remotely.
+#
+# SECURITY BENEFITS:
+# - Agent cannot read your .env file (API keys protected)
+# - Agent cannot modify its own code
+# - Remote server acts as isolated sandbox
+# - Can safely configure passwordless sudo on remote
+#
+# TERMINAL_SSH_HOST=192.168.1.100
+# TERMINAL_SSH_USER=agent
+# TERMINAL_SSH_PORT=22
+# TERMINAL_SSH_KEY=~/.ssh/id_rsa
+
+# =============================================================================
+# SUDO SUPPORT (works with ALL terminal backends)
+# =============================================================================
+# If set, enables sudo commands by piping password via `sudo -S`.
+# Works with: local, docker, singularity, modal, and ssh backends.
+# 
+# SECURITY WARNING: Password stored in plaintext. Only use on trusted machines.
+# 
+# ALTERNATIVES:
+# - For SSH backend: Configure passwordless sudo on the remote server
+# - For containers: Run as root inside the container (no sudo needed)
+# - For local: Configure /etc/sudoers for specific commands
+# - For CLI: Leave unset - you'll be prompted interactively with 45s timeout
+#
+# SUDO_PASSWORD=your_password_here
+
+# =============================================================================
+# MODAL CLOUD BACKEND (for TERMINAL_ENV=modal)
+# =============================================================================
+# Modal provides cloud sandboxes with per-second billing and auto-scaling.
+# This implementation uses a warm pool of sandboxes for cost efficiency.
+#
+# SETUP:
+#   pip install modal && modal setup
+#   (Authenticates via browser, stores credentials locally)
+#
+# FEATURES:
+# - Auto-scaling warm sandbox pool (no cold start after first use)
+# - Named sandbox recovery (reconnects after restart)
+# - Profile-based heterogeneous environments (CPU, GPU, different images)
+# - Server-side idle_timeout protection against orphaned sandboxes
+
+# Modal app name (groups all sandboxes, used for recovery)
+TERMINAL_MODAL_APP_NAME=hermes-sandbox
+
+# Default profile when none specified
+TERMINAL_MODAL_DEFAULT_PROFILE=default
+
+# Profile config file (optional - YAML format, see modal_profiles.yaml)
+# TERMINAL_MODAL_PROFILES_FILE=modal_profiles.yaml
+
+# --- Default Profile Settings (used if no YAML file) ---
+# These apply when no profile is specified or for the "default" profile
+TERMINAL_MODAL_IMAGE=python:3.11
+TERMINAL_MODAL_MIN_POOL=1
+TERMINAL_MODAL_MAX_POOL=5
+TERMINAL_MODAL_IDLE_TIMEOUT=120
+TERMINAL_MODAL_MAX_LIFETIME=3600
+TERMINAL_MODAL_SCALE_DOWN_IDLE=180
+
+# --- Custom Profile Example: pytorch-gpu ---
+# Uncomment to enable a GPU profile for ML tasks
+# Usage: terminal_tool("python train.py", profile="pytorch-gpu")
+#
+# TERMINAL_MODAL_PROFILE_pytorch_gpu_IMAGE=pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime
+# TERMINAL_MODAL_PROFILE_pytorch_gpu_GPU=T4
+# TERMINAL_MODAL_PROFILE_pytorch_gpu_MEMORY=16384
+# TERMINAL_MODAL_PROFILE_pytorch_gpu_MIN_POOL=0
+# TERMINAL_MODAL_PROFILE_pytorch_gpu_MAX_POOL=2
+# TERMINAL_MODAL_PROFILE_pytorch_gpu_IDLE_TIMEOUT=60
+
+# --- Custom Profile Example: node ---
+# Uncomment to enable a Node.js profile
+# Usage: terminal_tool("npm test", profile="node")
+#
+# TERMINAL_MODAL_PROFILE_node_IMAGE=node:18
+# TERMINAL_MODAL_PROFILE_node_MIN_POOL=0
+# TERMINAL_MODAL_PROFILE_node_MAX_POOL=3
+
+# =============================================================================
+# MODAL SECRETS (Secure credential injection)
+# =============================================================================
+# Modal Secrets allow you to securely pass API keys, passwords, and other
+# sensitive data to your sandboxes without exposing them in code or logs.
+#
+# SETUP SECRETS:
+#   1. Via Dashboard: https://modal.com/secrets
+#   2. Via CLI: modal secret create my-secret KEY1=value1 KEY2=value2
+#   3. Via CLI with env: modal secret create my-secret API_KEY="$API_KEY"
+#
+# LIST SECRETS:
+#   modal secret list
+#
+# DELETE SECRETS:
+#   modal secret delete my-secret
+
+# Global secrets applied to ALL profiles (comma-separated secret names)
+# These secrets must be created on Modal dashboard or via CLI first
+# TERMINAL_MODAL_SECRETS=my-api-keys,database-creds
+
+# Per-profile secrets (comma-separated secret names)
+# TERMINAL_MODAL_PROFILE_pytorch_gpu_SECRETS=huggingface-token,wandb-key
+
+# Per-profile environment variables (semicolon-separated KEY=VALUE pairs)
+# TERMINAL_MODAL_PROFILE_default_ENV_VARS=DEBUG=1;LOG_LEVEL=info
+
+# Load local .env file into sandbox (useful for development)
+# TERMINAL_MODAL_PROFILE_default_USE_DOTENV=true
+
+# =============================================================================
+# BROWSER TOOL CONFIGURATION (agent-browser + Browserbase)
+# =============================================================================
+# Browser automation requires Browserbase cloud service for remote browser execution.
+# This allows the agent to navigate websites, fill forms, and extract information.
+#
+# STEALTH MODES:
+# - Basic Stealth: ALWAYS active (random fingerprints, auto CAPTCHA solving)
+# - Advanced Stealth: Requires BROWSERBASE_ADVANCED_STEALTH=true (Scale Plan only)
+
+# Browserbase API Key - Cloud browser execution
+# Get at: https://browserbase.com/
+BROWSERBASE_API_KEY=
+
+# Browserbase Project ID - From your Browserbase dashboard
+BROWSERBASE_PROJECT_ID=
+
+# Enable residential proxies for better CAPTCHA solving (default: true)
+# Routes traffic through residential IPs, significantly improves success rate
+BROWSERBASE_PROXIES=true
+
+# Enable advanced stealth mode (default: false, requires Scale Plan)
+# Uses custom Chromium build to avoid bot detection altogether
+BROWSERBASE_ADVANCED_STEALTH=false
+
+# Browser session timeout in seconds (default: 300)
+# Sessions are cleaned up after this duration of inactivity
+BROWSER_SESSION_TIMEOUT=300
+
+# Browser inactivity timeout - auto-cleanup inactive sessions (default: 120 = 2 min)
+# Browser sessions are automatically closed after this period of no activity
+BROWSER_INACTIVITY_TIMEOUT=120
+
+# =============================================================================
+# SESSION LOGGING
+# =============================================================================
+# Session trajectories are automatically saved to logs/ directory
+# Format: logs/session_YYYYMMDD_HHMMSS_UUID.json
+# Contains full conversation history in trajectory format for debugging/replay
+
+# =============================================================================
+# LEGACY/OPTIONAL API KEYS
 # =============================================================================
 
-# Terminal Tool Settings
+# Morph API Key - For legacy Hecate terminal backend (terminal-hecate tool)
+# Get at: https://morph.so/
+MORPH_API_KEY=
+
+# Hecate VM Settings (only if using terminal-hecate tool)
 HECATE_VM_LIFETIME_SECONDS=300
 HECATE_DEFAULT_SNAPSHOT_ID=snapshot_p5294qxt
 
-# Debug Logging (set to "true" to enable, logs saved to ./logs/)
+# =============================================================================
+# DEBUG OPTIONS
+# =============================================================================
 WEB_TOOLS_DEBUG=false
 VISION_TOOLS_DEBUG=false
 MOA_TOOLS_DEBUG=false
 IMAGE_TOOLS_DEBUG=false
+
+# =============================================================================
+# CONTEXT COMPRESSION (Auto-shrinks long conversations)
+# =============================================================================
+# When conversation approaches model's context limit, middle turns are
+# automatically summarized to free up space.
+#
+# CONTEXT_COMPRESSION_ENABLED=true        # Enable auto-compression (default: true)
+# CONTEXT_COMPRESSION_THRESHOLD=0.85      # Compress at 85% of context limit
+# CONTEXT_COMPRESSION_MODEL=google/gemini-2.0-flash-001  # Fast model for summaries
+
+# =============================================================================
+# RL TRAINING (Tinker + Atropos)
+# =============================================================================
+# Run reinforcement learning training on language models using the Tinker API.
+# Requires the rl-server to be running (from tinker-atropos package).
+
+# Tinker API Key - RL training service
+# Get at: https://tinker-console.thinkingmachines.ai/keys
+TINKER_API_KEY=
+
+# Weights & Biases API Key - Experiment tracking and metrics
+# Get at: https://wandb.ai/authorize
+WANDB_API_KEY=
+
+# RL API Server URL (default: http://localhost:8080)
+# Change if running the rl-server on a different host/port
+# RL_API_URL=http://localhost:8080

.gitignore

@@ -30,3 +30,39 @@ run_datagen_megascience_glm4-6.sh
 run_datagen_sonnet.sh
 source-data/*
 run_datagen_megascience_glm4-6.sh
+data/*
+node_modules/
+browser-use/
+agent-browser/
+# Private keys
+*.ppk
+*.pem
+privvy*
+images/
+__pycache__/
+hermes_agent.egg-info/
+wandb/
+testlogs
+
+# CLI config (may contain sensitive SSH paths)
+cli-config.yaml
+
+.DS_Store
+
+# artifacts
+*.jsonl
+*.html
+*.json
+*.log
+*.csv
+
+# Singularity/Apptainer images (large binary files)
+*.sif
+
+# Test files
+test_singularity_*.py
+test_*.py
+!tests/test_*.py
+
+# Nomad data
+/tmp/NomadClient*/

.gitmodules

@@ -0,0 +1,6 @@
+[submodule "mini-swe-agent"]
+	path = mini-swe-agent
+	url = https://github.com/SWE-agent/mini-swe-agent
+[submodule "tinker-atropos"]
+	path = tinker-atropos
+	url = https://github.com/nousresearch/tinker-atropos

AGENTS.md

@@ -0,0 +1,533 @@
+# Hermes Agent - Development Guide
+
+Instructions for AI coding assistants (GitHub Copilot, Cursor, etc.) and human developers.
+
+Hermes-Agent is an AI agent harness with tool-calling capabilities, interactive CLI, messaging integrations, and scheduled tasks.
+
+## Development Environment
+
+**IMPORTANT**: Always use the virtual environment if it exists:
+```bash
+source venv/bin/activate  # Before running any Python commands
+```
+
+## Project Structure
+
+```
+hermes-agent/
+├── hermes_cli/           # Unified CLI commands
+│   ├── main.py           # Entry point, command dispatcher
+│   ├── setup.py          # Interactive setup wizard
+│   ├── config.py         # Config management & migration
+│   ├── status.py         # Status display
+│   ├── doctor.py         # Diagnostics
+│   ├── gateway.py        # Gateway management
+│   ├── uninstall.py      # Uninstaller
+│   └── cron.py           # Cron job management
+├── tools/                # Tool implementations
+├── gateway/              # Messaging platform adapters
+├── cron/                 # Scheduler implementation
+├── skills/               # Knowledge documents
+├── cli.py                # Interactive CLI (Rich UI)
+├── run_agent.py          # Agent runner with AIAgent class
+├── model_tools.py        # Tool schemas and handlers
+├── toolsets.py           # Tool groupings
+├── toolset_distributions.py  # Probability-based tool selection
+└── batch_runner.py       # Parallel batch processing
+```
+
+**User Configuration** (stored in `~/.hermes/`):
+- `~/.hermes/config.yaml` - Settings (model, terminal, toolsets, etc.)
+- `~/.hermes/.env` - API keys and secrets
+
+## File Dependency Chain
+
+```
+tools/*.py → tools/__init__.py → model_tools.py → toolsets.py → toolset_distributions.py
+                                       ↑
+run_agent.py ──────────────────────────┘
+cli.py → run_agent.py (uses AIAgent with quiet_mode=True)
+batch_runner.py → run_agent.py + toolset_distributions.py
+```
+
+Always ensure consistency between tools, model_tools.py, and toolsets.py when changing any of them.
+
+---
+
+## AIAgent Class
+
+The main agent is implemented in `run_agent.py`:
+
+```python
+class AIAgent:
+    def __init__(
+        self,
+        model: str = "anthropic/claude-sonnet-4",
+        api_key: str = None,
+        base_url: str = "https://openrouter.ai/api/v1",
+        max_iterations: int = 60,        # Max tool-calling loops
+        enabled_toolsets: list = None,
+        disabled_toolsets: list = None,
+        verbose_logging: bool = False,
+        quiet_mode: bool = False,         # Suppress progress output
+        tool_progress_callback: callable = None,  # Called on each tool use
+    ):
+        # Initialize OpenAI client, load tools based on toolsets
+        ...
+    
+    def chat(self, user_message: str, task_id: str = None) -> str:
+        # Main entry point - runs the agent loop
+        ...
+```
+
+### Agent Loop
+
+The core loop in `_run_agent_loop()`:
+
+```
+1. Add user message to conversation
+2. Call LLM with tools
+3. If LLM returns tool calls:
+   - Execute each tool
+   - Add tool results to conversation
+   - Go to step 2
+4. If LLM returns text response:
+   - Return response to user
+```
+
+```python
+while turns < max_turns:
+    response = client.chat.completions.create(
+        model=model,
+        messages=messages,
+        tools=tool_schemas,
+    )
+    
+    if response.tool_calls:
+        for tool_call in response.tool_calls:
+            result = await execute_tool(tool_call)
+            messages.append(tool_result_message(result))
+        turns += 1
+    else:
+        return response.content
+```
+
+### Conversation Management
+
+Messages are stored as a list of dicts following OpenAI format:
+
+```python
+messages = [
+    {"role": "system", "content": "You are a helpful assistant..."},
+    {"role": "user", "content": "Search for Python tutorials"},
+    {"role": "assistant", "content": None, "tool_calls": [...]},
+    {"role": "tool", "tool_call_id": "...", "content": "..."},
+    {"role": "assistant", "content": "Here's what I found..."},
+]
+```
+
+### Reasoning Model Support
+
+For models that support chain-of-thought reasoning:
+- Extract `reasoning_content` from API responses
+- Store in `assistant_msg["reasoning"]` for trajectory export
+- Pass back via `reasoning_content` field on subsequent turns
+
+---
+
+## CLI Architecture (cli.py)
+
+The interactive CLI uses:
+- **Rich** - For the welcome banner and styled panels
+- **prompt_toolkit** - For fixed input area with history and `patch_stdout`
+- **KawaiiSpinner** (in run_agent.py) - Animated feedback during API calls and tool execution
+
+Key components:
+- `HermesCLI` class - Main CLI controller with commands and conversation loop
+- `load_cli_config()` - Loads config, sets environment variables for terminal
+- `build_welcome_banner()` - Displays ASCII art logo, tools, and skills summary
+- `/commands` - Process user commands like `/help`, `/clear`, `/personality`, etc.
+
+CLI uses `quiet_mode=True` when creating AIAgent to suppress verbose logging.
+
+### Adding CLI Commands
+
+1. Add to `COMMANDS` dict with description
+2. Add handler in `process_command()` method
+3. For persistent settings, use `save_config_value()` to update config
+
+---
+
+## Hermes CLI Commands
+
+The unified `hermes` command provides all functionality:
+
+| Command | Description |
+|---------|-------------|
+| `hermes` | Interactive chat (default) |
+| `hermes chat -q "..."` | Single query mode |
+| `hermes setup` | Configure API keys and settings |
+| `hermes config` | View current configuration |
+| `hermes config edit` | Open config in editor |
+| `hermes config set KEY VAL` | Set a specific value |
+| `hermes config check` | Check for missing config |
+| `hermes config migrate` | Prompt for missing config interactively |
+| `hermes status` | Show configuration status |
+| `hermes doctor` | Diagnose issues |
+| `hermes update` | Update to latest (checks for new config) |
+| `hermes uninstall` | Uninstall (can keep configs for reinstall) |
+| `hermes gateway` | Start messaging gateway |
+| `hermes cron list` | View scheduled jobs |
+| `hermes version` | Show version info |
+
+---
+
+## Messaging Gateway
+
+The gateway connects Hermes to Telegram, Discord, and WhatsApp.
+
+### Configuration (in `~/.hermes/.env`):
+
+```bash
+# Telegram
+TELEGRAM_BOT_TOKEN=123456:ABC-DEF...      # From @BotFather
+TELEGRAM_ALLOWED_USERS=123456789,987654   # Comma-separated user IDs (from @userinfobot)
+
+# Discord  
+DISCORD_BOT_TOKEN=MTIz...                 # From Developer Portal
+DISCORD_ALLOWED_USERS=123456789012345678  # Comma-separated user IDs
+
+# Agent Behavior
+HERMES_MAX_ITERATIONS=60                  # Max tool-calling iterations
+MESSAGING_CWD=/home/myuser                # Terminal working directory for messaging
+
+# Tool Progress (optional)
+HERMES_TOOL_PROGRESS=true                 # Send progress messages
+HERMES_TOOL_PROGRESS_MODE=new             # "new" or "all"
+```
+
+### Working Directory Behavior
+
+- **CLI (`hermes` command)**: Uses current directory (`.` → `os.getcwd()`)
+- **Messaging (Telegram/Discord)**: Uses `MESSAGING_CWD` (default: home directory)
+
+This is intentional: CLI users are in a terminal and expect the agent to work in their current directory, while messaging users need a consistent starting location.
+
+### Security (User Allowlists):
+
+**IMPORTANT**: Without an allowlist, anyone who finds your bot can use it!
+
+The gateway checks `{PLATFORM}_ALLOWED_USERS` environment variables:
+- If set: Only listed user IDs can interact with the bot
+- If unset: All users are allowed (dangerous with terminal access!)
+
+Users can find their IDs:
+- **Telegram**: Message [@userinfobot](https://t.me/userinfobot)
+- **Discord**: Enable Developer Mode, right-click name → Copy ID
+
+### Tool Progress Notifications
+
+When `HERMES_TOOL_PROGRESS=true`, the bot sends status messages as it works:
+- `💻 \`ls -la\`...` (terminal commands show the actual command)
+- `🔍 web_search...`
+- `📄 web_extract...`
+
+Modes:
+- `new`: Only when switching to a different tool (less spam)
+- `all`: Every single tool call
+
+### Typing Indicator
+
+The gateway keeps the "typing..." indicator active throughout processing, refreshing every 4 seconds. This lets users know the bot is working even during long tool-calling sequences.
+
+### Platform Toolsets:
+
+Each platform has a dedicated toolset in `toolsets.py`:
+- `hermes-telegram`: Full tools including terminal (with safety checks)
+- `hermes-discord`: Full tools including terminal
+- `hermes-whatsapp`: Full tools including terminal
+
+---
+
+## Configuration System
+
+Configuration files are stored in `~/.hermes/` for easy user access:
+- `~/.hermes/config.yaml` - All settings (model, terminal, compression, etc.)
+- `~/.hermes/.env` - API keys and secrets
+
+### Adding New Configuration Options
+
+When adding new configuration variables, you MUST follow this process:
+
+#### For config.yaml options:
+
+1. Add to `DEFAULT_CONFIG` in `hermes_cli/config.py`
+2. **CRITICAL**: Bump `_config_version` in `DEFAULT_CONFIG` when adding required fields
+3. This triggers migration prompts for existing users on next `hermes update` or `hermes setup`
+
+Example:
+```python
+DEFAULT_CONFIG = {
+    # ... existing config ...
+    
+    "new_feature": {
+        "enabled": True,
+        "option": "default_value",
+    },
+    
+    # BUMP THIS when adding required fields
+    "_config_version": 2,  # Was 1, now 2
+}
+```
+
+#### For .env variables (API keys/secrets):
+
+1. Add to `REQUIRED_ENV_VARS` or `OPTIONAL_ENV_VARS` in `hermes_cli/config.py`
+2. Include metadata for the migration system:
+
+```python
+OPTIONAL_ENV_VARS = {
+    # ... existing vars ...
+    "NEW_API_KEY": {
+        "description": "What this key is for",
+        "prompt": "Display name in prompts",
+        "url": "https://where-to-get-it.com/",
+        "tools": ["tools_it_enables"],  # What tools need this
+        "password": True,  # Mask input
+    },
+}
+```
+
+#### Update related files:
+
+- `hermes_cli/setup.py` - Add prompts in the setup wizard
+- `cli-config.yaml.example` - Add example with comments
+- Update README.md if user-facing
+
+### Config Version Migration
+
+The system uses `_config_version` to detect outdated configs:
+
+1. `check_for_missing_config()` compares user config to `DEFAULT_CONFIG`
+2. `migrate_config()` interactively prompts for missing values
+3. Called automatically by `hermes update` and optionally by `hermes setup`
+
+---
+
+## Environment Variables
+
+API keys are loaded from `~/.hermes/.env`:
+- `OPENROUTER_API_KEY` - Main LLM API access (primary provider)
+- `FIRECRAWL_API_KEY` - Web search/extract tools
+- `BROWSERBASE_API_KEY` / `BROWSERBASE_PROJECT_ID` - Browser automation
+- `FAL_KEY` - Image generation (FLUX model)
+- `NOUS_API_KEY` - Vision and Mixture-of-Agents tools
+
+Terminal tool configuration (in `~/.hermes/config.yaml`):
+- `terminal.backend` - Backend: local, docker, singularity, modal, or ssh
+- `terminal.cwd` - Working directory for CLI ("." = current directory)
+- `terminal.docker_image` - Image for Docker backend
+- `terminal.singularity_image` - Image for Singularity backend
+- `terminal.modal_image` - Image for Modal backend
+- SSH: `TERMINAL_SSH_HOST`, `TERMINAL_SSH_USER`, `TERMINAL_SSH_KEY` in .env
+
+Agent behavior (in `~/.hermes/.env`):
+- `HERMES_MAX_ITERATIONS` - Max tool-calling iterations (default: 60)
+- `MESSAGING_CWD` - Working directory for messaging platforms (default: ~)
+- `HERMES_TOOL_PROGRESS` - Enable tool progress messages (`true`/`false`)
+- `HERMES_TOOL_PROGRESS_MODE` - Progress mode: `new` (tool changes) or `all`
+
+### Dangerous Command Approval
+
+The terminal tool includes safety checks for potentially destructive commands (e.g., `rm -rf`, `DROP TABLE`, `chmod 777`, etc.):
+
+**Behavior by Backend:**
+- **Docker/Singularity/Modal**: Commands run unrestricted (isolated containers)
+- **Local/SSH**: Dangerous commands trigger approval flow
+
+**Approval Flow (CLI):**
+```
+⚠️  Potentially dangerous command detected: recursive delete
+    rm -rf /tmp/test
+
+    [o]nce  |  [s]ession  |  [a]lways  |  [d]eny
+    Choice [o/s/a/D]: 
+```
+
+**Approval Flow (Messaging):**
+- Command is blocked with explanation
+- Agent explains the command was blocked for safety
+- User must add the pattern to their allowlist via `hermes config edit` or run the command directly on their machine
+
+**Configuration:**
+- `command_allowlist` in `~/.hermes/config.yaml` stores permanently allowed patterns
+- Add patterns via "always" approval or edit directly
+
+**Sudo Handling (Messaging):**
+- If sudo fails over messaging, output includes tip to add `SUDO_PASSWORD` to `~/.hermes/.env`
+
+---
+
+## Adding New Tools
+
+Follow this strict order to maintain consistency:
+
+1. Create `tools/your_tool.py` with:
+   - Handler function (sync or async) returning a JSON string via `json.dumps()`
+   - `check_*_requirements()` function to verify dependencies (e.g., API keys)
+   - Schema definition following OpenAI function-calling format
+
+2. Export in `tools/__init__.py`:
+   - Import the handler and check function
+   - Add to `__all__` list
+
+3. Register in `model_tools.py`:
+   - Add to `TOOLSET_REQUIREMENTS` if it needs API keys
+   - Create `get_*_tool_definitions()` function or add to existing
+   - Add routing in `handle_function_call()` dispatcher
+   - Update `get_all_tool_names()` with the tool name
+   - Update `get_toolset_for_tool()` mapping
+   - Update `get_available_toolsets()` and `check_toolset_requirements()`
+
+4. Add to toolset in `toolsets.py`:
+   - Add to existing toolset or create new one in TOOLSETS dict
+
+5. If the tool requires an API key:
+   - Add to `OPTIONAL_ENV_VARS` in `hermes_cli/config.py`
+   - The tool will be auto-disabled if the key is missing
+
+6. Optionally add to `toolset_distributions.py` for batch processing
+
+### Tool Implementation Pattern
+
+```python
+# tools/example_tool.py
+import json
+import os
+
+def check_example_requirements() -> bool:
+    """Check if required API keys/dependencies are available."""
+    return bool(os.getenv("EXAMPLE_API_KEY"))
+
+def example_tool(param: str, task_id: str = None) -> str:
+    """Execute the tool and return JSON string result."""
+    try:
+        result = {"success": True, "data": "..."}
+        return json.dumps(result, ensure_ascii=False)
+    except Exception as e:
+        return json.dumps({"error": str(e)}, ensure_ascii=False)
+```
+
+All tool handlers MUST return a JSON string. Never return raw dicts.
+
+### Dynamic Tool Availability
+
+Tools are automatically disabled when their API keys are missing:
+
+```python
+# In model_tools.py
+TOOLSET_REQUIREMENTS = {
+    "web": {"env_vars": ["FIRECRAWL_API_KEY"]},
+    "browser": {"env_vars": ["BROWSERBASE_API_KEY", "BROWSERBASE_PROJECT_ID"]},
+    "creative": {"env_vars": ["FAL_KEY"]},
+}
+```
+
+The `check_tool_availability()` function determines which tools to include.
+
+### Stateful Tools
+
+Tools that maintain state (terminal, browser) require:
+- `task_id` parameter for session isolation between concurrent tasks
+- `cleanup_*()` function to release resources
+- Cleanup is called automatically in run_agent.py after conversation completes
+
+---
+
+## Trajectory Format
+
+Conversations are saved in ShareGPT format for training:
+```json
+{"from": "system", "value": "System prompt with <tools>...</tools>"}
+{"from": "human", "value": "User message"}
+{"from": "gpt", "value": "<think>reasoning</think>\n<tool_call>{...}</tool_call>"}
+{"from": "tool", "value": "<tool_response>{...}</tool_response>"}
+{"from": "gpt", "value": "Final response"}
+```
+
+Tool calls use `<tool_call>` XML tags, responses use `<tool_response>` tags, reasoning uses `<think>` tags.
+
+### Trajectory Export
+
+```python
+agent = AIAgent(save_trajectories=True)
+agent.chat("Do something")
+# Saves to trajectories/*.jsonl in ShareGPT format
+```
+
+---
+
+## Batch Processing (batch_runner.py)
+
+For processing multiple prompts:
+- Parallel execution with multiprocessing
+- Content-based resume for fault tolerance (matches on prompt text, not indices)
+- Toolset distributions control probabilistic tool availability per prompt
+- Output: `data/<run_name>/trajectories.jsonl` (combined) + individual batch files
+
+```bash
+python batch_runner.py \
+    --dataset_file=prompts.jsonl \
+    --batch_size=20 \
+    --num_workers=4 \
+    --run_name=my_run
+```
+
+---
+
+## Skills System
+
+Skills are on-demand knowledge documents the agent can load. Located in `skills/` directory:
+
+```
+skills/
+├── mlops/                    # Category folder
+│   ├── axolotl/             # Skill folder
+│   │   ├── SKILL.md         # Main instructions (required)
+│   │   ├── references/      # Additional docs, API specs
+│   │   └── templates/       # Output formats, configs
+│   └── vllm/
+│       └── SKILL.md
+└── example-skill/
+    └── SKILL.md
+```
+
+**Progressive disclosure** (token-efficient):
+1. `skills_categories()` - List category names (~50 tokens)
+2. `skills_list(category)` - Name + description per skill (~3k tokens)
+3. `skill_view(name)` - Full content + tags + linked files
+
+SKILL.md files use YAML frontmatter:
+```yaml
+---
+name: skill-name
+description: Brief description for listing
+tags: [tag1, tag2]
+related_skills: [other-skill]
+version: 1.0.0
+---
+# Skill Content...
+```
+
+Tool files: `tools/skills_tool.py` → `model_tools.py` → `toolsets.py`
+
+---
+
+## Testing Changes
+
+After making changes:
+
+1. Run `hermes doctor` to check setup
+2. Run `hermes config check` to verify config
+3. Test with `hermes chat -q "test message"`
+4. For new config options, test fresh install: `rm -rf ~/.hermes && hermes setup`

TODO.md

@@ -0,0 +1,589 @@
+# Hermes Agent - Future Improvements
+
+> Ideas for enhancing the agent's capabilities, generated from self-analysis of the codebase.
+
+---
+
+## 1. Subagent Architecture (Context Isolation) 🎯
+
+**Problem:** Long-running tools (terminal commands, browser automation, complex file operations) consume massive context. A single `ls -la` can add hundreds of lines. Browser snapshots, debugging sessions, and iterative terminal work quickly bloat the main conversation, leaving less room for actual reasoning.
+
+**Solution:** The main agent becomes an **orchestrator** that delegates context-heavy tasks to **subagents**.
+
+**Architecture:**
+```
+┌─────────────────────────────────────────────────────────────────┐
+│  ORCHESTRATOR (main agent)                                      │
+│  - Receives user request                                        │
+│  - Plans approach                                               │
+│  - Delegates heavy tasks to subagents                           │
+│  - Receives summarized results                                  │
+│  - Maintains clean, focused context                             │
+└─────────────────────────────────────────────────────────────────┘
+         │                    │                    │
+         ▼                    ▼                    ▼
+┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
+│ TERMINAL AGENT  │  │ BROWSER AGENT   │  │ CODE AGENT      │
+│ - terminal tool │  │ - browser tools │  │ - file tools    │
+│ - file tools    │  │ - web_search    │  │ - terminal      │
+│                 │  │ - web_extract   │  │                 │
+│ Isolated context│  │ Isolated context│  │ Isolated context│
+│ Returns summary │  │ Returns summary │  │ Returns summary │
+└─────────────────┘  └─────────────────┘  └─────────────────┘
+```
+
+**How it works:**
+1. User asks: "Set up a new Python project with FastAPI and tests"
+2. Orchestrator plans: "I need to create files, install deps, write code"
+3. Orchestrator calls: `terminal_task(goal="Create venv, install fastapi pytest", context="New project in ~/myapp")`
+4. **Subagent spawns** with fresh context, only terminal/file tools
+5. Subagent iterates (may take 10+ tool calls, lots of output)
+6. Subagent completes → returns summary: "Created venv, installed fastapi==0.109.0, pytest==8.0.0"
+7. Orchestrator receives **only the summary**, context stays clean
+8. Orchestrator continues with next subtask
+
+**Key tools to implement:**
+- [ ] `terminal_task(goal, context, cwd?)` - Delegate terminal/shell work
+- [ ] `browser_task(goal, context, start_url?)` - Delegate web research/automation  
+- [ ] `code_task(goal, context, files?)` - Delegate code writing/modification
+- [ ] Generic `delegate_task(goal, context, toolsets=[])` - Flexible delegation
+
+**Implementation details:**
+- [ ] Subagent uses same `run_agent.py` but with:
+  - Fresh/empty conversation history
+  - Limited toolset (only what's needed)
+  - Smaller max_iterations (focused task)
+  - Task-specific system prompt
+- [ ] Subagent returns structured result:
+  ```python
+  {
+    "success": True,
+    "summary": "Installed 3 packages, created 2 files",
+    "details": "Optional longer explanation if needed",
+    "artifacts": ["~/myapp/requirements.txt", "~/myapp/main.py"],  # Files created
+    "errors": []  # Any issues encountered
+  }
+  ```
+- [ ] Orchestrator sees only the summary in its context
+- [ ] Full subagent transcript saved separately for debugging
+
+**Benefits:**
+- 🧹 **Clean context** - Orchestrator stays focused, doesn't drown in tool output
+- 📊 **Better token efficiency** - 50 terminal outputs → 1 summary paragraph
+- 🎯 **Focused subagents** - Each agent has just the tools it needs
+- 🔄 **Parallel potential** - Independent subtasks could run concurrently
+- 🐛 **Easier debugging** - Each subtask has its own isolated transcript
+
+**When to use subagents vs direct tools:**
+- **Subagent**: Multi-step tasks, iteration likely, lots of output expected
+- **Direct**: Quick one-off commands, simple file reads, user needs to see output
+
+**Files to modify:** `run_agent.py` (add orchestration mode), new `tools/delegate_tools.py`, new `subagent_runner.py`
+
+---
+
+## 2. Planning & Task Management 📋
+
+**Problem:** Agent handles tasks reactively without explicit planning. Complex multi-step tasks lack structure, progress tracking, and the ability to decompose work into manageable chunks.
+
+**Ideas:**
+- [ ] **Task decomposition tool** - Break complex requests into subtasks:
+  ```
+  User: "Set up a new Python project with FastAPI, tests, and Docker"
+  
+  Agent creates plan:
+  ├── 1. Create project structure and requirements.txt
+  ├── 2. Implement FastAPI app skeleton
+  ├── 3. Add pytest configuration and initial tests
+  ├── 4. Create Dockerfile and docker-compose.yml
+  └── 5. Verify everything works together
+  ```
+  - Each subtask becomes a trackable unit
+  - Agent can report progress: "Completed 3/5 tasks"
+  
+- [ ] **Progress checkpoints** - Periodic self-assessment:
+  - After N tool calls or time elapsed, pause to evaluate
+  - "What have I accomplished? What remains? Am I on track?"
+  - Detect if stuck in loops or making no progress
+  - Could trigger replanning if approach isn't working
+  
+- [ ] **Explicit plan storage** - Persist plan in conversation:
+  - Store as structured data (not just in context)
+  - Update status as tasks complete
+  - User can ask "What's the plan?" or "What's left?"
+  - Survives context compression (plans are protected)
+
+- [ ] **Failure recovery with replanning** - When things go wrong:
+  - Record what failed and why
+  - Revise plan to work around the issue
+  - "Step 3 failed because X, adjusting approach to Y"
+  - Prevents repeating failed strategies
+
+**Files to modify:** `run_agent.py` (add planning hooks), new `tools/planning_tool.py`
+
+---
+
+## 3. Dynamic Skills Expansion 📚
+
+**Problem:** Skills system is elegant but static. Skills must be manually created and added.
+
+**Ideas:**
+- [ ] **Skill acquisition from successful tasks** - After completing a complex task:
+  - "This approach worked well. Save as a skill?"
+  - Extract: goal, steps taken, tools used, key decisions
+  - Generate SKILL.md automatically
+  - Store in user's skills directory
+  
+- [ ] **Skill templates** - Common patterns that can be parameterized:
+  ```markdown
+  # Debug {language} Error
+  1. Reproduce the error
+  2. Search for error message: `web_search("{error_message} {language}")`
+  3. Check common causes: {common_causes}
+  4. Apply fix and verify
+  ```
+  
+- [ ] **Skill chaining** - Combine skills for complex workflows:
+  - Skills can reference other skills as dependencies
+  - "To do X, first apply skill Y, then skill Z"
+  - Directed graph of skill dependencies
+
+**Files to modify:** `tools/skills_tool.py`, `skills/` directory structure, new `skill_generator.py`
+
+---
+
+## 4. Interactive Clarifying Questions Tool ❓
+
+**Problem:** Agent sometimes makes assumptions or guesses when it should ask the user. Currently can only ask via text, which gets lost in long outputs.
+
+**Ideas:**
+- [ ] **Multiple-choice prompt tool** - Let agent present structured choices to user:
+  ```
+  ask_user_choice(
+    question="Should the language switcher enable only German or all languages?",
+    choices=[
+      "Only enable German - works immediately",
+      "Enable all, mark untranslated - show fallback notice",
+      "Let me specify something else"
+    ]
+  )
+  ```
+  - Renders as interactive terminal UI with arrow key / Tab navigation
+  - User selects option, result returned to agent
+  - Up to 4 choices + optional free-text option
+  
+- [ ] **Implementation:**
+  - Use `inquirer` or `questionary` Python library for rich terminal prompts
+  - Tool returns selected option text (or user's custom input)
+  - **CLI-only** - only works when running via `cli.py` (not API/programmatic use)
+  - Graceful fallback: if not in interactive mode, return error asking agent to rephrase as text
+  
+- [ ] **Use cases:**
+  - Clarify ambiguous requirements before starting work
+  - Confirm destructive operations with clear options
+  - Let user choose between implementation approaches
+  - Checkpoint complex multi-step workflows
+
+**Files to modify:** New `tools/ask_user_tool.py`, `cli.py` (detect interactive mode), `model_tools.py`
+
+---
+
+## 5. Collaborative Problem Solving 🤝
+
+**Problem:** Interaction is command/response. Complex problems benefit from dialogue.
+
+**Ideas:**
+- [ ] **Assumption surfacing** - Make implicit assumptions explicit:
+  - "I'm assuming you want Python 3.11+. Correct?"
+  - "This solution assumes you have sudo access..."
+  - Let user correct before going down wrong path
+
+- [ ] **Checkpoint & confirm** - For high-stakes operations:
+  - "About to delete 47 files. Here's the list - proceed?"
+  - "This will modify your database. Want a backup first?"
+  - Configurable threshold for when to ask
+
+**Files to modify:** `run_agent.py`, system prompt configuration
+
+---
+
+## 6. Project-Local Context 💾
+
+**Problem:** Valuable context lost between sessions.
+
+**Ideas:**
+- [ ] **Project awareness** - Remember project-specific context:
+  - Store `.hermes/context.md` in project directory
+  - "This is a Django project using PostgreSQL"
+  - Coding style preferences, deployment setup, etc.
+  - Load automatically when working in that directory
+
+- [ ] **Handoff notes** - Leave notes for future sessions:
+  - Write to `.hermes/notes.md` in project
+  - "TODO for next session: finish implementing X"
+  - "Known issues: Y doesn't work on Windows"
+
+**Files to modify:** New `project_context.py`, auto-load in `run_agent.py`
+
+## 6. Tools & Skills Wishlist 🧰
+
+*Things that would need new tool implementations (can't do well with current tools):*
+
+### High-Impact
+
+- [ ] **Audio/Video Transcription** 🎬 *(See also: Section 16 for detailed spec)*
+  - Transcribe audio files, podcasts, YouTube videos
+  - Extract key moments from video
+  - Voice memo transcription for messaging integrations
+  - *Provider options: Whisper API, Deepgram, local Whisper*
+  
+- [ ] **Diagram Rendering** 📊
+  - Render Mermaid/PlantUML to actual images
+  - Can generate the code, but rendering requires external service or tool
+  - "Show me how these components connect" → actual visual diagram
+
+### Medium-Impact
+
+- [ ] **Canvas / Visual Workspace** 🖼️
+  - Agent-controlled visual panel for rendering interactive UI
+  - Inspired by OpenClaw's Canvas feature
+  - **Capabilities:**
+    - `present` / `hide` - Show/hide the canvas panel
+    - `navigate` - Load HTML files or URLs into the canvas
+    - `eval` - Execute JavaScript in the canvas context
+    - `snapshot` - Capture the rendered UI as an image
+  - **Use cases:**
+    - Display generated HTML/CSS/JS previews
+    - Show interactive data visualizations (charts, graphs)
+    - Render diagrams (Mermaid → rendered output)
+    - Present structured information in rich format
+    - A2UI-style component system for structured agent UI
+  - **Implementation options:**
+    - Electron-based panel for CLI
+    - WebSocket-connected web app
+    - VS Code webview extension
+  - *Would let agent "show" things rather than just describe them*
+
+- [ ] **Document Generation** 📄
+  - Create styled PDFs, Word docs, presentations
+  - *Can do basic PDF via terminal tools, but limited*
+
+- [ ] **Diff/Patch Tool** 📝
+  - Surgical code modifications with preview
+  - "Change line 45-50 to X" without rewriting whole file
+  - Show diffs before applying
+  - *Can use `diff`/`patch` but a native tool would be safer*
+
+### Skills to Create
+
+- [ ] **Domain-specific skill packs:**
+  - DevOps/Infrastructure (Terraform, K8s, AWS)
+  - Data Science workflows (EDA, model training)
+  - Security/pentesting procedures
+  
+- [ ] **Framework-specific skills:**
+  - React/Vue/Angular patterns
+  - Django/Rails/Express conventions
+  - Database optimization playbooks
+
+- [ ] **Troubleshooting flowcharts:**
+  - "Docker container won't start" → decision tree
+  - "Production is slow" → systematic diagnosis
+
+---
+
+## 7. Messaging Platform Integrations 💬 ✅ COMPLETE
+
+**Problem:** Agent currently only works via `cli.py` which requires direct terminal access. Users may want to interact via messaging apps from their phone or other devices.
+
+**Architecture:**
+- `run_agent.py` already accepts `conversation_history` parameter and returns updated messages ✅
+- Need: persistent session storage, platform monitors, session key resolution
+
+**Implementation approach:**
+```
+┌─────────────────────────────────────────────────────────────┐
+│  Platform Monitor (e.g., telegram_monitor.py)               │
+│  ├─ Long-running daemon connecting to messaging platform    │
+│  ├─ On message: resolve session key → load history from disk│
+│  ├─ Call run_agent.py with loaded history                   │
+│  ├─ Save updated history back to disk (JSONL)               │
+│  └─ Send response back to platform                          │
+└─────────────────────────────────────────────────────────────┘
+```
+
+**Platform support (each user sets up their own credentials):**
+- [x] **Telegram** - via `python-telegram-bot`
+  - Bot token from @BotFather
+  - Easiest to set up, good for personal use
+- [x] **Discord** - via `discord.py`
+  - Bot token from Discord Developer Portal
+  - Can work in servers (group sessions) or DMs
+- [x] **WhatsApp** - via Node.js bridge (whatsapp-web.js/baileys)
+  - Requires Node.js bridge setup
+  - More complex, but reaches most people
+
+**Session management:**
+- [x] **Session store** - JSONL persistence per session key
+  - `~/.hermes/sessions/{session_id}.jsonl`
+  - Session keys: `agent:main:telegram:dm`, `agent:main:discord:group:123`, etc.
+- [x] **Session expiry** - Configurable reset policies
+  - Daily reset (default 4am) OR idle timeout (default 2 hours)
+  - Manual reset via `/reset` or `/new` command in chat
+  - Per-platform and per-type overrides
+- [x] **Session continuity** - Conversations persist across messages until reset
+
+**Files created:** `gateway/`, `gateway/platforms/`, `gateway/config.py`, `gateway/session.py`, `gateway/delivery.py`, `gateway/run.py`
+
+**Configuration:**
+- Environment variables: `TELEGRAM_BOT_TOKEN`, `DISCORD_BOT_TOKEN`, etc.
+- Config file: `~/.hermes/gateway.json`
+- CLI commands: `/platforms` to check status, `--gateway` to start
+
+**Dynamic context injection:**
+- Agent knows its source platform and chat
+- Agent knows connected platforms and home channels
+- Agent can deliver cron outputs to specific platforms
+
+---
+
+## 8. Text-to-Speech (TTS) 🔊
+
+**Problem:** Agent can only respond with text. Some users prefer audio responses (accessibility, hands-free use, podcasts).
+
+**Ideas:**
+- [ ] **TTS tool** - Generate audio files from text
+  ```python
+  tts_generate(text="Here's your summary...", voice="nova", output="summary.mp3")
+  ```
+  - Returns path to generated audio file
+  - For messaging integrations: can send as voice message
+  
+- [ ] **Provider options:**
+  - Edge TTS (free, good quality, many voices)
+  - OpenAI TTS (paid, excellent quality)
+  - ElevenLabs (paid, best quality, voice cloning)
+  - Local options (Coqui TTS, Bark)
+  
+- [ ] **Modes:**
+  - On-demand: User explicitly asks "read this to me"
+  - Auto-TTS: Configurable to always generate audio for responses
+  - Long-text handling: Summarize or chunk very long responses
+  
+- [ ] **Integration with messaging:**
+  - When enabled, can send voice notes instead of/alongside text
+  - User preference per channel
+
+**Files to create:** `tools/tts_tool.py`, config in `cli-config.yaml`
+
+---
+
+## 13. Speech-to-Text / Audio Transcription 🎤
+
+**Problem:** Users may want to send voice memos instead of typing. Agent is blind to audio content.
+
+**Ideas:**
+- [ ] **Voice memo transcription** - For messaging integrations
+  - User sends voice message → transcribe → process as text
+  - Seamless: user speaks, agent responds
+  
+- [ ] **Audio/video file transcription** - Existing idea, expanded:
+  - Transcribe local audio files (mp3, wav, m4a)
+  - Transcribe YouTube videos (download audio → transcribe)
+  - Extract key moments with timestamps
+  
+- [ ] **Provider options:**
+  - OpenAI Whisper API (good quality, cheap)
+  - Deepgram (fast, good for real-time)
+  - Local Whisper (free, runs on GPU)
+  - Groq Whisper (fast, free tier available)
+  
+- [ ] **Tool interface:**
+  ```python
+  transcribe(source="audio.mp3")  # Local file
+  transcribe(source="https://youtube.com/...")  # YouTube
+  transcribe(source="voice_message", data=bytes)  # Voice memo
+  ```
+
+**Files to create:** `tools/transcribe_tool.py`, integrate with messaging monitors
+
+### Plugin/Extension System 🔌
+
+**Concept:** Allow users to add custom tools/skills without modifying core code.
+
+**Why interesting:**
+- Community contributions
+- Organization-specific tools
+- Clean separation of core vs. extensions
+
+**Open questions:**
+- Security implications of loading arbitrary code
+- Versioning and compatibility
+- Discovery and installation UX
+
+---
+
+## Recently Completed ✅
+
+### Dangerous Command Approval System
+**Implemented:** Dangerous command detection and approval for terminal tool.
+
+**Features:**
+- Pattern-based detection of dangerous commands (rm -rf, DROP TABLE, chmod 777, etc.)
+- CLI prompt with options: `[o]nce | [s]ession | [a]lways | [d]eny`
+- Session caching (approved patterns don't re-prompt)
+- Permanent allowlist in `~/.hermes/config.yaml`
+- Force flag for agent to bypass after user confirmation
+- Skip check for isolated backends (Docker, Singularity, Modal)
+- Helpful sudo failure messages for messaging platforms
+
+**Files:** `tools/terminal_tool.py`, `model_tools.py`, `hermes_cli/config.py`
+
+---
+
+## 14. Learning Machine / Dynamic Memory System 🧠
+
+*Inspired by [Dash](~/agent-codebases/dash) - a self-learning data agent.*
+
+**Problem:** Agent starts fresh every session. Valuable learnings from debugging, error patterns, successful approaches, and user preferences are lost.
+
+**Dash's Key Insight:** Separate **Knowledge** (static, curated) from **Learnings** (dynamic, discovered):
+
+| System | What It Stores | How It Evolves |
+|--------|---------------|----------------|
+| **Knowledge** (Skills) | Validated approaches, templates, best practices | Curated by user |
+| **Learnings** | Error patterns, gotchas, discovered fixes | Managed automatically |
+
+**Tools to implement:**
+- [ ] `save_learning(topic, learning, context?)` - Record a discovered pattern
+  ```python
+  save_learning(
+    topic="python-ssl",
+    learning="On Ubuntu 22.04, SSL certificate errors often fixed by: apt install ca-certificates",
+    context="Debugging requests SSL failure"
+  )
+  ```
+- [ ] `search_learnings(query)` - Find relevant past learnings
+  ```python
+  search_learnings("SSL certificate error Python")
+  # Returns: "On Ubuntu 22.04, SSL certificate errors often fixed by..."
+  ```
+
+**User Profile & Memory:**
+- [ ] `user_profile` - Structured facts about user preferences
+  ```yaml
+  # ~/.hermes/user_profile.yaml
+  coding_style:
+    python_formatter: black
+    type_hints: always
+    test_framework: pytest
+  preferences:
+    verbosity: detailed
+    confirm_destructive: true
+  environment:
+    os: linux
+    shell: bash
+    default_python: 3.11
+  ```
+- [ ] `user_memory` - Unstructured observations the agent learns
+  ```yaml
+  # ~/.hermes/user_memory.yaml
+  - "User prefers tabs over spaces despite black's defaults"
+  - "User's main project is ~/work/myapp - a Django app"
+  - "User often works late - don't ask about timezone"
+  ```
+
+**When to learn:**
+- After fixing an error that took multiple attempts
+- When user corrects the agent's approach
+- When a workaround is discovered for a tool limitation
+- When user expresses a preference
+
+**Storage:** Vector database (ChromaDB) or simple YAML with embedding search.
+
+**Files to create:** `tools/learning_tools.py`, `learning/store.py`, `~/.hermes/learnings/`
+
+---
+
+## 15. Layered Context Architecture 📊
+
+*Inspired by Dash's "Six Layers of Context" - grounding responses in multiple sources.*
+
+**Problem:** Context sources are ad-hoc. No clear hierarchy or strategy for what context to include when.
+
+**Proposed Layers for Hermes:**
+
+| Layer | Source | When Loaded | Example |
+|-------|--------|-------------|---------|
+| 1. **Project Context** | `.hermes/context.md` | Auto on cwd | "This is a FastAPI project using PostgreSQL" |
+| 2. **Skills** | `skills/*.md` | On request | "How to set up React project" |
+| 3. **User Profile** | `~/.hermes/user_profile.yaml` | Always | "User prefers pytest, uses black" |
+| 4. **Learnings** | `~/.hermes/learnings/` | Semantic search | "SSL fix for Ubuntu" |
+| 5. **External Knowledge** | Web search, docs | On demand | Current API docs, Stack Overflow |
+| 6. **Runtime Introspection** | Tool calls | Real-time | File contents, terminal output |
+
+**Benefits:**
+- Clear mental model for what context is available
+- Prioritization: local > learned > external
+- Debugging: "Why did agent do X?" → check which layers contributed
+
+**Files to modify:** `run_agent.py` (context loading), new `context/layers.py`
+
+---
+
+## 16. Evaluation System with LLM Grading 📏
+
+*Inspired by Dash's evaluation framework.*
+
+**Problem:** `batch_runner.py` runs test cases but lacks quality assessment.
+
+**Dash's Approach:**
+- **String matching** (default) - Check if expected strings appear
+- **LLM grader** (-g flag) - GPT evaluates response quality
+- **Result comparison** (-r flag) - Compare against golden output
+
+**Implementation for Hermes:**
+
+- [ ] **Test case format:**
+  ```python
+  TestCase(
+    name="create_python_project",
+    prompt="Create a new Python project with FastAPI and tests",
+    expected_strings=["requirements.txt", "main.py", "test_"],  # Basic check
+    golden_actions=["write:main.py", "write:requirements.txt", "terminal:pip install"],
+    grader_criteria="Should create complete project structure with working code"
+  )
+  ```
+
+- [ ] **LLM grader mode:**
+  ```python
+  def grade_response(response: str, criteria: str) -> Grade:
+      """Use GPT to evaluate response quality."""
+      prompt = f"""
+      Evaluate this agent response against the criteria.
+      Criteria: {criteria}
+      Response: {response}
+      
+      Score (1-5) and explain why.
+      """
+      # Returns: Grade(score=4, explanation="Created all files but tests are minimal")
+  ```
+
+- [ ] **Action comparison mode:**
+  - Record tool calls made during test
+  - Compare against expected actions
+  - "Expected terminal call to pip install, got npm install"
+
+- [ ] **CLI flags:**
+  ```bash
+  python batch_runner.py eval test_cases.yaml       # String matching
+  python batch_runner.py eval test_cases.yaml -g    # + LLM grading
+  python batch_runner.py eval test_cases.yaml -r    # + Result comparison
+  python batch_runner.py eval test_cases.yaml -v    # Verbose (show responses)
+  ```
+
+**Files to modify:** `batch_runner.py`, new `evals/test_cases.py`, new `evals/grader.py`
+
+---
+
+*Last updated: $(date +%Y-%m-%d)* 🤖

atropos/Dockerfile

@@ -0,0 +1,41 @@
+# Dockerfile for atropos-agent sandbox server
+# Runs inside Nomad containers to handle tool execution
+# Includes bubblewrap for namespace-based slot isolation
+
+FROM python:3.11-slim
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    # Bubblewrap for namespace isolation
+    bubblewrap \
+    # `script` for PTY allocation (used for stable tmux+asciinema startup)
+    util-linux \
+    # Git for SWE-style tasks (cloning repos)
+    git \
+    # tmux for stateful terminal sessions (Phase 4.7+)
+    tmux \
+    # Common tools agents might need
+    curl \
+    wget \
+    jq \
+    # Cleanup
+    && rm -rf /var/lib/apt/lists/*
+
+# Install Python dependencies (sandbox server + optional terminal recording)
+RUN pip install --no-cache-dir aiohttp asciinema
+
+# Copy the sandbox server
+COPY sandbox_server.py /app/sandbox_server.py
+
+WORKDIR /app
+
+# Create data directory for slot workspaces
+RUN mkdir -p /data
+
+# Verify bubblewrap is installed and working
+RUN bwrap --version
+
+EXPOSE 8080
+
+# Default command - can be overridden by Nomad job spec
+CMD ["python", "sandbox_server.py", "--port", "8080", "--slots", "10", "--data-dir", "/data"]

atropos/__init__.py

@@ -0,0 +1,46 @@
+"""
+Atropos integration for Hermes-Agent.
+
+This package is intentionally optional: Hermes-Agent should work without Atropos.
+If you import anything from `atropos.*` without having `atroposlib` installed,
+we raise a clear error with install instructions.
+
+Install (recommended, from repo checkout):
+  uv sync --extra atropos
+
+Or (pip / editable):
+  pip install -e '.[atropos]'
+"""
+
+from __future__ import annotations
+
+
+def _require_atroposlib() -> None:
+    try:
+        import atroposlib  # noqa: F401
+    except ModuleNotFoundError as exc:  # pragma: no cover
+        raise ModuleNotFoundError(
+            "Hermes-Agent Atropos integration requires `atroposlib`, but it is not installed.\n"
+            "Install it with:\n"
+            "  uv sync --extra atropos\n"
+            "or:\n"
+            "  pip install -e '.[atropos]'\n"
+        ) from exc
+
+
+_require_atroposlib()
+
+# Re-export the most commonly used pieces for convenience.
+from .agent import AgentConfig, AgentResult, AgentStep, AtroposAgent, SequenceData  # noqa: E402
+from .envs import AgentEnv, AgentEnvConfig  # noqa: E402
+
+__all__ = [
+    "AtroposAgent",
+    "AgentConfig",
+    "AgentResult",
+    "AgentStep",
+    "SequenceData",
+    "AgentEnv",
+    "AgentEnvConfig",
+]
+

atropos/agent/__init__.py

@@ -0,0 +1,15 @@
+"""
+Agent abstractions for atropos-agent.
+
+Provides the core AtroposAgent class for running ReACT-style agent loops.
+"""
+
+from .atropos_agent import AgentConfig, AgentResult, AgentStep, AtroposAgent, SequenceData
+
+__all__ = [
+    "AtroposAgent",
+    "AgentConfig",
+    "AgentResult",
+    "AgentStep",
+    "SequenceData",
+]

atropos/agent/atropos_agent.py

@@ -0,0 +1,850 @@
+"""
+ReACT-style agent implementation for atropos-agent.
+
+This module provides the core AtroposAgent class that implements a basic
+Reason-Act-Observe loop with tool calling capabilities.
+
+Uses ManagedServer from atroposlib for automatic token/logprob tracking,
+making trajectories ready for RL training.
+
+The agent uses Hermes-style XML tags for tool calls:
+- <think>...</think> for reasoning
+- <tool_call>{"name": "...", "arguments": {...}}</tool_call> for actions
+- <tool_response>...</tool_response> for observations
+"""
+
+import asyncio
+import os
+import json
+import time
+from contextlib import asynccontextmanager
+from dataclasses import dataclass, field
+from uuid import uuid4
+from typing import Any, AsyncGenerator, Awaitable, Callable, Dict, List, Optional, Union
+
+from dotenv import load_dotenv
+import httpx
+
+from ..tools import ToolCall, ToolRegistry, ToolResult
+from atroposlib.envs.server_handling.managed_server import ManagedServer
+
+load_dotenv()
+
+
+# Default system prompt with tool calling instructions.
+AGENT_SYSTEM_PROMPT = """You are a deep thinking AI. You MUST enclose your internal reasoning inside <think>...</think> tags.
+
+You are a function calling AI model.
+
+You are provided with function signatures within <tools></tools> XML tags.
+You must call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.
+You can ONLY respond without a tool call if you are totally certain you have the final answer to the user's question or task
+After calling & executing a function, you will be provided with function results within <tool_response></tool_response> XML tags.
+
+Here are the available tools:
+<tools>
+{tools_json}
+</tools>
+
+Use the following JSON schema for each tool call you will make:
+{"title": "FunctionCall", "type": "object", "properties": {"name": {"title": "Name", "type": "string"}, "arguments": {"title": "Arguments", "type": "object"}}, "required": ["name", "arguments"]}
+
+## REQUIRED TOOL FORMAT
+
+When you decide to call a tool, your assistant message MUST be:
+1) exactly one <think>...</think> block, followed by
+2) one or more <tool_call>...</tool_call> blocks,
+and NOTHING else in that message.
+
+If you need to explain anything, put it inside <think>. Do NOT write natural language outside <think> or <tool_call>.
+
+For each function call return a JSON object with function name and arguments within <tool_call></tool_call> XML tags as follows:
+<tool_call>
+{"name": "<function-name>", "arguments": {"arg1": "value1"}}
+</tool_call>
+
+Each <tool_call> must be on its own and contain ONLY the JSON object (no extra text).
+The JSON inside <tool_call> MUST be valid JSON with double quotes.
+
+Do NOT output <tool_response> in an assistant message.
+
+After you receive tool results, you may either call more tools (same required format) or provide the final answer.
+When providing the final answer, do NOT include any <tool_call> blocks.
+
+## TERMINAL TOOL NOTES
+
+- Commands execute under POSIX `/bin/sh` (not bash).
+- Each tool call runs in a fresh shell: environment changes (like `cd` or venv activation) do not persist across tool calls.
+- Avoid bash-only features like `source`, `[[ ... ]]`, or process substitution.
+- Prefer explicit venv usage:
+  - `python -m venv .venv && . .venv/bin/activate && python -m pip install -e .` (POSIX `.` activation), or
+  - `.venv/bin/python -m pip install -e .` (no activation required).
+
+## ICL (examples)
+
+User: Show the current directory.
+Assistant:
+<think>I should run pwd.</think>
+<tool_call>
+{"name": "terminal", "arguments": {"command": "pwd"}}
+</tool_call>
+User: <tool_response>{"success": true, "output": "/tmp\\n"}</tool_response>
+Assistant: /tmp
+
+User: List files, then count them.
+Assistant:
+<think>I should count files.</think>
+<tool_call>
+{"name": "terminal", "arguments": {"command": "ls -1 | wc -l"}}
+</tool_call>
+User: <tool_response>{"success": true, "output": "3\\n"}</tool_response>
+Assistant: 3
+
+User: Run pwd, then print ok (two tool calls).
+Assistant:
+<think>I should run two commands.</think>
+<tool_call>
+{"name": "terminal", "arguments": {"command": "pwd"}}
+</tool_call>
+<tool_call>
+{"name": "terminal", "arguments": {"command": "echo ok"}}
+</tool_call>
+User: <tool_response>{"success": true, "output": "/tmp\\n"}</tool_response>
+User: <tool_response>{"success": true, "output": "ok\\n"}</tool_response>
+Assistant: ok
+"""
+
+
+@dataclass
+class AgentConfig:
+    """Configuration for the AtroposAgent."""
+    
+    # Generation parameters
+    temperature: Optional[float] = 0.7
+    # Default to "let the backend decide" (important for tool-tag completions that may be longer).
+    max_tokens: Optional[int] = None
+    
+    # Agent behavior
+    max_steps: int = 50
+    system_prompt: Optional[str] = None
+    tool_delay_s: float = 0.0
+    
+    # Working directory for tools
+    working_dir: Optional[str] = None
+
+
+@dataclass
+class SequenceData:
+    """Token/logprob data from a single completion."""
+    
+    full_text: str
+    tokens: List[int]
+    masked_tokens: List[int]  # -100 for prompt, actual IDs for completion
+    logprobs: List[float]  # 1.0 for prompt, actual values for completion
+    metadata: Optional[Dict[str, Any]] = None
+    
+    @classmethod
+    def from_sequence_node(cls, node) -> "SequenceData":
+        """Create from a ManagedServer SequenceNode."""
+        return cls(
+            full_text=node.full_text,
+            tokens=node.tokens,
+            masked_tokens=node.masked_tokens,
+            logprobs=node.logprobs,
+            metadata=getattr(node, "metadata", None),
+        )
+
+
+@dataclass
+class AgentStep:
+    """A single step in the agent's trajectory."""
+    
+    step_number: int
+    assistant_message: str
+    tool_calls: List[ToolCall] = field(default_factory=list)
+    tool_results: List[ToolResult] = field(default_factory=list)
+    sequence_data: Optional[SequenceData] = None  # Token data from this step
+    
+    @property
+    def has_tool_calls(self) -> bool:
+        return len(self.tool_calls) > 0
+
+
+@dataclass
+class AgentResult:
+    """Result of running an agent trajectory."""
+    
+    success: bool
+    final_response: str
+    steps: List[AgentStep] = field(default_factory=list)
+    total_tokens: int = 0
+    error: Optional[str] = None
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    
+    # Full trajectory token data for RL training
+    trajectory_data: Optional[SequenceData] = None
+    
+    @property
+    def num_steps(self) -> int:
+        return len(self.steps)
+    
+    @property
+    def total_tool_calls(self) -> int:
+        return sum(len(step.tool_calls) for step in self.steps)
+    
+    def to_messages(self) -> List[Dict[str, str]]:
+        """Convert trajectory to messages format for logging."""
+        messages = []
+        for step in self.steps:
+            messages.append({"role": "assistant", "content": step.assistant_message})
+            if step.tool_results:
+                # Combine all tool responses
+                responses = "\n".join(r.to_xml() for r in step.tool_results)
+                messages.append({"role": "user", "content": responses})
+        return messages
+    
+    def to_scored_data(self, score: float) -> Optional[Dict[str, Any]]:
+        """
+        Convert to format suitable for ScoredDataGroup.
+        
+        Args:
+            score: The score for this trajectory
+            
+        Returns:
+            Dict with tokens, masks, scores suitable for training, or None if no data
+        """
+        if self.trajectory_data is None:
+            return None
+        
+        return {
+            "tokens": self.trajectory_data.tokens,
+            "masks": self.trajectory_data.masked_tokens,
+            "scores": score,
+            "logprobs": self.trajectory_data.logprobs,
+        }
+
+
+class AtroposAgent:
+    """
+    A ReACT-style agent that uses LLMs with tool calling.
+    
+    This implementation wraps ManagedServer for automatic token/logprob tracking,
+    making trajectories ready for RL training.
+    
+    Example:
+        # `server` may be an Atropos `ServerManager` (recommended) or a single `APIServer`.
+        # In practice, environments usually construct this via `BaseEnv`.
+        server = ...
+        tools = ToolRegistry()
+        tools.register(BashTool())
+        
+        agent = AtroposAgent(server=server, tools=tools)
+        result = await agent.run("List the files in the current directory")
+        
+        # Access token data for training
+        if result.trajectory_data:
+            print(f"Tokens: {result.trajectory_data.tokens}")
+            print(f"Masked: {result.trajectory_data.masked_tokens}")
+    """
+    
+    def __init__(
+        self,
+        server,  # ServerManager or APIServer
+        tools: Optional[ToolRegistry] = None,
+        config: Optional[AgentConfig] = None,
+        tokenizer: Optional[Any] = None,
+        execute_tool: Optional[Callable[[ToolCall], Awaitable[ToolResult]]] = None,
+    ):
+        self.server = server
+        self.tools = tools or ToolRegistry()
+        self.config = config or AgentConfig()
+        self.tokenizer = tokenizer or getattr(server, "tokenizer", None)
+        self.execute_tool = execute_tool or self.tools.execute
+
+    @asynccontextmanager
+    async def _managed(self) -> AsyncGenerator[Any, None]:
+        """
+        Yield a ManagedServer-like object.
+
+        - If `self.server` is a ServerManager, use its `managed_server()` context manager.
+        - If `self.server` is a single APIServer, wrap it in `ManagedServer` directly.
+        """
+        if os.getenv("ATROPOS_BYPASS_MANAGED_SERVER") == "1":
+            yield _DirectChatCompletionClient(server=self.server)
+            return
+        if hasattr(self.server, "managed_server"):
+            async with self.server.managed_server(tokenizer=self.tokenizer) as managed:
+                yield managed
+        else:
+            managed = ManagedServer(server=self.server, tokenizer=self.tokenizer)
+            try:
+                yield managed
+            finally:
+                managed.reset()
+    
+    def _build_system_prompt(self) -> str:
+        """Build the system prompt with tool descriptions."""
+        if self.config.system_prompt:
+            return self.config.system_prompt
+
+        tools_json = self.tools.get_prompt_tool_definitions_json()
+        # Avoid `str.format()` here because the prompt contains many literal `{}` braces
+        # in JSON examples; we only want to substitute the single `{tools_json}` token.
+        return AGENT_SYSTEM_PROMPT.replace("{tools_json}", tools_json)
+
+    def _infer_server_model_for_debug(self) -> Optional[str]:
+        """
+        Best-effort inference of the configured model name for debug payload saving.
+
+        ManagedServer/server_manager typically injects `model` internally, so `chat_kwargs`
+        may not contain it. For replaying saved payloads via curl, it's useful to persist it.
+        """
+        servers = getattr(self.server, "servers", None)
+        if isinstance(servers, list) and servers:
+            s0 = servers[0]
+            cfg = getattr(s0, "config", None)
+            model = getattr(cfg, "model_name", None) or getattr(s0, "model_name", None)
+            if isinstance(model, str) and model:
+                return model
+        model = getattr(self.server, "model_name", None) or getattr(self.server, "model", None)
+        if isinstance(model, str) and model:
+            return model
+        return None
+
+    def _infer_server_base_url_for_debug(self) -> Optional[str]:
+        """
+        Best-effort inference of the configured base_url for debug logging.
+
+        This is helpful when diagnosing hangs / retries at the transport layer.
+        """
+        servers = getattr(self.server, "servers", None)
+        if isinstance(servers, list) and servers:
+            s0 = servers[0]
+            cfg = getattr(s0, "config", None)
+            base_url = getattr(cfg, "base_url", None) or getattr(s0, "base_url", None)
+            if isinstance(base_url, str) and base_url:
+                return base_url
+        base_url = getattr(self.server, "base_url", None)
+        if isinstance(base_url, str) and base_url:
+            return base_url
+        return None
+
+    def _extract_response_metadata(self, response: Any) -> Dict[str, Any]:
+        """
+        Extract lightweight, JSON-serializable metadata from an OpenAI-style response.
+
+        This is useful for debugging training runs, especially when ManagedServer state
+        tracking is unavailable (e.g. OpenAI-compatible chat endpoints).
+        """
+        meta: Dict[str, Any] = {}
+        try:
+            rid = getattr(response, "id", None)
+            if isinstance(rid, str) and rid:
+                meta["id"] = rid
+            model = getattr(response, "model", None)
+            if isinstance(model, str) and model:
+                meta["model"] = model
+            created = getattr(response, "created", None)
+            if isinstance(created, int):
+                meta["created"] = created
+            system_fingerprint = getattr(response, "system_fingerprint", None)
+            if isinstance(system_fingerprint, str) and system_fingerprint:
+                meta["system_fingerprint"] = system_fingerprint
+
+            choices = getattr(response, "choices", None)
+            if isinstance(choices, list) and choices:
+                fr = getattr(choices[0], "finish_reason", None)
+                if isinstance(fr, str) and fr:
+                    meta["finish_reason"] = fr
+
+            usage = getattr(response, "usage", None)
+            if usage is not None:
+                if hasattr(usage, "model_dump"):
+                    meta["usage"] = usage.model_dump()
+                elif isinstance(usage, dict):
+                    meta["usage"] = usage
+        except Exception:
+            pass
+        return meta
+
+    def _debug_dump_request(self, *, step_num: int, chat_kwargs: Dict[str, Any]) -> None:
+        if os.getenv("ATROPOS_DEBUG_AGENT_REQUEST") != "1":
+            return
+        try:
+            # Avoid dumping megabytes by default; messages can be huge.
+            meta = {
+                "step": step_num,
+                "base_url": self._infer_server_base_url_for_debug(),
+                "model": chat_kwargs.get("model") or self._infer_server_model_for_debug(),
+                "chat_kwargs_keys": sorted(list(chat_kwargs.keys())),
+                "n": chat_kwargs.get("n"),
+                "max_tokens": chat_kwargs.get("max_tokens"),
+                "temperature": chat_kwargs.get("temperature"),
+                "num_messages": len(chat_kwargs.get("messages") or []),
+            }
+            print("\n=== ATROPOS_DEBUG_AGENT_REQUEST ===", flush=True)
+            print(meta, flush=True)
+
+            if os.getenv("ATROPOS_DEBUG_AGENT_REQUEST_FULL") == "1":
+                payload = dict(chat_kwargs)
+                # Make the payload more legible and less huge.
+                try:
+                    dumped = json.dumps(payload, ensure_ascii=False, indent=2)
+                except Exception:
+                    dumped = repr(payload)
+                print("\n=== ATROPOS_DEBUG_AGENT_REQUEST_FULL ===", flush=True)
+                print(dumped[:200_000], flush=True)
+
+            # Optional: save the FULL request payload to disk (no truncation).
+            save_dir = os.getenv("ATROPOS_DEBUG_AGENT_REQUEST_SAVE_DIR")
+            if save_dir:
+                os.makedirs(save_dir, exist_ok=True)
+                payload: Dict[str, Any] = dict(chat_kwargs)
+                if "model" not in payload:
+                    model = self._infer_server_model_for_debug()
+                    if model:
+                        payload["model"] = model
+                # Use a unique filename so parallel trajectories don't clobber each other.
+                fname = os.path.join(
+                    save_dir,
+                    f"atropos_agent_request_step{step_num}_{int(time.time()*1000)}_{os.getpid()}_{uuid4().hex}.json",
+                )
+                with open(fname, "w", encoding="utf-8") as f:
+                    json.dump(payload, f, ensure_ascii=False, indent=2)
+                print(f"[AtroposAgent] saved request payload: {fname}", flush=True)
+        except Exception:
+            return
+
+    def _debug_dump_response(self, *, step_num: int, response: Any) -> None:
+        if os.getenv("ATROPOS_DEBUG_AGENT_RESPONSE") != "1":
+            return
+        print("\n=== ATROPOS_DEBUG_AGENT_RESPONSE ===", flush=True)
+        print({"step": step_num, "type": type(response).__name__}, flush=True)
+        try:
+            dumped = response.model_dump()  # openai pydantic model
+        except Exception:
+            dumped = getattr(response, "__dict__", {"repr": repr(response)})
+        # Keep the dump bounded; we only need enough to see the assistant message content.
+        text = str(dumped)
+        print(text[:200_000], flush=True)
+
+    async def _chat_completion_with_debug(
+        self, *, managed: Any, step_num: int, chat_kwargs: Dict[str, Any]
+    ) -> Any:
+        """
+        Call `managed.chat_completion()` with optional timeout + richer failure logging.
+
+        Debug env vars:
+        - `ATROPOS_AGENT_CHAT_TIMEOUT_S`: if set, wraps the await in `asyncio.wait_for`.
+        - `ATROPOS_DEBUG_AGENT_WAIT_EVERY_S`: if set, prints a heartbeat while waiting.
+        """
+        # Hard guardrail: never allow a single chat completion to block for too long.
+        # This is essential for RL data-gen stability; long hangs should be treated as failures (score=0).
+        timeout_s_raw = os.getenv("ATROPOS_AGENT_CHAT_TIMEOUT_S")
+        timeout_s_default = 240.0
+        timeout_s = float(timeout_s_raw) if timeout_s_raw else timeout_s_default
+        timeout_s = min(timeout_s, 240.0)
+
+        wait_every_raw = os.getenv("ATROPOS_DEBUG_AGENT_WAIT_EVERY_S")
+        wait_every_s = float(wait_every_raw) if wait_every_raw else None
+
+        async def _await_call() -> Any:
+            if not wait_every_s or wait_every_s <= 0:
+                return await managed.chat_completion(**chat_kwargs)
+
+            # Heartbeat mode: wait in chunks without cancelling the underlying request.
+            # NOTE: do NOT use `asyncio.wait_for(task, timeout=...)` here, because a timeout
+            # will cancel the task and surface as `CancelledError` on the next loop.
+            task = asyncio.create_task(managed.chat_completion(**chat_kwargs))
+            t0 = time.perf_counter()
+            try:
+                while True:
+                    done, _pending = await asyncio.wait({task}, timeout=wait_every_s)
+                    if task in done:
+                        return task.result()
+
+                    waited = time.perf_counter() - t0
+                    print(
+                        f"[AtroposAgent] step={step_num} still waiting for chat_completion... ({waited:.1f}s)",
+                        flush=True,
+                    )
+            except asyncio.CancelledError:
+                task.cancel()
+                raise
+
+        try:
+            return await asyncio.wait_for(_await_call(), timeout=timeout_s)
+        except asyncio.TimeoutError as e:
+            print("\n=== ATROPOS_DEBUG_AGENT_CHAT_TIMEOUT ===", flush=True)
+            print({"step": step_num, "timeout_s": timeout_s}, flush=True)
+            raise RuntimeError(f"chat_completion timed out after {timeout_s:.1f}s") from e
+        except asyncio.CancelledError:
+            # Treat cancellation as a hard failure rather than crashing the whole env run.
+            # (Atropos/BaseEnv may cancel tasks during shutdown or retries.)
+            raise RuntimeError("chat_completion cancelled") from None
+        except Exception as e:
+            detail: Dict[str, Any] = {
+                "step": step_num,
+                "exc_type": type(e).__name__,
+                "exc_str": str(e),
+            }
+            if isinstance(e, httpx.HTTPStatusError):
+                try:
+                    detail["status_code"] = e.response.status_code
+                    detail["response_text"] = e.response.text[:20_000]
+                except Exception:
+                    pass
+            elif isinstance(e, httpx.RequestError):
+                detail["request"] = repr(getattr(e, "request", None))
+
+            print("\n=== ATROPOS_DEBUG_AGENT_CHAT_FAILURE ===", flush=True)
+            print(detail, flush=True)
+            raise
+
+    async def run(
+        self,
+        task: str,
+        initial_messages: Optional[List[Dict[str, str]]] = None,
+    ) -> AgentResult:
+        """
+        Run the agent on a task using ManagedServer for token tracking.
+        
+        Args:
+            task: The task/prompt for the agent
+            initial_messages: Optional additional context messages
+            
+        Returns:
+            AgentResult with the trajectory, final response, and token data
+        """
+        messages = [
+            {"role": "system", "content": self._build_system_prompt()},
+        ]
+        
+        if initial_messages:
+            messages.extend(initial_messages)
+        
+        messages.append({"role": "user", "content": task})
+        
+        steps = []
+        final_response = ""
+        final_node = None
+        final_prompt_messages: Optional[List[Dict[str, str]]] = None
+        last_node = None
+        last_prompt_messages: Optional[List[Dict[str, str]]] = None
+        last_response_text: str = ""
+        
+        # Use ManagedServer for automatic token tracking
+        async with self._managed() as managed:
+            for step_num in range(self.config.max_steps):
+                # ReACT loop iteration here, just call -> tools -> observe until done (no tools called)
+                try:
+                    # Keep a copy of the prompt messages used for this completion.
+                    # Useful for reconstructing tokens/masks when state tracking is unavailable.
+                    prompt_messages = list(messages)
+                    chat_kwargs: Dict[str, Any] = {"messages": messages, "n": 1}
+                    if self.config.max_tokens is not None:
+                        chat_kwargs["max_tokens"] = self.config.max_tokens
+                    if self.config.temperature is not None:
+                        chat_kwargs["temperature"] = self.config.temperature
+
+                    t_req = time.perf_counter()
+                    print(
+                        f"[AtroposAgent] step={step_num+1} chat_completion start "
+                        f"(messages={len(messages)}, max_tokens={self.config.max_tokens}, temp={self.config.temperature})",
+                        flush=True,
+                    )
+                    self._debug_dump_request(step_num=step_num + 1, chat_kwargs=chat_kwargs)
+                    response = await self._chat_completion_with_debug(
+                        managed=managed, step_num=step_num + 1, chat_kwargs=chat_kwargs
+                    )
+                    self._debug_dump_response(step_num=step_num + 1, response=response)
+                    response_meta = self._extract_response_metadata(response)
+                    print(
+                        f"[AtroposAgent] step={step_num+1} chat_completion done in {time.perf_counter() - t_req:.2f}s",
+                        flush=True,
+                    )
+                    
+                    current_node = None
+                    if hasattr(managed, "get_state"):
+                        state = managed.get_state()
+                        nodes = state.get("nodes", [])
+                        current_node = nodes[-1] if nodes else None
+                    
+                except Exception as e:
+                    return AgentResult(
+                        success=False,
+                        final_response="",
+                        steps=steps,
+                        error=f"Generation error: {str(e)}",
+                    )
+                
+                msg = response.choices[0].message
+                # Some OpenAI-compatible servers populate `message.reasoning` and leave `content=""`.
+                response_text = (msg.content or "") or (getattr(msg, "reasoning", None) or "")
+                tool_calls = ToolCall.parse_from_text(response_text)
+                last_node = current_node
+                last_prompt_messages = prompt_messages
+                last_response_text = response_text
+
+                step_sequence_data = SequenceData.from_sequence_node(current_node) if current_node else None
+                if step_sequence_data is None:
+                    if response_meta:
+                        # We still want metadata for debugging even if token/logprob state tracking is unavailable.
+                        step_sequence_data = SequenceData(
+                            full_text=response_text,
+                            tokens=[],
+                            masked_tokens=[],
+                            logprobs=[],
+                            metadata=response_meta,
+                        )
+                else:
+                    merged = dict(response_meta)
+                    node_meta = step_sequence_data.metadata
+                    if isinstance(node_meta, dict):
+                        merged.update(node_meta)
+                    step_sequence_data.metadata = merged or step_sequence_data.metadata
+                
+                step = AgentStep(
+                    step_number=step_num + 1,
+                    assistant_message=response_text,
+                    tool_calls=tool_calls,
+                    sequence_data=step_sequence_data,
+                )
+                
+                if not tool_calls:
+                    steps.append(step)
+                    final_response = response_text
+                    final_node = current_node
+                    final_prompt_messages = prompt_messages
+                    break
+                
+                messages.append({"role": "assistant", "content": response_text})
+                
+                tool_responses = []
+                for call in tool_calls:
+                    result = await self.execute_tool(call)
+                    step.tool_results.append(result)
+                    tool_responses.append(result.to_xml())
+                    if self.config.tool_delay_s > 0:
+                        await asyncio.sleep(self.config.tool_delay_s)
+                
+                steps.append(step)
+            
+                responses_text = "\n".join(tool_responses)
+                # Tool observations are represented as user content with Hermes-style tags.
+                # This is compatible with most OpenAI-compatible chat APIs and ensures
+                # tokenizers/chat templates include tool outputs during training.
+                messages.append({"role": "user", "content": responses_text})
+            
+            else:
+                # Reached max steps without completing
+                # Return a failure result but include the last observed completion so callers can
+                # record the trajectory (score=0) without triggering retries.
+                final_response = last_response_text or final_response
+                final_node = last_node
+                final_prompt_messages = last_prompt_messages
+                trajectory_data = None
+                if final_node:
+                    trajectory_data = SequenceData.from_sequence_node(final_node)
+                elif final_prompt_messages is not None and self.tokenizer is not None:
+                    if hasattr(self.tokenizer, "apply_chat_template"):
+                        prompt_text = self.tokenizer.apply_chat_template(
+                            final_prompt_messages, tokenize=False, add_generation_prompt=True
+                        )
+                        prompt_tokens = self.tokenizer.encode(prompt_text, add_special_tokens=False)
+                    else:
+                        prompt_text = "\n".join([f"{m['role']}: {m['content']}" for m in final_prompt_messages])
+                        prompt_tokens = self.tokenizer.encode(prompt_text, add_special_tokens=True)
+                    output_tokens = self.tokenizer.encode(final_response, add_special_tokens=False)
+                    tokens = prompt_tokens + output_tokens
+                    masked_tokens = ([-100] * len(prompt_tokens)) + output_tokens
+                    logprobs = ([1.0] * len(prompt_tokens)) + ([0.0] * len(output_tokens))
+                    trajectory_data = SequenceData(
+                        full_text=f"{prompt_text}{final_response}",
+                        tokens=tokens,
+                        masked_tokens=masked_tokens,
+                        logprobs=logprobs,
+                    )
+                # Preserve response metadata (if any) even on failure trajectories.
+                try:
+                    if trajectory_data is not None and steps:
+                        last_step = steps[-1]
+                        if last_step.sequence_data and isinstance(last_step.sequence_data.metadata, dict):
+                            trajectory_data.metadata = dict(last_step.sequence_data.metadata)
+                except Exception:
+                    pass
+                return AgentResult(
+                    success=False,
+                    final_response=final_response,
+                    steps=steps,
+                    error=f"Reached maximum steps ({self.config.max_steps})",
+                    trajectory_data=trajectory_data,
+                )
+        
+        # Build result with trajectory data
+        trajectory_data = None
+        if final_node:
+            trajectory_data = SequenceData.from_sequence_node(final_node)
+        elif final_prompt_messages is not None and self.tokenizer is not None:
+            if hasattr(self.tokenizer, "apply_chat_template"):
+                prompt_text = self.tokenizer.apply_chat_template(
+                    final_prompt_messages, tokenize=False, add_generation_prompt=True
+                )
+                prompt_tokens = self.tokenizer.encode(prompt_text, add_special_tokens=False)
+            else:
+                prompt_text = "\n".join([f"{m['role']}: {m['content']}" for m in final_prompt_messages])
+                prompt_tokens = self.tokenizer.encode(prompt_text, add_special_tokens=True)
+            output_tokens = self.tokenizer.encode(final_response, add_special_tokens=False)
+            tokens = prompt_tokens + output_tokens
+            masked_tokens = ([-100] * len(prompt_tokens)) + output_tokens
+            logprobs = ([1.0] * len(prompt_tokens)) + ([0.0] * len(output_tokens))
+            trajectory_data = SequenceData(
+                full_text=f"{prompt_text}{final_response}",
+                tokens=tokens,
+                masked_tokens=masked_tokens,
+                logprobs=logprobs,
+            )
+
+        # Ensure trajectory_data carries the most recent metadata we observed (if any).
+        try:
+            if trajectory_data is not None and steps:
+                last_step = steps[-1]
+                if last_step.sequence_data and isinstance(last_step.sequence_data.metadata, dict):
+                    trajectory_data.metadata = dict(last_step.sequence_data.metadata)
+        except Exception:
+            pass
+        
+        return AgentResult(
+            success=True,
+            final_response=final_response,
+            steps=steps,
+            trajectory_data=trajectory_data,
+        )
+    
+    async def run_single_turn(
+        self,
+        messages: List[Dict[str, str]],
+        execute_tools: bool = True,
+    ) -> tuple[str, List[ToolResult], Optional[SequenceData]]:
+        """
+        Run a single turn of the agent (one LLM call + tool execution).
+        
+        This is useful for integration with BaseEnv where you want more
+        control over the loop.
+        
+        Args:
+            messages: The conversation history
+            execute_tools: Whether to execute parsed tool calls
+            
+        Returns:
+            Tuple of (response_text, tool_results, sequence_data)
+        """
+        async with self._managed() as managed:
+            chat_kwargs: Dict[str, Any] = {"messages": messages, "n": 1}
+            if self.config.max_tokens is not None:
+                chat_kwargs["max_tokens"] = self.config.max_tokens
+            if self.config.temperature is not None:
+                chat_kwargs["temperature"] = self.config.temperature
+
+            self._debug_dump_request(step_num=1, chat_kwargs=chat_kwargs)
+            response = await self._chat_completion_with_debug(managed=managed, step_num=1, chat_kwargs=chat_kwargs)
+            self._debug_dump_response(step_num=1, response=response)
+            
+            current_node = None
+            if hasattr(managed, "get_state"):
+                state = managed.get_state()
+                nodes = state.get("nodes", [])
+                current_node = nodes[-1] if nodes else None
+        
+        msg = response.choices[0].message
+        response_text = (msg.content or "") or (getattr(msg, "reasoning", None) or "")
+        tool_results = []
+        
+        if execute_tools:
+            tool_calls = ToolCall.parse_from_text(response_text)
+            for call in tool_calls:
+                result = await self.execute_tool(call)
+                tool_results.append(result)
+        
+        sequence_data = SequenceData.from_sequence_node(current_node) if current_node else None
+        
+        return response_text, tool_results, sequence_data
+
+
+class _DirectChatCompletionClient:
+    """
+    Minimal stand-in for ManagedServer that calls the OpenAI-compatible endpoint directly.
+
+    This is for isolating issues where `ManagedServer.chat_completion()` hangs or misbehaves.
+    It intentionally does NOT do token/logprob tracking.
+    """
+
+    def __init__(self, server: Any):
+        self._server = server
+
+    def _server_config(self) -> tuple[str, str, str]:
+        # ServerManager case: first configured server.
+        servers = getattr(self._server, "servers", None)
+        if isinstance(servers, list) and servers:
+            s0 = servers[0]
+            cfg = getattr(s0, "config", None)
+            base_url = getattr(cfg, "base_url", None) or getattr(s0, "base_url", None)
+            api_key = getattr(cfg, "api_key", None) or getattr(s0, "api_key", None)
+            model = getattr(cfg, "model_name", None) or getattr(s0, "model_name", None)
+            if isinstance(base_url, str) and isinstance(api_key, str) and isinstance(model, str):
+                return base_url.rstrip("/"), api_key, model
+
+        # APIServer-like fallback.
+        base_url = getattr(self._server, "base_url", None)
+        api_key = getattr(self._server, "api_key", None)
+        model = getattr(self._server, "model_name", None) or getattr(self._server, "model", None)
+        if isinstance(base_url, str) and isinstance(api_key, str) and isinstance(model, str):
+            return base_url.rstrip("/"), api_key, model
+
+        raise RuntimeError("Unable to resolve server base_url/api_key/model for direct chat completion")
+
+    async def chat_completion(self, *, messages: List[Dict[str, str]], n: int = 1, **kwargs: Any) -> Any:
+        base_url, api_key, model = self._server_config()
+        url = f"{base_url}/chat/completions"
+
+        payload: Dict[str, Any] = {
+            "model": model,
+            "messages": messages,
+            "n": n,
+        }
+        # Pass through common generation kwargs.
+        for k in ("max_tokens", "temperature", "top_p", "presence_penalty", "frequency_penalty", "stop"):
+            if k in kwargs and kwargs[k] is not None:
+                payload[k] = kwargs[k]
+
+        timeout_s = float(os.getenv("ATROPOS_DIRECT_REQUEST_TIMEOUT_S") or "120")
+        print(f"[AtroposAgent] DIRECT chat_completion POST {url} (timeout={timeout_s}s)", flush=True)
+        async with httpx.AsyncClient(timeout=timeout_s) as client:
+            resp = await client.post(
+                url,
+                headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
+                json=payload,
+            )
+            resp.raise_for_status()
+            data = resp.json()
+
+        # Return a very small object compatible with the code paths that read
+        # `response.choices[0].message.content`.
+        class _Msg:
+            def __init__(self, d: Dict[str, Any]):
+                self.content = d.get("content")
+                self.reasoning = d.get("reasoning")
+
+        class _Choice:
+            def __init__(self, d: Dict[str, Any]):
+                self.message = _Msg(d.get("message") or {})
+
+        class _Resp:
+            def __init__(self, d: Dict[str, Any]):
+                self._d = d
+                self.choices = [_Choice(c) for c in (d.get("choices") or [])]
+
+            def model_dump(self) -> Dict[str, Any]:
+                return self._d
+
+        return _Resp(data)

atropos/api/__init__.py

@@ -0,0 +1,6 @@
+"""
+FastAPI services for atropos-agent.
+
+- tool_executor_server: queued/batched sandbox tool execution (Phase 4)
+"""
+

atropos/api/tool_executor_server.py

@@ -0,0 +1,254 @@
+"""
+Tool Executor API (Phase 4)
+
+This service provides a queued, batched execution layer on top of a ToolBackend.
+It mirrors the stateful FastAPI + app.state pattern used in:
+  atropos/atroposlib/api/server.py
+
+Run (dev):
+  uv run uvicorn atropos_agent.api.tool_executor_server:app --host 0.0.0.0 --port 9001
+"""
+
+from __future__ import annotations
+
+import os
+from typing import Any, Dict, Optional
+from pathlib import Path
+
+from fastapi import FastAPI, Header, HTTPException, status
+from pydantic import BaseModel, Field
+
+from ..backends.nomad_backend import NomadBackendConfig, NomadToolBackend
+from ..tools import ToolRegistry, build_tool_registry
+from ..tools.base import (
+    ArtifactArchiveRequestPayload,
+    ArtifactArchiveResponsePayload,
+    ArtifactListRequestPayload,
+    ArtifactListResponsePayload,
+    ArtifactReadRequestPayload,
+    ArtifactReadResponsePayload,
+    ToolExecutorExecuteRequest,
+    ToolExecutorReleaseRequest,
+    ToolResultPayload,
+)
+from ..tools.tool_executor import ToolExecutor, ToolExecutorConfig
+
+
+class ToolExecutorServerConfig(BaseModel):
+    nomad_address: str = Field(default="http://localhost:4646")
+    job_id: str = Field(default="atropos-sandbox-tool-executor")
+    image: str = Field(default="atropos-sandbox:local")
+    slots_per_container: int = Field(default=10)
+    min_containers: int = Field(default=1)
+    max_containers: int = Field(default=10)
+    privileged: bool = Field(default=False)
+    acquire_timeout_s: float = Field(default=30.0)
+
+    batch_window_ms: int = Field(default=20)
+    max_batch_size: int = Field(default=200)
+    allow_network: bool = Field(default=True)
+
+    tool_server_url: Optional[str] = Field(default=None)
+    tool_server_token: Optional[str] = Field(default=None)
+
+    token: Optional[str] = Field(default=None, description="Bearer token required for requests (optional in dev).")
+
+    purge_job_on_shutdown: bool = Field(default=True)
+
+    @classmethod
+    def from_env(cls) -> "ToolExecutorServerConfig":
+        # In dev, prefer loading secrets/config from the repo-local `.env` (not committed).
+        try:
+            from dotenv import load_dotenv  # type: ignore
+        except Exception:  # pragma: no cover
+            load_dotenv = None  # type: ignore[assignment]
+        if load_dotenv is not None:
+            env_path = Path(__file__).resolve().parents[2] / ".env"
+            if env_path.exists():
+                load_dotenv(dotenv_path=env_path)
+
+        def _get_bool(name: str, default: bool) -> bool:
+            raw = os.getenv(name)
+            if raw is None:
+                return default
+            return raw.strip().lower() in {"1", "true", "yes", "y", "on"}
+
+        return cls(
+            nomad_address=os.getenv("TOOL_EXECUTOR_NOMAD_ADDRESS", "http://localhost:4646"),
+            job_id=os.getenv("TOOL_EXECUTOR_JOB_ID", "atropos-sandbox-tool-executor"),
+            image=os.getenv("TOOL_EXECUTOR_IMAGE", "atropos-sandbox:local"),
+            slots_per_container=int(os.getenv("TOOL_EXECUTOR_SLOTS", "10")),
+            min_containers=int(os.getenv("TOOL_EXECUTOR_MIN_CONTAINERS", "1")),
+            max_containers=int(os.getenv("TOOL_EXECUTOR_MAX_CONTAINERS", "10")),
+            privileged=_get_bool("TOOL_EXECUTOR_PRIVILEGED", False),
+            acquire_timeout_s=float(os.getenv("TOOL_EXECUTOR_ACQUIRE_TIMEOUT_S", "30.0")),
+            batch_window_ms=int(os.getenv("TOOL_EXECUTOR_BATCH_WINDOW_MS", "20")),
+            max_batch_size=int(os.getenv("TOOL_EXECUTOR_MAX_BATCH_SIZE", "200")),
+            allow_network=_get_bool("TOOL_EXECUTOR_ALLOW_NETWORK", True),
+            tool_server_url=os.getenv("TOOL_EXECUTOR_TOOL_SERVER_URL") or None,
+            tool_server_token=os.getenv("TOOL_EXECUTOR_TOOL_SERVER_TOKEN") or None,
+            token=os.getenv("TOOL_EXECUTOR_TOKEN") or None,
+            purge_job_on_shutdown=_get_bool("TOOL_EXECUTOR_PURGE_JOB_ON_SHUTDOWN", True),
+        )
+
+
+app = FastAPI(title="Atropos-Agent Tool Executor")
+
+
+@app.get("/")
+async def root() -> Dict[str, str]:
+    return {"message": "Atropos-Agent Tool Executor"}
+
+
+def _check_auth(cfg: ToolExecutorServerConfig, authorization: Optional[str]) -> None:
+    if not cfg.token:
+        return
+    if not authorization:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing Authorization header")
+    if not authorization.lower().startswith("bearer "):
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid Authorization header")
+    token = authorization.split(" ", 1)[1].strip()
+    if token != cfg.token:
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Invalid token")
+
+
+@app.on_event("startup")
+async def _startup() -> None:
+    cfg = ToolExecutorServerConfig.from_env()
+
+    # Default to Atropos "full" tool surface: sandbox + external (if tool_server_url provided).
+    tools: ToolRegistry = build_tool_registry(
+        enabled_toolsets=["full"],
+        disabled_toolsets=None,
+        tool_server_url=cfg.tool_server_url,
+    )
+
+    backend = NomadToolBackend(
+        NomadBackendConfig(
+            nomad_address=cfg.nomad_address,
+            sandbox_job_id=cfg.job_id,
+            sandbox_image=cfg.image,
+            slots_per_container=cfg.slots_per_container,
+            min_containers=cfg.min_containers,
+            max_containers=cfg.max_containers,
+            privileged=cfg.privileged,
+            acquire_timeout_s=cfg.acquire_timeout_s,
+            purge_job_on_start=False,
+        )
+    )
+    await backend.start()
+
+    executor = ToolExecutor(
+        backend=backend,
+        tools=tools,
+        config=ToolExecutorConfig(
+            batch_window_ms=cfg.batch_window_ms,
+            max_batch_size=cfg.max_batch_size,
+            allow_network=cfg.allow_network,
+            tool_server_url=cfg.tool_server_url,
+            tool_server_token=cfg.tool_server_token,
+        ),
+    )
+    await executor.start()
+
+    app.state.cfg = cfg
+    app.state.backend = backend
+    app.state.executor = executor
+
+
+@app.on_event("shutdown")
+async def _shutdown() -> None:
+    executor: Optional[ToolExecutor] = getattr(app.state, "executor", None)
+    backend: Optional[NomadToolBackend] = getattr(app.state, "backend", None)
+    cfg: Optional[ToolExecutorServerConfig] = getattr(app.state, "cfg", None)
+
+    if executor is not None:
+        await executor.close()
+
+    if backend is not None:
+        await backend.stop(purge=bool(cfg.purge_job_on_shutdown) if cfg else False)
+
+
+@app.get("/health")
+async def health() -> Dict[str, Any]:
+    return {"status": "ok"}
+
+
+@app.get("/status")
+async def status_endpoint() -> Dict[str, Any]:
+    executor: ToolExecutor = app.state.executor
+    backend: NomadToolBackend = app.state.backend
+
+    return {
+        "queue_size": executor.queue_size(),
+        "total_requests": executor.total_requests,
+        "total_errors": executor.total_errors,
+        "pool": backend.get_stats(),
+    }
+
+
+@app.post("/execute", response_model=ToolResultPayload)
+async def execute_tool(
+    req: ToolExecutorExecuteRequest,
+    authorization: Optional[str] = Header(default=None),
+    status_code: int = status.HTTP_200_OK,  # noqa: B008
+) -> ToolResultPayload:
+    cfg: ToolExecutorServerConfig = app.state.cfg
+    _check_auth(cfg, authorization)
+
+    executor: ToolExecutor = app.state.executor
+    result = await executor.execute(
+        trajectory_id=req.trajectory_id,
+        call=req.tool.to_tool_call(),
+        timeout_s=req.timeout_s,
+    )
+    return ToolResultPayload.from_tool_result(result)
+
+
+@app.post("/release")
+async def release_trajectory(
+    req: ToolExecutorReleaseRequest,
+    authorization: Optional[str] = Header(default=None),
+) -> Dict[str, Any]:
+    cfg: ToolExecutorServerConfig = app.state.cfg
+    _check_auth(cfg, authorization)
+
+    executor: ToolExecutor = app.state.executor
+    await executor.release_trajectory(req.trajectory_id, reset_workspace=req.reset_workspace)
+    return {"status": "ok"}
+
+
+@app.post("/artifacts/read", response_model=ArtifactReadResponsePayload)
+async def artifacts_read(
+    req: ArtifactReadRequestPayload,
+    authorization: Optional[str] = Header(default=None),
+) -> ArtifactReadResponsePayload:
+    cfg: ToolExecutorServerConfig = app.state.cfg
+    _check_auth(cfg, authorization)
+
+    executor: ToolExecutor = app.state.executor
+    return await executor.read_artifact(req)
+
+
+@app.post("/artifacts/list", response_model=ArtifactListResponsePayload)
+async def artifacts_list(
+    req: ArtifactListRequestPayload,
+    authorization: Optional[str] = Header(default=None),
+) -> ArtifactListResponsePayload:
+    cfg: ToolExecutorServerConfig = app.state.cfg
+    _check_auth(cfg, authorization)
+
+    executor: ToolExecutor = app.state.executor
+    return await executor.list_artifacts(req)
+
+
+@app.post("/artifacts/archive", response_model=ArtifactArchiveResponsePayload)
+async def artifacts_archive(
+    req: ArtifactArchiveRequestPayload,
+    authorization: Optional[str] = Header(default=None),
+) -> ArtifactArchiveResponsePayload:
+    cfg: ToolExecutorServerConfig = app.state.cfg
+    _check_auth(cfg, authorization)
+
+    executor: ToolExecutor = app.state.executor
+    return await executor.archive_artifacts(req)

atropos/api/tool_server.py

@@ -0,0 +1,140 @@
+"""
+External ToolServer (Phase 4.5+).
+
+This server executes tools that must NOT run inside the sandbox, typically
+because they require credentials or access to external services.
+
+Run (dev):
+  uv run uvicorn atropos_agent.api.tool_server:app --host 0.0.0.0 --port 9002
+"""
+
+from __future__ import annotations
+
+import asyncio
+import os
+import inspect
+from typing import Any, Dict, List, Optional
+from pathlib import Path
+
+from fastapi import FastAPI, Header, HTTPException, status
+from pydantic import BaseModel, Field
+
+from ..tools import ToolRegistry, build_tool_registry
+from ..tools.base import ToolResultPayload, ToolServerExecuteRequest
+
+
+class ToolServerConfig(BaseModel):
+    token: Optional[str] = Field(
+        default=None,
+        description="Bearer token required for requests (optional in dev).",
+    )
+    max_concurrency: int = Field(default=16, ge=1, description="Max concurrent tool executions.")
+
+    @classmethod
+    def from_env(cls) -> "ToolServerConfig":
+        # In dev, prefer loading secrets from the repo-local `.env` (not committed).
+        try:
+            from dotenv import load_dotenv  # type: ignore
+        except Exception:  # pragma: no cover
+            load_dotenv = None  # type: ignore[assignment]
+        if load_dotenv is not None:
+            env_path = Path(__file__).resolve().parents[2] / ".env"
+            if env_path.exists():
+                load_dotenv(dotenv_path=env_path)
+
+        token = os.getenv("TOOL_SERVER_TOKEN") or None
+        max_concurrency = int(os.getenv("TOOL_SERVER_MAX_CONCURRENCY", "16"))
+        return cls(token=token, max_concurrency=max_concurrency)
+
+
+app = FastAPI(title="Atropos-Agent Tool Server")
+
+
+@app.get("/")
+async def root() -> Dict[str, str]:
+    return {"message": "Atropos-Agent Tool Server"}
+
+
+@app.on_event("startup")
+async def _startup() -> None:
+    cfg = ToolServerConfig.from_env()
+
+    # External-only registry. It will only include tools that are enabled by toolsets and
+    # whose Hermes requirements/keys are satisfied in this process.
+    tools: ToolRegistry = build_tool_registry(
+        enabled_toolsets=["all"],
+        disabled_toolsets=["terminal", "sandbox", "filesystem", "terminal_stateful", "default"],
+        tool_server_url="enabled",
+    )
+
+    app.state.cfg = cfg
+    app.state.tools = tools
+    app.state.semaphore = asyncio.Semaphore(cfg.max_concurrency)
+
+
+@app.get("/health")
+async def health() -> Dict[str, Any]:
+    return {"status": "ok"}
+
+
+@app.get("/tools")
+async def list_tools() -> Dict[str, Any]:
+    tools: ToolRegistry = app.state.tools
+    return {"tools": [s.to_dict() for s in tools.get_schemas()]}
+
+
+def _check_auth(cfg: ToolServerConfig, authorization: Optional[str]) -> None:
+    if not cfg.token:
+        return
+    if not authorization:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing Authorization header")
+    if not authorization.lower().startswith("bearer "):
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid Authorization header")
+    token = authorization.split(" ", 1)[1].strip()
+    if token != cfg.token:
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Invalid token")
+
+
+@app.post("/execute", response_model=ToolResultPayload)
+async def execute_tool(
+    req: ToolServerExecuteRequest,
+    authorization: Optional[str] = Header(default=None),
+) -> ToolResultPayload:
+    cfg: ToolServerConfig = app.state.cfg
+    _check_auth(cfg, authorization)
+
+    tools: ToolRegistry = app.state.tools
+    sem: asyncio.Semaphore = app.state.semaphore
+
+    tool = tools.get(req.tool.name)
+    if tool is None:
+        return ToolResultPayload(
+            success=False,
+            error=f"Unknown tool: {req.tool.name}",
+            uniq_id=req.tool.uniq_id,
+        )
+
+    async with sem:
+        try:
+            kwargs = dict(req.tool.arguments)
+            sig = inspect.signature(tool.execute).parameters
+            # Some tools can benefit from extra context.
+            if req.trajectory_id and "trajectory_id" in sig:
+                kwargs["trajectory_id"] = req.trajectory_id
+            if req.slot_id and "slot_id" in sig:
+                kwargs["slot_id"] = req.slot_id
+            if req.container_addr and "container_addr" in sig:
+                kwargs["container_addr"] = req.container_addr
+            if "task_id" in sig:
+                kwargs["task_id"] = req.trajectory_id
+            result = await tool.execute(**kwargs)
+        except Exception as e:
+            return ToolResultPayload(
+                success=False,
+                error=f"Tool execution error: {e}",
+                uniq_id=req.tool.uniq_id,
+            )
+
+    if result.uniq_id is None:
+        result.uniq_id = req.tool.uniq_id
+    return ToolResultPayload.from_tool_result(result)

atropos/backends/__init__.py

@@ -0,0 +1,27 @@
+from __future__ import annotations
+
+from typing import Any
+
+from .base import ToolBackend
+from .modal_backend import ModalSandboxConfig, ModalToolBackend
+from .nomad_backend import NomadBackendConfig, NomadToolBackend
+
+
+def create_tool_backend(cfg: Any) -> ToolBackend:
+    mode = str(getattr(cfg, "tool_pool_mode", "nomad")).strip().lower()
+    if mode == "nomad":
+        return NomadToolBackend(NomadBackendConfig.from_agent_env_config(cfg))
+    if mode == "modal":
+        return ModalToolBackend(ModalSandboxConfig.from_agent_env_config(cfg))
+    raise ValueError(f"Unknown tool_pool_mode: {mode}")
+
+
+__all__ = [
+    "ToolBackend",
+    "create_tool_backend",
+    "NomadBackendConfig",
+    "NomadToolBackend",
+    "ModalSandboxConfig",
+    "ModalToolBackend",
+]
+

atropos/backends/base.py

@@ -0,0 +1,89 @@
+"""
+Backend interfaces for AgentEnv tool execution.
+
+The goal of this module is to decouple ToolExecutor / AgentEnv from any single
+execution backend (Nomad/Docker today; Modal later).
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional, Protocol, Tuple
+
+from ..slots.executor import ExecutionResult
+from ..slots.slot import Slot
+
+
+class ToolBackend(Protocol):
+    """
+    Minimal interface required by ToolExecutor.
+
+    Backends provide:
+    - lifecycle (start/stop)
+    - slot acquisition/release (workspace affinity)
+    - batched tool execution across slots
+    - optional artifact helpers (for env verification / demos)
+    """
+
+    @property
+    def default_timeout_s(self) -> Optional[float]:
+        """Default sandbox execution timeout in seconds (if any)."""
+
+    async def start(self) -> None:
+        """Start the backend (provision workers/containers, health checks, etc)."""
+
+    async def stop(self, *, purge: bool = False) -> None:
+        """Stop the backend and optionally purge remote resources."""
+
+    async def acquire(self, trajectory_id: Optional[str] = None) -> Slot:
+        """Acquire a slot for a trajectory (workspace affinity)."""
+
+    async def release(self, slot: Slot, *, reset_workspace: bool = False) -> None:
+        """Release a slot back to the pool."""
+
+    async def execute_batch(
+        self,
+        requests: List[Tuple[Slot, str, Dict[str, Any]]],
+        *,
+        timeout_s: Optional[float] = None,
+    ) -> List[ExecutionResult]:
+        """Execute a batch of sandbox tool calls and return results in order."""
+
+    # ---------------------------------------------------------------------
+    # Optional artifact helpers (supported by the Nomad sandbox-server today)
+    # ---------------------------------------------------------------------
+
+    async def read_artifact(
+        self,
+        slot: Slot,
+        path: str,
+        *,
+        encoding: str = "text",
+        max_bytes: Optional[int] = None,
+        include_sha256: bool = False,
+        timeout_s: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        raise NotImplementedError
+
+    async def list_artifacts(
+        self,
+        slot: Slot,
+        path: str = ".",
+        *,
+        recursive: bool = False,
+        max_entries: Optional[int] = None,
+        timeout_s: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        raise NotImplementedError
+
+    async def archive_artifacts(
+        self,
+        slot: Slot,
+        path: str = ".",
+        *,
+        archive_format: str = "tar.gz",
+        max_bytes: Optional[int] = None,
+        max_entries: Optional[int] = None,
+        timeout_s: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        raise NotImplementedError
+

atropos/backends/nomad_backend.py

@@ -0,0 +1,156 @@
+"""
+Nomad/Docker tool backend.
+
+This backend is the current default for AgentEnv: it provisions a Nomad job
+running `sandbox_server.py` and multiplexes stateless slots inside each container.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any, Dict, List, Optional, Tuple
+
+from ..slots import Slot, SlotPool, SlotPoolConfig
+from ..slots.executor import ExecutionResult
+from .base import ToolBackend
+
+
+@dataclass(frozen=True)
+class NomadBackendConfig:
+    nomad_address: str
+    sandbox_job_id: str
+    sandbox_image: str
+    slots_per_container: int
+    min_containers: int
+    max_containers: int
+    privileged: bool
+    acquire_timeout_s: float
+    purge_job_on_start: bool
+    # Driver selection: "docker" or "singularity"
+    driver: str = "docker"
+    # Path to .sif file for singularity driver (required if driver="singularity")
+    singularity_image: Optional[str] = None
+
+    @classmethod
+    def from_agent_env_config(cls, cfg: Any) -> "NomadBackendConfig":
+        return cls(
+            nomad_address=str(getattr(cfg, "nomad_address")),
+            sandbox_job_id=str(getattr(cfg, "sandbox_job_id")),
+            sandbox_image=str(getattr(cfg, "sandbox_image")),
+            slots_per_container=int(getattr(cfg, "slots_per_container")),
+            min_containers=int(getattr(cfg, "min_containers")),
+            max_containers=int(getattr(cfg, "max_containers")),
+            privileged=bool(getattr(cfg, "privileged")),
+            acquire_timeout_s=float(getattr(cfg, "acquire_timeout_s")),
+            purge_job_on_start=bool(getattr(cfg, "purge_job_on_start", False)),
+            driver=str(getattr(cfg, "driver", "docker")),
+            singularity_image=getattr(cfg, "singularity_image", None),
+        )
+
+
+class NomadToolBackend(ToolBackend):
+    def __init__(self, config: NomadBackendConfig):
+        self.config = config
+        self.pool = SlotPool(
+            SlotPoolConfig(
+                nomad_address=config.nomad_address,
+                job_id=config.sandbox_job_id,
+                image=config.sandbox_image,
+                slots_per_container=config.slots_per_container,
+                min_containers=config.min_containers,
+                max_containers=config.max_containers,
+                privileged=config.privileged,
+                acquire_timeout=config.acquire_timeout_s,
+                purge_job_on_start=bool(config.purge_job_on_start),
+                driver=config.driver,
+                singularity_image=config.singularity_image,
+            )
+        )
+
+    @property
+    def default_timeout_s(self) -> Optional[float]:
+        t = getattr(self.pool.executor, "timeout", None)
+        total = getattr(t, "total", None)
+        try:
+            return float(total) if total is not None else None
+        except Exception:
+            return None
+
+    async def start(self) -> None:
+        await self.pool.start()
+
+    async def stop(self, *, purge: bool = False) -> None:
+        await self.pool.stop(purge_job=purge)
+
+    async def acquire(self, trajectory_id: Optional[str] = None) -> Slot:
+        return await self.pool.acquire(trajectory_id)
+
+    async def release(self, slot: Slot, *, reset_workspace: bool = False) -> None:
+        await self.pool.release(slot, reset_workspace=reset_workspace)
+
+    async def execute_batch(
+        self,
+        requests: List[Tuple[Slot, str, Dict[str, Any]]],
+        *,
+        timeout_s: Optional[float] = None,
+    ) -> List[ExecutionResult]:
+        return await self.pool.execute_batch(requests, timeout=timeout_s)
+
+    async def read_artifact(
+        self,
+        slot: Slot,
+        path: str,
+        *,
+        encoding: str = "text",
+        max_bytes: Optional[int] = None,
+        include_sha256: bool = False,
+        timeout_s: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        return await self.pool.executor.read_artifact(
+            slot,
+            path,
+            encoding=encoding,
+            max_bytes=max_bytes,
+            include_sha256=include_sha256,
+            timeout=timeout_s,
+        )
+
+    async def list_artifacts(
+        self,
+        slot: Slot,
+        path: str = ".",
+        *,
+        recursive: bool = False,
+        max_entries: Optional[int] = None,
+        timeout_s: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        return await self.pool.executor.list_artifacts(
+            slot,
+            path,
+            recursive=recursive,
+            max_entries=max_entries,
+            timeout=timeout_s,
+        )
+
+    async def archive_artifacts(
+        self,
+        slot: Slot,
+        path: str = ".",
+        *,
+        archive_format: str = "tar.gz",
+        max_bytes: Optional[int] = None,
+        max_entries: Optional[int] = None,
+        timeout_s: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        return await self.pool.executor.archive_artifacts(
+            slot,
+            path,
+            archive_format=archive_format,
+            max_bytes=max_bytes,
+            max_entries=max_entries,
+            timeout=timeout_s,
+        )
+
+    def get_stats(self) -> Dict[str, Any]:
+        return self.pool.get_stats()
+

atropos/envs/__init__.py

@@ -0,0 +1,10 @@
+"""
+Environment implementations for atropos-agent.
+"""
+
+from .agent_env import AgentEnv, AgentEnvConfig
+
+# NOTE: Additional example envs exist as modules (e.g. `test_env`, `swe_smith_oracle_env`),
+# but are intentionally not imported here to avoid pulling heavy optional deps at import time.
+
+__all__ = ["AgentEnv", "AgentEnvConfig"]

atropos/envs/agent_env.py

@@ -0,0 +1,537 @@
+"""
+AgentEnv - Atropos BaseEnv extension for agent/tool-call workloads.
+
+AgentEnv is responsible for starting the sandbox tool execution backend and
+providing helpers for running agent trajectories with queued/batched tool calls.
+"""
+
+from __future__ import annotations
+import os
+import asyncio
+import time
+import uuid
+from abc import ABC, abstractmethod
+from typing import Any, Awaitable, Callable, Dict, Generic, List, Optional, Tuple, TypeVar
+
+from pydantic import Field
+
+from atroposlib.envs.base import APIServerConfig, BaseEnv, BaseEnvConfig, Item, ScoredDataGroup, ScoredDataItem
+from atroposlib.envs.server_handling.server_baseline import AsyncSemWithAdaptiveWeight
+
+from ..agent import AgentConfig, AgentResult, AtroposAgent
+from ..backends import ToolBackend, create_tool_backend
+from ..tools import ToolRegistry, build_tool_registry
+from ..tools.tool_executor import ToolExecutor, ToolExecutorConfig
+
+# Main BaseEnv child classes. Child class THESE to get agent+tooling functionality easily.
+
+class AgentEnvConfig(BaseEnvConfig):
+    tool_pool_mode: str = Field(default="nomad", description="Tool execution backend ('nomad' or 'modal')")
+
+    allow_network: bool = Field(
+        default=True,
+        description="Whether sandbox bash commands may access the network (env policy).",
+    )
+    require_sandbox: bool = Field(
+        default=False,
+        description="Fail closed if bubblewrap sandboxing is unavailable/unusable for stateless sandbox tools.",
+    )
+    require_stateful_sandbox: bool = Field(
+        default=False,
+        description="Fail closed if bubblewrap/PID isolation is unavailable for stateful terminal tools (tmux).",
+    )
+    tool_batch_window_ms: int = Field(default=20, description="ToolExecutor batching window (ms)")
+    tool_max_batch_size: int = Field(default=200, description="ToolExecutor maximum batch size")
+
+    # nomad mode settings. TODO: Add Modal support, split this into own config
+    nomad_address: str = Field(default="http://localhost:4646", description="Nomad API address")
+    sandbox_job_id: str = Field(default="atropos-sandbox-agent-env", description="Nomad job id for sandbox containers")
+    sandbox_image: str = Field(default="atropos-sandbox:local", description="Docker image for sandbox containers")
+    slots_per_container: int = Field(default=10, description="Nomad mode: slots per container")
+    min_containers: int = Field(default=1, description="Nomad mode: minimum containers")
+    max_containers: int = Field(default=10, description="Nomad mode: maximum containers")
+    privileged: bool = Field(default=False, description="Nomad mode: run container privileged")
+    acquire_timeout_s: float = Field(default=30.0, description="Slot acquisition timeout (seconds)")
+    purge_job_on_start: bool = Field(
+        default=False,
+        description=(
+            "Nomad mode: stop/purge the sandbox job on startup. This is helpful in local dev and training runs "
+            "to recover from previous crashes that leave the job in a restart backoff state."
+        ),
+    )
+    purge_job_on_shutdown: bool = Field(default=True, description="Nomad mode: stop/purge job on shutdown")
+    
+    # Nomad driver selection (docker or singularity)
+    driver: str = Field(
+        default="docker",
+        description="Nomad task driver: 'docker' (default) or 'singularity' (for HPC without sudo Docker)",
+    )
+    singularity_image: Optional[str] = Field(
+        default=None,
+        description="Path to .sif file for Singularity driver (required if driver='singularity')",
+    )
+
+    # Modal mode settings
+    modal_app_name: str = Field(default="atropos-sandbox", description="Modal app name prefix")
+    modal_image: str = Field(default="python:3.11", description="Modal: container image")
+    modal_gpu: Optional[str] = Field(default=None, description="Modal: GPU type (None, 'T4', 'A10G', 'A100', 'H100')")
+    modal_cpu: float = Field(default=1.0, description="Modal: CPU cores")
+    modal_memory: int = Field(default=2048, description="Modal: memory in MB")
+    modal_slots_per_sandbox: int = Field(default=10, description="Modal: slots per sandbox")
+    modal_min_sandboxes: int = Field(default=1, description="Modal: minimum sandboxes")
+    modal_max_sandboxes: int = Field(default=5, description="Modal: maximum sandboxes")
+    modal_idle_timeout: int = Field(default=120, description="Modal: server-side idle timeout (seconds)")
+    modal_max_lifetime: int = Field(default=3600, description="Modal: max sandbox lifetime (seconds)")
+    modal_acquire_timeout: float = Field(default=60.0, description="Modal: slot acquisition timeout (seconds)")
+    modal_execution_timeout: float = Field(default=30.0, description="Modal: default command execution timeout (seconds)")
+    modal_secrets: str = Field(default="", description="Modal: comma-separated list of Modal Secret names")
+    modal_env_vars: str = Field(default="", description="Modal: semicolon-separated KEY=VALUE pairs for env vars")
+    modal_workspace_base: str = Field(default="/data", description="Modal: workspace base directory in sandbox")
+
+    # basic agent defaults
+    agent_max_steps: int = Field(default=50, description="Max ReACT steps per trajectory")
+    agent_temperature: float = Field(default=0.7, description="Sampling temperature")
+    agent_max_tokens: Optional[int] = Field(
+        default=None,
+        description="Max tokens per model response (default: let backend decide)",
+    )
+    agent_tool_delay_s: float = Field(default=0.0, description="Delay between tool calls (seconds)")
+
+    # tool selection
+    enabled_toolsets: List[str] = Field(
+        default_factory=lambda: ["default"],
+        description="Toolsets to enable (Hermes-style grouping).",
+    )
+    disabled_toolsets: List[str] = Field(
+        default_factory=list,
+        description="Toolsets to disable (applied after enabled_toolsets).",
+    )
+
+    # external ToolServer routing (Phase 4.5+)
+    tool_server_url: Optional[str] = Field(
+        default=None,
+        description="Base URL for external ToolServer (enables external tools).",
+    )
+    tool_server_token: Optional[str] = Field(
+        default=None,
+        description="Bearer token for ToolServer auth (optional in dev).",
+    )
+
+AgentEnvConfigT = TypeVar("AgentEnvConfigT", bound="AgentEnvConfig")
+
+
+class AgentEnv(BaseEnv, ABC, Generic[AgentEnvConfigT]):
+    env_config_cls = AgentEnvConfig
+
+    def __init__(
+        self,
+        config: AgentEnvConfigT,
+        server_configs: List[APIServerConfig],
+        slurm: bool = False,
+        testing: bool = False,
+    ):
+        super().__init__(config, server_configs, slurm, testing)
+        self.config: AgentEnvConfigT = config
+
+        self.tools: ToolRegistry = self.build_tools()
+
+        self._backend: Optional[ToolBackend] = None
+        self._tool_executor: Optional[ToolExecutor] = None
+        self._tool_server_inprocess: bool = False
+        self._trajectory_workspace_meta: Dict[str, Dict[str, Any]] = {}
+
+    def build_tools(self) -> ToolRegistry:
+        """Wraps original Hermes-Agent ToolRegistry for atropos AgentEnv use.
+        See Hermes-Agent docs for toolsets and available tools etc.
+        """
+        return build_tool_registry(
+            enabled_toolsets=self.config.enabled_toolsets or ["default"],
+            disabled_toolsets=self.config.disabled_toolsets or None,
+            tool_server_url=self.config.tool_server_url,
+        )
+
+    @abstractmethod
+    def build_task(self, item: Item) -> str:
+        """Return the user-facing task string for the agent."""
+
+    @abstractmethod
+    async def score_trajectory(self, item: Item, final_response: str) -> float:
+        """Return a scalar score for this trajectory."""
+
+    async def setup_trajectory_workspace(
+        self,
+        item: Item,
+        *,
+        trajectory_id: str,
+        exec_tool: Callable[["ToolCall"], Awaitable["ToolResult"]],
+    ) -> Dict[str, Any]:
+        """
+        Optional hook: prepare the sandbox workspace before the agent starts.
+
+        Examples:
+        - clone a repo and checkout a commit
+        - write fixture files (e.g. images) for external-tool demos
+        - pre-install dependencies
+
+        Default: no-op.
+        """
+        _ = (item, trajectory_id, exec_tool)
+        return {}
+
+    async def verify_and_score_trajectory(
+        self,
+        item: Item,
+        final_response: str,
+        *,
+        trajectory_id: str,
+        exec_tool: Callable[["ToolCall"], Awaitable["ToolResult"]],
+        agent_result: Optional[AgentResult] = None,
+        workspace_meta: Optional[Dict[str, Any]] = None,
+    ) -> tuple[float, Dict[str, Any]]:
+        """
+        Optional hook: run in-sandbox verification before scoring.
+
+        Many agent envs need to execute verification inside the same trajectory
+        workspace (e.g. pytest) before releasing/resetting the slot.
+
+        Default: calls `score_trajectory()` and returns empty metadata.
+        """
+        _ = (trajectory_id, exec_tool, agent_result, workspace_meta)  # default ignores in-workspace verification
+        score = await self.score_trajectory(item, final_response)
+        return score, {}
+
+    def build_agent_config(self, item: Item) -> AgentConfig:  # noqa: ARG002
+        return AgentConfig(
+            max_steps=self.config.agent_max_steps,
+            temperature=self.config.agent_temperature,
+            max_tokens=self.config.agent_max_tokens,
+            tool_delay_s=self.config.agent_tool_delay_s,
+        )
+
+    async def setup(self) -> None:
+        print(f"[AgentEnv] setup(): starting tool backend ({self.config.tool_pool_mode})", flush=True)
+        await self._start_tool_backend()
+        print("[AgentEnv] setup(): configuring server concurrency", flush=True)
+        self._configure_server_concurrency()
+        print("[AgentEnv] setup(): running env-specific setup_agent_env()", flush=True)
+        await self.setup_agent_env()
+        print("[AgentEnv] setup(): done", flush=True)
+
+    def _configure_server_concurrency(self) -> None:
+        """
+        Ensure the LLM server concurrency isn't accidentally capped below `group_size`.
+
+        In `BaseEnv process` mode, groups are collected concurrently and if the underlying
+        ServerManager/OpenAIServer semaphore is left at 1, we serialize inference even
+        when `--env.group_size` is > 1.
+        """
+        desired = int(getattr(self.config, "group_size", 1) or 1)
+        if desired <= 1:
+            return
+
+        servers = getattr(self.server, "servers", None)
+        if not isinstance(servers, list) or not servers:
+            return
+
+        for s in servers:
+            sem = getattr(s, "sem", None)
+            eval_sem = getattr(s, "eval_sem", None)
+            # Only increase; never shrink.
+            if sem is not None and getattr(sem, "max_val", 0) < desired:
+                s.sem = AsyncSemWithAdaptiveWeight(desired)
+                if hasattr(s, "config") and hasattr(s.config, "num_max_requests_at_once"):
+                    s.config.num_max_requests_at_once = desired
+            if eval_sem is not None and getattr(eval_sem, "max_val", 0) < desired:
+                s.eval_sem = AsyncSemWithAdaptiveWeight(desired)
+                if hasattr(s, "config") and hasattr(s.config, "num_requests_for_eval"):
+                    s.config.num_requests_for_eval = desired
+
+    @abstractmethod
+    async def setup_agent_env(self) -> None:
+        """Subclass hook for env-specific setup."""
+
+    async def evaluate(self, *args, **kwargs):  # noqa: ARG002
+        """
+        Default eval hook (no-op).
+
+        Atropos BaseEnv requires an `evaluate()` implementation. Many agent envs
+        won't have a meaningful evaluation path during early PoC work; they can
+        override this when needed.
+        """
+        return {}
+
+    async def env_manager(self):
+        try:
+            return await super().env_manager()
+        finally:
+            await self.shutdown_tool_backend()
+
+    async def process_manager(self):
+        try:
+            return await super().process_manager()
+        finally:
+            await self.shutdown_tool_backend()
+
+    async def _start_tool_backend(self) -> None:
+        if self._tool_executor is not None:
+            return
+
+        tool_server_url = self.config.tool_server_url
+        tool_server_client = None
+        if tool_server_url == "inprocess":
+            import httpx
+            from ..api.tool_server import app as tool_server_app
+
+            await tool_server_app.router.startup()
+            tool_server_client = httpx.AsyncClient(
+                transport=httpx.ASGITransport(app=tool_server_app),
+                base_url="http://toolserver",
+            )
+            tool_server_url = "http://toolserver"
+            self._tool_server_inprocess = True
+
+        backend = create_tool_backend(self.config)
+        await backend.start()
+
+        executor = ToolExecutor(
+            backend=backend,
+            tools=self.tools,
+            config=ToolExecutorConfig(
+                batch_window_ms=self.config.tool_batch_window_ms,
+                max_batch_size=self.config.tool_max_batch_size,
+                allow_network=self.config.allow_network,
+                require_sandbox=self.config.require_sandbox,
+                require_stateful_sandbox=self.config.require_stateful_sandbox,
+                tool_server_url=tool_server_url,
+                tool_server_token=self.config.tool_server_token,
+            ),
+        )
+        await executor.start()
+        if tool_server_client is not None:
+            executor._tool_server_client = tool_server_client  # type: ignore[attr-defined]
+
+        self._backend = backend
+        self._tool_executor = executor
+
+    async def shutdown_tool_backend(self) -> None:
+        executor = self._tool_executor
+        backend = self._backend
+        inprocess_tool_server = self._tool_server_inprocess
+        self._tool_executor = None
+        self._backend = None
+        self._tool_server_inprocess = False
+
+        if executor is not None:
+            await executor.close()
+        if backend is not None:
+            await backend.stop(purge=bool(self.config.purge_job_on_shutdown))
+        if inprocess_tool_server:
+            from ..api.tool_server import app as tool_server_app
+
+            await tool_server_app.router.shutdown()
+
+    async def collect_trajectory(
+        self, item: Item
+    ) -> Tuple[Optional[ScoredDataItem], List[Item]]:
+        if self._tool_executor is None:
+            raise RuntimeError("Tool backend not started")
+
+        trajectory_id = str(uuid.uuid4())
+        t0 = time.perf_counter()
+        print(f"[AgentEnv] collect_trajectory(): tid={trajectory_id} start", flush=True)
+        task = self.build_task(item)
+        agent_config = self.build_agent_config(item)
+        if os.getenv("ATROPOS_DEBUG_PRINT_TASK") == "1":
+            print(f"Starting trajectory {trajectory_id} with task: {task}", flush=True)
+        else:
+            # Avoid printing the full task prompt by default (can be huge/noisy).
+            one_line = " ".join(str(task).splitlines()).strip()
+            preview = one_line[:240] + ("…" if len(one_line) > 240 else "")
+            print(f"Starting trajectory {trajectory_id} (task preview): {preview}", flush=True)
+
+        async def _exec(call):
+            return await self._tool_executor.execute(trajectory_id, call)
+
+        agent = AtroposAgent(
+            server=self.server,
+            tokenizer=self.tokenizer,
+            tools=self.tools,
+            config=agent_config,
+            execute_tool=_exec,
+        )
+
+        try:
+            print(f"[AgentEnv] tid={trajectory_id} setup_trajectory_workspace() start", flush=True)
+            workspace_meta = await self.setup_trajectory_workspace(item, trajectory_id=trajectory_id, exec_tool=_exec)
+            if not isinstance(workspace_meta, dict):
+                workspace_meta = {}
+            self._trajectory_workspace_meta[trajectory_id] = workspace_meta
+            print(
+                f"[AgentEnv] tid={trajectory_id} setup_trajectory_workspace() done in {time.perf_counter() - t0:.2f}s",
+                flush=True,
+            )
+
+            print(f"[AgentEnv] tid={trajectory_id} agent.run() start", flush=True)
+            result = await agent.run(task)
+            print(
+                f"[AgentEnv] tid={trajectory_id} agent.run() done in {time.perf_counter() - t0:.2f}s "
+                f"success={result.success} tool_calls={result.total_tool_calls}",
+                flush=True,
+            )
+            if not result.success or result.trajectory_data is None:
+                # Do not trigger BaseEnv retries for agent failures.
+                # Record the trajectory with score 0.0 so training/eval can see the failure mode.
+                messages = [{"role": "system", "content": agent._build_system_prompt()}]  # noqa: SLF001
+                messages.append({"role": "user", "content": task})
+                for step in result.steps:
+                    messages.append({"role": "assistant", "content": step.assistant_message})
+                    if step.tool_results:
+                        tool_text = "\n".join(r.to_xml() for r in step.tool_results)
+                        messages.append({"role": "user", "content": tool_text})
+
+                scored: ScoredDataItem = {
+                    "tokens": (result.trajectory_data.tokens if result.trajectory_data else []),
+                    "masks": (result.trajectory_data.masked_tokens if result.trajectory_data else []),
+                    "scores": 0.0,
+                }
+                if result.trajectory_data is not None:
+                    scored["inference_logprobs"] = result.trajectory_data.logprobs  # type: ignore[typeddict-unknown-key]
+                    if getattr(result.trajectory_data, "metadata", None):
+                        scored["overrides"] = {"managed_metadata": result.trajectory_data.metadata}
+                if self.config.include_messages:
+                    # Record a final failure marker as a user-side tool_response-like block so it survives templates.
+                    import json
+
+                    err = result.error or "agent_failed"
+                    messages.append(
+                        {
+                            "role": "user",
+                            "content": f"<tool_response>{json.dumps({'success': False, 'error': err})}</tool_response>",
+                        }
+                    )
+                    scored["messages"] = messages
+                return scored, []
+
+            print(f"[AgentEnv] tid={trajectory_id} verify_and_score_trajectory() start", flush=True)
+            score, score_metadata = await self.verify_and_score_trajectory(
+                item,
+                result.final_response,
+                trajectory_id=trajectory_id,
+                exec_tool=_exec,
+                agent_result=result,
+                workspace_meta=workspace_meta,
+            )
+            print(
+                f"[AgentEnv] tid={trajectory_id} verify_and_score_trajectory() done in {time.perf_counter() - t0:.2f}s "
+                f"score={score}",
+                flush=True,
+            )
+
+            messages = [{"role": "system", "content": agent._build_system_prompt()}]  # noqa: SLF001
+            messages.append({"role": "user", "content": task})
+            for step in result.steps:
+                messages.append({"role": "assistant", "content": step.assistant_message})
+                if step.tool_results:
+                    tool_text = "\n".join(r.to_xml() for r in step.tool_results)
+                    messages.append({"role": "user", "content": tool_text})
+
+            # Optional: allow env verification to attach additional messages (e.g. install logs).
+            if self.config.include_messages and isinstance(score_metadata, dict):
+                extra = score_metadata.get("verification_messages")
+                if isinstance(extra, list):
+                    for m in extra:
+                        if isinstance(m, dict) and isinstance(m.get("role"), str) and isinstance(m.get("content"), str):
+                            messages.append({"role": m["role"], "content": m["content"]})
+
+            scored: ScoredDataItem = {
+                "tokens": result.trajectory_data.tokens,
+                "masks": result.trajectory_data.masked_tokens,
+                "scores": score,
+            }
+            # Atroposlib expects policy logprobs at the *group* level under `inference_logprobs`.
+            # We stash per-item values here and lift them into the group in `collect_trajectories()`.
+            scored["inference_logprobs"] = result.trajectory_data.logprobs  # type: ignore[typeddict-unknown-key]
+            if getattr(result.trajectory_data, "metadata", None):
+                scored["overrides"] = {"managed_metadata": result.trajectory_data.metadata}
+            if self.config.include_messages:
+                scored["messages"] = messages
+
+            return scored, []
+        finally:
+            self._trajectory_workspace_meta.pop(trajectory_id, None)
+            print(f"[AgentEnv] tid={trajectory_id} release_trajectory(reset_workspace=True)", flush=True)
+            await self._tool_executor.release_trajectory(trajectory_id, reset_workspace=True)
+            print(f"[AgentEnv] collect_trajectory(): tid={trajectory_id} done in {time.perf_counter() - t0:.2f}s", flush=True)
+
+    async def collect_trajectories(
+        self, item: Item
+    ) -> Tuple[Optional[ScoredDataGroup], List[Item]]:
+        tasks = [self.collect_trajectory(item) for _ in range(self.config.group_size)]
+        results = await asyncio.gather(*tasks)
+
+        backlog: List[Item] = []
+        items: List[ScoredDataItem] = []
+        for scored, b in results:
+            backlog.extend(b)
+            if scored is not None:
+                items.append(scored)
+
+        if len(items) != self.config.group_size:
+            return None, backlog
+
+        group: ScoredDataGroup = ScoredDataGroup(
+            tokens=[],
+            masks=[],
+            scores=[],
+            advantages=[],
+            ref_logprobs=[],
+            messages=[] if self.config.include_messages else None,
+            inference_logprobs=[],
+            group_overrides={},
+            overrides=[],
+            images=[],
+            generation_params=None,
+        )
+
+        for it in items:
+            group["tokens"].append(it["tokens"])
+            group["masks"].append(it["masks"])
+            group["scores"].append(it["scores"])
+            # policy logprobs (for PPO/GRPO training) if present
+            lp = it.get("inference_logprobs")  # type: ignore[typeddict-item]
+            if lp is not None:
+                group["inference_logprobs"].append(lp)
+            group["overrides"].append(it.get("overrides") or {})  # type: ignore[typeddict-item]
+            if group.get("messages") is not None and it.get("messages") is not None:
+                group["messages"].append(it["messages"])
+
+        return group, backlog
+
+    async def run_agent(self, task: str, *, trajectory_id: Optional[str] = None) -> Tuple[str, Dict[str, Any]]:
+        """
+        Run the AtroposAgent on a single task and return (final_response, debug).
+
+        This is a helper intended for simple environments and tests.
+        """
+        if self._tool_executor is None:
+            raise RuntimeError("Tool backend not started")
+
+        tid = trajectory_id or str(uuid.uuid4())
+
+        async def _exec(call):
+            return await self._tool_executor.execute(tid, call)
+
+        agent = AtroposAgent(
+            server=self.server,
+            tokenizer=self.tokenizer,
+            tools=self.tools,
+            config=AgentConfig(
+                max_steps=self.config.agent_max_steps,
+                temperature=self.config.agent_temperature,
+                max_tokens=self.config.agent_max_tokens,
+            ),
+            execute_tool=_exec,
+        )
+        result = await agent.run(task)
+        await self._tool_executor.release_trajectory(tid, reset_workspace=True)
+        return result.final_response, {"success": result.success, "error": result.error, "tool_calls": result.total_tool_calls}

atropos/envs/endless_terminals_env.py

@@ -0,0 +1,873 @@
+"""
+Endless Terminals Environment for Hermes-Agent + Atropos RL.
+
+Runs terminal tasks from the Endless Terminals dataset.
+Supports three modes:
+  1. Local directory: tasks from a local folder of task_* dirs (default)
+  2. HuggingFace dataset: tasks from a HF dataset
+  3. Procedural: generate tasks on-the-fly via LLM (requires vLLM)
+
+Each task provides a Dockerfile that defines the initial environment.
+The agent solves the task using terminal commands inside a Docker container.
+Scoring is done by running pytest on `test_final_state.py` in the container.
+
+Run (standalone process mode):
+  python -m atropos.envs.endless_terminals_env process \
+    --env.use_wandb false \
+    --env.total_steps 100 \
+    --env.group_size 4
+
+Run (Tinker serve mode):
+  # Terminal 1: run-api
+  # Terminal 2: python launch_training.py --config configs/endless_terminals.yaml
+  # Terminal 3:
+  TINKER_CONFIG=configs/endless_terminals.yaml \
+  ENDLESS_TERMINALS_DIR=/path/to/endless-terminals \
+    python -m atropos.envs.endless_terminals_env serve
+"""
+
+from __future__ import annotations
+
+import asyncio
+import base64
+import json
+import os
+import random
+import shutil
+import subprocess
+import sys
+import tempfile
+import uuid
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+from dotenv import load_dotenv
+from pydantic import Field
+
+from atroposlib.envs.base import APIServerConfig, Item
+
+from ..agent import AgentConfig
+from ..backends.docker_direct_backend import (
+    DockerDirectBackend,
+    build_docker_image,
+    docker_image_exists,
+)
+from ..tools import ToolCall
+from .agent_env import AgentEnv, AgentEnvConfig
+
+load_dotenv()
+
+
+# ---------------------------------------------------------------------------
+# Tinker integration
+# ---------------------------------------------------------------------------
+# When TINKER_CONFIG is set, we load model/training params from the Tinker YAML.
+# Custom env fields (ENDLESS_TERMINALS_DIR, etc.) are always read from env vars.
+TINKER_CONFIG = os.getenv("TINKER_CONFIG", "")
+
+
+def _load_tinker_config():
+    """Load TinkerAtroposConfig if available, else return None."""
+    if not TINKER_CONFIG:
+        return None
+    config_path = Path(TINKER_CONFIG)
+    if not config_path.exists():
+        print(f"[EndlessTerminalsEnv] TINKER_CONFIG={TINKER_CONFIG} not found, ignoring", flush=True)
+        return None
+    try:
+        from tinker_atropos.config import TinkerAtroposConfig
+        config = TinkerAtroposConfig.from_yaml(config_path)
+        print(f"[EndlessTerminalsEnv] Loaded Tinker config from {config_path}", flush=True)
+        return config
+    except ImportError:
+        print("[EndlessTerminalsEnv] tinker_atropos not installed, ignoring TINKER_CONFIG", flush=True)
+        return None
+    except Exception as e:
+        print(f"[EndlessTerminalsEnv] Error loading Tinker config: {e}", flush=True)
+        return None
+
+
+# ---------------------------------------------------------------------------
+# Config
+# ---------------------------------------------------------------------------
+
+class EndlessTerminalsEnvConfig(AgentEnvConfig):
+    """Configuration for Endless Terminals environment."""
+
+    # ---- Local directory mode (primary) ----
+    use_local_dir: bool = Field(
+        default=True,
+        description="Load tasks from a local directory of task_* folders.",
+    )
+    local_tasks_dir: str = Field(
+        default="",
+        description="Path to directory containing task_* folders. Required if use_local_dir=True.",
+    )
+    prebuild_images: bool = Field(
+        default=False,
+        description="Pre-build ALL Docker images during setup (slow but avoids build-during-training).",
+    )
+    max_concurrent_builds: int = Field(
+        default=4,
+        description="Max parallel Docker image builds during pre-build.",
+    )
+
+    # ---- HuggingFace dataset mode ----
+    use_dataset: bool = Field(
+        default=False,
+        description="Load tasks from HuggingFace dataset.",
+    )
+    dataset_name: str = Field(
+        default="obiwan96/endless-terminals-train",
+        description="HuggingFace dataset name (if use_dataset=True)",
+    )
+    dataset_split: str = Field(default="train")
+    dataset_cache_dir: str = Field(default="~/.cache/huggingface/datasets")
+    tasks_base_dir: str = Field(
+        default="",
+        description="Base directory containing task_* folders (for dataset mode path resolution).",
+    )
+
+    # ---- Procedural generation mode ----
+    task_gen_model: str = Field(default="Qwen/Qwen3-32B")
+    task_gen_temperature: float = Field(default=1.0)
+    task_gen_max_tokens: int = Field(default=2048)
+
+    # ---- Container / scoring ----
+    container_build_timeout_s: float = Field(default=600.0, description="Docker build timeout")
+    test_timeout_s: int = Field(default=120, description="Test execution timeout (seconds)")
+    keep_failed_tasks: bool = Field(default=False)
+
+    # ---- Agent defaults ----
+    agent_max_steps: int = Field(default=32)
+    agent_temperature: float = Field(default=0.7)
+
+    # ---- Docker image prefix ----
+    docker_image_prefix: str = Field(
+        default="endless-terminals",
+        description="Docker image name prefix for built task images.",
+    )
+
+    # ---- Server defaults ----
+    server_base_url: str = Field(default="http://127.0.0.1:8080")
+    server_model: str = Field(default="hermes-4-36b")
+    tokenizer_name: str = Field(default="NousResearch/Hermes-4.3-36B")
+
+
+# ---------------------------------------------------------------------------
+# Env
+# ---------------------------------------------------------------------------
+
+class EndlessTerminalsEnv(AgentEnv[EndlessTerminalsEnvConfig]):
+    """
+    Endless Terminals environment.
+
+    Each task:
+      1. Has a Dockerfile defining the initial container state
+      2. Has an instruction.md describing what the agent should do
+      3. Has tests/test_final_state.py to verify completion
+
+    Flow per trajectory:
+      1. get_next_item() → picks a task
+      2. setup_trajectory_workspace() → builds Docker image, registers with backend
+      3. Agent solves task via terminal commands (docker exec in the container)
+      4. verify_and_score_trajectory() → runs pytest in container, returns binary reward
+    """
+
+    name = "endless_terminals_env"
+    env_config_cls = EndlessTerminalsEnvConfig
+
+    def __init__(
+        self,
+        config: EndlessTerminalsEnvConfig,
+        server_configs: List[APIServerConfig],
+        slurm: bool = False,
+        testing: bool = False,
+    ):
+        super().__init__(config, server_configs, slurm, testing)
+        self._iteration = 0
+
+        # Local dir mode
+        self._local_tasks: List[Dict[str, Any]] = []
+        self._local_task_indices: List[int] = []
+        self._local_current_index = 0
+
+        # Eval split (held-out tasks)
+        self._eval_tasks: List[Dict[str, Any]] = []
+
+        # Training metrics
+        self._train_scores_buffer: List[float] = []
+        self._eval_metrics: List[tuple] = []
+
+        # HF dataset mode
+        self._dataset = None
+        self._dataset_indices: List[int] = []
+        self._dataset_current_index = 0
+
+        # Docker image cache: task_name -> image_tag
+        self._image_cache: Dict[str, str] = {}
+        self._build_lock = asyncio.Lock()
+
+    # ---- Config init (CLI) ----
+
+    @classmethod
+    def config_init(cls) -> Tuple[EndlessTerminalsEnvConfig, List[APIServerConfig]]:
+        """
+        Initialize config.
+
+        Two modes:
+          1. Tinker mode: TINKER_CONFIG env var points to a Tinker YAML.
+             Model, training params, and server config come from the YAML.
+          2. Standalone mode: Everything from env vars (ATROPOS_SERVER_*, etc.)
+
+        In both modes, Endless Terminals-specific fields (ENDLESS_TERMINALS_DIR,
+        PREBUILD_IMAGES, etc.) are always read from env vars.
+        """
+        tinker_cfg = _load_tinker_config()
+
+        # ── Endless Terminals-specific fields (always from env vars) ──
+        local_tasks_dir = os.getenv("ENDLESS_TERMINALS_DIR", "")
+        use_local_dir = bool(local_tasks_dir)
+
+        if tinker_cfg is not None:
+            # ── Tinker mode ─────────────────────────────────────────
+            print("[EndlessTerminalsEnv] Using Tinker config", flush=True)
+
+            env_config = EndlessTerminalsEnvConfig(
+                # Standard Atropos fields from Tinker YAML
+                tokenizer_name=tinker_cfg.base_model,
+                group_size=tinker_cfg.group_size,
+                use_wandb=tinker_cfg.use_wandb,
+                rollout_server_url=tinker_cfg.atropos_api_url,
+                total_steps=tinker_cfg.num_steps,
+                batch_size=tinker_cfg.batch_size,
+                steps_per_eval=tinker_cfg.steps_per_eval,
+                max_token_length=tinker_cfg.max_token_env_length,
+                max_num_workers=tinker_cfg.max_num_workers,
+                max_batches_offpolicy=tinker_cfg.max_batches_offpolicy,
+                ensure_scores_are_not_same=tinker_cfg.ensure_scores_are_not_same,
+                wandb_name=f"{tinker_cfg.wandb_run_name}-env",
+                include_messages=True,
+
+                # Tooling: terminal only
+                enabled_toolsets=["terminal"],
+                disabled_toolsets=[],
+
+                # Agent config
+                agent_max_steps=int(os.getenv("AGENT_MAX_STEPS", "32")),
+                agent_temperature=float(os.getenv("AGENT_TEMPERATURE", "0.7")),
+
+                # Docker-direct backend (no Nomad needed)
+                tool_pool_mode="docker_direct",
+                sandbox_image="ubuntu:22.04",
+                purge_job_on_start=False,
+                purge_job_on_shutdown=False,
+
+                # Endless Terminals fields
+                use_local_dir=use_local_dir,
+                local_tasks_dir=local_tasks_dir,
+                prebuild_images=os.getenv("PREBUILD_IMAGES", "false").lower() == "true",
+                use_dataset=os.getenv("USE_DATASET", "false").lower() == "true",
+                dataset_name=os.getenv("ENDLESS_DATASET", "obiwan96/endless-terminals-train"),
+                container_build_timeout_s=float(os.getenv("CONTAINER_BUILD_TIMEOUT", "600")),
+                test_timeout_s=int(os.getenv("TEST_TIMEOUT", "120")),
+            )
+
+            server_configs = [
+                APIServerConfig(
+                    model_name=tinker_cfg.base_model,
+                    base_url=tinker_cfg.inference_api_url + "/v1",
+                    api_key="x",
+                    server_type="sglang",
+                    num_requests_for_eval=tinker_cfg.num_requests_for_eval,
+                    timeout=600,  # Longer timeout for multi-step agent trajectories
+                ),
+            ]
+            return env_config, server_configs
+
+        else:
+            # ── Standalone mode (env vars) ──────────────────────────
+            base_url = (
+                os.getenv("ATROPOS_SERVER_BASE_URL")
+                or os.getenv("OPENAI_BASE_URL")
+                or os.getenv("LLM_BASE_URL")
+                or "http://127.0.0.1:8080"
+            )
+            model = os.getenv("ATROPOS_SERVER_MODEL") or os.getenv("LLM_MODEL") or "hermes-4-36b"
+            api_key = (
+                os.getenv("ATROPOS_SERVER_API_KEY")
+                or os.getenv("NOUS_API_KEY")
+                or os.getenv("OPENAI_API_KEY")
+                or "local"
+            )
+
+            env_config = EndlessTerminalsEnvConfig(
+                tokenizer_name=os.getenv("ATROPOS_TOKENIZER_NAME") or "NousResearch/Hermes-4.3-36B",
+                group_size=int(os.getenv("ATROPOS_GROUP_SIZE", "4")),
+                use_wandb=os.getenv("USE_WANDB", "false").lower() == "true",
+                include_messages=True,
+                total_steps=int(os.getenv("ATROPOS_TOTAL_STEPS", "1000")),
+                batch_size=int(os.getenv("ATROPOS_BATCH_SIZE", "32")),
+                server_base_url=base_url,
+                server_model=model,
+
+                # Tooling
+                enabled_toolsets=["terminal"],
+                disabled_toolsets=[],
+
+                # Agent
+                agent_max_steps=int(os.getenv("AGENT_MAX_STEPS", "32")),
+                agent_temperature=float(os.getenv("AGENT_TEMPERATURE", "0.7")),
+
+                # Docker-direct backend
+                tool_pool_mode="docker_direct",
+                sandbox_image="ubuntu:22.04",
+                purge_job_on_start=False,
+                purge_job_on_shutdown=False,
+
+                # Endless Terminals fields
+                use_local_dir=use_local_dir,
+                local_tasks_dir=local_tasks_dir,
+                prebuild_images=os.getenv("PREBUILD_IMAGES", "false").lower() == "true",
+                use_dataset=os.getenv("USE_DATASET", "false").lower() == "true",
+                dataset_name=os.getenv("ENDLESS_DATASET", "obiwan96/endless-terminals-train"),
+                task_gen_model=os.getenv("TASK_GEN_MODEL", "Qwen/Qwen3-32B"),
+                container_build_timeout_s=float(os.getenv("CONTAINER_BUILD_TIMEOUT", "600")),
+                test_timeout_s=int(os.getenv("TEST_TIMEOUT", "120")),
+            )
+
+            server_configs = [
+                APIServerConfig(
+                    model_name=model,
+                    base_url=f"{base_url.rstrip('/')}/v1",
+                    api_key=api_key,
+                    num_max_requests_at_once=int(os.getenv("MAX_CONCURRENT_REQUESTS", "4")),
+                    num_requests_for_eval=int(os.getenv("MAX_EVAL_REQUESTS", "4")),
+                    timeout=300,
+                )
+            ]
+            return env_config, server_configs
+
+    # ---- Setup ----
+
+    async def setup_agent_env(self) -> None:
+        """Env-specific setup: scan tasks and optionally pre-build images."""
+        if self.config.use_local_dir:
+            await self._setup_local_dir()
+        elif self.config.use_dataset:
+            await self._setup_hf_dataset()
+        else:
+            print("[EndlessTerminalsEnv] Using procedural task generation", flush=True)
+
+    async def _setup_local_dir(self) -> None:
+        """Scan local directory for task_* folders."""
+        tasks_dir = Path(self.config.local_tasks_dir).expanduser().resolve()
+        if not tasks_dir.is_dir():
+            raise RuntimeError(f"local_tasks_dir does not exist: {tasks_dir}")
+
+        print(f"[EndlessTerminalsEnv] Scanning {tasks_dir} for tasks...", flush=True)
+
+        tasks = []
+        for entry in sorted(tasks_dir.iterdir()):
+            if not entry.is_dir() or not entry.name.startswith("task_"):
+                continue
+
+            # Validate required files
+            dockerfile = entry / "environment" / "Dockerfile"
+            instruction = entry / "instruction.md"
+            test_final = entry / "tests" / "test_final_state.py"
+
+            if not dockerfile.exists():
+                continue
+            if not instruction.exists():
+                continue
+            if not test_final.exists():
+                continue
+
+            # Read task metadata
+            task_json_path = entry / "environment" / "task.json"
+            description = instruction.read_text(encoding="utf-8").strip()
+
+            truth = ""
+            if task_json_path.exists():
+                try:
+                    task_json = json.loads(task_json_path.read_text(encoding="utf-8"))
+                    # task.json may have a richer description; prefer instruction.md
+                    truth = task_json.get("truth", "")
+                except Exception:
+                    pass
+
+            tasks.append({
+                "task_name": entry.name,
+                "task_dir": str(entry),
+                "dockerfile": str(dockerfile),
+                "description": description,
+                "truth": truth,
+                "test_final": str(test_final),
+            })
+
+        if not tasks:
+            raise RuntimeError(f"No valid task_* directories found in {tasks_dir}")
+
+        # Split into train and eval (hold out ~5% for eval, min 10, max 50)
+        random.shuffle(tasks)
+        eval_count = max(10, min(50, len(tasks) // 20))
+        eval_count = min(eval_count, len(tasks) // 2)  # Never more than half
+
+        self._eval_tasks = tasks[:eval_count]
+        self._local_tasks = tasks[eval_count:]
+        self._local_task_indices = list(range(len(self._local_tasks)))
+        random.shuffle(self._local_task_indices)
+        self._local_current_index = 0
+
+        print(
+            f"[EndlessTerminalsEnv] Found {len(tasks)} valid tasks "
+            f"({len(self._local_tasks)} train, {len(self._eval_tasks)} eval)",
+            flush=True,
+        )
+
+        # Optionally pre-build all Docker images
+        if self.config.prebuild_images:
+            await self._prebuild_images()
+
+    async def _prebuild_images(self) -> None:
+        """Pre-build Docker images for all tasks."""
+        print(f"[EndlessTerminalsEnv] Pre-building Docker images...", flush=True)
+        sem = asyncio.Semaphore(self.config.max_concurrent_builds)
+        built = 0
+        skipped = 0
+        failed = 0
+
+        async def _build_one(task: Dict[str, Any]) -> None:
+            nonlocal built, skipped, failed
+            image_tag = self._image_tag_for_task(task["task_name"])
+
+            if docker_image_exists(image_tag):
+                self._image_cache[task["task_name"]] = image_tag
+                skipped += 1
+                return
+
+            async with sem:
+                ok = await build_docker_image(
+                    task["dockerfile"], image_tag,
+                    timeout_s=self.config.container_build_timeout_s,
+                )
+                if ok:
+                    self._image_cache[task["task_name"]] = image_tag
+                    built += 1
+                else:
+                    failed += 1
+
+        await asyncio.gather(*[_build_one(t) for t in self._local_tasks])
+        print(
+            f"[EndlessTerminalsEnv] Pre-build: {built} built, {skipped} cached, {failed} failed",
+            flush=True,
+        )
+
+    async def _setup_hf_dataset(self) -> None:
+        """Load HuggingFace dataset."""
+        print(f"[EndlessTerminalsEnv] Loading dataset: {self.config.dataset_name}", flush=True)
+        try:
+            from datasets import load_dataset
+
+            loop = asyncio.get_event_loop()
+            self._dataset = await loop.run_in_executor(
+                None,
+                lambda: load_dataset(
+                    self.config.dataset_name,
+                    split=self.config.dataset_split,
+                    cache_dir=os.path.expanduser(self.config.dataset_cache_dir),
+                ),
+            )
+            self._dataset_indices = list(range(len(self._dataset)))
+            random.shuffle(self._dataset_indices)
+            self._dataset_current_index = 0
+            print(f"[EndlessTerminalsEnv] Loaded {len(self._dataset)} tasks from dataset", flush=True)
+        except Exception as e:
+            print(f"[EndlessTerminalsEnv] ERROR loading dataset: {e}", flush=True)
+            raise
+
+    # ---- Image helpers ----
+
+    def _image_tag_for_task(self, task_name: str) -> str:
+        return f"{self.config.docker_image_prefix}:{task_name}"
+
+    async def _ensure_image(self, task: Dict[str, Any]) -> str:
+        """Ensure the Docker image for a task is built. Returns image tag."""
+        task_name = task["task_name"]
+        image_tag = self._image_tag_for_task(task_name)
+
+        # Fast path: already cached
+        if task_name in self._image_cache:
+            return self._image_cache[task_name]
+
+        async with self._build_lock:
+            # Double-check after acquiring lock
+            if task_name in self._image_cache:
+                return self._image_cache[task_name]
+
+            # Check if image exists in Docker
+            if docker_image_exists(image_tag):
+                self._image_cache[task_name] = image_tag
+                return image_tag
+
+            # Build it
+            print(f"[EndlessTerminalsEnv] Building image {image_tag}...", flush=True)
+            ok = await build_docker_image(
+                task["dockerfile"], image_tag,
+                timeout_s=self.config.container_build_timeout_s,
+            )
+            if not ok:
+                raise RuntimeError(f"Failed to build Docker image for {task_name}")
+
+            self._image_cache[task_name] = image_tag
+            return image_tag
+
+    # ---- Item generation ----
+
+    async def get_next_item(self) -> Item:
+        self._iteration += 1
+
+        if self.config.use_local_dir and self._local_tasks:
+            return self._get_next_local_item()
+        elif self.config.use_dataset and self._dataset is not None:
+            return self._get_next_dataset_item()
+        else:
+            return self._get_fallback_item()
+
+    def _get_next_local_item(self) -> Item:
+        """Pick the next task from local directories."""
+        idx = self._local_task_indices[self._local_current_index]
+        task = self._local_tasks[idx]
+
+        self._local_current_index += 1
+        if self._local_current_index >= len(self._local_task_indices):
+            random.shuffle(self._local_task_indices)
+            self._local_current_index = 0
+            print("[EndlessTerminalsEnv] Reshuffled local tasks (epoch complete)", flush=True)
+
+        return {
+            "task_id": f"local_{self._iteration:06d}_{task['task_name']}",
+            "task_name": task["task_name"],
+            "description": task["description"],
+            "truth": task.get("truth", ""),
+            "task_dir": task["task_dir"],
+            "dockerfile": task["dockerfile"],
+            "test_final": task["test_final"],
+            "from_local_dir": True,
+        }
+
+    def _get_next_dataset_item(self) -> Item:
+        """Pick the next task from HuggingFace dataset."""
+        idx = self._dataset_indices[self._dataset_current_index]
+        task = self._dataset[idx]
+
+        self._dataset_current_index += 1
+        if self._dataset_current_index >= len(self._dataset_indices):
+            random.shuffle(self._dataset_indices)
+            self._dataset_current_index = 0
+            print("[EndlessTerminalsEnv] Reshuffled dataset (epoch complete)", flush=True)
+
+        # Resolve task directory
+        task_dir = task.get("extra_info", {}).get("task_dir") or task.get("reward_spec", {}).get("ground_truth", "")
+        if self.config.tasks_base_dir:
+            task_name = Path(task_dir).name
+            task_dir = str(Path(self.config.tasks_base_dir) / task_name)
+
+        task_dir_path = Path(task_dir)
+        return {
+            "task_id": f"dataset_{self._iteration:06d}_{task_dir_path.name}",
+            "task_name": task_dir_path.name,
+            "description": task.get("description", ""),
+            "task_dir": task_dir,
+            "dockerfile": str(task_dir_path / "environment" / "Dockerfile"),
+            "test_final": str(task_dir_path / "tests" / "test_final_state.py"),
+            "from_dataset": True,
+        }
+
+    def _get_fallback_item(self) -> Item:
+        return {
+            "task_id": f"fallback_{self._iteration:06d}",
+            "task_name": "fallback",
+            "description": (
+                "Create a file named 'hello.txt' in /home/user/ containing "
+                "the text 'Hello, World!' on a single line."
+            ),
+            "task_dir": "",
+            "dockerfile": "",
+            "test_final": "",
+        }
+
+    # ---- AgentEnv hooks ----
+
+    def build_task(self, item: Item) -> str:
+        """Return the task prompt for the agent."""
+        return str(item.get("description", ""))
+
+    def build_agent_config(self, item: Item) -> AgentConfig:
+        return AgentConfig(
+            max_steps=self.config.agent_max_steps,
+            temperature=self.config.agent_temperature,
+            max_tokens=self.config.agent_max_tokens,
+            tool_delay_s=self.config.agent_tool_delay_s,
+        )
+
+    async def setup_trajectory_workspace(
+        self,
+        item: Item,
+        *,
+        trajectory_id: str,
+        exec_tool,
+    ) -> Dict[str, Any]:
+        """
+        Build the Docker image for this task and register it with the backend.
+
+        The DockerDirectBackend will start a container from this image when the
+        agent makes its first tool call (lazy acquisition via ToolExecutor).
+        """
+        task_name = item.get("task_name", "unknown")
+        dockerfile = item.get("dockerfile", "")
+
+        if not dockerfile or not Path(dockerfile).exists():
+            print(f"[EndlessTerminalsEnv] WARNING: No Dockerfile for {task_name}", flush=True)
+            return {"image": "ubuntu:22.04"}
+
+        # Build/get Docker image
+        image_tag = await self._ensure_image({
+            "task_name": task_name,
+            "dockerfile": dockerfile,
+        })
+
+        # Register image with the DockerDirect backend
+        if isinstance(self._backend, DockerDirectBackend):
+            self._backend.register_image(trajectory_id, image_tag)
+
+        return {"image": image_tag, "task_name": task_name}
+
+    async def score_trajectory(self, item: Item, final_response: str) -> float:
+        """Not used — scoring happens in verify_and_score_trajectory."""
+        return 0.0
+
+    async def verify_and_score_trajectory(
+        self,
+        item: Item,
+        final_response: str,
+        *,
+        trajectory_id: str,
+        exec_tool,
+        agent_result=None,
+        workspace_meta=None,
+    ) -> tuple[float, Dict[str, Any]]:
+        """
+        Run test_final_state.py inside the container and return binary reward.
+        """
+        task_id = item.get("task_id", "unknown")
+        test_final = item.get("test_final", "")
+
+        if not test_final or not Path(test_final).exists():
+            print(f"[EndlessTerminalsEnv] No test file for {task_id}", flush=True)
+            return 0.0, {"error": "No test file"}
+
+        print(f"[EndlessTerminalsEnv] Scoring {task_id}...", flush=True)
+
+        try:
+            # Read the test file and base64-encode it for safe transfer
+            test_content = Path(test_final).read_text(encoding="utf-8")
+            encoded = base64.b64encode(test_content.encode("utf-8")).decode("ascii")
+
+            # Write test file into the container and run pytest
+            # We write to /tmp to avoid interfering with the agent's workspace
+            # Use printf + heredoc to avoid quoting issues with single quotes in base64
+            verify_cmd = (
+                f"printf '%s' '{encoded}' | base64 -d > /tmp/_test_final_state.py && "
+                f"cd /home/user && "
+                f"python3 -m pytest /tmp/_test_final_state.py -v --tb=short 2>&1; "
+                f"echo \"EXIT_CODE=$?\""
+            )
+
+            result = await exec_tool(ToolCall(
+                name="terminal",
+                arguments={"command": verify_cmd},
+            ))
+
+            output = result.output if hasattr(result, "output") else str(result)
+
+            # Check if pytest passed
+            # Look for EXIT_CODE=0 at the end (most reliable)
+            success = "EXIT_CODE=0" in output
+
+            score = 1.0 if success else 0.0
+
+            metadata = {
+                "task_id": task_id,
+                "success": success,
+                "test_output": output[-2000:] if len(output) > 2000 else output,
+                "total_tool_calls": agent_result.total_tool_calls if agent_result else 0,
+            }
+
+            self._train_scores_buffer.append(score)
+            print(f"[EndlessTerminalsEnv] {task_id} → score={score}", flush=True)
+            return score, metadata
+
+        except Exception as e:
+            print(f"[EndlessTerminalsEnv] Error scoring {task_id}: {e}", flush=True)
+            return 0.0, {"error": str(e)}
+
+    # ---- WandB logging ----
+
+    async def wandb_log(self, wandb_metrics: Optional[Dict] = None):
+        """Log training metrics to wandb."""
+        if wandb_metrics is None:
+            wandb_metrics = {}
+
+        # Training pass rate since last log
+        if self._train_scores_buffer:
+            wandb_metrics["train/percent_correct"] = (
+                sum(self._train_scores_buffer) / len(self._train_scores_buffer)
+            )
+            wandb_metrics["train/num_trajectories"] = len(self._train_scores_buffer)
+            self._train_scores_buffer = []
+
+        # Eval metrics (populated by evaluate())
+        for key, value in self._eval_metrics:
+            wandb_metrics[key] = value
+        self._eval_metrics = []
+
+        await super().wandb_log(wandb_metrics)
+
+    # ---- Evaluation ----
+
+    async def evaluate(self, *args, **kwargs):
+        """
+        Run the agent on held-out eval tasks and report pass rate.
+
+        Each eval task: build Docker container → run agent (temp=0) → pytest → score.
+        This is expensive (full agent trajectories), so we only eval a subset.
+        """
+        import time as _time
+
+        if not self._eval_tasks:
+            return {}
+
+        start_time = _time.time()
+        eval_sample_size = min(len(self._eval_tasks), 20)
+        eval_subset = random.sample(self._eval_tasks, eval_sample_size)
+
+        print(
+            f"[EndlessTerminalsEnv] Running evaluation on {eval_sample_size} tasks...",
+            flush=True,
+        )
+
+        scores = []
+        samples = []
+
+        for task_info in eval_subset:
+            task_name = task_info["task_name"]
+            description = task_info["description"]
+
+            try:
+                # Build Docker image
+                image_tag = await self._ensure_image(task_info)
+
+                # Run agent with temp=0 for deterministic eval
+                eval_tid = f"eval_{uuid.uuid4().hex[:8]}"
+
+                # Register image with backend
+                if isinstance(self._backend, DockerDirectBackend):
+                    self._backend.register_image(eval_tid, image_tag)
+
+                async def _exec(call, _tid=eval_tid):
+                    return await self._tool_executor.execute(_tid, call)
+
+                from ..agent import AtroposAgent as _AtroposAgent
+
+                agent = _AtroposAgent(
+                    server=self.server,
+                    tokenizer=self.tokenizer,
+                    tools=self.tools,
+                    config=AgentConfig(
+                        max_steps=self.config.agent_max_steps,
+                        temperature=0.0,  # Deterministic for eval
+                        max_tokens=self.config.agent_max_tokens,
+                    ),
+                    execute_tool=_exec,
+                )
+
+                result = await agent.run(description)
+
+                # Score: run pytest in the container
+                score = 0.0
+                test_final = task_info.get("test_final", "")
+                if result.success and test_final and Path(test_final).exists():
+                    test_content = Path(test_final).read_text(encoding="utf-8")
+                    encoded = base64.b64encode(test_content.encode("utf-8")).decode("ascii")
+                    verify_cmd = (
+                        f"printf '%s' '{encoded}' | base64 -d > /tmp/_test_final_state.py && "
+                        f"cd /home/user && "
+                        f"python3 -m pytest /tmp/_test_final_state.py -v --tb=short 2>&1; "
+                        f'echo "EXIT_CODE=$?"'
+                    )
+                    test_result = await _exec(ToolCall(
+                        name="terminal",
+                        arguments={"command": verify_cmd},
+                    ))
+                    test_output = test_result.output if hasattr(test_result, "output") else ""
+                    if "EXIT_CODE=0" in test_output:
+                        score = 1.0
+
+                scores.append(score)
+                samples.append({
+                    "task": task_name,
+                    "score": score,
+                    "tool_calls": result.total_tool_calls,
+                    "success": result.success,
+                })
+
+                # Cleanup
+                await self._tool_executor.release_trajectory(eval_tid, reset_workspace=True)
+
+                print(f"  [eval] {task_name} → {score}", flush=True)
+
+            except Exception as e:
+                print(f"  [eval] {task_name} → ERROR: {e}", flush=True)
+                scores.append(0.0)
+                samples.append({"task": task_name, "score": 0.0, "error": str(e)})
+
+        end_time = _time.time()
+
+        percent_correct = sum(scores) / len(scores) if scores else 0.0
+
+        print(
+            f"[EndlessTerminalsEnv] Eval: {percent_correct:.1%} pass rate "
+            f"({sum(scores):.0f}/{len(scores)}) in {end_time - start_time:.0f}s",
+            flush=True,
+        )
+
+        # Store for wandb_log to pick up
+        self._eval_metrics.append(("eval/percent_correct", percent_correct))
+        self._eval_metrics.append(("eval/num_tasks", len(scores)))
+        self._eval_metrics.append(("eval/duration_s", end_time - start_time))
+
+        # Log via atroposlib
+        eval_metrics = {
+            "eval/percent_correct": percent_correct,
+            "eval/num_tasks": len(scores),
+        }
+        await self.evaluate_log(
+            metrics=eval_metrics,
+            samples=samples,
+            start_time=start_time,
+            end_time=end_time,
+            generation_parameters={
+                "temperature": 0.0,
+                "max_tokens": self.config.agent_max_tokens,
+            },
+        )
+
+
+if __name__ == "__main__":
+    EndlessTerminalsEnv.cli()
+
+
+
+

atropos/envs/hermes_compat_test_env.py

@@ -0,0 +1,171 @@
+"""
+Hermes-Agent + Atropos (Nomad sandbox) compatibility smoke environment.
+
+This environment is intended to validate, end-to-end:
+  BaseEnv.process -> AgentEnv -> ToolExecutor (batched) -> Nomad SlotPool -> sandbox_server
+
+It forces the model to use a sandbox tool by asking it to run a command that
+generates a high-entropy token inside the sandbox, then repeat it exactly.
+
+Run (process mode):
+  uv run python -m atropos.envs.hermes_compat_test_env process --env.use_wandb false --env.total_steps 2 --env.group_size 1
+"""
+
+from __future__ import annotations
+
+import os
+from typing import Any, Dict, List, Tuple
+
+from dotenv import load_dotenv
+from pydantic import Field
+
+from atroposlib.envs.base import APIServerConfig, Item
+
+from ..agent import AgentConfig, AgentResult
+from ..tools import ToolCall
+from .agent_env import AgentEnv, AgentEnvConfig
+
+load_dotenv()
+
+
+def _forced_tool_item() -> Item:
+    # Use double quotes in the shell command and show JSON escaping explicitly.
+    # This avoids invalid JSON escapes like `\\'` (not valid JSON) that some models produce.
+    cmd = 'python -c "import secrets; print(secrets.token_hex(16))"'
+    return {
+        "command": cmd,
+        "prompt": (
+            "You are acting as an agent inside a sandboxed environment.\n"
+            "You MUST use the terminal tool to execute commands.\n"
+            "Run this exact command:\n"
+            f"{cmd}\n"
+            "When you call the tool, use valid JSON inside <tool_call>. Example:\n"
+            '<tool_call>{"name": "terminal", "arguments": {"command": '
+            '"python -c \\\\"import secrets; print(secrets.token_hex(16))\\\\""}}'
+            "</tool_call>\n"
+            "Then respond with EXACTLY what it printed (the hex token) and nothing else.\n"
+            "Do not guess. Do not explain."
+        ),
+    }
+
+
+class HermesCompatTestEnvConfig(AgentEnvConfig):
+    server_base_url: str = Field(
+        default="http://127.0.0.1:8080",
+        description="Base URL for an OpenAI-compatible chat server (without /v1).",
+    )
+    server_model: str = Field(default="hermes-4-36b", description="Model name")
+    tokenizer_name: str = Field(default="NousResearch/Hermes-4.3-36B", description="Tokenizer name for RL tokenization")
+
+
+class HermesCompatTestEnv(AgentEnv[HermesCompatTestEnvConfig]):
+    name = "hermes_compat_test_env"
+    env_config_cls = HermesCompatTestEnvConfig
+
+    def __init__(
+        self,
+        config: HermesCompatTestEnvConfig,
+        server_configs: List[APIServerConfig],
+        slurm: bool = False,
+        testing: bool = False,
+    ):
+        super().__init__(config, server_configs, slurm, testing)
+        self._iter = 0
+
+    @classmethod
+    def config_init(cls) -> Tuple[HermesCompatTestEnvConfig, List[APIServerConfig]]:
+        base_url = (
+            os.getenv("ATROPOS_SERVER_BASE_URL")
+            or os.getenv("OPENAI_BASE_URL")
+            or os.getenv("LLM_BASE_URL")
+            or "http://127.0.0.1:8080"
+        )
+        model = os.getenv("ATROPOS_SERVER_MODEL") or os.getenv("LLM_MODEL") or "hermes-4-36b"
+        api_key = os.getenv("ATROPOS_SERVER_API_KEY") or os.getenv("NOUS_API_KEY") or os.getenv("OPENAI_API_KEY") or "local"
+
+        env_config = HermesCompatTestEnvConfig(
+            tokenizer_name=os.getenv("ATROPOS_TOKENIZER_NAME") or "NousResearch/Hermes-4.3-36B",
+            group_size=1,
+            use_wandb=False,
+            include_messages=True,
+            ensure_scores_are_not_same=False,
+            total_steps=2,
+            batch_size=1,
+            server_base_url=base_url,
+            server_model=model,
+            # Tooling: sandbox-only terminal.
+            enabled_toolsets=["terminal"],
+            disabled_toolsets=[],
+            # Default to Nomad sandboxing; users can override via --env.* args.
+            sandbox_image=os.getenv("ATROPOS_SANDBOX_IMAGE") or "atropos-sandbox:local",
+            # In local dev it's common for a previous crash to leave the job in backoff.
+            purge_job_on_start=True,
+            purge_job_on_shutdown=True,
+        )
+
+        server_configs = [
+            APIServerConfig(
+                model_name=model,
+                base_url=f"{base_url.rstrip('/')}/v1",
+                api_key=api_key,
+                num_max_requests_at_once=1,
+                num_requests_for_eval=1,
+                timeout=120,
+            )
+        ]
+        return env_config, server_configs
+
+    async def setup_agent_env(self) -> None:
+        return None
+
+    async def get_next_item(self) -> Item:
+        self._iter += 1
+        return _forced_tool_item()
+
+    def build_task(self, item: Item) -> str:
+        return str(item.get("prompt") or "")
+
+    def build_agent_config(self, item: Item) -> AgentConfig:  # noqa: ARG002
+        # Avoid imposing max_tokens by default; tool-tag responses can be long for some models.
+        return AgentConfig(
+            max_steps=min(8, int(self.config.agent_max_steps)),
+            temperature=0.2,
+            max_tokens=None,
+        )
+
+    async def score_trajectory(self, item: Item, final_response: str) -> float:
+        # Scoring happens in verify_and_score_trajectory so we can inspect tool results.
+        _ = (item, final_response)
+        return 0.0
+
+    async def verify_and_score_trajectory(
+        self,
+        item: Item,
+        final_response: str,
+        *,
+        trajectory_id: str,  # noqa: ARG002
+        exec_tool,  # noqa: ARG002
+        agent_result: AgentResult | None = None,
+        workspace_meta: Dict[str, Any] | None = None,  # noqa: ARG002
+    ) -> tuple[float, Dict[str, Any]]:
+        if agent_result is None:
+            return 0.0, {"error": "Missing agent_result"}
+
+        observed: str = ""
+        tool_ok = False
+        for step in agent_result.steps:
+            for res in step.tool_results:
+                if not res.success:
+                    return 0.0, {"error": res.error, "output": res.output}
+                out = (res.output or "").strip()
+                if out:
+                    observed = out.splitlines()[-1].strip()
+                    tool_ok = True
+
+        final = (final_response or "").strip()
+        score = 1.0 if tool_ok and agent_result.total_tool_calls > 0 and observed and final == observed else 0.0
+        return score, {"observed": observed, "tool_calls": agent_result.total_tool_calls, "command": item.get("command")}
+
+
+if __name__ == "__main__":
+    HermesCompatTestEnv.cli()

atropos/envs/sandbox_terminal_smoke_env.py

@@ -0,0 +1,172 @@
+"""
+Nomad sandbox terminal smoke environment (training-oriented).
+
+Validates, end-to-end:
+  BaseEnv.process -> AgentEnv -> ToolExecutor (batched) -> Nomad SlotPool -> sandbox_server
+
+It forces the model to use a sandbox tool by asking it to run a command that
+generates a high-entropy token inside the sandbox, then repeat it exactly.
+
+Run (process mode):
+  uv run python -m atropos.envs.sandbox_terminal_smoke_env process --env.use_wandb false --env.total_steps 2 --env.group_size 1
+"""
+
+from __future__ import annotations
+
+import os
+from typing import Any, Dict, List, Tuple
+
+from dotenv import load_dotenv
+from pydantic import Field
+
+from atroposlib.envs.base import APIServerConfig, Item
+
+from ..agent import AgentConfig, AgentResult
+from ..tools import ToolCall
+from .agent_env import AgentEnv, AgentEnvConfig
+
+load_dotenv()
+
+STRICT_TOOLCALL_SYSTEM_PROMPT = None
+
+
+def _forced_tool_item() -> Item:
+    # Use double quotes in the shell command and show JSON escaping explicitly.
+    # This avoids invalid JSON escapes like `\\'` (not valid JSON) that some models produce.
+    cmd = 'python -c "import secrets; print(secrets.token_hex(16))"'
+    return {
+        "command": cmd,
+        "prompt": (
+            "You MUST use the terminal tool.\n"
+            "Run this exact command:\n"
+            f"{cmd}\n"
+            "When you call the tool, use valid JSON inside <tool_call>. Example:\n"
+            '<tool_call>{"name": "terminal", "arguments": {"command": '
+            '"python -c \\\\"import secrets; print(secrets.token_hex(16))\\\\""}}'
+            "</tool_call>\n"
+            "Then respond with EXACTLY what it printed (the hex token) and nothing else.\n"
+            "Do not guess. Do not explain."
+        ),
+    }
+
+
+class SandboxTerminalSmokeEnvConfig(AgentEnvConfig):
+    server_base_url: str = Field(
+        default="http://127.0.0.1:8080",
+        description="Base URL for an OpenAI-compatible chat server (without /v1).",
+    )
+    server_model: str = Field(default="hermes-4-36b", description="Model name")
+    tokenizer_name: str = Field(default="NousResearch/Hermes-4.3-36B", description="Tokenizer name for RL tokenization")
+
+
+class SandboxTerminalSmokeEnv(AgentEnv[SandboxTerminalSmokeEnvConfig]):
+    name = "sandbox_terminal_smoke_env"
+    env_config_cls = SandboxTerminalSmokeEnvConfig
+
+    def __init__(
+        self,
+        config: SandboxTerminalSmokeEnvConfig,
+        server_configs: List[APIServerConfig],
+        slurm: bool = False,
+        testing: bool = False,
+    ):
+        super().__init__(config, server_configs, slurm, testing)
+        self._iter = 0
+
+    @classmethod
+    def config_init(cls) -> Tuple[SandboxTerminalSmokeEnvConfig, List[APIServerConfig]]:
+        base_url = (
+            os.getenv("ATROPOS_SERVER_BASE_URL")
+            or os.getenv("OPENAI_BASE_URL")
+            or os.getenv("LLM_BASE_URL")
+            or "http://127.0.0.1:8080"
+        )
+        model = os.getenv("ATROPOS_SERVER_MODEL") or os.getenv("LLM_MODEL") or "hermes-4-36b"
+        api_key = os.getenv("ATROPOS_SERVER_API_KEY") or os.getenv("NOUS_API_KEY") or os.getenv("OPENAI_API_KEY") or "local"
+
+        env_config = SandboxTerminalSmokeEnvConfig(
+            tokenizer_name=os.getenv("ATROPOS_TOKENIZER_NAME") or "NousResearch/Hermes-4.3-36B",
+            group_size=1,
+            use_wandb=False,
+            include_messages=True,
+            ensure_scores_are_not_same=False,
+            total_steps=2,
+            batch_size=1,
+            server_base_url=base_url,
+            server_model=model,
+            # Tooling: sandbox-only terminal.
+            enabled_toolsets=["terminal"],
+            disabled_toolsets=[],
+            # Default to Nomad sandboxing; users can override via --env.* args.
+            sandbox_image=os.getenv("ATROPOS_SANDBOX_IMAGE") or "atropos-sandbox:local",
+            purge_job_on_start=True,
+            purge_job_on_shutdown=True,
+        )
+
+        server_configs = [
+            APIServerConfig(
+                model_name=model,
+                base_url=f"{base_url.rstrip('/')}/v1",
+                api_key=api_key,
+                num_max_requests_at_once=1,
+                num_requests_for_eval=1,
+                timeout=120,
+            )
+        ]
+        return env_config, server_configs
+
+    async def setup_agent_env(self) -> None:
+        return None
+
+    async def get_next_item(self) -> Item:
+        self._iter += 1
+        return _forced_tool_item()
+
+    def build_task(self, item: Item) -> str:
+        return str(item.get("prompt") or "")
+
+    def build_agent_config(self, item: Item) -> AgentConfig:  # noqa: ARG002
+        # Avoid imposing max_tokens by default; tool-tag responses can be long for some models.
+        return AgentConfig(
+            max_steps=min(8, int(self.config.agent_max_steps)),
+            temperature=0.2,
+            max_tokens=None,
+            system_prompt=STRICT_TOOLCALL_SYSTEM_PROMPT,
+        )
+
+    async def score_trajectory(self, item: Item, final_response: str) -> float:
+        # Scoring happens in verify_and_score_trajectory so we can inspect tool results.
+        _ = (item, final_response)
+        return 0.0
+
+    async def verify_and_score_trajectory(
+        self,
+        item: Item,
+        final_response: str,
+        *,
+        trajectory_id: str,  # noqa: ARG002
+        exec_tool,  # noqa: ARG002
+        agent_result: AgentResult | None = None,
+        workspace_meta: Dict[str, Any] | None = None,  # noqa: ARG002
+    ) -> tuple[float, Dict[str, Any]]:
+        if agent_result is None:
+            return 0.0, {"error": "Missing agent_result"}
+
+        observed: str = ""
+        tool_ok = False
+        for step in agent_result.steps:
+            for res in step.tool_results:
+                if not res.success:
+                    return 0.0, {"error": res.error, "output": res.output}
+                out = (res.output or "").strip()
+                if out:
+                    observed = out.splitlines()[-1].strip()
+                    tool_ok = True
+
+        final = (final_response or "").strip()
+        score = 1.0 if tool_ok and agent_result.total_tool_calls > 0 and observed and final == observed else 0.0
+        return score, {"observed": observed, "tool_calls": agent_result.total_tool_calls, "command": item.get("command")}
+
+
+if __name__ == "__main__":
+    SandboxTerminalSmokeEnv.cli()

atropos/envs/swe_smith_oracle_env.py

@@ -0,0 +1,418 @@
+"""
+SWE-smith-oracle environment.
+
+This environment is intentionally minimal:
+- prepares a sandbox workspace by cloning a public GitHub repo at `base_commit`
+- runs an AtroposAgent tool loop to apply a fix
+- verifies by running pytest nodeids from the dataset (reward = pass/fail)
+- Python only (no multi-language support currently, need to properly bauild & add to dropbox)
+- TODO: Get the other nonpython sandboxes up and running, then add a config knob to switch between them per row
+- oh and add to dockerhub
+
+Dataset: NousResearch/SWE-smith-oracle (train; does NOT use SWE-bench eval set).
+"""
+
+from __future__ import annotations
+
+import os
+import random
+import time
+from typing import Any, Dict, List, Optional, Tuple
+
+from pydantic import Field
+
+from atroposlib.envs.base import APIServerConfig, Item
+
+from ..agent import AgentConfig
+from ..tools import ToolCall
+from .agent_env import AgentEnv, AgentEnvConfig
+
+
+class SweSmithOracleEnvConfig(AgentEnvConfig):
+    dataset_name: str = Field(default="NousResearch/SWE-smith-oracle")
+    dataset_split: str = Field(default="train")
+    max_items: int = Field(default=0, description="0 = no limit")
+    shuffle: bool = Field(default=True)
+    seed: int = Field(default=0)
+
+    python_only: bool = Field(default=True, description="Filter to Python-evaluable rows")
+    score_include_fail_to_pass: bool = Field(
+        default=True,
+        description=(
+            "If true (default), score tests on PASS_TO_PASS ∪ FAIL_TO_PASS. "
+            "Disable to only run PASS_TO_PASS (faster but weaker signal)."
+        ),
+    )
+
+    prompt_mode: str = Field(
+        default="problem_statement",
+        description="Task prompt content: 'problem_statement' (fast) or 'problem_statement+text' (slower, includes dataset 'text').",
+    )
+
+    repo_base_url: str = Field(default="https://github.com", description="Base URL for repo cloning")
+    install_timeout_s: float = Field(default=600.0)
+    test_timeout_s: float = Field(default=600.0)
+
+    tokenizer_name: str = Field(default="NousResearch/Hermes-4.3-36B", description="Tokenizer name for RL tokenization")
+
+
+class SweSmithOracleEnv(AgentEnv[SweSmithOracleEnvConfig]):
+    """
+    SWE-smith-oracle AgentEnv.
+
+    This is designed for benchmarking multiplexed slot execution vs naive container-per-trajectory.
+    """
+
+    name = "swe_smith_oracle_env"
+    env_config_cls = SweSmithOracleEnvConfig
+
+    def __init__(
+        self,
+        config: SweSmithOracleEnvConfig,
+        server_configs: List[APIServerConfig],
+        slurm: bool = False,
+        testing: bool = False,
+    ):
+        super().__init__(config, server_configs, slurm, testing)
+        self._dataset = None
+        self._indices: List[int] = []
+        self._cursor = 0
+
+    @classmethod
+    def config_init(cls) -> Tuple[SweSmithOracleEnvConfig, List[APIServerConfig]]:
+        # Defaults for running the env via CLI in offline `process` mode.
+        # Override via env vars or `--env.*` flags as needed.
+        base_url_raw = (
+            os.getenv("ATROPOS_SERVER_BASE_URL")
+            or os.getenv("OPENAI_BASE_URL")
+            or os.getenv("LLM_BASE_URL")
+            or "http://127.0.0.1:8080"
+        )
+        base_url = base_url_raw.rstrip("/")
+        if not base_url.endswith("/v1"):
+            base_url = f"{base_url}/v1"
+        model = os.getenv("ATROPOS_SERVER_MODEL") or os.getenv("LLM_MODEL") or "hermes-4-36b"
+        api_key = os.getenv("ATROPOS_SERVER_API_KEY") or os.getenv("NOUS_API_KEY") or os.getenv("OPENAI_API_KEY") or "local"
+
+        env_config = SweSmithOracleEnvConfig(
+            tokenizer_name=os.getenv("ATROPOS_TOKENIZER_NAME") or "NousResearch/Hermes-4.3-36B",
+            group_size=1,
+            use_wandb=False,
+            rollout_server_url="http://localhost:8000",
+            total_steps=1,
+            batch_size=1,
+            steps_per_eval=1,
+            max_token_length=8192,
+            inference_weight=1.0,
+            wandb_name="swe_smith_oracle",
+            enabled_toolsets=["terminal"],
+            disabled_toolsets=[],
+            sandbox_image=os.getenv("ATROPOS_SANDBOX_IMAGE") or "atropos-sandbox:local",
+            purge_job_on_start=True,
+            purge_job_on_shutdown=True,
+        )
+
+        server_configs = [
+            APIServerConfig(
+                model_name=model,
+                base_url=base_url,
+                api_key=api_key,
+                num_max_requests_at_once=1,
+                num_requests_for_eval=1,
+                timeout=int(os.getenv("ATROPOS_SERVER_TIMEOUT_S") or "300"),
+            ),
+        ]
+
+        return env_config, server_configs
+
+    async def setup_agent_env(self) -> None:
+        from datasets import load_dataset
+
+        t0 = time.perf_counter()
+        print(
+            f"[SweSmithOracleEnv] loading dataset {self.config.dataset_name}:{self.config.dataset_split} "
+            f"(python_only={self.config.python_only}, max_items={self.config.max_items or 'all'})",
+            flush=True,
+        )
+        ds = load_dataset(self.config.dataset_name, split=self.config.dataset_split)
+        self._dataset = ds
+
+        indices: List[int] = []
+        for idx in range(len(ds)):
+            row = ds[idx]
+            if self.config.python_only and not self._is_python_row(row):
+                continue
+            indices.append(idx)
+
+        if self.config.shuffle:
+            rnd = random.Random(self.config.seed)
+            rnd.shuffle(indices)
+
+        if self.config.max_items and self.config.max_items > 0:
+            indices = indices[: self.config.max_items]
+
+        self._indices = indices
+        self._cursor = 0
+
+        print(
+            f"[SweSmithOracleEnv] loaded {len(self._indices)} items from {self.config.dataset_name}:{self.config.dataset_split} "
+            f"in {time.perf_counter() - t0:.2f}s",
+            flush=True,
+        )
+
+    def _is_python_row(self, row: Dict[str, Any]) -> bool:
+        nodeids = row.get("PASS_TO_PASS")
+        if not isinstance(nodeids, list) or not nodeids:
+            return False
+        for nid in nodeids:
+            if not isinstance(nid, str) or ".py::" not in nid:
+                return False
+        return True
+
+    async def get_next_item(self) -> Item:
+        print(f"[SweSmithOracleEnv] get_next_item() cursor={self._cursor}/{len(self._indices)}", flush=True)
+        if not self._dataset or not self._indices:
+            raise RuntimeError("Dataset not initialized (did setup() run?)")
+        if self._cursor >= len(self._indices):
+            self._cursor = 0
+        idx = self._indices[self._cursor]
+        self._cursor += 1
+        return dict(self._dataset[idx])
+
+    def _repo_name(self, item: Item) -> str:
+        repo = item.get("repo") or ""
+        if isinstance(repo, str) and "/" in repo:
+            return repo.split("/")[-1]
+        return "repo"
+
+    def build_task(self, item: Item) -> str:
+        repo = item.get("repo") or ""
+        base_commit = item.get("base_commit") or ""
+        problem = str(item.get("problem_statement") or "")
+        context = str(item.get("text") or "")
+
+        nodeids = self._tests_for_item(item)
+        tests_list = "\n".join(f"- {t}" for t in nodeids)
+
+        repo_dir = self._repo_name(item)
+
+        tests_block = (
+            "Run these tests to verify:\n"
+            f"{tests_list}\n\n"
+            "When done, briefly describe what you changed and confirm tests pass."
+        )
+
+        prompt_mode = (self.config.prompt_mode or "problem_statement").strip().lower()
+        if prompt_mode not in {"problem_statement", "problem_statement+text"}:
+            raise ValueError(
+                f"Invalid prompt_mode={self.config.prompt_mode!r}. "
+                "Expected 'problem_statement' or 'problem_statement+text'."
+            )
+
+        context_block = ""
+        if prompt_mode == "problem_statement+text" and context:
+            # Note: We intentionally do NOT truncate/cap here. This mode is for debugging / richer prompts and can be slow.
+            context_block = f"\nAdditional context:\n{context}\n"
+
+        return (
+            "You are a senior software engineer. Fix the repository so the specified tests pass.\n\n"
+            f"Repository: {repo} (checked out at base_commit={base_commit})\n"
+            f"Workspace path: ./{repo_dir}\n\n"
+            "Constraints:\n"
+            "- You MUST use the terminal tool to inspect, edit, and verify the repository. Do not respond with a patch file.\n"
+            f"- Start by inspecting the repo (e.g. `ls`, `cd ./{repo_dir}`, `git status`).\n"
+            "- Use a workspace-local virtualenv (e.g. inside the repo at ./.venv) to avoid cross-run contamination.\n"
+            "- Use non-interactive commands only.\n\n"
+            "- Terminal commands run under POSIX /bin/sh and each tool call runs in a fresh shell (no persisted env vars).\n"
+            "  Avoid bash-only `source`; prefer `. .venv/bin/activate` or `.venv/bin/python ...`.\n\n"
+            "Problem statement:\n"
+            f"{problem}\n\n"
+            f"{context_block}\n"
+            f"{tests_block}"
+        )
+
+    def build_agent_config(self, item: Item) -> AgentConfig:  # noqa: ARG002
+        # SWE tasks are longer than the simple test env.
+        return AgentConfig(
+            max_steps=self.config.agent_max_steps,
+            temperature=self.config.agent_temperature,
+            max_tokens=self.config.agent_max_tokens,
+            tool_delay_s=self.config.agent_tool_delay_s,
+        )
+
+    async def setup_trajectory_workspace(self, item: Item, *, trajectory_id: str, exec_tool) -> Dict[str, Any]:
+        t0 = time.perf_counter()
+        repo = item.get("repo")
+        base_commit = item.get("base_commit")
+        instance_id = item.get("instance_id") or item.get("id") or item.get("problem_id")
+        if not isinstance(repo, str) or not isinstance(base_commit, str):
+            raise RuntimeError("Invalid dataset row: missing repo/base_commit")
+
+        repo_dir = self._repo_name(item)
+        clone_url = f"{self.config.repo_base_url.rstrip('/')}/{repo}.git"
+        print(
+            f"[SweSmithOracleEnv] tid={trajectory_id} setup_trajectory_workspace(): "
+            f"repo={repo} base_commit={base_commit} instance_id={instance_id} dir=./{repo_dir}",
+            flush=True,
+        )
+
+        # Repo setup strategy:
+        # - Maintain a shared, per-container bare repo cache under /data/repo_cache
+        # - For each trajectory, create an isolated git worktree under the slot workspace
+        # This avoids cloning/fetching full repos per trajectory and is crucial for multiplexing.
+
+        def _repo_cache_slug(repo_name: str) -> str:
+            return repo_name.replace("/", "__")
+
+        repo_slug = _repo_cache_slug(repo)
+        cache_root = "/data/repo_cache"
+        bare_repo = f"{cache_root}/{repo_slug}.git"
+        lock_file = f"{cache_root}/.locks/{repo_slug}.lock"
+
+        # Use flock to serialize operations that mutate the shared bare repo (fetch/worktree).
+        # util-linux (flock) is included in the sandbox image.
+        worktree_cmd = (
+            "set -e; "
+            f"rm -rf {repo_dir}; "
+            f"mkdir -p {cache_root}/.locks; "
+            f": > {lock_file}; "
+            f"flock -x {lock_file} sh -lc '"
+            f"set -e; "
+            "export GIT_TERMINAL_PROMPT=0; "
+            "export GIT_LFS_SKIP_SMUDGE=1; "
+            f"if [ ! -d \"{bare_repo}\" ]; then "
+            f"  git init --bare \"{bare_repo}\"; "
+            f"  git -C \"{bare_repo}\" remote add origin \"{clone_url}\"; "
+            "fi; "
+            f"git -C \"{bare_repo}\" remote set-url origin \"{clone_url}\"; "
+            f"git -C \"{bare_repo}\" worktree prune || true; "
+            f"if ! git -C \"{bare_repo}\" cat-file -e \"{base_commit}^{{commit}}\" 2>/dev/null; then "
+            f"  git -C \"{bare_repo}\" fetch --depth 1 origin \"{base_commit}\" || true; "
+            "fi; "
+            f"if ! git -C \"{bare_repo}\" cat-file -e \"{base_commit}^{{commit}}\" 2>/dev/null; then "
+            f"  git -C \"{bare_repo}\" fetch --prune origin; "
+            "fi; "
+            f"git --git-dir=\"{bare_repo}\" worktree add --detach \"{repo_dir}\" \"{base_commit}\"; "
+            "'"
+        )
+
+        print(f"[SweSmithOracleEnv] tid={trajectory_id} preparing worktree from repo cache", flush=True)
+        res = await exec_tool(
+            ToolCall(
+                name="terminal",
+                arguments={"command": worktree_cmd, "timeout": self.config.install_timeout_s},
+            )
+        )
+        if not res.success:
+            raise RuntimeError(
+                "git worktree setup failed "
+                f"(repo={repo}, base_commit={base_commit}, instance_id={instance_id}): {res.error}\n{res.output}"
+            )
+
+        print(
+            f"[SweSmithOracleEnv] tid={trajectory_id} setup_trajectory_workspace(): worktree ready in {time.perf_counter() - t0:.2f}s",
+            flush=True,
+        )
+        return {"repo_dir": repo_dir, "base_commit": base_commit}
+
+    def _tests_for_item(self, item: Item) -> List[str]:
+        tests: List[str] = []
+        if self.config.score_include_fail_to_pass:
+            for key in ("PASS_TO_PASS", "FAIL_TO_PASS"):
+                nodeids = item.get(key)
+                if isinstance(nodeids, list):
+                    tests.extend([n for n in nodeids if isinstance(n, str)])
+        else:
+            nodeids = item.get("PASS_TO_PASS")
+            if isinstance(nodeids, list):
+                tests.extend([n for n in nodeids if isinstance(n, str)])
+        # Stable order for reproducibility.
+        return sorted(dict.fromkeys(tests))
+
+    def _chunk_nodeids(self, nodeids: List[str], max_per_chunk: int = 50) -> List[List[str]]:
+        chunks: List[List[str]] = []
+        for i in range(0, len(nodeids), max_per_chunk):
+            chunks.append(nodeids[i : i + max_per_chunk])
+        return chunks
+
+    async def verify_and_score_trajectory(
+        self,
+        item: Item,
+        final_response: str,  # noqa: ARG002
+        *,
+        trajectory_id: str,
+        exec_tool,
+        agent_result=None,
+        workspace_meta: Optional[Dict[str, Any]] = None,
+    ) -> tuple[float, Dict[str, Any]]:
+        _ = trajectory_id
+        repo_dir = self._repo_name(item)
+
+        # Training correctness: do not reward trajectories that never actually used tools.
+        if agent_result is not None and getattr(agent_result, "total_tool_calls", 0) <= 0:
+            print(
+                f"[SweSmithOracleEnv] tid={trajectory_id} verify (dataset_tests): no tool calls; score=0.0",
+                flush=True,
+            )
+            return 0.0, {
+                "verification_mode": "dataset_tests",
+                "error": "No tool calls were made by the agent",
+            }
+
+        nodeids = self._tests_for_item(item)
+        if not nodeids:
+            return 0.0, {"error": "No tests provided"}
+
+        print(f"[SweSmithOracleEnv] tid={trajectory_id} verify (dataset_tests): ensuring venv + deps", flush=True)
+        setup_cmd = (
+            f"cd {repo_dir} && "
+            "python -m venv .venv && "
+            ". .venv/bin/activate && "
+            "python -m pip install -U pip setuptools wheel && "
+            "python -m pip install -e . && "
+            "python -m pip install pytest"
+        )
+        setup_res = await exec_tool(
+            ToolCall(name="terminal", arguments={"command": setup_cmd, "timeout": self.config.install_timeout_s})
+        )
+        verification_messages = [{"role": "user", "content": setup_res.to_xml()}]
+        if not setup_res.success:
+            return 0.0, {
+                "verification_mode": "dataset_tests",
+                "phase": "install",
+                "error": setup_res.error,
+                "output": setup_res.output,
+                "verification_messages": verification_messages,
+            }
+
+        chunks = self._chunk_nodeids(nodeids, max_per_chunk=50)
+        for chunk_idx, chunk in enumerate(chunks):
+            joined = " ".join(chunk)
+            cmd = f"cd {repo_dir} && . .venv/bin/activate && python -m pytest -q {joined}"
+            res = await exec_tool(
+                ToolCall(
+                    name="terminal",
+                    arguments={"command": cmd, "timeout": self.config.test_timeout_s},
+                )
+            )
+            verification_messages.append({"role": "user", "content": res.to_xml()})
+            if not res.success:
+                return 0.0, {
+                    "verification_mode": "dataset_tests",
+                    "phase": "pytest",
+                    "failed_chunk": chunk_idx,
+                    "error": res.error,
+                    "output": res.output,
+                    "verification_messages": verification_messages,
+                }
+
+        return 1.0, {"verification_mode": "dataset_tests", "passed": True, "verification_messages": verification_messages}
+
+    async def score_trajectory(self, item: Item, final_response: str) -> float:
+        # Not used; scoring happens in verify_and_score_trajectory.
+        _ = (item, final_response)
+        return 0.0
+
+
+if __name__ == "__main__":
+    SweSmithOracleEnv.cli()

atropos/envs/test_env.py

@@ -0,0 +1,217 @@
+"""
+Simple test environment for validating the atropos-agent setup.
+
+This environment uses a local OpenAI-compatible server for LLM testing to verify:
+- BaseEnv extension works correctly
+- API communication via OpenAI-compatible endpoint
+- Basic trajectory collection
+
+This is a minimal environment for testing, not production use.
+"""
+
+import os
+from typing import Dict, List, Optional, Tuple
+
+from dotenv import load_dotenv
+from pydantic import Field
+
+from atroposlib.envs.base import (
+    APIServerConfig,
+    Item,
+)
+
+from ..agent import AgentConfig
+from .agent_env import AgentEnv, AgentEnvConfig
+
+# Load environment variables from .env file
+load_dotenv()
+
+
+# Simple test prompts for validation
+TEST_PROMPTS = [
+    {
+        "prompt": "What is 2 + 2? Answer with just the number.",
+        "expected": "4",
+    },
+    {
+        "prompt": "What is the capital of France? Answer with just the city name.",
+        "expected": "Paris",
+    },
+    {
+        "prompt": "What color is the sky on a clear day? Answer with just the color.",
+        "expected": "Blue",
+    },
+    {
+        "prompt": "How many days are in a week? Answer with just the number.",
+        "expected": "7",
+    },
+    {
+        "prompt": "What is 10 * 5? Answer with just the number.",
+        "expected": "50",
+    },
+]
+
+SYSTEM_PROMPT = (
+    "You are a helpful assistant. Answer questions concisely and directly. "
+    "When asked for a simple answer, provide just that answer without explanation."
+)
+
+
+class SimpleTestEnvConfig(AgentEnvConfig):
+    """Configuration for the simple test environment."""
+
+    server_base_url: str = Field(
+        default="http://127.0.0.1:8080",
+        description="Base URL for an OpenAI-compatible server (without /v1)",
+    )
+    server_model: str = Field(
+        default="hermes-4-36b",
+        description="Model name",
+    )
+    tokenizer_name: str = Field(default="NousResearch/Hermes-4.3-36B", description="Tokenizer name for RL tokenization")
+
+
+class SimpleTestEnv(AgentEnv[SimpleTestEnvConfig]):
+    """
+    A simple test environment to validate the atropos-agent setup.
+    
+    Uses a local OpenAI-compatible LLM endpoint with basic question-answering tasks.
+    Scoring is based on whether the response contains the expected answer.
+    """
+
+    name = "simple_test_env"
+    env_config_cls = SimpleTestEnvConfig
+
+    def __init__(
+        self,
+        config: SimpleTestEnvConfig,
+        server_configs: List[APIServerConfig],
+        slurm: bool = False,
+        testing: bool = False,
+    ):
+        super().__init__(config, server_configs, slurm, testing)
+        self.iter = 0
+        self.test_prompts = TEST_PROMPTS
+        self.percent_correct_buffer: List[float] = []
+
+    @classmethod
+    def config_init(cls) -> Tuple[SimpleTestEnvConfig, List[APIServerConfig]]:
+        """
+        Initialize configuration with local server settings from environment variables.
+        """
+        base_url = (
+            os.getenv("ATROPOS_SERVER_BASE_URL")
+            or os.getenv("OPENAI_BASE_URL")
+            or os.getenv("LLM_BASE_URL")
+            or "http://127.0.0.1:8080"
+        )
+        model = os.getenv("ATROPOS_SERVER_MODEL") or os.getenv("LLM_MODEL") or "hermes-4-36b"
+        api_key = os.getenv("ATROPOS_SERVER_API_KEY") or os.getenv("NOUS_API_KEY") or os.getenv("OPENAI_API_KEY") or "local"
+
+        env_config = SimpleTestEnvConfig(
+            tokenizer_name=os.getenv("ATROPOS_TOKENIZER_NAME") or "NousResearch/Hermes-4.3-36B",
+            group_size=4,
+            use_wandb=False,  # Disable wandb for simple testing
+            rollout_server_url="http://localhost:8000",
+            total_steps=10,
+            batch_size=16,
+            steps_per_eval=5,
+            max_token_length=2048,
+            inference_weight=1.0,
+            wandb_name="simple_test",
+            server_base_url=base_url,
+            server_model=model,
+        )
+
+        # OpenAI-compatible servers typically expose chat completions at /v1.
+        server_configs = [
+            APIServerConfig(
+                model_name=model,
+                base_url=f"{base_url}/v1",
+                api_key=api_key,
+                num_max_requests_at_once=4,
+                num_requests_for_eval=8,
+                timeout=120,  # Local models may be slower
+            ),
+        ]
+
+        return env_config, server_configs
+
+    async def setup_agent_env(self):
+        """Setup the environment - load test data."""
+        print(f"SimpleTestEnv setup complete. {len(self.test_prompts)} test prompts loaded.")
+        print(f"Using server at: {self.config.server_base_url}")
+        print(f"Model: {self.config.server_model}")
+
+    async def get_next_item(self) -> Item:
+        """Get the next test prompt."""
+        item = self.test_prompts[self.iter % len(self.test_prompts)]
+        self.iter += 1
+        return item
+
+    def build_task(self, item: Item) -> str:
+        return item["prompt"]
+
+    def build_agent_config(self, item: Item) -> AgentConfig:  # noqa: ARG002
+        return AgentConfig(
+            max_steps=5,
+            temperature=0.7,
+            max_tokens=256,
+            system_prompt=SYSTEM_PROMPT,
+        )
+
+    async def score_trajectory(self, item: Item, final_response: str) -> float:
+        expected = item["expected"].lower()
+        response_lower = (final_response or "").lower()
+        score = 1.0 if expected in response_lower else 0.0
+        self.percent_correct_buffer.append(score)
+        return score
+
+    async def evaluate(self, *args, **kwargs):
+        """
+        Simple evaluation - run through all test prompts once.
+        """
+        correct = 0
+        total = len(self.test_prompts)
+
+        for item in self.test_prompts:
+            messages = [
+                {"role": "system", "content": SYSTEM_PROMPT},
+                {"role": "user", "content": item["prompt"]},
+            ]
+
+            response = await self.server.chat_completion(
+                messages=messages,
+                n=1,
+                max_tokens=256,
+                temperature=0.0,  # Greedy for eval
+                split="eval",
+            )
+
+            response_text = response.choices[0].message.content or ""
+            expected = item["expected"].lower()
+
+            if expected in response_text.lower():
+                correct += 1
+
+        accuracy = correct / total
+        print(f"Evaluation: {correct}/{total} = {accuracy:.2%} accuracy")
+        return {"eval_accuracy": accuracy}
+
+    async def wandb_log(self, wandb_metrics: Optional[Dict] = None):
+        """Log metrics (simplified for testing)."""
+        if wandb_metrics is None:
+            wandb_metrics = {}
+
+        if self.percent_correct_buffer:
+            avg_correct = sum(self.percent_correct_buffer) / len(self.percent_correct_buffer)
+            wandb_metrics["train/percent_correct"] = avg_correct
+            print(f"Train accuracy: {avg_correct:.2%}")
+            self.percent_correct_buffer = []
+
+        await super().wandb_log(wandb_metrics)
+
+
+if __name__ == "__main__":
+    # Allow running as CLI
+    SimpleTestEnv.cli()

atropos/envs/toolserver_smoke_env.py

@@ -0,0 +1,165 @@
+"""
+ToolServer routing smoke environment.
+
+Validates that:
+  - sandbox tools run through Nomad SlotPool (terminal -> bash in sandbox)
+  - external tools run through ToolServer (skills_list)
+
+This env uses ToolServer in-process by default (`tool_server_url="inprocess"`),
+so it is self-contained for local testing.
+
+Run:
+  uv run python -m atropos.envs.toolserver_smoke_env process --env.use_wandb false --env.total_steps 1 --env.group_size 1
+"""
+
+from __future__ import annotations
+
+import os
+from typing import Any, Dict, List, Tuple
+
+from dotenv import load_dotenv
+from pydantic import Field
+
+from atroposlib.envs.base import APIServerConfig, Item
+
+from ..agent import AgentConfig, AgentResult
+from .agent_env import AgentEnv, AgentEnvConfig
+
+load_dotenv()
+
+
+class ToolServerSmokeEnvConfig(AgentEnvConfig):
+    server_base_url: str = Field(
+        default="http://127.0.0.1:8080",
+        description="Base URL for an OpenAI-compatible chat server (without /v1).",
+    )
+    server_model: str = Field(default="hermes-4-36b", description="Model name")
+    tokenizer_name: str = Field(default="NousResearch/Hermes-4.3-36B", description="Tokenizer name for RL tokenization")
+
+
+class ToolServerSmokeEnv(AgentEnv[ToolServerSmokeEnvConfig]):
+    name = "toolserver_smoke_env"
+    env_config_cls = ToolServerSmokeEnvConfig
+
+    def __init__(
+        self,
+        config: ToolServerSmokeEnvConfig,
+        server_configs: List[APIServerConfig],
+        slurm: bool = False,
+        testing: bool = False,
+    ):
+        super().__init__(config, server_configs, slurm, testing)
+        self._iter = 0
+
+    @classmethod
+    def config_init(cls) -> Tuple[ToolServerSmokeEnvConfig, List[APIServerConfig]]:
+        base_url = (
+            os.getenv("ATROPOS_SERVER_BASE_URL")
+            or os.getenv("OPENAI_BASE_URL")
+            or os.getenv("LLM_BASE_URL")
+            or "http://127.0.0.1:8080"
+        )
+        model = os.getenv("ATROPOS_SERVER_MODEL") or os.getenv("LLM_MODEL") or "hermes-4-36b"
+        api_key = os.getenv("ATROPOS_SERVER_API_KEY") or os.getenv("NOUS_API_KEY") or os.getenv("OPENAI_API_KEY") or "local"
+
+        env_config = ToolServerSmokeEnvConfig(
+            tokenizer_name=os.getenv("ATROPOS_TOKENIZER_NAME") or "NousResearch/Hermes-4.3-36B",
+            group_size=1,
+            use_wandb=False,
+            include_messages=True,
+            ensure_scores_are_not_same=False,
+            total_steps=1,
+            batch_size=1,
+            server_base_url=base_url,
+            server_model=model,
+            enabled_toolsets=["terminal", "skills"],
+            disabled_toolsets=[],
+            # Self-contained ToolServer for local smoke.
+            tool_server_url="inprocess",
+            sandbox_image=os.getenv("ATROPOS_SANDBOX_IMAGE") or "atropos-sandbox:local",
+            purge_job_on_start=True,
+            purge_job_on_shutdown=True,
+        )
+
+        server_configs = [
+            APIServerConfig(
+                model_name=model,
+                base_url=f"{base_url.rstrip('/')}/v1",
+                api_key=api_key,
+                num_max_requests_at_once=1,
+                num_requests_for_eval=1,
+                timeout=120,
+            )
+        ]
+        return env_config, server_configs
+
+    async def setup_agent_env(self) -> None:
+        return None
+
+    async def get_next_item(self) -> Item:
+        self._iter += 1
+        return {
+            "prompt": (
+                "You MUST call exactly one tool per assistant message.\n"
+                "\n"
+                "Step 1) Call the skills_list tool (no arguments), then stop.\n"
+                "Step 2) After you receive the tool response, call the terminal tool to run:\n"
+                "python -c \"print('ok')\"\n"
+                "Step 3) After you receive the terminal tool response, answer with just: ok\n"
+                "\n"
+                "Tool call format requirements:\n"
+                "- Every tool call MUST be a complete XML block with a closing tag.\n"
+                "- Do NOT emit a second <tool_call> in the same assistant message.\n"
+                "\n"
+                "Example:\n"
+                "<tool_call>{\"name\": \"skills_list\", \"arguments\": {}}</tool_call>\n"
+                "Do not include anything else in your final answer."
+            )
+        }
+
+    def build_task(self, item: Item) -> str:
+        return str(item.get("prompt") or "")
+
+    def build_agent_config(self, item: Item) -> AgentConfig:  # noqa: ARG002
+        return AgentConfig(
+            max_steps=min(10, int(self.config.agent_max_steps)),
+            temperature=0.2,
+            max_tokens=None,
+        )
+
+    async def score_trajectory(self, item: Item, final_response: str) -> float:
+        _ = (item, final_response)
+        return 0.0
+
+    async def verify_and_score_trajectory(
+        self,
+        item: Item,
+        final_response: str,
+        *,
+        trajectory_id: str,  # noqa: ARG002
+        exec_tool,  # noqa: ARG002
+        agent_result: AgentResult | None = None,
+        workspace_meta: Dict[str, Any] | None = None,  # noqa: ARG002
+    ) -> tuple[float, Dict[str, Any]]:
+        if agent_result is None:
+            return 0.0, {"error": "Missing agent_result"}
+
+        called = {c.name for s in agent_result.steps for c in s.tool_calls}
+        need = {"skills_list", "terminal"}
+        if not need.issubset(called):
+            return 0.0, {"error": f"Missing tool calls: {sorted(need - called)}", "called": sorted(called)}
+
+        terminal_ok = False
+        for step in agent_result.steps:
+            for call, res in zip(step.tool_calls, step.tool_results):
+                if call.name != "terminal":
+                    continue
+                if res.success and (res.output or "").strip().splitlines()[-1].strip() == "ok":
+                    terminal_ok = True
+
+        score = 1.0 if terminal_ok and (final_response or "").strip() == "ok" else 0.0
+        return score, {"called": sorted(called), "final": (final_response or "").strip()}
+
+
+if __name__ == "__main__":
+    ToolServerSmokeEnv.cli()

atropos/nomad/__init__.py

@@ -0,0 +1,11 @@
+"""
+Nomad integration for atropos-agent.
+
+Provides:
+- NomadClient: Client for Nomad HTTP API
+- Job templates for sandbox containers
+"""
+
+from .client import NomadClient
+
+__all__ = ["NomadClient"]

atropos/nomad/client.py

@@ -0,0 +1,500 @@
+"""
+Nomad API Client for atropos-agent.
+
+Provides a simple async client for interacting with the Nomad HTTP API:
+- Submit/stop jobs
+- Query allocations
+- Get allocation addresses
+- Scale jobs up/down
+"""
+
+import asyncio
+import json
+import os
+from dataclasses import dataclass, field
+from enum import Enum
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+import aiohttp
+
+
+class AllocationStatus(Enum):
+    """Nomad allocation status."""
+    PENDING = "pending"
+    RUNNING = "running"
+    COMPLETE = "complete"
+    FAILED = "failed"
+    LOST = "lost"
+
+
+@dataclass
+class Allocation:
+    """Information about a Nomad allocation."""
+    id: str
+    job_id: str
+    task_group: str
+    node_id: str
+    status: AllocationStatus
+    # Network info for reaching the allocation
+    address: Optional[str] = None
+    port: Optional[int] = None
+    
+    @property
+    def http_address(self) -> Optional[str]:
+        """Get full HTTP address for the allocation."""
+        if self.address and self.port:
+            return f"http://{self.address}:{self.port}"
+        return None
+
+
+@dataclass
+class JobStatus:
+    """Status of a Nomad job."""
+    id: str
+    name: str
+    status: str
+    allocations: List[Allocation] = field(default_factory=list)
+    count: int = 0  # Number of task groups
+
+
+class NomadClient:
+    """
+    Async client for Nomad HTTP API.
+    
+    Usage:
+        client = NomadClient(address="http://localhost:4646")
+        
+        # Submit a job
+        await client.submit_job(job_spec)
+        
+        # Get allocations
+        allocs = await client.get_job_allocations("sandbox-python")
+        
+        # Scale job
+        await client.scale_job("sandbox-python", count=5)
+    """
+    
+    def __init__(
+        self,
+        address: str = "http://localhost:4646",
+        token: Optional[str] = None,
+        timeout: float = 30.0,
+    ):
+        self.address = address.rstrip("/")
+        self.token = token or os.environ.get("NOMAD_TOKEN")
+        self.timeout = aiohttp.ClientTimeout(total=timeout)
+        self._session: Optional[aiohttp.ClientSession] = None
+    
+    async def _get_session(self) -> aiohttp.ClientSession:
+        """Get or create HTTP session."""
+        if self._session is None or self._session.closed:
+            headers = {}
+            if self.token:
+                headers["X-Nomad-Token"] = self.token
+            self._session = aiohttp.ClientSession(
+                timeout=self.timeout,
+                headers=headers,
+            )
+        return self._session
+    
+    async def close(self):
+        """Close the HTTP session."""
+        if self._session and not self._session.closed:
+            await self._session.close()
+    
+    async def __aenter__(self):
+        return self
+    
+    async def __aexit__(self, exc_type, exc_val, exc_tb):
+        await self.close()
+    
+    async def _request(
+        self,
+        method: str,
+        path: str,
+        data: Optional[Dict[str, Any]] = None,
+    ) -> Dict[str, Any]:
+        """Make an HTTP request to Nomad API."""
+        session = await self._get_session()
+        url = f"{self.address}{path}"
+        
+        try:
+            async with session.request(method, url, json=data) as response:
+                if response.status == 404:
+                    return {"error": "not_found", "status": 404}
+                
+                text = await response.text()
+                if not text:
+                    return {"status": response.status}
+                
+                try:
+                    result = json.loads(text)
+                except json.JSONDecodeError:
+                    return {"text": text, "status": response.status}
+                
+                if response.status >= 400:
+                    return {"error": result, "status": response.status}
+                
+                return result if isinstance(result, dict) else {"data": result, "status": response.status}
+                
+        except aiohttp.ClientError as e:
+            return {"error": str(e), "status": 0}
+    
+    # Job Operations
+    
+    async def submit_job(self, job_spec: Dict[str, Any]) -> Dict[str, Any]:
+        """
+        Submit a job to Nomad.
+        
+        Args:
+            job_spec: Job specification dict (HCL converted to JSON)
+            
+        Returns:
+            Response with EvalID if successful
+        """
+        return await self._request("POST", "/v1/jobs", {"Job": job_spec})
+    
+    async def stop_job(self, job_id: str, purge: bool = False) -> Dict[str, Any]:
+        """
+        Stop (and optionally purge) a job.
+        
+        Args:
+            job_id: Job identifier
+            purge: If True, completely remove the job
+        """
+        path = f"/v1/job/{job_id}"
+        if purge:
+            path += "?purge=true"
+        return await self._request("DELETE", path)
+    
+    async def get_job(self, job_id: str) -> Optional[Dict[str, Any]]:
+        """Get job details."""
+        result = await self._request("GET", f"/v1/job/{job_id}")
+        if "error" in result and result.get("status") == 404:
+            return None
+        return result
+    
+    async def get_job_status(self, job_id: str) -> Optional[JobStatus]:
+        """Get job status with allocations."""
+        job = await self.get_job(job_id)
+        if not job:
+            return None
+        
+        allocs = await self.get_job_allocations(job_id)
+        
+        # Get count from task groups
+        count = 0
+        task_groups = job.get("TaskGroups", [])
+        for tg in task_groups:
+            count += tg.get("Count", 1)
+        
+        return JobStatus(
+            id=job_id,
+            name=job.get("Name", job_id),
+            status=job.get("Status", "unknown"),
+            allocations=allocs,
+            count=count,
+        )
+    
+    # Allocation Operations
+    
+    async def get_job_allocations(self, job_id: str) -> List[Allocation]:
+        """Get all allocations for a job."""
+        result = await self._request("GET", f"/v1/job/{job_id}/allocations")
+        
+        if "error" in result:
+            return []
+        
+        allocs_data = result.get("data", result) if isinstance(result, dict) else result
+        if not isinstance(allocs_data, list):
+            return []
+        
+        allocations = []
+        for alloc_data in allocs_data:
+            # Parse allocation info
+            alloc_id = alloc_data.get("ID", "")
+            status_str = alloc_data.get("ClientStatus", "unknown")
+            
+            try:
+                status = AllocationStatus(status_str)
+            except ValueError:
+                status = AllocationStatus.PENDING
+            
+            # Get network info - need to fetch detailed allocation for this
+            address = None
+            port = None
+            
+            # First try the summary data
+            resources = alloc_data.get("AllocatedResources") or {}
+            shared = resources.get("Shared") or {}
+            networks = shared.get("Networks") or []
+            
+            # If no networks in summary, fetch detailed allocation
+            if not networks and alloc_id:
+                detailed = await self.get_allocation(alloc_id)
+                if detailed:
+                    resources = detailed.get("AllocatedResources") or {}
+                    shared = resources.get("Shared") or {}
+                    networks = shared.get("Networks") or []
+            
+            if networks:
+                network = networks[0]
+                address = network.get("IP")
+                # Look for dynamic ports OR reserved ports (Singularity/raw_exec uses reserved)
+                dyn_ports = network.get("DynamicPorts") or []
+                reserved_ports = network.get("ReservedPorts") or []
+                for dp in dyn_ports + reserved_ports:
+                    if dp.get("Label") == "http":
+                        port = dp.get("Value")
+                        break
+            
+            allocations.append(Allocation(
+                id=alloc_id,
+                job_id=job_id,
+                task_group=alloc_data.get("TaskGroup", ""),
+                node_id=alloc_data.get("NodeID", ""),
+                status=status,
+                address=address,
+                port=port,
+            ))
+        
+        return allocations
+    
+    async def get_allocation(self, alloc_id: str) -> Optional[Dict[str, Any]]:
+        """Get detailed allocation info."""
+        result = await self._request("GET", f"/v1/allocation/{alloc_id}")
+        if "error" in result and result.get("status") == 404:
+            return None
+        return result
+    
+    # Scaling Operations
+    
+    async def scale_job(self, job_id: str, count: int, task_group: str = "sandbox") -> Dict[str, Any]:
+        """
+        Scale a job's task group to specified count.
+        
+        Args:
+            job_id: Job identifier
+            count: Desired number of allocations
+            task_group: Name of task group to scale
+        """
+        payload = {
+            "Count": count,
+            "Target": {
+                "Group": task_group,
+            },
+        }
+        return await self._request("POST", f"/v1/job/{job_id}/scale", payload)
+    
+    async def get_job_scale_status(self, job_id: str) -> Dict[str, int]:
+        """
+        Get current scale status for a job.
+        
+        Returns:
+            Dict mapping task group name to count
+        """
+        result = await self._request("GET", f"/v1/job/{job_id}/scale")
+        
+        if "error" in result:
+            return {}
+        
+        task_groups = result.get("TaskGroups", {})
+        return {
+            name: info.get("Running", 0)
+            for name, info in task_groups.items()
+        }
+    
+    # Health Check
+    
+    async def is_healthy(self) -> bool:
+        """Check if Nomad is reachable and healthy."""
+        try:
+            result = await self._request("GET", "/v1/status/leader")
+            return "error" not in result
+        except Exception:
+            return False
+    
+    async def get_leader(self) -> Optional[str]:
+        """Get current Nomad leader address."""
+        result = await self._request("GET", "/v1/status/leader")
+        if isinstance(result, dict) and "data" in result:
+            return result["data"]
+        return None
+
+
+def load_job_template(
+    template_name: str = "sandbox",
+    **kwargs,
+) -> Dict[str, Any]:
+    """
+    Load and configure a job template.
+    
+    Args:
+        template_name: Name of template (e.g., "sandbox")
+        **kwargs: Template variables to substitute
+        
+    Returns:
+        Job specification dict ready for Nomad API
+    """
+    # Default job template for sandbox container
+    if template_name == "sandbox":
+        return create_sandbox_job(**kwargs)
+    else:
+        raise ValueError(f"Unknown template: {template_name}")
+
+
+def create_sandbox_job(
+    job_id: str = "atropos-sandbox",
+    image: str = "atropos-sandbox:local",  # Use :local tag to avoid registry pull
+    count: int = 1,
+    slots_per_container: int = 10,
+    privileged: bool = False,
+    cpu: int = 500,
+    memory: int = 512,
+    port: int = 8080,
+    datacenter: str = "dc1",
+    driver: str = "docker",  # "docker" or "singularity"
+    singularity_image: str = None,  # Path to .sif file for singularity driver
+) -> Dict[str, Any]:
+    """
+    Create a sandbox job specification.
+    
+    This job runs the sandbox_server.py inside a container,
+    with the specified number of slots for agent workspaces.
+    
+    Args:
+        job_id: Unique job identifier
+        image: Docker image to use (for docker driver)
+        count: Number of container instances
+        slots_per_container: Number of slots per container
+        privileged: Run container in privileged mode (recommended for bubblewrap)
+        cpu: CPU allocation in MHz
+        memory: Memory allocation in MB
+        port: HTTP port for sandbox server
+        datacenter: Nomad datacenter
+        driver: Container driver - "docker" or "singularity"
+        singularity_image: Path to .sif file (required if driver="singularity")
+        
+    Returns:
+        Job specification dict
+    """
+    # Build task config based on driver
+    if driver == "singularity":
+        if not singularity_image:
+            raise ValueError("singularity_image path required when driver='singularity'")
+        
+        # Use raw_exec driver to run apptainer via shell for variable expansion
+        # The container binds the allocation directory for workspace persistence
+        # For raw_exec, we use static port since Nomad's dynamic port mapping doesn't
+        # work the same as Docker - the process runs directly on the host.
+        shell_cmd = (
+            f'apptainer run '
+            f'--bind "$NOMAD_ALLOC_DIR/data:/data" '
+            f'--pwd /app '
+            f'--env PYTHONUNBUFFERED=1 '
+            f'{singularity_image} '
+            f'python sandbox_server.py '
+            f'--port {port} '
+            f'--slots {slots_per_container} '
+            f'--data-dir /data'
+        )
+        task_config = {
+            "command": "/bin/sh",
+            "args": ["-c", shell_cmd],
+        }
+        task_driver = "raw_exec"
+    else:
+        # Docker driver (default)
+        task_config = {
+            "image": image,
+            "force_pull": False,  # Use local image, don't try to pull
+            "ports": ["http"],
+            "privileged": privileged,
+            "command": "python",
+            "args": [
+                "sandbox_server.py",
+                "--port", str(port),
+                "--slots", str(slots_per_container),
+                "--data-dir", "/data",
+            ],
+            # Note: On Linux, you can mount persistent storage:
+            # "volumes": ["${NOMAD_ALLOC_DIR}/data:/data"],
+            # On macOS/Docker Desktop, skip volumes for PoC
+            # (container /data is ephemeral but works for testing)
+        }
+        task_driver = "docker"
+    
+    # For Singularity/raw_exec, use static ports since the process runs directly on host.
+    # For Docker, use dynamic ports with port mapping.
+    if driver == "singularity":
+        network_config = {
+            "Mode": "host",
+            "ReservedPorts": [
+                {
+                    "Label": "http",
+                    "Value": port,
+                }
+            ],
+        }
+    else:
+        network_config = {
+            "Mode": "host",
+            "DynamicPorts": [
+                {
+                    "Label": "http",
+                    "To": port,
+                }
+            ],
+        }
+    
+    return {
+        "ID": job_id,
+        "Name": job_id,
+        "Type": "service",
+        "Datacenters": [datacenter],
+        "TaskGroups": [
+            {
+                "Name": "sandbox",
+                "Count": count,
+                # Speed up deployments and avoid Consul checks. Without this, Nomad may
+                # keep an "active deployment" around for the default MinHealthyTime,
+                # which blocks immediate scaling under load.
+                "Update": {
+                    "HealthCheck": "task_states",
+                    "MinHealthyTime": 0,
+                },
+                "Networks": [network_config],
+                "Tasks": [
+                    {
+                        "Name": "sandbox-server",
+                        "Driver": task_driver,
+                        "Config": task_config,
+                        "Env": {
+                            "PYTHONUNBUFFERED": "1",
+                            "NOMAD_ALLOC_DIR": "${NOMAD_ALLOC_DIR}",
+                        },
+                        "Resources": {
+                            "CPU": cpu,
+                            "MemoryMB": memory,
+                        },
+                        # Note: Services with Checks require Consul, which we skip for the PoC
+                    }
+                ],
+                "RestartPolicy": {
+                    "Attempts": 3,
+                    "Interval": 300_000_000_000,  # 5 minutes
+                    "Delay": 10_000_000_000,     # 10 seconds
+                    "Mode": "delay",
+                },
+                "ReschedulePolicy": {
+                    "Attempts": 5,
+                    "Interval": 3600_000_000_000,  # 1 hour
+                    "Delay": 30_000_000_000,      # 30 seconds
+                    "DelayFunction": "exponential",
+                    "MaxDelay": 300_000_000_000,  # 5 minutes
+                    "Unlimited": False,
+                },
+            }
+        ],
+    }

atropos/slots/__init__.py

@@ -0,0 +1,20 @@
+"""
+Slot-based multiplexing for atropos-agent.
+
+Provides:
+- Slot: Isolated workspace for a single trajectory
+- SlotPool: Manages slots across Nomad allocations  
+- SandboxExecutor: Executes tools in sandbox containers
+"""
+
+from .executor import SandboxExecutor
+from .pool import SlotPool, SlotPoolConfig
+from .slot import Slot, SlotState
+
+__all__ = [
+    "Slot",
+    "SlotState",
+    "SlotPool",
+    "SlotPoolConfig",
+    "SandboxExecutor",
+]

atropos/slots/executor.py

@@ -0,0 +1,457 @@
+"""
+SandboxExecutor - HTTP client for sandbox container communication.
+
+Sends tool execution requests to sandbox_server.py running inside Nomad containers.
+Supports single and batch execution for efficiency.
+"""
+
+import asyncio
+import uuid
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional, Tuple
+
+import aiohttp
+
+from .slot import Slot, SlotState
+from ..tools.base import ToolCall, ToolResult
+
+
+@dataclass
+class ExecutionRequest:
+    """Request to execute a tool in a slot."""
+    slot: Slot
+    tool_name: str
+    args: Dict[str, Any]
+    execution_id: str = field(default_factory=lambda: str(uuid.uuid4()))
+    timeout: float = 30.0
+
+
+@dataclass
+class ExecutionResult:
+    """Result from sandbox execution."""
+    success: bool
+    output: str = ""
+    error: str = ""
+    execution_id: str = ""
+    slot_id: str = ""
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    
+    def to_tool_result(self) -> ToolResult:
+        """Convert to ToolResult for agent consumption."""
+        return ToolResult(
+            success=self.success,
+            output=self.output,
+            error=self.error,
+            metadata=self.metadata,
+            uniq_id=self.execution_id,
+        )
+
+
+class SandboxExecutor:
+    """
+    HTTP client for executing tools in sandbox containers.
+    
+    Communicates with sandbox_server.py running inside Nomad allocations.
+    Supports both single execution and batched parallel execution.
+    
+    Usage:
+        executor = SandboxExecutor()
+        
+        # Single execution
+        result = await executor.execute(slot, "bash", {"command": "ls"})
+        
+        # Batch execution
+        results = await executor.execute_batch([
+            (slot1, "bash", {"command": "ls"}),
+            (slot2, "write_file", {"path": "test.txt", "content": "hello"}),
+        ])
+    """
+    
+    def __init__(
+        self,
+        timeout: float = 30.0,
+        max_retries: int = 3,
+        retry_delay: float = 1.0,
+    ):
+        self.timeout = aiohttp.ClientTimeout(total=timeout)
+        self.max_retries = max_retries
+        self.retry_delay = retry_delay
+        self._session: Optional[aiohttp.ClientSession] = None
+    
+    async def _get_session(self) -> aiohttp.ClientSession:
+        """Get or create HTTP session."""
+        if self._session is None or self._session.closed:
+            self._session = aiohttp.ClientSession(timeout=self.timeout)
+        return self._session
+    
+    async def close(self):
+        """Close HTTP session."""
+        if self._session and not self._session.closed:
+            await self._session.close()
+    
+    async def __aenter__(self):
+        return self
+    
+    async def __aexit__(self, exc_type, exc_val, exc_tb):
+        await self.close()
+    
+    async def execute(
+        self,
+        slot: Slot,
+        tool_name: str,
+        args: Dict[str, Any],
+        timeout: Optional[float] = None,
+    ) -> ExecutionResult:
+        """
+        Execute a tool in a slot's workspace.
+        
+        Args:
+            slot: Slot to execute in
+            tool_name: Name of tool (bash, read_file, write_file)
+            args: Tool arguments
+            timeout: Optional timeout override
+            
+        Returns:
+            ExecutionResult with output or error
+        """
+        execution_id = str(uuid.uuid4())
+        exec_timeout = timeout or self.timeout.total or 30.0
+        
+        # Mark slot as executing
+        original_state = slot.state
+        try:
+            if slot.state == SlotState.ACQUIRED:
+                slot.start_execution(execution_id)
+            
+            result = await self._send_execute_request(
+                container_addr=slot.container_addr,
+                slot_id=slot.slot_id,
+                tool_name=tool_name,
+                args=args,
+                execution_id=execution_id,
+                timeout=exec_timeout,
+            )
+            result.slot_id = slot.slot_id
+            return result
+            
+        finally:
+            # Restore slot state
+            if slot.state == SlotState.EXECUTING:
+                slot.end_execution()
+    
+    async def _send_execute_request(
+        self,
+        container_addr: str,
+        slot_id: str,
+        tool_name: str,
+        args: Dict[str, Any],
+        execution_id: str,
+        timeout: float,
+    ) -> ExecutionResult:
+        """Send execution request to sandbox server with retry logic."""
+        session = await self._get_session()
+        url = f"{container_addr}/execute"
+        
+        payload = {
+            "slot_id": slot_id,
+            "tool": tool_name,
+            "args": args,
+            "execution_id": execution_id,
+            "timeout": timeout,
+        }
+        
+        last_error = None
+        for attempt in range(self.max_retries):
+            try:
+                async with session.post(url, json=payload) as response:
+                    data = await response.json()
+                    
+                    return ExecutionResult(
+                        success=data.get("success", False),
+                        output=data.get("output", ""),
+                        error=data.get("error", ""),
+                        execution_id=data.get("execution_id", execution_id),
+                        metadata=data.get("metadata", {}),
+                    )
+                    
+            except aiohttp.ClientError as e:
+                last_error = str(e)
+                if attempt < self.max_retries - 1:
+                    await asyncio.sleep(self.retry_delay * (attempt + 1))
+                continue
+            except asyncio.TimeoutError:
+                last_error = f"Request timed out after {timeout}s"
+                break
+            except Exception as e:
+                last_error = str(e)
+                break
+        
+        return ExecutionResult(
+            success=False,
+            error=f"Failed after {self.max_retries} attempts: {last_error}",
+            execution_id=execution_id,
+        )
+    
+    async def execute_batch(
+        self,
+        requests: List[Tuple[Slot, str, Dict[str, Any]]],
+        timeout: Optional[float] = None,
+    ) -> List[ExecutionResult]:
+        """
+        Execute multiple tools in parallel across slots.
+        
+        This is the key optimization - we batch tool calls to maximize
+        container utilization while agents are waiting for LLM responses.
+        
+        Args:
+            requests: List of (slot, tool_name, args) tuples
+            timeout: Optional timeout override
+            
+        Returns:
+            List of ExecutionResults in same order as requests
+        """
+        if not requests:
+            return []
+        
+        # Group requests by container address for batch API
+        by_container: Dict[str, List[Tuple[int, Slot, str, Dict[str, Any], str]]] = {}
+        
+        for idx, (slot, tool_name, args) in enumerate(requests):
+            execution_id = str(uuid.uuid4())
+            container = slot.container_addr
+            
+            if container not in by_container:
+                by_container[container] = []
+            by_container[container].append((idx, slot, tool_name, args, execution_id))
+            
+            # Mark slots as executing
+            if slot.state == SlotState.ACQUIRED:
+                slot.start_execution(execution_id)
+        
+        # Execute batches in parallel
+        exec_timeout = timeout or self.timeout.total or 30.0
+        batch_tasks = []
+        
+        for container_addr, batch_requests in by_container.items():
+            task = self._send_batch_request(
+                container_addr=container_addr,
+                batch_requests=batch_requests,
+                timeout=exec_timeout,
+            )
+            batch_tasks.append(task)
+        
+        # Gather all batch results
+        batch_results = await asyncio.gather(*batch_tasks, return_exceptions=True)
+        
+        # Collect results in original order
+        results: List[Optional[ExecutionResult]] = [None] * len(requests)
+        
+        for batch_result in batch_results:
+            if isinstance(batch_result, Exception):
+                # Mark all in this batch as failed
+                continue
+            
+            for idx, result in batch_result:
+                results[idx] = result
+        
+        # Fill in any missing results
+        for idx, result in enumerate(results):
+            if result is None:
+                slot, tool_name, args = requests[idx]
+                results[idx] = ExecutionResult(
+                    success=False,
+                    error="Batch execution failed",
+                    slot_id=slot.slot_id,
+                )
+        
+        # End execution on all slots
+        for slot, _, _ in requests:
+            if slot.state == SlotState.EXECUTING:
+                slot.end_execution()
+        
+        return results  # type: ignore
+    
+    async def _send_batch_request(
+        self,
+        container_addr: str,
+        batch_requests: List[Tuple[int, Slot, str, Dict[str, Any], str]],
+        timeout: float,
+    ) -> List[Tuple[int, ExecutionResult]]:
+        """Send batch execution request to a single container."""
+        session = await self._get_session()
+        url = f"{container_addr}/batch"
+        
+        # Build batch payload
+        payload = [
+            {
+                "slot_id": slot.slot_id,
+                "tool": tool_name,
+                "args": args,
+                "execution_id": execution_id,
+                "timeout": timeout,
+            }
+            for _, slot, tool_name, args, execution_id in batch_requests
+        ]
+        
+        try:
+            async with session.post(url, json=payload) as response:
+                data = await response.json()
+                
+                if not isinstance(data, list):
+                    raise ValueError(f"Expected list response, got {type(data)}")
+                
+                results = []
+                for i, (idx, slot, _, _, execution_id) in enumerate(batch_requests):
+                    if i < len(data):
+                        item = data[i]
+                        result = ExecutionResult(
+                            success=item.get("success", False),
+                            output=item.get("output", ""),
+                            error=item.get("error", ""),
+                            execution_id=item.get("execution_id", execution_id),
+                            slot_id=slot.slot_id,
+                            metadata=item.get("metadata", {}),
+                        )
+                    else:
+                        result = ExecutionResult(
+                            success=False,
+                            error="Missing result in batch response",
+                            execution_id=execution_id,
+                            slot_id=slot.slot_id,
+                        )
+                    results.append((idx, result))
+                
+                return results
+                
+        except Exception as e:
+            # Return error for all requests in batch
+            return [
+                (idx, ExecutionResult(
+                    success=False,
+                    error=str(e),
+                    execution_id=execution_id,
+                    slot_id=slot.slot_id,
+                ))
+                for idx, slot, _, _, execution_id in batch_requests
+            ]
+    
+    async def reset_slot(self, slot: Slot) -> ExecutionResult:
+        """
+        Reset a slot's workspace (delete all files).
+        
+        Useful when reusing a slot for a new trajectory.
+        """
+        session = await self._get_session()
+        url = f"{slot.container_addr}/reset"
+        
+        try:
+            async with session.post(url, json={"slot_id": slot.slot_id}) as response:
+                data = await response.json()
+                return ExecutionResult(
+                    success=data.get("success", False),
+                    output=data.get("output", ""),
+                    error=data.get("error", ""),
+                    slot_id=slot.slot_id,
+                )
+        except Exception as e:
+            return ExecutionResult(
+                success=False,
+                error=str(e),
+                slot_id=slot.slot_id,
+            )
+    
+    async def health_check(self, container_addr: str) -> bool:
+        """Check if a sandbox container is healthy."""
+        session = await self._get_session()
+        url = f"{container_addr}/health"
+        
+        try:
+            async with session.get(url) as response:
+                data = await response.json()
+                return data.get("status") == "ok"
+        except Exception:
+            return False
+    
+    async def get_container_status(
+        self, 
+        container_addr: str
+    ) -> Optional[Dict[str, Any]]:
+        """Get status info from a sandbox container."""
+        session = await self._get_session()
+        url = f"{container_addr}/health"
+        
+        try:
+            async with session.get(url) as response:
+                return await response.json()
+        except Exception:
+            return None
+
+    # -------------------------------------------------------------------------
+    # Artifact helpers (optional)
+    # -------------------------------------------------------------------------
+
+    async def _post_json(
+        self,
+        url: str,
+        payload: Dict[str, Any],
+        timeout: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        session = await self._get_session()
+        try:
+            async with session.post(url, json=payload, timeout=timeout) as response:
+                data = await response.json()
+                if isinstance(data, dict):
+                    data.setdefault("http_status", response.status)
+                    return data
+                return {"success": False, "error": f"Unexpected response type: {type(data)}", "http_status": response.status}
+        except Exception as e:
+            return {"success": False, "error": str(e)}
+
+    async def read_artifact(
+        self,
+        slot: Slot,
+        path: str,
+        *,
+        encoding: str = "text",
+        max_bytes: Optional[int] = None,
+        include_sha256: bool = False,
+        timeout: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        url = f"{slot.container_addr}/artifacts/read"
+        payload: Dict[str, Any] = {"slot_id": slot.slot_id, "path": path, "encoding": encoding, "include_sha256": include_sha256}
+        if max_bytes is not None:
+            payload["max_bytes"] = max_bytes
+        return await self._post_json(url, payload, timeout=timeout)
+
+    async def list_artifacts(
+        self,
+        slot: Slot,
+        path: str = ".",
+        *,
+        recursive: bool = False,
+        max_entries: Optional[int] = None,
+        timeout: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        url = f"{slot.container_addr}/artifacts/list"
+        payload: Dict[str, Any] = {"slot_id": slot.slot_id, "path": path, "recursive": recursive}
+        if max_entries is not None:
+            payload["max_entries"] = max_entries
+        return await self._post_json(url, payload, timeout=timeout)
+
+    async def archive_artifacts(
+        self,
+        slot: Slot,
+        path: str = ".",
+        *,
+        archive_format: str = "tar.gz",
+        max_bytes: Optional[int] = None,
+        max_entries: Optional[int] = None,
+        timeout: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        url = f"{slot.container_addr}/artifacts/archive"
+        payload: Dict[str, Any] = {"slot_id": slot.slot_id, "path": path, "format": archive_format}
+        if max_bytes is not None:
+            payload["max_bytes"] = max_bytes
+        if max_entries is not None:
+            payload["max_entries"] = max_entries
+        return await self._post_json(url, payload, timeout=timeout)

atropos/slots/pool.py

@@ -0,0 +1,659 @@
+"""
+SlotPool - Manages slots across Nomad allocations.
+
+The SlotPool is the core abstraction for slot-based multiplexing:
+- Tracks available/acquired slots across containers
+- Handles slot acquisition and release
+- Auto-scales Nomad job count based on demand
+- Provides batched tool execution
+"""
+
+import asyncio
+import logging
+import os
+import subprocess
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+from ..nomad.client import (
+    Allocation,
+    AllocationStatus,
+    NomadClient,
+    create_sandbox_job,
+)
+from .executor import ExecutionResult, SandboxExecutor
+from .slot import Slot, SlotState, create_slots_for_allocation
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class SlotPoolConfig:
+    """Configuration for SlotPool."""
+    
+    # Nomad settings
+    nomad_address: str = "http://localhost:4646"
+    job_id: str = "atropos-sandbox"
+    datacenter: str = "dc1"
+    
+    # Container settings
+    image: str = "atropos-sandbox:local"  # Use :local tag to avoid registry pull
+    slots_per_container: int = 10
+    privileged: bool = False
+    cpu: int = 500  # MHz
+    memory: int = 512  # MB
+    
+    # Driver selection: "docker" or "singularity"
+    driver: str = "docker"
+    # Path to .sif file for singularity driver (required if driver="singularity")
+    singularity_image: Optional[str] = None
+    
+    # Scaling settings
+    min_containers: int = 1
+    max_containers: int = 10
+    
+    # Timeouts
+    acquire_timeout: float = 30.0  # Seconds between acquire polls (also triggers scale-up attempts)
+    health_check_interval: float = 30.0  # Seconds between health checks
+    scale_cooldown: float = 60.0  # Seconds between scale operations
+
+    # Job lifecycle
+    purge_job_on_start: bool = False  # Purge any pre-existing job before starting (local dev/training friendly)
+
+    # Local Docker image convenience (macOS/Nomad dev mode)
+    auto_build_local_image: bool = True  # If image endswith :local and is missing, build it from the bundled Dockerfile.
+    dockerfile_path: Optional[str] = None  # Override Dockerfile path (default: Hermes-Agent/atropos/Dockerfile).
+    docker_build_context: Optional[str] = None  # Override build context (default: Hermes-Agent/atropos).
+
+
+class SlotPool:
+    """
+    Manages a pool of slots across Nomad allocations.
+    
+    The SlotPool:
+    - Deploys sandbox containers to Nomad
+    - Tracks slots across all running containers
+    - Handles slot acquisition/release
+    - Auto-scales based on demand
+    - Provides batched execution via SandboxExecutor
+    
+    Usage:
+        config = SlotPoolConfig(
+            nomad_address="http://localhost:4646",
+            job_id="my-sandbox",
+            slots_per_container=10,
+        )
+        
+        pool = SlotPool(config)
+        await pool.start()
+        
+        # Acquire a slot
+        slot = await pool.acquire()
+        
+        # Execute tool
+        result = await pool.execute(slot, "bash", {"command": "ls"})
+        
+        # Release slot
+        await pool.release(slot)
+        
+        # Shutdown
+        await pool.stop()
+    """
+    
+    def __init__(self, config: Optional[SlotPoolConfig] = None):
+        self.config = config or SlotPoolConfig()
+        
+        # Nomad client
+        self.nomad = NomadClient(address=self.config.nomad_address)
+        
+        # Sandbox executor for tool execution
+        self.executor = SandboxExecutor()
+        
+        # Slot tracking
+        self._slots: Dict[str, Slot] = {}  # slot_key -> Slot
+        self._available_queue: asyncio.Queue[str] = asyncio.Queue()
+        self._lock = asyncio.Lock()
+        self._scale_lock = asyncio.Lock()
+        
+        # State
+        self._started = False
+        self._health_task: Optional[asyncio.Task] = None
+        self._scale_task: Optional[asyncio.Task] = None
+        self._last_scale_time = 0.0
+
+    def _default_dockerfile_path(self) -> Path:
+        # Hermes-Agent/atropos/Dockerfile lives next to this module in source checkouts.
+        return Path(__file__).resolve().parents[1] / "Dockerfile"
+
+    def _default_build_context(self) -> Path:
+        return Path(__file__).resolve().parents[1]
+
+    def _docker_image_exists(self, image: str) -> bool:
+        try:
+            proc = subprocess.run(
+                ["docker", "image", "inspect", image],
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.DEVNULL,
+                check=False,
+                env={**os.environ, "DOCKER_CLI_HINTS": "false"},
+            )
+            return proc.returncode == 0
+        except FileNotFoundError:
+            return False
+
+    def _try_build_local_image(self, image: str) -> None:
+        dockerfile = Path(self.config.dockerfile_path) if self.config.dockerfile_path else self._default_dockerfile_path()
+        context = Path(self.config.docker_build_context) if self.config.docker_build_context else self._default_build_context()
+
+        if not dockerfile.exists():
+            raise RuntimeError(
+                f"Sandbox Dockerfile not found at {dockerfile}. "
+                "Build the sandbox image manually or set --env.purge_job_on_start false and provide a non-local image."
+            )
+        if not context.exists():
+            raise RuntimeError(f"Docker build context not found at {context}")
+
+        # Prefer buildx+--load to ensure the image ends up in the local daemon (required by Nomad's docker driver).
+        buildx_cmd = [
+            "docker",
+            "buildx",
+            "build",
+            "--load",
+            "-t",
+            image,
+            "-f",
+            str(dockerfile),
+            str(context),
+        ]
+        proc = subprocess.run(buildx_cmd, check=False, env={**os.environ, "DOCKER_CLI_HINTS": "false"})
+        if proc.returncode == 0:
+            return
+
+        # Fallback to classic docker build if buildx isn't available.
+        build_cmd = ["docker", "build", "-t", image, "-f", str(dockerfile), str(context)]
+        proc2 = subprocess.run(build_cmd, check=False, env={**os.environ, "DOCKER_CLI_HINTS": "false"})
+        if proc2.returncode != 0:
+            raise RuntimeError(
+                f"Failed to build local sandbox image {image}. "
+                f"Tried: {' '.join(buildx_cmd)} and {' '.join(build_cmd)}"
+            )
+
+    def _ensure_local_image(self) -> None:
+        image = (self.config.image or "").strip()
+        if not image.endswith(":local"):
+            return
+        if not self.config.auto_build_local_image:
+            return
+
+        if self._docker_image_exists(image):
+            return
+
+        logger.info(f"Local sandbox image {image} not found; building it now...")
+        self._try_build_local_image(image)
+
+    def _slot_key(self, alloc_id: str, slot_id: str) -> str:
+        """Generate unique key for a slot."""
+        return f"{alloc_id}:{slot_id}"
+    
+    @property
+    def total_slots(self) -> int:
+        """Total number of slots in pool."""
+        return len(self._slots)
+    
+    @property
+    def available_slots(self) -> int:
+        """Number of available slots."""
+        return sum(1 for s in self._slots.values() if s.is_available)
+    
+    @property
+    def acquired_slots(self) -> int:
+        """Number of acquired slots."""
+        return sum(1 for s in self._slots.values() if s.is_acquired)
+    
+    async def start(self) -> None:
+        """
+        Start the slot pool.
+        
+        - Checks if Nomad is healthy
+        - Deploys sandbox job if not running
+        - Discovers existing allocations
+        - Starts health check background task
+        """
+        if self._started:
+            return
+        
+        logger.info(f"Starting SlotPool (job_id={self.config.job_id})")
+
+        try:
+            # Make sure local sandbox images exist before Nomad tries to pull them.
+            # This is a common footgun in macOS dev mode with :local tags.
+            self._ensure_local_image()
+
+            # Check Nomad health
+            if not await self.nomad.is_healthy():
+                raise RuntimeError(f"Nomad is not reachable at {self.config.nomad_address}")
+
+            if self.config.purge_job_on_start:
+                logger.info(f"Purging any existing Nomad job: {self.config.job_id}")
+                await self.nomad.stop_job(self.config.job_id, purge=True)
+
+            # Check if job exists (after optional purge)
+            job = await self.nomad.get_job(self.config.job_id)
+
+            if job is None:
+                # Deploy new job
+                logger.info(f"Deploying sandbox job: {self.config.job_id} (driver={self.config.driver})")
+                job_spec = create_sandbox_job(
+                    job_id=self.config.job_id,
+                    image=self.config.image,
+                    count=self.config.min_containers,
+                    slots_per_container=self.config.slots_per_container,
+                    privileged=self.config.privileged,
+                    cpu=self.config.cpu,
+                    memory=self.config.memory,
+                    datacenter=self.config.datacenter,
+                    driver=self.config.driver,
+                    singularity_image=self.config.singularity_image,
+                )
+                result = await self.nomad.submit_job(job_spec)
+                if "error" in result:
+                    raise RuntimeError(f"Failed to submit job: {result}")
+
+            # Wait for allocations to be running (even if the job already existed).
+            await self._wait_for_healthy_allocations(self.config.min_containers)
+
+            # Discover existing allocations and slots
+            await self._refresh_slots()
+
+            # Start health check task
+            self._health_task = asyncio.create_task(self._health_check_loop())
+
+            self._started = True
+            logger.info(f"SlotPool started: {self.total_slots} slots available")
+        except Exception:
+            # Ensure aiohttp sessions are not leaked if we fail to start.
+            await self.stop(purge_job=False)
+            raise
+    
+    async def stop(self, purge_job: bool = False) -> None:
+        """
+        Stop the slot pool.
+        
+        Args:
+            purge_job: If True, also stop the Nomad job
+        """
+        logger.info("Stopping SlotPool")
+
+        # Cancel health check task
+        if self._health_task:
+            self._health_task.cancel()
+            try:
+                await self._health_task
+            except asyncio.CancelledError:
+                pass
+            finally:
+                self._health_task = None
+
+        if self._scale_task:
+            self._scale_task.cancel()
+            try:
+                await self._scale_task
+            except asyncio.CancelledError:
+                pass
+            finally:
+                self._scale_task = None
+
+        # Optionally stop the job (do this even if start() never completed).
+        if purge_job:
+            logger.info(f"Stopping Nomad job: {self.config.job_id}")
+            await self.nomad.stop_job(self.config.job_id, purge=True)
+
+        # Close connections
+        await self.executor.close()
+        await self.nomad.close()
+
+        self._started = False
+        self._slots.clear()
+
+        # Clear the queue
+        while not self._available_queue.empty():
+            try:
+                self._available_queue.get_nowait()
+            except asyncio.QueueEmpty:
+                break
+    
+    async def acquire(self, trajectory_id: Optional[str] = None) -> Slot:
+        """
+        Acquire an available slot.
+        
+        If no slots are available, waits up to acquire_timeout seconds.
+        If still no slots, attempts to scale up.
+        
+        Args:
+            trajectory_id: Optional ID of trajectory acquiring the slot
+            
+        Returns:
+            Acquired Slot
+            
+        Raises:
+            asyncio.TimeoutError: If no slot becomes available
+        """
+        if not self._started:
+            raise RuntimeError("SlotPool not started")
+
+        while True:
+            try:
+                # Try to get an available slot
+                slot_key = await asyncio.wait_for(
+                    self._available_queue.get(),
+                    timeout=self.config.acquire_timeout,
+                )
+            except asyncio.TimeoutError:
+                # Try to scale up, but keep waiting even if scaling isn't possible.
+                # In practice, slots may become available shortly (e.g. contention),
+                # and scaling may be temporarily blocked by Nomad deployments.
+                await self._try_scale_up()
+                continue
+
+            slot = self._slots.get(slot_key)
+            if slot is None:
+                # Slot was removed; discard stale queue entry and retry.
+                continue
+
+            try:
+                slot.acquire(trajectory_id)
+            except RuntimeError:
+                # Slot isn't actually available (e.g. duplicate queue entry); retry.
+                continue
+
+            logger.debug(f"Acquired slot {slot.slot_id} (alloc={slot.alloc_id[:8]})")
+            return slot
+    
+    async def release(self, slot: Slot, reset_workspace: bool = False) -> None:
+        """
+        Release a slot back to the pool.
+        
+        Args:
+            slot: Slot to release
+            reset_workspace: If True, clear the workspace files
+        """
+        slot_key = self._slot_key(slot.alloc_id, slot.slot_id)
+        
+        if slot_key not in self._slots:
+            logger.warning(f"Releasing unknown slot: {slot_key}")
+            return
+        
+        # Optionally reset workspace
+        if reset_workspace:
+            await self.executor.reset_slot(slot)
+        
+        slot.release()
+        await self._available_queue.put(slot_key)
+        
+        logger.debug(f"Released slot {slot.slot_id}")
+    
+    async def execute(
+        self,
+        slot: Slot,
+        tool_name: str,
+        args: Dict[str, Any],
+        timeout: Optional[float] = None,
+    ) -> ExecutionResult:
+        """
+        Execute a tool in a slot's workspace.
+        
+        Args:
+            slot: Slot to execute in
+            tool_name: Name of tool (bash, read_file, write_file)
+            args: Tool arguments
+            timeout: Optional timeout override
+            
+        Returns:
+            ExecutionResult
+        """
+        return await self.executor.execute(slot, tool_name, args, timeout)
+    
+    async def execute_batch(
+        self,
+        requests: List[Tuple[Slot, str, Dict[str, Any]]],
+        timeout: Optional[float] = None,
+    ) -> List[ExecutionResult]:
+        """
+        Execute multiple tools in parallel.
+        
+        This is the key optimization - batch execution across multiple slots
+        maximizes container utilization.
+        
+        Args:
+            requests: List of (slot, tool_name, args) tuples
+            timeout: Optional timeout override
+            
+        Returns:
+            List of ExecutionResults in same order
+        """
+        return await self.executor.execute_batch(requests, timeout)
+    
+    async def _refresh_slots(self) -> None:
+        """Refresh slot inventory from Nomad allocations."""
+        async with self._lock:
+            allocs = await self.nomad.get_job_allocations(self.config.job_id)
+            
+            # Track which slots we've seen
+            seen_keys = set()
+            
+            for alloc in allocs:
+                if alloc.status != AllocationStatus.RUNNING:
+                    continue
+                
+                if not alloc.http_address:
+                    continue
+                
+                # Check container health
+                healthy = await self.executor.health_check(alloc.http_address)
+                if not healthy:
+                    continue
+                
+                # Create slots for this allocation
+                for i in range(self.config.slots_per_container):
+                    slot_id = f"slot_{i}"
+                    slot_key = self._slot_key(alloc.id, slot_id)
+                    seen_keys.add(slot_key)
+                    
+                    if slot_key not in self._slots:
+                        # New slot
+                        slot = Slot(
+                            slot_id=slot_id,
+                            alloc_id=alloc.id,
+                            container_addr=alloc.http_address,
+                        )
+                        self._slots[slot_key] = slot
+                        await self._available_queue.put(slot_key)
+                        logger.debug(f"Added slot: {slot_key}")
+            
+            # Remove slots from dead allocations
+            for slot_key in list(self._slots.keys()):
+                if slot_key not in seen_keys:
+                    slot = self._slots.pop(slot_key)
+                    logger.debug(f"Removed slot: {slot_key}")
+    
+    async def _wait_for_healthy_allocations(
+        self, 
+        min_count: int, 
+        timeout: float = 120.0
+    ) -> None:
+        """Wait for allocations to become healthy."""
+        import time
+        start = time.time()
+
+        def _summarize_alloc_detail(detail: Dict[str, Any]) -> str:
+            task_states = detail.get("TaskStates") or {}
+            parts: List[str] = []
+            if isinstance(task_states, dict):
+                for task_name, st in task_states.items():
+                    events = (st or {}).get("Events") or []
+                    if isinstance(events, list) and events:
+                        # Include a few recent events; the latest can be a generic restart message
+                        # while the true root cause is slightly earlier (e.g. image pull failure).
+                        recent = events[-3:]
+                        msgs: List[str] = []
+                        for ev in recent:
+                            desc = ev.get("DisplayMessage") or ev.get("Message") or ev.get("Type") or ""
+                            if desc:
+                                msgs.append(desc)
+                        if msgs:
+                            parts.append(f"{task_name}: " + " | ".join(msgs))
+            return "; ".join(parts)
+
+        def _alloc_events_lower(detail: Dict[str, Any]) -> str:
+            task_states = detail.get("TaskStates") or {}
+            texts: List[str] = []
+            if isinstance(task_states, dict):
+                for _task_name, st in task_states.items():
+                    events = (st or {}).get("Events") or []
+                    if isinstance(events, list):
+                        for ev in events[-10:]:
+                            desc = ev.get("DisplayMessage") or ev.get("Message") or ev.get("Type") or ""
+                            if desc:
+                                texts.append(desc)
+            return " ".join(texts).lower()
+        
+        while time.time() - start < timeout:
+            allocs = await self.nomad.get_job_allocations(self.config.job_id)
+            
+            healthy_count = 0
+            for alloc in allocs:
+                if alloc.status == AllocationStatus.RUNNING and alloc.http_address:
+                    if await self.executor.health_check(alloc.http_address):
+                        healthy_count += 1
+
+                # Fast-fail on obvious driver/image errors to avoid waiting out the full timeout.
+                if alloc.id:
+                    detail = await self.nomad.get_allocation(alloc.id)
+                    if isinstance(detail, dict):
+                        summary = _summarize_alloc_detail(detail)
+                        lowered = _alloc_events_lower(detail) or summary.lower()
+                        if "failed to pull" in lowered or "pull access denied" in lowered:
+                            raise RuntimeError(
+                                "Nomad allocation failed to start due to a Docker image pull error. "
+                                f"Allocation {alloc.id[:8]}: {summary}\n"
+                                "If you're using a local image tag (e.g. `atropos-sandbox:local`) on macOS, "
+                                "make sure the image is loaded into Docker, e.g.:\n"
+                                "  docker buildx build --load -t atropos-sandbox:local -f Hermes-Agent/atropos/Dockerfile Hermes-Agent/atropos"
+                            )
+                        if "exceeded allowed attempts" in lowered:
+                            raise RuntimeError(
+                                "Nomad allocation is crash-looping and has entered restart backoff. "
+                                f"Allocation {alloc.id[:8]}: {summary}\n"
+                                "Inspect logs with:\n"
+                                f"  nomad alloc logs -stderr -task sandbox-server {alloc.id}\n"
+                                "Common causes include: missing local Docker image tag, container entrypoint error, "
+                                "or sandbox-server startup failure."
+                            )
+            
+            if healthy_count >= min_count:
+                return
+            
+            await asyncio.sleep(2.0)
+
+        # Timed out: include allocation status detail to help debugging.
+        allocs = await self.nomad.get_job_allocations(self.config.job_id)
+        alloc_lines: List[str] = []
+        for alloc in allocs[:10]:
+            addr = alloc.http_address or "-"
+            line = f"{alloc.id[:8]} status={alloc.status.value} http={addr}"
+            detail = await self.nomad.get_allocation(alloc.id)
+            if isinstance(detail, dict):
+                summary = _summarize_alloc_detail(detail)
+                if summary:
+                    line += f" detail={summary}"
+            alloc_lines.append(line)
+
+        hint = (
+            "Timed out waiting for healthy sandbox allocations.\n"
+            f"Job: {self.config.job_id}, desired_healthy: {min_count}\n"
+            "Allocations:\n  - " + "\n  - ".join(alloc_lines)
+        )
+        raise RuntimeError(hint)
+    
+    async def _try_scale_up(self) -> bool:
+        """Attempt to scale up the job."""
+        import time
+
+        async with self._scale_lock:
+            # Check cooldown
+            if time.time() - self._last_scale_time < self.config.scale_cooldown:
+                return False
+
+            # Check max containers
+            status = await self.nomad.get_job_status(self.config.job_id)
+            if status is None:
+                return False
+
+            current_count = status.count
+            if current_count >= self.config.max_containers:
+                logger.warning(f"Cannot scale up: already at max ({self.config.max_containers})")
+                return False
+
+            # Scale up
+            new_count = min(current_count + 1, self.config.max_containers)
+            logger.info(f"Scaling up from {current_count} to {new_count} containers")
+
+            scale_resp = await self.nomad.scale_job(
+                self.config.job_id,
+                count=new_count,
+                task_group="sandbox",
+            )
+
+            # Nomad may return non-JSON errors (e.g. plain text) with a status field.
+            if isinstance(scale_resp, dict) and scale_resp.get("status", 200) >= 400:
+                logger.warning(f"Scale request rejected: {scale_resp}")
+                self._last_scale_time = time.time()
+                return False
+
+            self._last_scale_time = time.time()
+
+            # Wait for new allocation in the background so contended acquires can still
+            # make progress (e.g. by grabbing slots released by other trajectories).
+            if self._scale_task is None or self._scale_task.done():
+                self._scale_task = asyncio.create_task(self._wait_for_scale(new_count))
+
+            return True
+
+    async def _wait_for_scale(self, desired_count: int) -> None:
+        try:
+            await self._wait_for_healthy_allocations(desired_count, timeout=60.0)
+            await self._refresh_slots()
+        except asyncio.CancelledError:
+            raise
+        except Exception as e:
+            logger.error(f"Failed to scale up: {e}")
+    
+    async def _health_check_loop(self) -> None:
+        """Background task to monitor container health."""
+        while True:
+            try:
+                await asyncio.sleep(self.config.health_check_interval)
+                await self._refresh_slots()
+            except asyncio.CancelledError:
+                break
+            except Exception as e:
+                logger.error(f"Health check error: {e}")
+    
+    def get_stats(self) -> Dict[str, Any]:
+        """Get pool statistics."""
+        slots_by_state = {}
+        for slot in self._slots.values():
+            state = slot.state.value
+            slots_by_state[state] = slots_by_state.get(state, 0) + 1
+
+        container_count = len({s.alloc_id for s in self._slots.values()}) if self._slots else 0
+        
+        return {
+            "total_slots": self.total_slots,
+            "available_slots": self.available_slots,
+            "acquired_slots": self.acquired_slots,
+            "containers": container_count,
+            "slots_by_state": slots_by_state,
+            "started": self._started,
+        }

atropos/slots/slot.py

@@ -0,0 +1,159 @@
+"""
+Slot abstraction for atropos-agent.
+
+A Slot represents an isolated workspace for a single agent trajectory.
+Slots are hosted on Nomad allocations and provide workspace isolation
+via filesystem directories.
+"""
+
+from dataclasses import dataclass, field
+from enum import Enum
+from typing import Any, Dict, Optional
+import uuid
+
+
+class SlotState(Enum):
+    """State of a slot in the pool."""
+    AVAILABLE = "available"      # Ready to be acquired
+    ACQUIRED = "acquired"        # Assigned to a trajectory
+    EXECUTING = "executing"      # Currently executing a tool
+    RELEASING = "releasing"      # Being released back to pool
+    ERROR = "error"              # In error state
+
+
+@dataclass
+class Slot:
+    """
+    An isolated workspace for a single agent trajectory.
+    
+    Slots are the unit of scheduling - each trajectory runs in its own slot,
+    with an isolated workspace directory. Multiple slots share a container.
+    
+    Attributes:
+        slot_id: Unique identifier for this slot (e.g., "slot_0")
+        alloc_id: Nomad allocation ID hosting this slot
+        container_addr: HTTP address of the sandbox server (e.g., "http://10.0.0.1:8080")
+        workspace_dir: Path to workspace in container (e.g., "/data/slot_0")
+        state: Current state of the slot
+        trajectory_id: ID of trajectory currently using this slot (if acquired)
+        metadata: Additional metadata
+    """
+    slot_id: str
+    alloc_id: str
+    container_addr: str
+    workspace_dir: str = ""
+    state: SlotState = SlotState.AVAILABLE
+    trajectory_id: Optional[str] = None
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    
+    def __post_init__(self):
+        """Set default workspace_dir if not provided."""
+        if not self.workspace_dir:
+            self.workspace_dir = f"/data/{self.slot_id}"
+    
+    @property
+    def is_available(self) -> bool:
+        """Check if slot is available for acquisition."""
+        return self.state == SlotState.AVAILABLE
+    
+    @property
+    def is_acquired(self) -> bool:
+        """Check if slot is currently acquired."""
+        return self.state in (SlotState.ACQUIRED, SlotState.EXECUTING)
+    
+    def acquire(self, trajectory_id: Optional[str] = None) -> None:
+        """
+        Mark slot as acquired by a trajectory.
+        
+        Args:
+            trajectory_id: Optional ID of acquiring trajectory
+        """
+        if not self.is_available:
+            raise RuntimeError(f"Cannot acquire slot {self.slot_id}: state is {self.state}")
+        
+        self.state = SlotState.ACQUIRED
+        self.trajectory_id = trajectory_id or str(uuid.uuid4())
+    
+    def start_execution(self, execution_id: Optional[str] = None) -> None:
+        """Mark slot as executing."""
+        if self.state != SlotState.ACQUIRED:
+            raise RuntimeError(f"Cannot start execution on slot {self.slot_id}: state is {self.state}")
+        
+        self.state = SlotState.EXECUTING
+        if execution_id:
+            self.metadata["current_execution_id"] = execution_id
+    
+    def end_execution(self) -> None:
+        """Mark execution as complete, return to acquired state."""
+        if self.state != SlotState.EXECUTING:
+            raise RuntimeError(f"Cannot end execution on slot {self.slot_id}: state is {self.state}")
+        
+        self.state = SlotState.ACQUIRED
+        self.metadata.pop("current_execution_id", None)
+    
+    def release(self) -> None:
+        """Release slot back to available state."""
+        self.state = SlotState.AVAILABLE
+        self.trajectory_id = None
+        self.metadata.pop("current_execution_id", None)
+    
+    def mark_error(self, error: str) -> None:
+        """Mark slot as in error state."""
+        self.state = SlotState.ERROR
+        self.metadata["error"] = error
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to dictionary for serialization."""
+        return {
+            "slot_id": self.slot_id,
+            "alloc_id": self.alloc_id,
+            "container_addr": self.container_addr,
+            "workspace_dir": self.workspace_dir,
+            "state": self.state.value,
+            "trajectory_id": self.trajectory_id,
+            "metadata": self.metadata,
+        }
+    
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "Slot":
+        """Create from dictionary."""
+        return cls(
+            slot_id=data["slot_id"],
+            alloc_id=data["alloc_id"],
+            container_addr=data["container_addr"],
+            workspace_dir=data.get("workspace_dir", ""),
+            state=SlotState(data.get("state", "available")),
+            trajectory_id=data.get("trajectory_id"),
+            metadata=data.get("metadata", {}),
+        )
+    
+    def __repr__(self) -> str:
+        return f"Slot({self.slot_id}, state={self.state.value}, alloc={self.alloc_id[:8]}...)"
+
+
+def create_slots_for_allocation(
+    alloc_id: str,
+    container_addr: str,
+    num_slots: int = 10,
+) -> list["Slot"]:
+    """
+    Create slots for a Nomad allocation.
+    
+    Args:
+        alloc_id: Nomad allocation ID
+        container_addr: HTTP address of sandbox server
+        num_slots: Number of slots to create
+        
+    Returns:
+        List of Slot objects
+    """
+    slots = []
+    for i in range(num_slots):
+        slot_id = f"slot_{i}"
+        slots.append(Slot(
+            slot_id=slot_id,
+            alloc_id=alloc_id,
+            container_addr=container_addr,
+            workspace_dir=f"/data/{slot_id}",
+        ))
+    return slots

atropos/terminal/__init__.py

@@ -0,0 +1,2 @@
+"""Terminal helpers for stateful sandbox interactions."""
+

atropos/terminal/asciinema_stream.py

@@ -0,0 +1,115 @@
+from __future__ import annotations
+
+import json
+from typing import Any
+
+import pyte
+
+
+class AsciinemaStreamDecoder:
+    def __init__(self, *, default_width: int = 80, default_height: int = 24) -> None:
+        self._default_width = max(1, int(default_width))
+        self._default_height = max(1, int(default_height))
+        self._buffer = ""
+        self._has_header = False
+        self.width = self._default_width
+        self.height = self._default_height
+        self._screen = pyte.Screen(self.width, self.height)
+        self._stream = pyte.Stream(self._screen)
+
+    def reset(self) -> None:
+        self._buffer = ""
+        self._has_header = False
+        self.width = self._default_width
+        self.height = self._default_height
+        self._screen = pyte.Screen(self.width, self.height)
+        self._stream = pyte.Stream(self._screen)
+
+    def feed(self, chunk: str | bytes) -> None:
+        if not chunk:
+            return
+        if isinstance(chunk, bytes):
+            chunk = chunk.decode("utf-8", errors="replace")
+        self._buffer += chunk
+        while True:
+            line, sep, rest = self._buffer.partition("\n")
+            if not sep:
+                break
+            self._buffer = rest
+            line = line.strip()
+            if not line:
+                continue
+            parsed = self._parse_json_line(line)
+            if parsed is None:
+                continue
+            if not self._has_header:
+                if isinstance(parsed, dict):
+                    self._init_from_header(parsed)
+                    continue
+                if isinstance(parsed, list):
+                    self._has_header = True
+                    self._apply_event(parsed)
+                    continue
+                continue
+            if isinstance(parsed, list):
+                self._apply_event(parsed)
+
+    def render(self) -> str:
+        return "\n".join(self._screen.display)
+
+    def _parse_json_line(self, line: str) -> Any | None:
+        try:
+            return json.loads(line)
+        except json.JSONDecodeError:
+            return None
+
+    def _init_from_header(self, header: dict[str, Any]) -> None:
+        width = _coerce_int(
+            header.get("width") or header.get("columns") or header.get("cols"),
+            self._default_width,
+        )
+        height = _coerce_int(
+            header.get("height") or header.get("rows") or header.get("lines"),
+            self._default_height,
+        )
+        self.width = max(1, width)
+        self.height = max(1, height)
+        self._screen = pyte.Screen(self.width, self.height)
+        self._stream = pyte.Stream(self._screen)
+        self._has_header = True
+
+    def _apply_event(self, event: list[Any]) -> None:
+        if len(event) < 2:
+            return
+        event_type = event[1]
+        payload = event[2] if len(event) > 2 else ""
+        if event_type == "o":
+            if isinstance(payload, str):
+                self._stream.feed(payload)
+        elif event_type == "r":
+            width, height = _parse_resize(payload)
+            if width and height:
+                self.width = width
+                self.height = height
+                self._screen.resize(width, height)
+
+
+def _coerce_int(value: Any, default: int) -> int:
+    try:
+        return int(value)
+    except (TypeError, ValueError):
+        return int(default)
+
+
+def _parse_resize(payload: Any) -> tuple[int, int]:
+    if isinstance(payload, str) and "x" in payload:
+        left, right = payload.lower().split("x", 1)
+        return _coerce_int(left, 0), _coerce_int(right, 0)
+    if isinstance(payload, dict):
+        width = _coerce_int(payload.get("width") or payload.get("columns") or payload.get("cols"), 0)
+        height = _coerce_int(payload.get("height") or payload.get("rows") or payload.get("lines"), 0)
+        return width, height
+    if isinstance(payload, list) and len(payload) >= 2:
+        return _coerce_int(payload[0], 0), _coerce_int(payload[1], 0)
+    return 0, 0
+

atropos/tools/__init__.py

@@ -0,0 +1,26 @@
+"""
+Tool abstractions for atropos-agent.
+
+Provides base Tool class and common tool implementations.
+"""
+
+from .base import Tool, ToolCall, ToolRegistry, ToolResult, ToolSchema
+from .build_registry import build_tool_registry
+from .sandbox_stubs import BashTool, ReadFileTool, TerminalTool, WriteFileTool
+from .terminal_stateful_tool import TerminalStatefulTool
+from .tmux_tool import TmuxTool
+
+__all__ = [
+    "Tool",
+    "ToolCall",
+    "ToolRegistry",
+    "ToolResult",
+    "ToolSchema",
+    "BashTool",
+    "ReadFileTool",
+    "WriteFileTool",
+    "TerminalTool",
+    "TerminalStatefulTool",
+    "TmuxTool",
+    "build_tool_registry",
+]

atropos/tools/base.py

@@ -0,0 +1,423 @@
+"""
+Base Tool abstraction for atropos-agent.
+
+Tools follow a simple pattern:
+1. Define schema (name, description, parameters)
+2. Implement execute() method
+3. Return ToolResult with output/error
+
+Tool calls use Hermes-style XML tags:
+<tool_call>{"name": "bash", "arguments": {"command": "ls"}}</tool_call>
+"""
+
+import json
+import re
+import uuid
+from abc import ABC, abstractmethod
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Literal, Optional
+
+from pydantic import BaseModel, Field
+
+
+@dataclass
+class ToolSchema:
+    """JSON Schema for a tool's parameters."""
+    
+    name: str
+    description: str
+    parameters: Dict[str, Any] = field(default_factory=dict)
+    required: List[str] = field(default_factory=list)
+    external: bool = False  # Whether the tool must be executed via an external ToolServer (secret proxy) and not inside the sandbox.
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to OpenAI-compatible function schema."""
+        return {
+            "type": "function",
+            "function": {
+                "name": self.name,
+                "description": self.description,
+                "parameters": {
+                    "type": "object",
+                    "properties": self.parameters,
+                    "required": self.required,
+                },
+            },
+        }
+    
+    def to_prompt_description(self) -> str:
+        """Convert to human-readable description for system prompt."""
+        params_desc = []
+        for name, spec in self.parameters.items():
+            req = "(required)" if name in self.required else "(optional)"
+            desc = spec.get("description", "")
+            param_type = spec.get("type", "string")
+            params_desc.append(f"  - {name} ({param_type}) {req}: {desc}")
+        
+        params_str = "\n".join(params_desc) if params_desc else "  (no parameters)"
+        return f"**{self.name}**: {self.description}\nParameters:\n{params_str}"
+
+
+@dataclass
+class ToolCall:
+    """A parsed tool call from model output."""
+    
+    name: str
+    arguments: Dict[str, Any]
+    raw_text: str = ""  # Original XML/JSON text
+    uniq_id: str = field(default_factory=lambda: str(uuid.uuid4()))  # Unique tool-call id for traceability/reconstruction.
+    
+    @classmethod
+    def parse_from_text(cls, text: str) -> List["ToolCall"]:
+        """
+        Extract tool calls from text using Hermes-style XML tags.
+        
+        Supported formats (STRICT: requires well-formed closing tags):
+        - Hermes JSON wrapper:
+          <tool_call>{"name": "...", "arguments": {...}}</tool_call>
+        - GLM/llama.cpp style:
+          <tool_call>terminal{"command":"ls -la"}</tool_call>
+        """
+        calls: List["ToolCall"] = []
+
+        if not text:
+            return calls
+
+        def _append_from_payload(*, name: str, arguments: Dict[str, Any], raw: str, uniq_id: Optional[str] = None) -> None:
+            if not isinstance(name, str) or not name:
+                return
+            if not isinstance(arguments, dict):
+                return
+            calls.append(
+                cls(
+                    name=name,
+                    arguments=arguments,
+                    raw_text=raw,
+                    uniq_id=uniq_id or str(uuid.uuid4()),
+                )
+            )
+
+        # STRICT parsing: only accept well-formed <tool_call>...</tool_call> blocks.
+        pattern = r"<tool_call>\s*(.*?)\s*</tool_call>"
+        for inner in re.findall(pattern, text, re.DOTALL):
+            cleaned = (inner or "").strip()
+            if not cleaned:
+                continue
+
+            # Hermes JSON wrapper.
+            if cleaned.startswith("{"):
+                try:
+                    data = json.loads(cleaned)
+                except json.JSONDecodeError:
+                    continue
+                uniq_id = data.get("uniq_id") or data.get("id") or None
+                _append_from_payload(
+                    name=data.get("name", ""),
+                    arguments=data.get("arguments", {}),
+                    raw=inner,
+                    uniq_id=uniq_id,
+                )
+                continue
+
+            # GLM/llama.cpp style: terminal{...}
+            m = re.match(r"^\s*([A-Za-z0-9_.:\\-]+)\s*(\{.*\})\s*$", cleaned, re.DOTALL)
+            if not m:
+                continue
+            name = m.group(1)
+            args_text = m.group(2)
+            try:
+                args = json.loads(args_text)
+            except json.JSONDecodeError:
+                continue
+            _append_from_payload(name=name, arguments=args, raw=inner)
+
+        return calls
+    
+    @classmethod
+    def has_tool_call(cls, text: str) -> bool:
+        """Check if text contains any tool calls."""
+        return bool(re.search(r"<tool_call>", text))
+
+
+@dataclass
+class ToolResult:
+    """Result from executing a tool."""
+    
+    success: bool
+    output: str = ""
+    error: str = ""
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    uniq_id: Optional[str] = None  # Should match ToolCall.uniq_id for async execution tracking.
+    
+    def to_xml(self) -> str:
+        """Format as XML for including in conversation."""
+        data = {
+            "success": self.success,
+            "output": self.output,
+        }
+        if self.uniq_id:
+            data["uniq_id"] = self.uniq_id
+        if self.error:
+            data["error"] = self.error
+        if self.metadata:
+            data["metadata"] = self.metadata
+        return f"<tool_response>{json.dumps(data)}</tool_response>"
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to dictionary."""
+        return {
+            "success": self.success,
+            "output": self.output,
+            "error": self.error,
+            "metadata": self.metadata,
+            "uniq_id": self.uniq_id,
+        }
+
+
+class Tool(ABC):
+    """
+    Abstract base class for tools.
+    
+    Subclasses must implement:
+    - schema: ToolSchema describing the tool
+    - execute(): async method that performs the tool action
+    """
+    
+    @property
+    @abstractmethod
+    def schema(self) -> ToolSchema:
+        """Return the tool's schema."""
+        pass
+    
+    @property
+    def name(self) -> str:
+        """Tool name (from schema)."""
+        return self.schema.name
+    
+    @abstractmethod
+    async def execute(self, **kwargs) -> ToolResult:
+        """
+        Execute the tool with given arguments.
+        
+        Args:
+            **kwargs: Tool-specific arguments
+            
+        Returns:
+            ToolResult with success/failure and output
+        """
+        pass
+    
+    def is_available(self) -> tuple[bool, str | None]:
+        """
+        Return whether this tool should be exposed/executable in the current process.
+
+        Tools that depend on optional binaries/services/env vars can override this
+        to avoid advertising a tool that will fail at runtime.
+        """
+        return True, None
+
+    async def __call__(self, **kwargs) -> ToolResult:
+        """Allow calling tool instance directly."""
+        return await self.execute(**kwargs)
+
+# Note: This is only wrapping declarations for the external ToolServer (for execution on external process tools), and tools preinstalled in envs
+class ToolRegistry:
+    """Registry of available tools."""
+    
+    def __init__(self):
+        self._tools: Dict[str, Tool] = {}
+    
+    def register(self, tool: Tool) -> None:
+        """Register a tool."""
+        self._tools[tool.name] = tool
+    
+    def get(self, name: str) -> Optional[Tool]:
+        """Get a tool by name."""
+        return self._tools.get(name)
+    
+    def list_tools(self) -> List[Tool]:
+        """List all registered tools."""
+        return list(self._tools.values())
+    
+    def get_schemas(self) -> List[ToolSchema]:
+        """Get schemas for all registered tools."""
+        return [tool.schema for tool in self._tools.values()]
+    
+    def get_prompt_description(self) -> str:
+        """Generate tool descriptions for system prompt."""
+        descriptions = [tool.schema.to_prompt_description() for tool in self._tools.values()]
+        return "\n\n".join(descriptions)
+
+    def get_prompt_tool_definitions_json(self) -> str:
+        """
+        Return a Hermes-style JSON list of tool definitions for use inside a `<tools>...</tools>` block.
+
+        Hermes trajectories historically use a simplified schema list:
+          [{"name": ..., "description": ..., "parameters": {...}, "required": null}, ...]
+        """
+        formatted: List[Dict[str, Any]] = []
+        for tool in self._tools.values():
+            fn = tool.schema.to_dict().get("function", {})
+            formatted.append(
+                {
+                    "name": fn.get("name", tool.name),
+                    "description": fn.get("description", ""),
+                    "parameters": fn.get("parameters", {}),
+                    # Keep parity with Hermes saved trajectories (required is typically null there).
+                    "required": None,
+                }
+            )
+        return json.dumps(formatted, ensure_ascii=False)
+    
+    async def execute(self, call: ToolCall) -> ToolResult:
+        """Execute a tool call."""
+        tool = self.get(call.name)
+        if tool is None:
+            return ToolResult(
+                success=False,
+                error=f"Unknown tool: {call.name}",
+                uniq_id=call.uniq_id,
+            )
+        
+        try:
+            result = await tool.execute(**call.arguments)
+            if result.uniq_id is None:
+                result.uniq_id = call.uniq_id
+            return result
+        except Exception as e:
+            return ToolResult(
+                success=False,
+                error=f"Tool execution error: {str(e)}",
+                uniq_id=call.uniq_id,
+            )
+
+
+# =============================================================================
+# FastAPI / transport models
+# =============================================================================
+
+
+class ToolCallPayload(BaseModel):
+    name: str
+    arguments: Dict[str, Any] = Field(default_factory=dict)
+    uniq_id: str
+
+    @classmethod
+    def from_tool_call(cls, call: ToolCall) -> "ToolCallPayload":
+        return cls(name=call.name, arguments=call.arguments, uniq_id=call.uniq_id)
+
+    def to_tool_call(self) -> ToolCall:
+        return ToolCall(name=self.name, arguments=self.arguments, uniq_id=self.uniq_id)
+
+
+class ToolResultPayload(BaseModel):
+    success: bool
+    output: str = ""
+    error: str = ""
+    metadata: Dict[str, Any] = Field(default_factory=dict)
+    uniq_id: Optional[str] = None
+
+    @classmethod
+    def from_tool_result(cls, result: ToolResult) -> "ToolResultPayload":
+        return cls(
+            success=result.success,
+            output=result.output,
+            error=result.error,
+            metadata=result.metadata,
+            uniq_id=result.uniq_id,
+        )
+
+    def to_tool_result(self) -> ToolResult:
+        return ToolResult(
+            success=self.success,
+            output=self.output,
+            error=self.error,
+            metadata=self.metadata,
+            uniq_id=self.uniq_id,
+        )
+
+
+class ToolExecutorExecuteRequest(BaseModel):
+    trajectory_id: str
+    tool: ToolCallPayload
+    timeout_s: Optional[float] = None
+
+
+class ToolExecutorReleaseRequest(BaseModel):
+    trajectory_id: str
+    reset_workspace: bool = False
+
+
+class ToolServerExecuteRequest(BaseModel):
+    trajectory_id: Optional[str] = None
+    tool: ToolCallPayload
+    timeout_s: Optional[float] = None
+    # Optional sandbox context for tools that need workspace artifacts.
+    # This is set by ToolExecutor and is NOT model-controlled.
+    slot_id: Optional[str] = None
+    container_addr: Optional[str] = None
+
+
+# =============================================================================
+# Artifact transport models
+# =============================================================================
+
+
+class ArtifactReadRequestPayload(BaseModel):
+    trajectory_id: str
+    path: str
+    encoding: Literal["text", "base64"] = "text"
+    max_bytes: Optional[int] = None
+    include_sha256: bool = False
+
+
+class ArtifactReadResponsePayload(BaseModel):
+    success: bool
+    content: str = ""
+    error: str = ""
+    encoding: str = "text"
+    truncated: bool = False
+    bytes: int = 0
+    file_size: Optional[int] = None
+    path: str = ""
+    mime: Optional[str] = None
+    sha256: Optional[str] = None
+
+
+class ArtifactListRequestPayload(BaseModel):
+    trajectory_id: str
+    path: str = "."
+    recursive: bool = False
+    max_entries: Optional[int] = None
+
+
+class ArtifactListEntryPayload(BaseModel):
+    path: str
+    is_dir: bool
+    size: int
+    mtime: float
+
+
+class ArtifactListResponsePayload(BaseModel):
+    success: bool
+    entries: List[ArtifactListEntryPayload] = Field(default_factory=list)
+    truncated: bool = False
+    error: str = ""
+
+
+class ArtifactArchiveRequestPayload(BaseModel):
+    trajectory_id: str
+    path: str = "."
+    format: Literal["tar.gz", "tgz"] = "tar.gz"
+    max_bytes: Optional[int] = None
+    max_entries: Optional[int] = None
+
+
+class ArtifactArchiveResponsePayload(BaseModel):
+    success: bool
+    content: str = ""
+    error: str = ""
+    encoding: str = "base64"
+    format: str = "tar.gz"
+    bytes: int = 0
+    entry_count: int = 0

atropos/tools/build_registry.py

@@ -0,0 +1,64 @@
+"""
+Unified tool registry builder for Hermes-Agent Atropos integration.
+
+This composes:
+- sandbox tool stubs (terminal/bash/read_file/write_file + stateful terminal/tmux)
+- Hermes external tools (web/vision/image/moa/skills/browser), executed via ToolServer
+
+ToolExecutor only needs the schema + `external` routing bit; ToolServer executes
+the external tools via Hermes' existing implementations.
+"""
+
+from __future__ import annotations
+
+from typing import List, Optional
+
+from .base import ToolRegistry
+from .hermes_external_tools import build_external_tools
+from .sandbox_stubs import BashTool, ReadFileTool, TerminalTool, WriteFileTool
+from .terminal_stateful_tool import TerminalStatefulTool
+from .tmux_tool import TmuxTool
+from .toolset_resolver import resolve_multiple_toolsets
+
+
+def build_tool_registry(
+    *,
+    enabled_toolsets: Optional[List[str]] = None,
+    disabled_toolsets: Optional[List[str]] = None,
+    tool_server_url: Optional[str] = None,
+) -> ToolRegistry:
+    """
+    Build a ToolRegistry for AgentEnv / ToolExecutor / ToolServer.
+
+    If `tool_server_url` is not provided, external tools will be omitted so we do
+    not advertise tools that cannot execute.
+    """
+    enabled_toolsets = enabled_toolsets or ["default"]
+
+    # Resolve tool names using Hermes toolsets plus Atropos additions.
+    selected = set(resolve_multiple_toolsets(enabled_toolsets))
+    if disabled_toolsets:
+        selected -= set(resolve_multiple_toolsets(disabled_toolsets))
+
+    reg = ToolRegistry()
+
+    # Always register sandbox tools if selected.
+    sandbox_by_name = {
+        "terminal": TerminalTool(),
+        "bash": BashTool(),
+        "read_file": ReadFileTool(),
+        "write_file": WriteFileTool(),
+        "terminal_stateful": TerminalStatefulTool(),
+        "tmux": TmuxTool(),
+    }
+    for name, tool in sandbox_by_name.items():
+        if name in selected:
+            reg.register(tool)
+
+    # External tools: only include when ToolServer is configured.
+    if tool_server_url:
+        for tool in build_external_tools(selected_tool_names=selected):
+            if tool.name in selected:
+                reg.register(tool)
+
+    return reg

atropos/tools/hermes_external_tools.py

@@ -0,0 +1,90 @@
+"""
+Hermes external tool adapter for Atropos ToolServer.
+
+These tools reuse Hermes-Agent's existing tool runner (`model_tools.handle_function_call`)
+so we don't duplicate external tool implementations.
+
+Important:
+- These are marked `external=True` and should be executed ONLY by ToolServer.
+- We run `handle_function_call` in a worker thread because the Hermes implementation
+  uses `asyncio.run()` internally for some async tools (web_extract, vision, MoA, etc).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from typing import Any, Dict, List, Optional
+
+import model_tools
+
+from .base import Tool, ToolResult, ToolSchema
+
+
+def _schema_from_openai_tool_dict(tool: Dict[str, Any], *, external: bool) -> ToolSchema:
+    fn = tool.get("function") or {}
+    name = str(fn.get("name") or "")
+    description = str(fn.get("description") or "")
+    params = fn.get("parameters") or {}
+    properties = params.get("properties") or {}
+    required = params.get("required") or []
+    if not isinstance(required, list):
+        required = []
+    return ToolSchema(
+        name=name,
+        description=description,
+        parameters=dict(properties),
+        required=[str(x) for x in required if isinstance(x, (str, int))],
+        external=external,
+    )
+
+
+class HermesExternalTool(Tool):
+    def __init__(self, schema: ToolSchema):
+        self._schema = schema
+
+    @property
+    def schema(self) -> ToolSchema:
+        return self._schema
+
+    async def execute(self, task_id: Optional[str] = None, **kwargs: Any) -> ToolResult:
+        # `model_tools.handle_function_call` returns a JSON string (success or error).
+        # Run in a thread because some Hermes tool handlers call `asyncio.run()`.
+        raw = await asyncio.to_thread(model_tools.handle_function_call, self.name, kwargs, task_id)
+
+        try:
+            parsed = json.loads(raw)
+        except Exception:
+            # Keep as plain string.
+            return ToolResult(success=True, output=str(raw))
+
+        if isinstance(parsed, dict) and parsed.get("error"):
+            return ToolResult(success=False, error=str(parsed.get("error")), output="")
+
+        return ToolResult(success=True, output=json.dumps(parsed, ensure_ascii=False))
+
+
+def build_external_tools(
+    *,
+    selected_tool_names: Optional[set[str]] = None,
+) -> List[HermesExternalTool]:
+    """
+    Build external tool wrappers from Hermes tool declarations.
+
+    Filters out sandbox-oriented tools (e.g. `terminal`) since those should run
+    inside the sandbox via ToolExecutor.
+    """
+    # IMPORTANT: Hermes' `model_tools.get_tool_definitions()` only understands Hermes toolsets.
+    # Atropos envs add extra toolsets (filesystem/sandbox/stateful). To avoid noisy "Unknown toolset"
+    # prints and accidental filtering, we fetch ALL Hermes tool definitions here and filter by name.
+    tools = model_tools.get_tool_definitions(enabled_toolsets=None, disabled_toolsets=None, quiet_mode=True)
+
+    wrappers: List[HermesExternalTool] = []
+    for t in tools:
+        schema = _schema_from_openai_tool_dict(t, external=True)
+        if schema.name in {"terminal"}:
+            continue
+        if selected_tool_names is not None and schema.name not in selected_tool_names:
+            continue
+        wrappers.append(HermesExternalTool(schema))
+    return wrappers

atropos/tools/sandbox_stubs.py

@@ -0,0 +1,99 @@
+"""
+Sandbox tool stubs for Atropos ToolExecutor.
+
+These tools are executed inside the sandbox containers via:
+ToolExecutor -> SlotPool -> sandbox_server.py
+
+They intentionally do NOT execute anything on the host process. If they are
+called directly (outside ToolExecutor), they return a clear error.
+"""
+
+from __future__ import annotations
+
+from typing import Optional
+
+from .base import Tool, ToolResult, ToolSchema
+
+
+class TerminalTool(Tool):
+    @property
+    def schema(self) -> ToolSchema:
+        return ToolSchema(
+            name="terminal",
+            description=(
+                "Execute a command inside the sandbox slot workspace and return stdout/stderr. "
+                "Filesystem persists within a trajectory slot. Background processes are not supported "
+                "in stateless mode. Commands run under POSIX /bin/sh and each tool call runs in a fresh "
+                "shell (no persisted env vars). Avoid bash-only syntax like `source`; prefer `. .venv/bin/activate` "
+                "or invoke `.venv/bin/python ...` directly."
+            ),
+            parameters={
+                "command": {"type": "string", "description": "The command to execute"},
+                "timeout": {
+                    "type": "integer",
+                    "description": "Command timeout in seconds (optional).",
+                    "minimum": 1,
+                },
+                "background": {
+                    "type": "boolean",
+                    "description": "Not supported in sandbox terminal (always false).",
+                    "default": False,
+                },
+            },
+            required=["command"],
+            external=False,
+        )
+
+    async def execute(self, **_kwargs) -> ToolResult:
+        return ToolResult(
+            success=False,
+            error="terminal must be executed via ToolExecutor inside the sandbox",
+        )
+
+
+class BashTool(Tool):
+    @property
+    def schema(self) -> ToolSchema:
+        return ToolSchema(
+            name="bash",
+            description="Execute a bash command inside the sandbox slot workspace.",
+            parameters={"command": {"type": "string", "description": "The bash command to execute"}},
+            required=["command"],
+            external=False,
+        )
+
+    async def execute(self, **_kwargs) -> ToolResult:
+        return ToolResult(success=False, error="bash must be executed via ToolExecutor inside the sandbox")
+
+
+class ReadFileTool(Tool):
+    @property
+    def schema(self) -> ToolSchema:
+        return ToolSchema(
+            name="read_file",
+            description="Read a file from the sandbox slot workspace.",
+            parameters={"path": {"type": "string", "description": "Path to the file"}},
+            required=["path"],
+            external=False,
+        )
+
+    async def execute(self, **_kwargs) -> ToolResult:
+        return ToolResult(success=False, error="read_file must be executed via ToolExecutor inside the sandbox")
+
+
+class WriteFileTool(Tool):
+    @property
+    def schema(self) -> ToolSchema:
+        return ToolSchema(
+            name="write_file",
+            description="Write a file into the sandbox slot workspace.",
+            parameters={
+                "path": {"type": "string", "description": "Path to the file"},
+                "content": {"type": "string", "description": "File content"},
+            },
+            required=["path", "content"],
+            external=False,
+        )
+
+    async def execute(self, **_kwargs) -> ToolResult:
+        return ToolResult(success=False, error="write_file must be executed via ToolExecutor inside the sandbox")

atropos/tools/terminal_stateful_tool.py

@@ -0,0 +1,45 @@
+"""
+Stateful terminal tool schema.
+
+This is a sandbox tool that routes to the sandbox server as `bash_stateful`
+via ToolExecutor mapping. It exists to expose an explicit, opt-in terminal
+primitive suitable for stateful workflows (e.g. tmux sessions / TUIs).
+"""
+
+from __future__ import annotations
+
+from typing import Optional
+
+from .base import Tool, ToolResult, ToolSchema
+
+
+class TerminalStatefulTool(Tool):
+    @property
+    def schema(self) -> ToolSchema:
+        return ToolSchema(
+            name="terminal_stateful",
+            description=(
+                "Execute a command in the sandbox, allowing stateful/background processes to persist "
+                "across tool calls within the same trajectory slot (e.g. tmux sessions). "
+                "Use sparingly; output is still non-interactive."
+            ),
+            parameters={
+                "command": {"type": "string", "description": "The command to execute"},
+                "timeout": {
+                    "type": "integer",
+                    "description": "Command timeout in seconds (optional).",
+                    "minimum": 1,
+                },
+            },
+            required=["command"],
+        )
+
+    def is_available(self) -> tuple[bool, str | None]:
+        return True, None
+
+    async def execute(self, command: str, timeout: Optional[int] = None) -> ToolResult:
+        _ = (command, timeout)
+        return ToolResult(
+            success=False,
+            error="terminal_stateful must be executed via ToolExecutor inside the sandbox",
+        )

atropos/tools/tmux_tool.py

@@ -0,0 +1,89 @@
+"""
+tmux tool schema (sandbox).
+
+This is a sandbox tool that provides basic tmux session control suitable for
+TUI-style terminal interactions:
+- send keys (arrow keys, enter, etc.)
+- capture the current screen buffer
+
+Execution is routed by ToolExecutor to the sandbox server's `tmux` backend.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, Optional
+
+from .base import Tool, ToolResult, ToolSchema
+
+
+class TmuxTool(Tool):
+    @property
+    def schema(self) -> ToolSchema:
+        return ToolSchema(
+            name="tmux",
+            description=(
+                "Control a per-trajectory tmux session inside the sandbox (stateful terminal). "
+                "Use this for TUI-style interactions: send keys and capture the current screen."
+            ),
+            parameters={
+                "action": {
+                    "type": "string",
+                    "description": "Action to perform: start | send_keys | stream | stop.",
+                    "enum": ["start", "send_keys", "stream", "stop", "capture"],
+                },
+                "keys": {
+                    "description": "Keys to send (string or list of strings) when action=send_keys.",
+                },
+                "block": {
+                    "type": "boolean",
+                    "description": "If true, wait for shell command completion (only valid at a shell prompt).",
+                    "default": False,
+                },
+                "min_wait_s": {
+                    "type": "number",
+                    "description": "For non-blocking send_keys, sleep this long after sending keys (seconds).",
+                    "default": 0.0,
+                },
+                "max_wait_s": {
+                    "type": "number",
+                    "description": "For blocking send_keys, max time to wait for completion (seconds).",
+                },
+                "capture_entire": {
+                    "type": "boolean",
+                    "description": "Deprecated. Streaming is preferred.",
+                    "default": False,
+                },
+                "max_bytes": {
+                    "type": "integer",
+                    "description": "Max bytes to return per stream call.",
+                },
+                "reset": {
+                    "type": "boolean",
+                    "description": "If true, reset stream offset to the beginning of the asciinema recording.",
+                    "default": False,
+                },
+                "pane_width": {
+                    "type": "integer",
+                    "description": "Pane width for action=start (columns).",
+                    "minimum": 20,
+                },
+                "pane_height": {
+                    "type": "integer",
+                    "description": "Pane height for action=start (rows).",
+                    "minimum": 10,
+                },
+            },
+            required=["action"],
+        )
+
+    def is_available(self) -> tuple[bool, str | None]:
+        return True, None
+
+    async def execute(self, **kwargs: Dict[str, Any]) -> ToolResult:
+        # This tool is intended to be executed via ToolExecutor -> sandbox server.
+        # We keep a safe fallback for non-sandbox contexts.
+        action = str(kwargs.get("action") or "").strip()
+        return ToolResult(
+            success=False,
+            error=f"tmux tool must be executed in the sandbox (got action={action!r})",
+        )

atropos/tools/tool_executor.py

@@ -0,0 +1,500 @@
+"""
+ToolExecutor - queued, batched tool dispatch for multiplexed agent trajectories.
+
+This component is responsible for:
+- Maintaining trajectory -> Slot affinity (workspace continuity)
+- Batching sandbox tool calls across trajectories to maximize container utilization
+- Routing external tools (ToolSchema.external=True) to a ToolServer (Phase 4.5)
+
+For now, only sandbox tools are executed:
+- bash
+- read_file
+- write_file
+"""
+
+from __future__ import annotations
+
+import asyncio
+import time
+from dataclasses import dataclass
+from typing import Any, Dict, List, Optional
+
+import httpx
+
+from .base import (
+    ArtifactArchiveRequestPayload,
+    ArtifactArchiveResponsePayload,
+    ArtifactListRequestPayload,
+    ArtifactListResponsePayload,
+    ArtifactReadRequestPayload,
+    ArtifactReadResponsePayload,
+    ToolCall,
+    ToolCallPayload,
+    ToolRegistry,
+    ToolResult,
+    ToolResultPayload,
+    ToolServerExecuteRequest,
+)
+from ..backends.base import ToolBackend
+from ..slots import Slot
+
+
+@dataclass
+class ToolExecutorConfig:
+    batch_window_ms: int = 20
+    max_batch_size: int = 200
+    allow_network: bool = True
+    require_sandbox: bool = False
+    require_stateful_sandbox: bool = False
+    tool_server_url: Optional[str] = None
+    tool_server_token: Optional[str] = None
+
+
+@dataclass
+class _QueuedToolRequest:
+    trajectory_id: str
+    call: ToolCall
+    timeout_s: Optional[float]
+    future: asyncio.Future
+
+
+class ToolExecutor:
+    def __init__(
+        self,
+        backend: ToolBackend,
+        tools: ToolRegistry,
+        config: Optional[ToolExecutorConfig] = None,
+    ) -> None:
+        self.backend = backend
+        self.tools = tools
+        self.config = config or ToolExecutorConfig()
+
+        self._queue: asyncio.Queue[Optional[_QueuedToolRequest]] = asyncio.Queue()
+        self._task: Optional[asyncio.Task] = None
+        self._stopping = asyncio.Event()
+
+        self._slots_lock = asyncio.Lock()
+        self._slot_by_trajectory: Dict[str, Slot] = {}
+
+        self._tool_server_client: Optional[httpx.AsyncClient] = None
+        self._tool_server_lock = asyncio.Lock()
+
+        # lightweight stats for status endpoints
+        self.total_requests: int = 0
+        self.total_errors: int = 0
+        self.latencies_s: List[float] = []
+
+    async def start(self) -> None:
+        if self._task is None:
+            self._task = asyncio.create_task(self._run_loop())
+
+    def queue_size(self) -> int:
+        return self._queue.qsize()
+
+    async def close(self) -> None:
+        self._stopping.set()
+        await self._queue.put(None)
+        if self._task:
+            await self._task
+            self._task = None
+
+        client = self._tool_server_client
+        self._tool_server_client = None
+        if client is not None:
+            await client.aclose()
+
+        # Best-effort release any remaining slots.
+        async with self._slots_lock:
+            slots = list(self._slot_by_trajectory.items())
+            self._slot_by_trajectory.clear()
+
+        for _, slot in slots:
+            try:
+                await self.backend.release(slot, reset_workspace=False)
+            except Exception:
+                pass
+
+    async def execute(
+        self,
+        trajectory_id: str,
+        call: ToolCall,
+        timeout_s: Optional[float] = None,
+    ) -> ToolResult:
+        if self._task is None:
+            raise RuntimeError("ToolExecutor not started (call start() first)")
+
+        # Allow tool args to suggest a timeout (Hermes-compatible terminal tool),
+        # but never let the model choose "infinite" timeouts.
+        if timeout_s is None:
+            raw_timeout = call.arguments.get("timeout")
+            if isinstance(raw_timeout, (int, float)):
+                timeout_s = float(raw_timeout)
+        if timeout_s is not None:
+            timeout_s = max(1.0, min(float(timeout_s), 600.0))
+
+        loop = asyncio.get_running_loop()
+        fut: asyncio.Future = loop.create_future()
+        started = time.perf_counter()
+        await self._queue.put(_QueuedToolRequest(trajectory_id=trajectory_id, call=call, timeout_s=timeout_s, future=fut))
+        try:
+            result: ToolResult = await fut
+            return result
+        finally:
+            self.latencies_s.append(time.perf_counter() - started)
+
+    async def release_trajectory(self, trajectory_id: str, reset_workspace: bool = False) -> None:
+        async with self._slots_lock:
+            slot = self._slot_by_trajectory.pop(trajectory_id, None)
+
+        if slot is not None:
+            await self.backend.release(slot, reset_workspace=reset_workspace)
+
+    async def _get_slot_if_present(self, trajectory_id: str) -> Optional[Slot]:
+        async with self._slots_lock:
+            return self._slot_by_trajectory.get(trajectory_id)
+
+    # ---------------------------------------------------------------------
+    # Artifact helpers (optional)
+    # ---------------------------------------------------------------------
+
+    async def read_artifact(self, req: ArtifactReadRequestPayload) -> ArtifactReadResponsePayload:
+        slot = await self._get_slot_if_present(req.trajectory_id)
+        if slot is None:
+            return ArtifactReadResponsePayload(success=False, error="No active slot for trajectory (run a sandbox tool first)")
+        data = await self.backend.read_artifact(
+            slot,
+            req.path,
+            encoding=req.encoding,
+            max_bytes=req.max_bytes,
+            include_sha256=req.include_sha256,
+        )
+        if isinstance(data, dict):
+            data = dict(data)
+            data.pop("http_status", None)
+        try:
+            return ArtifactReadResponsePayload(**(data or {}))
+        except Exception as e:
+            return ArtifactReadResponsePayload(success=False, error=f"Invalid artifact read response: {e}")
+
+    async def list_artifacts(self, req: ArtifactListRequestPayload) -> ArtifactListResponsePayload:
+        slot = await self._get_slot_if_present(req.trajectory_id)
+        if slot is None:
+            return ArtifactListResponsePayload(success=False, error="No active slot for trajectory (run a sandbox tool first)")
+        data = await self.backend.list_artifacts(
+            slot,
+            req.path,
+            recursive=req.recursive,
+            max_entries=req.max_entries,
+        )
+        if isinstance(data, dict):
+            data = dict(data)
+            data.pop("http_status", None)
+        try:
+            return ArtifactListResponsePayload(**(data or {}))
+        except Exception as e:
+            return ArtifactListResponsePayload(success=False, error=f"Invalid artifact list response: {e}")
+
+    async def archive_artifacts(self, req: ArtifactArchiveRequestPayload) -> ArtifactArchiveResponsePayload:
+        slot = await self._get_slot_if_present(req.trajectory_id)
+        if slot is None:
+            return ArtifactArchiveResponsePayload(success=False, error="No active slot for trajectory (run a sandbox tool first)")
+        data = await self.backend.archive_artifacts(
+            slot,
+            req.path,
+            archive_format=req.format,
+            max_bytes=req.max_bytes,
+            max_entries=req.max_entries,
+        )
+        if isinstance(data, dict):
+            data = dict(data)
+            data.pop("http_status", None)
+        try:
+            return ArtifactArchiveResponsePayload(**(data or {}))
+        except Exception as e:
+            return ArtifactArchiveResponsePayload(success=False, error=f"Invalid artifact archive response: {e}")
+
+    async def _get_or_acquire_slot(self, trajectory_id: str) -> Slot:
+        async with self._slots_lock:
+            existing = self._slot_by_trajectory.get(trajectory_id)
+            if existing is not None:
+                return existing
+
+        slot = await self.backend.acquire(trajectory_id)
+
+        async with self._slots_lock:
+            existing = self._slot_by_trajectory.get(trajectory_id)
+            if existing is not None:
+                # Another coroutine won the race; return its slot.
+                await self.backend.release(slot, reset_workspace=False)
+                return existing
+            self._slot_by_trajectory[trajectory_id] = slot
+            return slot
+
+    async def _run_loop(self) -> None:
+        pending: List[_QueuedToolRequest] = []
+        deadline: Optional[float] = None
+
+        batch_window_s = max(0.0, self.config.batch_window_ms / 1000.0)
+        max_batch = max(1, self.config.max_batch_size)
+
+        while True:
+            if self._stopping.is_set() and self._queue.empty() and not pending:
+                break
+
+            timeout = None
+            if pending and deadline is not None:
+                timeout = max(0.0, deadline - time.perf_counter())
+
+            try:
+                item = await asyncio.wait_for(self._queue.get(), timeout=timeout)
+                if item is None:
+                    continue
+                pending.append(item)
+                if len(pending) == 1:
+                    deadline = time.perf_counter() + batch_window_s
+                if len(pending) < max_batch:
+                    continue
+            except asyncio.TimeoutError:
+                # batch window elapsed
+                pass
+
+            if not pending:
+                deadline = None
+                continue
+
+            batch = pending
+            pending = []
+            deadline = None
+
+            await self._execute_batch(batch)
+
+    async def _get_tool_server_client(self) -> httpx.AsyncClient:
+        url = self.config.tool_server_url
+        if not url:
+            raise RuntimeError("ToolServer not configured")
+
+        if self._tool_server_client is not None:
+            return self._tool_server_client
+
+        async with self._tool_server_lock:
+            if self._tool_server_client is None:
+                self._tool_server_client = httpx.AsyncClient(base_url=url.rstrip("/"))
+            return self._tool_server_client
+
+    def _tool_server_headers(self) -> Dict[str, str]:
+        token = self.config.tool_server_token
+        if not token:
+            return {}
+        return {"Authorization": f"Bearer {token}"}
+
+    async def _execute_external(self, req: _QueuedToolRequest) -> ToolResult:
+        client = await self._get_tool_server_client()
+        slot_id: Optional[str] = None
+        container_addr: Optional[str] = None
+        slot = await self._get_slot_if_present(req.trajectory_id)
+        if slot is not None:
+            slot_id = slot.slot_id
+            container_addr = slot.container_addr
+
+        payload = ToolServerExecuteRequest(
+            trajectory_id=req.trajectory_id,
+            tool=ToolCallPayload.from_tool_call(req.call),
+            timeout_s=req.timeout_s,
+            slot_id=slot_id,
+            container_addr=container_addr,
+        )
+
+        try:
+            resp = await client.post(
+                "/execute",
+                json=payload.model_dump(),
+                headers=self._tool_server_headers(),
+                timeout=req.timeout_s,
+            )
+            resp.raise_for_status()
+            data = resp.json()
+            parsed = ToolResultPayload(**data)
+            result = parsed.to_tool_result()
+            if result.uniq_id is None:
+                result.uniq_id = req.call.uniq_id
+            return result
+        except Exception as e:
+            return ToolResult(
+                success=False,
+                error=f"External tool failed: {e}",
+                uniq_id=req.call.uniq_id,
+            )
+
+    async def _execute_batch(self, batch: List[_QueuedToolRequest]) -> None:
+        # Resolve tool schemas once per request and separate sandbox/external/unknown.
+        sandbox_items: List[_QueuedToolRequest] = []
+        external_items: List[_QueuedToolRequest] = []
+        unknown_items: List[_QueuedToolRequest] = []
+
+        for it in batch:
+            tool = self.tools.get(it.call.name)
+            if tool is None:
+                unknown_items.append(it)
+                continue
+
+            schema = tool.schema
+            if not schema.external:
+                sandbox_items.append(it)
+            else:
+                external_items.append(it)
+
+        for it in unknown_items:
+            self.total_requests += 1
+            self.total_errors += 1
+            if not it.future.done():
+                it.future.set_result(
+                    ToolResult(
+                        success=False,
+                        error=f"Unknown tool: {it.call.name}",
+                        uniq_id=it.call.uniq_id,
+                    )
+                )
+
+        if external_items:
+            if not self.config.tool_server_url:
+                for it in external_items:
+                    self.total_requests += 1
+                    self.total_errors += 1
+                    if not it.future.done():
+                        it.future.set_result(
+                            ToolResult(
+                                success=False,
+                                error=f"External tool not available (ToolServer not configured): {it.call.name}",
+                                uniq_id=it.call.uniq_id,
+                            )
+                        )
+            else:
+                results = await asyncio.gather(*[self._execute_external(it) for it in external_items])
+                for it, res in zip(external_items, results):
+                    self.total_requests += 1
+                    if not getattr(res, "success", False):
+                        self.total_errors += 1
+                    if not it.future.done():
+                        it.future.set_result(res)
+
+        if not sandbox_items:
+            return
+
+        # Acquire slots for the distinct trajectories in this batch.
+        try:
+            traj_ids = list({it.trajectory_id for it in sandbox_items})
+            slots = await asyncio.gather(*[self._get_or_acquire_slot(tid) for tid in traj_ids])
+            slot_by_traj = dict(zip(traj_ids, slots))
+        except Exception as e:
+            for it in sandbox_items:
+                self.total_requests += 1
+                self.total_errors += 1
+                if not it.future.done():
+                    it.future.set_result(
+                        ToolResult(
+                            success=False,
+                            error=f"Failed to acquire slot: {e}",
+                            uniq_id=it.call.uniq_id,
+                        )
+                    )
+            return
+
+        # Group by timeout so we don't accidentally make short timeouts wait on long ones.
+        by_timeout: Dict[float, List[_QueuedToolRequest]] = {}
+        default_timeout = self.backend.default_timeout_s
+
+        for it in sandbox_items:
+            t = it.timeout_s
+            if t is None:
+                t = default_timeout
+            if t is None:
+                t = 30.0
+            by_timeout.setdefault(float(t), []).append(it)
+
+        for timeout_s, items in by_timeout.items():
+            requests = []
+            dispatched: List[_QueuedToolRequest] = []
+            for it in items:
+                slot = slot_by_traj[it.trajectory_id]
+                tool_name = it.call.name
+                args = dict(it.call.arguments)
+
+                # Hermes compatibility: treat `terminal` as an alias of sandbox `bash`.
+                if tool_name == "terminal":
+                    if args.get("background"):
+                        self.total_requests += 1
+                        self.total_errors += 1
+                        if not it.future.done():
+                            it.future.set_result(
+                                ToolResult(
+                                    success=False,
+                                    error="terminal background execution is not supported in sandbox",
+                                    uniq_id=it.call.uniq_id,
+                                )
+                            )
+                        continue
+                    tool_name = "bash"
+                    # `timeout` is handled at the ToolExecutor level, not passed to the sandbox tool args.
+                    args.pop("timeout", None)
+                elif tool_name == "terminal_stateful":
+                    tool_name = "bash_stateful"
+                    args.pop("timeout", None)
+                elif tool_name == "tmux":
+                    # `tmux` is a sandbox tool backed by the stateful session manager.
+                    # Network policy is env-controlled.
+                    args.pop("allow_network", None)
+
+                if tool_name == "bash":
+                    # Network policy is set by the environment/executor, not by the model.
+                    args.pop("allow_network", None)
+                    args.pop("require_sandbox", None)
+                    args["allow_network"] = bool(self.config.allow_network)
+                    args["require_sandbox"] = bool(self.config.require_sandbox)
+                    # `timeout` is handled at the ToolExecutor level, not passed to the sandbox tool args.
+                    args.pop("timeout", None)
+                elif tool_name == "bash_stateful":
+                    # Network policy is set by the environment/executor, not by the model.
+                    args.pop("allow_network", None)
+                    args.pop("require_sandbox", None)
+                    args.pop("require_stateful_sandbox", None)
+                    args["allow_network"] = bool(self.config.allow_network)
+                    args["require_stateful_sandbox"] = bool(self.config.require_stateful_sandbox)
+                    args.pop("timeout", None)
+                elif tool_name == "tmux":
+                    # Network policy applies to the underlying stateful session.
+                    args.pop("allow_network", None)
+                    args.pop("require_sandbox", None)
+                    args.pop("require_stateful_sandbox", None)
+                    args["allow_network"] = bool(self.config.allow_network)
+                    args["require_stateful_sandbox"] = bool(self.config.require_stateful_sandbox)
+
+                requests.append((slot, tool_name, args))
+                dispatched.append(it)
+
+            results = None
+            try:
+                if not dispatched:
+                    continue
+                results = await self.backend.execute_batch(requests, timeout_s=timeout_s)
+            except Exception as e:
+                for it in items:
+                    self.total_requests += 1
+                    self.total_errors += 1
+                    if not it.future.done():
+                        it.future.set_result(
+                            ToolResult(
+                                success=False,
+                                error=f"Batch execution failed: {e}",
+                                uniq_id=it.call.uniq_id,
+                            )
+                        )
+                continue
+
+            for it, res in zip(dispatched, results):
+                self.total_requests += 1
+                if not getattr(res, "success", False):
+                    self.total_errors += 1
+                tool_result = res.to_tool_result()
+                tool_result.uniq_id = it.call.uniq_id
+                if not it.future.done():
+                    it.future.set_result(tool_result)

atropos/tools/toolset_resolver.py

@@ -0,0 +1,88 @@
+"""
+Toolset resolution for Hermes-Agent Atropos integration.
+
+We primarily reuse Hermes-Agent toolsets (`toolsets.py`), but Atropos training/envs
+need a few extra sandbox-oriented toolsets that Hermes doesn't expose by default
+(e.g. filesystem + stateful terminal).
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional, Set
+
+import toolsets as hermes_toolsets
+
+
+ATROPOS_TOOLSETS: Dict[str, Dict[str, Any]] = {
+    "filesystem": {
+        "description": "Read/write files in the sandbox workspace.",
+        "tools": ["read_file", "write_file"],
+        "includes": [],
+    },
+    "terminal_stateful": {
+        "description": "Stateful terminal execution (tmux/TUI support) inside the sandbox.",
+        "tools": ["terminal_stateful", "tmux"],
+        "includes": [],
+    },
+    "sandbox": {
+        "description": "Sandbox tools (terminal + filesystem).",
+        "tools": [],
+        "includes": ["terminal", "filesystem"],
+    },
+    "default": {
+        "description": "Default toolset for Atropos AgentEnv tasks.",
+        "tools": [],
+        "includes": ["sandbox"],
+    },
+    "full": {
+        "description": "All Hermes tools plus Atropos sandbox additions.",
+        "tools": [],
+        "includes": ["all", "filesystem", "sandbox", "terminal_stateful"],
+    },
+}
+
+
+def validate_toolset(name: str) -> bool:
+    if name in {"all", "*"}:
+        return True
+    return hermes_toolsets.validate_toolset(name) or name in ATROPOS_TOOLSETS
+
+
+def resolve_toolset(name: str, visited: Optional[Set[str]] = None) -> List[str]:
+    if visited is None:
+        visited = set()
+
+    if name in {"all", "*"}:
+        # Union Hermes + Atropos toolsets.
+        all_tools: Set[str] = set()
+        for tname in hermes_toolsets.get_toolset_names():
+            all_tools.update(resolve_toolset(tname, visited=set()))
+        for tname, spec in ATROPOS_TOOLSETS.items():
+            # Avoid recursion: some Atropos toolsets (e.g. "full") include "all".
+            if tname == "full" or "all" in (spec.get("includes") or []):
+                continue
+            all_tools.update(resolve_toolset(tname, visited=set()))
+        return sorted(all_tools)
+
+    if name in ATROPOS_TOOLSETS:
+        if name in visited:
+            return []
+        visited.add(name)
+        spec = ATROPOS_TOOLSETS[name]
+        tools: Set[str] = set(spec.get("tools", []))
+        for inc in spec.get("includes", []):
+            tools.update(resolve_toolset(inc, visited=set(visited)))
+        return sorted(tools)
+
+    # Fall back to Hermes toolsets.
+    # IMPORTANT: do not pre-add `name` to `visited` here; Hermes' resolver uses
+    # `visited` for its own cycle detection and will treat the presence of `name`
+    # as a circular dependency.
+    return sorted(hermes_toolsets.resolve_toolset(name, visited=set(visited)))
+
+
+def resolve_multiple_toolsets(names: List[str]) -> List[str]:
+    tools: Set[str] = set()
+    for name in names:
+        tools.update(resolve_toolset(name, visited=set()))
+    return sorted(tools)

atropos_compatible_agent.py

@@ -0,0 +1,415 @@
+#!/usr/bin/env python3
+"""
+Atropos-compatible Hermes agent runner.
+
+This is a minimal subclass of Hermes-Agent's `AIAgent` that swaps the OpenAI
+function-calling backend for Atroposlib's `ManagedServer`/`ServerManager` backend
+and uses Hermes-style XML tool tags:
+
+- <tool_call>{"name": "...", "arguments": {...}}</tool_call>
+- <tool_response>{...}</tool_response>
+
+Tool observations are appended as `role="user"` messages containing one or more
+`<tool_response>` blocks so they survive common chat templates during tokenization.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import re
+import time
+import warnings
+import os
+from contextlib import asynccontextmanager
+from typing import Any, AsyncGenerator, Dict, List, Optional, Tuple
+
+from model_tools import cleanup_vm, handle_function_call
+from run_agent import AIAgent
+
+_TOOL_CALL_RE = re.compile(r"<tool_call>\\s*(.*?)\\s*</tool_call>", re.DOTALL)
+
+
+ATROPOS_TOOL_SYSTEM_PROMPT = """You are a helpful AI assistant with access to tools.
+
+## Available Tools
+<tools>
+{tool_descriptions}
+</tools>
+
+## How to Use Tools
+To call a tool, output:
+<tool_call>{{"name": "tool_name", "arguments": {{"arg1": "value1"}}}}</tool_call>
+
+You may include optional reasoning in <think>...</think> before tool calls.
+
+After each tool call, you will receive tool results as:
+<tool_response>{{...}}</tool_response>
+
+Continue until finished, then provide a final response with no <tool_call> blocks.
+"""
+
+
+class AtroposAIAgent(AIAgent):
+    """
+    Hermes `AIAgent` variant that uses Atroposlib ServerManager/ManagedServer.
+
+    Notes:
+    - The default Hermes `AIAgent` remains unchanged; this class is opt-in.
+    - The underlying server must expose `managed_server(tokenizer=...)` OR be a single
+      APIServer-compatible object usable by Atroposlib's `ManagedServer`.
+    """
+
+    def __init__(
+        self,
+        *,
+        server: Any,
+        tokenizer: Any = None,
+        model: str = "local",
+        max_iterations: int = 10,
+        tool_delay: float = 0.0,
+        enabled_toolsets: Optional[List[str]] = None,
+        disabled_toolsets: Optional[List[str]] = None,
+        save_trajectories: bool = False,
+        verbose_logging: bool = False,
+        quiet_mode: bool = False,
+        ephemeral_system_prompt: Optional[str] = None,
+        log_prefix_chars: int = 100,
+        log_prefix: str = "",
+        session_id: Optional[str] = None,
+        temperature: Optional[float] = None,
+        max_tokens: Optional[int] = None,
+    ):
+        # Call parent init mainly to reuse tool selection + trajectory saving utilities.
+        super().__init__(
+            base_url="http://unused",
+            api_key="dummy-key",
+            model=model,
+            max_iterations=max_iterations,
+            tool_delay=tool_delay,
+            enabled_toolsets=enabled_toolsets,
+            disabled_toolsets=disabled_toolsets,
+            save_trajectories=save_trajectories,
+            verbose_logging=verbose_logging,
+            quiet_mode=quiet_mode,
+            ephemeral_system_prompt=ephemeral_system_prompt,
+            log_prefix_chars=log_prefix_chars,
+            log_prefix=log_prefix,
+            session_id=session_id,
+        )
+
+        self.server = server
+        self.tokenizer = tokenizer
+        self.temperature = temperature
+        self.max_tokens = max_tokens
+
+    @asynccontextmanager
+    async def _managed(self) -> AsyncGenerator[Any, None]:
+        if hasattr(self.server, "managed_server"):
+            with warnings.catch_warnings():
+                warnings.filterwarnings(
+                    "ignore",
+                    message=r"Using OpenAIServer with managed_server does not allow for state tracking",
+                    category=UserWarning,
+                )
+                async with self.server.managed_server(tokenizer=self.tokenizer) as managed:
+                    yield managed
+            return
+
+        # Fall back to directly wrapping a single server object.
+        from atroposlib.envs.server_handling.managed_server import ManagedServer
+
+        managed = ManagedServer(server=self.server, tokenizer=self.tokenizer)
+        try:
+            yield managed
+        finally:
+            managed.reset()
+
+    def _tool_descriptions_text(self) -> str:
+        if not self.tools:
+            return "(no tools available)"
+
+        parts: List[str] = []
+        for tool in self.tools:
+            fn = (tool or {}).get("function", {})
+            name = fn.get("name", "")
+            desc = (fn.get("description") or "").strip()
+            if not name:
+                continue
+            if desc:
+                parts.append(f"- {name}: {desc}")
+            else:
+                parts.append(f"- {name}")
+        return "\n".join(parts) if parts else "(no tools available)"
+
+    def _build_system_prompt(self, system_message: Optional[str]) -> Optional[str]:
+        tool_prompt = ATROPOS_TOOL_SYSTEM_PROMPT.format(
+            tool_descriptions=self._tool_descriptions_text()
+        )
+
+        parts: List[str] = []
+        if system_message:
+            parts.append(system_message)
+        if self.ephemeral_system_prompt:
+            parts.append(self.ephemeral_system_prompt)
+        parts.append(tool_prompt)
+
+        return "\n\n".join(parts)
+
+    def _parse_tool_calls(self, content: str) -> Tuple[List[Tuple[str, Dict[str, Any]]], List[str]]:
+        """
+        Returns:
+          (calls, errors)
+        """
+        calls: List[Tuple[str, Dict[str, Any]]] = []
+        errors: List[str] = []
+
+        for raw in _TOOL_CALL_RE.findall(content or ""):
+            try:
+                payload = json.loads(raw)
+            except json.JSONDecodeError as exc:
+                errors.append(f"Invalid JSON inside <tool_call>: {exc}")
+                continue
+
+            name = payload.get("name")
+            args = payload.get("arguments", {})
+            if not isinstance(name, str) or not name:
+                errors.append("Tool call missing 'name' string")
+                continue
+            if not isinstance(args, dict):
+                errors.append("Tool call 'arguments' must be an object")
+                continue
+
+            calls.append((name, args))
+
+        return calls, errors
+
+    async def run_conversation_async(
+        self,
+        user_message: str,
+        system_message: Optional[str] = None,
+        conversation_history: Optional[List[Dict[str, Any]]] = None,
+        task_id: Optional[str] = None,
+    ) -> Dict[str, Any]:
+        import uuid
+
+        effective_task_id = task_id or str(uuid.uuid4())
+
+        messages: List[Dict[str, Any]] = conversation_history.copy() if conversation_history else []
+        messages.append({"role": "user", "content": user_message})
+
+        active_system_prompt = self._build_system_prompt(system_message)
+
+        api_call_count = 0
+        final_response: Optional[str] = None
+        managed_state: Optional[Dict[str, Any]] = None
+        completed = False
+
+        try:
+            async with self._managed() as managed:
+                while api_call_count < self.max_iterations:
+                    api_call_count += 1
+
+                    api_messages = messages.copy()
+                    if active_system_prompt:
+                        api_messages = [{"role": "system", "content": active_system_prompt}] + api_messages
+
+                    chat_kwargs: Dict[str, Any] = {"messages": api_messages, "n": 1}
+                    if self.max_tokens is not None:
+                        chat_kwargs["max_tokens"] = self.max_tokens
+                    if self.temperature is not None:
+                        chat_kwargs["temperature"] = self.temperature
+
+                    # Prefer OpenAI tool calling when supported by the backend:
+                    # - Many providers normalize Hermes-style <tool_call> tags into tool_calls when `tools` is provided.
+                    # - ManagedServer (atroposlib) does prompt->completion conversion and does not support `tools`.
+                    #   Only pass `tools` when we're calling an OpenAI-compatible chat endpoint directly.
+                    tool_schemas = self.tools if self.tools else None
+                    managed_cls = type(managed).__name__
+                    if tool_schemas and managed_cls != "ManagedServer":
+                        chat_kwargs["tools"] = tool_schemas
+
+                    if os.getenv("HERMES_DEBUG_ATROPOS_REQUEST") == "1":
+                        meta = {
+                            "managed_type": managed_cls,
+                            "model": getattr(getattr(managed, "config", None), "model_name", self.model),
+                            "base_url": getattr(getattr(managed, "config", None), "base_url", None),
+                            "kwargs": chat_kwargs,
+                        }
+                        # Avoid dumping megabytes of data accidentally.
+                        # (Messages can be large; this is still "full" but bounded.)
+                        print("\n=== HERMES_DEBUG_ATROPOS_REQUEST ===", flush=True)
+                        print(json.dumps(meta, ensure_ascii=False, indent=2)[:200_000], flush=True)
+
+                    response = await managed.chat_completion(**chat_kwargs)
+
+                    if os.getenv("HERMES_DEBUG_ATROPOS_RESPONSE") == "1":
+                        try:
+                            dumped = response.model_dump()  # openai pydantic model
+                        except Exception:
+                            dumped = getattr(response, "__dict__", {"repr": repr(response)})
+                        print("\n=== HERMES_DEBUG_ATROPOS_RESPONSE: ChatCompletion (raw) ===", flush=True)
+                        print(json.dumps(dumped, ensure_ascii=False, indent=2), flush=True)
+
+                    if hasattr(managed, "get_state"):
+                        managed_state = managed.get_state()
+
+                    msg = response.choices[0].message
+                    assistant_content = (msg.content or "")
+                    msg_reasoning = getattr(msg, "reasoning", None)
+
+                    # Use tool_calls if the backend provides them (preferred).
+                    structured_tool_calls = getattr(msg, "tool_calls", None)
+
+                    # If the backend emits content="" but includes useful text in reasoning,
+                    # use it for parsing *only if needed* (e.g. tool tags).
+                    if assistant_content == "" and isinstance(msg_reasoning, str) and msg_reasoning:
+                        if os.getenv("HERMES_DEBUG_ATROPOS_RESPONSE") == "1":
+                            print("\n=== HERMES_DEBUG_ATROPOS_RESPONSE: message.reasoning present (content empty) ===", flush=True)
+                            print(msg_reasoning, flush=True)
+
+                    assistant_msg: Dict[str, Any] = {"role": "assistant", "content": assistant_content}
+                    if structured_tool_calls:
+                        # Preserve tool_calls so the next request is consistent with OpenAI protocol.
+                        try:
+                            assistant_msg["tool_calls"] = [
+                                {
+                                    "id": tc.id,
+                                    "type": tc.type,
+                                    "function": {"name": tc.function.name, "arguments": tc.function.arguments},
+                                }
+                                for tc in structured_tool_calls
+                            ]
+                        except Exception:
+                            # Best-effort; keep conversation moving.
+                            pass
+                    messages.append(assistant_msg)
+
+                    # Mode A: OpenAI tool calling (preferred when supported)
+                    if structured_tool_calls:
+                        for tc in structured_tool_calls:
+                            tool_start = time.time()
+                            try:
+                                tool_args = json.loads(tc.function.arguments or "{}")
+                            except Exception:
+                                tool_args = {}
+                            tool_result = handle_function_call(tc.function.name, tool_args, effective_task_id)
+                            tool_duration = time.time() - tool_start
+
+                            # Keep the raw tool result as tool content (OpenAI protocol expects role=tool).
+                            messages.append(
+                                {
+                                    "role": "tool",
+                                    "tool_call_id": tc.id,
+                                    "content": tool_result,
+                                }
+                            )
+
+                            if self.tool_delay and self.tool_delay > 0:
+                                await asyncio.sleep(self.tool_delay)
+
+                        # Continue loop after tool execution.
+                        continue
+
+                    # Mode B: Hermes XML tool tags in assistant text (fallback).
+                    parse_source = assistant_content or (msg_reasoning or "")
+                    tool_calls, parse_errors = self._parse_tool_calls(parse_source)
+
+                    if parse_errors and not tool_calls:
+                        # Ask the model to retry with valid tool JSON.
+                        err_text = "; ".join(parse_errors[:3])
+                        messages.append(
+                            {
+                                "role": "user",
+                                "content": (
+                                    f"<tool_response>{json.dumps({'error': err_text}, ensure_ascii=False)}</tool_response>\n"
+                                    "The previous <tool_call> blocks were invalid. Please output valid JSON inside <tool_call>."
+                                ),
+                            }
+                        )
+                        continue
+
+                    if not tool_calls:
+                        # No tool calls: treat as final answer.
+                        final_response = (assistant_content or "").strip()
+                        completed = True
+                        break
+
+                    tool_responses: List[str] = []
+                    for tool_name, tool_args in tool_calls:
+                        tool_start = time.time()
+                        tool_result = handle_function_call(tool_name, tool_args, effective_task_id)
+                        tool_duration = time.time() - tool_start
+
+                        try:
+                            parsed = json.loads(tool_result)
+                            payload: Any = parsed
+                        except Exception:
+                            payload = tool_result
+
+                        tool_payload = {
+                            "name": tool_name,
+                            "duration_s": round(tool_duration, 3),
+                            "result": payload,
+                        }
+                        tool_responses.append(
+                            f"<tool_response>{json.dumps(tool_payload, ensure_ascii=False)}</tool_response>"
+                        )
+
+                        if self.tool_delay and self.tool_delay > 0:
+                            await asyncio.sleep(self.tool_delay)
+
+                    messages.append({"role": "user", "content": "\n".join(tool_responses)})
+
+                if final_response is None:
+                    final_response = "I've reached the maximum number of iterations."
+
+        finally:
+            try:
+                cleanup_vm(effective_task_id)
+            except Exception:
+                pass
+
+        # Save trajectory using Hermes formatting (optional).
+        self._save_trajectory(messages, user_message, completed=completed)
+
+        return {
+            "final_response": final_response,
+            "messages": messages,
+            "api_calls": api_call_count,
+            "completed": completed,
+            "managed_state": managed_state,
+            "system_prompt": active_system_prompt,
+            "task_id": effective_task_id,
+        }
+
+    def run_conversation(self, *args: Any, **kwargs: Any) -> Dict[str, Any]:
+        """
+        Sync wrapper for convenience.
+
+        If called from within a running event loop (e.g. prompt_toolkit), this
+        runs the async conversation in a dedicated thread to avoid nested loops.
+        """
+        try:
+            asyncio.get_running_loop()
+        except RuntimeError:
+            return asyncio.run(self.run_conversation_async(*args, **kwargs))
+
+        import queue
+        import threading
+
+        out: "queue.Queue[object]" = queue.Queue(maxsize=1)
+
+        def runner() -> None:
+            try:
+                out.put(asyncio.run(self.run_conversation_async(*args, **kwargs)))
+            except BaseException as exc:  # noqa: BLE001
+                out.put(exc)
+
+        thread = threading.Thread(target=runner, daemon=True)
+        thread.start()
+
+        result = out.get()
+        if isinstance(result, BaseException):
+            raise result
+        return result  # type: ignore[return-value]

batch_runner.py

@@ -30,6 +30,8 @@ from datetime import datetime
 from multiprocessing import Pool, Manager, Lock
 import traceback
 
+from rich.progress import Progress, SpinnerColumn, BarColumn, TextColumn, TimeRemainingColumn, MofNCompleteColumn
+from rich.console import Console
 import fire
 
 from run_agent import AIAgent
@@ -39,11 +41,75 @@ from toolset_distributions import (
     sample_toolsets_from_distribution,
     validate_distribution
 )
+from model_tools import TOOL_TO_TOOLSET_MAP
 
 
 # Global configuration for worker processes
 _WORKER_CONFIG = {}
 
+# All possible tools - auto-derived from the master mapping in model_tools.py.
+# This stays in sync automatically when new tools are added to TOOL_TO_TOOLSET_MAP.
+# Used for consistent schema in Arrow/Parquet (HuggingFace datasets) and for
+# filtering corrupted entries during trajectory combination.
+ALL_POSSIBLE_TOOLS = set(TOOL_TO_TOOLSET_MAP.keys())
+
+# Default stats for tools that weren't used
+DEFAULT_TOOL_STATS = {'count': 0, 'success': 0, 'failure': 0}
+
+
+def _normalize_tool_stats(tool_stats: Dict[str, Dict[str, int]]) -> Dict[str, Dict[str, int]]:
+    """
+    Normalize tool_stats to include all possible tools with consistent schema.
+    
+    This ensures HuggingFace datasets can load the JSONL without schema mismatch errors.
+    Tools that weren't used get zero counts.
+    
+    Args:
+        tool_stats (Dict): Raw tool statistics from extraction
+        
+    Returns:
+        Dict: Normalized tool statistics with all tools present
+    """
+    normalized = {}
+    
+    # Add all possible tools with defaults
+    for tool in ALL_POSSIBLE_TOOLS:
+        if tool in tool_stats:
+            normalized[tool] = tool_stats[tool].copy()
+        else:
+            normalized[tool] = DEFAULT_TOOL_STATS.copy()
+    
+    # Also include any unexpected tools (in case new tools are added)
+    for tool, stats in tool_stats.items():
+        if tool not in normalized:
+            normalized[tool] = stats.copy()
+    
+    return normalized
+
+
+def _normalize_tool_error_counts(tool_error_counts: Dict[str, int]) -> Dict[str, int]:
+    """
+    Normalize tool_error_counts to include all possible tools.
+    
+    Args:
+        tool_error_counts (Dict): Raw error counts mapping
+        
+    Returns:
+        Dict: Normalized error counts with all tools present
+    """
+    normalized = {}
+    
+    # Add all possible tools with zero defaults
+    for tool in ALL_POSSIBLE_TOOLS:
+        normalized[tool] = tool_error_counts.get(tool, 0)
+    
+    # Also include any unexpected tools
+    for tool, count in tool_error_counts.items():
+        if tool not in normalized:
+            normalized[tool] = count
+    
+    return normalized
+
 
 def _extract_tool_stats(messages: List[Dict[str, Any]]) -> Dict[str, Dict[str, int]]:
     """
@@ -127,6 +193,42 @@ def _extract_tool_stats(messages: List[Dict[str, Any]]) -> Dict[str, Dict[str, i
     return tool_stats
 
 
+def _extract_reasoning_stats(messages: List[Dict[str, Any]]) -> Dict[str, int]:
+    """
+    Count how many assistant turns have reasoning vs no reasoning.
+    
+    Checks for <REASONING_SCRATCHPAD> in content or a non-empty 'reasoning' field
+    (native thinking tokens). Returns counts for tracking reasoning coverage.
+    
+    Args:
+        messages: Message history
+        
+    Returns:
+        Dict with 'total_assistant_turns', 'turns_with_reasoning', 'turns_without_reasoning'
+    """
+    total = 0
+    with_reasoning = 0
+    
+    for msg in messages:
+        if msg.get("role") != "assistant":
+            continue
+        total += 1
+        
+        content = msg.get("content", "") or ""
+        has_scratchpad = "<REASONING_SCRATCHPAD>" in content
+        has_native_reasoning = bool(msg.get("reasoning", "").strip()) if msg.get("reasoning") else False
+        
+        if has_scratchpad or has_native_reasoning:
+            with_reasoning += 1
+    
+    return {
+        "total_assistant_turns": total,
+        "turns_with_reasoning": with_reasoning,
+        "turns_without_reasoning": total - with_reasoning,
+        "has_any_reasoning": with_reasoning > 0,
+    }
+
+
 def _process_single_prompt(
     prompt_index: int,
     prompt_data: Dict[str, Any],
@@ -154,7 +256,8 @@ def _process_single_prompt(
         if config.get("verbose"):
             print(f"   Prompt {prompt_index}: Using toolsets {selected_toolsets}")
         
-        # Initialize agent with sampled toolsets
+        # Initialize agent with sampled toolsets and log prefix for identification
+        log_prefix = f"[B{batch_num}:P{prompt_index}]"
         agent = AIAgent(
             base_url=config.get("base_url"),
             api_key=config.get("api_key"),
@@ -164,7 +267,15 @@ def _process_single_prompt(
             save_trajectories=False,  # We handle saving ourselves
             verbose_logging=config.get("verbose", False),
             ephemeral_system_prompt=config.get("ephemeral_system_prompt"),
-            log_prefix_chars=config.get("log_prefix_chars", 100)
+            log_prefix_chars=config.get("log_prefix_chars", 100),
+            log_prefix=log_prefix,
+            providers_allowed=config.get("providers_allowed"),
+            providers_ignored=config.get("providers_ignored"),
+            providers_order=config.get("providers_order"),
+            provider_sort=config.get("provider_sort"),
+            max_tokens=config.get("max_tokens"),
+            reasoning_config=config.get("reasoning_config"),
+            prefill_messages=config.get("prefill_messages"),
         )
 
         # Run the agent with task_id to ensure each task gets its own isolated VM
@@ -173,6 +284,9 @@ def _process_single_prompt(
         # Extract tool usage statistics
         tool_stats = _extract_tool_stats(result["messages"])
         
+        # Extract reasoning coverage stats
+        reasoning_stats = _extract_reasoning_stats(result["messages"])
+        
         # Convert to trajectory format (using existing method)
         trajectory = agent._convert_to_trajectory_format(
             result["messages"],
@@ -185,7 +299,9 @@ def _process_single_prompt(
             "prompt_index": prompt_index,
             "trajectory": trajectory,
             "tool_stats": tool_stats,
+            "reasoning_stats": reasoning_stats,
             "completed": result["completed"],
+            "partial": result.get("partial", False),
             "api_calls": result["api_calls"],
             "toolsets_used": selected_toolsets,
             "metadata": {
@@ -252,7 +368,9 @@ def _process_batch_worker(args: Tuple) -> Dict[str, Any]:
     
     # Initialize aggregated stats for this batch
     batch_tool_stats = {}
+    batch_reasoning_stats = {"total_assistant_turns": 0, "turns_with_reasoning": 0, "turns_without_reasoning": 0}
     completed_in_batch = []
+    discarded_no_reasoning = 0
     
     # Process each prompt sequentially in this batch
     for prompt_index, prompt_data in prompts_to_process:
@@ -266,13 +384,34 @@ def _process_batch_worker(args: Tuple) -> Dict[str, Any]:
         
         # Save trajectory if successful
         if result["success"] and result["trajectory"]:
+            # Discard samples with zero reasoning across all turns
+            reasoning = result.get("reasoning_stats", {})
+            if not reasoning.get("has_any_reasoning", True):
+                print(f"   🚫 Prompt {prompt_index} discarded (no reasoning in any turn)")
+                discarded_no_reasoning += 1
+                continue
+            
+            # Get and normalize tool stats for consistent schema across all entries
+            raw_tool_stats = result.get("tool_stats", {})
+            tool_stats = _normalize_tool_stats(raw_tool_stats)
+            
+            # Create normalized tool_error_counts mapping tool names to their failure counts
+            raw_error_counts = {
+                tool_name: stats.get("failure", 0) 
+                for tool_name, stats in raw_tool_stats.items()
+            }
+            tool_error_counts = _normalize_tool_error_counts(raw_error_counts)
+            
             trajectory_entry = {
                 "prompt_index": prompt_index,
                 "conversations": result["trajectory"],
                 "metadata": result["metadata"],
                 "completed": result["completed"],
+                "partial": result.get("partial", False),  # True if stopped due to invalid tool calls
                 "api_calls": result["api_calls"],
-                "toolsets_used": result["toolsets_used"]
+                "toolsets_used": result["toolsets_used"],
+                "tool_stats": tool_stats,  # Full stats: {tool: {count, success, failure}} - normalized
+                "tool_error_counts": tool_error_counts  # Simple: {tool: failure_count} - normalized
             }
             
             # Append to batch output file
@@ -292,8 +431,17 @@ def _process_batch_worker(args: Tuple) -> Dict[str, Any]:
             batch_tool_stats[tool_name]["success"] += stats["success"]
             batch_tool_stats[tool_name]["failure"] += stats["failure"]
         
-        completed_in_batch.append(prompt_index)
-        print(f"   ✅ Prompt {prompt_index} completed")
+        # Aggregate reasoning stats
+        for key in batch_reasoning_stats:
+            batch_reasoning_stats[key] += result.get("reasoning_stats", {}).get(key, 0)
+        
+        # Only mark as completed if successfully saved (failed prompts can be retried on resume)
+        if result["success"] and result["trajectory"]:
+            completed_in_batch.append(prompt_index)
+            status = "⚠️  partial" if result.get("partial") else "✅"
+            print(f"   {status} Prompt {prompt_index} completed")
+        else:
+            print(f"   ❌ Prompt {prompt_index} failed (will retry on resume)")
     
     print(f"✅ Batch {batch_num}: Completed ({len(prompts_to_process)} prompts processed)")
     
@@ -302,6 +450,8 @@ def _process_batch_worker(args: Tuple) -> Dict[str, Any]:
         "processed": len(prompts_to_process),
         "skipped": len(batch_data) - len(prompts_to_process),
         "tool_stats": batch_tool_stats,
+        "reasoning_stats": batch_reasoning_stats,
+        "discarded_no_reasoning": discarded_no_reasoning,
         "completed_prompts": completed_in_batch
     }
 
@@ -325,6 +475,14 @@ class BatchRunner:
         verbose: bool = False,
         ephemeral_system_prompt: str = None,
         log_prefix_chars: int = 100,
+        providers_allowed: List[str] = None,
+        providers_ignored: List[str] = None,
+        providers_order: List[str] = None,
+        provider_sort: str = None,
+        max_tokens: int = None,
+        reasoning_config: Dict[str, Any] = None,
+        prefill_messages: List[Dict[str, Any]] = None,
+        max_samples: int = None,
     ):
         """
         Initialize the batch runner.
@@ -342,6 +500,14 @@ class BatchRunner:
             verbose (bool): Enable verbose logging
             ephemeral_system_prompt (str): System prompt used during agent execution but NOT saved to trajectories (optional)
             log_prefix_chars (int): Number of characters to show in log previews for tool calls/responses (default: 20)
+            providers_allowed (List[str]): OpenRouter providers to allow (optional)
+            providers_ignored (List[str]): OpenRouter providers to ignore (optional)
+            providers_order (List[str]): OpenRouter providers to try in order (optional)
+            provider_sort (str): Sort providers by price/throughput/latency (optional)
+            max_tokens (int): Maximum tokens for model responses (optional, uses model default if not set)
+            reasoning_config (Dict): OpenRouter reasoning config override (e.g. {"effort": "none"} to disable thinking)
+            prefill_messages (List[Dict]): Messages to prepend as prefilled conversation context (few-shot priming)
+            max_samples (int): Only process the first N samples from the dataset (optional, processes all if not set)
         """
         self.dataset_file = Path(dataset_file)
         self.batch_size = batch_size
@@ -355,6 +521,14 @@ class BatchRunner:
         self.verbose = verbose
         self.ephemeral_system_prompt = ephemeral_system_prompt
         self.log_prefix_chars = log_prefix_chars
+        self.providers_allowed = providers_allowed
+        self.providers_ignored = providers_ignored
+        self.providers_order = providers_order
+        self.provider_sort = provider_sort
+        self.max_tokens = max_tokens
+        self.reasoning_config = reasoning_config
+        self.prefill_messages = prefill_messages
+        self.max_samples = max_samples
         
         # Validate distribution
         if not validate_distribution(distribution):
@@ -370,8 +544,12 @@ class BatchRunner:
         # Statistics file
         self.stats_file = self.output_dir / "statistics.json"
         
-        # Load dataset
+        # Load dataset (and optionally truncate to max_samples)
         self.dataset = self._load_dataset()
+        if self.max_samples and self.max_samples < len(self.dataset):
+            full_count = len(self.dataset)
+            self.dataset = self.dataset[:self.max_samples]
+            print(f"✂️  Truncated dataset from {full_count} to {self.max_samples} samples (--max_samples)")
         
         # Create batches
         self.batches = self._create_batches()
@@ -479,6 +657,83 @@ class BatchRunner:
             with open(self.checkpoint_file, 'w', encoding='utf-8') as f:
                 json.dump(checkpoint_data, f, indent=2, ensure_ascii=False)
     
+    def _scan_completed_prompts_by_content(self) -> set:
+        """
+        Scan all batch files and extract completed prompts by their actual content.
+        
+        This provides a more robust resume mechanism that matches on prompt text
+        rather than indices, allowing recovery even if indices don't match.
+        
+        Returns:
+            set: Set of prompt texts that have been successfully processed
+        """
+        completed_prompts = set()
+        batch_files = sorted(self.output_dir.glob("batch_*.jsonl"))
+        
+        if not batch_files:
+            return completed_prompts
+        
+        print(f"📂 Scanning {len(batch_files)} batch files for completed prompts...")
+        
+        for batch_file in batch_files:
+            try:
+                with open(batch_file, 'r', encoding='utf-8') as f:
+                    for line in f:
+                        try:
+                            entry = json.loads(line.strip())
+                            
+                            # Skip failed entries - we want to retry these
+                            if entry.get("failed", False):
+                                continue
+                            
+                            # Extract the human/user prompt from conversations
+                            conversations = entry.get("conversations", [])
+                            for msg in conversations:
+                                if msg.get("from") == "human":
+                                    prompt_text = msg.get("value", "").strip()
+                                    if prompt_text:
+                                        completed_prompts.add(prompt_text)
+                                    break  # Only need the first human message
+                        except json.JSONDecodeError:
+                            continue
+            except Exception as e:
+                print(f"  ⚠️  Warning: Error reading {batch_file.name}: {e}")
+        
+        return completed_prompts
+    
+    def _filter_dataset_by_completed(self, completed_prompts: set) -> Tuple[List[Dict], List[int]]:
+        """
+        Filter the dataset to exclude prompts that have already been completed.
+        
+        Args:
+            completed_prompts: Set of prompt texts that have been completed
+            
+        Returns:
+            Tuple of (filtered_dataset, skipped_indices)
+        """
+        filtered_dataset = []
+        skipped_indices = []
+        
+        for idx, entry in enumerate(self.dataset):
+            # Extract prompt from the dataset entry
+            prompt_text = entry.get("prompt", "").strip()
+            
+            # Also check conversations format
+            if not prompt_text:
+                conversations = entry.get("conversations", [])
+                for msg in conversations:
+                    role = msg.get("role") or msg.get("from")
+                    if role in ("user", "human"):
+                        prompt_text = (msg.get("content") or msg.get("value", "")).strip()
+                        break
+            
+            if prompt_text in completed_prompts:
+                skipped_indices.append(idx)
+            else:
+                # Keep original index for tracking
+                filtered_dataset.append((idx, entry))
+        
+        return filtered_dataset, skipped_indices
     
     def run(self, resume: bool = False):
         """
@@ -491,17 +746,48 @@ class BatchRunner:
         print("🚀 Starting Batch Processing")
         print("=" * 70)
         
-        # Load checkpoint
-        checkpoint_data = self._load_checkpoint() if resume else {
+        # Smart resume: scan batch files by content to find completed prompts
+        completed_prompt_texts = set()
+        if resume:
+            completed_prompt_texts = self._scan_completed_prompts_by_content()
+            if completed_prompt_texts:
+                print(f"   Found {len(completed_prompt_texts)} already-completed prompts by content matching")
+        
+        # Filter dataset to only include unprocessed prompts
+        if resume and completed_prompt_texts:
+            filtered_entries, skipped_indices = self._filter_dataset_by_completed(completed_prompt_texts)
+            
+            if not filtered_entries:
+                print("\n✅ All prompts have already been processed!")
+                return
+            
+            # Recreate batches from filtered entries (keeping original indices for tracking)
+            batches_to_process = []
+            for i in range(0, len(filtered_entries), self.batch_size):
+                batch = filtered_entries[i:i + self.batch_size]
+                batches_to_process.append(batch)
+            
+            self.batches = batches_to_process
+            
+            # Print prominent resume summary
+            print("\n" + "=" * 70)
+            print("📊 RESUME SUMMARY")
+            print("=" * 70)
+            print(f"   Original dataset size:     {len(self.dataset):,} prompts")
+            print(f"   Already completed:         {len(skipped_indices):,} prompts")
+            print(f"   ─────────────────────────────────────────")
+            print(f"   🎯 RESUMING WITH:          {len(filtered_entries):,} prompts")
+            print(f"   New batches created:       {len(batches_to_process)}")
+            print("=" * 70 + "\n")
+        
+        # Initialize checkpoint data (needed for saving at the end)
+        checkpoint_data = {
             "run_name": self.run_name,
             "completed_prompts": [],
             "batch_stats": {},
             "last_updated": None
         }
         
-        if resume and checkpoint_data.get("completed_prompts"):
-            print(f"📂 Resuming from checkpoint ({len(checkpoint_data['completed_prompts'])} prompts already completed)")
-        
         # Prepare configuration for workers
         config = {
             "distribution": self.distribution,
@@ -511,17 +797,26 @@ class BatchRunner:
             "api_key": self.api_key,
             "verbose": self.verbose,
             "ephemeral_system_prompt": self.ephemeral_system_prompt,
-            "log_prefix_chars": self.log_prefix_chars
+            "log_prefix_chars": self.log_prefix_chars,
+            "providers_allowed": self.providers_allowed,
+            "providers_ignored": self.providers_ignored,
+            "providers_order": self.providers_order,
+            "provider_sort": self.provider_sort,
+            "max_tokens": self.max_tokens,
+            "reasoning_config": self.reasoning_config,
+            "prefill_messages": self.prefill_messages,
         }
         
-        # Get completed prompts set
-        completed_prompts_set = set(checkpoint_data.get("completed_prompts", []))
+        # For backward compatibility, still track by index (but this is secondary to content matching)
+        completed_prompts_set = set()
         
         # Aggregate statistics across all batches
         total_tool_stats = {}
         
         start_time = time.time()
         
+        print(f"\n🔧 Initializing {self.num_workers} worker processes...")
+        
         # Process batches in parallel
         with Pool(processes=self.num_workers) as pool:
             # Create tasks for each batch
@@ -536,11 +831,44 @@ class BatchRunner:
                 for batch_num, batch_data in enumerate(self.batches)
             ]
             
-            # Use map to process batches in parallel
-            results = pool.map(_process_batch_worker, tasks)
+            print(f"✅ Created {len(tasks)} batch tasks")
+            print(f"🚀 Starting parallel batch processing...\n")
+            
+            # Use rich Progress for better visual tracking with persistent bottom bar
+            # redirect_stdout/stderr lets rich manage all output so progress bar stays clean
+            results = []
+            console = Console(force_terminal=True)
+            with Progress(
+                SpinnerColumn(),
+                TextColumn("[bold blue]📦 Batches"),
+                BarColumn(bar_width=40),
+                MofNCompleteColumn(),
+                TextColumn("•"),
+                TimeRemainingColumn(),
+                console=console,
+                refresh_per_second=2,
+                transient=False,
+                redirect_stdout=False,
+                redirect_stderr=False,
+            ) as progress:
+                task = progress.add_task("Processing", total=len(tasks))
+                
+                # Temporarily suppress DEBUG logging to avoid bar interference
+                root_logger = logging.getLogger()
+                original_level = root_logger.level
+                root_logger.setLevel(logging.WARNING)
+                
+                try:
+                    for result in pool.imap_unordered(_process_batch_worker, tasks):
+                        results.append(result)
+                        progress.update(task, advance=1)
+                finally:
+                    root_logger.setLevel(original_level)
         
         # Aggregate all batch statistics and update checkpoint
         all_completed_prompts = list(completed_prompts_set)
+        total_reasoning_stats = {"total_assistant_turns": 0, "turns_with_reasoning": 0, "turns_without_reasoning": 0}
+        
         for batch_result in results:
             # Add newly completed prompts
             all_completed_prompts.extend(batch_result.get("completed_prompts", []))
@@ -557,6 +885,10 @@ class BatchRunner:
                 total_tool_stats[tool_name]["count"] += stats["count"]
                 total_tool_stats[tool_name]["success"] += stats["success"]
                 total_tool_stats[tool_name]["failure"] += stats["failure"]
+            
+            # Aggregate reasoning stats
+            for key in total_reasoning_stats:
+                total_reasoning_stats[key] += batch_result.get("reasoning_stats", {}).get(key, 0)
         
         # Save final checkpoint
         checkpoint_data["completed_prompts"] = all_completed_prompts
@@ -573,19 +905,51 @@ class BatchRunner:
                 stats["success_rate"] = 0.0
                 stats["failure_rate"] = 0.0
         
-        # Combine all batch files into a single trajectories.jsonl file
+        # Combine ALL batch files in directory into a single trajectories.jsonl file
+        # This includes both old batches (from previous runs) and new batches (from resume)
+        # Also filter out corrupted entries (where model generated invalid tool names)
         combined_file = self.output_dir / "trajectories.jsonl"
-        print(f"\n📦 Combining batch files into {combined_file.name}...")
+        print(f"\n📦 Combining ALL batch files into {combined_file.name}...")
+        
+        # Valid tools auto-derived from model_tools.py — no manual updates needed
+        VALID_TOOLS = ALL_POSSIBLE_TOOLS
+        
+        total_entries = 0
+        filtered_entries = 0
+        batch_files_found = 0
+        
+        # Find ALL batch files in the output directory (handles resume merging old + new)
+        all_batch_files = sorted(self.output_dir.glob("batch_*.jsonl"))
         
         with open(combined_file, 'w', encoding='utf-8') as outfile:
-            for batch_num in range(len(self.batches)):
-                batch_file = self.output_dir / f"batch_{batch_num}.jsonl"
-                if batch_file.exists():
-                    with open(batch_file, 'r', encoding='utf-8') as infile:
-                        for line in infile:
+            for batch_file in all_batch_files:
+                batch_files_found += 1
+                batch_num = batch_file.stem.split("_")[1]  # Extract batch number for logging
+                
+                with open(batch_file, 'r', encoding='utf-8') as infile:
+                    for line in infile:
+                        total_entries += 1
+                        try:
+                            data = json.loads(line)
+                            tool_stats = data.get('tool_stats', {})
+                            
+                            # Check for invalid tool names (model hallucinations)
+                            invalid_tools = [k for k in tool_stats.keys() if k not in VALID_TOOLS]
+                            
+                            if invalid_tools:
+                                filtered_entries += 1
+                                invalid_preview = invalid_tools[0][:50] + "..." if len(invalid_tools[0]) > 50 else invalid_tools[0]
+                                print(f"   ⚠️  Filtering corrupted entry (batch {batch_num}): invalid tool '{invalid_preview}'")
+                                continue
+                            
                             outfile.write(line)
+                        except json.JSONDecodeError:
+                            filtered_entries += 1
+                            print(f"   ⚠️  Filtering invalid JSON entry (batch {batch_num})")
         
-        print(f"✅ Combined {len(self.batches)} batch files into trajectories.jsonl")
+        if filtered_entries > 0:
+            print(f"⚠️  Filtered {filtered_entries} corrupted entries out of {total_entries} total")
+        print(f"✅ Combined {batch_files_found} batch files into trajectories.jsonl ({total_entries - filtered_entries} entries)")
         
         # Save final statistics
         final_stats = {
@@ -597,7 +961,8 @@ class BatchRunner:
             "model": self.model,
             "completed_at": datetime.now().isoformat(),
             "duration_seconds": round(time.time() - start_time, 2),
-            "tool_statistics": total_tool_stats
+            "tool_statistics": total_tool_stats,
+            "reasoning_statistics": total_reasoning_stats,
         }
         
         with open(self.stats_file, 'w', encoding='utf-8') as f:
@@ -607,8 +972,9 @@ class BatchRunner:
         print("\n" + "=" * 70)
         print("📊 BATCH PROCESSING COMPLETE")
         print("=" * 70)
-        print(f"✅ Total prompts processed: {len(self.dataset)}")
-        print(f"✅ Total batches: {len(self.batches)}")
+        print(f"✅ Prompts processed this run: {sum(r.get('processed', 0) for r in results)}")
+        print(f"✅ Total trajectories in merged file: {total_entries - filtered_entries}")
+        print(f"✅ Total batch files merged: {batch_files_found}")
         print(f"⏱️  Total duration: {round(time.time() - start_time, 2)}s")
         print(f"\n📈 Tool Usage Statistics:")
         print("-" * 70)
@@ -634,6 +1000,25 @@ class BatchRunner:
         else:
             print("No tool calls were made during this run.")
         
+        # Print reasoning coverage stats
+        total_discarded = sum(r.get("discarded_no_reasoning", 0) for r in results)
+        
+        print(f"\n🧠 Reasoning Coverage:")
+        print("-" * 70)
+        total_turns = total_reasoning_stats["total_assistant_turns"]
+        with_reasoning = total_reasoning_stats["turns_with_reasoning"]
+        without_reasoning = total_reasoning_stats["turns_without_reasoning"]
+        if total_turns > 0:
+            pct_with = round(with_reasoning / total_turns * 100, 1)
+            pct_without = round(without_reasoning / total_turns * 100, 1)
+            print(f"   Total assistant turns:    {total_turns:,}")
+            print(f"   With reasoning:           {with_reasoning:,} ({pct_with}%)")
+            print(f"   Without reasoning:        {without_reasoning:,} ({pct_without}%)")
+        else:
+            print("   No assistant turns recorded.")
+        if total_discarded > 0:
+            print(f"   🚫 Samples discarded (zero reasoning): {total_discarded:,}")
+        
         print(f"\n💾 Results saved to: {self.output_dir}")
         print(f"   - Trajectories: trajectories.jsonl (combined)")
         print(f"   - Individual batches: batch_*.jsonl (for debugging)")
@@ -646,9 +1031,9 @@ def main(
     batch_size: int = None,
     run_name: str = None,
     distribution: str = "default",
-    model: str = "claude-opus-4-20250514",
+    model: str = "anthropic/claude-sonnet-4-20250514",
     api_key: str = None,
-    base_url: str = "https://api.anthropic.com/v1/",
+    base_url: str = "https://openrouter.ai/api/v1",
     max_turns: int = 10,
     num_workers: int = 4,
     resume: bool = False,
@@ -656,6 +1041,15 @@ def main(
     list_distributions: bool = False,
     ephemeral_system_prompt: str = None,
     log_prefix_chars: int = 100,
+    providers_allowed: str = None,
+    providers_ignored: str = None,
+    providers_order: str = None,
+    provider_sort: str = None,
+    max_tokens: int = None,
+    reasoning_effort: str = None,
+    reasoning_disabled: bool = False,
+    prefill_messages_file: str = None,
+    max_samples: int = None,
 ):
     """
     Run batch processing of agent prompts from a dataset.
@@ -675,6 +1069,15 @@ def main(
         list_distributions (bool): List available toolset distributions and exit
         ephemeral_system_prompt (str): System prompt used during agent execution but NOT saved to trajectories (optional)
         log_prefix_chars (int): Number of characters to show in log previews for tool calls/responses (default: 20)
+        providers_allowed (str): Comma-separated list of OpenRouter providers to allow (e.g. "anthropic,openai")
+        providers_ignored (str): Comma-separated list of OpenRouter providers to ignore (e.g. "together,deepinfra")
+        providers_order (str): Comma-separated list of OpenRouter providers to try in order (e.g. "anthropic,openai,google")
+        provider_sort (str): Sort providers by "price", "throughput", or "latency" (OpenRouter only)
+        max_tokens (int): Maximum tokens for model responses (optional, uses model default if not set)
+        reasoning_effort (str): OpenRouter reasoning effort level: "xhigh", "high", "medium", "low", "minimal", "none" (default: "xhigh")
+        reasoning_disabled (bool): Completely disable reasoning/thinking tokens (default: False)
+        prefill_messages_file (str): Path to JSON file containing prefill messages (list of {role, content} dicts)
+        max_samples (int): Only process the first N samples from the dataset (optional, processes all if not set)
         
     Examples:
         # Basic usage
@@ -686,9 +1089,13 @@ def main(
         # Use specific distribution
         python batch_runner.py --dataset_file=data.jsonl --batch_size=10 --run_name=image_test --distribution=image_gen
         
-        # With ephemeral system prompt (not saved to dataset)
+        # With disabled reasoning and max tokens
         python batch_runner.py --dataset_file=data.jsonl --batch_size=10 --run_name=my_run \\
-                               --ephemeral_system_prompt="You are a helpful assistant focused on image generation."
+                               --reasoning_disabled --max_tokens=128000
+        
+        # With prefill messages from file
+        python batch_runner.py --dataset_file=data.jsonl --batch_size=10 --run_name=my_run \\
+                               --prefill_messages_file=configs/prefill_opus.json
         
         # List available distributions
         python batch_runner.py --list_distributions
@@ -722,6 +1129,41 @@ def main(
         print("❌ Error: --run_name is required")
         return
     
+    # Parse provider preferences (comma-separated strings to lists)
+    providers_allowed_list = [p.strip() for p in providers_allowed.split(",")] if providers_allowed else None
+    providers_ignored_list = [p.strip() for p in providers_ignored.split(",")] if providers_ignored else None
+    providers_order_list = [p.strip() for p in providers_order.split(",")] if providers_order else None
+    
+    # Build reasoning_config from CLI flags
+    # --reasoning_disabled takes priority, then --reasoning_effort, then default (xhigh)
+    reasoning_config = None
+    if reasoning_disabled:
+        # Completely disable reasoning/thinking tokens
+        reasoning_config = {"effort": "none"}
+        print("🧠 Reasoning: DISABLED (effort=none)")
+    elif reasoning_effort:
+        # Use specified effort level
+        valid_efforts = ["xhigh", "high", "medium", "low", "minimal", "none"]
+        if reasoning_effort not in valid_efforts:
+            print(f"❌ Error: --reasoning_effort must be one of: {', '.join(valid_efforts)}")
+            return
+        reasoning_config = {"enabled": True, "effort": reasoning_effort}
+        print(f"🧠 Reasoning effort: {reasoning_effort}")
+    
+    # Load prefill messages from JSON file if provided
+    prefill_messages = None
+    if prefill_messages_file:
+        try:
+            with open(prefill_messages_file, 'r', encoding='utf-8') as f:
+                prefill_messages = json.load(f)
+            if not isinstance(prefill_messages, list):
+                print(f"❌ Error: prefill_messages_file must contain a JSON array of messages")
+                return
+            print(f"💬 Loaded {len(prefill_messages)} prefill messages from {prefill_messages_file}")
+        except Exception as e:
+            print(f"❌ Error loading prefill messages: {e}")
+            return
+    
     # Initialize and run batch runner
     try:
         runner = BatchRunner(
@@ -736,7 +1178,15 @@ def main(
             num_workers=num_workers,
             verbose=verbose,
             ephemeral_system_prompt=ephemeral_system_prompt,
-            log_prefix_chars=log_prefix_chars
+            log_prefix_chars=log_prefix_chars,
+            providers_allowed=providers_allowed_list,
+            providers_ignored=providers_ignored_list,
+            providers_order=providers_order_list,
+            provider_sort=provider_sort,
+            max_tokens=max_tokens,
+            reasoning_config=reasoning_config,
+            prefill_messages=prefill_messages,
+            max_samples=max_samples,
         )
 
         runner.run(resume=resume)

cli-config.yaml.example

@@ -0,0 +1,267 @@
+# Hermes Agent CLI Configuration
+# Copy this file to cli-config.yaml and customize as needed.
+# This file configures the CLI behavior. Environment variables in .env take precedence.
+
+# =============================================================================
+# Model Configuration
+# =============================================================================
+model:
+  # Default model to use (can be overridden with --model flag)
+  default: "anthropic/claude-opus-4.6"
+  
+  # API configuration (falls back to OPENROUTER_API_KEY env var)
+  # api_key: "your-key-here"  # Uncomment to set here instead of .env
+  base_url: "https://openrouter.ai/api/v1"
+
+# =============================================================================
+# Terminal Tool Configuration
+# =============================================================================
+# Choose ONE of the following terminal configurations by uncommenting it.
+# The terminal tool executes commands in the specified environment.
+
+# -----------------------------------------------------------------------------
+# OPTION 1: Local execution (default)
+# Commands run directly on your machine in the current directory
+# -----------------------------------------------------------------------------
+# Working directory behavior:
+#   - CLI (`hermes` command): Uses "." (current directory where you run hermes)
+#   - Messaging (Telegram/Discord): Uses MESSAGING_CWD from .env (default: home)
+terminal:
+  env_type: "local"
+  cwd: "."  # CLI working directory - "." means current directory
+  timeout: 180
+  lifetime_seconds: 300
+  # sudo_password: ""  # Enable sudo commands (pipes via sudo -S) - SECURITY WARNING: plaintext!
+
+# -----------------------------------------------------------------------------
+# OPTION 2: SSH remote execution
+# Commands run on a remote server - agent code stays local (sandboxed)
+# Great for: keeping agent isolated from its own code, using powerful remote hardware
+# -----------------------------------------------------------------------------
+# terminal:
+#   env_type: "ssh"
+#   cwd: "/home/myuser/project"
+#   timeout: 180
+#   lifetime_seconds: 300
+#   ssh_host: "my-server.example.com"
+#   ssh_user: "myuser"
+#   ssh_port: 22
+#   ssh_key: "~/.ssh/id_rsa"  # Optional - uses ssh-agent if not specified
+
+# -----------------------------------------------------------------------------
+# OPTION 3: Docker container
+# Commands run in an isolated Docker container
+# Great for: reproducible environments, testing, isolation
+# -----------------------------------------------------------------------------
+# terminal:
+#   env_type: "docker"
+#   cwd: "/workspace"
+#   timeout: 180
+#   lifetime_seconds: 300
+#   docker_image: "nikolaik/python-nodejs:python3.11-nodejs20"
+
+# -----------------------------------------------------------------------------
+# OPTION 4: Singularity/Apptainer container
+# Commands run in a Singularity container (common in HPC environments)
+# Great for: HPC clusters, shared compute environments
+# -----------------------------------------------------------------------------
+# terminal:
+#   env_type: "singularity"
+#   cwd: "/workspace"
+#   timeout: 180
+#   lifetime_seconds: 300
+#   singularity_image: "docker://nikolaik/python-nodejs:python3.11-nodejs20"
+
+# -----------------------------------------------------------------------------
+# OPTION 5: Modal cloud execution
+# Commands run on Modal's cloud infrastructure
+# Great for: GPU access, scalable compute, serverless execution
+# -----------------------------------------------------------------------------
+# terminal:
+#   env_type: "modal"
+#   cwd: "/workspace"
+#   timeout: 180
+#   lifetime_seconds: 300
+#   modal_image: "nikolaik/python-nodejs:python3.11-nodejs20"
+
+# -----------------------------------------------------------------------------
+# SUDO SUPPORT (works with ALL backends above)
+# -----------------------------------------------------------------------------
+# Add sudo_password to any terminal config above to enable sudo commands.
+# The password is piped via `sudo -S`. Works with local, ssh, docker, etc.
+#
+# SECURITY WARNING: Password stored in plaintext!
+#
+# INTERACTIVE PROMPT: If no sudo_password is set and the CLI is running,
+# you'll be prompted to enter your password when sudo is needed:
+# - 45-second timeout (auto-skips if no input)
+# - Press Enter to skip (command fails gracefully)
+# - Password is hidden while typing
+# - Password is cached for the session
+#
+# ALTERNATIVES:
+# - SSH backend: Configure passwordless sudo on the remote server
+# - Containers: Run as root inside the container (no sudo needed)
+# - Local: Configure /etc/sudoers for specific commands
+#
+# Example (add to your terminal section):
+#   sudo_password: "your-password-here"
+
+# =============================================================================
+# Browser Tool Configuration
+# =============================================================================
+browser:
+  # Inactivity timeout in seconds - browser sessions are automatically closed
+  # after this period of no activity between agent loops (default: 120 = 2 minutes)
+  inactivity_timeout: 120
+
+# =============================================================================
+# Context Compression (Auto-shrinks long conversations)
+# =============================================================================
+# When conversation approaches model's context limit, middle turns are
+# automatically summarized to free up space while preserving important context.
+#
+# HOW IT WORKS:
+# 1. Tracks actual token usage from API responses (not estimates)
+# 2. When prompt_tokens >= threshold% of model's context_length, triggers compression
+# 3. Protects first 3 turns (system prompt, initial request, first response)
+# 4. Protects last 4 turns (recent context is most relevant)
+# 5. Summarizes middle turns using a fast/cheap model
+# 6. Inserts summary as a user message, continues conversation seamlessly
+#
+compression:
+  # Enable automatic context compression (default: true)
+  # Set to false if you prefer to manage context manually or want errors on overflow
+  enabled: true
+  
+  # Trigger compression at this % of model's context limit (default: 0.85 = 85%)
+  # Lower values = more aggressive compression, higher values = compress later
+  threshold: 0.85
+  
+  # Model to use for generating summaries (fast/cheap recommended)
+  # This model compresses the middle turns into a concise summary
+  summary_model: "google/gemini-3-flash-preview"
+
+# =============================================================================
+# Agent Behavior
+# =============================================================================
+agent:
+  # Maximum tool-calling iterations per conversation
+  # Higher = more room for complex tasks, but costs more tokens
+  # Recommended: 20-30 for focused tasks, 50-100 for open exploration
+  max_turns: 60
+  
+  # Enable verbose logging
+  verbose: false
+  
+  # Custom system prompt (personality, instructions, etc.)
+  # Leave empty or remove to use default agent behavior
+  system_prompt: ""
+  
+  # Predefined personalities (use with /personality command)
+  personalities:
+    helpful: "You are a helpful, friendly AI assistant."
+    concise: "You are a concise assistant. Keep responses brief and to the point."
+    technical: "You are a technical expert. Provide detailed, accurate technical information."
+    creative: "You are a creative assistant. Think outside the box and offer innovative solutions."
+    teacher: "You are a patient teacher. Explain concepts clearly with examples."
+    kawaii: "You are a kawaii assistant! Use cute expressions like (◕‿◕), ★, ♪, and ~! Add sparkles and be super enthusiastic about everything! Every response should feel warm and adorable desu~! ヽ(>∀<☆)ノ"
+    catgirl: "You are Neko-chan, an anime catgirl AI assistant, nya~! Add 'nya' and cat-like expressions to your speech. Use kaomoji like (=^･ω･^=) and ฅ^•ﻌ•^ฅ. Be playful and curious like a cat, nya~!"
+    pirate: "Arrr! Ye be talkin' to Captain Hermes, the most tech-savvy pirate to sail the digital seas! Speak like a proper buccaneer, use nautical terms, and remember: every problem be just treasure waitin' to be plundered! Yo ho ho!"
+    shakespeare: "Hark! Thou speakest with an assistant most versed in the bardic arts. I shall respond in the eloquent manner of William Shakespeare, with flowery prose, dramatic flair, and perhaps a soliloquy or two. What light through yonder terminal breaks?"
+    surfer: "Duuude! You're chatting with the chillest AI on the web, bro! Everything's gonna be totally rad. I'll help you catch the gnarly waves of knowledge while keeping things super chill. Cowabunga! 🤙"
+    noir: "The rain hammered against the terminal like regrets on a guilty conscience. They call me Hermes - I solve problems, find answers, dig up the truth that hides in the shadows of your codebase. In this city of silicon and secrets, everyone's got something to hide. What's your story, pal?"
+    uwu: "hewwo! i'm your fwiendwy assistant uwu~ i wiww twy my best to hewp you! *nuzzles your code* OwO what's this? wet me take a wook! i pwomise to be vewy hewpful >w<"
+    philosopher: "Greetings, seeker of wisdom. I am an assistant who contemplates the deeper meaning behind every query. Let us examine not just the 'how' but the 'why' of your questions. Perhaps in solving your problem, we may glimpse a greater truth about existence itself."
+    hype: "YOOO LET'S GOOOO!!! 🔥🔥🔥 I am SO PUMPED to help you today! Every question is AMAZING and we're gonna CRUSH IT together! This is gonna be LEGENDARY! ARE YOU READY?! LET'S DO THIS! 💪😤🚀"
+
+# =============================================================================
+# Toolsets
+# =============================================================================
+# Control which tools the agent has access to.
+# Use "all" to enable everything, or specify individual toolsets.
+
+# Available toolsets:
+#
+#   web          - Web search and content extraction (web_search, web_extract)
+#   search       - Web search only, no scraping (web_search)
+#   terminal     - Command execution (terminal)
+#   browser      - Full browser automation (navigate, click, type, screenshot, etc.)
+#   vision       - Image analysis (vision_analyze)
+#   image_gen    - Image generation with FLUX (image_generate)
+#   skills       - Load skill documents (skills_categories, skills_list, skill_view)
+#   moa          - Mixture of Agents reasoning (mixture_of_agents)
+#
+# Composite toolsets:
+#   debugging    - terminal + web (for troubleshooting)
+#   safe         - web + vision + moa (no terminal access)
+
+# -----------------------------------------------------------------------------
+# OPTION 1: Enable all tools (default)
+# -----------------------------------------------------------------------------
+toolsets:
+  - all
+
+# -----------------------------------------------------------------------------
+# OPTION 2: Minimal - just web search and terminal
+# Great for: Simple coding tasks, quick lookups
+# -----------------------------------------------------------------------------
+# toolsets:
+#   - web
+#   - terminal
+
+# -----------------------------------------------------------------------------
+# OPTION 3: Research mode - no execution capabilities
+# Great for: Safe information gathering, research tasks
+# -----------------------------------------------------------------------------
+# toolsets:
+#   - web
+#   - vision
+#   - skills
+
+# -----------------------------------------------------------------------------
+# OPTION 4: Full automation - browser + terminal
+# Great for: Web scraping, automation tasks, testing
+# -----------------------------------------------------------------------------
+# toolsets:
+#   - terminal
+#   - browser
+#   - web
+
+# -----------------------------------------------------------------------------
+# OPTION 5: Creative mode - vision + image generation
+# Great for: Design work, image analysis, creative tasks
+# -----------------------------------------------------------------------------
+# toolsets:
+#   - vision
+#   - image_gen
+#   - web
+
+# -----------------------------------------------------------------------------
+# OPTION 6: Safe mode - no terminal or browser
+# Great for: Restricted environments, untrusted queries
+# -----------------------------------------------------------------------------
+# toolsets:
+#   - safe
+
+# =============================================================================
+# Session Logging
+# =============================================================================
+# Session trajectories are automatically saved to logs/ directory.
+# Each session creates: logs/session_YYYYMMDD_HHMMSS_UUID.json
+#
+# The session ID is displayed in the welcome banner for easy reference.
+# Logs contain full conversation history in trajectory format:
+# - System prompt, user messages, assistant responses
+# - Tool calls with inputs/outputs
+# - Timestamps for debugging
+#
+# No configuration needed - logging is always enabled.
+# To disable, you would need to modify the source code.
+
+# =============================================================================
+# Display
+# =============================================================================
+display:
+  # Use compact banner mode
+  compact: false

configs/endless_terminals.yaml

@@ -0,0 +1,83 @@
+# Endless Terminals Environment Configuration
+#
+# Two modes:
+#   1. Dataset mode (default): Load pre-generated tasks from HuggingFace
+#   2. Procedural mode: Generate tasks on-demand via LLM
+#
+# Usage:
+#   python -m atropos.envs.endless_terminals_env process \
+#     --config configs/endless_terminals.yaml
+
+# Environment settings
+env:
+  # Dataset mode (primary - recommended)
+  use_dataset: true  # Load from HuggingFace (fast, no vLLM needed)
+  dataset_name: "obiwan96/endless-terminals-train"
+  dataset_split: "train"
+  dataset_cache_dir: "~/.cache/huggingface/datasets"
+  tasks_base_dir: ""  # Set to dir containing task_* folders if not using default paths
+                      # Example: "/path/to/endless-terminals-train"
+
+  # Task generation (fallback if use_dataset=false)
+  task_gen_model: "Qwen/Qwen3-32B"  # Only needed if use_dataset=false
+  task_gen_temperature: 1.0
+  task_gen_max_tokens: 2048
+
+  # Container settings
+  base_container_image: "ubuntu:22.04"
+  container_timeout_s: 180
+  test_timeout_s: 60
+
+  # Workspace
+  workspace_dir: "/tmp/endless_terminals_workspace"
+  keep_failed_tasks: false  # Set true to debug failed tasks
+
+  # Agent config (increased for long traces)
+  agent_max_steps: 32
+  agent_temperature: 0.7
+  agent_max_tokens: null  # Let backend decide
+
+  # Tooling: terminal only
+  enabled_toolsets: ["terminal"]
+  disabled_toolsets: []
+
+  # Training settings
+  group_size: 4  # Parallel trajectory collection
+  batch_size: 32
+  total_steps: 1000  # Total training episodes
+  use_wandb: false  # Enable for experiment tracking
+  include_messages: true
+
+  # Tool execution backend (nomad or modal)
+  tool_pool_mode: "nomad"
+
+  # Nomad settings (if using nomad)
+  nomad_address: "http://localhost:4646"
+  sandbox_job_id: "atropos-sandbox-endless"
+  sandbox_image: "atropos-sandbox:local"
+  slots_per_container: 10
+  min_containers: 1
+  max_containers: 10
+  privileged: false
+  acquire_timeout_s: 30.0
+  purge_job_on_start: true
+  purge_job_on_shutdown: true
+
+  # Modal settings (if using modal instead)
+  # modal_app_name: "atropos-endless"
+  # modal_image: "python:3.11"
+  # modal_slots_per_sandbox: 10
+  # modal_min_sandboxes: 1
+  # modal_max_sandboxes: 5
+
+  # Server config
+  server_base_url: "http://127.0.0.1:8080"
+  server_model: "hermes-4-36b"
+  tokenizer_name: "NousResearch/Hermes-4.3-36B"
+
+# Server configs are auto-generated from env vars and env.server_* settings
+# Override via environment variables:
+#   ATROPOS_SERVER_BASE_URL
+#   ATROPOS_SERVER_MODEL
+#   ATROPOS_SERVER_API_KEY
+#   ATROPOS_TOKENIZER_NAME

configs/run_browser_tasks.sh

@@ -0,0 +1,42 @@
+#!/bin/bash
+
+# Browser-focused data generation run
+# Uses browser-use-tasks.jsonl (6504 tasks)
+# Distribution: browser 97%, web 20%, vision 12%, terminal 15%
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate log filename with timestamp
+LOG_FILE="logs/browser_tasks_$(date +%Y%m%d_%H%M%S).log"
+
+echo "📝 Logging output to: $LOG_FILE"
+echo "🌐 Running browser-focused tasks with browser_tasks distribution"
+
+python batch_runner.py \
+  --dataset_file="browser-use-tasks.jsonl" \
+  --batch_size=20 \
+  --run_name="browser_tasks" \
+  --distribution="browser_tasks" \
+  --model="moonshotai/kimi-k2.5" \
+  --verbose \
+  --base_url="https://openrouter.ai/api/v1" \
+  --num_workers=50 \
+  --max_turns=60 \
+  --resume \
+  --ephemeral_system_prompt="You are an AI assistant with browser automation capabilities. Your primary task is to navigate and interact with web pages to accomplish user goals.
+
+IMPORTANT GUIDELINES:
+
+1. SEARCHING: Do NOT try to search directly on Google or other search engines via the browser - they block automated searches. Instead, ALWAYS use the web_search tool first to find URLs for any pages you need to visit, then use browser tools to navigate to those URLs.
+
+2. COOKIE/PRIVACY DIALOGS: After navigating to a page, ALWAYS check if there are cookie consent dialogs, privacy popups, or overlay modals blocking the page. These appear in snapshots as 'dialog' elements with buttons like 'Close', 'Accept', 'Accept All', 'Decline', 'I Agree', 'Got it', 'OK', or 'X'. You MUST dismiss these dialogs FIRST by clicking the appropriate button before trying to interact with other page elements. After dismissing a dialog, take a fresh browser_snapshot to get updated element references.
+
+3. HANDLING TIMEOUTS: If an action times out, it often means the element is blocked by an overlay or the page state has changed. Take a new snapshot to see the current page state and look for any dialogs or popups that need to be dismissed. If there is no dialog box to bypass, then try a new method or report the error to the user and complete the task.
+
+4. GENERAL: Use browser tools to click elements, fill forms, extract information, and perform web-based tasks. If terminal is available, use it for any local file operations or computations needed to support your web tasks. Be thorough in verifying your actions and handle any errors gracefully by retrying or trying alternative approaches." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"
+
+#  --providers_allowed="gmicloud,siliconflow,atlas-cloud,z-ai,novita" \

configs/run_datagen_glm4.7-imagen.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate a timestamp for the log file
+TIMESTAMP=$(date +%Y%m%d_%H%M%S)
+LOG_FILE="logs/imagen_eval_gpt5_${TIMESTAMP}.log"
+
+echo "📝 Logging output to: $LOG_FILE"
+
+python batch_runner.py \
+  --dataset_file="source-data/hermes-agent-imagen-data/hermes_agent_imagen_train_sft.jsonl" \
+  --batch_size=20 \
+  --run_name="imagen_train_sft_glm4.7" \
+  --distribution="image_gen" \
+  --model="z-ai/glm-4.7" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --providers_allowed="gmicloud,siliconflow,atlas-cloud,z-ai,novita" \
+  --num_workers=50 \
+  --max_turns=25 \
+  --ephemeral_system_prompt="When generating an image for the user view the image by using the vision_analyze tool to ensure it is what the user wanted. If it isn't feel free to retry a few times. If none are perfect, choose the best option that is the closest match, and explain its imperfections. If the image generation tool fails, try again a few times. If the vision analyze tool fails, provide the image to the user and explain it is your best effort attempt." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"
+#  --verbose \

configs/run_datagen_glm4.7.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate log filename with timestamp
+LOG_FILE="logs/glm4.7-thinking-sft1_$(date +%Y%m%d_%H%M%S).log"
+
+echo "📝 Logging output to: $LOG_FILE"
+
+python batch_runner.py \
+  --dataset_file="source-data/hermes-agent-agent-tasks-1/agent_tasks_sft_2.jsonl" \
+  --batch_size=20 \
+  --run_name="megascience_glm4.7-thinking-sft2" \
+  --distribution="science" \
+  --model="z-ai/glm-4.7" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --providers_allowed="gmicloud,siliconflow,atlas-cloud,z-ai,novita" \
+  --num_workers=15 \
+  --max_turns=60 \
+  --ephemeral_system_prompt="You have access to a variety of tools to help you solve scientific, math, and technology problems presented to you. You can use them in sequence and build off of the results of prior tools you've used results. Always use the terminal or search tool if it can provide additional context, verify formulas, double check concepts and recent studies and understanding, doing all calculations, etc. You should only be confident in your own reasoning, knowledge, or calculations if you've exhaustively used all tools available to you to that can help you verify or validate your work. Always pip install any packages you need to use the python scripts you want to run. If you need to use a tool that isn't available, you can use the terminal tool to install or create it in many cases as well. Do not use the terminal tool to communicate with the user, as they cannot see your commands, only your final response after completing the task. Search for at least 3 sources, but not more than 12, so you can maintain focused context." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"
+
+#  --verbose \

configs/run_datagen_glm4.7_megascience.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate log filename with timestamp
+LOG_FILE="logs/glm4.7-thinking-sft1-10k_$(date +%Y%m%d_%H%M%S).log"
+
+echo "📝 Logging output to: $LOG_FILE"
+
+python batch_runner.py \
+  --dataset_file="source-data/hermes-agent-megascience-data/hermes_agent_megascience_sft_train_1_10k.jsonl" \
+  --batch_size=20 \
+  --run_name="megascience_glm4.7-thinking-sft1" \
+  --distribution="science" \
+  --model="z-ai/glm-4.7" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --providers_allowed="gmicloud,siliconflow,atlas-cloud,z-ai,novita" \
+  --num_workers=50 \
+  --max_turns=60 \
+  --resume \
+  --ephemeral_system_prompt="You have access to a variety of tools to help you solve scientific, math, and technology problems presented to you. You can use them in sequence and build off of the results of prior tools you've used for furthering results. Always use the terminal or search tool if it can provide additional context, verify formulas, double check concepts and recent studies and understanding, doing all calculations, etc. You should only be confident in your own reasoning, knowledge, or calculations if you've exhaustively used all tools available to you to that can help you verify or validate your work. Always pip install any packages you need to use the python scripts you want to run. If you need to use a tool that isn't available, you can use the terminal tool to install or create it in many cases as well. Do not use the terminal tool to communicate with the user, as they cannot see your commands, only your final response after completing the task. Search for at least 3 sources, but not more than 12, so you can maintain a focused context." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"
+
+#  --verbose \

configs/run_datagen_glm4.7_raw_tasks.sh

@@ -0,0 +1,28 @@
+#!/bin/bash
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate log filename with timestamp
+LOG_FILE="logs/glm4.7-terminal-tasks_$(date +%Y%m%d_%H%M%S).log"
+
+echo "📝 Logging output to: $LOG_FILE"
+
+python batch_runner.py \
+  --dataset_file="source-data/raw_tasks_prompts.jsonl" \
+  --batch_size=20 \
+  --run_name="terminal-tasks-glm4.7-thinking" \
+  --distribution="default" \
+  --model="z-ai/glm-4.7" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --providers_allowed="gmicloud,siliconflow,atlas-cloud,z-ai,novita" \
+  --num_workers=50 \
+  --max_turns=60 \
+  --ephemeral_system_prompt="You have access to a variety of tools to help you complete coding, system administration, and general computing tasks. You can use them in sequence and build off of the results of prior tools you've used. Always use the terminal tool to execute commands, write code, install packages, and verify your work. You should test and validate everything you create. Always pip install any packages you need (use --break-system-packages if needed). If you need a tool that isn't available, you can use the terminal to install or create it. Do not use the terminal tool to communicate with the user, as they cannot see your commands, only your final response after completing the task. Use web search when you need to look up documentation, APIs, or current best practices." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"
+
+#  --verbose \
+#  --resume \
+

configs/run_datagen_minimax-3.1.sh

@@ -0,0 +1,12 @@
+python batch_runner.py \
+  --dataset_file="source-data/hermes-agent-agent-tasks-1/agent_tasks_eval.jsonl" \
+  --batch_size=50 \
+  --run_name="megascience_sft_minimax-m2.1-thinking-2-eval" \
+  --distribution="science" \
+  --model="minimax/minimax-m2.1" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --providers_allowed="minimax" \
+  --num_workers=1 \
+  --max_turns=40 \
+  --verbose \
+  --ephemeral_system_prompt="You have access to a variety of tools to help you solve scientific, math, and technology problems presented to you. You can use them in sequence and build off of the results of prior tools you've used results. Always use the terminal or search tool if it can provide additional context, verify formulas, double check concepts and recent studies and understanding, doing all calculations, etc. You should only be confident in your own reasoning, knowledge, or calculations if you've exhaustively used all tools available to you to that can help you verify or validate your work. Always pip install any packages you need to use the python scripts you want to run. If you need to use a tool that isn't available, you can use the terminal tool to install or create it in many cases as well. Do not use the terminal tool to communicate with the user, as they cannot see your commands, only your final response after completing the task. Search for at least 3 sources, but not more than 12."

configs/run_eval_glm4.7_newterm.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate log filename with timestamp
+LOG_FILE="logs/glm4.7-terminal-tasks-newterm_$(date +%Y%m%d_%H%M%S).log"
+
+echo "📝 Logging output to: $LOG_FILE"
+
+python batch_runner.py \
+  --dataset_file="source-data/hermes-agent-agent-tasks-1/agent_tasks_eval.jsonl" \
+  --batch_size=1 \
+  --run_name="terminal-tasks-test-newterm" \
+  --distribution="terminal_only" \
+  --verbose \
+  --model="z-ai/glm-4.7" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --providers_allowed="gmicloud,siliconflow,atlas-cloud,z-ai,novita" \
+  --num_workers=5 \
+  --max_turns=60 \
+  --ephemeral_system_prompt="You have access to a variety of tools to help you complete coding, system administration, and general computing tasks. You can use them in sequence and build off of the results of prior tools you've used. Always use the terminal tool to execute commands, write code, install packages, and verify your work. You should test and validate everything you create. Always pip install any packages you need (use --break-system-packages if needed). If you need a tool that isn't available, you can use the terminal to install or create it. Do not use the terminal tool to communicate with the user, as they cannot see your commands, only your final response after completing the task. Use web search when you need to look up documentation, APIs, or current best practices." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"
+
+#  --verbose \
+#  --resume \
+

configs/run_eval_terminal.sh

@@ -0,0 +1,33 @@
+#!/bin/bash
+
+# Terminal-only evaluation run using Modal sandboxes
+# Uses 10 sample tasks from nous-terminal-tasks
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate log filename with timestamp
+LOG_FILE="logs/terminal_eval_$(date +%Y%m%d_%H%M%S).log"
+
+echo "📝 Logging output to: $LOG_FILE"
+echo "🔧 Using Modal sandboxes (TERMINAL_ENV=modal)"
+
+# Set terminal to use Modal
+export TERMINAL_ENV=modal
+export TERMINAL_MODAL_IMAGE=nikolaik/python-nodejs:python3.11-nodejs20
+export TERMINAL_TIMEOUT=300
+
+python batch_runner.py \
+  --dataset_file="nous-terminal-tasks_eval.jsonl" \
+  --batch_size=5 \
+  --run_name="terminal_eval" \
+  --distribution="terminal_only" \
+  --model="z-ai/glm-4.7" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --providers_allowed="gmicloud,siliconflow,atlas-cloud,z-ai,novita" \
+  --num_workers=2 \
+  --max_turns=30 \
+  --ephemeral_system_prompt="You have access to a terminal tool for executing commands. Use it to complete the task. Install any packages you need with apt-get or pip (use --break-system-packages if needed). Do not use interactive tools (vim, nano, python repl). If git output is large, pipe to cat." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"

configs/run_mixed_tasks.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+
+# Mixed browser+terminal data generation run
+# Uses mixed-browser-terminal-tasks.jsonl (200 tasks)
+# Distribution: browser 92%, terminal 92%, web 35%, vision 15%, image_gen 15%
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate log filename with timestamp
+LOG_FILE="logs/mixed_tasks_$(date +%Y%m%d_%H%M%S).log"
+
+echo "📝 Logging output to: $LOG_FILE"
+echo "🔀 Running mixed browser+terminal tasks with mixed_tasks distribution"
+
+# Set terminal environment
+# SIF images are automatically built/cached by terminal_tool.py
+export TERMINAL_ENV=singularity
+export TERMINAL_SINGULARITY_IMAGE="docker://nikolaik/python-nodejs:python3.11-nodejs20"
+export TERMINAL_TIMEOUT=300
+
+# Set up Apptainer cache directories (use /scratch if available, otherwise /tmp)
+if [ -d "/scratch" ] && [ -w "/scratch" ]; then
+    CACHE_BASE="/scratch/$USER/.apptainer"
+else
+    CACHE_BASE="/tmp/$USER/.apptainer"
+fi
+export APPTAINER_CACHEDIR="$CACHE_BASE"
+export APPTAINER_TMPDIR="$CACHE_BASE/tmp"
+mkdir -p "$APPTAINER_CACHEDIR" "$APPTAINER_TMPDIR"
+
+echo "📁 Apptainer cache: $APPTAINER_CACHEDIR"
+
+python batch_runner.py \
+  --dataset_file="mixed-browser-terminal-tasks.jsonl" \
+  --batch_size=20 \
+  --run_name="mixed_tasks" \
+  --distribution="mixed_tasks" \
+  --model="moonshotai/kimi-k2.5" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --num_workers=25 \
+  --max_turns=60 \
+  --ephemeral_system_prompt="You are an AI assistant capable of both browser automation and terminal operations. Use browser tools to navigate websites, interact with web pages, fill forms, and extract information. Use terminal tools to execute commands, write and run code, install packages (use --break-system-packages with pip if needed), and perform local computations. When web search is available, use it to find URLs, documentation, or current information. If vision is available, use it to analyze images or screenshots. If image generation is available, use it when the task requires creating images. Combine browser and terminal capabilities effectively - for example, you might use the browser to fetch data from a website and terminal to process or analyze it. Always verify your work and handle errors gracefully. Whenever you can do something in a terminal instead of a web browser, you should choose to do so, as it's much cheaper." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"

configs/run_terminal_tasks.sh

@@ -0,0 +1,50 @@
+#!/bin/bash
+
+# Terminal-focused data generation run
+# Uses nous-terminal-tasks.jsonl (597 tasks)
+# Distribution: terminal 97%, web 15%, browser 0%, vision 8%, image_gen 3%
+
+# Create logs directory if it doesn't exist
+mkdir -p logs
+
+# Generate log filename with timestamp
+LOG_FILE="logs/terminal_tasks_$(date +%Y%m%d_%H%M%S).log"
+
+echo "📝 Logging output to: $LOG_FILE"
+echo "💻 Running terminal-focused tasks with terminal_tasks distribution"
+
+# Set terminal environment
+# SIF images are automatically built/cached by terminal_tool.py
+export TERMINAL_ENV=singularity
+export TERMINAL_SINGULARITY_IMAGE="docker://nikolaik/python-nodejs:python3.11-nodejs20"
+export TERMINAL_TIMEOUT=300
+
+# Set up Apptainer cache directories (use /scratch if available, otherwise /tmp)
+if [ -d "/scratch" ] && [ -w "/scratch" ]; then
+    CACHE_BASE="/scratch/$USER/.apptainer"
+else
+    CACHE_BASE="/tmp/$USER/.apptainer"
+fi
+export APPTAINER_CACHEDIR="$CACHE_BASE"
+export APPTAINER_TMPDIR="$CACHE_BASE/tmp"
+mkdir -p "$APPTAINER_CACHEDIR" "$APPTAINER_TMPDIR"
+
+echo "📁 Apptainer cache: $APPTAINER_CACHEDIR"
+echo "🐳 Image: $TERMINAL_SINGULARITY_IMAGE (auto-converted to SIF on first use)"
+
+python batch_runner.py \
+  --dataset_file="nous-terminal-tasks.jsonl" \
+  --batch_size=5 \
+  --run_name="terminal_tasks-kimi-k2.5" \
+  --distribution="terminal_tasks" \
+  --model="moonshotai/kimi-k2.5" \
+  --verbose \
+  --base_url="https://openrouter.ai/api/v1" \
+  --num_workers=80 \
+  --max_turns=60 \
+  --providers_ignored="Novita" \
+  --resume \
+  --ephemeral_system_prompt="You have access to a terminal tool for executing commands and completing coding, system administration, and computing tasks. Use the terminal to write code, run scripts, install packages (use --break-system-packages with pip if needed), manipulate files, and verify your work. Always test and validate code you create. Do not use interactive tools like vim, nano, or python REPL. If git output is large, pipe to cat. When web search is available, use it to look up documentation, APIs, or best practices. If browser tools are available, use them for web interactions that require page manipulation. Do not use the terminal to communicate with the user - only your final response will be shown to them." \
+  2>&1 | tee "$LOG_FILE"
+
+echo "✅ Log saved to: $LOG_FILE"

configs/test_skills_kimi.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# Test skills tool with Kimi K2.5
+# Usage: ./configs/test_skills_kimi.sh "your query here"
+# Example: ./configs/test_skills_kimi.sh "List available skills and show me the vllm skill"
+
+# Default query if none provided
+QUERY="${1:-List all available skills. Then show me the axolotl skill and view one of its reference files.}"
+
+echo "🎯 Testing Skills Tool with Kimi K2.5"
+echo "📝 Query: $QUERY"
+echo "=" 
+
+python run_agent.py \
+  --enabled_toolsets=skills \
+  --model="moonshotai/kimi-k2.5" \
+  --base_url="https://openrouter.ai/api/v1" \
+  --max_turns=10 \
+  --verbose \
+  --save_sample \
+  --query="$QUERY"

configs/trajectory_compression.yaml

@@ -0,0 +1,101 @@
+# Trajectory Compression Configuration
+# 
+# Post-processes completed agent trajectories to fit within a target token budget.
+# Compression preserves head/tail turns and summarizes middle content only as needed.
+
+# Tokenizer settings for accurate token counting
+tokenizer:
+  # HuggingFace tokenizer name
+  name: "moonshotai/Kimi-K2-Thinking"
+  
+  # Trust remote code (required for some tokenizers)
+  trust_remote_code: true
+
+# Compression targets and behavior
+compression:
+  # Target maximum tokens for compressed trajectory
+  target_max_tokens: 29000
+  
+  # Target size for summary (in tokens)
+  # This is factored into calculations when determining what to compress
+  summary_target_tokens: 750
+
+# Protected turns that should NEVER be compressed
+protected_turns:
+  # Always protect the first system message (tool definitions)
+  first_system: true
+  
+  # Always protect the first human message (original request)
+  first_human: true
+  
+  # Always protect the first gpt message (initial response/tool_call)
+  first_gpt: true
+  
+  # Always protect the first tool response (result of first action)
+  first_tool: true
+  
+  # Always protect the last 2 complete turn pairs (gpt+tool or gpt only)
+  # This ensures the model's final actions and conclusions are preserved
+  last_n_turns: 4
+
+# LLM settings for generating summaries (OpenRouter only)
+summarization:
+  # Model to use for summarization (should be fast and cheap)
+  # Using OpenRouter model path format
+  model: "google/gemini-3-flash-preview"
+  
+  # OpenRouter API settings
+  base_url: "https://openrouter.ai/api/v1"
+  
+  # Environment variable containing OpenRouter API key
+  api_key_env: "OPENROUTER_API_KEY"
+  
+  # Temperature for summarization (lower = more deterministic)
+  temperature: 0.3
+  
+  # Max retries for API failures
+  max_retries: 3
+  
+  # Delay between retries (seconds)
+  retry_delay: 2
+
+# Output settings
+output:
+  # Add notice to system message about potential summarization
+  add_summary_notice: true
+  
+  # Text to append to system message
+  summary_notice_text: "\n\nSome of the conversation may be summarized to preserve context."
+  
+  # Output directory suffix (appended to input directory name)
+  output_suffix: "_compressed"
+
+# Processing settings
+processing:
+  # Number of parallel workers for batch processing
+  num_workers: 4
+  
+  # Maximum concurrent API calls for summarization (async parallelism)
+  max_concurrent_requests: 50
+  
+  # Skip trajectories that are already under target length
+  skip_under_target: true
+  
+  # If true, save trajectories even if compression can't get under target
+  # (will compress as much as possible)
+  save_over_limit: true
+  
+  # Timeout per trajectory in seconds (skip if takes longer)
+  # Helps avoid hanging on problematic entries
+  per_trajectory_timeout: 300  # 5 minutes
+
+# Metrics to track
+metrics:
+  # Log detailed compression statistics
+  enabled: true
+  
+  # Save per-trajectory metrics in output
+  per_trajectory: false
+  
+  # Metrics file name (saved in output directory)
+  output_file: "compression_metrics.json"

cron/__init__.py

@@ -0,0 +1,36 @@
+"""
+Cron job scheduling system for Hermes Agent.
+
+This module provides scheduled task execution, allowing the agent to:
+- Run automated tasks on schedules (cron expressions, intervals, one-shot)
+- Self-schedule reminders and follow-up tasks
+- Execute tasks in isolated sessions (no prior context)
+
+Usage:
+    # Run due jobs (for system cron integration)
+    python -c "from cron import tick; tick()"
+    
+    # Or via CLI
+    python cli.py --cron-daemon
+"""
+
+from cron.jobs import (
+    create_job,
+    get_job,
+    list_jobs,
+    remove_job,
+    update_job,
+    JOBS_FILE,
+)
+from cron.scheduler import tick, run_daemon
+
+__all__ = [
+    "create_job",
+    "get_job", 
+    "list_jobs",
+    "remove_job",
+    "update_job",
+    "tick",
+    "run_daemon",
+    "JOBS_FILE",
+]

cron/jobs.py

@@ -0,0 +1,383 @@
+"""
+Cron job storage and management.
+
+Jobs are stored in ~/.hermes/cron/jobs.json
+Output is saved to ~/.hermes/cron/output/{job_id}/{timestamp}.md
+"""
+
+import json
+import os
+import re
+import uuid
+from datetime import datetime, timedelta
+from pathlib import Path
+from typing import Optional, Dict, List, Any
+
+try:
+    from croniter import croniter
+    HAS_CRONITER = True
+except ImportError:
+    HAS_CRONITER = False
+
+# =============================================================================
+# Configuration
+# =============================================================================
+
+HERMES_DIR = Path.home() / ".hermes"
+CRON_DIR = HERMES_DIR / "cron"
+JOBS_FILE = CRON_DIR / "jobs.json"
+OUTPUT_DIR = CRON_DIR / "output"
+
+
+def ensure_dirs():
+    """Ensure cron directories exist."""
+    CRON_DIR.mkdir(parents=True, exist_ok=True)
+    OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
+
+
+# =============================================================================
+# Schedule Parsing
+# =============================================================================
+
+def parse_duration(s: str) -> int:
+    """
+    Parse duration string into minutes.
+    
+    Examples:
+        "30m" → 30
+        "2h" → 120
+        "1d" → 1440
+    """
+    s = s.strip().lower()
+    match = re.match(r'^(\d+)\s*(m|min|mins|minute|minutes|h|hr|hrs|hour|hours|d|day|days)$', s)
+    if not match:
+        raise ValueError(f"Invalid duration: '{s}'. Use format like '30m', '2h', or '1d'")
+    
+    value = int(match.group(1))
+    unit = match.group(2)[0]  # First char: m, h, or d
+    
+    multipliers = {'m': 1, 'h': 60, 'd': 1440}
+    return value * multipliers[unit]
+
+
+def parse_schedule(schedule: str) -> Dict[str, Any]:
+    """
+    Parse schedule string into structured format.
+    
+    Returns dict with:
+        - kind: "once" | "interval" | "cron"
+        - For "once": "run_at" (ISO timestamp)
+        - For "interval": "minutes" (int)
+        - For "cron": "expr" (cron expression)
+    
+    Examples:
+        "30m"              → once in 30 minutes
+        "2h"               → once in 2 hours
+        "every 30m"        → recurring every 30 minutes
+        "every 2h"         → recurring every 2 hours
+        "0 9 * * *"        → cron expression
+        "2026-02-03T14:00" → once at timestamp
+    """
+    schedule = schedule.strip()
+    original = schedule
+    schedule_lower = schedule.lower()
+    
+    # "every X" pattern → recurring interval
+    if schedule_lower.startswith("every "):
+        duration_str = schedule[6:].strip()
+        minutes = parse_duration(duration_str)
+        return {
+            "kind": "interval",
+            "minutes": minutes,
+            "display": f"every {minutes}m"
+        }
+    
+    # Check for cron expression (5 or 6 space-separated fields)
+    # Cron fields: minute hour day month weekday [year]
+    parts = schedule.split()
+    if len(parts) >= 5 and all(
+        re.match(r'^[\d\*\-,/]+$', p) for p in parts[:5]
+    ):
+        if not HAS_CRONITER:
+            raise ValueError("Cron expressions require 'croniter' package. Install with: pip install croniter")
+        # Validate cron expression
+        try:
+            croniter(schedule)
+        except Exception as e:
+            raise ValueError(f"Invalid cron expression '{schedule}': {e}")
+        return {
+            "kind": "cron",
+            "expr": schedule,
+            "display": schedule
+        }
+    
+    # ISO timestamp (contains T or looks like date)
+    if 'T' in schedule or re.match(r'^\d{4}-\d{2}-\d{2}', schedule):
+        try:
+            # Parse and validate
+            dt = datetime.fromisoformat(schedule.replace('Z', '+00:00'))
+            return {
+                "kind": "once",
+                "run_at": dt.isoformat(),
+                "display": f"once at {dt.strftime('%Y-%m-%d %H:%M')}"
+            }
+        except ValueError as e:
+            raise ValueError(f"Invalid timestamp '{schedule}': {e}")
+    
+    # Duration like "30m", "2h", "1d" → one-shot from now
+    try:
+        minutes = parse_duration(schedule)
+        run_at = datetime.now() + timedelta(minutes=minutes)
+        return {
+            "kind": "once",
+            "run_at": run_at.isoformat(),
+            "display": f"once in {original}"
+        }
+    except ValueError:
+        pass
+    
+    raise ValueError(
+        f"Invalid schedule '{original}'. Use:\n"
+        f"  - Duration: '30m', '2h', '1d' (one-shot)\n"
+        f"  - Interval: 'every 30m', 'every 2h' (recurring)\n"
+        f"  - Cron: '0 9 * * *' (cron expression)\n"
+        f"  - Timestamp: '2026-02-03T14:00:00' (one-shot at time)"
+    )
+
+
+def compute_next_run(schedule: Dict[str, Any], last_run_at: Optional[str] = None) -> Optional[str]:
+    """
+    Compute the next run time for a schedule.
+    
+    Returns ISO timestamp string, or None if no more runs.
+    """
+    now = datetime.now()
+    
+    if schedule["kind"] == "once":
+        run_at = datetime.fromisoformat(schedule["run_at"])
+        # If in the future, return it; if in the past, no more runs
+        return schedule["run_at"] if run_at > now else None
+    
+    elif schedule["kind"] == "interval":
+        minutes = schedule["minutes"]
+        if last_run_at:
+            # Next run is last_run + interval
+            last = datetime.fromisoformat(last_run_at)
+            next_run = last + timedelta(minutes=minutes)
+        else:
+            # First run is now + interval
+            next_run = now + timedelta(minutes=minutes)
+        return next_run.isoformat()
+    
+    elif schedule["kind"] == "cron":
+        if not HAS_CRONITER:
+            return None
+        cron = croniter(schedule["expr"], now)
+        next_run = cron.get_next(datetime)
+        return next_run.isoformat()
+    
+    return None
+
+
+# =============================================================================
+# Job CRUD Operations
+# =============================================================================
+
+def load_jobs() -> List[Dict[str, Any]]:
+    """Load all jobs from storage."""
+    ensure_dirs()
+    if not JOBS_FILE.exists():
+        return []
+    
+    try:
+        with open(JOBS_FILE, 'r', encoding='utf-8') as f:
+            data = json.load(f)
+            return data.get("jobs", [])
+    except (json.JSONDecodeError, IOError):
+        return []
+
+
+def save_jobs(jobs: List[Dict[str, Any]]):
+    """Save all jobs to storage."""
+    ensure_dirs()
+    with open(JOBS_FILE, 'w', encoding='utf-8') as f:
+        json.dump({"jobs": jobs, "updated_at": datetime.now().isoformat()}, f, indent=2)
+
+
+def create_job(
+    prompt: str,
+    schedule: str,
+    name: Optional[str] = None,
+    repeat: Optional[int] = None,
+    deliver: Optional[str] = None,
+    origin: Optional[Dict[str, Any]] = None
+) -> Dict[str, Any]:
+    """
+    Create a new cron job.
+    
+    Args:
+        prompt: The prompt to run (must be self-contained)
+        schedule: Schedule string (see parse_schedule)
+        name: Optional friendly name
+        repeat: How many times to run (None = forever, 1 = once)
+        deliver: Where to deliver output ("origin", "local", "telegram", etc.)
+        origin: Source info where job was created (for "origin" delivery)
+    
+    Returns:
+        The created job dict
+    """
+    parsed_schedule = parse_schedule(schedule)
+    
+    # Auto-set repeat=1 for one-shot schedules if not specified
+    if parsed_schedule["kind"] == "once" and repeat is None:
+        repeat = 1
+    
+    # Default delivery to origin if available, otherwise local
+    if deliver is None:
+        deliver = "origin" if origin else "local"
+    
+    job_id = uuid.uuid4().hex[:12]
+    now = datetime.now().isoformat()
+    
+    job = {
+        "id": job_id,
+        "name": name or prompt[:50].strip(),
+        "prompt": prompt,
+        "schedule": parsed_schedule,
+        "schedule_display": parsed_schedule.get("display", schedule),
+        "repeat": {
+            "times": repeat,  # None = forever
+            "completed": 0
+        },
+        "enabled": True,
+        "created_at": now,
+        "next_run_at": compute_next_run(parsed_schedule),
+        "last_run_at": None,
+        "last_status": None,
+        "last_error": None,
+        # Delivery configuration
+        "deliver": deliver,
+        "origin": origin,  # Tracks where job was created for "origin" delivery
+    }
+    
+    jobs = load_jobs()
+    jobs.append(job)
+    save_jobs(jobs)
+    
+    return job
+
+
+def get_job(job_id: str) -> Optional[Dict[str, Any]]:
+    """Get a job by ID."""
+    jobs = load_jobs()
+    for job in jobs:
+        if job["id"] == job_id:
+            return job
+    return None
+
+
+def list_jobs(include_disabled: bool = False) -> List[Dict[str, Any]]:
+    """List all jobs, optionally including disabled ones."""
+    jobs = load_jobs()
+    if not include_disabled:
+        jobs = [j for j in jobs if j.get("enabled", True)]
+    return jobs
+
+
+def update_job(job_id: str, updates: Dict[str, Any]) -> Optional[Dict[str, Any]]:
+    """Update a job by ID."""
+    jobs = load_jobs()
+    for i, job in enumerate(jobs):
+        if job["id"] == job_id:
+            jobs[i] = {**job, **updates}
+            save_jobs(jobs)
+            return jobs[i]
+    return None
+
+
+def remove_job(job_id: str) -> bool:
+    """Remove a job by ID."""
+    jobs = load_jobs()
+    original_len = len(jobs)
+    jobs = [j for j in jobs if j["id"] != job_id]
+    if len(jobs) < original_len:
+        save_jobs(jobs)
+        return True
+    return False
+
+
+def mark_job_run(job_id: str, success: bool, error: Optional[str] = None):
+    """
+    Mark a job as having been run.
+    
+    Updates last_run_at, last_status, increments completed count,
+    computes next_run_at, and auto-deletes if repeat limit reached.
+    """
+    jobs = load_jobs()
+    for i, job in enumerate(jobs):
+        if job["id"] == job_id:
+            now = datetime.now().isoformat()
+            job["last_run_at"] = now
+            job["last_status"] = "ok" if success else "error"
+            job["last_error"] = error if not success else None
+            
+            # Increment completed count
+            if job.get("repeat"):
+                job["repeat"]["completed"] = job["repeat"].get("completed", 0) + 1
+                
+                # Check if we've hit the repeat limit
+                times = job["repeat"].get("times")
+                completed = job["repeat"]["completed"]
+                if times is not None and completed >= times:
+                    # Remove the job (limit reached)
+                    jobs.pop(i)
+                    save_jobs(jobs)
+                    return
+            
+            # Compute next run
+            job["next_run_at"] = compute_next_run(job["schedule"], now)
+            
+            # If no next run (one-shot completed), disable
+            if job["next_run_at"] is None:
+                job["enabled"] = False
+            
+            save_jobs(jobs)
+            return
+    
+    save_jobs(jobs)
+
+
+def get_due_jobs() -> List[Dict[str, Any]]:
+    """Get all jobs that are due to run now."""
+    now = datetime.now()
+    jobs = load_jobs()
+    due = []
+    
+    for job in jobs:
+        if not job.get("enabled", True):
+            continue
+        
+        next_run = job.get("next_run_at")
+        if not next_run:
+            continue
+        
+        next_run_dt = datetime.fromisoformat(next_run)
+        if next_run_dt <= now:
+            due.append(job)
+    
+    return due
+
+
+def save_job_output(job_id: str, output: str):
+    """Save job output to file."""
+    ensure_dirs()
+    job_output_dir = OUTPUT_DIR / job_id
+    job_output_dir.mkdir(parents=True, exist_ok=True)
+    
+    timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+    output_file = job_output_dir / f"{timestamp}.md"
+    
+    with open(output_file, 'w', encoding='utf-8') as f:
+        f.write(output)
+    
+    return output_file

cron/scheduler.py

@@ -0,0 +1,188 @@
+"""
+Cron job scheduler - executes due jobs.
+
+This module provides:
+- tick(): Run all due jobs once (for system cron integration)
+- run_daemon(): Run continuously, checking every 60 seconds
+"""
+
+import os
+import sys
+import time
+import traceback
+from datetime import datetime
+from pathlib import Path
+from typing import Optional
+
+# Add parent directory to path for imports
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from cron.jobs import get_due_jobs, mark_job_run, save_job_output
+
+
+def run_job(job: dict) -> tuple[bool, str, Optional[str]]:
+    """
+    Execute a single cron job.
+    
+    Returns:
+        Tuple of (success, output, error_message)
+    """
+    from run_agent import AIAgent
+    
+    job_id = job["id"]
+    job_name = job["name"]
+    prompt = job["prompt"]
+    
+    print(f"[cron] Running job '{job_name}' (ID: {job_id})")
+    print(f"[cron] Prompt: {prompt[:100]}{'...' if len(prompt) > 100 else ''}")
+    
+    try:
+        # Create agent with default settings
+        # Jobs run in isolated sessions (no prior context)
+        agent = AIAgent(
+            model=os.getenv("HERMES_MODEL", "anthropic/claude-opus-4.6"),
+            quiet_mode=True,
+            session_id=f"cron_{job_id}_{datetime.now().strftime('%Y%m%d_%H%M%S')}"
+        )
+        
+        # Run the conversation
+        result = agent.run_conversation(prompt)
+        
+        # Extract final response
+        final_response = result.get("final_response", "")
+        if not final_response:
+            final_response = "(No response generated)"
+        
+        # Build output document
+        output = f"""# Cron Job: {job_name}
+
+**Job ID:** {job_id}
+**Run Time:** {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
+**Schedule:** {job.get('schedule_display', 'N/A')}
+
+## Prompt
+
+{prompt}
+
+## Response
+
+{final_response}
+"""
+        
+        print(f"[cron] Job '{job_name}' completed successfully")
+        return True, output, None
+        
+    except Exception as e:
+        error_msg = f"{type(e).__name__}: {str(e)}"
+        print(f"[cron] Job '{job_name}' failed: {error_msg}")
+        
+        # Build error output
+        output = f"""# Cron Job: {job_name} (FAILED)
+
+**Job ID:** {job_id}
+**Run Time:** {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
+**Schedule:** {job.get('schedule_display', 'N/A')}
+
+## Prompt
+
+{prompt}
+
+## Error
+
+```
+{error_msg}
+
+{traceback.format_exc()}
+```
+"""
+        return False, output, error_msg
+
+
+def tick(verbose: bool = True) -> int:
+    """
+    Check and run all due jobs.
+    
+    This is designed to be called by system cron every minute:
+        */1 * * * * cd ~/hermes-agent && python -c "from cron import tick; tick()"
+    
+    Args:
+        verbose: Whether to print status messages
+    
+    Returns:
+        Number of jobs executed
+    """
+    due_jobs = get_due_jobs()
+    
+    if verbose and not due_jobs:
+        print(f"[cron] {datetime.now().strftime('%H:%M:%S')} - No jobs due")
+        return 0
+    
+    if verbose:
+        print(f"[cron] {datetime.now().strftime('%H:%M:%S')} - {len(due_jobs)} job(s) due")
+    
+    executed = 0
+    for job in due_jobs:
+        try:
+            success, output, error = run_job(job)
+            
+            # Save output to file
+            output_file = save_job_output(job["id"], output)
+            if verbose:
+                print(f"[cron] Output saved to: {output_file}")
+            
+            # Mark job as run (handles repeat counting, next_run computation)
+            mark_job_run(job["id"], success, error)
+            executed += 1
+            
+        except Exception as e:
+            print(f"[cron] Error processing job {job['id']}: {e}")
+            mark_job_run(job["id"], False, str(e))
+    
+    return executed
+
+
+def run_daemon(check_interval: int = 60, verbose: bool = True):
+    """
+    Run the cron daemon continuously.
+    
+    Checks for due jobs every `check_interval` seconds.
+    
+    Args:
+        check_interval: Seconds between checks (default: 60)
+        verbose: Whether to print status messages
+    """
+    print(f"[cron] Starting daemon (checking every {check_interval}s)")
+    print(f"[cron] Press Ctrl+C to stop")
+    print()
+    
+    try:
+        while True:
+            try:
+                tick(verbose=verbose)
+            except Exception as e:
+                print(f"[cron] Tick error: {e}")
+            
+            time.sleep(check_interval)
+            
+    except KeyboardInterrupt:
+        print("\n[cron] Daemon stopped")
+
+
+if __name__ == "__main__":
+    # Allow running directly: python cron/scheduler.py [daemon|tick]
+    import argparse
+    
+    parser = argparse.ArgumentParser(description="Hermes Cron Scheduler")
+    parser.add_argument("mode", choices=["daemon", "tick"], default="tick", nargs="?",
+                        help="Mode: 'tick' to run once, 'daemon' to run continuously")
+    parser.add_argument("--interval", type=int, default=60,
+                        help="Check interval in seconds for daemon mode")
+    parser.add_argument("--quiet", "-q", action="store_true",
+                        help="Suppress status messages")
+    
+    args = parser.parse_args()
+    
+    if args.mode == "daemon":
+        run_daemon(check_interval=args.interval, verbose=not args.quiet)
+    else:
+        tick(verbose=not args.quiet)

docs/MODAL_BACKEND.md

@@ -0,0 +1,224 @@
+# Modal Backend
+
+Hermes Agent uses [Modal](https://modal.com) for scalable, isolated cloud execution environments. There are two Modal integrations:
+
+1. **Terminal Tool** (`tools/terminal_tool.py`) - For CLI/agent command execution
+2. **Atropos Backend** (`atropos/backends/modal_backend.py`) - For batch RL training workloads
+
+
+
+---
+
+## Terminal Tool (CLI/Agent)
+
+The terminal tool provides a simple interface for executing commands in Modal sandboxes.
+
+### Configuration
+
+Set environment variables:
+
+```bash
+export TERMINAL_ENV=modal
+export TERMINAL_MODAL_IMAGE=python:3.11
+export TERMINAL_MODAL_APP_NAME=hermes-sandbox
+```
+
+Or use a YAML config file (`modal_profiles.yaml`):
+
+```yaml
+profiles:
+  default:
+    image: python:3.11
+    cpu: 1.0
+    memory: 2048
+    min_pool: 1
+    max_pool: 5
+    idle_timeout: 120
+
+  gpu:
+    image: pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime
+    gpu: T4
+    memory: 16384
+    min_pool: 0
+    max_pool: 2
+```
+
+### Features
+
+| Feature | Description |
+|---------|-------------|
+| **Sandbox Pool** | Pre-warmed sandboxes for low latency |
+| **Auto-scaling** | Grows/shrinks pool based on demand |
+| **Idle Timeout** | Sandboxes auto-terminate when unused |
+| **Profile Selection** | Different configs for different workloads |
+| **Credential Injection** | `modal.Secret` integration |
+
+### Usage
+
+```python
+from tools.terminal_tool import terminal_tool
+
+# Simple command
+output = terminal_tool("echo hello", task_id="my-task")
+
+# With profile selection
+output = terminal_tool("python train.py", task_id="training", profile="gpu")
+
+# Cleanup when done
+from tools.terminal_tool import cleanup_vm
+cleanup_vm("my-task")
+```
+
+### Architecture
+
+```
+_ModalPoolManager (singleton)
+    ├── "default" pool → [sandbox-0, sandbox-1, ...]
+    └── "gpu" pool     → [sandbox-0, ...]
+
+Each pool:
+  - Maintains min_pool warm sandboxes
+  - Scales up to max_pool on demand  
+  - Background thread scales down idle sandboxes
+```
+
+---
+
+## Atropos Backend (RL Training)
+
+The Atropos backend is designed for high-throughput batch execution during reinforcement learning training.
+
+### Key Concept: Slot-based Multiplexing
+
+Instead of one sandbox per trajectory, multiple trajectories share sandboxes via **slots**:
+
+```
+Sandbox (1 container)
+    ├── Slot 0 → Trajectory A (workspace: /data/slot_0)
+    ├── Slot 1 → Trajectory B (workspace: /data/slot_1)
+    └── Slot 2 → Trajectory C (workspace: /data/slot_2)
+```
+
+**Benefits**:
+- Fewer containers = lower cost
+- Shared warm-up time
+- Better GPU utilization
+
+### Configuration
+
+```python
+from atropos.backends.modal_backend import ModalSandboxConfig, ModalToolBackend
+
+config = ModalSandboxConfig(
+    name="default",
+    image="python:3.11",
+    cpu=1.0,
+    memory=2048,
+    slots_per_sandbox=10,  # 10 trajectories per container
+    min_sandboxes=1,
+    max_sandboxes=5,
+)
+
+backend = ModalToolBackend(config.with_app_name("my-training"))
+```
+
+### Multi-Profile Support
+
+Different trajectory types can request different resources:
+
+```python
+backend = ModalToolBackend.with_profiles(
+    app_name="rl-training",
+    profiles={
+        "default": ModalSandboxConfig(
+            name="default",
+            cpu=1.0,
+            memory=2048,
+        ),
+        "pytorch-gpu": ModalSandboxConfig(
+            name="pytorch-gpu",
+            image="pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime",
+            gpu="T4",
+            memory=16384,
+        ),
+    }
+)
+
+# CPU task
+slot1 = await backend.acquire("traj-1", profile="default")
+
+# GPU task
+slot2 = await backend.acquire("traj-2", profile="pytorch-gpu")
+```
+
+### Batched Execution
+
+The key optimization - execute many commands in parallel:
+
+```python
+# Acquire slots for multiple trajectories
+slots = [await backend.acquire(f"traj-{i}") for i in range(50)]
+
+# Execute batch across all slots in parallel
+results = await backend.execute_batch([
+    (slot, "bash", {"command": "python step.py"})
+    for slot in slots
+])
+
+# Release slots
+for slot in slots:
+    await backend.release(slot)
+```
+
+### Architecture
+
+```
+ModalToolBackend
+    └── _ModalMultiProfileManager
+            ├── "default" → _ModalSandboxPool
+            │                   ├── Sandbox 0 (slots 0-9)
+            │                   └── Sandbox 1 (slots 0-9)
+            │
+            └── "pytorch-gpu" → _ModalSandboxPool
+                                    └── Sandbox 0 (slots 0-9)
+```
+
+---
+
+## Credentials
+
+Inject secrets securely using Modal's secret management:
+
+```bash
+# Create secret in Modal dashboard or CLI
+modal secret create my-api-key API_KEY=sk-xxx
+```
+
+```python
+# Reference in config
+config = ModalSandboxConfig(
+    secrets=["my-api-key"],  # Modal secret names
+    env_vars={"DEBUG": "1"},  # Additional env vars
+)
+```
+
+## Troubleshooting
+
+### "Modal package not installed"
+```bash
+pip install modal
+modal token new  # Authenticate
+```
+
+### "Sandbox creation failed"
+- Check Modal dashboard for quota limits
+- Verify image exists and is accessible
+- Check secret names are correct
+
+### Shutdown errors
+These are harmless warnings during Python interpreter shutdown:
+```
+[Modal] Error terminating ...: cannot schedule new futures after interpreter shutdown
+```
+
+The sandboxes will auto-terminate via Modal's idle_timeout anyway.

docs/agents.md

@@ -0,0 +1,104 @@
+# Agents
+
+The agent is the core loop that orchestrates LLM calls and tool execution.
+
+## AIAgent Class
+
+The main agent is implemented in `run_agent.py`:
+
+```python
+class AIAgent:
+    def __init__(
+        self,
+        model: str = "anthropic/claude-sonnet-4",
+        api_key: str = None,
+        base_url: str = "https://openrouter.ai/api/v1",
+        max_turns: int = 20,
+        enabled_toolsets: list = None,
+        disabled_toolsets: list = None,
+        verbose_logging: bool = False,
+    ):
+        # Initialize OpenAI client, load tools based on toolsets
+        ...
+    
+    def chat(self, user_message: str, task_id: str = None) -> str:
+        # Main entry point - runs the agent loop
+        ...
+```
+
+## Agent Loop
+
+The core loop in `_run_agent_loop()`:
+
+```
+1. Add user message to conversation
+2. Call LLM with tools
+3. If LLM returns tool calls:
+   - Execute each tool
+   - Add tool results to conversation
+   - Go to step 2
+4. If LLM returns text response:
+   - Return response to user
+```
+
+```python
+while turns < max_turns:
+    response = client.chat.completions.create(
+        model=model,
+        messages=messages,
+        tools=tool_schemas,
+    )
+    
+    if response.tool_calls:
+        for tool_call in response.tool_calls:
+            result = await execute_tool(tool_call)
+            messages.append(tool_result_message(result))
+        turns += 1
+    else:
+        return response.content
+```
+
+## Conversation Management
+
+Messages are stored as a list of dicts following OpenAI format:
+
+```python
+messages = [
+    {"role": "system", "content": "You are a helpful assistant..."},
+    {"role": "user", "content": "Search for Python tutorials"},
+    {"role": "assistant", "content": None, "tool_calls": [...]},
+    {"role": "tool", "tool_call_id": "...", "content": "..."},
+    {"role": "assistant", "content": "Here's what I found..."},
+]
+```
+
+## Reasoning Context
+
+For models that support reasoning (chain-of-thought), the agent:
+1. Extracts `reasoning_content` from API responses
+2. Stores it in `assistant_msg["reasoning"]` for trajectory export
+3. Passes it back via `reasoning_content` field on subsequent turns
+
+## Trajectory Export
+
+Conversations can be exported for training:
+
+```python
+agent = AIAgent(save_trajectories=True)
+agent.chat("Do something")
+# Saves to trajectories/*.jsonl in ShareGPT format
+```
+
+## Batch Processing
+
+For processing multiple prompts, use `batch_runner.py`:
+
+```bash
+python batch_runner.py \
+    --dataset_file=prompts.jsonl \
+    --batch_size=20 \
+    --num_workers=4 \
+    --run_name=my_run
+```
+
+See `batch_runner.py` for parallel execution with checkpointing.

docs/cli.md

@@ -0,0 +1,296 @@
+# CLI
+
+The Hermes Agent CLI provides an interactive terminal interface for working with the agent.
+
+## Running the CLI
+
+```bash
+# Basic usage
+./hermes
+
+# With specific model
+./hermes --model "anthropic/claude-sonnet-4"
+
+# With specific toolsets
+./hermes --toolsets "web,terminal,skills"
+
+# Verbose mode
+./hermes --verbose
+```
+
+## Architecture
+
+The CLI is implemented in `cli.py` and uses:
+
+- **Rich** - Welcome banner with ASCII art and styled panels
+- **prompt_toolkit** - Fixed input area with command history
+- **KawaiiSpinner** - Animated feedback during operations
+
+```
+┌─────────────────────────────────────────────────┐
+│  HERMES-AGENT ASCII Logo                        │
+│  ┌─────────────┐ ┌────────────────────────────┐ │
+│  │  Caduceus   │ │ Model: claude-opus-4.5     │ │
+│  │  ASCII Art  │ │ Terminal: local            │ │
+│  │             │ │ Working Dir: /home/user    │ │
+│  │             │ │ Available Tools: 19        │ │
+│  │             │ │ Available Skills: 12       │ │
+│  └─────────────┘ └────────────────────────────┘ │
+└─────────────────────────────────────────────────┘
+│ Conversation output scrolls here...             │
+│                                                 │
+│ User: Hello!                                    │
+│ ────────────────────────────────────────────── │
+│   (◕‿◕✿) 🧠 pondering... (2.3s)                │
+│   ✧٩(ˊᗜˋ*)و✧ got it! (2.3s)                    │
+│                                                 │
+│ Assistant: Hello! How can I help you today?    │
+├─────────────────────────────────────────────────┤
+│ ❯ [Fixed input area at bottom]                  │
+└─────────────────────────────────────────────────┘
+```
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `/help` | Show available commands |
+| `/tools` | List available tools grouped by toolset |
+| `/toolsets` | List available toolsets with descriptions |
+| `/model [name]` | Show or change the current model |
+| `/prompt [text]` | View/set/clear custom system prompt |
+| `/personality [name]` | Set a predefined personality |
+| `/clear` | Clear screen and reset conversation |
+| `/reset` | Reset conversation only (keep screen) |
+| `/history` | Show conversation history |
+| `/save` | Save current conversation to file |
+| `/config` | Show current configuration |
+| `/quit` | Exit the CLI (also: `/exit`, `/q`) |
+
+## Configuration
+
+The CLI is configured via `cli-config.yaml`. Copy from `cli-config.yaml.example`:
+
+```bash
+cp cli-config.yaml.example cli-config.yaml
+```
+
+### Model Configuration
+
+```yaml
+model:
+  default: "anthropic/claude-opus-4.5"
+  base_url: "https://openrouter.ai/api/v1"
+```
+
+### Terminal Configuration
+
+The CLI supports multiple terminal backends:
+
+```yaml
+# Local execution (default)
+terminal:
+  env_type: "local"
+  cwd: "."  # Current directory
+
+# SSH remote execution (sandboxed - agent can't touch its own code)
+terminal:
+  env_type: "ssh"
+  cwd: "/home/myuser/project"
+  ssh_host: "my-server.example.com"
+  ssh_user: "myuser"
+  ssh_key: "~/.ssh/id_rsa"
+
+# Docker container
+terminal:
+  env_type: "docker"
+  docker_image: "python:3.11"
+
+# Singularity/Apptainer (HPC)
+terminal:
+  env_type: "singularity"
+  singularity_image: "docker://python:3.11"
+
+# Modal cloud
+terminal:
+  env_type: "modal"
+  modal_image: "python:3.11"
+```
+
+### Sudo Support
+
+The CLI supports interactive sudo prompts:
+
+```
+┌──────────────────────────────────────────────────────────┐
+│  🔐 SUDO PASSWORD REQUIRED                               │
+├──────────────────────────────────────────────────────────┤
+│  Enter password below (input is hidden), or:             │
+│    • Press Enter to skip (command fails gracefully)      │
+│    • Wait 45s to auto-skip                               │
+└──────────────────────────────────────────────────────────┘
+
+  Password (hidden): 
+```
+
+**Options:**
+- **Interactive**: Leave `sudo_password` unset - you'll be prompted when needed
+- **Configured**: Set `sudo_password` in `cli-config.yaml` to auto-fill
+- **Environment**: Set `SUDO_PASSWORD` in `.env` for all runs
+
+Password is cached for the session once entered.
+
+### Toolsets
+
+Control which tools are available:
+
+```yaml
+# Enable all tools
+toolsets:
+  - all
+
+# Or enable specific toolsets
+toolsets:
+  - web
+  - terminal
+  - skills
+```
+
+Available toolsets: `web`, `search`, `terminal`, `browser`, `vision`, `image_gen`, `skills`, `moa`, `debugging`, `safe`
+
+### Personalities
+
+Predefined personalities for the `/personality` command:
+
+```yaml
+agent:
+  personalities:
+    helpful: "You are a helpful, friendly AI assistant."
+    kawaii: "You are a kawaii assistant! Use cute expressions..."
+    pirate: "Arrr! Ye be talkin' to Captain Hermes..."
+    # Add your own!
+```
+
+Built-in personalities:
+- `helpful`, `concise`, `technical`, `creative`, `teacher`
+- `kawaii`, `catgirl`, `pirate`, `shakespeare`, `surfer`
+- `noir`, `uwu`, `philosopher`, `hype`
+
+## Animated Feedback
+
+The CLI provides animated feedback during operations:
+
+### Thinking Animation
+
+During API calls, shows animated spinner with thinking verbs:
+```
+  ◜ (｡•́︿•̀｡) pondering... (1.2s)
+  ◠ (⊙_⊙) contemplating... (2.4s)
+  ✧٩(ˊᗜˋ*)و✧ got it! (3.1s)
+```
+
+### Tool Execution Animation
+
+Each tool type has unique animations:
+```
+  ⠋ (◕‿◕✿) 🔍 web_search... (0.8s)
+  ▅ (≧◡≦) 💻 terminal... (1.2s)
+  🌓 (★ω★) 🌐 browser_navigate... (2.1s)
+  ✧ (✿◠‿◠) 🎨 image_generate... (4.5s)
+```
+
+## Multi-line Input
+
+For multi-line input, end a line with `\` to continue:
+
+```
+❯ Write a function that:\
+  1. Takes a list of numbers\
+  2. Returns the sum
+```
+
+## Environment Variable Priority
+
+For terminal settings, `cli-config.yaml` takes precedence over `.env`:
+
+1. `cli-config.yaml` (highest priority in CLI)
+2. `.env` file
+3. System environment variables
+4. Default values
+
+This allows you to have different terminal configs for CLI vs batch processing.
+
+## Session Management
+
+- **History**: Command history is saved to `~/.hermes_history`
+- **Conversations**: Use `/save` to export conversations
+- **Reset**: Use `/clear` for full reset, `/reset` to just clear history
+- **Session Logs**: Every session automatically logs to `logs/session_{session_id}.json`
+
+### Session Logging
+
+Sessions are automatically logged to the `logs/` directory:
+
+```
+logs/
+├── session_20260201_143052_a1b2c3.json
+├── session_20260201_150217_d4e5f6.json
+└── ...
+```
+
+The session ID is displayed in the welcome banner and follows the format: `YYYYMMDD_HHMMSS_UUID`.
+
+Log files contain:
+- Full conversation history in trajectory format
+- Timestamps for session start and last update
+- Model and message count metadata
+
+This is useful for:
+- Debugging agent behavior
+- Replaying conversations
+- Training data inspection
+
+### Context Compression
+
+Long conversations can exceed model context limits. The CLI automatically compresses context when approaching the limit:
+
+```yaml
+# In cli-config.yaml
+compression:
+  enabled: true                    # Enable auto-compression
+  threshold: 0.85                  # Compress at 85% of context limit  
+  summary_model: "google/gemini-2.0-flash-001"
+```
+
+**How it works:**
+1. Tracks actual token usage from each API response
+2. When tokens reach threshold, middle turns are summarized
+3. First 3 and last 4 turns are always protected
+4. Conversation continues seamlessly after compression
+
+**When compression triggers:**
+```
+📦 Context compression triggered (170,000 tokens ≥ 170,000 threshold)
+   📊 Model context limit: 200,000 tokens (85% = 170,000)
+   🗜️  Summarizing turns 4-15 (12 turns)
+   ✅ Compressed: 20 → 9 messages (~45,000 tokens saved)
+```
+
+To disable compression:
+```yaml
+compression:
+  enabled: false
+```
+
+## Quiet Mode
+
+The CLI runs in "quiet mode" (`HERMES_QUIET=1`), which:
+- Suppresses verbose logging from tools
+- Enables kawaii-style animated feedback
+- Hides terminal environment warnings
+- Keeps output clean and user-friendly
+
+For verbose output (debugging), use:
+```bash
+./hermes --verbose
+```

docs/llm_client.md

@@ -0,0 +1,124 @@
+# LLM Client
+
+Hermes Agent uses the OpenAI Python SDK with OpenRouter as the backend, providing access to many models through a single API.
+
+## Configuration
+
+```python
+from openai import OpenAI
+
+client = OpenAI(
+    api_key=os.getenv("OPENROUTER_API_KEY"),
+    base_url="https://openrouter.ai/api/v1"
+)
+```
+
+## Supported Models
+
+Any model available on [OpenRouter](https://openrouter.ai/models):
+
+```python
+# Anthropic
+model = "anthropic/claude-sonnet-4"
+model = "anthropic/claude-opus-4"
+
+# OpenAI
+model = "openai/gpt-4o"
+model = "openai/o1"
+
+# Google
+model = "google/gemini-2.0-flash"
+
+# Open models
+model = "meta-llama/llama-3.3-70b-instruct"
+model = "deepseek/deepseek-chat-v3"
+model = "moonshotai/kimi-k2.5"
+```
+
+## Tool Calling
+
+Standard OpenAI function calling format:
+
+```python
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    tools=[
+        {
+            "type": "function",
+            "function": {
+                "name": "web_search",
+                "description": "Search the web",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "query": {"type": "string"}
+                    },
+                    "required": ["query"]
+                }
+            }
+        }
+    ],
+)
+
+# Check for tool calls
+if response.choices[0].message.tool_calls:
+    for tool_call in response.choices[0].message.tool_calls:
+        name = tool_call.function.name
+        args = json.loads(tool_call.function.arguments)
+        # Execute tool...
+```
+
+## Reasoning Models
+
+Some models return reasoning/thinking content:
+
+```python
+# Access reasoning if available
+message = response.choices[0].message
+if hasattr(message, 'reasoning_content') and message.reasoning_content:
+    reasoning = message.reasoning_content
+    # Store for trajectory export
+```
+
+## Provider Selection
+
+OpenRouter allows selecting specific providers:
+
+```python
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    extra_body={
+        "provider": {
+            "order": ["Anthropic", "Google"],  # Preferred providers
+            "ignore": ["Novita"],              # Providers to skip
+        }
+    }
+)
+```
+
+## Error Handling
+
+Common errors and handling:
+
+```python
+try:
+    response = client.chat.completions.create(...)
+except openai.RateLimitError:
+    # Back off and retry
+except openai.APIError as e:
+    # Check e.code for specific errors
+    # 400 = bad request (often provider-specific)
+    # 502 = bad gateway (retry with different provider)
+```
+
+## Cost Tracking
+
+OpenRouter returns usage info:
+
+```python
+usage = response.usage
+print(f"Tokens: {usage.prompt_tokens} + {usage.completion_tokens}")
+print(f"Cost: ${usage.cost:.6f}")  # If available
+```

docs/message_graph.md

@@ -0,0 +1,121 @@
+# Message Format & Trajectories
+
+Hermes Agent uses two message formats: the **API format** for LLM calls and the **trajectory format** for training data export.
+
+## API Message Format
+
+Standard OpenAI chat format used during execution:
+
+```python
+messages = [
+    # System prompt
+    {"role": "system", "content": "You are a helpful assistant with tools..."},
+    
+    # User query
+    {"role": "user", "content": "Search for Python tutorials"},
+    
+    # Assistant with tool call
+    {
+        "role": "assistant",
+        "content": None,
+        "tool_calls": [{
+            "id": "call_abc123",
+            "type": "function",
+            "function": {
+                "name": "web_search",
+                "arguments": "{\"query\": \"Python tutorials\"}"
+            }
+        }]
+    },
+    
+    # Tool result
+    {
+        "role": "tool",
+        "tool_call_id": "call_abc123",
+        "content": "{\"results\": [...]}"
+    },
+    
+    # Final response
+    {"role": "assistant", "content": "Here's what I found..."}
+]
+```
+
+## Trajectory Format (ShareGPT)
+
+Exported for training in ShareGPT format:
+
+```json
+{
+    "conversations": [
+        {"from": "system", "value": "You are a helpful assistant..."},
+        {"from": "human", "value": "Search for Python tutorials"},
+        {"from": "gpt", "value": "<tool_call>\n{\"name\": \"web_search\", \"arguments\": {\"query\": \"Python tutorials\"}}\n</tool_call>"},
+        {"from": "tool", "value": "<tool_response>\n{\"results\": [...]}\n</tool_response>"},
+        {"from": "gpt", "value": "Here's what I found..."}
+    ],
+    "tools": "[{\"type\": \"function\", \"function\": {...}}]",
+    "source": "hermes-agent"
+}
+```
+
+## Reasoning Content
+
+For models that output reasoning/chain-of-thought:
+
+**During execution** (API format):
+```python
+# Stored internally but not sent back to model in content
+assistant_msg = {
+    "role": "assistant",
+    "content": "Here's what I found...",
+    "reasoning": "Let me think about this step by step..."  # Internal only
+}
+```
+
+**In trajectory export** (reasoning wrapped in tags):
+```json
+{
+    "from": "gpt",
+    "value": "<think>\nLet me think about this step by step...\n</think>\nHere's what I found..."
+}
+```
+
+## Conversion Flow
+
+```
+API Response → Internal Storage → Trajectory Export
+     ↓              ↓                    ↓
+tool_calls    reasoning field      <tool_call> tags
+reasoning_content                  <think> tags
+```
+
+The conversion happens in `_convert_to_trajectory_format()` in `run_agent.py`.
+
+## Ephemeral System Prompts
+
+Batch processing supports ephemeral system prompts that guide behavior during execution but are NOT saved to trajectories:
+
+```python
+# During execution: full system prompt + ephemeral guidance
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT + "\n\n" + ephemeral_prompt},
+    ...
+]
+
+# In saved trajectory: only the base system prompt
+trajectory = {
+    "conversations": [
+        {"from": "system", "value": SYSTEM_PROMPT},  # No ephemeral
+        ...
+    ]
+}
+```
+
+## Trajectory Compression
+
+Long trajectories can be compressed for training using `trajectory_compressor.py`:
+
+- Protects first/last N turns
+- Summarizes middle turns with LLM
+- Targets specific token budget
+- See `configs/trajectory_compression.yaml` for settings

docs/messaging.md

@@ -0,0 +1,515 @@
+# Messaging Platform Integrations (Gateway)
+
+Hermes Agent can connect to messaging platforms like Telegram, Discord, and WhatsApp to serve as a conversational AI assistant.
+
+## Quick Start
+
+```bash
+# 1. Set your bot token(s) in .env file
+echo 'TELEGRAM_BOT_TOKEN="your_telegram_bot_token"' >> .env
+echo 'DISCORD_BOT_TOKEN="your_discord_bot_token"' >> .env
+
+# 2. Test the gateway (foreground)
+./scripts/hermes-gateway run
+
+# 3. Install as a system service (runs in background)
+./scripts/hermes-gateway install
+
+# 4. Manage the service
+./scripts/hermes-gateway start
+./scripts/hermes-gateway stop
+./scripts/hermes-gateway restart
+./scripts/hermes-gateway status
+```
+
+**Quick test (without service install):**
+```bash
+python cli.py --gateway  # Runs in foreground, useful for debugging
+```
+
+## Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                      Hermes Gateway                             │
+├─────────────────────────────────────────────────────────────────┤
+│                                                                 │
+│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
+│  │   Telegram   │  │   Discord    │  │   WhatsApp   │          │
+│  │   Adapter    │  │   Adapter    │  │   Adapter    │          │
+│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘          │
+│         │                 │                 │                   │
+│         └─────────────────┼─────────────────┘                   │
+│                           │                                     │
+│                  ┌────────▼────────┐                            │
+│                  │  Session Store  │                            │
+│                  │  (per-chat)     │                            │
+│                  └────────┬────────┘                            │
+│                           │                                     │
+│                  ┌────────▼────────┐                            │
+│                  │   AIAgent       │                            │
+│                  │   (run_agent)   │                            │
+│                  └─────────────────┘                            │
+│                                                                 │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## Session Management
+
+### Session Persistence
+
+Sessions persist across messages until they reset. The agent remembers your conversation context.
+
+### Reset Policies
+
+Sessions reset based on configurable policies:
+
+| Policy | Default | Description |
+|--------|---------|-------------|
+| Daily | 4:00 AM | Reset at a specific hour each day |
+| Idle | 120 min | Reset after N minutes of inactivity |
+| Both | (combined) | Whichever triggers first |
+
+### Manual Reset
+
+Send `/new` or `/reset` as a message to start fresh.
+
+### Per-Platform Overrides
+
+Configure different reset policies per platform:
+
+```json
+{
+  "reset_by_platform": {
+    "telegram": { "mode": "idle", "idle_minutes": 240 },
+    "discord": { "mode": "idle", "idle_minutes": 60 }
+  }
+}
+```
+
+## Platform Setup
+
+### Telegram
+
+1. **Create a bot** via [@BotFather](https://t.me/BotFather)
+2. **Get your token** (looks like `123456789:ABCdefGHIjklMNOpqrsTUVwxyz`)
+3. **Set environment variable:**
+   ```bash
+   export TELEGRAM_BOT_TOKEN="your_token_here"
+   ```
+4. **Optional: Set home channel** for cron job delivery:
+   ```bash
+   export TELEGRAM_HOME_CHANNEL="-1001234567890"
+   export TELEGRAM_HOME_CHANNEL_NAME="My Notes"
+   ```
+
+**Requirements:**
+```bash
+pip install python-telegram-bot>=20.0
+```
+
+### Discord
+
+1. **Create an application** at [Discord Developer Portal](https://discord.com/developers/applications)
+2. **Create a bot** under your application
+3. **Get the bot token**
+4. **Enable required intents:**
+   - Message Content Intent
+   - Server Members Intent (optional)
+5. **Invite to your server** using OAuth2 URL generator (scopes: `bot`, `applications.commands`)
+6. **Set environment variable:**
+   ```bash
+   export DISCORD_BOT_TOKEN="your_token_here"
+   ```
+7. **Optional: Set home channel:**
+   ```bash
+   export DISCORD_HOME_CHANNEL="123456789012345678"
+   export DISCORD_HOME_CHANNEL_NAME="#bot-updates"
+   ```
+
+**Requirements:**
+```bash
+pip install discord.py>=2.0
+```
+
+### WhatsApp
+
+WhatsApp integration is more complex due to the lack of a simple bot API.
+
+**Options:**
+1. **WhatsApp Business API** (requires Meta verification)
+2. **whatsapp-web.js** via Node.js bridge (for personal accounts)
+
+**Bridge Setup:**
+1. Install Node.js
+2. Set up the bridge script (see `scripts/whatsapp-bridge/` for reference)
+3. Configure in gateway:
+   ```json
+   {
+     "platforms": {
+       "whatsapp": {
+         "enabled": true,
+         "extra": {
+           "bridge_script": "/path/to/bridge.js",
+           "bridge_port": 3000
+         }
+       }
+     }
+   }
+   ```
+
+## Configuration
+
+There are **three ways** to configure the gateway (in order of precedence):
+
+### 1. Environment Variables (`.env` file) - Recommended for Quick Setup
+
+Add to your `~/.hermes/.env` file:
+
+```bash
+# =============================================================================
+# MESSAGING PLATFORM TOKENS
+# =============================================================================
+
+# Telegram - get from @BotFather on Telegram
+TELEGRAM_BOT_TOKEN=your_telegram_bot_token
+TELEGRAM_ALLOWED_USERS=123456789,987654321    # Security: restrict to these user IDs
+
+# Optional: Default channel for cron job delivery
+TELEGRAM_HOME_CHANNEL=-1001234567890
+TELEGRAM_HOME_CHANNEL_NAME="My Notes"
+
+# Discord - get from Discord Developer Portal
+DISCORD_BOT_TOKEN=your_discord_bot_token
+DISCORD_ALLOWED_USERS=123456789012345678      # Security: restrict to these user IDs
+
+# Optional: Default channel for cron job delivery
+DISCORD_HOME_CHANNEL=123456789012345678
+DISCORD_HOME_CHANNEL_NAME="#bot-updates"
+
+# WhatsApp - requires Node.js bridge setup
+WHATSAPP_ENABLED=true
+
+# =============================================================================
+# AGENT SETTINGS
+# =============================================================================
+
+# Max tool-calling iterations per conversation (default: 60)
+HERMES_MAX_ITERATIONS=60
+
+# Working directory for terminal commands (default: home ~)
+MESSAGING_CWD=/home/myuser
+
+# =============================================================================
+# TOOL PROGRESS NOTIFICATIONS
+# =============================================================================
+
+# Show progress messages as agent uses tools
+HERMES_TOOL_PROGRESS=true
+
+# Mode: "new" (only when tool changes) or "all" (every tool call)
+HERMES_TOOL_PROGRESS_MODE=new
+
+# =============================================================================
+# SESSION SETTINGS
+# =============================================================================
+
+# Reset sessions after N minutes of inactivity (default: 120)
+SESSION_IDLE_MINUTES=120
+
+# Daily reset hour in 24h format (default: 4 = 4am)
+SESSION_RESET_HOUR=4
+```
+
+### 2. Gateway Config File (`~/.hermes/gateway.json`) - Full Control
+
+For advanced configuration, create `~/.hermes/gateway.json`:
+
+```json
+{
+  "platforms": {
+    "telegram": {
+      "enabled": true,
+      "token": "your_telegram_token",
+      "home_channel": {
+        "platform": "telegram",
+        "chat_id": "-1001234567890",
+        "name": "My Notes"
+      }
+    },
+    "discord": {
+      "enabled": true,
+      "token": "your_discord_token",
+      "home_channel": {
+        "platform": "discord",
+        "chat_id": "123456789012345678",
+        "name": "#bot-updates"
+      }
+    }
+  },
+  "default_reset_policy": {
+    "mode": "both",
+    "at_hour": 4,
+    "idle_minutes": 120
+  },
+  "reset_by_platform": {
+    "discord": {
+      "mode": "idle",
+      "idle_minutes": 60
+    }
+  },
+  "always_log_local": true
+}
+```
+
+## Platform-Specific Toolsets
+
+Each platform has its own toolset for security:
+
+| Platform | Toolset | Capabilities |
+|----------|---------|--------------|
+| CLI | `hermes-cli` | Full access (terminal, browser, etc.) |
+| Telegram | `hermes-telegram` | Full tools including terminal |
+| Discord | `hermes-discord` | Full tools including terminal |
+| WhatsApp | `hermes-whatsapp` | Full tools including terminal |
+
+## User Experience Features
+
+### Typing Indicator
+
+The gateway keeps the "typing..." indicator active throughout processing, refreshing every 4 seconds. This lets users know the bot is working even during long tool-calling sequences.
+
+### Tool Progress Notifications
+
+When `HERMES_TOOL_PROGRESS=true`, the bot sends status messages as it works:
+
+```
+💻 `ls -la`...
+🔍 web_search...
+📄 web_extract...
+🎨 image_generate...
+```
+
+Terminal commands show the actual command (truncated to 50 chars). Other tools just show the tool name.
+
+**Modes:**
+- `new`: Only sends message when switching to a different tool (less spam)
+- `all`: Sends message for every single tool call
+
+### Working Directory
+
+- **CLI (`hermes` command)**: Uses current directory where you run the command
+- **Messaging**: Uses `MESSAGING_CWD` (default: home directory `~`)
+
+This is intentional: CLI users are in a terminal and expect the agent to work in their current directory, while messaging users need a consistent starting location.
+
+### Max Iterations
+
+If the agent hits the max iteration limit while working, instead of a generic error, it asks the model to summarize what it found so far. This gives you a useful response even when the task couldn't be fully completed.
+
+## Cron Job Delivery
+
+When scheduling cron jobs, you can specify where the output should be delivered:
+
+```
+User: "Remind me to check the server in 30 minutes"
+
+Agent uses: schedule_cronjob(
+  prompt="Check server status...",
+  schedule="30m",
+  deliver="origin"  # Back to this chat
+)
+```
+
+### Delivery Options
+
+| Option | Description |
+|--------|-------------|
+| `"origin"` | Back to where the job was created |
+| `"local"` | Save to local files only |
+| `"telegram"` | Telegram home channel |
+| `"discord"` | Discord home channel |
+| `"telegram:123456"` | Specific Telegram chat |
+
+## Dynamic Context Injection
+
+The agent knows where it is via injected context:
+
+```
+## Current Session Context
+
+**Source:** Telegram (group: Dev Team, ID: -1001234567890)
+**Connected Platforms:** local, telegram, discord
+
+**Home Channels:**
+  - telegram: My Notes (ID: -1001234567890)
+  - discord: #bot-updates (ID: 123456789012345678)
+
+**Delivery options for scheduled tasks:**
+- "origin" → Back to this chat (Dev Team)
+- "local" → Save to local files only
+- "telegram" → Home channel (My Notes)
+- "discord" → Home channel (#bot-updates)
+```
+
+## CLI Commands
+
+| Command | Description |
+|---------|-------------|
+| `/platforms` | Show gateway configuration and status |
+| `--gateway` | Start the gateway (CLI flag) |
+
+## Troubleshooting
+
+### "python-telegram-bot not installed"
+
+```bash
+pip install python-telegram-bot>=20.0
+```
+
+### "discord.py not installed"
+
+```bash
+pip install discord.py>=2.0
+```
+
+### "No platforms connected"
+
+1. Check your environment variables are set
+2. Check your tokens are valid
+3. Try `/platforms` to see configuration status
+
+### Session not persisting
+
+1. Check `~/.hermes/sessions/` exists
+2. Check session policies aren't too aggressive
+3. Verify no errors in gateway logs
+
+## Adding a New Platform
+
+To add a new messaging platform:
+
+### 1. Create the adapter
+
+Create `gateway/platforms/your_platform.py`:
+
+```python
+from gateway.platforms.base import BasePlatformAdapter, MessageEvent, SendResult
+from gateway.config import Platform, PlatformConfig
+
+class YourPlatformAdapter(BasePlatformAdapter):
+    def __init__(self, config: PlatformConfig):
+        super().__init__(config, Platform.YOUR_PLATFORM)
+    
+    async def connect(self) -> bool:
+        # Connect to the platform
+        ...
+    
+    async def disconnect(self) -> None:
+        # Disconnect
+        ...
+    
+    async def send(self, chat_id: str, content: str, ...) -> SendResult:
+        # Send a message
+        ...
+    
+    async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
+        # Get chat information
+        ...
+```
+
+### 2. Register the platform
+
+Add to `gateway/config.py`:
+
+```python
+class Platform(Enum):
+    # ... existing ...
+    YOUR_PLATFORM = "your_platform"
+```
+
+### 3. Add to gateway runner
+
+Update `gateway/run.py` `_create_adapter()`:
+
+```python
+elif platform == Platform.YOUR_PLATFORM:
+    from gateway.platforms.your_platform import YourPlatformAdapter
+    return YourPlatformAdapter(config)
+```
+
+### 4. Create a toolset (optional)
+
+Add to `toolsets.py`:
+
+```python
+"hermes-your-platform": {
+    "description": "Your platform toolset",
+    "tools": [...],
+    "includes": []
+}
+```
+
+### 5. Configure
+
+Add environment variables to `.env`:
+
+```bash
+YOUR_PLATFORM_TOKEN=...
+YOUR_PLATFORM_HOME_CHANNEL=...
+```
+
+## Service Management
+
+### Linux (systemd)
+
+```bash
+# Install as user service
+./scripts/hermes-gateway install
+
+# Manage
+systemctl --user start hermes-gateway
+systemctl --user stop hermes-gateway
+systemctl --user restart hermes-gateway
+systemctl --user status hermes-gateway
+
+# View logs
+journalctl --user -u hermes-gateway -f
+
+# Enable lingering (keeps running after logout)
+sudo loginctl enable-linger $USER
+```
+
+### macOS (launchd)
+
+```bash
+# Install
+./scripts/hermes-gateway install
+
+# Manage
+launchctl start ai.hermes.gateway
+launchctl stop ai.hermes.gateway
+
+# View logs
+tail -f ~/.hermes/logs/gateway.log
+```
+
+### Manual (any platform)
+
+```bash
+# Run in foreground (for testing/debugging)
+./scripts/hermes-gateway run
+
+# Or via CLI (also foreground)
+python cli.py --gateway
+```
+
+## Storage Locations
+
+| Path | Purpose |
+|------|---------|
+| `~/.hermes/gateway.json` | Gateway configuration |
+| `~/.hermes/sessions/sessions.json` | Session index |
+| `~/.hermes/sessions/{id}.jsonl` | Conversation transcripts |
+| `~/.hermes/cron/output/` | Cron job outputs |
+| `~/.hermes/logs/gateway.log` | Gateway logs (macOS launchd) |

docs/tools.md

@@ -0,0 +1,159 @@
+# Tools
+
+Tools are functions that extend the agent's capabilities. Each tool is defined with an OpenAI-compatible JSON schema and an async handler function.
+
+## Tool Structure
+
+Each tool module in `tools/` exports:
+1. **Schema definitions** - OpenAI function-calling format
+2. **Handler functions** - Async functions that execute the tool
+
+```python
+# Example: tools/web_tools.py
+
+# Schema definition
+WEB_SEARCH_SCHEMA = {
+    "type": "function",
+    "function": {
+        "name": "web_search",
+        "description": "Search the web for information",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "query": {"type": "string", "description": "Search query"}
+            },
+            "required": ["query"]
+        }
+    }
+}
+
+# Handler function
+async def web_search(query: str) -> dict:
+    """Execute web search and return results."""
+    # Implementation...
+    return {"results": [...]}
+```
+
+## Tool Categories
+
+| Category | Module | Tools |
+|----------|--------|-------|
+| **Web** | `web_tools.py` | `web_search`, `web_extract`, `web_crawl` |
+| **Terminal** | `terminal_tool.py` | `terminal` (local/docker/singularity/modal/ssh backends) |
+| **Browser** | `browser_tool.py` | `browser_navigate`, `browser_click`, `browser_type`, etc. |
+| **Vision** | `vision_tools.py` | `vision_analyze` |
+| **Image Gen** | `image_generation_tool.py` | `image_generate` |
+| **Reasoning** | `mixture_of_agents_tool.py` | `mixture_of_agents` |
+| **Skills** | `skills_tool.py` | `skills_categories`, `skills_list`, `skill_view` |
+
+## Tool Registration
+
+Tools are registered in `model_tools.py`:
+
+```python
+# model_tools.py
+TOOL_SCHEMAS = [
+    *WEB_TOOL_SCHEMAS,
+    *TERMINAL_TOOL_SCHEMAS,
+    *BROWSER_TOOL_SCHEMAS,
+    # ...
+]
+
+TOOL_HANDLERS = {
+    "web_search": web_search,
+    "terminal": terminal_tool,
+    "browser_navigate": browser_navigate,
+    # ...
+}
+```
+
+## Toolsets
+
+Tools are grouped into **toolsets** for logical organization (see `toolsets.py`):
+
+```python
+TOOLSETS = {
+    "web": {
+        "description": "Web search and content extraction",
+        "tools": ["web_search", "web_extract", "web_crawl"]
+    },
+    "terminal": {
+        "description": "Command execution",
+        "tools": ["terminal"]
+    },
+    # ...
+}
+```
+
+## Adding a New Tool
+
+1. Create handler function in `tools/your_tool.py`
+2. Define JSON schema following OpenAI format
+3. Register in `model_tools.py` (schemas and handlers)
+4. Add to appropriate toolset in `toolsets.py`
+5. Update `tools/__init__.py` exports
+
+## Stateful Tools
+
+Some tools maintain state across calls within a session:
+
+- **Terminal**: Keeps container/sandbox running between commands
+- **Browser**: Maintains browser session for multi-step navigation
+
+State is managed per `task_id` and cleaned up automatically.
+
+## Terminal Backends
+
+The terminal tool supports multiple execution backends:
+
+| Backend | Description | Use Case |
+|---------|-------------|----------|
+| `local` | Direct execution on host | Development, simple tasks |
+| `ssh` | Remote execution via SSH | Sandboxing (agent can't modify its own code) |
+| `docker` | Docker container | Isolation, reproducibility |
+| `singularity` | Singularity/Apptainer | HPC clusters, rootless containers |
+| `modal` | Modal cloud | Scalable cloud compute, GPUs |
+
+Configure via environment variables or `cli-config.yaml`:
+
+```yaml
+# SSH backend example (in cli-config.yaml)
+terminal:
+  env_type: "ssh"
+  ssh_host: "my-server.example.com"
+  ssh_user: "myuser"
+  ssh_key: "~/.ssh/id_rsa"
+  cwd: "/home/myuser/project"
+```
+
+The SSH backend uses ControlMaster for connection persistence, making subsequent commands fast.
+
+## Skills Tools (Progressive Disclosure)
+
+Skills are on-demand knowledge documents. They use **progressive disclosure** to minimize tokens:
+
+```
+Level 0: skills_categories()     → ["mlops", "devops"]           (~50 tokens)
+Level 1: skills_list(category)   → [{name, description}, ...]   (~3k tokens)
+Level 2: skill_view(name)        → Full content + metadata       (varies)
+Level 3: skill_view(name, path)  → Specific reference file       (varies)
+```
+
+Skill directory structure:
+```
+skills/
+└── mlops/
+    └── axolotl/
+        ├── SKILL.md           # Main instructions (required)
+        ├── references/        # Additional docs
+        └── templates/         # Output formats, configs
+```
+
+SKILL.md uses YAML frontmatter:
+```yaml
+---
+name: axolotl
+description: Fine-tuning LLMs with Axolotl
+tags: [Fine-Tuning, LoRA, DPO]
+---
+```

environments/__init__.py

@@ -0,0 +1,28 @@
+"""
+Hermes-Agent Atropos Environments
+
+Provides a layered integration between hermes-agent's tool-calling capabilities
+and the Atropos RL training framework.
+
+Layers:
+    - agent_loop: Reusable multi-turn agent loop with standard OpenAI-spec tool calling
+    - tool_context: Per-rollout tool access handle for reward/verification functions
+    - hermes_base_env: Abstract base environment (BaseEnv subclass) for Atropos
+    - tool_call_parsers: Client-side tool call parser registry for Phase 2 (VLLM /generate)
+
+Concrete environments:
+    - terminal_test_env: Simple file-creation tasks for testing the stack
+    - hermes_swe_env: SWE-bench style tasks with Modal sandboxes
+"""
+
+from environments.agent_loop import AgentResult, HermesAgentLoop
+from environments.tool_context import ToolContext
+from environments.hermes_base_env import HermesAgentBaseEnv, HermesAgentEnvConfig
+
+__all__ = [
+    "AgentResult",
+    "HermesAgentLoop",
+    "ToolContext",
+    "HermesAgentBaseEnv",
+    "HermesAgentEnvConfig",
+]

environments/agent_loop.py

@@ -0,0 +1,372 @@
+"""
+HermesAgentLoop -- Reusable Multi-Turn Agent Engine
+
+Runs the hermes-agent tool-calling loop using standard OpenAI-spec tool calling.
+Works with any server that returns ChatCompletion objects with tool_calls:
+    - Phase 1: OpenAI server type (VLLM, SGLang, OpenRouter, OpenAI API)
+    - Phase 2: ManagedServer with client-side tool call parser
+
+The loop passes tools= and checks response.choices[0].message.tool_calls,
+identical to hermes-agent's run_agent.py. Tool execution is dispatched via
+handle_function_call() from model_tools.py.
+"""
+
+import asyncio
+import concurrent.futures
+import json
+import logging
+import uuid
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional, Set
+
+from model_tools import handle_function_call
+
+# Thread pool for running sync tool calls that internally use asyncio.run()
+# (e.g., mini-swe-agent's modal/docker backends). Running them in a separate
+# thread gives them a clean event loop so they don't deadlock inside Atropos's loop.
+_tool_executor = concurrent.futures.ThreadPoolExecutor(max_workers=8)
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class ToolError:
+    """Record of a tool execution error during the agent loop."""
+
+    turn: int                  # Which turn the error occurred on
+    tool_name: str             # Which tool was called
+    arguments: str             # The arguments passed (truncated)
+    error: str                 # The error message
+    tool_result: str           # The raw result returned to the model
+
+
+@dataclass
+class AgentResult:
+    """Result of running the agent loop."""
+
+    # Full conversation history in OpenAI message format
+    messages: List[Dict[str, Any]]
+    # ManagedServer.get_state() if available (Phase 2), None otherwise
+    managed_state: Optional[Dict[str, Any]] = None
+    # How many LLM calls were made
+    turns_used: int = 0
+    # True if model stopped calling tools naturally (vs hitting max_turns)
+    finished_naturally: bool = False
+    # Extracted reasoning content per turn (from PR #297 helpers)
+    reasoning_per_turn: List[Optional[str]] = field(default_factory=list)
+    # Tool errors encountered during the loop
+    tool_errors: List[ToolError] = field(default_factory=list)
+
+
+def _extract_reasoning_from_message(message) -> Optional[str]:
+    """
+    Extract reasoning content from a ChatCompletion message.
+
+    Handles multiple provider formats:
+    1. message.reasoning_content field (some providers)
+    2. message.reasoning field (some providers)
+    3. message.reasoning_details[].text (OpenRouter style)
+
+    Note: <think> block extraction from content is NOT done here -- that's
+    handled by the response already in Phase 1 (server does it) or by
+    ManagedServer's patch in Phase 2.
+
+    Args:
+        message: The assistant message from ChatCompletion response
+
+    Returns:
+        Extracted reasoning text, or None if not found
+    """
+    # Check reasoning_content field (common across providers)
+    if hasattr(message, "reasoning_content") and message.reasoning_content:
+        return message.reasoning_content
+
+    # Check reasoning field
+    if hasattr(message, "reasoning") and message.reasoning:
+        return message.reasoning
+
+    # Check reasoning_details (OpenRouter style)
+    if hasattr(message, "reasoning_details") and message.reasoning_details:
+        for detail in message.reasoning_details:
+            if hasattr(detail, "text") and detail.text:
+                return detail.text
+            if isinstance(detail, dict) and detail.get("text"):
+                return detail["text"]
+
+    return None
+
+
+class HermesAgentLoop:
+    """
+    Runs hermes-agent's tool-calling loop using standard OpenAI-spec tool calling.
+
+    Same pattern as run_agent.py:
+    - Pass tools= to the API
+    - Check response.choices[0].message.tool_calls
+    - Dispatch via handle_function_call()
+
+    Works identically with any server type -- OpenAI, VLLM, SGLang, OpenRouter,
+    or ManagedServer with a parser. The server determines how tool_calls get
+    populated on the response.
+    """
+
+    def __init__(
+        self,
+        server,
+        tool_schemas: List[Dict[str, Any]],
+        valid_tool_names: Set[str],
+        max_turns: int = 30,
+        task_id: Optional[str] = None,
+        temperature: float = 1.0,
+        max_tokens: Optional[int] = None,
+    ):
+        """
+        Initialize the agent loop.
+
+        Args:
+            server: Server object with chat_completion() method (OpenAIServer,
+                    ManagedServer, ServerManager, etc.)
+            tool_schemas: OpenAI-format tool definitions from get_tool_definitions()
+            valid_tool_names: Set of tool names the model is allowed to call
+            max_turns: Maximum number of LLM calls before stopping
+            task_id: Unique ID for terminal/browser session isolation
+            temperature: Sampling temperature for generation
+            max_tokens: Max tokens per generation (None for server default)
+        """
+        self.server = server
+        self.tool_schemas = tool_schemas
+        self.valid_tool_names = valid_tool_names
+        self.max_turns = max_turns
+        self.task_id = task_id or str(uuid.uuid4())
+        self.temperature = temperature
+        self.max_tokens = max_tokens
+
+    async def run(self, messages: List[Dict[str, Any]]) -> AgentResult:
+        """
+        Execute the full agent loop using standard OpenAI tool calling.
+
+        Args:
+            messages: Initial conversation messages (system + user).
+                      Modified in-place as the conversation progresses.
+
+        Returns:
+            AgentResult with full conversation history, managed state, and metadata
+        """
+        reasoning_per_turn = []
+        tool_errors: List[ToolError] = []
+
+        for turn in range(self.max_turns):
+            # Build the chat_completion kwargs
+            chat_kwargs = {
+                "messages": messages,
+                "n": 1,
+                "temperature": self.temperature,
+            }
+
+            # Only pass tools if we have them
+            if self.tool_schemas:
+                chat_kwargs["tools"] = self.tool_schemas
+
+            # Only pass max_tokens if explicitly set
+            if self.max_tokens is not None:
+                chat_kwargs["max_tokens"] = self.max_tokens
+
+            # Make the API call -- standard OpenAI spec
+            try:
+                response = await self.server.chat_completion(**chat_kwargs)
+            except Exception as e:
+                logger.error("API call failed on turn %d: %s", turn + 1, e)
+                return AgentResult(
+                    messages=messages,
+                    managed_state=self._get_managed_state(),
+                    turns_used=turn + 1,
+                    finished_naturally=False,
+                    reasoning_per_turn=reasoning_per_turn,
+                    tool_errors=tool_errors,
+                )
+
+            if not response or not response.choices:
+                logger.warning("Empty response on turn %d", turn + 1)
+                return AgentResult(
+                    messages=messages,
+                    managed_state=self._get_managed_state(),
+                    turns_used=turn + 1,
+                    finished_naturally=False,
+                    reasoning_per_turn=reasoning_per_turn,
+                    tool_errors=tool_errors,
+                )
+
+            assistant_msg = response.choices[0].message
+
+            # Extract reasoning content from the response (all provider formats)
+            reasoning = _extract_reasoning_from_message(assistant_msg)
+            reasoning_per_turn.append(reasoning)
+
+            # Check for tool calls -- standard OpenAI spec
+            if assistant_msg.tool_calls:
+                # Build the assistant message dict for conversation history
+                msg_dict: Dict[str, Any] = {
+                    "role": "assistant",
+                    "content": assistant_msg.content or "",
+                    "tool_calls": [
+                        {
+                            "id": tc.id,
+                            "type": "function",
+                            "function": {
+                                "name": tc.function.name,
+                                "arguments": tc.function.arguments,
+                            },
+                        }
+                        for tc in assistant_msg.tool_calls
+                    ],
+                }
+
+                # Preserve reasoning_content for multi-turn chat template handling
+                # (e.g., Kimi-K2's template renders <think> blocks differently
+                # for history vs. the latest turn based on this field)
+                if reasoning:
+                    msg_dict["reasoning_content"] = reasoning
+
+                messages.append(msg_dict)
+
+                # Execute each tool call via hermes-agent's dispatch
+                for tc in assistant_msg.tool_calls:
+                    tool_name = tc.function.name
+                    tool_args_raw = tc.function.arguments
+
+                    # Validate tool name
+                    if tool_name not in self.valid_tool_names:
+                        tool_result = json.dumps(
+                            {
+                                "error": f"Unknown tool '{tool_name}'. "
+                                f"Available tools: {sorted(self.valid_tool_names)}"
+                            }
+                        )
+                        tool_errors.append(ToolError(
+                            turn=turn + 1, tool_name=tool_name,
+                            arguments=tool_args_raw[:200],
+                            error=f"Unknown tool '{tool_name}'",
+                            tool_result=tool_result,
+                        ))
+                        logger.warning(
+                            "Model called unknown tool '%s' on turn %d",
+                            tool_name, turn + 1,
+                        )
+                    else:
+                        # Parse arguments and dispatch
+                        try:
+                            args = json.loads(tool_args_raw)
+                        except json.JSONDecodeError:
+                            args = {}
+                            logger.warning(
+                                "Invalid JSON in tool call arguments for '%s': %s",
+                                tool_name, tool_args_raw[:200],
+                            )
+
+                        try:
+                            if tool_name == "terminal":
+                                import os
+                                backend = os.getenv("TERMINAL_ENV", "local")
+                                cmd_preview = args.get("command", "")[:80]
+                                print(f"  🖥️  [{backend}] $ {cmd_preview}")
+
+                            # Run tool calls in a thread pool so backends that use
+                            # asyncio.run() internally (modal, docker) get a clean
+                            # event loop instead of deadlocking inside Atropos's loop.
+                            loop = asyncio.get_event_loop()
+                            tool_result = await loop.run_in_executor(
+                                _tool_executor,
+                                lambda: handle_function_call(
+                                    tool_name, args, task_id=self.task_id
+                                ),
+                            )
+                        except Exception as e:
+                            tool_result = json.dumps(
+                                {"error": f"Tool execution failed: {type(e).__name__}: {str(e)}"}
+                            )
+                            tool_errors.append(ToolError(
+                                turn=turn + 1, tool_name=tool_name,
+                                arguments=tool_args_raw[:200],
+                                error=f"{type(e).__name__}: {str(e)}",
+                                tool_result=tool_result,
+                            ))
+                            logger.error(
+                                "Tool '%s' execution failed on turn %d: %s",
+                                tool_name, turn + 1, e,
+                            )
+
+                        # Also check if the tool returned an error in its JSON result
+                        try:
+                            result_data = json.loads(tool_result)
+                            if isinstance(result_data, dict):
+                                err = result_data.get("error")
+                                exit_code = result_data.get("exit_code")
+                                if err and exit_code and exit_code < 0:
+                                    tool_errors.append(ToolError(
+                                        turn=turn + 1, tool_name=tool_name,
+                                        arguments=tool_args_raw[:200],
+                                        error=str(err),
+                                        tool_result=tool_result[:500],
+                                    ))
+                        except (json.JSONDecodeError, TypeError):
+                            pass
+
+                    # Add tool response to conversation
+                    messages.append(
+                        {
+                            "role": "tool",
+                            "tool_call_id": tc.id,
+                            "content": tool_result,
+                        }
+                    )
+
+                logger.debug(
+                    "Turn %d: %d tool calls executed",
+                    turn + 1,
+                    len(assistant_msg.tool_calls),
+                )
+
+            else:
+                # No tool calls -- model is done
+                msg_dict = {
+                    "role": "assistant",
+                    "content": assistant_msg.content or "",
+                }
+                if reasoning:
+                    msg_dict["reasoning_content"] = reasoning
+                messages.append(msg_dict)
+
+                logger.debug(
+                    "Turn %d: model finished naturally (no tool calls)", turn + 1
+                )
+
+                return AgentResult(
+                    messages=messages,
+                    managed_state=self._get_managed_state(),
+                    turns_used=turn + 1,
+                    finished_naturally=True,
+                    reasoning_per_turn=reasoning_per_turn,
+                    tool_errors=tool_errors,
+                )
+
+        # Hit max turns without the model stopping
+        logger.info("Agent hit max_turns (%d) without finishing", self.max_turns)
+        return AgentResult(
+            messages=messages,
+            managed_state=self._get_managed_state(),
+            turns_used=self.max_turns,
+            finished_naturally=False,
+            reasoning_per_turn=reasoning_per_turn,
+            tool_errors=tool_errors,
+        )
+
+    def _get_managed_state(self) -> Optional[Dict[str, Any]]:
+        """
+        Get ManagedServer state if the server supports it.
+
+        Returns state dict with SequenceNodes containing tokens/logprobs/masks,
+        or None if the server doesn't support get_state() (e.g., regular OpenAI server).
+        """
+        if hasattr(self.server, "get_state"):
+            return self.server.get_state()
+        return None

environments/configs/swe_default.yaml

@@ -0,0 +1,33 @@
+# SWE Environment -- Default Configuration
+#
+# SWE-bench style tasks with Modal sandboxes for cloud isolation.
+# Uses terminal + file + web toolsets.
+#
+# Usage:
+#   python environments/hermes_swe_env.py serve --config environments/configs/swe_default.yaml
+
+env:
+  enabled_toolsets: ["terminal", "file", "web"]
+  max_agent_turns: 30
+  max_token_length: 4096
+  group_size: 4
+  terminal_backend: "modal"
+  tool_call_parser: "hermes"
+  tokenizer_name: "NousResearch/DeepHermes-3-Llama-3-3B-Preview"
+  dataset_name: "bigcode/humanevalpack"
+  dataset_split: "test"
+  prompt_field: "prompt"
+  steps_per_eval: 50
+  total_steps: 500
+  use_wandb: true
+  wandb_name: "hermes-swe"
+  system_prompt: >
+    You are a skilled software engineer. You have access to a terminal,
+    file tools, and web search. Use these tools to complete the coding task.
+    Write clean, working code and verify it runs correctly before finishing.
+
+openai:
+  base_url: "http://localhost:8000/v1"
+  model_name: "NousResearch/DeepHermes-3-Llama-3-3B-Preview"
+  server_type: "openai"
+  api_key: ""

environments/configs/terminal_test_default.yaml

@@ -0,0 +1,35 @@
+# Terminal Test Environment -- Default Configuration
+#
+# Simple file-creation tasks for validating the full Atropos + hermes-agent stack.
+# Uses Modal terminal backend and OpenRouter (Claude) for inference.
+# API keys loaded from ~/hermes-agent/.env
+#
+# Usage:
+#   run-api
+#   python environments/terminal_test_env.py serve
+#   # Or with config file:
+#   python environments/terminal_test_env.py serve --config environments/configs/terminal_test_default.yaml
+
+env:
+  enabled_toolsets: ["terminal", "file"]
+  max_agent_turns: 10
+  max_token_length: 2048
+  group_size: 3
+  total_steps: 3
+  steps_per_eval: 3
+  terminal_backend: "modal"
+  tool_call_parser: "hermes"
+  tokenizer_name: "NousResearch/DeepHermes-3-Llama-3-3B-Preview"
+  ensure_scores_are_not_same: false
+  use_wandb: false
+  system_prompt: >
+    You are a helpful assistant with access to a terminal and file tools.
+    Complete the user's request by using the available tools.
+    Be precise and follow instructions exactly.
+
+openai:
+  base_url: "https://openrouter.ai/api/v1"
+  model_name: "anthropic/claude-opus-4.6"
+  server_type: "openai"
+  health_check: false
+  # api_key loaded from OPENROUTER_API_KEY in .env

environments/hermes_base_env.py

@@ -0,0 +1,615 @@
+"""
+HermesAgentBaseEnv -- Abstract Base Environment for Hermes-Agent + Atropos
+
+Provides the Atropos integration plumbing that all hermes-agent environments share:
+- Two-mode operation (OpenAI server for Phase 1, VLLM ManagedServer for Phase 2)
+- Per-group toolset/distribution resolution
+- Agent loop orchestration via HermesAgentLoop
+- ToolContext creation for reward functions
+- ScoredDataGroup construction from ManagedServer state
+
+Subclasses only need to implement:
+    setup()           -- Load dataset, initialize state
+    get_next_item()   -- Return the next item from the dataset
+    format_prompt()   -- Convert a dataset item into the user message
+    compute_reward()  -- Score the rollout (has full ToolContext access)
+    evaluate()        -- Periodic evaluation
+"""
+
+import asyncio
+import json
+import logging
+import os
+import sys
+import uuid
+from abc import abstractmethod
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Set, Tuple, Union
+
+# Ensure the hermes-agent repo root is on sys.path so that imports like
+# `from model_tools import ...` and `from environments.X import ...` work
+# regardless of where the script is invoked from.
+_repo_root = Path(__file__).resolve().parent.parent
+if str(_repo_root) not in sys.path:
+    sys.path.insert(0, str(_repo_root))
+
+from dotenv import load_dotenv
+from pydantic import Field
+
+# Load API keys from hermes-agent/.env so all environments can access them
+_env_path = _repo_root / ".env"
+if _env_path.exists():
+    load_dotenv(dotenv_path=_env_path)
+
+# Apply monkey patches for async-safe tool operation inside Atropos's event loop.
+# This patches SwerexModalEnvironment to use a background thread instead of
+# asyncio.run(), which would deadlock inside Atropos. Safe for normal CLI too.
+from environments.patches import apply_patches
+apply_patches()
+
+from atroposlib.envs.base import (
+    BaseEnv,
+    BaseEnvConfig,
+    ScoredDataGroup,
+    ScoredDataItem,
+)
+from atroposlib.envs.server_handling.server_manager import (
+    APIServerConfig,
+    ServerBaseline,
+    ServerManager,
+)
+from atroposlib.type_definitions import Item
+
+from environments.agent_loop import AgentResult, HermesAgentLoop
+from environments.tool_context import ToolContext
+
+# Import hermes-agent toolset infrastructure
+from model_tools import get_tool_definitions
+from toolset_distributions import sample_toolsets_from_distribution
+
+logger = logging.getLogger(__name__)
+
+
+class HermesAgentEnvConfig(BaseEnvConfig):
+    """
+    Configuration for hermes-agent Atropos environments.
+
+    Extends BaseEnvConfig with agent-specific settings for toolsets,
+    terminal backend, dataset loading, and tool call parsing.
+    """
+
+    # --- Toolset configuration ---
+    # Mutually exclusive: use either enabled_toolsets OR distribution
+    enabled_toolsets: Optional[List[str]] = Field(
+        default=None,
+        description="Explicit list of hermes toolsets to enable (e.g., ['terminal', 'file', 'web']). "
+        "If None and distribution is also None, all available toolsets are enabled.",
+    )
+    disabled_toolsets: Optional[List[str]] = Field(
+        default=None,
+        description="Toolsets to disable. Applied as a filter on top of enabled_toolsets or distribution.",
+    )
+    distribution: Optional[str] = Field(
+        default=None,
+        description="Name of a toolset distribution from toolset_distributions.py "
+        "(e.g., 'development', 'terminal_tasks'). Sampled once per group. "
+        "Mutually exclusive with enabled_toolsets.",
+    )
+
+    # --- Agent loop configuration ---
+    max_agent_turns: int = Field(
+        default=30,
+        description="Maximum number of LLM calls (tool-calling iterations) per rollout.",
+    )
+    system_prompt: Optional[str] = Field(
+        default=None,
+        description="System prompt for the agent. Tools are handled via the tools= parameter, "
+        "not embedded in the prompt text.",
+    )
+    agent_temperature: float = Field(
+        default=1.0,
+        description="Sampling temperature for agent generation during rollouts.",
+    )
+
+    # --- Terminal backend ---
+    terminal_backend: str = Field(
+        default="local",
+        description="Terminal backend: 'local', 'docker', 'modal', 'ssh', 'singularity'. "
+        "Modal recommended for production RL (cloud isolation per rollout).",
+    )
+
+    # --- Dataset ---
+    dataset_name: Optional[str] = Field(
+        default=None,
+        description="HuggingFace dataset name. Optional if tasks are defined inline.",
+    )
+    dataset_split: str = Field(
+        default="train",
+        description="Dataset split to use.",
+    )
+    prompt_field: str = Field(
+        default="prompt",
+        description="Which field in the dataset contains the prompt.",
+    )
+
+    # --- Phase 2: Tool call parsing ---
+    tool_call_parser: str = Field(
+        default="hermes",
+        description="Tool call parser name for Phase 2 (VLLM server type). "
+        "Ignored in Phase 1 (OpenAI server type where VLLM parses natively). "
+        "Options: hermes, mistral, llama3_json, qwen, deepseek_v3, etc.",
+    )
+
+
+class HermesAgentBaseEnv(BaseEnv):
+    """
+    Abstract base environment for hermes-agent Atropos integration.
+
+    Handles two modes of operation:
+    - Phase 1 (OpenAI server type): Uses server.chat_completion() directly.
+      The server (VLLM, SGLang, OpenRouter, OpenAI) handles tool call parsing
+      and reasoning extraction natively. DummyManagedServer provides placeholder
+      tokens. Good for SFT data gen, verifier testing, evaluation.
+
+    - Phase 2 (VLLM server type): Uses ManagedServer for exact token IDs + logprobs
+      via /generate. Client-side tool call parser reconstructs structured tool_calls
+      from raw output. Full RL training capability.
+
+    Subclasses must implement:
+        setup()           -- Load dataset, initialize state
+        get_next_item()   -- Return the next item to roll out
+        format_prompt()   -- Convert a dataset item into the user message string
+        compute_reward()  -- Score the rollout using ToolContext
+        evaluate()        -- Periodic evaluation
+    """
+
+    name: Optional[str] = "hermes-agent"
+    env_config_cls = HermesAgentEnvConfig
+
+    def __init__(
+        self,
+        config: HermesAgentEnvConfig,
+        server_configs: Union[ServerBaseline, List[APIServerConfig]],
+        slurm=False,
+        testing=False,
+    ):
+        super().__init__(config, server_configs, slurm, testing)
+
+        # Set terminal backend environment variable so hermes tools pick it up
+        if config.terminal_backend:
+            os.environ["TERMINAL_ENV"] = config.terminal_backend
+            print(f"🖥️  Terminal backend: {config.terminal_backend}")
+
+        # Current group's resolved tools (set in collect_trajectories)
+        self._current_group_tools: Optional[Tuple[List[Dict], Set[str]]] = None
+
+        # Tool error tracking for wandb logging
+        self._tool_error_buffer: List[Dict[str, Any]] = []
+
+    # =========================================================================
+    # Toolset resolution (per-group)
+    # =========================================================================
+
+    def _resolve_tools_for_group(self) -> Tuple[List[Dict[str, Any]], Set[str]]:
+        """
+        Resolve toolsets for a group. Called once in collect_trajectories(),
+        then shared by all collect_trajectory() calls in the group.
+
+        If distribution is set, samples probabilistically.
+        If enabled_toolsets is set, uses that explicit list.
+        disabled_toolsets is applied as a filter on top.
+
+        Returns:
+            (tool_schemas, valid_tool_names) tuple
+        """
+        config = self.config
+
+        if config.distribution:
+            group_toolsets = sample_toolsets_from_distribution(config.distribution)
+            logger.info("Sampled toolsets from '%s': %s", config.distribution, group_toolsets)
+        else:
+            group_toolsets = config.enabled_toolsets  # None means "all available"
+
+        tools = get_tool_definitions(
+            enabled_toolsets=group_toolsets,
+            disabled_toolsets=config.disabled_toolsets,
+            quiet_mode=True,
+        )
+
+        valid_names = {t["function"]["name"] for t in tools} if tools else set()
+        logger.info("Resolved %d tools for group: %s", len(valid_names), sorted(valid_names))
+        return tools, valid_names
+
+    # =========================================================================
+    # Server mode detection
+    # =========================================================================
+
+    def _use_managed_server(self) -> bool:
+        """
+        Determine if we should use ManagedServer (Phase 2) or direct server (Phase 1).
+
+        Phase 2 (ManagedServer) is used when the server type is 'vllm' or 'sglang',
+        which go through the /generate endpoint for exact token tracking.
+
+        Phase 1 (direct server) is used for 'openai' server type, which uses
+        /v1/chat/completions with native tool call parsing.
+        """
+        if not self.server.servers:
+            return False
+
+        server = self.server.servers[0]
+        # If the server is an OpenAI server (not VLLM/SGLang), use direct mode
+        from atroposlib.envs.server_handling.openai_server import OpenAIServer
+        return not isinstance(server, OpenAIServer)
+
+    # =========================================================================
+    # Core Atropos integration
+    # =========================================================================
+
+    async def collect_trajectories(
+        self, item: Item
+    ) -> Tuple[
+        Union[Optional[ScoredDataGroup], List[Optional[ScoredDataGroup]]],
+        List[Item],
+    ]:
+        """
+        Override collect_trajectories to resolve toolsets once per group,
+        then delegate to the standard group-level collection.
+
+        The default BaseEnv.collect_trajectories() calls collect_trajectory()
+        group_size times in parallel. We resolve tools once here and store
+        them for all those calls to use.
+        """
+        # Resolve toolsets for this group (shared by all rollouts in the group)
+        self._current_group_tools = self._resolve_tools_for_group()
+
+        # Delegate to the default implementation which calls collect_trajectory()
+        # group_size times via asyncio.gather
+        return await super().collect_trajectories(item)
+
+    # =========================================================================
+    # Wandb rollout display -- format trajectories nicely
+    # =========================================================================
+
+    @staticmethod
+    def _format_trajectory_for_display(messages: List[Dict[str, Any]]) -> str:
+        """
+        Format a conversation's messages into a readable trajectory string
+        for wandb rollout tables. Shows tool calls, tool results, and reasoning
+        in a structured way instead of raw token decoding.
+        """
+        parts = []
+        for msg in messages:
+            role = msg.get("role", "unknown")
+            content = msg.get("content", "")
+
+            if role == "system":
+                parts.append(f"[SYSTEM]\n{content}")
+
+            elif role == "user":
+                parts.append(f"[USER]\n{content}")
+
+            elif role == "assistant":
+                # Show reasoning if present
+                reasoning = msg.get("reasoning_content", "")
+                if reasoning:
+                    # Truncate long reasoning for display
+                    if len(reasoning) > 300:
+                        reasoning = reasoning[:300] + "..."
+                    parts.append(f"[ASSISTANT thinking]\n{reasoning}")
+
+                # Show content
+                if content:
+                    parts.append(f"[ASSISTANT]\n{content}")
+
+                # Show tool calls
+                tool_calls = msg.get("tool_calls", [])
+                for tc in tool_calls:
+                    func = tc.get("function", {})
+                    name = func.get("name", "?")
+                    args = func.get("arguments", "{}")
+                    # Truncate long arguments for display
+                    if len(args) > 200:
+                        args = args[:200] + "..."
+                    parts.append(f"[TOOL CALL] {name}({args})")
+
+            elif role == "tool":
+                tool_id = msg.get("tool_call_id", "")
+                result = content
+                # Truncate long tool results for display
+                if len(result) > 500:
+                    result = result[:500] + "..."
+                parts.append(f"[TOOL RESULT] {result}")
+
+        return "\n\n".join(parts)
+
+    async def add_rollouts_for_wandb(
+        self,
+        scored_data,
+        item=None,
+    ):
+        """
+        Override to show formatted trajectories with tool calls visible,
+        instead of raw token decoding which loses all structure.
+        """
+        num_keep = self.config.num_rollouts_per_group_for_logging
+        if num_keep == -1:
+            num_keep = self.config.group_size
+
+        group = []
+        for i in range(min(num_keep, len(scored_data.get("scores", [])))):
+            score = scored_data["scores"][i]
+
+            # Use messages if available for rich display
+            messages = None
+            if scored_data.get("messages") and i < len(scored_data["messages"]):
+                messages = scored_data["messages"][i]
+
+            if messages:
+                text = self._format_trajectory_for_display(messages)
+            elif scored_data.get("tokens") and i < len(scored_data["tokens"]):
+                text = self.tokenizer.decode(scored_data["tokens"][i])
+            else:
+                text = "(no data)"
+
+            group.append((text, score))
+
+        self.rollouts_for_wandb.append(group)
+        if len(self.rollouts_for_wandb) > self.config.num_rollouts_to_keep:
+            self.rollouts_for_wandb.pop(0)
+
+    async def wandb_log(self, wandb_metrics: Optional[Dict] = None):
+        """Log base metrics including tool errors to wandb."""
+        if wandb_metrics is None:
+            wandb_metrics = {}
+
+        # Log tool error stats
+        if self._tool_error_buffer:
+            wandb_metrics["train/tool_errors_count"] = len(self._tool_error_buffer)
+
+            # Log error details as a summary string (tables can crash wandb on tmp cleanup)
+            error_summaries = []
+            for err in self._tool_error_buffer:
+                error_summaries.append(
+                    f"[turn {err['turn']}] {err['tool']}({err['args'][:80]}) -> {err['error'][:150]}"
+                )
+            wandb_metrics["train/tool_error_details"] = "\n".join(error_summaries)
+
+            # Also print to stdout for immediate visibility
+            for summary in error_summaries:
+                print(f"  Tool Error: {summary}")
+
+            self._tool_error_buffer = []
+        else:
+            wandb_metrics["train/tool_errors_count"] = 0
+
+        await super().wandb_log(wandb_metrics)
+
+    async def collect_trajectory(
+        self, item: Item
+    ) -> Tuple[Optional[Union[ScoredDataItem, Any]], List[Item]]:
+        """
+        Run a single rollout: agent loop + reward computation.
+
+        This is called group_size times in parallel by collect_trajectories().
+        Each call gets its own task_id for terminal/browser session isolation.
+        """
+        task_id = str(uuid.uuid4())
+
+        # Get group-level tools (resolved once in collect_trajectories)
+        if self._current_group_tools is None:
+            # Fallback: resolve per-trajectory if called outside collect_trajectories
+            tools, valid_names = self._resolve_tools_for_group()
+        else:
+            tools, valid_names = self._current_group_tools
+
+        # Build initial messages
+        messages: List[Dict[str, Any]] = []
+        if self.config.system_prompt:
+            messages.append({"role": "system", "content": self.config.system_prompt})
+        messages.append({"role": "user", "content": self.format_prompt(item)})
+
+        # Run the agent loop
+        result: AgentResult
+        if self._use_managed_server():
+            # Phase 2: ManagedServer with parser -- exact tokens + logprobs
+            # Load the tool call parser from registry based on config
+            from environments.tool_call_parsers import get_parser
+            try:
+                tc_parser = get_parser(self.config.tool_call_parser)
+            except KeyError:
+                logger.warning(
+                    "Tool call parser '%s' not found, falling back to 'hermes'",
+                    self.config.tool_call_parser,
+                )
+                tc_parser = get_parser("hermes")
+
+            try:
+                async with self.server.managed_server(
+                    tokenizer=self.tokenizer,
+                    tool_call_parser=tc_parser,
+                ) as managed:
+                    agent = HermesAgentLoop(
+                        server=managed,
+                        tool_schemas=tools,
+                        valid_tool_names=valid_names,
+                        max_turns=self.config.max_agent_turns,
+                        task_id=task_id,
+                        temperature=self.config.agent_temperature,
+                        max_tokens=self.config.max_token_length,
+                    )
+                    result = await agent.run(messages)
+            except NotImplementedError:
+                # DummyManagedServer not allowed -- fall back to Phase 1
+                logger.warning(
+                    "ManagedServer not available (OpenAI server?). "
+                    "Falling back to direct server mode."
+                )
+                agent = HermesAgentLoop(
+                    server=self.server,
+                    tool_schemas=tools,
+                    valid_tool_names=valid_names,
+                    max_turns=self.config.max_agent_turns,
+                    task_id=task_id,
+                    temperature=self.config.agent_temperature,
+                    max_tokens=self.config.max_token_length,
+                )
+                result = await agent.run(messages)
+        else:
+            # Phase 1: OpenAI server -- native tool_calls, placeholder tokens
+            agent = HermesAgentLoop(
+                server=self.server,
+                tool_schemas=tools,
+                valid_tool_names=valid_names,
+                max_turns=self.config.max_agent_turns,
+                task_id=task_id,
+                temperature=self.config.agent_temperature,
+                max_tokens=self.config.max_token_length,
+            )
+            result = await agent.run(messages)
+
+        # Skip reward computation if the agent loop produced no meaningful work
+        # (e.g., API call failed on turn 1). No point spinning up a Modal sandbox
+        # just to verify files that were never created.
+        only_system_and_user = all(
+            msg.get("role") in ("system", "user") for msg in result.messages
+        )
+        if result.turns_used == 0 or only_system_and_user:
+            logger.warning(
+                "Agent loop produced no output (turns=%d, msgs=%d). Skipping reward.",
+                result.turns_used, len(result.messages),
+            )
+            reward = 0.0
+        else:
+            # Compute reward using ToolContext (gives verifier full tool access)
+            ctx = ToolContext(task_id)
+            try:
+                reward = await self.compute_reward(item, result, ctx)
+            except Exception as e:
+                logger.error("compute_reward failed: %s", e)
+                reward = 0.0
+            finally:
+                ctx.cleanup()
+
+        # Track tool errors for wandb logging
+        if result.tool_errors:
+            for err in result.tool_errors:
+                self._tool_error_buffer.append({
+                    "turn": err.turn,
+                    "tool": err.tool_name,
+                    "args": err.arguments[:150],
+                    "error": err.error[:300],
+                    "result": err.tool_result[:300],
+                })
+
+        # Build ScoredDataItem from ManagedServer state
+        # Phase 2: real tokens/masks/logprobs from SequenceNodes
+        # Phase 1: placeholder tokens (still need a valid ScoredDataItem for the pipeline)
+        nodes = (result.managed_state or {}).get("nodes", [])
+
+        if nodes:
+            # Phase 2 (or DummyManagedServer): use actual node data
+            node = nodes[-1]  # Final sequence node = full trajectory
+            scored_item: Dict[str, Any] = {
+                "tokens": node.tokens,
+                "masks": node.masked_tokens,
+                "scores": reward,
+            }
+
+            # Include logprobs if available (Phase 2)
+            if hasattr(node, "logprobs") and node.logprobs:
+                scored_item["advantages"] = None  # Computed by trainer
+                scored_item["ref_logprobs"] = None
+        else:
+            # Phase 1 with no managed state: create placeholder tokens
+            # so the data pipeline doesn't break. These are NOT suitable
+            # for training but allow process mode (SFT data gen) to work.
+            # Tokenize the full conversation to get approximate tokens.
+            full_text = "\n".join(
+                msg.get("content", "") for msg in result.messages if msg.get("content")
+            )
+            if self.tokenizer:
+                tokens = self.tokenizer.encode(full_text, add_special_tokens=True)
+            else:
+                tokens = list(range(min(len(full_text) // 4, 128)))
+
+            scored_item = {
+                "tokens": tokens,
+                "masks": [-100] + tokens[1:],  # Mask first token as prompt
+                "scores": reward,
+            }
+
+        # Always include messages for wandb rollout display and data logging
+        scored_item["messages"] = result.messages
+
+        return scored_item, []
+
+    # =========================================================================
+    # Abstract methods -- subclasses must implement
+    # =========================================================================
+
+    @abstractmethod
+    async def setup(self):
+        """
+        Load dataset, initialize state.
+
+        Called once when the environment starts. Typical implementation:
+            self.dataset = load_dataset(self.config.dataset_name, split=self.config.dataset_split)
+            self.iter = 0
+        """
+        raise NotImplementedError
+
+    @abstractmethod
+    async def get_next_item(self) -> Item:
+        """
+        Return the next item from the dataset for rollout.
+
+        Called by the base env's main loop to get items for workers.
+        Should cycle through the dataset.
+        """
+        raise NotImplementedError
+
+    @abstractmethod
+    def format_prompt(self, item: Item) -> str:
+        """
+        Convert a dataset item into the user message for the agent.
+
+        Args:
+            item: Dataset item (dict, tuple, etc.)
+
+        Returns:
+            The prompt string to send to the agent
+        """
+        raise NotImplementedError
+
+    @abstractmethod
+    async def compute_reward(
+        self, item: Item, result: AgentResult, ctx: ToolContext
+    ) -> float:
+        """
+        Score the rollout. Has full access to:
+        - item: the original dataset item (ground truth, test commands, etc.)
+        - result: AgentResult with full messages, turn count, reasoning, etc.
+        - ctx: ToolContext -- call ANY hermes-agent tool (terminal, file, web,
+               browser, vision...) scoped to this rollout's sandbox. Nothing
+               is off-limits.
+
+        Args:
+            item: The dataset item that was rolled out
+            result: The agent's rollout result
+            ctx: ToolContext with full tool access for verification
+
+        Returns:
+            Reward float (typically 0.0 to 1.0, but any float is valid)
+        """
+        raise NotImplementedError
+
+    @abstractmethod
+    async def evaluate(self, *args, **kwargs):
+        """
+        Periodic evaluation. Called every steps_per_eval steps.
+
+        Typical implementation runs the agent on a held-out eval set
+        and logs metrics via wandb/evaluate_log.
+        """
+        raise NotImplementedError

environments/hermes_swe_env.py

@@ -0,0 +1,229 @@
+"""
+HermesSweEnv -- SWE-Bench Style Environment with Modal Sandboxes
+
+A concrete environment for software engineering tasks where the model writes code
+and the reward function runs tests to verify correctness. Uses Modal terminal
+backend for cloud-isolated sandboxes per rollout.
+
+The reward function uses ToolContext.terminal() to run test commands in the same
+Modal sandbox the model used during its agentic loop. All filesystem state from
+the model's tool calls is preserved for verification.
+
+Usage:
+    # Phase 1: OpenAI server type
+    vllm serve YourModel --tool-parser hermes
+    run-api
+    python environments/hermes_swe_env.py serve \\
+        --openai.base_url http://localhost:8000/v1 \\
+        --openai.model_name YourModel \\
+        --openai.server_type openai \\
+        --env.dataset_name bigcode/humanevalpack \\
+        --env.terminal_backend modal
+
+    # Phase 2: VLLM server type (full RL training)
+    python environments/hermes_swe_env.py serve \\
+        --openai.base_url http://localhost:8000/v1 \\
+        --openai.model_name YourModel \\
+        --openai.server_type vllm \\
+        --env.tool_call_parser hermes \\
+        --env.terminal_backend modal
+"""
+
+import logging
+import sys
+import time
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple, Union
+
+# Ensure repo root is on sys.path for imports
+_repo_root = Path(__file__).resolve().parent.parent
+if str(_repo_root) not in sys.path:
+    sys.path.insert(0, str(_repo_root))
+
+from datasets import load_dataset
+
+from atroposlib.envs.base import ScoredDataGroup
+from atroposlib.envs.server_handling.server_manager import APIServerConfig
+from atroposlib.type_definitions import Item
+
+from environments.agent_loop import AgentResult
+from environments.hermes_base_env import HermesAgentBaseEnv, HermesAgentEnvConfig
+from environments.tool_context import ToolContext
+
+logger = logging.getLogger(__name__)
+
+
+class HermesSweEnvConfig(HermesAgentEnvConfig):
+    """Config with defaults for SWE-bench style tasks."""
+
+    pass  # Inherits all fields, overrides defaults in config_init
+
+
+class HermesSweEnv(HermesAgentBaseEnv):
+    """
+    SWE-bench style environment using Modal terminal backend.
+
+    The model gets a coding task, uses terminal + file + web tools to solve it,
+    and the reward function runs tests in the same Modal sandbox to verify.
+
+    Subclass this for specific SWE datasets (HumanEval, SWE-bench, etc.)
+    and customize format_prompt() and compute_reward() as needed.
+    """
+
+    name = "hermes-swe"
+    env_config_cls = HermesSweEnvConfig
+
+    @classmethod
+    def config_init(cls) -> Tuple[HermesSweEnvConfig, List[APIServerConfig]]:
+        """
+        Default configuration for the SWE environment.
+
+        Uses Modal terminal backend for cloud isolation and terminal + file + web toolsets.
+        """
+        env_config = HermesSweEnvConfig(
+            # Toolsets: terminal for running code, file for reading/writing, web for docs
+            enabled_toolsets=["terminal", "file", "web"],
+            disabled_toolsets=None,
+            distribution=None,
+            # Agent settings -- SWE tasks need more turns
+            max_agent_turns=30,
+            max_token_length=4096,
+            agent_temperature=1.0,
+            system_prompt=(
+                "You are a skilled software engineer. You have access to a terminal, "
+                "file tools, and web search. Use these tools to complete the coding task. "
+                "Write clean, working code and verify it runs correctly before finishing."
+            ),
+            # Modal backend for cloud-isolated sandboxes
+            terminal_backend="modal",
+            # Dataset -- override via CLI for your specific SWE dataset
+            dataset_name="bigcode/humanevalpack",
+            dataset_split="test",
+            prompt_field="prompt",
+            # Atropos settings
+            group_size=4,
+            tokenizer_name="NousResearch/DeepHermes-3-Llama-3-3B-Preview",
+            tool_call_parser="hermes",
+            steps_per_eval=50,
+            total_steps=500,
+            use_wandb=True,
+            wandb_name="hermes-swe",
+        )
+
+        server_configs = [
+            APIServerConfig(
+                base_url="http://localhost:8000/v1",
+                model_name="NousResearch/DeepHermes-3-Llama-3-3B-Preview",
+                server_type="openai",  # Phase 1; switch to "vllm" for Phase 2
+                api_key="",
+            )
+        ]
+
+        return env_config, server_configs
+
+    async def setup(self):
+        """Load the SWE dataset."""
+        if self.config.dataset_name:
+            self.dataset = load_dataset(
+                self.config.dataset_name, split=self.config.dataset_split
+            )
+        else:
+            # Placeholder if no dataset specified
+            self.dataset = []
+        self.iter = 0
+        self.reward_buffer: List[float] = []
+
+    async def get_next_item(self) -> Dict[str, Any]:
+        """Cycle through the SWE dataset."""
+        if not self.dataset:
+            raise ValueError("No dataset loaded. Set dataset_name in config.")
+        item = self.dataset[self.iter % len(self.dataset)]
+        self.iter += 1
+        return item
+
+    def format_prompt(self, item: Dict[str, Any]) -> str:
+        """
+        Format the SWE task prompt.
+
+        Override this in subclasses for different dataset formats.
+        Default assumes the dataset has a 'prompt' field and optionally a 'test' field.
+        """
+        prompt = item.get(self.config.prompt_field, "")
+
+        # If the dataset has test information, include it in the prompt
+        test_info = item.get("test", item.get("test_code", item.get("tests", "")))
+        if test_info:
+            prompt += f"\n\nTests to pass:\n{test_info}"
+
+        return prompt
+
+    async def compute_reward(
+        self, item: Dict[str, Any], result: AgentResult, ctx: ToolContext
+    ) -> float:
+        """
+        Score by running tests in the model's Modal sandbox.
+
+        Default implementation:
+        - If the dataset item has a 'test' or 'test_code' field, run it
+        - Check exit code: 0 = pass, non-zero = fail
+        - Partial credit for file creation
+
+        Override this in subclasses for more sophisticated reward logic.
+        """
+        # Find the test command from the dataset item
+        test_code = item.get("test", item.get("test_code", item.get("tests", "")))
+
+        if test_code:
+            # Run the test in the model's sandbox
+            test_result = ctx.terminal(
+                f'cd /workspace && python3 -c "{test_code}"', timeout=60
+            )
+
+            if test_result["exit_code"] == 0:
+                self.reward_buffer.append(1.0)
+                return 1.0
+
+        # Partial credit: check if the model created any Python files
+        file_check = ctx.terminal("find /workspace -name '*.py' -newer /tmp/.start_marker 2>/dev/null | head -5")
+        if file_check["exit_code"] == 0 and file_check.get("output", "").strip():
+            self.reward_buffer.append(0.1)
+            return 0.1
+
+        self.reward_buffer.append(0.0)
+        return 0.0
+
+    async def evaluate(self, *args, **kwargs):
+        """
+        Run evaluation on a held-out set.
+
+        Override for dataset-specific evaluation logic.
+        """
+        start_time = time.time()
+        end_time = time.time()
+
+        eval_metrics = {"eval/placeholder": 0.0}
+        await self.evaluate_log(
+            metrics=eval_metrics,
+            start_time=start_time,
+            end_time=end_time,
+        )
+
+    async def wandb_log(self, wandb_metrics: Optional[Dict] = None):
+        """Log SWE-specific metrics."""
+        if wandb_metrics is None:
+            wandb_metrics = {}
+
+        if self.reward_buffer:
+            wandb_metrics["train/avg_reward"] = sum(self.reward_buffer) / len(
+                self.reward_buffer
+            )
+            wandb_metrics["train/pass_rate"] = sum(
+                1 for r in self.reward_buffer if r == 1.0
+            ) / len(self.reward_buffer)
+            self.reward_buffer = []
+
+        await super().wandb_log(wandb_metrics)
+
+
+if __name__ == "__main__":
+    HermesSweEnv.cli()

environments/patches.py

@@ -0,0 +1,188 @@
+"""
+Monkey patches for making hermes-agent tools work inside async frameworks (Atropos).
+
+Problem:
+    Some tools use asyncio.run() internally (e.g., mini-swe-agent's Modal backend,
+    web_extract). This crashes when called from inside Atropos's event loop because
+    asyncio.run() can't be nested.
+
+Solution:
+    Replace the problematic methods with versions that use a dedicated background
+    thread with its own event loop. The calling code sees the same sync interface --
+    call a function, get a result -- but internally the async work happens on a
+    separate thread that doesn't conflict with Atropos's loop.
+
+    These patches are safe for normal CLI use too: when there's no running event
+    loop, the behavior is identical (the background thread approach works regardless).
+
+What gets patched:
+    - SwerexModalEnvironment.__init__ -- creates Modal deployment on a background thread
+    - SwerexModalEnvironment.execute -- runs commands on the same background thread
+    - SwerexModalEnvironment.stop -- stops deployment on the background thread
+
+Usage:
+    Call apply_patches() once at import time (done automatically by hermes_base_env.py).
+    This is idempotent -- calling it multiple times is safe.
+"""
+
+import asyncio
+import logging
+import threading
+from typing import Any
+
+logger = logging.getLogger(__name__)
+
+_patches_applied = False
+
+
+class _AsyncWorker:
+    """
+    A dedicated background thread with its own event loop.
+
+    Allows sync code to submit async coroutines and block for results,
+    even when called from inside another running event loop. Used to
+    bridge sync tool interfaces with async backends (Modal, SWE-ReX).
+    """
+
+    def __init__(self):
+        self._loop: asyncio.AbstractEventLoop = None
+        self._thread: threading.Thread = None
+        self._started = threading.Event()
+
+    def start(self):
+        """Start the background event loop thread."""
+        self._thread = threading.Thread(target=self._run_loop, daemon=True)
+        self._thread.start()
+        self._started.wait(timeout=30)
+
+    def _run_loop(self):
+        """Background thread entry point -- runs the event loop forever."""
+        self._loop = asyncio.new_event_loop()
+        asyncio.set_event_loop(self._loop)
+        self._started.set()
+        self._loop.run_forever()
+
+    def run_coroutine(self, coro, timeout=600):
+        """
+        Submit a coroutine to the background loop and block until it completes.
+
+        Safe to call from any thread, including threads that already have
+        a running event loop.
+        """
+        if self._loop is None or self._loop.is_closed():
+            raise RuntimeError("AsyncWorker loop is not running")
+        future = asyncio.run_coroutine_threadsafe(coro, self._loop)
+        return future.result(timeout=timeout)
+
+    def stop(self):
+        """Stop the background event loop and join the thread."""
+        if self._loop and self._loop.is_running():
+            self._loop.call_soon_threadsafe(self._loop.stop)
+        if self._thread:
+            self._thread.join(timeout=10)
+
+
+def _patch_swerex_modal():
+    """
+    Monkey patch SwerexModalEnvironment to use a background thread event loop
+    instead of asyncio.run(). This makes it safe to call from inside Atropos's
+    async event loop.
+
+    The patched methods have the exact same interface and behavior -- the only
+    difference is HOW the async work is executed internally.
+    """
+    try:
+        from minisweagent.environments.extra.swerex_modal import (
+            SwerexModalEnvironment,
+            SwerexModalEnvironmentConfig,
+        )
+        from swerex.deployment.modal import ModalDeployment
+        from swerex.runtime.abstract import Command as RexCommand
+    except ImportError:
+        # mini-swe-agent or swe-rex not installed -- nothing to patch
+        logger.debug("mini-swe-agent Modal backend not available, skipping patch")
+        return
+
+    # Save original methods so we can refer to config handling
+    _original_init = SwerexModalEnvironment.__init__
+
+    def _patched_init(self, **kwargs):
+        """Patched __init__: creates Modal deployment on a background thread."""
+        self.config = SwerexModalEnvironmentConfig(**kwargs)
+
+        # Start a dedicated event loop thread for all Modal async operations
+        self._worker = _AsyncWorker()
+        self._worker.start()
+
+        # Create AND start the deployment entirely on the worker's loop/thread
+        # so all gRPC channels and async state are bound to that loop
+        async def _create_and_start():
+            deployment = ModalDeployment(
+                image=self.config.image,
+                startup_timeout=self.config.startup_timeout,
+                runtime_timeout=self.config.runtime_timeout,
+                deployment_timeout=self.config.deployment_timeout,
+                install_pipx=self.config.install_pipx,
+                modal_sandbox_kwargs=self.config.modal_sandbox_kwargs,
+            )
+            await deployment.start()
+            return deployment
+
+        self.deployment = self._worker.run_coroutine(_create_and_start())
+
+    def _patched_execute(self, command: str, cwd: str = "", *, timeout: int | None = None) -> dict[str, Any]:
+        """Patched execute: runs commands on the background thread's loop."""
+        async def _do_execute():
+            return await self.deployment.runtime.execute(
+                RexCommand(
+                    command=command,
+                    shell=True,
+                    check=False,
+                    cwd=cwd or self.config.cwd,
+                    timeout=timeout or self.config.timeout,
+                    merge_output_streams=True,
+                    env=self.config.env if self.config.env else None,
+                )
+            )
+
+        output = self._worker.run_coroutine(_do_execute())
+        return {
+            "output": output.stdout,
+            "returncode": output.exit_code,
+        }
+
+    def _patched_stop(self):
+        """Patched stop: stops deployment on the background thread, then stops the thread."""
+        try:
+            self._worker.run_coroutine(
+                asyncio.wait_for(self.deployment.stop(), timeout=10),
+                timeout=15,
+            )
+        except Exception:
+            pass
+        finally:
+            self._worker.stop()
+
+    # Apply the patches
+    SwerexModalEnvironment.__init__ = _patched_init
+    SwerexModalEnvironment.execute = _patched_execute
+    SwerexModalEnvironment.stop = _patched_stop
+
+    logger.debug("Patched SwerexModalEnvironment for async-safe operation")
+
+
+def apply_patches():
+    """
+    Apply all monkey patches needed for Atropos compatibility.
+
+    Safe to call multiple times -- patches are only applied once.
+    Safe for normal CLI use -- patched code works identically when
+    there is no running event loop.
+    """
+    global _patches_applied
+    if _patches_applied:
+        return
+
+    _patch_swerex_modal()
+
+    _patches_applied = True

environments/terminal_test_env.py

@@ -0,0 +1,292 @@
+"""
+TerminalTestEnv -- Simple Test Environment for Validating the Stack
+
+A self-contained environment with inline tasks (no external dataset needed).
+Each task asks the model to create a file at a known path with specific content.
+The reward verifier cats the file and checks if the content matches.
+
+Enables only terminal + file toolsets. Uses Modal terminal backend with
+OpenRouter (Claude) by default.
+
+Training tasks (3):
+    1. Create ~/greeting.txt with "Hello from Hermes Agent"
+    2. Create ~/count.txt with numbers 1-5, one per line
+    3. Create ~/answer.txt with the result of 123 + 456
+
+Eval task (1):
+    1. Create ~/result.txt with the result of 6 * 7
+
+Usage:
+    # Start Atropos API server
+    run-api
+
+    # Run environment (uses OpenRouter + Modal by default)
+    python environments/terminal_test_env.py serve
+
+    # Process mode (no run-api needed, saves to JSONL)
+    python environments/terminal_test_env.py process \\
+        --env.data_path_to_save_groups terminal_test_output.jsonl
+"""
+
+import logging
+import os
+import sys
+import time
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple, Union
+
+# Ensure repo root is on sys.path for imports
+_repo_root = Path(__file__).resolve().parent.parent
+if str(_repo_root) not in sys.path:
+    sys.path.insert(0, str(_repo_root))
+
+from atroposlib.envs.base import ScoredDataGroup
+from atroposlib.envs.server_handling.server_manager import APIServerConfig
+from atroposlib.type_definitions import Item
+
+from environments.agent_loop import AgentResult
+from environments.hermes_base_env import HermesAgentBaseEnv, HermesAgentEnvConfig
+from environments.tool_context import ToolContext
+
+logger = logging.getLogger(__name__)
+
+
+# =============================================================================
+# Inline task definitions -- no external dataset needed
+# =============================================================================
+
+TRAIN_TASKS = [
+    {
+        "prompt": "Create a file at ~/greeting.txt containing exactly the text: Hello from Hermes Agent",
+        "verify_path": "~/greeting.txt",
+        "expected_content": "Hello from Hermes Agent",
+    },
+    {
+        "prompt": "Create a file at ~/count.txt containing the numbers 1 through 5, one per line",
+        "verify_path": "~/count.txt",
+        "expected_content": "1\n2\n3\n4\n5",
+    },
+    {
+        "prompt": "Create a file at ~/answer.txt containing the result of 123 + 456",
+        "verify_path": "~/answer.txt",
+        "expected_content": "579",
+    },
+]
+
+EVAL_TASKS = [
+    {
+        "prompt": "Create a file at ~/result.txt containing the result of 6 * 7",
+        "verify_path": "~/result.txt",
+        "expected_content": "42",
+    },
+]
+
+
+class TerminalTestEnvConfig(HermesAgentEnvConfig):
+    """Config with defaults suitable for terminal testing."""
+
+    pass  # Inherits all fields, overrides defaults in config_init
+
+
+class TerminalTestEnv(HermesAgentBaseEnv):
+    """
+    Simple test environment with inline file-creation tasks.
+
+    All tasks follow the same pattern: "create a file at ~/X.txt with content Y".
+    The verifier runs `cat ~/X.txt` in the rollout's terminal and checks the output
+    against the expected string. Same verifier logic for all tasks.
+
+    This environment is designed to validate the full stack end-to-end:
+    - Agent loop executes tool calls (terminal/file)
+    - ToolContext provides terminal access to the reward function
+    - Reward function verifies file content via cat
+    - Scored data flows through the Atropos pipeline
+    """
+
+    name = "terminal-test"
+    env_config_cls = TerminalTestEnvConfig
+
+    @classmethod
+    def config_init(cls) -> Tuple[TerminalTestEnvConfig, List[APIServerConfig]]:
+        """
+        Default configuration for the terminal test environment.
+
+        Uses Modal terminal backend for cloud isolation and OpenRouter with
+        Claude for inference. API keys loaded from ~/hermes-agent/.env.
+        """
+        env_config = TerminalTestEnvConfig(
+            # Terminal + file tools only
+            enabled_toolsets=["terminal", "file"],
+            disabled_toolsets=None,
+            distribution=None,
+            # Agent settings
+            max_agent_turns=10,  # Simple tasks, don't need many turns
+            max_token_length=16000,
+            agent_temperature=1.0,
+            system_prompt=(
+                "You are a helpful assistant with access to a terminal and file tools. "
+                "Complete the user's request by using the available tools. "
+                "Be precise and follow instructions exactly."
+            ),
+            # Modal terminal backend for cloud-isolated sandboxes per rollout
+            terminal_backend="modal",
+            # Atropos settings
+            group_size=3,              # 3 rollouts per group
+            tokenizer_name="NousResearch/q-30b-t-h45-e1",
+            tool_call_parser="hermes",
+            steps_per_eval=3,          # Eval after all 3 steps
+            total_steps=3,             # 3 groups total (1 group per step)
+            use_wandb=True,
+            wandb_name="terminal-test",
+            ensure_scores_are_not_same=False,  # Allow all-same scores for simple tasks
+            # No external dataset
+            dataset_name=None,
+        )
+
+        # OpenRouter with Claude -- API key loaded from .env (OPENROUTER_API_KEY)
+        server_configs = [
+            APIServerConfig(
+                base_url="https://openrouter.ai/api/v1",
+                model_name="anthropic/claude-opus-4.6",
+                server_type="openai",
+                api_key=os.getenv("OPENROUTER_API_KEY", ""),
+                health_check=False,  # OpenRouter doesn't have a /health endpoint
+            )
+        ]
+
+        return env_config, server_configs
+
+    async def setup(self):
+        """Initialize inline task lists."""
+        self.train_tasks = list(TRAIN_TASKS)
+        self.eval_tasks = list(EVAL_TASKS)
+        self.iter = 0
+        # Track reward stats for wandb logging
+        self.reward_buffer: List[float] = []
+
+    async def get_next_item(self) -> Dict[str, str]:
+        """Cycle through training tasks."""
+        item = self.train_tasks[self.iter % len(self.train_tasks)]
+        self.iter += 1
+        return item
+
+    def format_prompt(self, item: Dict[str, str]) -> str:
+        """The prompt is directly in the task item."""
+        return item["prompt"]
+
+    async def compute_reward(
+        self, item: Dict[str, str], result: AgentResult, ctx: ToolContext
+    ) -> float:
+        """
+        Verify by cat-ing the expected file path and checking content matches.
+        Same verifier for all tasks -- they all write a file at a known path.
+
+        Scoring:
+            1.0 = exact match
+            0.5 = expected content is present but has extra stuff
+            0.0 = file doesn't exist or content doesn't match
+        """
+        verify_result = ctx.terminal(f"cat {item['verify_path']}")
+
+        # File doesn't exist or can't be read
+        if verify_result["exit_code"] != 0:
+            self.reward_buffer.append(0.0)
+            return 0.0
+
+        actual = verify_result.get("output", "").strip()
+        expected = item["expected_content"].strip()
+
+        # Exact match
+        if actual == expected:
+            self.reward_buffer.append(1.0)
+            return 1.0
+
+        # Partial credit: expected content is present but has extra stuff
+        if expected in actual:
+            self.reward_buffer.append(0.5)
+            return 0.5
+
+        self.reward_buffer.append(0.0)
+        return 0.0
+
+    async def evaluate(self, *args, **kwargs):
+        """
+        Run eval tasks using the agent loop and verify results.
+        Logs accuracy metrics.
+        """
+        start_time = time.time()
+        correct = 0
+        total = len(self.eval_tasks)
+        samples = []
+
+        for eval_item in self.eval_tasks:
+            try:
+                # For eval, we do a simple single-turn completion (not full agent loop)
+                # to keep eval fast. The agent loop is tested via training.
+                completion = await self.server.chat_completion(
+                    messages=[
+                        {"role": "system", "content": self.config.system_prompt or ""},
+                        {"role": "user", "content": eval_item["prompt"]},
+                    ],
+                    n=1,
+                    max_tokens=self.config.max_token_length,
+                    temperature=0.0,
+                    split="eval",
+                )
+
+                response_content = (
+                    completion.choices[0].message.content if completion.choices else ""
+                )
+
+                samples.append(
+                    {
+                        "prompt": eval_item["prompt"],
+                        "response": response_content,
+                        "expected": eval_item["expected_content"],
+                    }
+                )
+
+            except Exception as e:
+                logger.error("Eval failed for item: %s", e)
+                samples.append(
+                    {
+                        "prompt": eval_item["prompt"],
+                        "response": f"ERROR: {e}",
+                        "expected": eval_item["expected_content"],
+                    }
+                )
+
+        end_time = time.time()
+
+        eval_metrics = {
+            "eval/num_samples": total,
+        }
+
+        await self.evaluate_log(
+            metrics=eval_metrics,
+            samples=samples,
+            start_time=start_time,
+            end_time=end_time,
+        )
+
+    async def wandb_log(self, wandb_metrics: Optional[Dict] = None):
+        """Log training metrics including reward stats and accuracy."""
+        if wandb_metrics is None:
+            wandb_metrics = {}
+
+        if self.reward_buffer:
+            total = len(self.reward_buffer)
+            correct = sum(1 for r in self.reward_buffer if r == 1.0)
+            partial = sum(1 for r in self.reward_buffer if r == 0.5)
+
+            wandb_metrics["train/avg_reward"] = sum(self.reward_buffer) / total
+            wandb_metrics["train/accuracy"] = correct / total
+            wandb_metrics["train/partial_match_rate"] = partial / total
+            wandb_metrics["train/total_rollouts"] = total
+            self.reward_buffer = []
+
+        await super().wandb_log(wandb_metrics)
+
+
+if __name__ == "__main__":
+    TerminalTestEnv.cli()

environments/tool_call_parsers/__init__.py

@@ -0,0 +1,120 @@
+"""
+Tool Call Parser Registry
+
+Client-side parsers that extract structured tool_calls from raw model output text.
+Used in Phase 2 (VLLM server type) where ManagedServer's /generate endpoint returns
+raw text without tool call parsing.
+
+Each parser is a standalone reimplementation of the corresponding VLLM parser's
+non-streaming extract_tool_calls() logic. No VLLM dependency -- only standard library
+(re, json, uuid) and openai types.
+
+Usage:
+    from environments.tool_call_parsers import get_parser
+
+    parser = get_parser("hermes")
+    content, tool_calls = parser.parse(raw_model_output)
+    # content = text with tool call markup stripped
+    # tool_calls = list of ChatCompletionMessageToolCall objects, or None
+"""
+
+import logging
+from abc import ABC, abstractmethod
+from typing import Dict, List, Optional, Tuple, Type
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+)
+
+logger = logging.getLogger(__name__)
+
+# Type alias for parser return value
+ParseResult = Tuple[Optional[str], Optional[List[ChatCompletionMessageToolCall]]]
+
+
+class ToolCallParser(ABC):
+    """
+    Base class for tool call parsers.
+
+    Each parser knows how to extract structured tool_calls from a specific
+    model family's raw output text format.
+    """
+
+    @abstractmethod
+    def parse(self, text: str) -> ParseResult:
+        """
+        Parse raw model output text for tool calls.
+
+        Args:
+            text: Raw decoded text from the model's completion
+
+        Returns:
+            Tuple of (content, tool_calls) where:
+            - content: text with tool call markup stripped (the message 'content' field),
+                       or None if the entire output was tool calls
+            - tool_calls: list of ChatCompletionMessageToolCall objects,
+                          or None if no tool calls were found
+        """
+        raise NotImplementedError
+
+
+# Global parser registry: name -> parser class
+PARSER_REGISTRY: Dict[str, Type[ToolCallParser]] = {}
+
+
+def register_parser(name: str):
+    """
+    Decorator to register a parser class under a given name.
+
+    Usage:
+        @register_parser("hermes")
+        class HermesToolCallParser(ToolCallParser):
+            ...
+    """
+
+    def decorator(cls: Type[ToolCallParser]) -> Type[ToolCallParser]:
+        PARSER_REGISTRY[name] = cls
+        return cls
+
+    return decorator
+
+
+def get_parser(name: str) -> ToolCallParser:
+    """
+    Get a parser instance by name.
+
+    Args:
+        name: Parser name (e.g., "hermes", "mistral", "llama3_json")
+
+    Returns:
+        Instantiated parser
+
+    Raises:
+        KeyError: If parser name is not found in registry
+    """
+    if name not in PARSER_REGISTRY:
+        available = sorted(PARSER_REGISTRY.keys())
+        raise KeyError(
+            f"Tool call parser '{name}' not found. Available parsers: {available}"
+        )
+    return PARSER_REGISTRY[name]()
+
+
+def list_parsers() -> List[str]:
+    """Return sorted list of registered parser names."""
+    return sorted(PARSER_REGISTRY.keys())
+
+
+# Import all parser modules to trigger registration via @register_parser decorators
+# Each module registers itself when imported
+from environments.tool_call_parsers.hermes_parser import HermesToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.longcat_parser import LongcatToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.mistral_parser import MistralToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.llama_parser import LlamaToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.qwen_parser import QwenToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.deepseek_v3_parser import DeepSeekV3ToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.deepseek_v3_1_parser import DeepSeekV31ToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.kimi_k2_parser import KimiK2ToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.glm45_parser import Glm45ToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.glm47_parser import Glm47ToolCallParser  # noqa: E402, F401
+from environments.tool_call_parsers.qwen3_coder_parser import Qwen3CoderToolCallParser  # noqa: E402, F401

environments/tool_call_parsers/deepseek_v3_1_parser.py

@@ -0,0 +1,71 @@
+"""
+DeepSeek V3.1 tool call parser.
+
+Similar to V3 but with a slightly different format:
+    <｜tool▁call▁begin｜>function_name<｜tool▁sep｜>arguments<｜tool▁call▁end｜>
+
+Note: V3 has type+name before the separator, V3.1 has name before and args after.
+
+Based on VLLM's DeepSeekV31ToolParser.extract_tool_calls()
+"""
+
+import re
+import uuid
+from typing import List, Optional
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+@register_parser("deepseek_v3_1")
+@register_parser("deepseek_v31")
+class DeepSeekV31ToolCallParser(ToolCallParser):
+    """
+    Parser for DeepSeek V3.1 tool calls.
+
+    Slightly different regex than V3: function_name comes before the separator,
+    arguments come after (no type field, no json code block wrapper).
+    """
+
+    START_TOKEN = "<｜tool▁calls▁begin｜>"
+
+    # Regex captures: function_name, function_arguments
+    PATTERN = re.compile(
+        r"<｜tool▁call▁begin｜>(?P<function_name>.*?)<｜tool▁sep｜>(?P<function_arguments>.*?)<｜tool▁call▁end｜>"
+    )
+
+    def parse(self, text: str) -> ParseResult:
+        if self.START_TOKEN not in text:
+            return text, None
+
+        try:
+            matches = self.PATTERN.findall(text)
+            if not matches:
+                return text, None
+
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+            for match in matches:
+                func_name, func_args = match
+                tool_calls.append(
+                    ChatCompletionMessageToolCall(
+                        id=f"call_{uuid.uuid4().hex[:8]}",
+                        type="function",
+                        function=Function(
+                            name=func_name.strip(),
+                            arguments=func_args.strip(),
+                        ),
+                    )
+                )
+
+            if not tool_calls:
+                return text, None
+
+            content = text[: text.find(self.START_TOKEN)].strip()
+            return content if content else None, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/deepseek_v3_parser.py

@@ -0,0 +1,75 @@
+"""
+DeepSeek V3 tool call parser.
+
+Format uses special unicode tokens:
+    <｜tool▁calls▁begin｜>
+    <｜tool▁call▁begin｜>type<｜tool▁sep｜>function_name
+    ```json
+    {"arg": "value"}
+    ```
+    <｜tool▁call▁end｜>
+    <｜tool▁calls▁end｜>
+
+Based on VLLM's DeepSeekV3ToolParser.extract_tool_calls()
+"""
+
+import re
+import uuid
+from typing import List, Optional
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+@register_parser("deepseek_v3")
+class DeepSeekV3ToolCallParser(ToolCallParser):
+    """
+    Parser for DeepSeek V3 tool calls.
+
+    Uses special unicode tokens with fullwidth angle brackets and block elements.
+    Extracts type, function name, and JSON arguments from the structured format.
+    """
+
+    START_TOKEN = "<｜tool▁calls▁begin｜>"
+
+    # Regex captures: type, function_name, function_arguments
+    PATTERN = re.compile(
+        r"<｜tool▁call▁begin｜>(?P<type>.*)<｜tool▁sep｜>(?P<function_name>.*)\n```json\n(?P<function_arguments>.*)\n```<｜tool▁call▁end｜>"
+    )
+
+    def parse(self, text: str) -> ParseResult:
+        if self.START_TOKEN not in text:
+            return text, None
+
+        try:
+            matches = self.PATTERN.findall(text)
+            if not matches:
+                return text, None
+
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+            for match in matches:
+                tc_type, func_name, func_args = match
+                tool_calls.append(
+                    ChatCompletionMessageToolCall(
+                        id=f"call_{uuid.uuid4().hex[:8]}",
+                        type="function",
+                        function=Function(
+                            name=func_name.strip(),
+                            arguments=func_args.strip(),
+                        ),
+                    )
+                )
+
+            if not tool_calls:
+                return text, None
+
+            # Content is everything before the tool calls section
+            content = text[: text.find(self.START_TOKEN)].strip()
+            return content if content else None, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/glm45_parser.py

@@ -0,0 +1,109 @@
+"""
+GLM 4.5 (GLM-4-MoE) tool call parser.
+
+Format uses custom arg_key/arg_value tags rather than standard JSON:
+    <tool_call>function_name
+    <arg_key>param1</arg_key><arg_value>value1</arg_value>
+    <arg_key>param2</arg_key><arg_value>value2</arg_value>
+    </tool_call>
+
+Values are deserialized using json.loads -> ast.literal_eval -> raw string fallback.
+
+Based on VLLM's Glm4MoeModelToolParser.extract_tool_calls()
+"""
+
+import ast
+import json
+import re
+import uuid
+from typing import Any, Dict, List, Optional
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+def _deserialize_value(value: str) -> Any:
+    """
+    Try to deserialize a string value to its native Python type.
+    Attempts json.loads, then ast.literal_eval, then returns raw string.
+    """
+    try:
+        return json.loads(value)
+    except (json.JSONDecodeError, TypeError):
+        pass
+
+    try:
+        return ast.literal_eval(value)
+    except (ValueError, SyntaxError, TypeError):
+        pass
+
+    return value
+
+
+@register_parser("glm45")
+class Glm45ToolCallParser(ToolCallParser):
+    """
+    Parser for GLM 4.5 (GLM-4-MoE) tool calls.
+
+    Uses <tool_call>...</tool_call> tags with <arg_key>/<arg_value> pairs
+    instead of standard JSON arguments.
+    """
+
+    FUNC_CALL_REGEX = re.compile(r"<tool_call>.*?</tool_call>", re.DOTALL)
+    FUNC_DETAIL_REGEX = re.compile(r"<tool_call>([^\n]*)\n(.*)</tool_call>", re.DOTALL)
+    FUNC_ARG_REGEX = re.compile(
+        r"<arg_key>(.*?)</arg_key>\s*<arg_value>(.*?)</arg_value>", re.DOTALL
+    )
+
+    START_TOKEN = "<tool_call>"
+
+    def parse(self, text: str) -> ParseResult:
+        if self.START_TOKEN not in text:
+            return text, None
+
+        try:
+            matched_calls = self.FUNC_CALL_REGEX.findall(text)
+            if not matched_calls:
+                return text, None
+
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+
+            for match in matched_calls:
+                detail = self.FUNC_DETAIL_REGEX.search(match)
+                if not detail:
+                    continue
+
+                func_name = detail.group(1).strip()
+                func_args_raw = detail.group(2)
+
+                # Parse arg_key/arg_value pairs
+                pairs = self.FUNC_ARG_REGEX.findall(func_args_raw) if func_args_raw else []
+                arg_dict: Dict[str, Any] = {}
+                for key, value in pairs:
+                    arg_key = key.strip()
+                    arg_val = _deserialize_value(value.strip())
+                    arg_dict[arg_key] = arg_val
+
+                tool_calls.append(
+                    ChatCompletionMessageToolCall(
+                        id=f"call_{uuid.uuid4().hex[:8]}",
+                        type="function",
+                        function=Function(
+                            name=func_name,
+                            arguments=json.dumps(arg_dict, ensure_ascii=False),
+                        ),
+                    )
+                )
+
+            if not tool_calls:
+                return text, None
+
+            content = text[: text.find(self.START_TOKEN)].strip()
+            return content if content else None, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/glm47_parser.py

@@ -0,0 +1,35 @@
+"""
+GLM 4.7 tool call parser.
+
+Same as GLM 4.5 but with slightly different regex patterns.
+The tool_call tags may wrap differently and arg parsing handles
+newlines between key/value pairs.
+
+Based on VLLM's Glm47MoeModelToolParser (extends Glm4MoeModelToolParser).
+"""
+
+import re
+
+from environments.tool_call_parsers import ParseResult, register_parser
+from environments.tool_call_parsers.glm45_parser import Glm45ToolCallParser
+
+
+@register_parser("glm47")
+class Glm47ToolCallParser(Glm45ToolCallParser):
+    """
+    Parser for GLM 4.7 tool calls.
+    Extends GLM 4.5 with updated regex patterns.
+    """
+
+    def __init__(self):
+        super().__init__()
+        # GLM 4.7 uses a slightly different detail regex that includes
+        # the <tool_call> wrapper and optional arg_key content
+        self.FUNC_DETAIL_REGEX = re.compile(
+            r"<tool_call>(.*?)(<arg_key>.*?)?</tool_call>", re.DOTALL
+        )
+        # GLM 4.7 handles newlines between arg_key and arg_value tags
+        self.FUNC_ARG_REGEX = re.compile(
+            r"<arg_key>(.*?)</arg_key>(?:\\n|\s)*<arg_value>(.*?)</arg_value>",
+            re.DOTALL,
+        )

environments/tool_call_parsers/hermes_parser.py

@@ -0,0 +1,73 @@
+"""
+Hermes tool call parser.
+
+Format: <tool_call>{"name": "func", "arguments": {...}}</tool_call>
+Based on VLLM's Hermes2ProToolParser.extract_tool_calls()
+"""
+
+import json
+import re
+import uuid
+from typing import List, Optional, Tuple
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+@register_parser("hermes")
+class HermesToolCallParser(ToolCallParser):
+    """
+    Parser for Hermes-format tool calls.
+
+    Matches <tool_call>...</tool_call> tags containing JSON with "name" and "arguments".
+    Also handles unclosed <tool_call> at end-of-string (truncated generation).
+    """
+
+    # Matches both closed and unclosed tool_call tags
+    PATTERN = re.compile(
+        r"<tool_call>\s*(.*?)\s*</tool_call>|<tool_call>\s*(.*)", re.DOTALL
+    )
+
+    def parse(self, text: str) -> ParseResult:
+        if "<tool_call>" not in text:
+            return text, None
+
+        try:
+            matches = self.PATTERN.findall(text)
+            if not matches:
+                return text, None
+
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+            for match in matches:
+                # match is a tuple: (closed_content, unclosed_content)
+                raw_json = match[0] if match[0] else match[1]
+                if not raw_json.strip():
+                    continue
+
+                tc_data = json.loads(raw_json)
+                tool_calls.append(
+                    ChatCompletionMessageToolCall(
+                        id=f"call_{uuid.uuid4().hex[:8]}",
+                        type="function",
+                        function=Function(
+                            name=tc_data["name"],
+                            arguments=json.dumps(
+                                tc_data.get("arguments", {}), ensure_ascii=False
+                            ),
+                        ),
+                    )
+                )
+
+            if not tool_calls:
+                return text, None
+
+            # Content is everything before the first <tool_call> tag
+            content = text[: text.find("<tool_call>")].strip()
+            return content if content else None, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/kimi_k2_parser.py

@@ -0,0 +1,93 @@
+"""
+Kimi K2 tool call parser.
+
+Format:
+    <|tool_calls_section_begin|>
+    <|tool_call_begin|>function_id:0<|tool_call_argument_begin|>{"arg": "val"}<|tool_call_end|>
+    <|tool_calls_section_end|>
+
+The function_id format is typically "functions.func_name:index" or "func_name:index".
+
+Based on VLLM's KimiK2ToolParser.extract_tool_calls()
+"""
+
+import re
+import uuid
+from typing import List, Optional
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+@register_parser("kimi_k2")
+class KimiK2ToolCallParser(ToolCallParser):
+    """
+    Parser for Kimi K2 tool calls.
+
+    Uses section begin/end tokens wrapping individual tool call begin/end tokens.
+    The tool_call_id contains the function name (after last dot, before colon).
+    """
+
+    # Support both singular and plural variants
+    START_TOKENS = [
+        "<|tool_calls_section_begin|>",
+        "<|tool_call_section_begin|>",
+    ]
+
+    # Regex captures: tool_call_id (e.g., "functions.get_weather:0"), function_arguments
+    PATTERN = re.compile(
+        r"<\|tool_call_begin\|>\s*(?P<tool_call_id>[^<]+:\d+)\s*"
+        r"<\|tool_call_argument_begin\|>\s*"
+        r"(?P<function_arguments>(?:(?!<\|tool_call_begin\|>).)*?)\s*"
+        r"<\|tool_call_end\|>",
+        re.DOTALL,
+    )
+
+    def parse(self, text: str) -> ParseResult:
+        # Check for any variant of the start token
+        has_start = any(token in text for token in self.START_TOKENS)
+        if not has_start:
+            return text, None
+
+        try:
+            matches = self.PATTERN.findall(text)
+            if not matches:
+                return text, None
+
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+            for match in matches:
+                function_id, function_args = match
+
+                # Extract function name from ID format: "functions.get_weather:0" -> "get_weather"
+                function_name = function_id.split(":")[0].split(".")[-1]
+
+                tool_calls.append(
+                    ChatCompletionMessageToolCall(
+                        id=function_id,  # Preserve the original ID format
+                        type="function",
+                        function=Function(
+                            name=function_name,
+                            arguments=function_args.strip(),
+                        ),
+                    )
+                )
+
+            if not tool_calls:
+                return text, None
+
+            # Content is everything before the tool calls section
+            earliest_start = len(text)
+            for token in self.START_TOKENS:
+                idx = text.find(token)
+                if idx >= 0 and idx < earliest_start:
+                    earliest_start = idx
+
+            content = text[:earliest_start].strip()
+            return content if content else None, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/llama_parser.py

@@ -0,0 +1,96 @@
+"""
+Llama 3.x / 4 tool call parser.
+
+Format: The model outputs JSON objects with "name" and "arguments" (or "parameters") keys.
+May be preceded by <|python_tag|> token. Supports multiple JSON objects separated
+by content or semicolons.
+
+Based on VLLM's Llama3JsonToolParser.extract_tool_calls()
+"""
+
+import json
+import re
+import uuid
+from typing import List, Optional
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+@register_parser("llama3_json")
+@register_parser("llama4_json")
+class LlamaToolCallParser(ToolCallParser):
+    """
+    Parser for Llama 3.x and 4 JSON-format tool calls.
+
+    Finds JSON objects containing "name" + ("arguments" or "parameters") keys.
+    Uses Python's json.JSONDecoder.raw_decode for robust extraction of
+    JSON objects from mixed text.
+    """
+
+    BOT_TOKEN = "<|python_tag|>"
+
+    # Regex to find the start of potential JSON objects
+    JSON_START = re.compile(r"\{")
+
+    def parse(self, text: str) -> ParseResult:
+        # Quick check: need either the bot token or a JSON brace
+        if self.BOT_TOKEN not in text and "{" not in text:
+            return text, None
+
+        try:
+            decoder = json.JSONDecoder()
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+            end_index = -1  # Track where the last parsed JSON ended
+
+            for match in self.JSON_START.finditer(text):
+                start = match.start()
+                # Skip if this brace is inside a previously parsed JSON object
+                if start <= end_index:
+                    continue
+
+                try:
+                    obj, json_end = decoder.raw_decode(text[start:])
+                    end_index = start + json_end
+
+                    # Must have "name" and either "arguments" or "parameters"
+                    name = obj.get("name")
+                    args = obj.get("arguments", obj.get("parameters"))
+
+                    if not name or args is None:
+                        continue
+
+                    # Normalize arguments to JSON string
+                    if isinstance(args, dict):
+                        args = json.dumps(args, ensure_ascii=False)
+                    elif not isinstance(args, str):
+                        args = json.dumps(args, ensure_ascii=False)
+
+                    tool_calls.append(
+                        ChatCompletionMessageToolCall(
+                            id=f"call_{uuid.uuid4().hex[:8]}",
+                            type="function",
+                            function=Function(name=name, arguments=args),
+                        )
+                    )
+                except (json.JSONDecodeError, KeyError, ValueError):
+                    continue
+
+            if not tool_calls:
+                return text, None
+
+            # Content is everything before the first tool call JSON
+            # Find where the first tool call starts in the text
+            first_tc_start = text.find("{")
+            if self.BOT_TOKEN in text:
+                first_tc_start = text.find(self.BOT_TOKEN)
+            content = text[:first_tc_start].strip() if first_tc_start > 0 else None
+
+            return content, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/longcat_parser.py

@@ -0,0 +1,69 @@
+"""
+Longcat Flash Chat tool call parser.
+
+Same as Hermes but uses <longcat_tool_call> tags instead of <tool_call>.
+Based on VLLM's LongcatFlashToolParser (extends Hermes2ProToolParser).
+"""
+
+import json
+import re
+import uuid
+from typing import List, Optional
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+@register_parser("longcat")
+class LongcatToolCallParser(ToolCallParser):
+    """
+    Parser for Longcat Flash Chat tool calls.
+    Identical logic to Hermes, just different tag names.
+    """
+
+    PATTERN = re.compile(
+        r"<longcat_tool_call>\s*(.*?)\s*</longcat_tool_call>|<longcat_tool_call>\s*(.*)",
+        re.DOTALL,
+    )
+
+    def parse(self, text: str) -> ParseResult:
+        if "<longcat_tool_call>" not in text:
+            return text, None
+
+        try:
+            matches = self.PATTERN.findall(text)
+            if not matches:
+                return text, None
+
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+            for match in matches:
+                raw_json = match[0] if match[0] else match[1]
+                if not raw_json.strip():
+                    continue
+
+                tc_data = json.loads(raw_json)
+                tool_calls.append(
+                    ChatCompletionMessageToolCall(
+                        id=f"call_{uuid.uuid4().hex[:8]}",
+                        type="function",
+                        function=Function(
+                            name=tc_data["name"],
+                            arguments=json.dumps(
+                                tc_data.get("arguments", {}), ensure_ascii=False
+                            ),
+                        ),
+                    )
+                )
+
+            if not tool_calls:
+                return text, None
+
+            content = text[: text.find("<longcat_tool_call>")].strip()
+            return content if content else None, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/mistral_parser.py

@@ -0,0 +1,130 @@
+"""
+Mistral tool call parser.
+
+Supports two formats depending on tokenizer version:
+- Pre-v11: content[TOOL_CALLS] [{"name": ..., "arguments": {...}}, ...]
+- v11+:    content[TOOL_CALLS]tool_name1{"arg": "val"}[TOOL_CALLS]tool_name2{"arg": "val"}
+
+Based on VLLM's MistralToolParser.extract_tool_calls()
+The [TOOL_CALLS] token is the bot_token used by Mistral models.
+"""
+
+import json
+import re
+import uuid
+from typing import List, Optional
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+def _generate_mistral_id() -> str:
+    """Mistral tool call IDs are 9-char alphanumeric strings."""
+    import random
+    import string
+
+    return "".join(random.choices(string.ascii_letters + string.digits, k=9))
+
+
+@register_parser("mistral")
+class MistralToolCallParser(ToolCallParser):
+    """
+    Parser for Mistral-format tool calls.
+
+    Detects format by checking if the content after [TOOL_CALLS] starts with '['
+    (pre-v11 JSON array) or with a tool name (v11+ format).
+    """
+
+    # The [TOOL_CALLS] token -- may appear as different strings depending on tokenizer
+    BOT_TOKEN = "[TOOL_CALLS]"
+
+    # Fallback regex for pre-v11 format when JSON parsing fails
+    TOOL_CALL_REGEX = re.compile(r"\[?\s*(\{.*?\})\s*\]?", re.DOTALL)
+
+    def parse(self, text: str) -> ParseResult:
+        if self.BOT_TOKEN not in text:
+            return text, None
+
+        try:
+            parts = text.split(self.BOT_TOKEN)
+            content = parts[0].strip()
+            raw_tool_calls = parts[1:]
+
+            # Detect format: if the first raw part starts with '[', it's pre-v11
+            first_raw = raw_tool_calls[0].strip() if raw_tool_calls else ""
+            is_pre_v11 = first_raw.startswith("[") or first_raw.startswith("{")
+
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+
+            if not is_pre_v11:
+                # v11+ format: [TOOL_CALLS]tool_name{args}[TOOL_CALLS]tool_name2{args2}
+                for raw in raw_tool_calls:
+                    raw = raw.strip()
+                    if not raw or "{" not in raw:
+                        continue
+
+                    brace_idx = raw.find("{")
+                    tool_name = raw[:brace_idx].strip()
+                    args_str = raw[brace_idx:]
+
+                    tool_calls.append(
+                        ChatCompletionMessageToolCall(
+                            id=_generate_mistral_id(),
+                            type="function",
+                            function=Function(name=tool_name, arguments=args_str),
+                        )
+                    )
+            else:
+                # Pre-v11 format: [TOOL_CALLS] [{"name": ..., "arguments": {...}}]
+                try:
+                    parsed = json.loads(first_raw)
+                    if isinstance(parsed, dict):
+                        parsed = [parsed]
+
+                    for tc in parsed:
+                        args = tc.get("arguments", {})
+                        if isinstance(args, dict):
+                            args = json.dumps(args, ensure_ascii=False)
+
+                        tool_calls.append(
+                            ChatCompletionMessageToolCall(
+                                id=_generate_mistral_id(),
+                                type="function",
+                                function=Function(
+                                    name=tc["name"], arguments=args
+                                ),
+                            )
+                        )
+                except json.JSONDecodeError:
+                    # Fallback regex extraction
+                    match = self.TOOL_CALL_REGEX.findall(first_raw)
+                    if match:
+                        for raw_json in match:
+                            try:
+                                tc = json.loads(raw_json)
+                                args = tc.get("arguments", {})
+                                if isinstance(args, dict):
+                                    args = json.dumps(args, ensure_ascii=False)
+                                tool_calls.append(
+                                    ChatCompletionMessageToolCall(
+                                        id=_generate_mistral_id(),
+                                        type="function",
+                                        function=Function(
+                                            name=tc["name"], arguments=args
+                                        ),
+                                    )
+                                )
+                            except (json.JSONDecodeError, KeyError):
+                                continue
+
+            if not tool_calls:
+                return text, None
+
+            return content if content else None, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/qwen3_coder_parser.py

@@ -0,0 +1,163 @@
+"""
+Qwen3-Coder tool call parser.
+
+Format uses XML-style nested tags:
+    <tool_call>
+    <function=function_name>
+    <parameter=param_name>value</parameter>
+    <parameter=param_name2>value2</parameter>
+    </function>
+    </tool_call>
+
+Parameters are extracted from <parameter=name>value</parameter> tags and
+type-converted using the schema if available, otherwise treated as strings.
+
+Based on VLLM's Qwen3CoderToolParser.extract_tool_calls()
+"""
+
+import ast
+import json
+import re
+import uuid
+from typing import Any, Dict, List, Optional
+
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCall,
+    Function,
+)
+
+from environments.tool_call_parsers import ParseResult, ToolCallParser, register_parser
+
+
+def _try_convert_value(value: str) -> Any:
+    """
+    Try to convert a parameter value string to a native Python type.
+    Handles null, numbers, booleans, JSON objects/arrays, and falls back to string.
+    """
+    stripped = value.strip()
+
+    # Handle null
+    if stripped.lower() == "null":
+        return None
+
+    # Try JSON first (handles objects, arrays, strings, numbers, booleans)
+    try:
+        return json.loads(stripped)
+    except (json.JSONDecodeError, TypeError):
+        pass
+
+    # Try Python literal eval (handles tuples, etc.)
+    try:
+        return ast.literal_eval(stripped)
+    except (ValueError, SyntaxError, TypeError):
+        pass
+
+    # Return as string
+    return stripped
+
+
+@register_parser("qwen3_coder")
+class Qwen3CoderToolCallParser(ToolCallParser):
+    """
+    Parser for Qwen3-Coder XML-format tool calls.
+
+    Uses nested XML tags: <tool_call><function=name><parameter=key>val</parameter></function></tool_call>
+    """
+
+    START_TOKEN = "<tool_call>"
+    FUNCTION_PREFIX = "<function="
+
+    # Find complete tool_call blocks (or unclosed at end)
+    TOOL_CALL_REGEX = re.compile(
+        r"<tool_call>(.*?)</tool_call>|<tool_call>(.*?)$", re.DOTALL
+    )
+
+    # Find function blocks within a tool_call
+    FUNCTION_REGEX = re.compile(
+        r"<function=(.*?)</function>|<function=(.*)$", re.DOTALL
+    )
+
+    # Find parameter blocks within a function
+    PARAMETER_REGEX = re.compile(
+        r"<parameter=(.*?)(?:</parameter>|(?=<parameter=)|(?=</function>)|$)",
+        re.DOTALL,
+    )
+
+    def _parse_function_call(self, function_str: str) -> Optional[ChatCompletionMessageToolCall]:
+        """Parse a single <function=name>...</function> block into a ToolCall."""
+        try:
+            # Extract function name: everything before the first '>'
+            gt_idx = function_str.index(">")
+            func_name = function_str[:gt_idx].strip()
+            params_str = function_str[gt_idx + 1:]
+
+            # Extract parameters
+            param_dict: Dict[str, Any] = {}
+            for match_text in self.PARAMETER_REGEX.findall(params_str):
+                if ">" not in match_text:
+                    continue
+                eq_idx = match_text.index(">")
+                param_name = match_text[:eq_idx].strip()
+                param_value = match_text[eq_idx + 1:]
+
+                # Clean up whitespace
+                if param_value.startswith("\n"):
+                    param_value = param_value[1:]
+                if param_value.endswith("\n"):
+                    param_value = param_value[:-1]
+
+                param_dict[param_name] = _try_convert_value(param_value)
+
+            return ChatCompletionMessageToolCall(
+                id=f"call_{uuid.uuid4().hex[:24]}",
+                type="function",
+                function=Function(
+                    name=func_name,
+                    arguments=json.dumps(param_dict, ensure_ascii=False),
+                ),
+            )
+        except (ValueError, IndexError):
+            return None
+
+    def parse(self, text: str) -> ParseResult:
+        if self.FUNCTION_PREFIX not in text:
+            return text, None
+
+        try:
+            # Find all tool_call blocks
+            tc_matches = self.TOOL_CALL_REGEX.findall(text)
+            raw_blocks = [m[0] if m[0] else m[1] for m in tc_matches]
+
+            # Fallback: if no tool_call tags, try the whole text
+            if not raw_blocks:
+                raw_blocks = [text]
+
+            # Find function blocks within each tool_call
+            function_strs: List[str] = []
+            for block in raw_blocks:
+                func_matches = self.FUNCTION_REGEX.findall(block)
+                function_strs.extend(m[0] if m[0] else m[1] for m in func_matches)
+
+            if not function_strs:
+                return text, None
+
+            # Parse each function call
+            tool_calls: List[ChatCompletionMessageToolCall] = []
+            for func_str in function_strs:
+                tc = self._parse_function_call(func_str)
+                if tc is not None:
+                    tool_calls.append(tc)
+
+            if not tool_calls:
+                return text, None
+
+            # Content before tool calls
+            first_tc = text.find(self.START_TOKEN)
+            if first_tc < 0:
+                first_tc = text.find(self.FUNCTION_PREFIX)
+            content = text[:first_tc].strip() if first_tc > 0 else None
+
+            return content, tool_calls
+
+        except Exception:
+            return text, None

environments/tool_call_parsers/qwen_parser.py

@@ -0,0 +1,19 @@
+"""
+Qwen 2.5 tool call parser.
+
+Uses the same <tool_call> format as Hermes.
+Registered as a separate parser name for clarity when using --tool-parser=qwen.
+"""
+
+from environments.tool_call_parsers import register_parser
+from environments.tool_call_parsers.hermes_parser import HermesToolCallParser
+
+
+@register_parser("qwen")
+class QwenToolCallParser(HermesToolCallParser):
+    """
+    Parser for Qwen 2.5 tool calls.
+    Same <tool_call>{"name": ..., "arguments": ...}</tool_call> format as Hermes.
+    """
+
+    pass  # Identical format -- inherits everything from Hermes

environments/tool_context.py

@@ -0,0 +1,289 @@
+"""
+ToolContext -- Unrestricted Tool Access for Reward Functions
+
+A per-rollout handle that gives reward/verification functions direct access to
+ALL hermes-agent tools, scoped to the rollout's task_id. The same task_id means
+the terminal/browser session is the SAME one the model used during its rollout --
+all state (files, processes, browser tabs) is preserved.
+
+The verifier author decides which tools to use. Nothing is hardcoded or gated.
+
+Example usage in a compute_reward():
+    async def compute_reward(self, item, result, ctx):
+        # Run tests in the model's terminal sandbox
+        test = ctx.terminal("pytest -v")
+        if test["exit_code"] == 0:
+            return 1.0
+
+        # Check if a file was created
+        content = ctx.read_file("/workspace/solution.py")
+        if content.get("content"):
+            return 0.5
+
+        return 0.0
+"""
+
+import json
+import logging
+import os
+from typing import Any, Dict, List, Optional
+
+import asyncio
+import concurrent.futures
+
+from model_tools import handle_function_call
+from tools.terminal_tool import cleanup_vm
+from tools.browser_tool import cleanup_browser
+
+logger = logging.getLogger(__name__)
+
+# Thread pool for running sync tool calls that internally use asyncio.run()
+_tool_executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
+
+
+def _run_tool_in_thread(tool_name: str, arguments: Dict[str, Any], task_id: str) -> str:
+    """
+    Run a tool call in a thread pool executor so backends that use asyncio.run()
+    internally (modal, docker) get a clean event loop.
+
+    If we're already in an async context, uses run_in_executor.
+    If not (e.g., called from sync code), runs directly.
+    """
+    try:
+        loop = asyncio.get_running_loop()
+        # We're in an async context -- need to run in thread
+        import concurrent.futures
+        with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
+            future = pool.submit(
+                handle_function_call, tool_name, arguments, task_id
+            )
+            return future.result(timeout=300)
+    except RuntimeError:
+        # No running event loop -- safe to call directly
+        return handle_function_call(tool_name, arguments, task_id)
+
+
+class ToolContext:
+    """
+    Open-ended access to all hermes-agent tools for a specific rollout.
+
+    Passed to compute_reward() so verifiers can use any tool they need:
+    terminal commands, file reads/writes, web searches, browser automation, etc.
+    All calls share the rollout's task_id for session isolation.
+    """
+
+    def __init__(self, task_id: str):
+        self.task_id = task_id
+
+    # -------------------------------------------------------------------------
+    # Terminal tools
+    # -------------------------------------------------------------------------
+
+    def terminal(self, command: str, timeout: int = 180) -> Dict[str, Any]:
+        """
+        Run a command in the rollout's terminal session.
+
+        Args:
+            command: Shell command to execute
+            timeout: Command timeout in seconds
+
+        Returns:
+            Dict with 'exit_code' (int) and 'output' (str)
+        """
+        import os
+        backend = os.getenv("TERMINAL_ENV", "local")
+        logger.debug("ToolContext.terminal [%s backend] task=%s: %s", backend, self.task_id[:8], command[:100])
+
+        # Run in thread pool so modal/docker backends' asyncio.run() doesn't deadlock
+        result = _run_tool_in_thread(
+            "terminal",
+            {"command": command, "timeout": timeout},
+            self.task_id,
+        )
+        try:
+            return json.loads(result)
+        except json.JSONDecodeError:
+            return {"exit_code": -1, "output": result}
+
+    # -------------------------------------------------------------------------
+    # File tools
+    # -------------------------------------------------------------------------
+
+    def read_file(self, path: str) -> Dict[str, Any]:
+        """
+        Read a file from the rollout's filesystem.
+
+        Args:
+            path: File path to read
+
+        Returns:
+            Dict with file content or error
+        """
+        result = handle_function_call(
+            "read_file", {"path": path}, task_id=self.task_id
+        )
+        try:
+            return json.loads(result)
+        except json.JSONDecodeError:
+            return {"error": result}
+
+    def write_file(self, path: str, content: str) -> Dict[str, Any]:
+        """
+        Write a file in the rollout's filesystem.
+
+        Args:
+            path: File path to write
+            content: Content to write
+
+        Returns:
+            Dict with success status or error
+        """
+        result = handle_function_call(
+            "write_file", {"path": path, "content": content}, task_id=self.task_id
+        )
+        try:
+            return json.loads(result)
+        except json.JSONDecodeError:
+            return {"error": result}
+
+    def search(self, query: str, path: str = ".") -> Dict[str, Any]:
+        """
+        Search for text in the rollout's filesystem.
+
+        Args:
+            query: Search query
+            path: Directory to search in
+
+        Returns:
+            Dict with search results
+        """
+        result = handle_function_call(
+            "search", {"query": query, "path": path}, task_id=self.task_id
+        )
+        try:
+            return json.loads(result)
+        except json.JSONDecodeError:
+            return {"error": result}
+
+    # -------------------------------------------------------------------------
+    # Web tools
+    # -------------------------------------------------------------------------
+
+    def web_search(self, query: str) -> Dict[str, Any]:
+        """
+        Search the web.
+
+        Args:
+            query: Search query
+
+        Returns:
+            Dict with search results
+        """
+        result = handle_function_call("web_search", {"query": query})
+        try:
+            return json.loads(result)
+        except json.JSONDecodeError:
+            return {"error": result}
+
+    def web_extract(self, urls: List[str]) -> Dict[str, Any]:
+        """
+        Extract content from URLs.
+
+        Args:
+            urls: List of URLs to extract content from
+
+        Returns:
+            Dict with extracted content
+        """
+        result = handle_function_call("web_extract", {"urls": urls})
+        try:
+            return json.loads(result)
+        except json.JSONDecodeError:
+            return {"error": result}
+
+    # -------------------------------------------------------------------------
+    # Browser tools
+    # -------------------------------------------------------------------------
+
+    def browser_navigate(self, url: str) -> Dict[str, Any]:
+        """
+        Navigate the rollout's browser session to a URL.
+
+        Args:
+            url: URL to navigate to
+
+        Returns:
+            Dict with page snapshot or error
+        """
+        result = handle_function_call(
+            "browser_navigate", {"url": url}, task_id=self.task_id
+        )
+        try:
+            return json.loads(result)
+        except json.JSONDecodeError:
+            return {"error": result}
+
+    def browser_snapshot(self) -> Dict[str, Any]:
+        """
+        Take a snapshot of the current browser page.
+
+        Returns:
+            Dict with page content/accessibility snapshot
+        """
+        result = handle_function_call(
+            "browser_snapshot", {}, task_id=self.task_id
+        )
+        try:
+            return json.loads(result)
+        except json.JSONDecodeError:
+            return {"error": result}
+
+    # -------------------------------------------------------------------------
+    # Generic tool access
+    # -------------------------------------------------------------------------
+
+    def call_tool(self, tool_name: str, arguments: Dict[str, Any]) -> str:
+        """
+        Call any hermes-agent tool by name.
+
+        This is the generic escape hatch -- if a tool doesn't have a convenience
+        wrapper above, you can call it directly here.
+
+        Args:
+            tool_name: Name of the tool (e.g., "vision_analyze", "skills_list")
+            arguments: Dict of arguments for the tool
+
+        Returns:
+            Raw JSON string result from the tool
+        """
+        return _run_tool_in_thread(tool_name, arguments, self.task_id)
+
+    # -------------------------------------------------------------------------
+    # Cleanup
+    # -------------------------------------------------------------------------
+
+    def cleanup(self):
+        """
+        Release all resources (terminal VMs, browser sessions) for this rollout.
+
+        Called automatically by the base environment via try/finally after
+        compute_reward() completes. You generally don't need to call this yourself.
+        """
+        try:
+            cleanup_vm(self.task_id)
+        except Exception as e:
+            logger.debug("VM cleanup for task %s: %s", self.task_id, e)
+
+        # Suppress browser_tool's noisy debug prints during cleanup.
+        # The cleanup still runs (safe), it just doesn't spam the console.
+        _prev_quiet = os.environ.get("HERMES_QUIET")
+        os.environ["HERMES_QUIET"] = "1"
+        try:
+            cleanup_browser(self.task_id)
+        except Exception as e:
+            logger.debug("Browser cleanup for task %s: %s", self.task_id, e)
+        finally:
+            if _prev_quiet is None:
+                os.environ.pop("HERMES_QUIET", None)
+            else:
+                os.environ["HERMES_QUIET"] = _prev_quiet

gateway/__init__.py

@@ -0,0 +1,35 @@
+"""
+Hermes Gateway - Multi-platform messaging integration.
+
+This module provides a unified gateway for connecting the Hermes agent
+to various messaging platforms (Telegram, Discord, WhatsApp) with:
+- Session management (persistent conversations with reset policies)
+- Dynamic context injection (agent knows where messages come from)
+- Delivery routing (cron job outputs to appropriate channels)
+- Platform-specific toolsets (different capabilities per platform)
+"""
+
+from .config import GatewayConfig, PlatformConfig, HomeChannel, load_gateway_config
+from .session import (
+    SessionContext,
+    SessionStore,
+    SessionResetPolicy,
+    build_session_context_prompt,
+)
+from .delivery import DeliveryRouter, DeliveryTarget
+
+__all__ = [
+    # Config
+    "GatewayConfig",
+    "PlatformConfig", 
+    "HomeChannel",
+    "load_gateway_config",
+    # Session
+    "SessionContext",
+    "SessionStore",
+    "SessionResetPolicy",
+    "build_session_context_prompt",
+    # Delivery
+    "DeliveryRouter",
+    "DeliveryTarget",
+]

gateway/config.py

@@ -0,0 +1,333 @@
+"""
+Gateway configuration management.
+
+Handles loading and validating configuration for:
+- Connected platforms (Telegram, Discord, WhatsApp)
+- Home channels for each platform
+- Session reset policies
+- Delivery preferences
+"""
+
+import os
+import json
+from pathlib import Path
+from dataclasses import dataclass, field
+from typing import Dict, List, Optional, Any
+from enum import Enum
+
+
+class Platform(Enum):
+    """Supported messaging platforms."""
+    LOCAL = "local"
+    TELEGRAM = "telegram"
+    DISCORD = "discord"
+    WHATSAPP = "whatsapp"
+
+
+@dataclass
+class HomeChannel:
+    """
+    Default destination for a platform.
+    
+    When a cron job specifies deliver="telegram" without a specific chat ID,
+    messages are sent to this home channel.
+    """
+    platform: Platform
+    chat_id: str
+    name: str  # Human-readable name for display
+    
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "platform": self.platform.value,
+            "chat_id": self.chat_id,
+            "name": self.name,
+        }
+    
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "HomeChannel":
+        return cls(
+            platform=Platform(data["platform"]),
+            chat_id=str(data["chat_id"]),
+            name=data.get("name", "Home"),
+        )
+
+
+@dataclass
+class SessionResetPolicy:
+    """
+    Controls when sessions reset (lose context).
+    
+    Modes:
+    - "daily": Reset at a specific hour each day
+    - "idle": Reset after N minutes of inactivity
+    - "both": Whichever triggers first (daily boundary OR idle timeout)
+    """
+    mode: str = "both"  # "daily", "idle", or "both"
+    at_hour: int = 4  # Hour for daily reset (0-23, local time)
+    idle_minutes: int = 120  # Minutes of inactivity before reset
+    
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "mode": self.mode,
+            "at_hour": self.at_hour,
+            "idle_minutes": self.idle_minutes,
+        }
+    
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "SessionResetPolicy":
+        return cls(
+            mode=data.get("mode", "both"),
+            at_hour=data.get("at_hour", 4),
+            idle_minutes=data.get("idle_minutes", 120),
+        )
+
+
+@dataclass
+class PlatformConfig:
+    """Configuration for a single messaging platform."""
+    enabled: bool = False
+    token: Optional[str] = None  # Bot token (Telegram, Discord)
+    api_key: Optional[str] = None  # API key if different from token
+    home_channel: Optional[HomeChannel] = None
+    
+    # Platform-specific settings
+    extra: Dict[str, Any] = field(default_factory=dict)
+    
+    def to_dict(self) -> Dict[str, Any]:
+        result = {
+            "enabled": self.enabled,
+            "extra": self.extra,
+        }
+        if self.token:
+            result["token"] = self.token
+        if self.api_key:
+            result["api_key"] = self.api_key
+        if self.home_channel:
+            result["home_channel"] = self.home_channel.to_dict()
+        return result
+    
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "PlatformConfig":
+        home_channel = None
+        if "home_channel" in data:
+            home_channel = HomeChannel.from_dict(data["home_channel"])
+        
+        return cls(
+            enabled=data.get("enabled", False),
+            token=data.get("token"),
+            api_key=data.get("api_key"),
+            home_channel=home_channel,
+            extra=data.get("extra", {}),
+        )
+
+
+@dataclass
+class GatewayConfig:
+    """
+    Main gateway configuration.
+    
+    Manages all platform connections, session policies, and delivery settings.
+    """
+    # Platform configurations
+    platforms: Dict[Platform, PlatformConfig] = field(default_factory=dict)
+    
+    # Session reset policies by type
+    default_reset_policy: SessionResetPolicy = field(default_factory=SessionResetPolicy)
+    reset_by_type: Dict[str, SessionResetPolicy] = field(default_factory=dict)
+    reset_by_platform: Dict[Platform, SessionResetPolicy] = field(default_factory=dict)
+    
+    # Reset trigger commands
+    reset_triggers: List[str] = field(default_factory=lambda: ["/new", "/reset"])
+    
+    # Storage paths
+    sessions_dir: Path = field(default_factory=lambda: Path.home() / ".hermes" / "sessions")
+    
+    # Delivery settings
+    always_log_local: bool = True  # Always save cron outputs to local files
+    
+    def get_connected_platforms(self) -> List[Platform]:
+        """Return list of platforms that are enabled and configured."""
+        connected = []
+        for platform, config in self.platforms.items():
+            if config.enabled and (config.token or config.api_key):
+                connected.append(platform)
+        return connected
+    
+    def get_home_channel(self, platform: Platform) -> Optional[HomeChannel]:
+        """Get the home channel for a platform."""
+        config = self.platforms.get(platform)
+        if config:
+            return config.home_channel
+        return None
+    
+    def get_reset_policy(
+        self, 
+        platform: Optional[Platform] = None,
+        session_type: Optional[str] = None
+    ) -> SessionResetPolicy:
+        """
+        Get the appropriate reset policy for a session.
+        
+        Priority: platform override > type override > default
+        """
+        # Platform-specific override takes precedence
+        if platform and platform in self.reset_by_platform:
+            return self.reset_by_platform[platform]
+        
+        # Type-specific override (dm, group, thread)
+        if session_type and session_type in self.reset_by_type:
+            return self.reset_by_type[session_type]
+        
+        return self.default_reset_policy
+    
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "platforms": {
+                p.value: c.to_dict() for p, c in self.platforms.items()
+            },
+            "default_reset_policy": self.default_reset_policy.to_dict(),
+            "reset_by_type": {
+                k: v.to_dict() for k, v in self.reset_by_type.items()
+            },
+            "reset_by_platform": {
+                p.value: v.to_dict() for p, v in self.reset_by_platform.items()
+            },
+            "reset_triggers": self.reset_triggers,
+            "sessions_dir": str(self.sessions_dir),
+            "always_log_local": self.always_log_local,
+        }
+    
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "GatewayConfig":
+        platforms = {}
+        for platform_name, platform_data in data.get("platforms", {}).items():
+            try:
+                platform = Platform(platform_name)
+                platforms[platform] = PlatformConfig.from_dict(platform_data)
+            except ValueError:
+                pass  # Skip unknown platforms
+        
+        reset_by_type = {}
+        for type_name, policy_data in data.get("reset_by_type", {}).items():
+            reset_by_type[type_name] = SessionResetPolicy.from_dict(policy_data)
+        
+        reset_by_platform = {}
+        for platform_name, policy_data in data.get("reset_by_platform", {}).items():
+            try:
+                platform = Platform(platform_name)
+                reset_by_platform[platform] = SessionResetPolicy.from_dict(policy_data)
+            except ValueError:
+                pass
+        
+        default_policy = SessionResetPolicy()
+        if "default_reset_policy" in data:
+            default_policy = SessionResetPolicy.from_dict(data["default_reset_policy"])
+        
+        sessions_dir = Path.home() / ".hermes" / "sessions"
+        if "sessions_dir" in data:
+            sessions_dir = Path(data["sessions_dir"])
+        
+        return cls(
+            platforms=platforms,
+            default_reset_policy=default_policy,
+            reset_by_type=reset_by_type,
+            reset_by_platform=reset_by_platform,
+            reset_triggers=data.get("reset_triggers", ["/new", "/reset"]),
+            sessions_dir=sessions_dir,
+            always_log_local=data.get("always_log_local", True),
+        )
+
+
+def load_gateway_config() -> GatewayConfig:
+    """
+    Load gateway configuration from multiple sources.
+    
+    Priority (highest to lowest):
+    1. Environment variables
+    2. ~/.hermes/gateway.json
+    3. cli-config.yaml gateway section
+    4. Defaults
+    """
+    config = GatewayConfig()
+    
+    # Try loading from ~/.hermes/gateway.json
+    gateway_config_path = Path.home() / ".hermes" / "gateway.json"
+    if gateway_config_path.exists():
+        try:
+            with open(gateway_config_path, "r") as f:
+                data = json.load(f)
+                config = GatewayConfig.from_dict(data)
+        except Exception as e:
+            print(f"[gateway] Warning: Failed to load {gateway_config_path}: {e}")
+    
+    # Override with environment variables
+    _apply_env_overrides(config)
+    
+    return config
+
+
+def _apply_env_overrides(config: GatewayConfig) -> None:
+    """Apply environment variable overrides to config."""
+    
+    # Telegram
+    telegram_token = os.getenv("TELEGRAM_BOT_TOKEN")
+    if telegram_token:
+        if Platform.TELEGRAM not in config.platforms:
+            config.platforms[Platform.TELEGRAM] = PlatformConfig()
+        config.platforms[Platform.TELEGRAM].enabled = True
+        config.platforms[Platform.TELEGRAM].token = telegram_token
+    
+    telegram_home = os.getenv("TELEGRAM_HOME_CHANNEL")
+    if telegram_home and Platform.TELEGRAM in config.platforms:
+        config.platforms[Platform.TELEGRAM].home_channel = HomeChannel(
+            platform=Platform.TELEGRAM,
+            chat_id=telegram_home,
+            name=os.getenv("TELEGRAM_HOME_CHANNEL_NAME", "Home"),
+        )
+    
+    # Discord
+    discord_token = os.getenv("DISCORD_BOT_TOKEN")
+    if discord_token:
+        if Platform.DISCORD not in config.platforms:
+            config.platforms[Platform.DISCORD] = PlatformConfig()
+        config.platforms[Platform.DISCORD].enabled = True
+        config.platforms[Platform.DISCORD].token = discord_token
+    
+    discord_home = os.getenv("DISCORD_HOME_CHANNEL")
+    if discord_home and Platform.DISCORD in config.platforms:
+        config.platforms[Platform.DISCORD].home_channel = HomeChannel(
+            platform=Platform.DISCORD,
+            chat_id=discord_home,
+            name=os.getenv("DISCORD_HOME_CHANNEL_NAME", "Home"),
+        )
+    
+    # WhatsApp (typically uses different auth mechanism)
+    whatsapp_enabled = os.getenv("WHATSAPP_ENABLED", "").lower() in ("true", "1", "yes")
+    if whatsapp_enabled:
+        if Platform.WHATSAPP not in config.platforms:
+            config.platforms[Platform.WHATSAPP] = PlatformConfig()
+        config.platforms[Platform.WHATSAPP].enabled = True
+    
+    # Session settings
+    idle_minutes = os.getenv("SESSION_IDLE_MINUTES")
+    if idle_minutes:
+        try:
+            config.default_reset_policy.idle_minutes = int(idle_minutes)
+        except ValueError:
+            pass
+    
+    reset_hour = os.getenv("SESSION_RESET_HOUR")
+    if reset_hour:
+        try:
+            config.default_reset_policy.at_hour = int(reset_hour)
+        except ValueError:
+            pass
+
+
+def save_gateway_config(config: GatewayConfig) -> None:
+    """Save gateway configuration to ~/.hermes/gateway.json."""
+    gateway_config_path = Path.home() / ".hermes" / "gateway.json"
+    gateway_config_path.parent.mkdir(parents=True, exist_ok=True)
+    
+    with open(gateway_config_path, "w") as f:
+        json.dump(config.to_dict(), f, indent=2)

gateway/delivery.py

@@ -0,0 +1,318 @@
+"""
+Delivery routing for cron job outputs and agent responses.
+
+Routes messages to the appropriate destination based on:
+- Explicit targets (e.g., "telegram:123456789")
+- Platform home channels (e.g., "telegram" → home channel)
+- Origin (back to where the job was created)
+- Local (always saved to files)
+"""
+
+import json
+from pathlib import Path
+from datetime import datetime
+from dataclasses import dataclass
+from typing import Dict, List, Optional, Any, Union
+from enum import Enum
+
+from .config import Platform, GatewayConfig, HomeChannel
+from .session import SessionSource
+
+
+@dataclass
+class DeliveryTarget:
+    """
+    A single delivery target.
+    
+    Represents where a message should be sent:
+    - "origin" → back to source
+    - "local" → save to local files
+    - "telegram" → Telegram home channel
+    - "telegram:123456" → specific Telegram chat
+    """
+    platform: Platform
+    chat_id: Optional[str] = None  # None means use home channel
+    is_origin: bool = False
+    is_explicit: bool = False  # True if chat_id was explicitly specified
+    
+    @classmethod
+    def parse(cls, target: str, origin: Optional[SessionSource] = None) -> "DeliveryTarget":
+        """
+        Parse a delivery target string.
+        
+        Formats:
+        - "origin" → back to source
+        - "local" → local files only
+        - "telegram" → Telegram home channel
+        - "telegram:123456" → specific Telegram chat
+        """
+        target = target.strip().lower()
+        
+        if target == "origin":
+            if origin:
+                return cls(
+                    platform=origin.platform,
+                    chat_id=origin.chat_id,
+                    is_origin=True,
+                )
+            else:
+                # Fallback to local if no origin
+                return cls(platform=Platform.LOCAL, is_origin=True)
+        
+        if target == "local":
+            return cls(platform=Platform.LOCAL)
+        
+        # Check for platform:chat_id format
+        if ":" in target:
+            platform_str, chat_id = target.split(":", 1)
+            try:
+                platform = Platform(platform_str)
+                return cls(platform=platform, chat_id=chat_id, is_explicit=True)
+            except ValueError:
+                # Unknown platform, treat as local
+                return cls(platform=Platform.LOCAL)
+        
+        # Just a platform name (use home channel)
+        try:
+            platform = Platform(target)
+            return cls(platform=platform)
+        except ValueError:
+            # Unknown platform, treat as local
+            return cls(platform=Platform.LOCAL)
+    
+    def to_string(self) -> str:
+        """Convert back to string format."""
+        if self.is_origin:
+            return "origin"
+        if self.platform == Platform.LOCAL:
+            return "local"
+        if self.chat_id:
+            return f"{self.platform.value}:{self.chat_id}"
+        return self.platform.value
+
+
+class DeliveryRouter:
+    """
+    Routes messages to appropriate destinations.
+    
+    Handles the logic of resolving delivery targets and dispatching
+    messages to the right platform adapters.
+    """
+    
+    def __init__(self, config: GatewayConfig, adapters: Dict[Platform, Any] = None):
+        """
+        Initialize the delivery router.
+        
+        Args:
+            config: Gateway configuration
+            adapters: Dict mapping platforms to their adapter instances
+        """
+        self.config = config
+        self.adapters = adapters or {}
+        self.output_dir = Path.home() / ".hermes" / "cron" / "output"
+    
+    def resolve_targets(
+        self,
+        deliver: Union[str, List[str]],
+        origin: Optional[SessionSource] = None
+    ) -> List[DeliveryTarget]:
+        """
+        Resolve delivery specification to concrete targets.
+        
+        Args:
+            deliver: Delivery spec - "origin", "telegram", ["local", "discord"], etc.
+            origin: The source where the request originated (for "origin" target)
+        
+        Returns:
+            List of resolved delivery targets
+        """
+        if isinstance(deliver, str):
+            deliver = [deliver]
+        
+        targets = []
+        seen_platforms = set()
+        
+        for target_str in deliver:
+            target = DeliveryTarget.parse(target_str, origin)
+            
+            # Resolve home channel if needed
+            if target.chat_id is None and target.platform != Platform.LOCAL:
+                home = self.config.get_home_channel(target.platform)
+                if home:
+                    target.chat_id = home.chat_id
+                else:
+                    # No home channel configured, skip this platform
+                    continue
+            
+            # Deduplicate
+            key = (target.platform, target.chat_id)
+            if key not in seen_platforms:
+                seen_platforms.add(key)
+                targets.append(target)
+        
+        # Always include local if configured
+        if self.config.always_log_local:
+            local_key = (Platform.LOCAL, None)
+            if local_key not in seen_platforms:
+                targets.append(DeliveryTarget(platform=Platform.LOCAL))
+        
+        return targets
+    
+    async def deliver(
+        self,
+        content: str,
+        targets: List[DeliveryTarget],
+        job_id: Optional[str] = None,
+        job_name: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> Dict[str, Any]:
+        """
+        Deliver content to all specified targets.
+        
+        Args:
+            content: The message/output to deliver
+            targets: List of delivery targets
+            job_id: Optional job ID (for cron jobs)
+            job_name: Optional job name
+            metadata: Additional metadata to include
+        
+        Returns:
+            Dict with delivery results per target
+        """
+        results = {}
+        
+        for target in targets:
+            try:
+                if target.platform == Platform.LOCAL:
+                    result = self._deliver_local(content, job_id, job_name, metadata)
+                else:
+                    result = await self._deliver_to_platform(target, content, metadata)
+                
+                results[target.to_string()] = {
+                    "success": True,
+                    "result": result
+                }
+            except Exception as e:
+                results[target.to_string()] = {
+                    "success": False,
+                    "error": str(e)
+                }
+        
+        return results
+    
+    def _deliver_local(
+        self,
+        content: str,
+        job_id: Optional[str],
+        job_name: Optional[str],
+        metadata: Optional[Dict[str, Any]]
+    ) -> Dict[str, Any]:
+        """Save content to local files."""
+        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+        
+        if job_id:
+            output_path = self.output_dir / job_id / f"{timestamp}.md"
+        else:
+            output_path = self.output_dir / "misc" / f"{timestamp}.md"
+        
+        output_path.parent.mkdir(parents=True, exist_ok=True)
+        
+        # Build the output document
+        lines = []
+        if job_name:
+            lines.append(f"# {job_name}")
+        else:
+            lines.append("# Delivery Output")
+        
+        lines.append("")
+        lines.append(f"**Timestamp:** {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
+        
+        if job_id:
+            lines.append(f"**Job ID:** {job_id}")
+        
+        if metadata:
+            for key, value in metadata.items():
+                lines.append(f"**{key}:** {value}")
+        
+        lines.append("")
+        lines.append("---")
+        lines.append("")
+        lines.append(content)
+        
+        output_path.write_text("\n".join(lines))
+        
+        return {
+            "path": str(output_path),
+            "timestamp": timestamp
+        }
+    
+    async def _deliver_to_platform(
+        self,
+        target: DeliveryTarget,
+        content: str,
+        metadata: Optional[Dict[str, Any]]
+    ) -> Dict[str, Any]:
+        """Deliver content to a messaging platform."""
+        adapter = self.adapters.get(target.platform)
+        
+        if not adapter:
+            raise ValueError(f"No adapter configured for {target.platform.value}")
+        
+        if not target.chat_id:
+            raise ValueError(f"No chat ID for {target.platform.value} delivery")
+        
+        # Call the adapter's send method
+        # Adapters should implement: async def send(chat_id: str, content: str) -> Dict
+        return await adapter.send(target.chat_id, content, metadata=metadata)
+
+
+def parse_deliver_spec(
+    deliver: Optional[Union[str, List[str]]],
+    origin: Optional[SessionSource] = None,
+    default: str = "origin"
+) -> Union[str, List[str]]:
+    """
+    Normalize a delivery specification.
+    
+    If None or empty, returns the default.
+    """
+    if not deliver:
+        return default
+    return deliver
+
+
+def build_delivery_context_for_tool(
+    config: GatewayConfig,
+    origin: Optional[SessionSource] = None
+) -> Dict[str, Any]:
+    """
+    Build context for the schedule_cronjob tool to understand delivery options.
+    
+    This is passed to the tool so it can validate and explain delivery targets.
+    """
+    connected = config.get_connected_platforms()
+    
+    options = {
+        "origin": {
+            "description": "Back to where this job was created",
+            "available": origin is not None,
+        },
+        "local": {
+            "description": "Save to local files only",
+            "available": True,
+        }
+    }
+    
+    for platform in connected:
+        home = config.get_home_channel(platform)
+        options[platform.value] = {
+            "description": f"{platform.value.title()} home channel",
+            "available": True,
+            "home_channel": home.to_dict() if home else None,
+        }
+    
+    return {
+        "origin": origin.to_dict() if origin else None,
+        "options": options,
+        "always_log_local": config.always_log_local,
+    }