Token-by-token streaming of LLM responses, disabled by default.
Enable via streaming.enabled: true in config.yaml or
HERMES_STREAMING_ENABLED=true env var.
Core (run_agent.py):
- stream_callback parameter on AIAgent
- _run_streaming_chat_completion() for Chat Completions streaming
- _run_codex_stream() now emits tokens via callback
- _interruptible_api_call routes to streaming when callback is set
- Graceful fallback to non-streaming on any error
Gateway (gateway/run.py):
- Read streaming config (master switch + per-platform overrides)
- Queue-based callback bridging agent thread to async event loop
- stream_preview task: progressive message editing with cursor
- Skip normal send when streaming already delivered the response
- Thread-safe, respects platform rate limits (1.5s edit interval)
API Server (gateway/platforms/api_server.py):
- Real token-by-token SSE when stream=true (replaces pseudo-streaming)
- stream_callback wired through _create_agent and _run_agent
- Background agent task + queue for concurrent streaming
Config:
- streaming.enabled (master switch, default: false)
- Per-platform: streaming.telegram, streaming.discord, etc.
- HERMES_STREAMING_ENABLED env var override
teknium1
5a426e084a