Closes#33163.
When _try_activate_fallback() switches from one provider to another (e.g.
openai-codex → openrouter), the credential pool still belongs to the
primary provider. This causes two compounding bugs:
1. The pool retains the primary's base_url. Downstream pool recovery
(rate_limit / billing / auth) calls _swap_credential() with a primary
entry which overwrites the agent's base_url back to the primary's
endpoint. Every fallback request then 404s against the wrong host.
2. Pool recovery acting on errors from the FALLBACK provider mutates the
PRIMARY's pool state (#33088 reported a related corruption pattern),
exhausting/rotating entries that have nothing to do with the failure.
Two layered fixes:
a) try_activate_fallback (agent/chat_completion_helpers.py): on fallback
activation, clear agent._credential_pool when the fallback provider
doesn't match the pool's provider. Pool is preserved when the fallback
shares the pool's provider (e.g. multiple openrouter entries).
b) recover_with_credential_pool (agent/agent_runtime_helpers.py):
defensive guard rejects any pool mutation when agent.provider doesn't
match pool.provider. Defense-in-depth — should never fire after (a)
is in place, but covers any future path that attaches a stale pool.
Salvaged from @zccyman's PR #33217. The original PR was written against
the pre-refactor monolithic run_agent.py; both target functions have
since been extracted to module-level helpers. Behavior is identical —
the guards live in the canonical extracted locations.
Tests
- New tests/run_agent/test_fallback_credential_isolation.py (7 tests
covering: fallback clears mismatched pool, fallback preserves matching
pool, recovery rejects mismatched pool, recovery accepts matching
pool, 429-from-z.ai-doesn't-exhaust-codex-pool, _client_kwargs
base_url survives pool clear, _swap_credential doesn't restore
primary URL after fallback).
- Cross-verified: 77/77 passing across fallback isolation tests +
agent/test_credential_pool.py — no regression.
Co-authored-by: zccyman <16263913+zccyman@users.noreply.github.com>
zccyman
2e181602a1