Add optional x/y parameters to browser_click for viewport-coordinate
clicking via CDP Input.dispatchMouseEvent. When coordinates are provided,
clicks are dispatched at the browser compositor level — Chrome does its own
hit-testing, bypassing DOM selectors entirely.
Use cases where ref-based click fails but coordinate click works:
- Cross-origin iframes (OOPIFs)
- Closed shadow DOM
- Canvas/WebGL elements
- Dynamic overlays where the snapshot may be stale
Implementation:
- CDP path (preferred): Input.dispatchMouseEvent via WebSocket
(Target.getTargets + mousePressed + mouseReleased)
- agent-browser fallback: mouse move/down/up when no CDP endpoint available
- ref is no longer required — either ref OR x+y must be provided
Benchmark (real Lightpanda WS at ws://127.0.0.1:63372, 200 iterations):
CDP coord click: 3.71ms mean (2.97ms median, 2.61ms min, 7.01ms p95)
Single WS conn baseline: 1.57ms mean (cost per connection open+call)
agent-browser IPC: 0.20ms mean per HTTP call
The 3.71ms per CDP click comes from 3 sequential fresh WS connections
(pre-existing architecture in browser_cdp_tool.py). A persistent WS
connection pool would bring this to ~3.1ms (just the 2 mouse events).
Both paths are well under the 100ms human perception threshold.
Files:
- tools/browser_tool.py: schema update (x/y, ref no longer required),
_cdp_coordinate_click(), _coordinate_click_via_agent_browser(),
updated browser_click() with validation and dispatch
- tests/tools/test_browser_coordinate_click.py: 21 tests covering
validation, CDP path, fallback path, ref preservation, schema, registry
- scripts/benchmark_click_paths.py: real-browser latency benchmark
kshitijk4poor