Explorer
Roku Explorer: Autonomous Testing with RL & RAG
Autonomous Exploratory Testing Platform
A production autonomous QA platform for firmware and Grand Central product areas. Combines PPO/LSTM reinforcement learning, ChromaDB multimodal RAG, dual validators, and MCP-based device control to navigate device UIs, surface potential issues, and flag visual regressions with minimal human oversight.
The Core Challenge
Engineering Solution & Implementation
Designed a multi-service Docker Compose stack (API, worker, UI, dual validators, scheduler, MCP) around an autonomous agent loop: PPO/LSTM policies plus multimodal vision reasoning explore device UIs, while change-driven planning targets high-risk paths and MCP commands physical test devices. UI states are indexed in ChromaDB for retrieval-augmented reasoning across firmware and Grand Central product areas.
Figure 1: High-level architectural dataflow of Roku Explorer's change-driven autonomous loop.
Repo diffs route into scenario synthesis and then into the Explorer core, which commands the device fleet over MCP. ChromaDB stores UI state, and screen frames return to the policy so the loop can continue. Dual validators record validation signals from that loop.
My contribution
I designed the platform and still lead it, including the change-driven path from code diffs to device exploration with PPO/LSTM policies, multimodal retrieval, MCP device control, and dual validators. Roughly 70% of the version-control history is mine, across a 17-engineer contributor base and ~4,000 commits.
Tradeoff
The 16,266 figure counts validation signals. Those signals are not confirmed defects and not duplicate reports.
Verified Outcomes & Deliverables
In production for ~16 months with 249,868 device actions across 1,065 runs on 92 devices, serving 30+ users and surfacing 16,266 validation signals over a 1,993-node / 5,966-transition exploration graph. Those 16,266 validation signals are not confirmed defects and not duplicate reports.
Designed and still lead the platform — roughly 70% of commit history across a 17-engineer contributor base (~4,000 commits).
Closed the loop from code change → automated risk scenario generation → device exploration with dual validators and MCP device control.