Engineering Journal
ML Systems DRAFT

Debugging Browser-Based ML Inference: Lessons from ONNX Runtime Web

Running MobileViT inside a browser extension means WebGPU when you can get it, WASM when you can't, and failure modes that only exist at the intersection.

Pavan Ganesh Indupudi (Infernix) Published 2026-10-19 Draft — incomplete 6 min read
This article is a draft. The verified background sections below come from the PRIVYSE repository's documented design. The debugging sections are marked TODO — the specific failure stories, error messages, and fixes have not been verified against notes or logs yet and will not be published as facts until they are.

PRIVYSE runs a quantized MobileViT-Small inside the browser — not on a server, not in a worker talking to a GPU daemon, but through ONNX Runtime Web in the extension itself. The model classifies screenshot tiles so the privacy pipeline knows which regions need Tier C protection. This article is meant to collect the debugging lessons from making that reliable. The setup is verified; the war stories are still being checked against notes, so they are stubs for now.

The setup (verified)

What is documented in the PRIVYSE repo and can be stated as fact:

  • Model: MobileViT-Small, quantized to q8 ONNX, 6.3 MB, bundled in the extension (extension/public/models/) — no CDN downloads, ever.
  • Runtime: ONNX Runtime Web with the WebGPU execution provider first. When navigator.gpu is present, tiles classify on the GPU; any op the WebGPU EP lacks falls through to the bundled WASM EP. The same .jsep runtime serves both paths, so nothing is re-downloaded on fallback.
  • Host context: Chrome MV3 service workers have no DOM, so the vision stack (MobileViT + BlazeFace + Tesseract.js) runs in a hidden offscreen document. Firefox MV2 has neither the offscreen API nor the side panel API — the background page is the DOM host, sharing the same core/vision-host.ts.
  • Visibility: the active provider is surfaced live in the side panel resource meter (perception · ⚡ WebGPU / wasm), and reported in perception stats alongside tile tags.
  • Caching: a pixel-hash LRU reuses the tile map when the page has not visually changed — cached perception ≈ 0 ms vs ≈ 1.4 s first run. Faces, OCR, and the privacy gate still run fresh every step.
  • Measured accuracy: ViT page classification 7/7 on the labeled benchmark, photo-vs-blank 100%.
javascript — provider selection as documented (illustrative shape) 
// WebGPU first, WASM fallback — same bundled .jsep runtime
const hasWebGPU = typeof navigator !== "undefined" && !!navigator.gpu;
const providers = hasWebGPU ? ["webgpu", "wasm"] : ["wasm"];
// active provider is reported to the side-panel resource meter
Note on the snippet above

This code block illustrates the documented behavior (WebGPU-first with WASM fallback, provider reported to the UI). It is not copied from the source tree. The verified implementation lives in extension/core/perception.ts in the PRIVYSE repo.

Failure modes worth writing up (TODO)

The following debugging sessions happened while building the perception stack. Each one needs its notes re-checked before it can be written as fact:

TODO — WebGPU init failures in offscreen documents

Notes exist on at least one environment where navigator.gpu was present but WebGPU device requests failed inside the MV3 offscreen document, silently dropping to WASM with a latency jump. To verify: exact browser version, the error surfaced, whether requestAdapter() returned null, and the fallback log line. Do not publish until the console capture is recovered.

TODO — unsupported ops forcing partial WASM execution

Some MobileViT ops were expected to miss the WebGPU EP and fall through. To verify: which ops, whether ORT logged partition boundaries, and the measured GPU-vs-WASM tile-classification latency delta on the same page.

TODO — WASM memory growth across long agent runs

Long-running sessions appeared to accumulate memory in the WASM heap. To verify: whether session reuse or per-step session creation was the cause, the observed growth curve, and what the current code does (release vs reuse).

TODO — pixel-hash cache false hits

The same-page cache keys on a pixel hash; dynamic pages (clocks, spinners) were expected to thrash it, and near-identical frames to false-hit it. To verify: whether any false hit was actually observed in benchmarks, the hash scheme, and the eviction policy in the LRU.

Debugging playbook (TODO — outline only)

The shape of the playbook this article will end with, pending verification:

  1. Always log the active execution provider per step — silent fallback is the root cause of most "it got slow" reports.
  2. Keep a WASM-only baseline measurement per machine; compare, don't assume.
  3. Test the offscreen-document path and the Firefox background-page path separately; they fail differently.
  4. Treat cache hits as untrusted until the hash scheme is re-read — measure first-run vs cached latencies in CI-style runs.

Takeaways (provisional)

Two things are already safe to claim because they are documented and benchmarked, not anecdotal:

  1. Surface the execution provider in the UI. Making WebGPU/WASM visible in the resource meter turned "why is it slow?" from a mystery into a screenshot.
  2. Bundle both runtimes. Shipping the .jsep build with WASM fallback inside the extension means first-run latency never depends on a CDN, and fallback never depends on a network fetch.

References

AI Agents · Privacy Engineering Building PRIVYSE: Designing a Privacy-First Browser Agent RAG · ML Systems Inside VOICE-RAG: From Speech to Grounded Answers Portfolio All projects — PRIVYSE, VOICE-RAG, SourceTrail, Viora