Keeping Web AR Fast: Bundle Budgets, Lazy Loading and Adaptive Tracking

Real-time augmented reality in the browser is heavy. A virtual try-on needs a WebAssembly runtime, machine learning models measured in megabytes, WebGL shaders and sometimes a full 3D engine. Yet it lives on product pages, where every kilobyte competes with the photos, price and “Add to cart” button that actually sell the product.

When I built TryOnIt, I worked from one rule: nothing heavy loads until a shopper opens try-on, and then only the pieces the current product needs. This article covers how that rule is implemented and enforced.

Budgets that fail the build

Performance goals that are not measured on every change quietly regress. Each TryOnIt entry point has a size budget, minified and gzipped, checked by size-limit as part of the release checks:

Entry point	Budget	Measured
@tryonit/core, typical import	8 KB	about 5.9 KB
@tryonit/core, entire public API	14 KB	about 12.9 KB
@tryonit/web initial entry (MediaPipe and three.js excluded)	25 KB	about 22.6 KB
@tryonit/react	15 KB	about 7.9 KB
@tryonit/react/styles.css	6 KB	about 2.5 KB
@tryonit/web/styles.css (vanilla mount() UI)	3 KB	about 0.9 KB

The core budget taught me something about how to set budgets. My original target was 8 KB for the entire core. Measuring every export at once, which includes a validator for 15 asset types, the anchor math, the filters, the homography and the resolver, came to about 12.9 KB. Rather than pretend, I split it into two budgets: 14 KB for the whole API, and a second check that keeps a typical tree-shaken import (validateManifest, createSessionStore, resolveAsset) under the original 8 KB.

The second number is the one users actually pay. A budget should measure what people import, not what the package exports.

Load only what the product needs

The biggest savings do not come from shaving bytes but from not loading code at all. Every asset type declares what it requires, and the engine maps that to lazily imported chunks:

Asset types	Tracker	Renderer	Chunks loaded on first use
makeup.*	face	makeup (WebGL2)	MediaPipe runtime, face model, GL renderers
hair.color	segmenter	hair (WebGL2)	MediaPipe runtime, hair model, GL renderers
face.overlay2d	face	2D overlay (WebGL2)	MediaPipe runtime, face model, GL renderers
glasses, hat, earrings	face	three.js	MediaPipe runtime, face model, three.js
watch, ring	hand	three.js	MediaPipe runtime, hand model, three.js
clothing.top	pose	2D overlay (WebGL2)	MediaPipe runtime, pose model, GL renderers

The costs behind those chunks are very different:

• The MediaPipe JavaScript runtime is about 45 KB gzipped, plus its WebAssembly files, and loads the first time any tracker is needed.
• Each model file (.task or .tflite) weighs 1 to 8 MB. It downloads once and stays in memory.
• The GL renderers load for makeup, hair, stickers and clothing.
• three.js and the 3D renderer load only for glasses, hats, earrings, watches and rings. It is an optional peer dependency, so a makeup-only store never ships it.

Trackers and renderers stay warm until the session is destroyed. Switching between two lipsticks loads nothing. Switching from a lipstick to glasses loads only three.js, because the face tracker is already running.

Warm up before the click

Lazy loading has a cost: the first open of try-on has to fetch the runtime and a model. TryOnIt hides most of that latency by starting the download when the shopper signals intent, not when they click.

The React button preloads on hover or focus by default, or when it scrolls into view:

<TryOnButton asset={asset} preload="hover" />   // default: warm on hover or focus
<TryOnButton asset={asset} preload="visible" /> // warm when the button scrolls into view

Outside React, or for a whole catalogue page, preloadTryOn warms the runtime, models, chunks and product files ahead of time:

import { preloadTryOn } from '@tryonit/web';

preloadTryOn(['/assets/aviator.json', '/assets/ruby.json'], { modelBaseUrl: '/models' });

Detect when needed, render every frame

Inside a running session, the expensive operation is detection: running a neural network on a camera frame. Rendering the result is comparatively cheap. TryOnIt treats them as separate loops:

• Rendering runs on requestAnimationFrame, every frame, using the latest smoothed pose.
• Detection runs on requestVideoFrameCallback where the browser supports it, so it happens exactly once per new camera frame instead of re-processing a frame the camera has not replaced yet.
• Detection input is downscaled to 640 px on the long side, independent of the 720p display. The model does not need full resolution to find a face.
• MediaPipe runs on the GPU delegate with an automatic fallback to the CPU.

The detection rate adapts to the device. The target is 30 detections per second; when detection plus rendering exceeds the frame budget, the engine drops to 15, and it recovers after three stable seconds. Apps can choose a strategy:

Mode	Detection rate	Use it when
auto	30, dropping to 15	You want the engine to adapt (the default)
quality	fixed 30	Smooth tracking matters more than battery life
balanced	24, dropping to 15	You want to cap the load on every device
battery	fixed 15	You target kiosks or low-end devices

import { createTryOnEngine } from '@tryonit/web';

const engine = createTryOnEngine({ container, performance: 'balanced' });

Because rendering keeps going at full speed with the last smoothed pose, a lower detection rate shows up as slightly less responsive tracking rather than a choppy picture.

Keep React out of the hot path

In @tryonit/react, per-frame data never goes through React state. Landmarks and anchors stay inside the engine, and the store updates its performance numbers at most twice per second. Components subscribe to narrow slices:

const faceVisible = useTryOnState((s) => s.tracking.faceVisible);
const perf = useTryOnState((s) => s.perf, shallowEqual);

useTryOnState re-renders only when the selected value changes, so a “look at the camera” hint re-renders when the face is found or lost, not 30 times a second. I explain the full split between the fast path and UI state in the architecture article.

Clean up like you mean it

Performance is also about what happens after the feature is used. A camera left running drains batteries and keeps the browser’s camera indicator on, which shoppers rightly find alarming.

• The session pauses automatically when the tab is hidden.
• The camera stops when the dialog closes.
• destroy() stops every camera track, closes the MediaPipe tasks, disposes WebGL resources and removes event listeners.

A test mounts and unmounts the React component 20 times in StrictMode, which double-invokes effects on purpose, and checks that no camera tracks or listeners are left behind. Leaks in camera code tend to appear exactly in these mount and unmount races, so it is worth testing them directly.

Why tracking does not run in a Web Worker (yet)

Moving tracking into a Web Worker looks like an obvious win, since it would free the main thread. It is planned behind the existing Tracker interface, but it is not the default today for two reasons: iOS Safari has limited support for WebGL on an OffscreenCanvas inside workers, and transferring camera frames to a worker adds latency. For a feature whose whole value is that the product moves with your face, latency is the wrong trade.

A production checklist

• Self-host the models next to your site with modelBaseUrl, and serve them with long cache headers. This also removes third-party requests, which I cover in Privacy by Design for Camera Features.
• Keep GLB files under 1 MB with Meshopt or Draco compression. The details are in Preparing 3D Models for AR Try-On.
• Use performance: 'battery' for kiosks and low-end devices.
• Turn on debug to see FPS, detection time, render time, detection rate and loaded modules while you tune.

You can install the engine with npm install @tryonit/web three from npm, or the React components from @tryonit/react.