SplatShop rendering a Gaussian splat scene
github.com/NarinderBrar/SplatShop
Project 2 · Open source

SplatShop

A WebGPU-first renderer for 3D Gaussian splats: photoreal scenes made of millions of soft, transparent blobs, which have to be sorted and blended every frame in a browser.

WebGPU / WGSLWebGL 2 fallbackTypeScriptRust → WASMWeb WorkersBabylon.js
5.7Msplats in a test scene
70+documented optimizations
13×less upload per sort
6file formats
What I built

Features

01EWA splat rendering

Full 3D covariance projected to a screen-space ellipse per splat, drawn as batched instanced quads.

02GPU radix sort

Depth keys and histogram → prefix → scatter on the GPU; a CPU worker sort as fallback.

03SSOG LOD streaming

A hierarchical spatial LOD format with on-demand chunk loading under a splat budget.

04GPU visibility

Chunk frustum culling and Hi-Z occlusion in compute, with a compacted visible list.

05Memory system

GPU page pool, buffer arenas, readback and typed-array pools, dirty-range uploads.

06Formats

.ply .splat .spz .ksplat .sog .ssog, with packed data decoded in the shader.

07Quality adaptation

5 presets, device tiers, DPR cap, temporal jitter accumulation, reverse-Z depth.

08Debug tooling

Tile heatmaps, chunk colours, sort stats, and de-duplicated WebGPU error capture.

09Editing tools

GPU nearest-splat and colour selection passes exist; select, move and paint tools are designed in the UI.

Architecture

The frame pipeline: CPU decides, worker selects, GPU does the heavy work

MAIN WORKER GPU loaderSOG ranges · Morton streaming queuepriority · budget decoded chunks GPU pagepool ≤ budget/frame camera LOD selectionRust → WASM · budget if moved selected (node, lod)transferable buffer chunk cullfrustum · compact Hi-Z testocclusion depth keys20-bit radix sort5 × 4-bit raster128 / instance postHalton jitter resident chunks the main thread only schedules · heavy CPU work is in WASM · per-splat work is on the GPU
Each stage has a budget: LOD reruns every 15 frames only if the camera moved, uploads are capped per frame, and sorting is skipped while the camera is still. A CPU sort worker and a WebGL 2 path cover devices without WebGPU compute.
Architecture decisions

What I chose, and what I didn't

DecisionRejectedWhy
Sort an index bufferReorder every attribute array~23 MB instead of ~298 MB per sort; the splat data never moves
GPU radix sortCPU sort + uploadNo readback, no upload; parallel and comparison-free
Packed data, decoded in the shaderExpanded float attributes~20 B vs 52 B per splat; bandwidth is the real limit for splats
Budget-driven LODFixed distance thresholdsPredictable frame cost on any device and any scene size
LOD in Rust → WASM, in a workerTypeScript on the main threadTight loop, no GC pauses; the main thread stays free; zero-copy results
Coarse fallback while streamingShow nothing until loadedNever a hole on screen; fine detail replaces coarse when ready
WebGPU first, WebGL 2 fallbackWebGPU onlyCompute where available, still runs everywhere
Feature · streaming format

SSOG: a spatial tree whose nodes point at byte ranges

spatial nodes (nested boxes) node 5 far → coarsenear: fine node 5 · chunk entries LOD 2 · coarsest, fewest splats LOD 1 LOD 0 · finest packed file · entry = { nodeId, lod, fileIndex, offset, count, bound } HTTP range select: start coarsest → upgrade by screen radius · distance · view · foveation until the budget finer chunk not loaded yet → keep the coarse one visible → no holes hysteresis ×1.15 for selected nodes · cost^0.55 favours cheap upgrades
The index is small and known up front; the bytes are fetched only for selected (node, lod) pairs. It's the same shape as map tiles and COG overviews: an index, byte ranges, detail by distance.
Feature · GPU sorting

Depth keys + radix sort, entirely on the GPU

① depth key per splat camera forward key = dot(center − cam, fwd) normalised → 20-bit integer ② one pass · 4-bit digit keys 3a30fa13 histogram 16 buckets prefix 0 1 2 2 5 … offset per bucket scatter 01333aaf stable: equal keys keep their order ③ five passes, ping-pong buffer A buffer B pass 1…5 20 bits ÷ 4 = 5 passes bits adapt to splat count indices move, data doesn't
Radix sort needs no comparisons, so every step is parallel: count per workgroup, prefix-sum the counts into offsets, scatter each key to its slot. The sort is skipped when the camera hasn't moved, and camera inertia is off so the view never drifts between sorts.
Feature · streaming & memory

What loads first, and what gets evicted

priority lanes ① fallback · coarse ② selected · visible ③ prefetch · wider ④ idle background no longer selected → dropped from queue upload gate ≤ N bytes and≤ K chunks / frame no frame stalls GPU page pool · first-fit fallback pressure > 98% or over limit → evict first: behind camera · idle prefetch · unused 4–8 frames never evicted: visible and fallback chunks CPU decoded cache · LRU by lastUsedFrame freed after GPU eviction · LRU · until under budget
The rule: show a rough version of what's visible first, then make it sharp, then prepare what's nearby, and only use spare time for the rest. Memory stays bounded because every cache has a budget and an eviction order.
Feature · rendering

Batched instancing: 128 splats per instance

one instance = 128 quads … ×128 vertex.xy = corner · vertex.z = localId LOD chunk 4,096 splats= 32 instances order = instance × 128 + localId sorted index buffer (from the radix sort) splat data (storage buffers, packed) center quat scale color EWA: 3D covariance → 2D ellipse Σ′ = J W Σ Wᵀ Jᵀ alpha = opacity · exp(−r²) · premultiplied
Batching 128 splats per instance keeps the instance count small while the shader still reads each splat through the sorted index, so the data is never duplicated per vertex. Quads are sized to the projected ellipse, which avoids wasting fragments.
Memory & buffer packing

52 bytes → ~20 bytes per splat, decoded in the shader

expanded · 52 B / splat (float32)12 px = 1 B position 12 color 12 α 4 covA 16 covB 8 5.72M × 52 B ≈ 298 MB packed SOG · ~20 B / splat mLmUquatsclsh0 means: 16-bit per axis split into lo / hi bytes, log-encoded in [min, max] quat: smallest-three · 3 × 8 bit + which component was dropped scale · colour: 8-bit indices into small codebooks inside meansL / meansU / quat (RGBA8 words) x loy loz lo— x hiy hiz hi— abcidx center = lerp(min, max, (hi·256 + lo) / 65535) sort index · 4 B u32 5.72M × 4 B ≈ 23 MB · generated on the GPU state · 1 × u32 bitfield SEL HID LCK FLT DEL · 27 bits free dirty-range upload: only changed words go to the GPU
The shader expands packed values into registers per vertex: a little more ALU work for far less VRAM and bandwidth. For splats, bandwidth is the real limit.
Debugging story

The WebGL context kept dying

My first renderer (a Three.js prototype) lost the WebGL context when I moved the camera over a 320 MB, 5,722,544-splat scene on a 6 GB GPU.

  • Cause: every frame a CPU sort physically reordered all five attribute arrays and marked them for re-upload, about 298 MB per sort.
  • Fix 1: sort an index list instead: ~23 MB, ~13× less.
  • Fix 2: move the sort to the GPU, so nothing is uploaded per sort.
  • Fix 3: keep the data packed and decode it in the shader.
  • Fix 4: stream under a budget, which is the real protection.

That bug is why SplatShop is designed the way it is.

UPLOADED PER SORT reorder attributes ~298 MB sort indices ~23 MB GPU sort 0 MB 5.72M splats · widths to scale
Optimizations & results

70+ techniques; these matter most

13×

Index sort, then GPU sort

~298 → ~23 MB per sort, then zero uploads; skipped when the camera is still.

LOD

Budget-aware hierarchical LOD

Foveated, with hysteresis; Rust → WASM in a worker with transferable results.

GPU

Frustum + Hi-Z culling

A compacted visible-chunk list on the GPU; occlusion protected against popping.

pool

GPU page pool

First-fit pages, pressure-based eviction, per-frame upload budget, stale-drop.

2.6×

Packed data

52 → ~20 B per splat, decoded in the shader; codebooks for scale and colour.

0 GC

Arenas and pools

Buffer and uniform arenas, typed-array pools, dirty-range uploads, bind-group caching.

tiles

Compute tile pipeline

32×32 px tiles, depth bands, work queue; many small sorts instead of one big one.

adapt

Quality adaptation

5 presets, device tiers, DPR cap, reverse-Z, Halton temporal accumulation.

The full catalogue, 70+ techniques by pipeline stage, is in docs/optimization-techniques.md, plus a 14-part article series.