RobotLens
robotlens.dev · github.com/NarinderBrar/RobotLens
Project 1 · Open source · 2026 →

RobotLens

A native ROS 2 engineering workspace. Topology, robot models, TF, diagnostics, logs, 3D visualization, MoveIt, Nav2 and Gazebo in one app, built on one shared graph of the robot system.

C++20VulkanDear ImGuiROS 2 JazzyGTestMCP
~200klines of C++
57decision records
359test files
25+display types
What I built

Features

Everything in one dockable workspace, sharing one selection model, one timeline and one message pipeline.

01Robot knowledge graph

Nodes, topics, services, TF frames, joints, diagnostics and logs as entities with stable IDs; immutable per-frame snapshots.

02Live ROS 2 discovery

Nodes, endpoints with QoS, services, actions, parameters and TF at ~1 Hz; failure-safe (last good state survives).

03Vulkan viewport

PBR, shadow maps, IBL, SSAO, ray-query contact shadows, MSAA 1–8×, GPU picking, gizmos.

04GPU-driven rendering

Compute frustum culling with indirect draws, Hi-Z occlusion, packed vertices, front-to-back sorting.

0525+ display types

Point clouds (chunked, partially resident), laser scans, maps, markers, paths, cameras, virtual LIDAR.

06Robot models & sim

URDF/SDF parse, render and edit; MoveIt 2, Nav2 and Gazebo workflows with approval steps.

07Health & evidence

12 health rules with evidence citations; a deterministic investigation assistant with 4 LLM adapters.

08MCP server

30+ read-only tools for AI agents, an evidence envelope on every reply, propose → approve → execute.

09Investigation agent

A tool-calling loop with budgets, guards and evals, plus tiny specialist models (“mini-brains”) as tools.

Architecture

Workers produce, one thread writes, everyone reads a snapshot

WORKER THREADS · produce discovery · ~1 Hz diagnostics · logs URDF fetch · meshes MoveIt · Nav2 · Gazebo MCP transport bounded queueObservationBatch drain Main thread single writer Graph::applyBatch() deterministic deltas publish GraphSnapshot immutable · per frame ImGuipanels Vulkanrenderer read-onlyread-only edits return as typed commands · never direct mutation no locks on the UI path · one consistent frame failed ROS queries → last good state survives
ADR 0004: the graph has one logical writer. Workers never touch shared state, and readers get an immutable snapshot once per frame. It's the same idea as Redux: a single source of truth.
Architecture decisions

What I chose, and what I didn't

DecisionRejectedWhy
Single writer + immutable snapshotsShared graph with locksNo lock contention on the UI path; consistent frames; deterministic add/update/stale/remove deltas
Domain graph independent of ROS, ImGui, Vulkan, JSONROS types everywhereCore logic testable without a robot; adapters at the edges; backends swappable (ADR 0022)
Stable entity IDs from kind + source + identityPointers / countersSelection survives reconnects, reloads and snapshots; shared across every panel
Dirty-domain invalidation (bit flags)Rebuild the scene on any changeA transform change updates matrices only; a display message re-renders displays only
Fence-safe resource retirementDestroy immediatelyWith 2 frames in flight the GPU may still read it; avoids device-lost crashes
MCP read-only, actions via propose → approveLet agents act directlyA robot is physical; humans approve with risk shown; the same typed commands as the UI
57 written ADRsDecisions in chatReasons survive; new contributors and AI agents follow the same boundaries
Feature · performance

Incremental snapshots: 60 ms → 1.5 ms

With 12,000 entities, rebuilding the whole snapshot on every change caused visible hitches. I split it into one block per entity kind: rebuild only the changed kinds and share the rest with the previous snapshot.

Snapshot NSnapshot N + 1 Nodes Topics (old) TF frames Diagnostics Logs → shared Topics · rebuilt → shared → shared → shared same pointer FULL REBUILD~60 ms INCREMENTAL~1.5 ms change: /scan QoS updated → only the Topics block is rebuilt+ cached ID hashes+ refs instead of copies
Structural sharing, the same trick immutable data libraries use. Being immutable doesn't mean copying everything. It's measured in commit a72c1bf9, which also stopped one display message from rebuilding the scene (~30 ms).
Feature · GPU-driven rendering

Compute culling writes its own draw calls

Instead of CPU culling and one draw per visible instance, a compute shader tests every instance and writes VkDrawIndirectCommands. The CPU issues one indirect draw per topology group.

MeshCullRecord[]112 B · std430 · per instance AABB + model + draw range Hi-Z pyramidfrom last frame's depth mesh_cull.comp 1 threadper instance ① frustum: positive-vertex ② occluded by Hi-Z? 10110 VkDrawIndirectCommand[]instanceCount = visible ? 1 : 0 DrawIndirect1 per topologygroup HOST→COMPUTEbarrier COMPUTE→DRAW_INDIRECT double-buffered records · commands · stats (2 frames in flight) · stats read back one frame late
GP-10 (compute culling and indirect draws) and GP-12 (Hi-Z occlusion). A CPU reference implementation mirrors the shader math, so the tests prove both give the same answer.
Feature · occlusion culling

Hi-Z: skip what's hidden behind other things

mip 0 · scene depthAABB footprint mip 1 mip 2≈ 1 texel 2×2 max 2×2 max Per instance, in the cull shader 1 project the 8 AABB corners → screen rectangle 2 level = ceil(log2(largest side in px)) 3 read the texels under it (+1 border) 4 hidden if its nearest depth is behind the stored max depth everywhere corner behind near plane → no decision (draw it) BUG I FIXED · GPU hang (device lost) near-plane footprints made the texel loop unbounded fix: clamp to viewport + CPU mirror + regression test
Max reduction keeps the farthest depth per texel, so “behind everything stored here” is a safe, conservative test. The pyramid is built from depth the scene already wrote, so it costs one small compute pass.
Feature · memory

Point clouds bigger than one GPU buffer

A single cloud can exceed one practical GPU allocation, and scrubbing a recording grows memory. Over a byte budget, only visible chunks become resident, in slot arenas with LRU reuse (ADR 0034).

cloud → fixed-size chunks camera in view = requested, nearest first GPU slot arena · byte budget c14c15c21c22c9 ←c16 least recently used → slot reused maxResident = ⌊budget / (chunkPts × (pointStep + 32))⌋ one compute dispatch per resident chunk (slot offset) upload Retirement queue free only when theframe's fence signals 2 frames in flight resize picking uses the sametranslated ranges → you canonly click what's drawn
Eviction only invalidates slots. Arena buffers are destroyed solely through the fence-safe retirement path, so nothing an in-flight frame still reads can be freed.
Memory & buffer packing

Vertex 56 → 20 bytes, and the cull record

Vertex · 56 B (float32) position 3×f32 · 12 color 4×f32 · 16 normal 3×f32 · 12 met 4 rgh 4 uv 2×f32 · 8 14 px = 1 B VertexPacked · 20 B2.8× smaller · static_assert(sizeof == 20) pos 3×u16 n oct rgba8 uv f16 0610141620 met · rgh u8 position: u16 relative to the mesh AABB (min/max in push constants) normal: octahedral 2×snorm16 · UV: half floats shared by 7 pipelines: scene, prepass, 2 shadow, SSAO, picking MeshCullRecord · 112 B · std4306 px = 1 B localMin 16 localMax 16 model mat4 · 64 vtxCount · firstVtx · firstInst · pad vec4-aligned for std430
Fewer bytes per vertex means less memory and less bandwidth for every pass that reads geometry. Layouts are pinned with static_asserts on size and offsets, so the CPU struct can never silently drift from the shader.
Feature · AI

MCP server: agents inspect, humans approve

AI clientClaude Code · CursorCodex · OpenCode READ PATH · 30+ tools stdio · socket transport worker bounded queue main threadresolve on snapshot evidence envelope: result · citations · captured_at · limitations · truncated? ACTION PATH · never autonomous propose_* proposal · riskimpact · expiry human approvesin the app typed commandsame path as the UI
ADR 0035. External processes never mutate state. Tools are resolved on the main thread against immutable snapshots, and every queue, deadline and response size is bounded. Next up is an in-app agent loop (planned) that uses these same tools, with evals from day one.
Optimizations & results

Measured, not guessed

40×

Incremental graph snapshots

~60 ms → ~1.5 ms with 12k entities: per-kind blocks and structural sharing.

2.8×

Packed vertices

56 → 20 bytes: u16 positions, octahedral normals, u8 materials, f16 UVs.

GPU

Compute culling + indirect draws

The GPU decides visibility and writes its own draw commands.

Hi-Z

Occlusion culling

A max-reduced depth pyramid; conservative test; device-lost bug fixed with a regression test.

ΔZ

Front-to-back sorting

Early-Z rejects hidden fragments; less overdraw without a second pass.

LRU

Partial residency

Byte-budgeted point clouds; 256 MiB history cap; fence-safe freeing.

dirty

Dirty-domain invalidation

11 flags; a display message no longer rebuilds the scene (~30 ms).

auto

Quality profiles

GPU timestamps drive MSAA and render scale; picking stays pixel-exact.

Every change is measured before and after on the same scenario, and the perf snapshot (CPU sections, GPU timestamps per pass, memory by owner) is queryable over MCP.