Vaahan
A pipeline that turns fixed-camera traffic video into a queryable, simulatable digital twin of the road scene.
Traffic simulators like CARLA and SUMO are genuinely impressive. The catch is they need you to describe the scene: road geometry, signal timings, vehicle mix, behaviour parameters. For anything real, that’s a lot of manual work.
Vaahan is a perception and reconstruction pipeline that extracts all of that from video. Point it at a fixed camera feed, and it produces a structured scene model you can query and simulate. The interesting thing is that scene model is the actual product. Everything before it (detection, tracking, depth estimation) is just populating it. Everything after it (simulation, analytics, visualisation) is just consuming it.
You get two outputs from the same model: a replay that faithfully recreates what happened (useful as a correctness check), and a simulation mode where you can change things and see what would have happened differently.
What the pipeline does
Fixed-cam video
-> Detect + Track + Segment (YOLO11, BoT-SORT, SAM2)
-> 3D Lift (homography -> metric BEV trajectories)
-> Scene Reconstruction (3D Gaussian Splatting for background)
-> Canonical Scene Model (Postgres + PostGIS)
-> Simulation (SUMO, CARLA via OpenDRIVE export)
-> Query / Analytics API
Stack
I’m reusing everything I can. The interesting IP here is the extraction pipeline, the schema design, and the per-site calibration. Not another object detector.
| Task | Tool |
|---|---|
| Object detection | YOLO11 / RT-DETRv2, fine-tuned on traffic |
| Tracking and re-ID | BoT-SORT / ByteTrack |
| Segmentation | SAM 2 |
| Mono depth | Depth Anything V2 / UniDepth |
| Photoreal background | 3D Gaussian Splatting |
| Traffic micro-sim | Eclipse SUMO |
| Photoreal driving sim | CARLA |
| Road-network format | OpenDRIVE (.xodr) |
| Scene schema | Postgres + PostGIS |
Where it’s at
Architecture and schema are locked. The local non-GPU stack (PostGIS, MinIO, Redis) runs via Docker. Next up is the perception pipeline: GPU workers for detection, tracking, and segmentation. That work has to run on a remote Linux host because Docker on Mac can’t access CUDA.
What I’m drawing from
- YOLO11 and BoT-SORT for detection and tracking
- 3D Gaussian Splatting for scene reconstruction
- CARLA and Eclipse SUMO as simulation backends
- Depth Anything V2 for monocular depth
If you want to help
I need GPU infrastructure to run the perception pipeline, and I’d love to talk to anyone who’s done BEV projection or worked with BoT-SORT integrations before. Also looking for people with access to real CCTV footage for calibration experiments. Get in touch.
