Visual-SLAM Developer Roadmap - 2026

Visual-SLAM is a special case of 'Simultaneous Localization and Mapping', which you use a camera device to gather exteroceptive sensory data.

Below there is a set of topics you need to understand in Visual-SLAM, from an absolute beginner difficulty to getting ready to become a Visual-SLAM engineer / researcher.

Visual-SLAM is often portrayed as a rather difficult topic - many think good C++ programming skills and deep understanding of mathematics is necessary.

On the other hand, there are not many courses provided for beginners, especially in non-English languages.

I made these charts to share my thoughts and experience on studying Visual-SLAM, and hopefully the beginner learners can get a grasp of where to start from.

Purpose of these Roadmaps

The purpose of these roadmaps is to give you an idea about the general overview of Visual-SLAM, and to guide you if you are confused about where to start from.

Note to Beginners

Acknowledge that SLAM has a relatively high entry barrier - it's not because of the requirement of undertanding difficult mathematics, but the requirement of equipping yourself with various types of skills. Don't feel overwhelmed - you don't need to learn everything if you are just getting started. Instead, enjoy the journey itself and progress topic by topic. The result will be very rewarding.

Table of Contents

Level 1: Beginner

Programming

• C++: Pointer, OOP
• Python
• Bash/Linux: Basic terminal usage

Mathematics

• Basic Probability & Statistics: Gaussian distribution, Bayes' theorem
• Basic Linear Algebra: Vectors & Matrices, Determinant, Dot & Cross product, Rank, Inverse matrix, Transpose matrix, SVD, Eigenvalues/Eigenvectors
• Logarithm & Exponential
• Basic Calculus: Differentiation, Taylor expansion

Projective Geometry

• Pinhole camera model → Image projection
• Camera calibration: Intrinsic/Extrinsic parameters, Lens distortion
• Rigid body motion: Euler/Quaternion/Rotation Matrix, Projective space & Vanishing point, Homogeneous transformation
• Epipolar geometry → Essential & Fundamental matrix
• Triangulation

Camera Device

• Lens, Sensor, Resolution/ISO/Aperture

Image Data

• Colour image, Resolution, Grayscale image
• Thresholding, Gaussian blur
• Corner detector: Harris corner
• Edge detector: Sobel & Canny Edge
• Stereovision, RGB-D, Disparity, Depth

Level 2: Getting Familiar with SLAM

Programming

• C++: OOP, Modern C++, Data structures & Algorithms, Compilers, CMake/Makefile/Ninja, Design patterns, OpenCV C++
• C
• Git/GitHub
• OpenCV (opencv-python)
• Python: Deep learning, Graph plots, System scripts
• Bash/Linux: ssh, CLI text editor/Vim/tmux
• Concurrency: SIMD-SSE/AVX/Neon, OpenMP, CUDA
• Mobile: Android (Java/Kotlin), iOS (Objective-C/Swift)
• Maths library: Eigen, Ceres-solver/GTSAM/g2o
• C++/Python interop: PyBind11, nanobind
• Docker
• C#: COLMAP, Unity AR, Microsoft Hololens
• CI/CD: GitHub Actions, Apache Airflow
• ROS/ROS2
• Simulation: Gazebo, Isaac Sim

Image Processing

• Keypoints → Detector/Descriptor
  • SIFT, FAST, ORB, AKAZE
  • Deep features: R2D2, Superpoint
• Image pyramid, oFAST, rBRIEF

Local Feature Matching

• Brute-Force, FLANN, Kd-Tree
• LSH, Multi-probe LSH, HBST
• Superglue

Global Feature Matching

• Bag of Visual Words, NetVLAD
• Deep image retrieval, Hierarchical localization

Feature Tracking

• Optical flow, KLT Tracker

Multiple View Geometry

• 2D-2D correspondence: Essential/Fundamental, Homography
• 2D-3D correspondence: P3P, PnP, SVD
• 3D-3D correspondence: ICP

Outlier Rejection

• RANSAC, PROSAC, M-Estimator, MAXCON, Convex relaxation

Least Squares Optimisation

• Reprojection error, Bundle adjustment
• Non-linear optimisation, Lie algebra
• Lie groups: SO(3), SE(3)
• Gauss-Newton, Levenberg-Marquardt
• Pose graph optimization
• Schur complement / Sparsity

Motion Model

• Proprioceptive sensor: IMU, Wheel
• Odometry (pose)

Observation Model

• Exteroceptive sensor: Camera, LiDAR
• Landmark (Map)
• Joint optimisation, MLE & MAP

Factor Graph Optimisation

Mapping

• Point cloud, Occupancy grid mapping, TSDF, Surfel, Voxel map

Sensors

• Camera device: Wide/telecentric lens, Lens MTF, CCD/CMOS, Rolling/Global shutter, Exposure/ISO, Stereovision, RGB-D, Structured light, Active IR/ToF
• LiDAR → Visual-LiDAR fusion
• IMU → VIO
• RADAR → Sensor fusion, Extended Kalman filter
• Sonar
• Multi-sensor calibration: Camera-IMU, Camera-LiDAR

Evaluation

• Metrics: ATE (Absolute Trajectory Error), RPE (Relative Pose Error)
• Datasets: KITTI, TUM RGB-D, EuRoC

Next Levels

Monocular SLAM · VIO/VINS · Stereo SLAM · Visual-LiDAR Fusion · RGB-D SLAM · Collaborative SLAM · Deep SLAM/Localization

Level 3: Monocular Visual-SLAM

Key Concepts

• VO vs SLAM — VO is local (no loop closure), SLAM includes global map + loop closure
• Scale ambiguity — Fundamental limitation of monocular SLAM; absolute scale is unrecoverable from images alone
• Covisibility graph — Shared map point visibility between keyframes; core data structure in ORB-SLAM
• Visual Place Recognition (VPR) — Recognising previously visited places for loop closure
• Self-supervised depth — Learning monocular depth without ground truth (Monodepth2, Godard 2019)

Feature-based SLAM

Direct SLAM

Hybrid (Feature + Direct)

Learning-based SLAM

Foundation Model SLAM

SfM Tools

Neural Representation SLAM

NeRF-based

3DGS-based

Semantic / Language-Grounded SLAM

Also see: LEGS, Open-S3SLAM (3DGS-based section above); Open-YOLO 3D (Level 5 Object Detection)

Level 4: RGB-D Visual-SLAM

RGB-D Camera Devices

• Intel RealSense D series
• Microsoft Kinect v1/v2
• Azure Kinect DK
• Occipital Structure Core
• Orbbec Astra

GPGPU Programming

• CUDA, OpenGL GLSL

Systems

Level 5: Applying Deep Learning

Level 5 is organized into four pillars: A. Frontend — learned perception components replacing hand-crafted modules B. Backend — learned/certifiable optimization replacing classical solvers C. Systems — end-to-end deep VO/SLAM pipelines D. Scene Understanding — semantic, language, and relational reasoning on SLAM maps

A. Deep Frontend — Perception

Feature Detection & Matching

Depth Estimation

Optical Flow & Scene Flow

Camera Pose Regression & Relocalization

Object Detection & Segmentation for SLAM

B. Deep Backend — Optimization

Differentiable Bundle Adjustment

Certifiably Optimal Algorithms

Gaussian Belief Propagation & Graph Processors

C. End-to-End Deep VO / SLAM Systems

Self-supervised & Learned VO

Latent Representation SLAM

Neural Rendering (reference)

NeRF/3DGS-based SLAM systems → see Level 3: Neural Representation SLAM

D. Scene Understanding

Benchmarks & Foundations

3D Scene Graph

Level 6: VIO / VINS

Key Concepts

• Tightly-coupled vs Loosely-coupled — Joint vs separate optimization of visual and inertial measurements
• Filter-based vs Optimization-based — EKF approaches vs nonlinear optimization (BA)
• IMU preintegration — On-manifold IMU integration between keyframes (Forster 2015)
• IMU noise model — Bias, random walk, Allan variance
• Observability — Yaw and global position are unobservable in VIO

Foundations

Filter-based

Optimization-based

Level 7: Stereo SLAM

Key Concepts

• Stereo rectification — Epipolar alignment for efficient disparity search
• Disparity vs Depth — d = f·B/Z, baseline determines depth range/accuracy
• Scale observability — Stereo provides metric scale (unlike monocular)

Systems

Level 8: Collaborative / Multi-Robot SLAM

Key Concepts

• Centralized vs Decentralized — Single server vs peer-to-peer map merging
• Inter-robot loop closure — Place recognition across robots with different viewpoints
• Communication constraints — Bandwidth-limited map sharing, sparse descriptors
• Map merging — Aligning submaps from different robots into a global map

Systems

Level 9: LiDAR & Visual-LiDAR Fusion SLAM

Key Concepts

• LiDAR-Visual-Inertial (LVI) — Triple fusion for robust outdoor SLAM
• Tightly-coupled LiDAR-camera — Joint optimization of point cloud and visual features
• Direct LiDAR-camera alignment — Photometric/geometric alignment without feature extraction
• Degradation handling — Graceful fallback when one modality fails (e.g., LiDAR in rain, camera in darkness)
• Range image — 2D projection of LiDAR scans for efficient processing (SuMa, RangeNet++)

LiDAR / LiDAR-Inertial SLAM

Visual-LiDAR Fusion SLAM

Resources

Level 10: Event Camera SLAM

Key Concepts

• Event cameras (DVS) — Asynchronous per-pixel brightness change detection, μs temporal resolution
• Advantages — HDR (140dB+), no motion blur, low latency, low power
• Challenges — No absolute intensity, sparse asynchronous output, requires new algorithms
• Event representations — Event frames, time surfaces, voxel grids, spike tensors

Foundations

Systems

Level 11: World Models & Spatial AI

World Models

Generative 3D

Vision-Language Models (VLM)

Vision-Language-Action Models (VLA)

Resources

Study Resources

YouTube Lecture Series

Books

Code & Practice

Wrap Up

If you think any of the roadmaps can be improved, please do open a PR with any updates and submit any issues. Also, I will continue to improve this, so you might want to watch/star this repository to revisit.

Also, check out my GitHub and blog :smiley_cat:

Contribution

• Open pull request with improvements
• Discuss ideas in issues
• Spread the word
• Reach out to me directly at hyunggi.chang95[at]gmail.com.

Discussion

To discuss any topics or ask questions, please use the issue tab.

License

The class is licensed under the MIT License:

Copyright © 2026 Hyunggi Chang.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.