Coarse-to-fine Semantic Localization with HD Map for Autonomous Driving In Structural Scenes

Published in IROS 2021, 2021

Robust, low-cost localization for autonomous driving using cameras together with an HD map. Instead of relying on expensive sensor suites, the method casts visual localization as a data-association problem — matching image semantics to landmarks in the HD map — and refines the vehicle pose with pose-graph optimization.

Key ideas

  • Semantic ↔ map association. Vehicle localization is framed as associating visual semantics (lane lines, poles, signs) in the image with landmarks stored in the HD map.
  • Coarse-to-fine initialization. A coarse GPS prior is refined by a fine pose search, avoiding the accurate-initial-pose requirement that trips up prior methods.
  • Semantic tracking. During tracking, the pose is refined by implicitly aligning the image’s semantic segmentation with HD-map landmarks under a photometric-consistency objective.
  • Sliding-window optimization. The final trajectory is computed by pose-graph optimization in a sliding-window fashion.
  • Mono & multi-camera. The same formulation works for a single camera or a multi-camera rig, improving robustness.
Semantic lane detection on a highway scene
Semantic lane detection on a highway scene — the visual semantics that get matched against the HD map.
HD-map landmark view with the estimated vehicle pose
HD-map landmarks (lanes, poles, overhead signs) with the estimated vehicle pose during tracking.

Evaluated on two datasets, the approach yields promising localization across different driving scenarios. Accepted at IROS 2021, and featured by Nullmax.ai.

Read the paper on arXiv →

Comments