ORB-SLAM2 Feature Extraction

ORBSLAM2 Analysis - Feature Extration

Feature Extraction

Recently I read through the ORB-SLAM2 source code again and was deeply impressed by the author’s ability. Here I summarize its ORB feature extraction. The Zhihu expert Xiaoputao (https://zhuanlan.zhihu.com/p/61738607) has already made a great summary of this topic, and here I once again act as a porter of knowledge, following along with the code.

####Several characteristics of ORB features:

1. Scale invariance: first, an image pyramid is computed, which guarantees the scale invariance of feature points within a certain range
2. Rotation invariance: the orientation of a feature point is computed using the intensity centroid method, and then the rotated neighborhood point pairs are used when computing the descriptor
3. Some illumination invariance: both FAST extraction and BRIEF computation are based on comparing the relative magnitudes of intensity values
4. Fast: both the feature extraction method and the descriptor computation method are far faster than SIFT and SURF
5. Noise resistance: the image is Gaussian-blurred when computing the descriptor, and when comparing the intensities of neighborhood point pairs, the intensity of the surrounding region rather than a single point is used (ORB-SLAM2 does not use it this way)

The extraction process is as follows:

1. Pyramid computation In ORB-SLAM, feature extraction is performed at every level of the image. The number of feature points extracted at each level is determined by a geometric sequence governed by the scaling factor. Ultimately, the information contained in each feature point is (pt.x, pt.y, octave, angle, size).

    resize(mvImagePyramid[level-1],	//input image
            mvImagePyramid[level], 	//output image
            sz, 						//size of the output image
            0, 						//horizontal scaling factor; leave as 0 for automatic computation
            0,  						//vertical scaling factor; leave as 0 for automatic computation
            cv::INTER_LINEAR);		//interpolation algorithm type for image scaling; here it is linear interpolation
   

   mvImagePyramid is built according to the scaling factor.

2. FAST feature-point extraction & quadtree homogenization Feature points are extracted from all CELLs (30 * 30 squares) of the image at each level. If the initial threshold fails to extract feature points, the threshold is lowered for extraction. The feature points extracted at each level are then homogenized with a quadtree.

        keypoints = DistributeOctTree(vToDistributeKeys, 			
                                      minBorderX, maxBorderX,		
                                      minBorderY, maxBorderY,
                                      mnFeaturesPerLevel[level],
                                      level);            
   

   The process of quadtree-based uniform extraction is shown in the figure (figure by Xiaoputao on Zhihu):

Each split splits all current nodes. The newly split nodes are placed at the front of the node list, waiting to be split first in the next round. After the splitting finally terminates, the feature point with the strongest response is selected from each node and added to the final set of feature points.

Then the orientation of the extracted feature points is computed (intensity centroid method), thereby obtaining all the information of the feature points.

1. Descriptor computation The corresponding descriptor is computed for the feature points at each level. The descriptor computation uses the Steered BRIEF method, i.e. the point pairs in the pattern are rotated according to the feature point’s angle before the intensity comparison. Although this orientation gives the feature-point description rotation invariance, it also causes a drop in performance, as the distinctiveness between different features decreases. Rotation-Aware BRIEF seems to be able to solve this problem, using a statistical method to select the best 256 point pairs.

2. Reduce the feature-point coordinates to level 0 Now that the feature-point information for each level has been obtained, the feature points of the other levels are reduced to level 0 according to scale, forming the final set of feature points, ultimately yielding all the feature points and their corresponding descriptors.

Extraction Results

opencv orb extration:

orbslam2 orb extration:

As you can see, the feature points are more uniform, which helps improve the accuracy of the pose solution.