ORB Descriptor
The ORB (Oriented Fast and Rotated Brief) descriptor is a somewhat similar to BRIEF. It doesn’t have an elaborate sampling pattern as BRISK or FREAK.
However, there are two main differences between ORB and BRIEF:
- ORB uses an orientation compensation mechanism, making it rotation invariant.
- ORB learns the optimal sampling pairs, whereas BRIEF uses randomly chosen sampling pairs.
The ORB descriptor uses the intensity centroid as a measure of orientation. To calculate the centroid, we first need to find the moment of a patch, which is given by Mpq = x,yxpyqI(x,y). The centroid, or ‘centre of mass' is then given by C=(M10M00, M01M00).
The vector from the corner’s center to the centroid gives the orientation of the patch. Now, the patch can be rotated to some predefined canonical orientation before calculating the descriptor, thus achieving rotation invariance.
ORB tries to take sampling pairs which are uncorrelated so that each new pair will bring new information to the descriptor, thus maximizing the amount of information the descriptor carries. We also want high variance among the pairs making a feature more discriminative, since it responds differently to inputs. To do this, we consider the sampling pairs over keypoints in standard datasets and then do a greedy evaluation of all the pairs in order of distance from mean till the number of desired pairs are obtained i.e. the size of the descriptor.
The descriptor is built using intensity comparisons of the pairs. For each pair if the first point has greater intensity than the second, then 1 is written else 0 is written to the corresponding bit of the descriptor.
Example
Let us take a look at a simple example where the ORB descriptor is used to match two images where one has been translated by (50, 40) pixels and then rotated by an angle of 75 degrees. We will use the lighthouse image from the TestImages package for this example.
First, let us create the two images we will match using ORB.
using ImageFeatures, TestImages, Images, ImageDraw, CoordinateTransformations, Rotations
img = testimage("cameraman")
img1 = Gray.(img)
rot = recenter(RotMatrix(5pi/6), [size(img1)...] .÷ 2) # a rotation around the center
tform = rot ∘ Translation(-50, -40)
img2 = warp(img1, tform, axes(img1))
The ORB descriptor calculates the keypoints as well as the descriptor, unlike BRIEF. To create the ORB descriptor, we first need to define the parameters by calling the ORB constructor.
orb_params = ORB(num_keypoints = 1000)ORB(1000, 12, 0.25, 0.04, 1.3, 8, 1.2)Now pass the image with the parameters to the create_descriptor function.
desc_1, ret_keypoints_1 = create_descriptor(img1, orb_params)
desc_2, ret_keypoints_2 = create_descriptor(img2, orb_params)(BitVector[[0, 0, 0, 0, 1, 1, 0, 0, 0, 0 … 1, 0, 1, 0, 0, 0, 1, 1, 0, 1], [0, 1, 0, 0, 1, 1, 0, 0, 0, 1 … 1, 0, 1, 1, 0, 0, 1, 1, 0, 1], [0, 1, 1, 0, 0, 1, 1, 0, 0, 0 … 1, 1, 0, 1, 1, 0, 0, 1, 1, 0], [0, 0, 1, 1, 1, 1, 1, 0, 0, 0 … 1, 1, 0, 1, 0, 0, 1, 0, 0, 0], [0, 0, 0, 0, 0, 1, 0, 0, 0, 1 … 1, 0, 1, 1, 0, 0, 1, 1, 0, 1], [1, 0, 1, 1, 0, 0, 1, 0, 1, 1 … 1, 1, 1, 0, 0, 0, 0, 1, 1, 1], [0, 1, 1, 0, 1, 1, 1, 0, 0, 0 … 1, 0, 0, 1, 1, 0, 0, 1, 1, 0], [0, 0, 1, 1, 1, 1, 1, 0, 0, 0 … 1, 0, 0, 1, 0, 1, 1, 0, 0, 1], [0, 1, 0, 0, 1, 0, 1, 0, 1, 0 … 0, 0, 0, 0, 1, 0, 0, 0, 1, 0], [0, 1, 0, 1, 0, 0, 1, 0, 0, 0 … 0, 0, 1, 1, 0, 1, 1, 0, 1, 0] … [0, 0, 0, 0, 0, 0, 1, 0, 1, 1 … 1, 1, 1, 0, 0, 1, 1, 1, 1, 1], [0, 1, 1, 1, 0, 1, 1, 0, 1, 0 … 1, 0, 0, 1, 0, 1, 0, 1, 1, 1], [0, 1, 1, 0, 1, 1, 1, 1, 1, 0 … 0, 0, 1, 1, 0, 0, 0, 1, 0, 0], [0, 1, 1, 1, 0, 0, 0, 1, 0, 1 … 0, 1, 1, 1, 1, 0, 0, 0, 1, 0], [0, 1, 0, 0, 0, 0, 0, 1, 1, 0 … 1, 1, 1, 1, 0, 1, 1, 0, 1, 1], [0, 1, 1, 0, 0, 0, 1, 1, 1, 0 … 0, 1, 1, 0, 0, 0, 0, 0, 0, 1], [0, 0, 1, 1, 1, 1, 1, 0, 0, 0 … 1, 1, 0, 1, 0, 0, 0, 1, 0, 0], [1, 1, 0, 0, 0, 1, 1, 0, 1, 0 … 0, 1, 0, 0, 0, 1, 1, 0, 0, 1], [1, 0, 0, 1, 1, 0, 0, 0, 0, 1 … 0, 0, 1, 0, 1, 1, 0, 0, 0, 1], [1, 1, 0, 0, 1, 0, 0, 1, 1, 1 … 0, 0, 1, 1, 1, 1, 1, 0, 0, 0]], CartesianIndex{2}[CartesianIndex(160, 37), CartesianIndex(161, 37), CartesianIndex(336, 296), CartesianIndex(382, 299), CartesianIndex(161, 38), CartesianIndex(158, 242), CartesianIndex(336, 295), CartesianIndex(382, 300), CartesianIndex(150, 241), CartesianIndex(196, 298) … CartesianIndex(122, 235), CartesianIndex(248, 273), CartesianIndex(227, 162), CartesianIndex(365, 347), CartesianIndex(234, 184), CartesianIndex(263, 265), CartesianIndex(258, 262), CartesianIndex(164, 247), CartesianIndex(277, 283), CartesianIndex(275, 287)], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0 … 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0])The obtained descriptors can be used to find the matches between the two images using the match_keypoints function.
matches = match_keypoints(ret_keypoints_1, ret_keypoints_2, desc_1, desc_2, 0.2)122-element Vector{Vector{CartesianIndex{2}}}:
[CartesianIndex(230, 271), CartesianIndex(337, 296)]
[CartesianIndex(189, 291), CartesianIndex(382, 300)]
[CartesianIndex(230, 272), CartesianIndex(337, 295)]
[CartesianIndex(146, 315), CartesianIndex(432, 301)]
[CartesianIndex(147, 314), CartesianIndex(431, 301)]
[CartesianIndex(230, 275), CartesianIndex(338, 294)]
[CartesianIndex(172, 209), CartesianIndex(355, 380)]
[CartesianIndex(199, 287), CartesianIndex(371, 299)]
[CartesianIndex(229, 271), CartesianIndex(338, 296)]
[CartesianIndex(199, 286), CartesianIndex(370, 299)]
⋮
[CartesianIndex(136, 283), CartesianIndex(425, 333)]
[CartesianIndex(343, 239), CartesianIndex(254, 279)]
[CartesianIndex(145, 201), CartesianIndex(374, 400)]
[CartesianIndex(182, 236), CartesianIndex(360, 351)]
[CartesianIndex(179, 240), CartesianIndex(364, 348)]
[CartesianIndex(240, 259), CartesianIndex(322, 302)]
[CartesianIndex(162, 289), CartesianIndex(404, 315)]
[CartesianIndex(180, 239), CartesianIndex(363, 349)]
[CartesianIndex(278, 251), CartesianIndex(287, 291)]We can use the ImageDraw.jl package to view the results.
grid = hcat(img1, img2)
offset = CartesianIndex(0, size(img1, 2))
map(m -> draw!(grid, LineSegment(m[1], m[2] + offset)), matches)
grid
This page was generated using DemoCards.jl and Literate.jl.