Using scale as a feature for correspondence#

One important detail about the Harris corner detector is that, it does not provide the best patch size for corner detection. This means if we zoom in or zoom out of the image, harris corner detector fails.

Harris corner detector is not scale invariant.

Note

What is the optimal patch size to find the good features?

In other words, we would like the features to be scale invariant.

Harris detector is good in the sense that even though the image undergoes projective transformation, the corners are still preserved and detected by the Harris corner detector.

Is there a way to modify Harris corner detector to be scale invariant?

The idea is to find the local maximum of the Harris Operator \(f\).

For a given location, we find \(f\) for \(n\) different patch sizes. Finally we pick the patch size that gives the maximum value of \(f\). This incorporates for the scale change issue in the Harris corner detector.

A better alternative to Harris Operator \(f\)#

Laplacian Operator

The Laplacian of a function \(f = f(x,y)\) is defined as the divergence of its gradient which is equal to the sum of the function’s second spatial derivatives in cartesian coordinates.

(22)#\[\nabla \nabla f = \nabla ^2 f= \frac{\partial ^2f}{\partial x^2} + \frac{\partial ^2 f}{\partial y^2}\]

For more information on laplacian, refer this

For the corner detection, the Laplacian Operator becomes:

(23)#\[\nabla ^2 f(x,y) = I_{xx}+I_{yy}\]

There is another alternative to Laplacian detector.

../_images/dog.PNG

Fig. 39 Laplacian v/s Difference of Gaussians#

Automatic patch size selection in corner detection after zoom in or zoom out.

../_images/auto-scale.PNG

Fig. 40 Automatic patch size selection in corner detection#

../_images/eigen-affine.PNG

Fig. 41 Matching under affine transformation#

Note that we still have not solved the correspondence problem. We have the scale invariant features, how to find what features in the left image match with the features in the right image?

../_images/towards-scale-invariance.PNG

Fig. 42 Normalization#

../_images/affine-normalization-challenge.PNG

Fig. 43 Affine Normalization challenge#

At every pixel in the image patch we calculate the vector \((I_x,I_y)\).

../_images/gradient-orientation.PNG

Fig. 44 Gradient Orientation#

../_images/rotational-ambiguity.PNG

Fig. 45 Eliminating Rotational Ambiguity#

../_images/rotational-ambiguity1.PNG

Fig. 46 Eliminating Rotational Ambiguity#

SIFT Vector#

A more simplified version of gradient of intensity representation of ellipse is to consider a circle and calculate gradient for each pixel and maintain a histogram of gradients at every pixel.

The histogram represents the distribution of gradients in the image patch. Even when the image patch is rotated, the histogram undergoes circular shift.

../_images/sift-features.PNG

Fig. 47 SIFT Features#

../_images/sift-hand-drawn.PNG

Fig. 48 SIFT Features#

An important detail to note is that SIFT features consider the gradient of intensity values and not the pixel values.