Image Preprocessing and Feature Extraction¶

Local Binary Patterns¶

A local binary pattern (LBP) operator computes a value at each pixel that is partially invariant to luminance changes.

Histograms¶

Histogram descriptors are useful for tasks with plenty of spatial resolution. They are largely unsuitable when an object’s spatial layout is important for identification.

Exercise 13.1¶

It will be the diagonal line in Figure 13.2 where the gray level intensity is between \([0, 127]\) and the cumulative proportion is between \([0, 1]\). Starting at the gray level intensity \(128\), the diagonal line switches to horizontal line. Applying histogram equalization changes the ordinate scale i.e. changes the range of the pixel intensities to \([0, 255]\).

[1]:

import matplotlib.pyplot as plt
import numpy

I = J = 512
K = 255
_ = numpy.random.randint(0, K / 2, I * J)

X = numpy.sort(_)
F = numpy.arange(I * J) / (I * J)

plt.plot(X, F)

h, X1 = numpy.histogram(_, bins=(K + 1), range=(0, K), density=True)
dx = X1[1] - X1[0]
F1 = numpy.cumsum(h) * dx
plt.plot(X1[1:], F1)

plt.tight_layout()
plt.title('CDF')
plt.xlabel('Pixel Intensity')
plt.ylabel('Fraction')
plt.xlim([0, 255])
plt.ylim([-0.05, 1.05])
plt.show();

<Figure size 640x480 with 1 Axes>

Exercise 13.2¶

Define images \(a\) and \(b\) as

\[\begin{split}a[i, j] = [f \otimes p](i, j) = \int_{-\infty}^\infty \int_{-\infty}^\infty p[i - m, j - n] f[m, n] dm dn\\\\ b[m', n'] = [g \otimes f](m', n') = \int_{-\infty}^\infty \int_{-\infty}^\infty f[m' - m, n' - n] g[m, n] dm dn\end{split}\]

where

\[\begin{split}\left[ (g \otimes f) \otimes p \right](k, l) &= [b \otimes p](k, l)\\ &= \int_{-\infty}^\infty \int_{-\infty}^\infty p[k - i, l - j] b[i, j] di dj\\ &= \iiiint_{-\infty}^\infty p[k - i, l - j] f[i - m, j - n] g[m, n] dm dn di dj\\ &= \iiiint_{-\infty}^\infty p[k - m - m', l - n - n'] f[m', n'] g[m, n] dm dn dm' dn' & \quad & \text{change of variables where } i = m' + m \text{ and } j = n' + n\\ &= \int_{-\infty}^\infty \int_{-\infty}^\infty g[m, n] \int_{-\infty}^\infty \int_{-\infty}^\infty p[k - m - m', l - n - n'] f[m', n'] dm' dn' dm dn & \quad & \text{Fubini-Tonelli Theorem}\\ &= \int_{-\infty}^\infty \int_{-\infty}^\infty a[k - m, l - n] g[m, n] dm dn\\ &= [g \otimes a](k, l)\\ &= [g \otimes (f \otimes p)](k, l).\end{split}\]

The result extends to infinite discrete images. For finite sized images, the boundary pixels may have different results depending on the padding.

Exercise 13.3¶

(a, b)¶

The two-rectangle feature requires six operations.

(c)¶

The three-rectangle feature requires eight operations.

(d)¶

The four-rectangle feature requires nine operations.

Exercise 13.4¶

Bilateral filters preserve sharp edges.

Exercise 13.5¶

\[\begin{split}\begin{bmatrix} 0 & -1 & -2\\ 1 & 0 & -1\\ 2 & 1 & 0 \end{bmatrix}\end{split}\]

Exercise 13.6¶

The coefficients of the Sobel \(F_x\) can be modified to satisfy these requirements (e.g. Scharr operator). Taking a page out of bilateral filters, each coefficient can be weighted according to their spatial distance from the kernel’s center.

Exercise 13.7¶

LBPs have been crafted to be invariant to gray scale and rotation. What’s left in a natural image are primitive microfeatures (e.g. edges, lines, corners, spots, and flat areas).

Exercise 13.8¶

K-means assumes the clusters will have spherical variance and gives more weight to larger clusters. Data with uneven clusters will cause k-means to fail while GMM succeeds.

Exercise 13.9¶

The single outlier will drag one of the two cluster means into the empty region. One surefire approach is to use a GMM using two t-distributions; another approach could be to weigh the contributions of data points depending on distance.

Exercise 13.10¶

These peaks correspond typically to the number of data clusters. [ZMP05] computes the affinity matrix and analyze the gaps between eigenvalues as well as the eigenvectors’ to estimate the number of clusters.

References

ZMP05: Lihi Zelnik-Manor and Pietro Perona. Self-tuning spectral clustering. In Advances in neural information processing systems, 1601–1608. 2005.