Camera Models

(i)

Recall that a general projective camera \(\mathtt{P}\) maps a point in space \(\mathbf{X}\) to an image point according to the mapping \(\mathbf{x} = \mathtt{P} \mathbf{X}\). Let \(I_0\) be a projective image of said points \(\mathbf{x}\).

The camera centre \(\mathbf{C} \in \mathbb{R}^4\) is the homogeneous vector that satisfies \(\mathtt{P} \mathbf{C} = \boldsymbol{0}\).

An image of an image can be specified as \(I' = \mathtt{H} \mathtt{P} \mathbf{X}\) where \(\mathtt{H} \in \mathbb{R}^{3 \times 3}\) represents a planar homography mapping between \(I'\) and \(I_0\). Notice that the homography cannot change the camera centre i.e. eyes drawn at the camera centre will follow you around the room. By inspection, \(\mathtt{H} \mathtt{P}\) is not necessarily equal to \(\mathtt{P}\).

(ii)

Recall from section 3.2.2 Lines that the dual \(\text{Pl}\mathrm{\ddot{u}}\text{cker}\) representation \(\mathtt{L}^*\) can be obtained directly from \(\mathtt{L}\) via (3.10). The join of the line \(\mathtt{L}\) and point \(\mathbf{X} \in \mathbb{R}^4\) defines a plane

\[\boldsymbol{\pi} = \mathtt{L}^* \mathbf{X}\]

where \(\mathtt{L}^* \mathbf{X} = \boldsymbol{0}\) if, and only if, \(\mathbf{X}\) is on \(\mathtt{L}\).

The ray back-projected from an image point \(\mathbf{x}\) under a projective camera \(\mathtt{P}\) may be written as (6.28) because

\[\begin{split}\mathtt{L}^* \mathbf{X}(\mu) &= \mathtt{P}^\top \left[ \mathbf{x} \right]_\times \mathtt{P} \begin{pmatrix} \mathtt{M}^{-1} \left( \mu \mathbf{x} - \mathbf{p}_4 \right)\\ 1 \end{pmatrix} & \quad & \text{(6.14)}\\ &= \mathtt{P}^\top \left[ \mathbf{x} \right]_\times \begin{bmatrix} \left( \mu \mathbf{x} - \mathbf{p}_4 \right) + \mathbf{p}_4 \end{bmatrix} & \quad & \mathtt{P} = \left[ \begin{array}{c|c} \mathtt{M} & \mathbf{p}_4 \end{array} \right]\\ &= \mathtt{P}^\top \left[ \mathbf{x} \right]_\times \mu \mathbf{x}\\ &= \boldsymbol{0} & \quad & \text{(A4.6).}\end{split}\]

(iii) The Affine Camera

(a)

Recall that a point \(\mathbf{X}\) on a plane \(\boldsymbol{\pi}\) satisfies

\[\boldsymbol{\pi}^\top \mathbf{X} = 0 \qquad \text{(3.2).}\]

If the points on the plane at infinity \(\boldsymbol{\pi}_\infty\) map to points at infinity in the image, then \(\mathtt{P}\) must have the affine form. Otherwise, the projected point is no longer at infinity because its last coordinate is not zero:

\[\begin{split}\mathtt{P} \mathbf{X} &= \begin{bmatrix} \mathbf{m}_1^\top & t_1\\ \mathbf{m}_2^\top & t_2\\ \mathbf{v}^\top & v \end{bmatrix} \begin{bmatrix} \mathbf{x}\\ 0 \end{bmatrix} & \quad & \boldsymbol{\pi}_\infty = (0, 0, 0, 1)\\ &= \begin{bmatrix} \mathbf{m}_1^\top \mathbf{x}\\ \mathbf{m}_2^\top \mathbf{x}\\ \mathbf{v}^\top \mathbf{x} \end{bmatrix}\end{split}\]

(b)

The ratio of lengths on parallel line segments is an invariant under an affine camera. The proof is essentially the same as the one for affinities.

The previous exercise already demonstrated that parallel lines still map to parallel lines. \(\mathbf{l}_\infty\) and \(\boldsymbol{\pi}_\infty\) are also invariant.

(iv)

See [HS97] for more details on rational polynomial cameras. Beyond defining what this type of camera is and presenting results, there isn’t anything new per say. The coefficients are fitted using normalized DLT followed by Levenberg-Marquardt with a regularization term. Regularization is needed to reduce the variance of the solution.

(v)

Given a finite projective camera (6.11) \(\mathtt{P}\), observe that

\[\begin{split}\mathtt{R} \left[ \begin{array}{c|c} \mathtt{I} & -\tilde{\mathbf{C}} \end{array} \right] \mathtt{H}_0 = \begin{bmatrix} \mathbf{r}_1^\top & -\mathbf{r}_1^\top \tilde{\mathbf{C}}\\ \mathbf{r}_2^\top & -\mathbf{r}_2^\top \tilde{\mathbf{C}}\\ \mathbf{r}_3^\top & -\mathbf{r}_3^\top \tilde{\mathbf{C}} \end{bmatrix} \begin{bmatrix} \mathbf{r}_1 & \mathbf{r}_2 & \tilde{\mathbf{C}} & \mathbf{r}_3\\ 0 & 0 & 1 & 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1 \end{bmatrix} = \mathtt{P}_{\text{orthog}}\end{split}\]

and

\[\begin{split}\mathtt{K} \mathtt{P}_{\text{orthog}} \mathtt{H}_1 &= \begin{bmatrix} \alpha_x & s & x_0\\ & \alpha_y & y_0\\ & & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1 \end{bmatrix} \mathtt{H}_1 & \quad & \text{(6.10)}\\ &= \begin{bmatrix} \alpha_x & s & 0 & x_0\\ 0 & \alpha_y & 0 & y_0\\ 0 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} \frac{1}{\alpha_x} & -\frac{s}{\alpha_x \alpha_y} & 0 & \frac{-x_0}{\alpha_x} + \frac{s y_0}{\alpha_x \alpha_y}\\ 0 & \frac{1}{\alpha_y} & 0 & -\frac{y_0}{\alpha_y}\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1 \end{bmatrix}\\ &= \mathtt{P}_{\text{orthog}}.\end{split}\]

By inspection, \(\mathtt{H} = \mathtt{H}_0 \mathtt{H}_1\) has a rank of four and transforms \(\mathtt{P}\) to an orthographic camera via

\[\mathtt{P} \mathtt{H} = \mathtt{K} \mathtt{R} \left[ \begin{array}{c|c} \mathtt{I} & -\tilde{\mathbf{C}} \end{array} \right] \mathtt{H}_0 \mathtt{H}_1 = \mathtt{K} \mathtt{P}_{\text{orthog}} \mathtt{H}_1 = \mathtt{P}_{\text{orthog}}.\]

The action of any camera may be considered as a projective transformation of 3-space followed by orthographic projection because

\[\mathbf{x} = \mathtt{P} \left( \mathtt{H} \mathtt{H}^{-1} \right) \mathbf{X} = \left( \mathtt{P} \mathtt{H} \right) \left( \mathtt{H}^{-1} \mathbf{X} \right) = \mathtt{P}_{\text{orthog}} \mathbf{X}'.\]

References

HS97

Richard I Hartley and Tushar Saxena. The cubic rational polynomial camera model. In Image Understanding Workshop, volume 649, 653. 1997.