Arbitrary Motion

Let O be a set of $N$ points on the planar trajectory of a point and let ${\bf P}_l$ be its images using an affine camera in view ${\bf V}_l$, where $l$ is the view index. Let $(u^l[i], v^l[i])$ be the image coordinates of points on the contour traced out in view ${\bf V}_l$. We represent this contour using a sequence of vectors of complex numbers as given below.

$${\bf x}^l[i] = \left[ \begin{array}{c} u^l[i] + j 0 v^l[i] + j 0 \end{array} \right]$$

Under affine projection, the image-to-image homography between a pair of views is affine also. Thus, the corresponding points of the contour in view $l$ are related to the points in the reference view $0$ by the relation

$${\bf x}^l[i] = {\bf A}_l {\bf x}^0[i] + {\bf b_l}, \;\; 0 \leq i < N$$ (15)

where ${\bf A_l}$ is an arbitrary $2\times 2$ matrix. Taking the Fourier transform of Equation 15, we get

$${\bf X}^l[k] = {\bf A_l} {\bf X}^0[k] + {\bf b_l} \delta[0],\;\; 0 \leq k < N$$

which can be rewritten as (ignoring the DC component k = 0)

$${\bf X}^l[k] = {\bf A_l} {\bf X}^0[k],\;\; 0 < k < N$$ (16)

where ${\bf X}^l = \left[{\bf U}^l,\;\; {\bf V}^l\right]^{\sc T}$ ; ${\bf U}^l$ and ${\bf V}^l$ are Fourier transform sequences of $u^l$ and $v^l$ respectively. We can now define the following measure $\kappa$ for the points on the contour in the view $l$ as

$$\kappa^l[k] = ({\bf {X}}^l[k])^{*{\sc T}} \left[ \begin{arr... ...1\;\; & 0 \end{array} \right] {\bf\bar{X}}^l[k],\;\; 0 < k < N$$ (17)

(* denotes the complex conjugate). It can be shown that [10]

$$\kappa^l[k] = \vert{\bf A_l}\vert\; \kappa^0[k],\;\; 0 < k < N.$$ (18)

where, $\vert{\bf A_l}\vert$ denotes the determinant of ${\bf A_l}$. The $\kappa$ values defined by Equation 17, which can be computed independently for each view from the Fourier transform of the contour points, identify the contour formed by the motion, upto a scale factor. Consider the following $M\times (N-1)$ matrix for $M$ views of the planar contour

$$\Theta = \left[ \begin{array}{cccc} \kappa^0[1] & \kappa^... ...1] & \cdots & \cdots & \kappa^{M-1}[N-1] \end{array} \right].$$ (19)

It can be seen from Equation 18 that rank$(\Theta) = 1$. See reference [10] for a detailed discussion on rank constraints for shape matching.

Equation 19 gives a constraint on measures that can be computed from each view independently. The trajectory of a point moving in a plane can be tracked over time to generate the contour in each view. This constraint is view and time independent. There are no restrictions on the number of views or frames per se. In practice, however, the Fourier transform will be reliable only if the curve has sufficient length. The motion of the point is arbitrary. If a number of points can be tracked independently, each contour will yield a different constraint, all of which have to be satisfied simultaneously. It is clear that non-rigid motion is also covered by these constraints.