The objects of interest in the billiards game (the table, the pockets and the balls) possess some properties which make it particularly well-suited for computer vision. An important feature that comes to mind is color. The green felt of the table and the distinctive contrasting colors of the balls and pockets allow robust color segmentation. In addition, the structures we are tracking have simple geometric shapes. The balls and pockets have a standard circular structure so radial symmetry can be used.
Figure: The Algorithm's Main Loop
Figure depicts the algorithm loop that iterates repeatedly over video data. The system begins by searching the image for a region that contains green felt colors. The largest green felt patch corresponds to the top of the table and its contour is determined. The interior of the table is then analyzed to find balls by computing symmetry. The centers of circular objects are detected in this region. Similarly, the periphery of the table is also processed to compute radial symmetry and generates symmetry peaks at the centers of the pockets. Each radial symmetry peak is then classified to determine which object generated it (ball, pocket, table). This is done by analyzing the colors of the symmetric objects and checking them against a database of color models for each ball, pocket and table. Once all symmetry peaks are classified, we can compute linear trajectories from the cue ball to every other solid or striped ball (depending on what the user selects). Then, we compute trajectories from each of these to a destination pocket. Each of these trajectories is evaluated to see if it is feasible and rated in difficulty depending on how accurate the shot must be to sink the ball. The easiest shot is then displayed on-screen using linear trajectories. These trace out the paths of the cue ball and the ball to sink.