Abstract for group VGLR

Vision-Guided Lego Robot

Peripheral: Lego Robot with motors and camera
Hardware/Software workload: Camera interface and image data
reception done in hardware. Image processing done in software.
Controller (consumes results from image processing and issues
commands to Lego motors) can be done either in software or
hardware.

We are going to build a Lego robot guided by camera inputs. We will
first make FPGA board work with a camera, being able to receive
video inputs. Then our software component of the project will
process the video input images to some data (maybe a short array of
bits indicating where the objects are blocking in the view). A
controller receives this result from the image processing stage and
issues commands to the Lego motors via the driver provided by Lego
through a USB wire. Finally, the Lego robot receives the commands
and make the correct turns to avoid the objects.

For the image processing stage done in software, given an image
matrix, we can either use machine learning techniques (such as
support vector machines), or just simple statistics. For example,
given an image vector A of length n, we want to output a result
array of bits of length 5, indicating where the object in front
occupy in the view. For example, a result array of "00000" means
there's no object in front. An array of "00001" means there's an
object on the extreme right side of the view. An array of "11111"
means the object completely blocks the view. To derive such a
function that maps the image vector to the result array, we can use
simple statistics to figure out, say, for the first 1/5 of the view
(if the view is sliced vertically), how many pixels are dark versus
how many are light. If the percentage of dark pixels is
overwhelming, we assign "31" to that 1/5 of the view, indicating
there is an object. Alternatively, we can use machine learning
classification methods to train on sample image vectors to decide
which result array a particular image vector maps to. Currently, we
prefer the simpler method, and will see how well it works.

With the result array mentioned above, a controller (probably a FSM)
is needed to map that array to an actual command. This controller
can be implemented in software or hardware.