A wide array of possible perceptual inputs and possible output modalities can be considered for a system that interacts with and learns from human users. A primary concern in the perceptual system is the recovery of action parameters which are particularly expressive and interactive. In addition, to maintain real-time interactiveness and fast training, the input/output parameters should be compact (low dimensional). Another equally important attribute of the perceptual features is linearity and smooth behaviour in their parametric space. This property is more difficult to characterize and the representation of data is an ongoing research issue. It is sufficient to say that the learning system already has a significant task in learning mappings between actions and reactions to acquire behaviour. The difficulties in the learning task can be severely compounded by the use of spurious, complex or strangely non-linear perceptual inputs. Thus, in designing a perceptual system, we begin cautiously and use only the simplest features.