DisCo: Displays that Communicate

We present DisCo, a novel display-camera communication system. DisCo enables displays and cameras to communicate with each other, while also displaying and capturing images for human consumption. Messages are transmitted by temporally modulating the display brightness at high frequencies so that they are imperceptible to humans. Messages are received by a rolling shutter camera which converts the temporally modulated incident light into a spatial flicker pattern. In the captured image, the flicker pattern is superimposed on the pattern shown on the display. The flicker and the display pattern are separated by capturing two images with different exposures. The proposed system performs robustly in challenging real-world situations such as occlusion, variable display size, defocus blur, perspective distortion and camera rotation. Unlike several existing visible light communication methods, DisCo works with off-the-shelf image sensors. It is compatible with a variety of sources (including displays, single LEDs), as well as reflective surfaces illuminated with light sources. We have built hardware prototypes that demonstrate DisCo's performance in several scenarios. Because of its robustness, speed, ease of use and generality, DisCo can be widely deployed in several CHI applications, such as advertising, pairing of displays with cell-phones, tagging objects in stores and museums, and indoor navigation.

Publications

"DisCo: Display Camera Communication Using Rolling Shutter Sensors,"
K. Jo, M. Gupta and S.K. Nayar,
ACM Trans. on Graphics (also Proc. of ACM SIGGRAPH),
Vol. 35, No. 5, pp. 150:1-13, Jul. 2016.
[PDF] [bib] [©]

"DisCo: Displays that Communicate,"
K. Jo, M. Gupta and S. K. Nayar,
Technical Report, Department of Computer Science, Columbia University,
Dec. 2014.
[PDF] [bib] [©]

Images

  Concept of DisCo:

DisCo can work with a broad range of devices. (a) For transmitters, DisCo can use display monitors, light sources or objects illuminated by light as the transmitter (Display). (b) The display sends information to the receiver, which can be implemented using rolling shutter cameras. (c) The display also shows images for human consumption, while simultaneously communicating with the camera.

     
  Input and Output of DisCo with Simultaneous Dual Exposure (SDE) Sensor:

(a) Exposure timing of an SDE rolling shutter sensor with two different exposures. (b) An example captured image (input) with a rolling shutter SDE sensor. The display was modulated with a 500 Hz. temporal sinusoid. (c-d) Images of different exposures are extracted from (b), from which the signal (e) and the flicker-free image (f) are recovered (output).

     
  Experimental Demonstration of DisCo in Challenging Imaging Scenarios:

We have extensively tested the proposed system in a variety of real-world imaging situations. (a) Display smaller than camerafs FOV due to large display-camera distance. (b) Display larger than camerafs FOV due to large zoom/small display-camera distance. (c) Display blurred due to camera defocus. (d-e) Occluding objects between camera and display. In these examples, the signal (URL https://www.google.com/) was transmitted and received by the camera. The system adapts to challenging conditions by capturing multiple frames.

     

Video

  Project Video:

This video introduces the concept of DisCo, summarizes the signal decode algorithm, and demonstrates several applications of our method, including advertising and communicating meta-information, pairing of devices, and Indoor navigation and location specific services. (With narration)