Six SEAS students traveled this summer to Ghana to train university students to lead CS outreach that will impact hundreds of Ghanian students. The trip was organized by a nonprofit headed by Chelsey Roebuck (BS’10).
The paper, widely covered in the press, describes an automatic method for error-checking thousands to millions of neurons in a deep-learning neural network. Authors are Kexin Pei, Yinzhi Cao (Lehigh), Junfeng Yang, Suman Jana.
Developed by Kexin Pei, Yinzhi Cao (Lehigh), Junfeng Yang, and Suman Jana, DeepXplore error-checks thousands to millions of neurons in deep learning neural networks. In tests, DeepXplore found thousands of bugs missed by previous methods.
To uniquely identify and encode information in printed objects, Columbia researchers Dingzeyu Li, Avinash S. Nair, Shree K. Nayar, and Changxi Zheng have invented a process that embeds carefully designed air pockets, or AirCode tags, just below the surface of an object. By manipulating the size and configuration of these air pockets, the researchers cause light to scatter below the object surface in a distinctive profile they can exploit to encode information. Information encoded using this method allows 3D-fabricated objects to be tracked, linked to online content, tagged with metadata, and embedded with copyright or licensing information. Under an object’s surface, AirCode tags are invisible to human eyes but easily readable using off-the-shelf digital cameras and projectors.
The AirCode system has several advantages over existing tagging methods, including the highly visible barcodes, QR codes, and RFID circuits: AirCode tags can be generated during the 3D printing process, removing the need for post-processing steps to apply tags. Being built into a printed object, the tags cannot be removed, either inadvertently or intentionally; nor do they obscure any part of the object or detract from its visual appearance. Invisibility of the tags also means that the presence of information can remain hidden.
“With the increasing popularity of 3D printing, it’s more important than ever to personalize and identify objects,” says Changxi Zheng, who helped develop the method. “We were motivated to find an easy, unobtrusive way to link digital information to physical objects. Among their many uses, AirCode tags provide a way for artists to authenticate their work and for companies to protect their branded products.”
One additional use for AirCode tags is robotic grasping. By encoding both an object’s 3D model and its orientation into an AirCode tag, a robot would just need to read the tag rather than rely on visual or other sensors to locate the graspable part of an object (such as the handle of a mug), which might be occluded depending on the object’s orientation.
AirCode tags, which work with existing 3D printers and with almost every 3D printing material, are easy to incorporate into 3D object fabrication. A user would install the AirCode software and supply a bitstring of the information to be encoded. From this bitstring, AirCode software automatically generates air pockets of the right size and configuration to encode the supplied information, inserting the air pockets at the precise depth to be invisible but still readable using a conventional camera-projector setup.
How the AirCode system works
The AirCode system takes advantage of the scattering of light that occurs when light penetrates an object. Subsurface scattering in 3D materials, which is not normally noticeable to people, will be different depending on whether the light hits an air pocket or hits solid material.
Computational imaging techniques are able to differentiate reflective light from scattered light and decompose a photo into two separate components: a direct component produced by reflected light, and a global component produced from the scattered light waves that first penetrate an object. It’s this global component that AirCode tags manipulate to encode information.
AirCode represents the first use of subsurface light scattering to relay information, and the paper detailing the method, AirCode: Unobtrusive Physical Tags for Digital Fabrication, was named Best Paper at this year’s User Interface Software and Technology (UIST), the premier forum for innovations in Human-Computer Interfaces.
Other innovations are algorithmic, falling into three main steps:
Analyzing the density and optical properties of a material. Most plastic 3D printer materials exhibit strong subsurface scattering, which will be different for each printing material and will thus affect the ideal depth, size, and geometry of an AirCode structure. For each printing material, the researchers analyze the optical properties to model how light penetrates the surface and its interactions with air pockets.
“The technical difficulty here was to create a physics-based model that can predict the light scattering properties of 3D printed materials especially when air pockets are present inside of the materials,” says PhD student Dingzeyu Li, a coauthor on the paper. “It’s only by doing these analyses were we able to determine the size and depth of individual air pockets.”
Analyzing the optical properties of a material is done once with results stored in a database.
Constructing the AirCode tag for a given material. Like QR codes, AirCode tags are made up of cells arranged on a grid, with each cell representing either a 1 (air-filled) or a 0 (filled with printing material), according to the user-supplied bitstring. Circular-shaped markers, easily detected by computer vision algorithms, help orient the tag and locate different cell regions.
Unlike QR codes, which are clearly visible with sharp distinctions between white and black, AirCode tags are often noisy and blurred, with many levels of gray. To overcome these problems, the researchers insert predefined bits to serve as training data for calibrating in real time the threshold for classifying cells as 0s or 1s.
Detecting and decoding the AirCode tag. To read the tag, a projector shines light on the object (multiple tags might be used to ensure the projector easily finds a tag) and a camera captures images of the object. A computer vision algorithm previously created by coauthor Shree Nayar and adapted for AirCode tags separates each image into its direct component and global component, making the AirCode tag clearly visible so it can be decoded.
While the AirCode system has certain limitations—it requires materials to be homogeneous and semitransparent (though most 3D printer materials fit this description) and the tags become unreadable when covered by opaque paint—tagging printed objects during fabrication has substantial benefits in cost, aesthetics, and function.
Posted 10/24/2017
– Linda Crane
Interviewed by Columbia Technology Ventures (CTV), Allen discusses his recent work on surgical robots and how incorporating big data, machine learning, and computer vision is contributing to smarter robots able to cope with the unexpected.
The timing isn’t great—mid-semester with early projects coming due and midterms beginning—but still they come; 37 from Columbia and Barnard traveled to Orlando earlier this month to join 18,000 other women in tech for the Grace Hopper Celebration, an annual gathering co-produced by AnitaB.org (formerly the Anita Borg Institute) and the Association for Computing Machinery.
For three days (Oct 4-6) attendees listened to 16 keynote speakers—among them Melinda Gates, Fei-Fei Li (Professor and Director of Stanford’s AI Lab and Chief Scientist at Google Cloud), and Megan Smith (Former US Chief Technology Officer). They signed up for technical panels on AI, wearable technologies, data science, software engineering, and dozens of other innovative technologies. They networked with their peers and listened to pitches from recruiters who flock to the event.
But for many, the main draw is just being among so many other women who share their interests in computer science and engineering. In fields dominated by men, the Grace Hopper Celebration (GHC) is one of the few tech venues where women run the show. Here they are the speakers, panelists, and attendees, sharing what they love about technology and what they hope to accomplish in their careers or in their research. They share also stories of workplace discrimination, slights, and sometimes blatant sexism as well as tangible recommendations for what works to keep women in technology.
Columbia CS major Tessa Hurr, attending for the fourth time, describes it this way: “GHC is a community of women who are there to support one another and lift one another up and encourage one other to pursue a career in STEM.” A senior about to embark on a career, she especially wanted to hear from women about their work experiences. “Coming from Columbia, where the engineering school and computer science department have done a lot of work to balance the ratio of males to females, you see a lot of other women and you don’t feel alone. But in industry, you see the problem of gender imbalance so glaringly. Being at GHC, I know there are support systems if I need them.”
Women at GHC may be excited about supporting other women in technology, but they’re just as enthusiastic about technology itself and the good it can do in the world. Says Hurr, “Sometimes when you’re learning different concepts in class you don’t necessarily see how they translate over to the real world; GHC tech talks help bridge that gap so you better understand how you can have an impact on the world and work towards social good through tech.”
Myra Deng, a CS student attending for the first time, appreciated the emphasis on new technologies, especially AI. A talk by keynote speaker Fei-Fei Li linking AI and diversity was especially inspiring to Deng, who is on the Intelligent Systems track. “Professor Li talked about how AI is supposed to represent the entire human experience but you can’t really model or build that with just a small section of the human population. Diversity isn’t just being nice in the workplace; it’s essential to getting the technology right.”
This mix of technology and support system is a powerful thing, and GHC has been growing by leaps and bounds. In four years, GHC has grown from 8000 to 18,000 participants.
Many attend by taking advantage of scholarships offered by some of the big tech companies. “If the concern is finances, there are lots of resources, including a Github page listing scholarships,” says Julia Di, president of WiCS, which also sponsors students. This year WiCS raised enough funding to send 16 students, though only six were able to purchase tickets in the few hours it took before tickets sold out. Next year, WiCS may follow the lead of tech companies and make a donation to pre-purchase tickets.
Some scholarships require students take the entire week off, not just the three days of the conference, making GHC even more of a time commitment as students scramble to get school work done ahead of time, and scramble again to catch up when they return to campus. That so many do shows how much importance they attach to continuing in tech and supporting others who want to do the same.
Deng encourages women to make the most of the opportunity offered by GHC. “Every now and then, it’s good to zoom out from school and see what’s going on in the world. At GHC you meet so many incredible people you might not otherwise meet. I came back a lot more motivated because I know what I’m working on is important. It’s why I’m in Tech. You can always catch up on school work later.”
Posted 10/18/2017
– Linda Crane
For the headline Liquid Water Found on Mars, which response is the least funny? Hint: One is professionally done, and two are crowdsourced. Voting results at end.
Creativity and computation are often thought to be incompatible: one open-ended and requiring imagination, originality of thought, and perhaps even a little magic; the other logical, linear, and broken down into concrete steps. But solving the hard problems of today in medicine, environmental science, biology, and software engineering will require both.
For Lydia Chilton, who joined the Computer Science department this fall, inventing new solutions is fundamentally about design. “When people start solving a problem, they often brainstorm over a broad range of possibilities,” says Chilton, whose research focuses on human-computer interaction and crowdsourcing. “Then there is a mysterious process by which those ideas are transformed into a solution. How can we break down this process into computational steps so that we can repeat them for new problems?” This question motivates her research into tools that are part human intelligence, part computer intelligence.
How this works in practice is illustrated by a pipeline she built to automatically generate visual metaphors, where two objects, similar in shape but having different conceptual associations, are blended to create something entirely new. It’s a juxtaposition of images and concepts intended to communicate a new idea, doing so in a way that grabs people’s attention by upending expectations.
Chilton decomposes the process of creating visual metaphors into a series of microtasks where people and machines collaborate by working on those microtasks they are good at. Defining the microtasks and the pipeline to make them flow together coherently is the major intellectual piece.
“The key is to identify the pieces you will need, and what relationships the pieces need to have to fit together. After you define it that way, it becomes a search problem.” Because it’s a search problem over conceptual spaces computers don’t fully understand, Chilton has people fill in the gaps and direct the search. People might examine the space of objects representing Starbucks and the space representing Summer, picking the most simple, meaningful, and iconic. The computer then searches for pairs of similarly shaped objects (as annotated by people), blending them together into an initial mockup of the visual metaphor. Humans come in at the last step to tweak the blend to be visually pleasing. At every stage in the pipeline, humans and computers work together based on their different strengths.
Crowdsourcing serves another purpose in Chilton’s research: harnessing many people’s intuitions. Foreshadowing her work on pipelines, Chilton created crowd algorithms that, more than simply aggregating uninformed opinions or intuitions, aggregate intuitions in intelligent ways to lead to correct solutions.
For example, deciphering this intentionally illegible handwriting would not be possible for any single person, but a crowd algorithm enables people to work towards a solution iteratively. People in the crowd suggest partial solutions, and then others, also in the crowd, vote on which partial solution seems like the right one to continue iterating. Those in later stages benefit from seeing contextual clues and thus build on the current solution, even if they wouldn’t have had those insights without seeing others’ partial solutions. “It’s an iterative algorithm that keeps improving on the partial solutions in every iteration until the problem is solved,” says Chilton.
Allowing people to collaborate in productive ways is the power of crowd algorithms and interactive pipelines. Her research into crowdsourcing and computational design has already earned her a Facebook Fellowship and a Brown Institute Grant. This year, she was named to the inaugural class of the ACM Future of Computing Academy.
At Columbia, she will continue applying interactive pipelines and computational design to new domains: authoring compelling arguments for ideas, finding ways to integrate existing knowledge of health and nutrition into people’s existing habits and routines, and creating humor, a known, very hard problem for computers because of the large amount of implicit communication and emotional impact.
“Although humor is valuable as a source of entertainment and clever observations about the world, humor is also a great problem to study because it is a microcosm of the fundamental process of creating novel and useful things. If we can figure out the mechanics of humor, and orchestrate it in an interactive pipeline, we would be even further towards the grand vision of computational design that could be applied to any domain.”
Humor is also a realm where human intelligence is still necessary. Computers lack the contextual clues and real world knowledge that enable people to know intuitively that a joke insulting McDonald’s or Justin Bieber is funny but one that insults refugees or clean air is not. As she did for visual metaphors, Chilton breaks down the humor creation process into microtasks that are distributed to humans and machines. This pipeline, HumorTools, was created to compete with The Onion. It generated two of the responses to the liquid water headline. The Onion writers wrote the third.
“I pick creative problems that involve images (like visual metaphors) and text (like humor) because I think both are fundamental to the human problem-solving ability,” says Chilton. “Sometimes a picture says 1000 words, and sometimes words lay out logic in ways that might be deceiving in images. The department here is strong in graphics and in speech and language processing, and I look forward to collaborating with both groups to build tools that enhance people’s problem-solving abilities.”
One of the people she will collaborate with is Steven Feiner, who directs the Computer Graphics and User Interfaces Lab. “It’s important to extend people’s capabilities, augmenting them through computation expressed as visualization,” says Feiner. “Here, the hybrid approaches between humans and computers that Lydia is exploring are especially important because these are difficult problems that we do not yet know how to do algorithmically.”
Chilton’s first class, to be taught this spring, will be User Interface Design (W4170).
Voting results for headline Liquid Water Found on Mars.
Posted 10/17/2017
– Linda Crane
Add to GMail