Junior Faculty Awardee Profile: Lydia Chilton, PhD

Tell me a little bit about your background and what brought you here.

I'm here in part because I love New York. I love places where you have a little bit of everything going on. It's not all tech. It's not all business. It's not all government. It's everybody working together. You never know what you're going to need to solve a problem. And as a test bed for solving problems, any problem the world has happens in New York first. 

What first sparked your interest in computer science?

I was a senior at MIT majoring in economics, working at an internship in China, and I realized that I was just analyzing economic programs that somebody else had built. I learned that engineering was not just analysis—it was a natural choice if you wanted to do things with data. Facebook was coming up, and I was interested in the ability to search anything in the knowledge production space.

Were there particular problems you were hoping to solve at that point?

I was doing all this economic stuff, which is largely pattern recognition—machine learning. And I wanted to solve problems like predicting where epidemics are going to occur—not based on 10-year-old census data, but on the things that people are searching for today. I began thinking that search logs contained all the information we need to know about the world because people tell their search engines things they won't even tell their spouse or their therapist.

I went back to MIT and did a super senior year and switched majors. After graduation, I went to the University of Washington. I found probably one of my most supportive mentors there, Jamie Steven, who is now the chief science officer at Microsoft. Then my advisor moved to Beijing, and then Stanford, and I went with him. In Asia, I was first exposed to real design—not just graphic design and making things pretty, but building things that fit the needs and abilities of people.

At the time, crowdsourcing was just emerging. McHugh's Mechanical Turk was a way to hire people for a few seconds for a few pennies to do a quick task. It's not just for simple things. You can build big programs—human/machine hybrids—out of this. Although not everyone uses Mechanical Turk, we still use the ideas to think about work in a more precise way. At the time, people really didn't understand what this had to do with computer science.

How have you applied that thinking in your current research?

So basically what I do now is I try to make people more productive on hard creative tasks by making the design process a little bit more concrete. If you go to most design workshops, they'll tell you about brainstorming: brainstorm, brainstorm, brainstorm. And I've sat in so many brainstorms and very rarely has the right answer just popped out. So although brainstorming clearly has some benefits, how does it really get used as a part of a solution? I want to mechanically understand all the pieces. There's no simple recipe. But we can still scaffold the creative process so we don't get stuck.

How would that play out in a room of creatives who are trying to solve a particular problem?

The keys of productivity are giving people all the assets and resources, and the tools to put them together in the right environment so that they can use them easily and in concert to achieve their goals. So if you're trying to create an ad for a particular message, you need to have like all the visual assets in front of you—ways to put them together. And right now, people are doing that with Photoshop, stitching things together using stock photography. Photoshop is also a highly expert tool and not a particularly fast one. But we can help even novices put them together in many ways.

Is this your visual blend?

Yes. We break down the creative process using A.I. tools to search for and synthesize images. My group uses it to make graphics for various organizations on campus. At the end of the summer, we will have an end user version for people to use. I've got three undergrads working on it now.

That's exactly it. Imagine that you have a new idea. First you have to prove it's possible. You have to then communicate what the heart of that is that people should be aware of. What I've learned is that you actually learn the most and get your next idea from making the tool, from taking those ideas and making up a production-worthy version of it, because then you see how people naturally use it.

You see the suite of problems and issues that arise in the real world, which are just as interesting. Each and every one of these frictions from idea to launch is something that we have to smooth out. And if you're stopping at the research prototype, you are going to get to the end and you want something that people can just ship. Once we deploy, we can observe its usage, its shortcomings and continue to iterate. I'm building that out this summer.

What potential end users do you hope to reach—and what are their needs?

I'm creating graphics and ads for events, which is important. People don't read—they want images. Journalism outlets know that articles and headlines and stock photography don't cut it anymore. People want interesting visuals because there are so many things fighting for their attention. So being able to produce that stuff quickly is really important, especially for editorial content and journalism, where you don't have two days to wait for an artist—you want the tools to create the image in ten minutes. We organize a lot of events and producing that material is not easy and not fast. So it's really to help groups get their messages, events, and news out. Even on this campus, how do you break through the noise to reach your target audience? You have to have a good message and good visuals.

What has the response been from designers?

As I develop these tools. I work with graphic designers, particularly students. Designers don't want to sit there and manipulate every pixel. Photoshop is really good at producing a pixel-perfect image at the end, but it's not that great at helping you explore. A trained designer can sketch by hand, but this makes their work faster and easier.

So I would say that in general, since we look pretty carefully at the whole pantheon of tasks, the danger of the computer scientist is thinking that you can do the whole job by yourself. We don’t want to replace the designer's job. We are striving to support people, and not assume that AI is the whole solution. Putting them together requires knowledge and unwritten rules.  

Democratizing design is extremely important because it's one of those things that seems like a burst of creativity, but it can actually be learned through a combination of principles and practice. And right now, there is no design major, but in reality, the jobs that most of us do involve a lot of design. All jobs in the future will be design-oriented, extracting human needs and creating tools to find solutions. Here at Columbia, I've worked with some of the journalism and architecture faculty who really know the built world—and they are great at interviewing people and eliciting problems. By bringing together this sort of problem-solving design mentality in elucidating community problems in the physical world around us, we can really start to tackle a new suite of issues that had previously been isolated. Solving big problems requires a far more interdisciplinary effort. I am using this funding to bring in a domain expert as an equal partner with this design team. 

That's wonderful—this idea of design thinking in everyday life. When you experience bad design, you realize how it can be an afterthought.

Exactly. The hard thing about design is it doesn't fit in any department. It's neither arts nor engineering. It's its own separate discipline. And some faculty are really pushing towards design as an undergraduate major. They might know how they tackle it within their field, but how can you reach people across other fields?  So I actually had a student leading this effort called Design Day, with a keynote and a panel of students who have found a path through Columbia in design, whatever their field is, and are now going off and working on that. And they are speaking about navigating design as a career, given that it's not a major. It is an Intellectual pursuit on top of a body of knowledge. The students and faculty are excited to work on this. We are also working with the Design and Entrepreneurship Center, with students in my class on user interface design.

You can't discover your passion in the spring of your senior year, as I did. You need to discover this your freshman year, so you can build a portfolio.

Can you speak about the classes you teach and the students and your approach to teaching?

I have so much to say about this. I require not only attendance, but everyone has to participate in every class. And I do this because it helps you stay engaged. If before you see the answer, you guess what the answer is, you're sort of warming up the right place in your brain to put it, and then when you hear the answer, you try to connect it to things you already know. You're just building out the structure that is your brain by adding nodes and connecting them the things you're already got. So that's why it's highly interactive. And I don't allow technology in my class. No phones, no screens. I really want students to engage and realize why something does not work. It's about getting it done, not just about showing up, and I try to facilitate the best conditions for that to happen. 

My class is half lecture and then half studios where we’re meeting with the students. And they do individual projects and some group projects. With group work, each member only gets to do a part of it. And usually it's women that end up doing the management; that's what happened to me. My students need to learn every part of the projects, so we mentor them as groups so they can learn from each other.  And you really have to look in their eyes and show them exactly how their code is failing, and say that if you don't fix us by next week, you're not going to get the points. So there's a lot of accountability in the system.  I wanted applications that students could literally deploy and make into startups—some really cool startups have come out of class projects.

Can you give an example?

One student developed Course Dog, a course scheduling application for administrators. Booking rooms is a huge problem for every school. Originally, he made a planner for students to plan their classes. But after talking to people, he realized that the real problem isn't the students: how does the school schedule these things so they don't conflict? He pivoted to solve that problem from the organizational perspective. He got Y Combinator funding and the Law School was one of their first clients. They are just taking over that academic organization space. 

Another project is an application that helps artists in New York find coffee shops with blank walls to meet the market of how artists can gain more visibility for their work. They realized that someone also has to hang the art, so they pick up the art, hang it, and add a little QR code with the price. They get some part of the profits, and everybody benefits. Now, the cafes also have art events. It’s great for students who do amazing photography, and would love to have the experience of an art pop-up in a coffee shop. 

If someone is reading this and they might be interested in collaborating in some way, what kind of collaborators are you looking for?

If you have a problem that you are solving in an ad hoc way, or you find it really slow and really laborious, we can come in. How can we create a coherent environment to give you all the tools you need to solve this? If you have something that is really killing your productivity, I want to know what that is, because you're not the only one that has it. There will be a solution that will, with some modification, be generalizable to other people.

Do you have any books, websites, applications or even podcasts that you'd recommend?

99% Invisible is a short design podcast that tells you all of the amazing engineering and design work to make things that you never thought about before. It really highlights how you just can start with a small idea and adopt what you have to solve your own problems. And those problems usually apply to other people, and I think that's really empowering. You don't need a million dollar investment—why seek funding?

Google Sites is a website builder that doesn’t require any coding. It's free and it comes with the Google suite. You can make a really nice looking site by yourself. You shouldn't have to hire a developer for something as simple as a homepage.

Assuming that you have any, how do you spend your free time?

I am a big NBA fan. My team is the Knicks, but they are hard to root for because they don't win very much. I also got married last fall, so it seems like my life events are now my hobbies.

Is there anything else that you would like to add?

I really love giving talks. It doesn't matter the size of the audience. If any faculty member or class would like my input on design, I'm happy to come anytime. 

To learn more about Dr. Chilton's work, visit her website or email her directly.