CS Student to Present at Google Cloud Next ’19

Tomer Aharoni (GS ’19) will present “Nagish”, a software that helps people with hearing and speech difficulties communicate over the phone, at the Google Cloud Next ‘19 conference in April.

It all started with a voice message and follow up text to Aharoni – his friend needed him to reply and he was in class and could not use his phone. Curious about what was so important, he decided to create a hack – he took the sound output of his computer and redirected it into the input, then he opened a Google Doc, selected the dictation option, played the voice message and the app translated the audio into text. He was finally able to read his friend’s message.

“I realized that I might be onto something,” said Aharoni, a veteran of the Israeli Defense Force who is a general studies student majoring in computer science. “The next week I teamed up with friends to join the DevFest hackathon and we won second place.”

The decision to join DevFest, Columbia’s annual hackathon, proved to be a way to open doors for Nagish. Aside from the second place win, the team also won the Best Hack for Social Good award and caught the eye of Google Cloud.

For the first iteration of Nagish, the team stayed up all night to create a product linked to only one phone number. They had to pre-code the number and link it to Facebook messenger. When Google showed interest and gave them a grant through the Google Cloud credits program, they focused on using Google’s APIs and added other messenger apps to reach more users.

Each call begins with a text message informing the registered user that they have an incoming call. The user can either pick up or decline the call. If the user picks up the call, the caller is informed that this is a special phone call for someone with hearing or speech difficulties so they know that they should wait on the line.

From then on the caller can speak and whatever they say will be translated into text. While the registered user will be able to reply via text that will be translated into a voice message for the caller. So one side of the call can type and read and the other side can speak and hear. The reverse is also true – registered users can place calls with the help of the software.

Video of Nagish from DevFest 2018

“The first time we built it it was like a fraction of a second of delay,” shared Aharoni. “It sounds like nothing but the delay is super annoying when you speak with someone over the phone.”

Since then they have worked on making the software more efficient and faster in time for beta testing set for March. Right now the team is looking for 10 people with hearing or speech difficulties to participate in a day of testing.

One of the things that is holding them back at the moment is the cost of using Nagish. While they have a grant from Google, not all of the APIs they use are from that suite of products. To connect the call to Nagish they use an API called Twilio which is costly. Each incoming or outgoing call costs 20 cents per minute. Even though they plan to release Nagish for free, users still have to pay Twilio to be able to use Nagish.

Tomer Aharoni and Alon Ezer at DevFest 2019. Photo credit : Vivian Shen

In the beginning, Aharoni had little experience in programming and credits the past year of working on Nagish as invaluable. “I learned a lot from my classes but it is different to actually work on a project and build my skills that way,” he said. Since DevFest he has continuously worked with Alon Ezer, who also gave him a lot of guidance in the beginning. Now he knows the whole process from project conception, implementation, to testing. He hopes to release Nagish, which means accessible in Hebrew, for free sometime this year.

Bloomberg also approached him after the hackathon to apply as an intern. He spent last summer as an intern for the equities group within the engineering department and will join the company as an employee after graduation this May.

“We’ve spent many hours working on Nagish to help others but all that hard work has been fulfilling in more ways than one,” said Aharoni.

Columbia Advances To ICPC World Finals

A Columbia team has advanced to the world finals of the International Collegiate Programming Contest (ICPC). They will go to Porto, Portugal to compete against teams from universities around the world.

Composed of computer science students – Victor Lecomte, Bicheng Gao, and Aditya Shah – the team finished first place at the greater New York regional championship held last November at Manhattan College.

“The teamwork aspect of this competition is really nice,” said Victor Lecomte, a first year MS student who has tried for five years to get into the world finals. “Working together as a team makes the competition even more fun.”

A unique aspect of this programming competition is that teams have five hours to complete ten computational problems with only one computer to use between them. Lecomte shared that their game plan was to get the easiest problems out of the way quickly then figure out which harder problems to tackle. By the end of five hours they had finished eight problems.

Coaches Chengyu Lin and Peilin Zhong helped Team Kington prepare for the regionals by holding weekend training sessions for two months. Through that the team got to know each other and figure out how to best work together. In preparation of the world finals they are headed to Beijing, China for a week-long intensive training camp at the end of February.

The training camp serves as a boot camp with a five-hour contest in the morning, a workshop to review the problems in the afternoon and evenings reserved for even more practice. Participating teams are also regional winners. Teams are ranked so it is much like a real competition. Continued Lecomte, “You can usually figure out who the top teams are and who will likely win in the championship.”

On-campus training sessions are open to everyone that want to sharpen their skills, shared Chengyu Lin, one of the coaches and a third year PhD student. The sessions are also a good way to prepare for potential interview questions as the practice problems are often the same as what Google recruiters ask. Lecomte has been contacted by recruiters because of his involvement in the competition.

“This is challenging work but if you love programming, if you love algorithms, this is definitely a way to improve your coding skills,” said Lin.

Exploring The Intersection Of AI And Philosophy

Kara Schechtman (CC ‘19) has been selected as one of the recipients of the Senator George J. Mitchell Scholarship Program to pursue a one-year postgraduate degree in Ireland. The fellowship is awarded to 12 individuals between the ages of 18 and 30 by the US-Ireland Alliance. Schechtman is headed to Trinity College Dublin where she will study philosophy.

“Artificial intelligence is advancing and we are at a point where ethics has to be considered,” said Schechtman, who is majoring in English and computer science at Columbia. “I believe studying philosophy will help me prepare for further studies in computer science.”

A point of frustration for her has been finding an area in computer science where both the humanities and computing overlap in a way that fits her interests. In artificial intelligence (AI), computing and philosophical questions can overlap, an intersection she finds fulfilling.

The development of AI poses a whole gamut of challenges to humanity, ranging from legislation challenges, to AI bias, to ethical concerns about potential machine consciousness, and even to potential existential threat. But these challenges have remained while the technical developments of AI has grown leaps and bounds.

One thing Schechtman hopes to answer through her studies is how society can act responsibly despite all that is unknown. “I think it also demands technical expertise to suggest actionable paths for responsibility now,” continued Schechtman. “Which is why it is so important for computer scientists and philosophers to work together, and for some people to study both.”

The Ireland location of the fellowship is also ideal. Dublin hosts the European Union headquarters of a number of tech giants. And having double majored in English, she looks forward to “nerding out” over Samuel Beckett and James Joyce in their home country.

“More broadly, the circumstances couldn’t be better — the other fellowship recipients seem amazing and I can’t wait to get to know them better, Trinity College Dublin is a wonderful school, and I am sure I will have a lot of fun exploring Ireland. I’m excited to grow from the experience in ways I don’t yet even expect,” said Schechtman.

CS Undergraduate Students Recognized By The Computing Research Association

The Computing Research Association (CRA) awarded Serina Chang as one of four awardees of the 2019 Outstanding Undergraduate Researcher Award. Three other computer science students were also nominated, including Ruiqi Zhong, Justin Whitehouse, Hamed Nilforoshan, all of whom received honorable mentions. All four undergraduates were recognized for their work in an area of computing research.

“I am always looking for innovative ways to combine computer science and social science,” said Serina Chang, a senior studying computer science and sociology. “Not only looking at research methods but to have a social impact.”

Chang’s work on detecting emotion in social media, specifically tweets of Chicago gang members, led to a paper in EMNLP ’18. Under the guidance of Professor Kathy Mckeown she also published a paper on automatic text summarization in EACL ’17.

The paper on gang-involved youth in Chicago is one example of how computer science and sociology can intersect. The research continued the work of Desmond Patton, a professor at the School of Social Work and a Data Science Institute member, who found expression of loss over the death of a friend often led to tweets of aggression and real-world violence.

One of their key insights include that domain-specificity and context are extremely important. Interpreting the tweets was also context-critical, as it often depends on the emotional state of the user or what they had been tweeting prior to the current one, and may also refer to context such as the names of rival gangs or deceased friends.

The researchers hypothesized that domain-specificity and context would be important, but were surprised by the degree to which it affected convolutional neural network performance. For example, one set of experiments compared the word embeddings trained on domain-specific unlabeled corpus against word embeddings that trained on other corpora, including standard GloVE and Google News as well as an African-American vernacular English-specific corpus and a location-specific one, which they drew from the same area of Southside Chicago as their users.

The results found that their word embeddings outperformed the others by nearly 4 F1 points, which demonstrates the importance of constructing a large domain-specific corpus instead of resorting to “similar” corpora based on location or demographics. Said Chang, “Our hope is that we can eventually use our system to save critical time, scale their reach, and intervene before more young lives are lost.”

Chang’s thoughtfulness extends into other parts of her life. In her sophomore year she became a teaching assistant so she could work with students in Professor Paul Blaer’s data structures in JAVA course. She worked one-on-one with students to help them with assignments, with the additional goal of building their confidence when it comes to their abilities.

Mentoring others also came hand-in-hand with her involvement in Lean In at Columbia and Womxn in Computer Science (WiCS). In both organizations Chang held a leadership position and made it a point to foster relationships and empower women through various programs that celebrated diversity and feminism.

“I do still have time to study,” laughed Chang when asked about time management. This semester she is focused on working on her thesis. Grad school is also on the horizon and she will make her final decision in April. Part of the CRA award is financial assistance to attend a conference of their choice, she is still unsure of where she wants to go.

Looking back at all she has accomplished while at Columbia makes her feel grateful for all the opportunities she had to work with colleagues and professors, the relationships that have grown with friends and family, and the research that she hopes to continue.

“I am incredibly honored by this award and excited to be recognized for research that I am so passionate about,” said Chang.