Tomer Aharoni (GS ’19) will present “Nagish”, a software that helps people with hearing and speech difficulties communicate over the phone, at the Google Cloud Next ‘19 conference in April.
It
all started with a voice message and follow up text to Aharoni – his friend
needed him to reply and he was in class and could not use his phone. Curious
about what was so important, he decided to create a hack – he took the sound
output of his computer and redirected it into the input, then he opened a
Google Doc, selected the dictation option, played the voice message and the app
translated the audio into text. He was finally able to read his friend’s
message.
“I
realized that I might be onto something,” said Aharoni, a veteran of the Israeli Defense Force
who is a
general studies student majoring in computer science. “The
next week I teamed up with friends to join the DevFest hackathon and we won
second place.”
The decision to join DevFest,
Columbia’s annual hackathon, proved to be a way to open doors for Nagish. Aside
from the second place win, the team also won the Best Hack for
Social Good award and caught the eye of Google Cloud.
For the first iteration of Nagish, the team stayed up all night to create a product linked to only one phone number. They had to pre-code the number and link it to Facebook messenger. When Google showed interest and gave them a grant through the Google Cloud credits program, they focused on using Google’s APIs and added other messenger apps to reach more users.
Each call begins with a text message informing the registered user that they have an incoming call. The user can either pick up or decline the call. If the user picks up the call, the caller is informed that this is a special phone call for someone with hearing or speech difficulties so they know that they should wait on the line.
From then on the caller can speak and whatever they say will be translated into text. While the registered user will be able to reply via text that will be translated into a voice message for the caller. So one side of the call can type and read and the other side can speak and hear. The reverse is also true – registered users can place calls with the help of the software.
Video of Nagish from DevFest 2018
“The first time we built it it was like a
fraction of a second of delay,” shared Aharoni. “It sounds like nothing but the
delay is super annoying when you speak with someone over the phone.”
Since then they have worked on making the software more efficient and faster in time for beta testing set for March. Right now the team is looking for 10 people with hearing or speech difficulties to participate in a day of testing.
One of the things that is holding them back
at the moment is the cost of using Nagish. While they have a grant from Google,
not all of the APIs they use are from that suite of products. To connect the
call to Nagish they use an API called Twilio which is costly. Each incoming or
outgoing call costs 20 cents per minute. Even though they plan to release
Nagish for free, users still have to pay Twilio to be able to use Nagish.
Tomer Aharoni and Alon Ezer at DevFest 2019. Photo credit : Vivian Shen
In the beginning, Aharoni had little
experience in programming and credits the past year of working on Nagish as
invaluable. “I learned a lot from my classes but it is different to actually
work on a project and build my skills that way,” he said. Since DevFest he has continuously
worked with Alon Ezer, who also gave him a lot of guidance in the beginning. Now
he knows the whole process from project conception, implementation, to testing.
He hopes to release Nagish, which means accessible in Hebrew, for free sometime
this year.
Bloomberg also approached him after the
hackathon to apply as an intern. He spent last summer as an intern for the equities
group within the engineering department and will join the company as an employee after graduation this May.
“We’ve spent many hours working on Nagish to help others but all that
hard work has been fulfilling in more ways than one,” said Aharoni.
Composed of computer science students – Victor Lecomte, Bicheng Gao, and Aditya Shah – the team finished first place at the greater New York regional championship held last November at Manhattan College.
“The
teamwork aspect of this competition is really nice,” said Victor Lecomte, a
first year MS student who has tried for five years to get into the world
finals. “Working together as a team makes the competition even more fun.”
A
unique aspect of this programming competition is that teams have five hours to
complete ten computational problems with only one computer to use
between them. Lecomte shared that their game plan was to get the easiest
problems out of the way quickly then figure out which harder problems to
tackle. By the end of five hours they had finished eight problems.
Coaches
Chengyu Lin and Peilin Zhong helped Team Kington prepare for the regionals by holding
weekend training sessions for two months. Through that the team got to know each
other and figure out how to best work together. In preparation of the world
finals they are headed to Beijing, China for a week-long intensive training
camp at the end of February.
The
training camp serves as a boot camp with a five-hour contest in the morning, a
workshop to review the problems in the afternoon and evenings reserved for even
more practice. Participating teams are also regional winners. Teams are ranked
so it is much like a real competition. Continued Lecomte, “You can usually
figure out who the top teams are and who will likely win in the championship.”
On-campus
training sessions are open to everyone that want to sharpen their skills, shared
Chengyu Lin, one of the coaches and a third year PhD student. The sessions are
also a good way to prepare for potential interview questions as the practice
problems are often the same as what Google recruiters ask. Lecomte has been
contacted by recruiters because of his involvement in the competition.
“This
is challenging
work but if you love programming, if you love algorithms, this is definitely a
way to improve your coding skills,” said Lin.
Kara Schechtman
(CC ‘19) has been selected as one of the recipients of the Senator
George J. Mitchell Scholarship Program to pursue a
one-year postgraduate degree in Ireland. The fellowship is awarded to 12 individuals between the ages of 18
and 30 by the US-Ireland Alliance.
Schechtman is headed to Trinity College Dublin where she will study philosophy.
“Artificial
intelligence is advancing and we are at a point where ethics has to be
considered,” said Schechtman, who is majoring in English and
computer science at Columbia. “I believe studying philosophy will help me
prepare for further studies in computer science.”
A point of frustration
for her has been finding an area in computer science where both the
humanities and computing overlap in a way that fits her interests. In artificial
intelligence (AI), computing and philosophical questions can overlap, an
intersection she finds fulfilling.
The development of AI poses a whole gamut of challenges to humanity, ranging from legislation challenges, to AI bias, to ethical concerns about potential machine consciousness, and even to potential existential threat. But these challenges have remained while the technical developments of AI has grown leaps and bounds.
One thing Schechtman
hopes to answer through her studies is how society can act responsibly despite
all that is unknown. “I think it also demands technical expertise to suggest
actionable paths for responsibility now,” continued Schechtman. “Which is why
it is so important for computer scientists and philosophers to work together,
and for some people to study both.”
The
Ireland location of the fellowship is also ideal. Dublin hosts the European
Union headquarters of a number of tech giants. And having double majored in
English, she looks forward to “nerding out” over Samuel Beckett and James Joyce
in their home country.
“More broadly, the circumstances couldn’t be better — the other fellowship recipients seem amazing and I can’t wait to get to know them better, Trinity College Dublin is a wonderful school, and I am sure I will have a lot of fun exploring Ireland. I’m excited to grow from the experience in ways I don’t yet even expect,” said Schechtman.
The Computing Research Association (CRA) awarded Serina Chang as one of four awardees of the 2019 Outstanding Undergraduate Researcher Award. Three other computer science students were also nominated, including Ruiqi Zhong, Justin Whitehouse, Hamed Nilforoshan, all of whom received honorable mentions. All four undergraduates were recognized for their work in an area of computing research.
“I am always looking for innovative ways to combine computer science and social science,” said Serina Chang, a senior studying computer science and sociology. “Not only looking at research methods but to have a social impact.”
The paper on gang-involved youth in Chicago is one example of how computer science and sociology can intersect. The research continued the work of Desmond Patton, a professor at the School of Social Work and a Data Science Institute member, who found expression of loss over the death of a friend often led to tweets of aggression and real-world violence.
One of their key insights include that domain-specificity and context are extremely important. Interpreting the tweets was also context-critical, as it often depends on the emotional state of the user or what they had been tweeting prior to the current one, and may also refer to context such as the names of rival gangs or deceased friends.
The researchers hypothesized that domain-specificity and context would be important, but were surprised by the degree to which it affected convolutional neural network performance. For example, one set of experiments compared the word embeddings trained on domain-specific unlabeled corpus against word embeddings that trained on other corpora, including standard GloVE and Google News as well as an African-American vernacular English-specific corpus and a location-specific one, which they drew from the same area of Southside Chicago as their users.
The results found that their word embeddings outperformed the others by nearly 4 F1 points, which demonstrates the importance of constructing a large domain-specific corpus instead of resorting to “similar” corpora based on location or demographics. Said Chang, “Our hope is that we can eventually use our system to save critical time, scale their reach, and intervene before more young lives are lost.”
Chang’s thoughtfulness extends into other parts of her life. In her sophomore year she became a teaching assistant so she could work with students in Professor Paul Blaer’s data structures in JAVA course. She worked one-on-one with students to help them with assignments, with the additional goal of building their confidence when it comes to their abilities.
Mentoring others also came hand-in-hand with her involvement in Lean In at Columbia and Womxn in Computer Science (WiCS). In both organizations Chang held a leadership position and made it a point to foster relationships and empower women through various programs that celebrated diversity and feminism.
“I do still have time to study,” laughed Chang when asked about time management. This semester she is focused on working on her thesis. Grad school is also on the horizon and she will make her final decision in April. Part of the CRA award is financial assistance to attend a conference of their choice, she is still unsure of where she wants to go.
Looking back at all she has accomplished while at Columbia makes her feel grateful for all the opportunities she had to work with colleagues and professors, the relationships that have grown with friends and family, and the research that she hopes to continue.
“I am incredibly honored by this award and excited to be recognized for research that I am so passionate about,” said Chang.
Dean Boyce's statement on amicus brief filed by President Bollinger
President Bollinger announced that Columbia University along with many other academic institutions (sixteen, including all Ivy League universities) filed an amicus brief in the U.S. District Court for the Eastern District of New York challenging the Executive Order regarding immigrants from seven designated countries and refugees. Among other things, the brief asserts that “safety and security concerns can be addressed in a manner that is consistent with the values America has always stood for, including the free flow of ideas and people across borders and the welcoming of immigrants to our universities.”
This recent action provides a moment for us to collectively reflect on our community within Columbia Engineering and the importance of our commitment to maintaining an open and welcoming community for all students, faculty, researchers and administrative staff. As a School of Engineering and Applied Science, we are fortunate to attract students and faculty from diverse backgrounds, from across the country, and from around the world. It is a great benefit to be able to gather engineers and scientists of so many different perspectives and talents – all with a commitment to learning, a focus on pushing the frontiers of knowledge and discovery, and with a passion for translating our work to impact humanity.
I am proud of our community, and wish to take this opportunity to reinforce our collective commitment to maintaining an open and collegial environment. We are fortunate to have the privilege to learn from one another, and to study, work, and live together in such a dynamic and vibrant place as Columbia.
Sincerely,
Mary C. Boyce
Dean of Engineering
Morris A. and Alma Schapiro Professor