When 108,000 students enrolled in her inaugural online edX artificial intelligence course, Ansaf Salleb-Aouissi and two assistants divided up among them the task of handling student questions. There was frustration on both sides; students sometimes had to wait up to 48 hours for a reply, and Salleb-Aouissi and her small team found themselves answering the same questions multiple times, many of them logistical in nature (“what materials can I use for the final”), leaving less time for those related directly to course content.
Even after putting together a FAQ, the volume of questions remained high.
With $20K in funding provided through a Provost Award for Hybrid Learning, enough to fund at least two graduate students, Salleb-Aouissi will now begin investigating building a chatbot to automatically answer student questions. Collaborating closely with Columbia Video Network (CVN), which produces Columbia Engineering edX courses, the goal will be a chatbot intelligent enough to “converse” with students in real time.
“We will first look at the state-of-the-art chatbots that are out there now,” says Salleb-Aouissi. “If there is open-source software that can work on our data, it makes sense to begin there.”
Though chatbots are currently in use—including by some MOOCs—machine question and answering is far from solved; it is in fact a hard problem and an active area of research. Many speech interfaces, including Siri and Alexa, involve relatively simple tasks—scheduling meetings, setting alarms, searching the web—with little back-and-forth conversation. These systems work by transcribing speech to text and then attempting to understand and classify the meaning. The answer, once retrieved, is transcribed back to speech. Some of the more difficult questions may be handled through hand-coding or automated to some extent using AI.
But true conversation, where an answer begets another question, will require much more intelligence and more automation, which Salleb-Aouissi hopes to achieve using methods borrowed from AI and machine learning.
“There is research now into using deep-learning models for text, and while we know they work well with images, text is a much, much harder problem than images. Text has many layers—lexical, semantic, contextual, metaphorical. There is ambiguity and intentional double-meaning in words, and understanding what is meant often requires considering the specific context or employing real-world knowledge. Conversation with all its stops and starts and disfluencies adds to the complexity. We will be looking to put together an interdisciplinary team with expertise in machine learning, in psychology, and of course natural language processing, which is necessary to build a language model for predicting, in the context of our class, the next word in a sentence.”
Predicting the probability of the next word will require data, the more the better. In Salleb-Aouissi’s case, the data will consist of questions taken from her MOOC supplemented with others from discussion forums (e.g., Piazza) of her on-campus AI courses, as well as from text-mining the course material. One requirement is for the chatbot to be trainable on new data so it can be used in other MOOCs (such as the three other Columbia classes that are part of the edX AI series).
The chatbot represents an ambitious project; still she hopes to have a proof of concept in a year. “You can accomplish a lot working with two or three students, especially when they are excited about artificial intelligence, and always willing to try new ideas, new avenues. It’s an inspiration to work with them.”
– Linda Crane