Spring 2014

ABSTRACT: The combined emergence of very large datasets, powerful parallel computers, and new machine learning methods, has enabled the deployment of highly-acurate computer perception systems, and is opening the door to a wide deployment of AI systems.

A key component in systems that can understand natural data is a module that turns the raw data into an suitable internal representation. But designing and building such a module, often called a feature extractor, requires a considerable amount of engineering efforts and domain expertise.

The main objective of ‘Deep Learning’ is to come up with learning methods that can automatically produce good representations of data from labeled or unlabeled samples. Deep learning allows us to construct systems that are trained end to end, from raw inputs to ultimate output. Instead of having a separate feature extractor and perdictor, deep architectures have multiple stages in which the data is represented hierarchically: features in successive stages are increasingly global, abstract, and invariant to irrelevant transformations of the input.

The convolutional network model (ConvNet) is a particular type of deep architecture that is somewhat inspired by biology, and consist of multiple stages of filter banks, interspersed with non-linear operations, and spatial pooling. ConvNets, have become the record holder for a wide variety of benchmarks and competition, including object detection, localization, and recognition in image, semantic image segmentation and labeling (2D and 3D), acoustic modeling for speech recognition, drug design, handwriting recognition, biological image segmentation, etc.

The most recent speech recognition and image understanding systems deployed by Facebook, Google, IBM, Microsoft, Baidu, NEC and others use deep learning, and many use convolutional networks. Such systems use very large and very deep ConvNets with billions of connections, trained using backpropagation with stochastic gradient, with heavy regularization. But many new applications require the use of unsupervised feature learning methods. A number of methods based on sparse auto-encoder will be presented.

Several applications will be shown through videos and live demos, including a category-level object recognition system that can be trained on the fly, a system that can label every pixel in an image with the category of the object it belongs to (scene parsing), a pedestrian detector, and object localization and detection systems that rank first on the ImageNet Large Scale Visual Recognition Challenge data. Specialized hardware architecture that run these systems in real time will also be described.

ABSTRACT: Programming is a skill that is best learned actively. I will discuss several websites that I have developed with this aim: Computer Science Circles (Python), Websheets (Java), and a Java execution visualizer.

While these are student-centric tools, the instructor’s perspective is also important. What do instructors learn about their students from these tools? How can we maximally enable creativity and efficiency on the part of the educator? Along the way I’ll discuss open-source software and correctional institutes.

ABSTRACT: Huge amounts of data are being collected everywhere when we browse the web, go to the doctor’s office, visit the supermarket, or watch a movie, we are providing information that fills in records on a database. Advances in fields like machine learning have shown promise with respect to digging through the data to make it more useful.

In the first part of my talk, I will present ongoing research by my group in medical informatics with emphasis on the application of machine learning to the prediction of preterm birth. I will present our analysis of a dataset collected by the NIH-NICHD Maternal Fetal Medicine Units (MFMU) Network, a high-quality data for over 3,000 singleton pregnancies having detailed study visits and biospecimen collection at 24, 26, 28 and 30 weeks gestation. Multiple processing steps were required to prepare this rich and highly structured data. We faced several challenges including: (1) presence of missing data, (2) varying sample size over time, (3) skewed class distributions due to preterm birth rate, and (4) the need to consider different subsets of data, such as nulliparous (first-time) mothers. In the second part, I will describe our efforts toward harnessing Electronic Health Records to prepare data for machine learning. I will show our preliminary work on prediction of preterm birth using a 5-year snapshot of data for mothers and babies from the New York Presbyterian Hospital EHR systems.

I will conclude my talk with my education and research objectives and how they intertwine with each other.

ABSTRACT: We propose a dynamic allocation procedure that increases power and efficiency when measuring an average treatment effect in fixed sample randomized trials with sequential allocation. Subjects arrive iteratively and are either randomized or paired via a matching criterion to a previously randomized subject and administered the alternate treatment. We develop estimators for the average treatment effect that combine information from both the matched pairs and unmatched subjects as well as an exact test. Simulations illustrate the method’s higher efficiency and power over several competing allocation procedures in both simulations and in data from a clinical trial.

ABSTRACT: The capacity of a communication channel is the maximum rate at which information can be reliably transmitted over the channel. In this work I consider the capacity of the binary deletion channel, where bits are deleted independently with a certain probability. This represents perhaps the simplest channel with synchronization errors but a characterization of its capacity remains an open question. I will present several techniques to lower bound the capacity, including Markov chain methods, Poisson-repeat channels, and ideas from renewal theory. The quality of these lower bounds is evaluated via numerical simulations.

ABSTRACT: In recent decades college graduation rates have soared for wealthier young adults but have stagnated for the poor. While many colleges are committed to admitting and supporting poorer students, they have been stymied by a dearth of qualified low income applicants. Studies have shown that a majority of highly qualified low income students employ inadequate strategies in the college application process and simply do not apply to the elite institutions that would likely accept them and offer support. A paucity of college counseling resources is largely to blame. Admitster.com is a web app that we hope will help bridge the college counseling gap by enabling students to more easily assess and improve their chances of gaining admissions at American universities. I will describe Admitster.com in the context of computer aided decision making. I’ll talk about technical issues that must be overcome in designing such a system. One particular issue that often occurs is when the variables for a dataset are only partially known for each observation. In such situations we expect machine learning results to get less robust as we increase the number of variables and therefore decrease the number of available observations. I’ll propose a pairwise variable ensembling technique that attempts to increase the reliability of predictions in these situations without imputing missing values.