Ishaan Preetam Chandratreya

I graduated with a B.S. in Computer Science ( magna cum laude ) from Columbia University, with a focus on Intelligent Systems, Computer Vision and Robotics. I am interested in building representations of the 3D world that lend themselves to robust predictive models of physical trajectories, and enable direct downstream applications in robotics and design. I have also previously worked on vision-and-language in the context of "foundation" models, and am interested in how we can interface the "coarse" representational power of these models with the finer, exact representations that we use to reason about trajectories.

Email  /  CV  /  Github

profile photo

I am fortunate to be advised by Prof. Carl Vondrick and have worked with an inspiring group of collaborators as part of CV Lab on a range of problems in cross-modal learning. I have also been fortunate to work on projects co-advised with Prof. Shuran Song, Prof. Dan Roth, Prof. Richard Zemel , and Prof. Hod Lipson.

There is a Time and Place for Reasoning Beyond the Image
Xingyu Fu, Ben Zhou*, Ishaan Chandratreya*, Carl Vondrick, Dan Roth
ACL 2022 (Oral)
arXiv / Code+Models

In this work, we formulate this problem and introduce TARA: a dataset with 16k images with their associated news, time and location automatically extracted from New York Times (NYT), and an additional 61k examples as distant supervision from WIT. We show that there exists a 70% gap between a state-of-the-art joint model and human performance, which is slightly filled by our proposed model that uses segment-wise reasoning, motivating higher-level vision-language joint models that can conduct open-ended reasoning with world knowledge.

Discovering State Variables Hidden in Experimental Data
Boyuan Chen, Kuang Huang, Sunand Raghupathi, Ishaan Chandratreya, Qiang Du, Hod Lipson
Nature Computational Science
Project Page / arXiv / Video / Code+Models

Most data-driven methods for modeling physical phenomena still assume that observed data streams already correspond to relevant state variables. A key challenge is to identify the possible sets of state variables from scratch, given only high-dimensional observational data. Here we propose a new principle for determining how many state variables an observed system is likely to have.

Class Projects

At Columbia, I've completed coursework in both theory and practice of machine learning and algorithms. You can find some of my key projects here.

Exploring Test Time Optimization for Discrete Neural Representation Learners
Ishaan Chandratreya, Max Helman, Raghav Mecheri
Self Supervised Learning , Spring 2022 (Prof. Richard Zemel)

Many methods seek to better understand the latent space of image synthesis models (eg. VAEs, GANs) in order to dissect the generative process and to enable easy latent space search: the task of navigating the latent space of the models such that the image produced by it follows some pre-conceived notion. It is not obvious how to carry search methods for continuous latent spaces over to autoencoders that have discrete latent spaces. In this paper, we provide empirical analysis of the latent space of Vector Quantized Variational Autoencoders (VQVAE), and propose a framework to extend continuous distribution search to the latent space of discrete models.

How much do Label Representations Matter for Image Classification?
Ishaan Chandratreya, Katon Luaces
Unsupervised Machine Learning , Fall 2021 (Prof. Nakul Verma)

We investigate how using a range of unsupervised methods for organizing the space in which the representations of the label reside affects the performance and training dynamics of a supervised image classification task. We present a range of possibilities to learn a "label" space prior to image classification, including imposing ordinal constraints and geometric inductive biases.

Learning to Cut with Reinforcement Learning
Ishaan Chandratreya
Reinforcement Learning: Theory and Applications , Spring 2021 (Prof. Shipra Agrawal)

The issue of selecting cuts as part of cutting-plane methods to solve integer programming (IP) problems can be framed as a Markov Decision Process, and hence the policy to select these cuts can be learnt using policy gradient methods.


At Columbia, I've helped assist with instructing classes covering various theoretical and applied matters in building intelligent systems. I am fortunate to have Prof. Nakul Verma as my guide here.

Head Instructional Assistant: COMS 4771 Machine Learning (Spring 2021, Fall 2021, Spring 2022)

Head Instructional Assistant: COMS 3251 Computational Linear Algebra (Summer 2021)

Teaching Assistant: COMS 4732 Computer Vision (Learning) (Summer 2021)

Teaching Assistant: COMS 3137 Honors Data Structures (Spring 2020)

Teaching Assistant: COMS 3134 Data Structure (Fall 2019)

Credits to Jon Barron for the website template.