Stepping Forward: MS student Develops VR Full-Body Tracking Shoes

When Columbia engineer Axl Chen, MS '23, enrolled in "Practical Deep Learning System Performance," he couldn’t have imagined he would soon be leading a start-up that develops full-body VR tracking shoes. But that’s exactly what happened. The course—taught by Professor Parijat Dube—introduced Chen to algorithms that can boost the accuracy of human activity recognition. A year later, he’s the proud founder and CEO of Surplex, a company developing a full-body VR tracker, hidden inside a pair of shoes.

"I [was] really interested in AI applications, but I [hadn’t] found a lot of applications where it is a novel kind of research that can be applied into the real world," says Chen. Now, Surplex is working on one such application: capturing full-body human motion for practical virtual reality uses like gaming and professional sports training.

Bridging the Gap Between Technology and Imagination

Combining dense sensors, IMU, and deep learning, Chen and a team of engineers and developers from Columbia, MIT, UPenn, Rutgers, and China's Tsinghua University have created a pair of shoes that have the potential to revolutionize today's VR experience. 

"We are combining textile sensing and AI wisdom for human pose estimation," Chen says.

The Surplex shoes use 480 pressure sensors that track body movements with "great accuracy, even when making tiny motions," according to the project's Kickstarter campaign. "Better still, you don't need to be confined to limited space, or tangled up with cords and straps as with other VR trackers.”

Chen, who has an undergraduate degree from Parsons School of Design, took his first steps at Columbia Engineering in a summer program. 

"I actually came to Columbia for an exchange [program], basically a summer semester. Here, I took a class of Machine Learning and also deep learning,” Chen says, “and I was actually quite fascinated by AI technology. Then, I applied for a Masters at Columbia in computer science."

Chen sees the potential to change the way professional sports approaches motion tracking. Currently approaches are either majorly expensive or demand an awkward base station. 

"Before, you would need to have a huge amount of professional equipment to capture your full body motion, several cameras looking at you to get a good capture,” Chen says. “Some use carpets [that have pressure sensors] so let's say, for example if you are lifting your leg, kicking around, dancing around, in that kind of scenario, it is hard to capture your full body motion by just using pressure sensing. So, I was thinking, how can we bridge this gap between textile sensing and full body capture with good quality?”

Start-Up State of Mind

The project has already garnered public interest, raising over $60,000 dollars through a Kickstarter campaign. Stories about Surplex in INPUT, VRScout,TrendHunter and Yahoo News praise Chen's new developments and predict his project could open up new possibilities for gaming and entertainment.

When asked about what kind of advice he would offer to young engineers looking to launch their careers, Chen says that students and young entrepreneurs have a unique vantage point that allows them to think big and see opportunities and ideas that more seasoned professionals may be missing. 

"When you try to start a startup when you're an undergrad, you can think about fields that cannot be seen by others,” says Chen, “you have a much broader view of what is happening right now in the world and what's possible. When you are young you have a chance to try and test and see if that works out.”