Please scan to add as friend.


PH. D.




Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers
NeurIPS 2020
Kiwon Um, Robert Brand, Yun (Raymond) Fei, Philipp Holl, Nils Thuerey
Abstract | Project Page | arXiv | GitHub | YouTube
simulation machine learning

Finding accurate solutions to partial differential equations (PDEs) is a crucial task in all scientific and engineering disciplines. It has recently been shown that machine learning methods can improve the solution accuracy by correcting for effects not captured by the discretized PDE. We target the problem of reducing numerical errors of iterative PDE solvers and compare different learning approaches for finding complex correction functions. We find that previously used learning approaches are significantly outperformed by methods that integrate the solver into the training loop and thereby allow the model to interact with the PDE during training. This provides the model with realistic input distributions that take previous corrections into account, yielding improvements in accuracy with stable rollouts of several hundred recurrent evaluation steps and surpassing even tailored supervised variants. We highlight the performance of the differentiable physics networks for a wide variety of PDEs, from non-linear advection-diffusion systems to three-dimensional Navier-Stokes flows.


Multi-Scale Models to Simulate Interactions between Liquid and Thin Structures
Dissertation at Columbia University
Advisors: Eitan Grinspun, Changxi Zheng
Abstract | Preprint | Columbia Academic Commons

In this dissertation, we introduce a framework for simulating the dynamics between liquid and thin structures, including the effects of buoyancy, drag, capillary cohesion, dripping, and diffusion. After introducing related works, Part I begins with a discussion on the interactions between Newtonian fluid and fabrics. In this discussion, we treat both the fluid and the fabrics as continuum media; thus, the physical model is built from mixture theory. In Part II, we discuss the interactions between Newtonian fluid and hairs. To have more detailed dynamics, we no longer treat the hairs as continuum media. Instead, we treat them as discrete Kirchhoff rods. To deal with the thin layer of liquid that clings to the hairs, we augment each hair strand with a height field representation, through which we introduce a new reduced-dimensional flow model to solve the motion of liquid along the longitudinal direction of each hair. In addition, we develop a faithful model for the hairs' cohesion induced by surface tension, where a penalty force is applied to simulate the collision and cohesion between hairs. To enable the discrete strands interact with continuum-based, shear-dependent liquid, in Part III, we develop models that account for the volume change of the liquid as it passes through strands and the momentum exchange between the strands and the liquid. Accordingly, we extend the reduced-dimensional flow model to simulate liquid with elastoviscoplastic behavior. Furthermore, we use a constraint-based model to replace the penalty-force model to handle contact, which enables an accurate simulation of the frictional and adhesive effects between wet strands. We also present a principled method to preserve the total momentum of a strand and its surface flow, as well as an analytic plastic flow approach for Herschel-Bulkley fluid that enables stable semi-implicit integration at larger time steps.
We demonstrate a wide range of effects, including the challenging animation scenarios involving splashing, wringing, and colliding of wet clothes, as well as flipping of hair, animals shaking, spinning roller brushes from car washes being dunked in water, and intricate hair coalescence effects. For complex liquids, we explore a series of challenging scenarios, including strands interacting with oil paint, mud, cream, melted chocolate, and pasta sauce.


A Multi-Scale Model for Coupling Strands with Shear-Dependent Liquid
ACM Transactions on Graphics (SIGGRAPH ASIA 2019)
Yun (Raymond) Fei, Christopher Batty, Eitan Grinspun, Changxi Zheng
Abstract | Project Page | In Press: TechXplore, AAAS EurekAlert!, Two Minute Papers

We propose a framework for simulating the complex dynamics of strands interacting with compressible, shear-dependent liquids, such as oil paint, mud, cream, melted chocolate, and pasta sauce. Our framework contains three main components: the strands modeled as discrete rods, the bulk liquid represented as a continuum (material point method), and a reduced-dimensional flow of liquid on the surface of the strands with detailed elastoviscoplastic behavior. These three components are tightly coupled together. To enable discrete strands interacting with continuum-based liquid, we develop models that account for the volume change of the liquid as it passes through strands and the momentum exchange between the strands and the liquid. We also develop an extended constraint-based collision handling method that supports cohesion between strands. Furthermore, we present a principled method to preserve the total momentum of a strand and its surface flow, as well as an analytic plastic flow approach for Herschel-Bulkley fluid that enables stable semi-implicit integration at larger time steps. We explore a series of challenging scenarios, involving splashing, shaking, and agitating the liquid which causes the strands to stick together and become entangled.


Addressing Troubles with Double Bubbles: Convergence and Stability at Multi-Bubble Junctions
Arxiv:1910.06402, 2019
Yun (Raymond)Fei, Christopher Batty, Eitan Grinspun
Abstract | Project Page

In this report we discuss and propose a correction to a convergence and stability issue occurring in the work of Da et al.[2015], in which they proposed a numerical model to simulate soap bubbles.


Mechanics-Aware Modeling of Cloth Appearance
IEEE Transactions on Visualization and Computer Graphics, 2019
Zahra Montazeri, Chang Xiao, Yun (Raymond) Fei, Changxi Zheng, Shuang Zhao,
Abstract | Project Page | IEEE Digital Library | Video: YouTube, bilibili | Talk at Pacific Graphics 2019: YouTube
simulation rendering machine learning

Micro-appearance models have brought unprecedented fidelity and details to cloth rendering. Yet, these models neglect fabric mechanics: when a piece of cloth interacts with the environment, its yarn and fiber arrangement usually changes in response to external contact and tension forces. Since subtle changes of a fabric's microstructures can greatly affect its macroscopic appearance, mechanics-driven appearance variation of fabrics has been a phenomenon that remains to be captured. We introduce a mechanics-aware model that adapts the microstructures of cloth yarns in a physics-based manner. Our technique works on two distinct physical scales: using physics-based simulations of individual yarns, we capture the rearrangement of yarn-level structures in response to external forces. These yarn structures are further enriched to obtain appearance-driving fiber-level details. The cross-scale enrichment is made practical through a new parameter fitting algorithm for simulation, an augmented procedural yarn model coupled with a custom-design regression neural network. We train the network using a dataset generated by joint simulations at both the yarn and the fiber levels. Through several examples, we demonstrate that our model is capable of synthesizing photorealistic cloth appearance in a %dynamic and mechanically plausible way.


A Multi-Scale Model for Simulating Liquid-Fabric Interactions
ACM Transactions on Graphics (SIGGRAPH 2018)
Yun (Raymond) Fei, Christopher Batty, Eitan Grinspun, Changxi Zheng
Abstract | Project Page | In Press: Deutschlandfunk (in German), Two Minute Papers, 3DVF (in French), Columbia Engineering, CGWorld (in Chinese)

We propose a method for simulating the complex dynamics of partially and fully saturated woven and knit fabrics interacting with liquid, including the effects of buoyancy, nonlinear drag, pore (capillary) pressure, dripping, and convection-diffusion. Our model evolves the velocity fields of both the liquid and solid relying on mixture theory, as well as tracking a scalar saturation variable that affects the pore pressure forces in the fluid. We consider the porous microstructure implied by the fibers composing individual threads, and use it to derive homogenized drag and pore pressure models that faithfully reflect the anisotropy of fabrics. In addition to the bulk liquid and fabric motion, we derive a quasi-static flow model that accounts for liquid spreading within the fabric itself. Our implementation significantly extends standard numerical cloth and fluid models to support the diverse behaviors of wet fabric, and includes a numerical method tailored to cope with the challenging nonlinearities of the problem. We explore a range of fabric-water interactions to validate our model, including challenging animation scenarios involving splashing, wringing, and collisions with obstacles, along with qualitative comparisons against simple physical experiments.


A Multi-Scale Model for Simulating Liquid-Hair Interactions
ACM Transactions on Graphics (SIGGRAPH 2017)
Yun (Raymond) Fei, Henrique Teles Maia, Christopher Batty, Changxi Zheng, Eitan Grinspun
Abstract | Project Page | In Press:, Two Minute Papers, CG Channel, PC Gamer, CGWorld (in Chinese), 3Dnchu (in Japanese), GeekiMovie (in Chinese)

The diverse interactions between hair and liquid are complex and span multiple length scales, yet are central to the appearance of humans and animals in many situations. We therefore propose a novel multi-component simulation framework that treats many of the key physical mechanisms governing the dynamics of wet hair. The foundations of our approach are a discrete rod model for hair and a particle-in-cell model for fluids. To treat the thin layer of liquid that clings to the hair, we augment each hair strand with a height field representation. Our contribution is to develop the necessary physical and numerical models to evolve this new system and the interactions among its components. We develop a new reduced-dimensional liquid model to solve the motion of the liquid along the length of each hair, while accounting for its moving reference frame and influence on the hair dynamics. We derive a faithful model for surface tension-induced cohesion effects between adjacent hairs, based on the geometry of the liquid bridges that connect them. We adopt an empirically-validated drag model to treat the effects of coarse-scale interactions between hair and surrounding fluid, and propose new volume-conserving dripping and absorption strategies to transfer liquid between the reduced and particle-in-cell liquid representations. The synthesis of these techniques yields an effective wet hair simulator, which we use to animate hair flipping, an animal shaking itself dry, a spinning car wash roller brush dunked in liquid, and intricate hair coalescence effects, among several additional scenarios.


Interactive Acoustic Transfer Approximation for Modal Sound
ACM Transactions on Graphics (SIGGRAPH 2016)
Dingzeyu Li, Yun Fei, Changxi Zheng
Abstract | Preprint: Compressed, Regular | ACM Digital Library | Video: YouTube, bilibili | Fast Forward | Project Page

Current linear modal sound models are tightly coupled with their frequency content. Both the modal vibration of object surfaces and the resulting sound radiation depend on the vibration frequency. Whenever the user tweaks modal parameters to adjust frequencies the modal sound model changes completely, necessitating expensive recomputation of modal vibration and sound radiation.

We propose a new method for interactive and continuous editing as well as exploration of modal sound parameters. We start by sampling a number of key points around a vibrating object, and then devise a compact, low-memory representation of frequency-varying acoustic transfer values at each key point using Prony series. We efficiently precompute these series using an adaptive frequency sweeping algorithm and volume-velocity-preserving mesh simplification. At runtime, we approximate acoustic transfer values using standard multipole expansions. Given user-specified modal frequencies, we solve a small least-squares system to estimate the expansion coefficients, and thereby quickly compute the resulting sound pressure value at arbitrary listening locations. We demonstrate the numerical accuracy, the runtime performance of our method on a set of comparisons and examples, and evaluate sound quality with user perception studies.



Computational Design of Metallophone Contact Sounds
ACM Transactions on Graphics (SIGGRAPH Asia 2015)
Gaurav Bharaj, David Levin, James Tompkin, Yun Fei, Hanspeter Pfister, Wojciech Matusik, Changxi Zheng
Abstract | Preprint | ACM Digital Library | Supplementary | Video: YouTube, bilibili | Project page | Industrial Reproduction | In Press: Science 360, Wired, Popular Science, Gizmodo, Inverse, Columbia Engineering, Science Daily, Gizmag, TechRadar, 3DFab+Print
simulation fabrication

Metallophones such as glockenspiels produce sounds in response to contact. Building these instruments is a complicated process, limiting their shapes to well-understood designs such as bars. We automatically optimize the shape of arbitrary 2D and 3D objects through deformation and perforation to produce sounds when struck which match user-supplied frequency and amplitude spectra. This optimization requires navigating a complex energy landscape, for which we develop Latin Complement Sampling to both speed up finding minima and provide probabilistic bounds on landscape exploration. Our method produces instruments which perform similarly to those that have been professionally-manufactured, while also expanding the scope of shape and sound that can be realized, e.g., single object chords. Furthermore, we can optimize sound spectra to create overtones and to dampen specific frequencies. Thus our technique allows even novices to design metallophones with unique sound and appearance.


On the Hessian of Shape Matching Energy
Arxiv:1604.02483, 2016
Yun Fei
Preprint | Project Page

SmartGuide: Towards Single-image Building Localization with Smartphone
Proceedings of MobiHoc 2015
Xi Xiong, Yang Zheng, Longfei Shangguan, Yun Fei, Miloš Stojmenović, Yunhao Liu
ACM Digital Library
mobile computing

Parallel L-BFGS-B algorithm on GPU
Computers and Graphics, 2014
Yun Fei, Guodong Rong, Bin Wang, Wenping Wang
Abstract | ScienceDirect | Preprint | Github
GPU optimization

Due to the rapid advance of general-purpose graphics processing unit (GPU), it is an active research topic to study performance improvement of non-linear optimization with parallel implementation on GPU, as attested by the much research on parallel implementation of relatively simple optimization methods, such as the conjugate gradient method. We study in this context the L-BFGS-B method, or the limited memory Broyden–Fletcher–Goldfarb–Shanno with boundaries, which is a sophisticated yet efficient optimization method widely used in computer graphics as well as general scientific computation. By analyzing and resolving the inherent dependencies of some of its search steps, we propose an efficient GPU-based parallel implementation of L-BFGS-B on the GPU. We justify our design decisions and demonstrate significant speed-up by our parallel implementation in solving the centroidal Voronoi tessellation (CVT) problem as well as some typical computing problems.

由于通用图形处理单元(GPU)的飞速发展,通过使用GPU对非线性优化进行性能改进已经成为一项活跃的研究课题,这一点已得到大量对相对简单的优化方法的并行执行的研究的证明(例如共轭梯度法)。 在这样的情况下,我们研究了L-BFGS-B方法(即带边界的有限内存Broyden-Fletcher-Goldfarb-Shanno方法)的并行实现。L-BFGS-B是一种复杂而高效的非线性优化方法,广泛用于计算机图形学以及一般科学计算中。 通过分析和解决其某些搜索步骤的内在依赖性,我们提出了一种L-BFGS-B在GPU上的高效并行实现。 我们通过合理的并行解决方案解决重心Voronoi细分(CVT)问题以及一些典型的计算问题,证明了我们的设计决策的合理性,并证明了我们的方法可以有显着的加速效果。

Towards Photo Watercolorization with Artistic Verisimilitude
IEEE Transactions on Visualization and Computer Graphics, 2014
Miaoyi Wang, Bin Wang, Yun Fei, Kang-Lai Qian, Wenping Wang, Jiating Chen, Jun-Hai Yong
Abstract | Preprint | IEEE Xplore | Gallery
image rendering

We present a novel artistic-verisimilitude driven system for watercolor rendering of images and photos. Our system achieves realistic simulation of a set of important characteristics of watercolor paintings that have not been well implemented before. Specifically, we designed several image filters to achieve: 1) watercolor-specified color transferring; 2) saliency-based level-of-detail drawing; 3) hand tremor effect due to human neural noise; and 4) an artistically controlled wet-in-wet effect in the border regions of different wet pigments. A user study indicates that our method can produce watercolor results of artistic verisimilitude better than previous filter-based or physical-based methods. Furthermore, our algorithm is efficient and can easily be parallelized, making it suitable for interactive image watercolorization.

我们提出了一种新颖的以美学标准驱动的系统,用于图像和照片的水彩渲染。 我们的系统对水彩画的一组重要特征进行了逼真的仿真,这些水彩画特征以前没有很好地实现。 具体来说,我们设计了几种图像滤镜来实现:1)水彩特有的颜色转移效果; 2)基于显著性对不同的部分使用不同的详细程度来绘制; 3)模拟由于人的神经噪声造成的手部震颤效应; 4)在不同的湿颜料的边界区域中通过艺术控制的湿中湿效果。 一项用户研究表明,与以前的基于过滤器或基于物理的方法相比,我们的方法可以产生具有艺术真实性的水彩效果。 此外,我们的算法高效且易于并行化,使其适用于交互式图像水彩化。

Bilateral Blue Noise Sampling
ACM Transactions on Graphics (SIGGRAPH Asia 2013)
Jiating Chen*, Xiaoyin Ge*, Li-Yi Wei, Bin Wang, Yusu Wang, Huamin Wang, Yun Fei, Kang-Lai Qian, Jun-Hai Yong, Wenping Wang (*Joint 1st authors)
Abstract | Preprint, Compressed (1.9MB) , Regular (23.0MB) | Video: YouTube, bilibili | Image Stippling | GitHub | Project Page
geometry rendering

Blue noise sampling is an important component in many graphics applications, but existing techniques consider mainly the spatial positions of samples, making them less effective when handling problems with non-spatial features. Examples include biological distribution in which plant spacing is influenced by non-positional factors such as tree type and size, photon mapping in which photon flux and direction are not a direct function of the attached surface, and point cloud sampling in which the underlying surface is unknown a priori. These scenarios can benefit from blue noise sample distributions, but cannot be adequately handled by prior art.

Inspired by bilateral filtering, we propose a bilateral blue noise sampling strategy. Our key idea is a general formulation to modulate the traditional sample distance measures, which are determined by sample position in spatial domain, with a similarity measure that considers arbitrary per sample attributes. This modulation leads to the notion of bilateral blue noise whose properties are influenced by not only the uniformity of the sample positions but also the similarity of the sample attributes. We describe how to incorporate our modulation into various sample analysis and synthesis methods, and demonstrate applications in object distribution, photon density estimation, and point cloud sub-sampling.



Research on GPU Acceleration of Incompressible Smoothed Particle Hydrodynamics and Applications
Bachelor Thesis at Tsinghua University (in Chinese), 2013
Advisor: Bin Wang
Abstract | Preprint | Tsinghua University Library | Source (Direct3D) | Video: Youku, YouTube
simulation GPU

Recently, the power of graphics processing unit (GPU) has been incredibly improved due to the development of hardware technology, in programming shading and in general purpose computing, which is much higher than traditional central processing units (CPU). Hence, in this thesis, we apply it to fluid simulation and rendering, which includes two main contributions: 1. a parallelized implementation of predictive-corrective incompressible smoothed particle hydrodynamics, which is a recently popular method for the simulation of incompressible fluid, producing real-time frame rate. 2. a parallelized implementation of smoothed surface reconstruction from 3D point cloud, for real-time rendering of the fluids motion. Furthermore, the implementation accompanied with this thesis is the first time that the simulation of incompressible fluids, surface reconstruction and rendering are integrated into an open source package, which can be an effective start for further research and applications. To be concrete, we demonstrate two application of our fluid solver: 1. by introducing geometry-based caustics and shadows, we plausibly simulate and render the fluid in real-time. 2. by introducing temperate-related viscosity, and fluid-solid coupling, we simulate the painting process of Chinese sugar-coating in an interactive way.


Point-Tessellated Voxelization
Proceedings of Graphics Interface 2012
Yun Fei, Bin Wang, and Jiating Chen
Abstract | ACM digital library | Preprint | Video: YouTube, bilibili | Source (Direct3D)
GPU geometry rendering

Applications such as shape matching, visibility processing, rapid manufacturing, and 360 degree display usually require the generation of a voxel representation from a triangle mesh interactively or in real-time. In this paper, we describe a novel framework that uses the hardware tessellation support on the graphics processing unit (GPU) for surface voxelization. To generate gap-free voxelization results with superior performance, our framework uses three stages: triangle subdivision, point generation, and point injection. For even higher temporal efficiency we introduce PN-triangles and displacement mapping to voxelize meshes with rugged surfaces in high resolution.

Our framework can be implemented with simple shader programming, making it readily applicable to a number of real-time applications where both development and runtime efficiencies are of concern.

诸如形状匹配,可见性处理,快速制造和360度显示之类的应用程序通常需要交互式或实时地从三角形网格生成体素表示。 在本文中,我们描述了一种新颖的框架,该框架使用图形处理单元(GPU)上的硬件细分支持来进行表面体素化。 为了生成具有出色性能的无间隙体素化结果,我们的框架使用三个阶段:三角形细分,点生成和点注入。 为了获得更高的时间效率,我们引入了PN三角形和位移映射,以高分辨率对具有粗糙表面的网格进行体素化。


Fast multi-image-based photon tracing with grid-based gathering
SIGGRAPH 2012 Poster
Yun Fei, Bin Wang
Brief Report | Poster | Technical Report | YouTube
GPU rendering


Alita: Battle Angel
Research & Development. Directed by Robert Rodriguez. 2019; 20th Century Fox, Lightstorm Entertainment and Troublemaker Studios.


Mesh2Centroids: SPH-based Lloyd's Relaxation on GPU inside 3D Meshes
Technical Demo

Affine Particle-in-Cell in 2D
Educational Demo

A Single C++ Header Generating 3D Triangulated Rounded Corner Box

4 Minutes for 40 Years: A Brief Review of Modern Computer Graphics