Several deep learning methods have been proposed for completing partial data from shape acquisition setups, i.e., filling the regions that were missing in the shape. These methods, however, only complete the partial shape with a single output, ignoring the ambiguity when reasoning the missing geometry. Hence, we pose a multi-modal shape completion problem, in which we seek to complete the partial shape with multiple outputs by learning a one-to-many mapping. We develop the first multimodal shape completion method that completes the partial shape via conditional generative modeling, without requiring paired training data.
We design a conditional generative adversarial network (cGAN) wherein a generator learns to map incomplete training data, combined with a latent vector sampled from a learned multimodal shape distribution, to a suitable latent representation such that a discriminator cannot differentiate between the mapped latent variables and the latent variables obtained from complete training data (i.e., complete shape models).
Our method can complete the partial shape with multiple plausible outputs. We show result examples, where the input partial shape is colored in grey and is followed by five different completions in yellow.
To allow more explicit control over the modes in completion results, the mode condition z can also be encoded from a user-specified shape. This enables us to complete the partial shape under the guidance of a given reference shape.
We present both qualitative and quantitative comparisons against baseline methods and variants of our method on three datasets, which have different forms of shape incompleteness. Our method can produce results that are both diverse and plausible.
@InProceedings{wu_2020_ECCV,
author = {Wu, Rundi and Chen, Xuelin and Zhuang, Yixin and Chen, Baoquan},
title = {Multimodal Shape Completion via Conditional Generative Adversarial Networks},
booktitle = {The European Conference on Computer Vision (ECCV)},
month = {August},
year = {2020}
}