Abstract
Due to the inherent robustness of segmentation, traditional methods fail to attack it in a few steps. We demonstrate a simple and effective method that changes the nuisance factors to fool semantic segmentation models in a single step, even if the models have been adversarially trained for defense. By training conditional generative models with a designed adversarial loss, we can generate realistic adversarial attack by changing nuisance factors. Our method does not require neural network back-propagation, which is even faster than fast gradient sign method (FGSM). We validate our approach on the popular Cityscapes and ADE20K datasets, and demonstrate better attack success rate compared to the existed adversarial attacks for semantic segmentation including PGD, Houdini and DAG with over 100x speed-up, which is the first method that can attack semantic segmentation models online.