FurryGAN: High-quality foreground-aware image synthesis

Like common rival network (GAN) improvements, new methods for image synthesis are being developed. One of the tasks is to synthesize foreground-aware images by modeling the image as a component of the foreground and background images. under a mask.

Example image and corresponding foreground mask. Both
generated concurrently by the proposed new model. Image credit: arXiv: 2208.10422 [cs.CV]

A recent article on arXiv.org recommends FurryGAN, which learns to synthesize images with a clear understanding of the foreground giving only a collection of images.

Firstly, foreground and composite images are encouraged to resemble the training distribution. In addition, coarse and fine masks are also introduced. Finally, the researchers propose an auxiliary task for the discriminator to predict the mask from the generated image so that the generator generates a foreground image aligned with the mask.

The tests demonstrate the superiority of the proposed framework over previous methods.

Foreground-aware image synthesis aims to create foreground images as well as their foreground mask. A common approach is to create the image as a masked blend of the foreground and background images. This is a challenging problem because it tends to reach a trivial solution where one of the two images overwhelms the other, i.e., the masks become completely full or empty, and the foreground and the background are not meaningfully separated. We present FurryGAN with three main components: 1) imposing both foreground and composite images to be realistic, 2) mask design as a combination of coarse and fine masks, and 3) instruct the generator with a sub-mask predictor in the discriminator. Our method creates realistic images with extremely detailed alpha masks that cover hair, fur and beard in a completely unattended manner.

Research articles: Bae, J., Kwon, M. and Uh, Y., “FurryGAN: High Quality Foreground-Aware Image Synthesis”, 2022. Link: https://arxiv.org/abs/2208.10422

You can provide your link to a page relevant to the topic of this post.