Common rival model can create realistic images. They use the GAN to take a latent code as input and produce an image as output. In contrast, GAN reverse engineering allows a real image to be projected onto hidden space of a pre-trained GAN and retrieve its corresponding latent code.
Implicit code modification enables high-level editing tasks. The latent code modification with semantic change in the image is called semantic editing.
A recent paper on arXiv.org presents a method for time-consistent video semantic editing. A simple and efficient stream-based approach is proposed to minimize temporal inconsistencies of live (frame-by-frame) edited video. The two-stage optimization method allows both the underlying code and the generator to be updated to preserve video details. This method is model agnostic and can be applied to various GAN manipulation and inversion techniques.
General adversarial networks (GANs) have demonstrated the impressive image creation quality and semantic editing capabilities of real images, e.g. changing feature classes, modifying properties, or switching styles. However, applying these GAN-based edits to the video independently for each frame will inevitably result in temporary flickering. We present a simple yet effective method to facilitate transient coherent video editing. Our core idea is to minimize photometric inconsistencies over time by optimizing both the latent code and the pre-trained generator. We evaluate the quality of our editing across different domains and GAN reverse engineering and show favorable results relative to baseline.
Research articles: Xu, Y., AlBahar, B., and Huang, J.-B., “Temporary Consistent Semantic Video Editing”, 2022. Link: https://arxiv.org/abs/2206.10590
