The ability to seamlessly replace the head in the target image with the watershed is important in a variety of situations, such as film composition or deepfake video detection.
A recent paper published on arXiv.org proposes a framework that generates high fidelity head-swapping results in nature based on a few frames.
The researchers propose to align the position and emotion of the watershed with the target in a unified blender. First, the watershed is aligned to have the same pose and expression as the target image overall. To that end, multi-scale global and local information from both images is encoded in a style-based generator. Then the mismatch in skin color and edge background image is handled.
Extensive experiments demonstrate that this method achieves outstanding head swapping results in a variety of scenes.
The head swap mission aims to place the water head in the target body perfectly, which is very important for different entertainment scenarios. While face-swapping gets a lot of attention, head-swapping quests are rarely explored, especially in low-scenes settings. It is inherently challenging due to its unique needs in head prototyping and background blending. In this paper, we present Head Swapper (HeSer), which achieves head swapping in the wild through two sophisticatedly designed modules. First, Head2Head Aligner is designed to comprehensively move postural and expression information from target to watershed by examining multi-scale information. Second, to address the challenges of skin discolouration and head background mismatch in the swap process, the Head2Scene Blender is introduced to simultaneously modify facial skin color and fill gaps. blanks do not match on the background around the head. In particular, seamless blending is achieved with the help of Semantic Guided Color Reference Generation and a Blender. Extensive experiments demonstrate that the proposed method produces superior head-swapping results in a variety of scenes.
Research articles: Shu, Changyong et al. “Head swaps are less common in the wild.” (In 2022). Link: https://arxiv.org/abs/2204.13100
Project page: https://jmliu88.github.io/HeSer/
