Recent advances in 3D imaging and acquisition technology require tools for creating and editing 3D content. However, most of the previous work on 2D-to-3D shape manipulation tailored to a specific editing task and interactive format.
A recent article on arXiv.org proposes a 2D-to-3D framework that works on a single control method. It is capable of flexibly handling different types of 2D interactions without architecture changes or retraining.
The framework constructs a shared latent representation on the common models of each 2D and 3D method. Represents that an arbitrary implicit code corresponds to a consistent 3D model for all methods.
Evaluation on two representative 2D modes (grayscale line outline and rendered color image) is performed. It turns out that the proposed method is easy to implement and generalizable to new methods without special requirements on network architecture.
Creating and editing the shapes and colors of 3D objects requires human effort and expertise. Compared to direct manipulation in the 3D interface, 2D interactions such as sketches and doodles are often much more natural and intuitive for the user. In this paper, we propose a multimodal general model that combines 2D methods and implicit 3D representations through shared latent spaces. With the proposed model, flexible 3D creation and manipulation is enabled by simply passing editing from a specific 2D control method through implicit spaces. For example, edit 3D shapes by drawing a sketch, retouch 3D surface colors through color doodles on a 2D render, or create 3D shapes of a given category for one or more Some reference pictures. Unlike previous products, our model does not require retraining or fine-tuning for each editing task and is also conceptually simple, easy to implement, powerful for changing input domains and flexibility to reproduce diversity on partial 2D input. We evaluate our framework on two representative 2D methods of grayscale line sketches and rendered color images, and demonstrate that our method enables image manipulation and creation tasks. different forms with these 2D methods.
Research articles: Cheng, Z., “Creating and Manipulating Multimodal 3D Shapes”, 2022. Link: https://arxiv.org/abs/2207.11795
Project page: https://people.cs.umass.edu/~zezhoucheng/edit3d/
