NVIDIA Research unveils GauGAN2, a new AI art demo that generates terrifyingly accurate images from text: Digital Photography Review

Main Image: Image created from the following text, ‘A peaceful lake surrounded by tall trees on a foggy day.’

NVIDIA has announced latest version of NVIDIA Research AI painting demo, GauGAN2. NS model powered by deep learning and now features text-to-image. While the original version could only turn rough sketches into detailed images, GauGAN 2 can generate images from phrases like ‘sunset at the beach’, which can then be further modified using features like ‘sunset at the beach’. words like ‘rocky beach’ or by changing ‘sunset’ to a different time of day or even changing weather conditions. GauGAN is powered by general adversarial networks (GANs), you can learn more about this network NVIDIA article.

Back at GauGAN2, NVIDIA writes, ‘With the push of a button, the user can create a segment map, a high-level contour showing the position of objects in the scene. From there, they can move on to painting, retouching the scene with rough sketches using labels like sky, trees, rocks, and rivers, allowing the brush to intelligently incorporate scribbles these into beautiful images. ‘

You can try GauGAN2 for yourself on NVIDIA AI Demo. You can also see it in action in the video below.

By adding text-to-image capabilities, the new version of GauGAN is more customizable and can be adjusted much faster. Even a quick sketch is not as quick and simple as typing a phrase. The latest version is also one of the first AI models to combine multiple methods, text, semantic segmentation, outline, and style, in a single GAN.

Your text-based starting point, such as ‘snow capped mountains’, can be further customized by sketching. You can add trees, change the height and size of objects, add clouds to the sky, and more. And then GauGAN2 creates a new, modified image.

‘Endless high mountains on a sunny day’

You also don’t have to keep your ideas based on facts. GauGAN2 can be useful for concept artists, as you can create worlds with two suns, like Tatooine in Star Wars. NVIDIA writes, ‘It’s an iterative process where each word the user enters into the text box adds more to the AI-generated image.’

Click to see animated GIF

GauGAN continues to improve its results. When are we? reviewed it in early 2019, the results are impressive, but there are obvious limitations. NVIDIA released a tool earlier this year built on top of GauGAN, NVIDIA Canvas, which can be used on any NVIDIA RTX GPU. At this point, GauGAN2 has been trained on 10 million landscape images using NVIDIA Selene Supercomputer, among the top 10 most powerful supercomputers in the world.

To learn more about NVIDIA Research and its projects, click here. There are a lot of exciting AI-powered projects underway and it’s impressive to see how far GauGAN has come in just a few years.