AI Accelerated Rendering using DLSS

AI Accelerated Rendering

DLSS or Deep Learning Super Sampling is an AI Rendering Technology that increases the graphical performance using AI. It is also arguably one of the most innovative technologies that have emerged in the real-time 3D rendering and visual graphics industry. It has made possible the rendering of high-resolution images with far less the computational power that would otherwise be required. Here we will be going into AI Accelerated Rendering using DLSS as an example.

Before going into AI Accelerated Rendering we must first understand why 3D Rendering at a high resolution is difficult. A 3D render would typically consist of several frames of images each having millions of individual pixels giving the image its features, color, and sharpness. In higher resolution images the pixel count will be much higher compared to normal images to account for the increased clarity or sharpness of the image. This is where the major issue becomes evident. Since for rendering at high resolution, we need to render out several frames of images in real-time, each of which consists of a large number of pixels, several times in any given second to get a proper and smooth render. So as the resolution increases the number of pixels we need to render per frame increases drastically thus increasing the load on the computer to a large extent.

What is a high-Resolution Image?

The definition of a high resolution is very ambiguous. It changes with the advancements in technology and hence does have a fixed meaning. By current standards, we may call resolutions above 4K (3840 x 2160 pixels) as high resolution. Resolution is also a measure of pixels per inch of the display which accounts for how many pixels are there in any given section of the image. For comparison, a single frame of an 8K image would have 16 times the pixels in the same 1080p image and that is a massive increase. Because we have to render several frames in any given second this easily becomes a task that even the best hardware struggles to perform.

Comparison of a 4k image and a 1080p image (The sharpness of the 4k image can be observed to be much higher than the 1080p image)

Upscaling Images

Upscaling of images is a process where we take an image in a lower resolution and then convert it to a higher resolution. For example, instead of rendering a native 8K image, we could render an image in 1080p and then scale its resolution to 8K. What this means is that we take each pixel in the 1080p image and create 16 individual pixels out of it (Since 8K has 16 times the pixels of 1080p). Here we reduce the load on the computer as we are just rendering the image in 1080p which takes only a fraction of the computing power required for a native 8K image. But naturally, this has its issues. Since with upscaling we are converting a single pixel to several pixels there will definitely be a loss in the quality of the image. This has been the main issue with upscaling of images since it usually results in blurry or less sharp images than a non-upscaled native image.

AI Accelerated Rendering: DLSS

With the advancements in AI technology DLSS (Deep Learning Super Sampling) a technology developed by Nvidia Corporation has been able to address the main issue with Upscaling of images. DLSS makes use of AI that can reconstruct pixels for upscaled images to restore the sharpness of the image. Here similar to upscaling of images the image is rendered in a lower resolution and then with the help of AI, it is upscaled to higher resolutions but retaining the quality of the image. This is the main reason why DLSS is so powerful as it reduces the load on the computer and provides enhanced performance without much loss in quality

A Video game rendered in native 4k in comparison to 4k with DLSS

How it Works

DLSS makes use of a neural network that has been trained using “ideal” images of video games of ultra-high resolution on supercomputers and low-resolution images of the same games. The neural network compares the input low-resolution image with the reference and produces a full high-resolution result. The training data is used by AI accelerators in the Graphics Processing Units of the computer to perform the processing of the image.

Impacts

What DLSS has achieved is to give the average consumer improved performance without much sacrifice in quality even in very high resolutions. What this means is that we get more life out of our hardware and great performance that improves year by year with improvements to the AI that is used. It has also made possible the availability of Real-Time Ray Traced 3D Graphics for video games and other purposes which provide hyper-realistic renders without having to require the best hardware.

Have a project in mind that includes Deep Learning Super Sampling? Contact us here.