Comment on Google is going ‘all in’ on AI. It’s part of a troubling trend in big tech
FreedomAdvocate@lemmy.net.au 6 days agoWhat are you talking about “temporal+quality” for DLSS? That’s not a thing.
DLSS I’m talking about. There are many comparisons out there showing how amazing it is, often resulting in better IQ than native.
LainTrain@lemmy.dbzer0.com 6 days ago
Sorry I was mistaken, it’s not “temporal”, I meant “transformer”, as in the “transformer model”, as here in CP2077.
Image
No, AI upscaling from a lower resolution will never be better than just running the game at the native resolution it’s being upscaled to. Most of the time it’s just blurry slop with sharpening applied.
Even if it was, any added detail is either literally just a sharpening post-processing filter akin to the added detail of a cheap Walmart “HD Ready” TV circa 2007 with sharpening cranked up, or outright fictional, and does not exist within the game files itself, and if by “better” we agree that it’s the most high fidelity representation of the game as it exists on disk, then AI cannot ever be better.
By it’s very nature, the ML model is just “guessing” what the frame might look like if it was rendered at native resolution. It’s not an accurate representation of the render output or artistic intent. Is it impressive? Yes of course, it’s a miracle of technology and a result of brilliant engineering and research in the ML field applied creatively and practically, but it does not result in a better image than native, nor does it aim to do so.
I mention FXAA because really the only reason we use “AI upscalers” is because anti-aliasing is really really computationally expensive.
The single most immediately evident and obvious consequence of a low render resolution is aliasing first and foremost. Almost all other aspects of a game’s graphics are usually completely detached from this like e.g. texture resolution.
The reason aliasing happens in the first place is because our ability to create, ship, process and render increasingly high polygon count games has massive surpassed our ability to push pixels on screen in real time.
Or course legibility suffers at lower resolution as well, but not nearly as much as smoothness of edges on high-polygon objects.
So for assets that would look really good at say, 4K, we run them at 720p instead, and this creates jagged edges because we literally cannot make the thing fit into the pixels we’re pushing.
The best and most direct solution will always be just to render the game at a much higher resolution. But that kills framerates.
We can’t do that, so we resort to Anti-Aliasing techniques instead. The most simple of which is MSAA which just multi-samples (renders at higher res) those edges and downscales them.
But it’s also very very expensive to do computationally. GPUs capable of doing it alongside other bells and whistles we have like Ray Tracing simply don’t exist, and if they did they’d cost too much, and even then, most games have to target consoles, which are solidly beat out by a flagship GPU even from several years ago.
One other solution is to blur these jagged edges out, sacrificing detail for a “smooth” look.
This is what FXAA does, but this creates a blurry image. This became very prevalent during the 7th Gen console era in particular because they simply couldn’t push more than 720p in most games, in an era where Full HD TVs had become fairly common towards the end and shiny, polished graphics in trailers became a major way to make sales, this was further worsened by the fact Motion Blur was often used to cover up low framerates and replicate the look of sleek modern (at the time) digital blockbusters.
SMAA fixed some of FXAA’s issues by being more selective about which pixels were blurred, and TAA eliminated the shimmering effect by also taking into account which pixels should be blurred across multiple frames.
Beyond this there are other tricks, like checkerboard rendering, where we render the frame in chunks at different resolutions based on what the player may or may not be looking at.
In VR we also use foveated rendering to render an FOV cone in front of the players immediate vision at a higher res than what would be in their periphery/outside the eye’s natural focus, with eye tracking tech, this actually works really well.
But none of these are very good solutions, so we resort to another ugly, but potentially less bad solution, solution, which is just rendering the game at a lower resolution and upscaling it, like a DVD played on an HDTV, but instead of a traditional upscaling algo like Lanczoz, we use DLSS, which reconstructs detail lost from a lower resolution render, based on context of the frame using machine learning, which is efficient because of tensor cores now included on every GPU making N-dimensional array multiplication and mixed precision FP math relatively computationally cheap.
DLSS often looks better compared to FXAA, SMAA and TAA because all of those just literally blur the image in different ways, without any detail reconstruction, but it is not comparable to any real anti-aliasing technique like MSAA.
But DLSS always renders at a lower res than native, so it will never be 1:1 a true native image, it’s just an upscale. That’s okay, because that’s not the point. The purpose of DLSS isn’t to boost quality, it’s to be a crutch for low performance, it’s why turning off even Quality presets for DLSS will often tank performance.
There is one situation where DLSS can look better than native, and it’s if you instead of typical applications of DLSS which downscales the image, then upscales it with ML guesswork, use it to upscale the image from native, to a higher target res instead and output that.
In Nvidia settings I believe this is called DL DSR factors.
FreedomAdvocate@lemmy.net.au 6 days ago
I don’t even know where to begin, so much wrong with this. I’ll have to come back when I’ve got more time.
LainTrain@lemmy.dbzer0.com 4 days ago
Okay, I’d be interested to hear what you think is wrong with this, because I’m pretty sure it’s more or less correct.
Some sources for you to help you understand these concepts a bit better:
What DLSS is and how it works as a starter: en.wikipedia.org/…/Deep_Learning_Super_Sampling
Issues with modern “optimization”, including DLSS: www.youtube.com/watch?v=lJu_DgCHfx4
TAA comparisons (yes, biased, but accurate): old.reddit.com/r/FuckTAA/…/rfucktaa_resource/