Virtual Method

@Big__TeeJ @nimlot26 If there’s a camera-mounted light, its specular contribution would appear in the original image as well. And it’s not rim lighting either since it lits the character from behind/laterally.

@virtual_method @nimlot26 Would it not have something to do with this? The engine essentially lights the characters at all times to keep them viewable. I'm no expert but my first thought on something like this is it has to be based on that.

After this whole debate about DLSS 5 I came to the conclusion that most of the people talking about it are completely unaware of what they don't know...they're on the peak of ignorance and don't even grasp how little they understand. They just heard generative AI and like Pavlov's dog they just start drooling thinking it's the same shit as unethical slop image generators...for the love of Christ...go and educate yourself before raging on the internet for no reason. DLLS 5 is not a prompt based generator...it's not creating stuff based on someone else's images and hallucinates results. It uses the information from the raster to build up a final render frame with the same information but with better lighting and shading... I'll even give you an example on how much of an impact better shading and lighting has. This is a character I've worked on not long ago. On the left you have a raster render, with some bad shaders. On the right you have a render with raytrace on, a much better shader for both hair and skin. They don't even look like the same person...do they? This is what DLSS5 is doing....getting a result like the one on the right(tbh a lot better) at a smaller cost than actually rendering it. Still the same geo, same textures, same light sources. Some of you will go and say the one on the left is better and it's the artist's vision. It's not...it's just the artist's limitation due to shading and lighting constrains. Every single artist out there would love to get the right result in real time.

@CryZe107 @IanCutress That's barely any control. What I mean is things like smoothness, albedo color, intensity/color of rim lighting, specular highlight placement, etc. The demos so far seem to arbitrarily change those (lower temp lighting, different eye/lip albedo, additional highlights, etc)

@virtual_method @IanCutress You almost certainly can choose how much each pixel gets affected by this, as that‘s the most obvious simple shader you could do on top of this. So you almost certainly could for example turn it off for faces if you don‘t like how it changes those.

"It changes the models! It affects the art!" So does tesselation. We've had that since DX11. x.com/IanCutress/sta…

@JTCoz @nimlot26 True, the Starfield one looks best compared to the original. It doesn’t take much for Starfield characters to look better, though… :D

@virtual_method @nimlot26 I can understand people not liking the RER effect because the game was already hi-fi (other than faces...) and Grace's model has that unfortunate AI slop babe aesthetic anyway, but Starfield, say?? It looks thoroughly improved in every way imo. But each to their own

@JTCoz @nimlot26 Well if you love mac and cheese, it may be unfathomable to you that I don’t. That’s ok. We all perceive things differently. I’m not worried about the technology, I’m worried that most games will end up having mac and cheese on them since I don’t like it- if that makes sense.

@virtual_method @nimlot26 Well, again: proportionality. I don't know how anyone can look at most of these examples and seriously claim they downgrade the overall fidelity. That's just mad. I mean, it's subjective - but that opinion still seems mad.

@JTCoz @nimlot26 In other words, this seems to trade control for perceptual “realism”. I don’t want perfect results, but let me determine if I want her eyes to have a glint or not, and where it shoud be.

@virtual_method @nimlot26 Isn't there also just a sense of proportion to consider here? Maybe you're right about this particular highlight being inaccurate. Would that REALLY override all the other huge improvements to the image?! Tech doesn't have to perfect in ALL RESPECTS to still be a net good.

@MisterSpace3 Why do their tunics look like plastic coats with dlss on? Why the overblown lighting and the extreme contrast? Don’t know man, it just looks wrong to me.

Do you still trust your eyes? Or do you trust someone else's opinions more than your eyes? #dlss5 #rtxOn

@kimmonismus That's kind of a half-truth: I can radically alter the look of an asset using eg. deferred decals without modifying the asset itself. However, adding wrinkles is not just "lighting". At the very least, it's modifying surface properties (roughness, normals, etc).

Important update: at the press conference I was able to talk to NVIDIA employees about DLSS5. They assured me that DLSS5 makes no changes to the characters at all (!), no changes to the assets, and no "AI filter" is applied to the characters. It's solely the lighting that makes it appear that way; they are completely the same, unaltered assets.

@nimlot26 In any case, Imho this shouldn't add specular highlight where there were none before. They're not like subsurface scattering or area light shadows that are expensive to even approximate, they're cheap and we've had a closed form solution for decades. No need to vibe-place them.

Actually AO is affecting specular also. You can turn it off for specular, or for diffuse for that matter, but normally you have it on for both in most game engines. Imagine this reflected on her eyeball. I think it's quite close. Light up sky and dark building building up a top reflection with an enough degree of contrast to match up what we have in the DLSS on image.

@nimlot26 That would be specular occlusion, which is different from ambient (more directional, ambient light tends to be isotropic). But yeah, I get what you mean.

@nimlot26 Don't know man. Part of my job involves writing shaders and lighting systems, and this looks quite off to me. It could boil down to subjective/personal preference of course, so let's agree to disagree. I just hope we get something more than an on/off switch.

There is definitely some specular in there but the shader is simple and real-time shadows and AO muted down all the specular that should come from the direction she's looking at. Even if you consider just the cloudy ambient sky everywhere in the world you should have some reflection of it on the eyeball.

@nimlot26 Could also be that her origional eye shading model is just diffuse and no specular, but that would be weird since specular highlights from light sources are pretty much free.

@nimlot26 No, I meant this one. The key light in this scene is placed on the right side. The specular in her eye suggests there's a light right in front of her and slightly above her head, but that's not there in the original.

@nimlot26 Overall light temperature also drastically changes: everything’s noticeably colder with DLSS on. Local constrast is also way too high, like in fake HDR photos. I’m not sure if all of that will be controllable, but it really should imho.

We will have to wait for the release to understand better how it works. My explanation was simplified and based on the little information that they gave us. I am not a technical artist myself but I understand enough to counter this whole AI slop filter thing. I keep hearing people saying it adds lightings sources. I don't see it in any of their examples. It uses the same light sources as the scene.

@Sebasti66855537 Yes, it’s *miles* better than the originals they showed. It no longer looks like they cranked local contrast sharpening to 200%.

I think this looks quite a bit better indeed.

@_BlessedRed Silhouette is not the same thing as shape. The volume of her facial features is completely different: more pronounced cheekbones, fuller lips, more prominent brows, thicker jaw, slighly thinner nasal bridge.

Literally all that was changed is lighting and shading. Its funny watching people that have no idea what they are talk about just disregard HOW MUCH lighting can change a person or how something looks.

@nimlot26 @engineseer We are aware of what they say it does. The problem for me is that it doesn’t seem to do that. Shadows are added/removed, light sources have their color/position changed, etc. It simply doesn’t blend right, results look wrong to me. Plausible, but wrong.

They literally tell in the nvidia post what it is trained for. It's trained to identify materiality and improve the render of it. Any art style will benefit from this. "The AI model is trained end to end to understand complex scene semantics such as characters, hair, fabric and translucent skin, along with environmental lighting conditions like front-lit, back-lit or overcast — all by analyzing a single frame. DLSS 5 then uses its deep understanding to generate visually precise images that handle complex elements such as subsurface scattering on skin, the delicate sheen of fabric and light-material interactions on hair, all while retaining the structure and semantics of the original scene."