32 bit normal maps

[Vanguard]

GIMP doesn't support the latest such as BC7.

[LucShep]

You can also do the old school way by batch conversion (instead of one by one).

Pick whatever group of textures you want to edit, convert them to TGA, then after editing (resizing, etc, whatever) and saving them, convert them to DDS texture format again.

AMD COMPRESSONATOR (v1.50.1731) is what I'd recommend for this, also if you got no DDS plugin. It's available in both 32bit and 64bit app installer format.

It's not recent so does not support latest BC6 and BC7 format, but it's still widely used, a simple and complete tool.

There are plenty options for DDS file formats, delivers fast conversion and quality compression no matter the system.

PS: a very quick guide (image example) on how to do batch convert to DDS format is found inside the downloaded ZIP - procedure is identical for other texture formats (TGA, BMP, PNG, TIF, etc).

Edited October 18, 2020 by LucShep
added link for Compressonator 1.50

[Taz1004]

Good thing about Visual Studio... and the reason why I use it even tho I have Photoshop is because it tells you what format and compression the file is already in. Like the last image I posted on second post of this thread.

 

Photoshop with Nvidia plugin doesn't tell you that. So if you open the dds files with plugins, it may do "Open As" into 8 bit format (which is Photoshop default) and you might think they're already 8 bit files.

Edited October 18, 2020 by Taz1004

[mkiii]

Just so you know - Blender3d from Blender.org, (or Steam), which in case you live under a rock, is free. Loads these DDS files - you can just drag them onto a cube object to view them as a Colour map, or attach them as a normal map to see the result if you delve a little deeper. THey can only be saved as a different format however - such as TGA.

A word to the wise though, you might be wasting your time, and all you might be saving is some disk space. I wouldn't use jpg files for normal maps either. Also, just because a 32bit file is loaded, it does not necessarily mean it will be store in memory at that bit-depth. Finally - I'm not sure where this notion that 32bit is not a power of 2. Count along 2, 4, 8, 16, 32, 64, 128, ...:music_whistling:

Edited October 23, 2020 by mkiii

[Taz1004]

Also, just because a 32bit file is loaded, it does not necessarily mean it will be store in memory at that bit-depth. Finally - I'm not sure where this notion that 32bit is not a power of 2. Count along 2, 4, 8, 16, 32, 64, 128, ...:music_whistling:

 

That is incorrect as I proved on the first post. Graphic card or engine will not automatically downsize texture files at runtime. It will display different mip level based on size and distance but entire file will be loaded. Always.

And maybe I missed a post. Who said 32 is not a power of 2?

[Minsky]

bundle your normals as a custom cockpit, and it should pass IC

[Taz1004]

bundle your normals as a custom cockpit, and it should pass IC

 

That's actually a good idea.

[mkiii]

 

That is incorrect as I proved on the first post. Graphic card or engine will not automatically downsize texture files at runtime. It will display different mip level based on size and distance but entire file will be loaded. Always.

And maybe I missed a post. Who said 32 is not a power of 2?

 

If you have direct knowledge off how the engine loads, manipulates, and stores bitmaps, I defer to your greater experience. However, I did not say it was automatically downsized, and I didn't say it would not load the entire file. I said "it does not necessarily mean it will be store in memory at that bit-depth", meaning that not all of the information in a bitmap file has to be kept in VRAM. How a file is used by an engine is at the discretion of the developer, the libraries they use, and how they choose to manipulate a file after it is loaded into memory. I am saying that editing files just because they seem too large, may not always yield the desired benefits. File compression is also a factor, especially with Normal maps - the colours in the rgb channels are not just bump maps, they represent a vector, so the colour is the important thing.

 

But yes, I agree that generally, it is best to use a bitmap no larger than necessary, regardless of any shenanigans that are happening in the code before rasterization, but the notion that a 4k image is too big because the display is only 4k in total is not strictly true when you consider that for example, that portion of instrument panel that takes up perhaps half of the 4k image might be filling your vision for much of the time, so you have in fact, an image size that is lower than the maximum resolution if you have a 4k monitor. Also, you have to account for the fact that the player will often be zoomed in to an instrument or console. No one wants to see jaggies when zoomed in.

 

I was not hinting that you said 32 was not a power of 2. Search for "GPUs usually internally access image data as 32bit per pixel for performance reasons (unlike 24bit, its a power of two which is practical for computating pixel offsets and such)"

 

Either way, have fun with this & maybe persuade ED to do some more optimisations.:joystick:

[dorianR666]

I was not hinting that you said 32 was not a power of 2. Search for "GPUs usually internally access image data as 32bit per pixel for performance reasons (unlike 24bit, its a power of two which is practical for computating pixel offsets and such)"

yeah it was me who said that.

"32bit (...) unlike 24bit, its a power of two" means that 32 is a power of two and 24 is not. i didnt say 32 isnt a power of two.

 

i agree with what you wrote above this though.

[twistking]

taz, my experience comes only from offline rendering, so i don't know if it applies here at all, but i would have guessed that normal maps profit more from depth than from resolution.

it's obvious that 600 mb just for cockpit normals is excessive and probably (hopefully) just an oversight that will get corrected, but have you tried if downscaling the normal maps to quarter resolution, is not even more effective than reducing bit depth?

 

i must admit that i have never really experimented with it, but i have always been under the impression that normal maps can be significantly smaller resolution than diffuse maps, as long as they have no lossy compression, which can obviously ruin the resulting image.

[Taz1004]

but the notion that a 4k image is too big because the display is only 4k in total is not strictly true when you consider that for example, that portion of instrument panel that takes up perhaps half of the 4k image might be filling your vision for much of the time, so you have in fact, an image size that is lower than the maximum resolution if you have a 4k monitor. Also, you have to account for the fact that the player will often be zoomed in to an instrument or console. No one wants to see jaggies when zoomed in.

 

Your last reply makes more sense but what you said above is also out of context. I said 4k is too big for "NORMAL MAPS"

4k texture for diffuse might improve quality but normal maps are just supplement to diffuse texture. For rivets, bevels on gauge dials, grooves on knobs... 4k normal maps for those is just overkill.

 

Even if you put your nose right up on a rivet, you will not notice 4K normal map vs 2K normal map. In fact, not even 1K normal map.

 

File compression is also a factor, especially with Normal maps - the colours in the rgb channels are not just bump maps, they represent a vector, so the colour is the important thing.

Representing a vector simply means 3 channel. Versus 1 channel of bump map. And at 8 bit, it's 256 colors. Versus 16,777,215 colors of 32 bit. Imagine you're 3D printing a rivet head only. 8 bit is printing it with 256 layers. And 32 bit is with 16,777,215 layers. Would 32 bit better quality? Possibly but will you notice it? No. Will it take forever to print? Yes.

 

Nobody here's saying normal map isn't important. But is 32 bit overkill? Hugely.

By the way, all diffuse map in DCS is 8 bit. If they used 32bit on purpose, use it on diffuse where it would make more difference. Not on normals.

Edited October 23, 2020 by Taz1004

[Taz1004]

taz, my experience comes only from offline rendering, so i don't know if it applies here at all, but i would have guessed that normal maps profit more from depth than from resolution.

it's obvious that 600 mb just for cockpit normals is excessive and probably (hopefully) just an oversight that will get corrected, but have you tried if downscaling the normal maps to quarter resolution, is not even more effective than reducing bit depth?

 

i must admit that i have never really experimented with it, but i have always been under the impression that normal maps can be significantly smaller resolution than diffuse maps, as long as they have no lossy compression, which can obviously ruin the resulting image.

 

Yes, as I just replied to mkiii, pixel dimension also plays big part. Probably more so than bit depth. Going from 32bit to 8bit reduces from 65MB to 21MB per texture. But going from 4K to 2K will reduce it by 1/4. Because it's 2x width and height.

 

The files I uploaded on first post, I reduced the normals from 4K to 2K. But I do not notice any difference. I probably could've went with 1K textures. So combined with reduction in pixel dimension and bit depth, it went from 65MB to 5MB per texture.

 

DCS swaps a lot. And most of these textures are stored in compressed zip format. Bigger the texture size, longer DCS will take to extract from the zip file and then swap. And it doesn't only do that on modules you own. I don't have FA18 but if I'm on multiplayer and someone else joins with FA18, I have to load his 4K textures.

 

Of course, there's no strict rule and depends on situations but in game design, it's generally accepted that normal maps, specular maps, and other "supplemental" maps to be 1/2 in dimensional size to its diffuse texture. So if it's 4K diffuse texture, normal and specular maps should be 2K. But I usually go one step further and make them 1K.

Edited October 23, 2020 by Taz1004

[twistking]

 

Yes, as I just replied to mkiii, pixel dimension also plays big part. Probably more so than bit depth. Going from 32bit to 8bit reduces from 65MB to 21MB per texture. But going from 4K to 2K will reduce it by 1/4. Because it's 2x width and height.

 

The files I uploaded on first post, I reduced them from 4K to 2K. But I do not notice any difference. I probably could've went with 1K textures. So combined with reduction in pixel dimension and bit depth, it went from 65MB to 5MB per texture.

 

DCS swaps a lot. And most of these textures are stored in compressed zip format. Bigger the texture size, longer DCS will take to extract from the zip file and then swap. And it doesn't only do that on modules you own. I don't have FA18 but if I'm on multiplayer and someone else joins with FA18, I have to load his 4K textures.

 

Of course, there's no strict rule and depends on situations but in game design, it's generally accepted that normal maps, specular maps, and other "supplemental" maps to be 1/2th in dimensional size to its diffuse texture. So if it's 4K diffuse texture, normal and specular maps should be 2K. But I usually go one step further and make them 1K.

 

thanks for the insight. interestingly i always used that rule of thumb (1/2th size for supplemental maps) for my offline rendering work and this despite the performance impact of texture size on offline (!) rendering being neglectible perhaps; but i simply never saw a real benefit in going higher that would have justified dealing with the larger file sizes...

 

you mention the f-18: do you know if this oversized aux maps are present on all recent full fidelity modules? i think they are, purely from noticing the performance impact of adding a different aircraft to a custom mission (i have 4gb of vram and when flying the new a-10c i cannot add a single f-14, f-16 or f-18 to the mission without my system completely clogging up, going from 60 to 15fps).

i ask, because i want to make a dedicated thread in the bugs/performance section to help shine some light on the issue.

i'm quite sure that wasteful texture authoring is one of the main culprit for bad dcs performance. the outrageous thing is, that it would be so easy to fix without any difference in visual quality.

[Taz1004]

you mention the f-18: do you know if this oversized aux maps are present on all recent full fidelity modules? i think they are, purely from noticing the performance impact of adding a different aircraft to a custom mission (i have 4gb of vram and when flying the new a-10c i cannot add a single f-14, f-16 or f-18 to the mission without my system completely clogging up, going from 60 to 15fps).

i ask, because i want to make a dedicated thread in the bugs/performance section to help shine some light on the issue.

i'm quite sure that wasteful texture authoring is one of the main culprit for bad dcs performance. the outrageous thing is, that it would be so easy to fix without any difference in visual quality.

 

Other planes will load full exterior texture. It wont load cockpit textures. But even exterior textures are about 1-3GB per plane. F14 has 1.2GB of exterior textures and then around 200MB per each skin. I do notice that HeatBlur did not compress their texture files into zip however. This is better for performance but will take up more disk space obviously.

 

I have downsized most of the other plane textures and package them as mods. For myself of course. I believe this helps FPS immensely. I will probably do the same for ground units too. But there are people who likes to take HD screenshot of other airplanes and ground units too so this is different subject and will be difficult to implement by ED.

 

What I was originally proposing was be efficient with textures that are needlessly large and inefficient. Ones that wouldn't make any difference in visual quality. Sure, one or two oversized textures wont give huge performance boost but there are too many of these. Like 2048x512 for tree bark. And when the game is full of these, you can bet it'll hurt performance. Otherwise there wouldn't be entire subject regarding game profiling or optimization.

 

I don't have the module so I'm just guessing but I suspect there are a lot of these on Supercarrier and Syria as I see many people complaining about performance and them being relatively new.

[twistking]

Other planes will load full exterior texture. It wont load cockpit textures. But even exterior textures are about 1-3GB per plane. [...] But there are people who likes to take HD screenshot of other airplanes and ground units too so this is different subject and will be difficult to implement by ED.

 

What I was originally proposing was be efficient with textures that are needlessly large and inefficient. Ones that wouldn't make any difference in visual quality. [...] Like 2048x512 for tree bark. ...

Do those external texturesets of aircrafts also have their aux maps 1/1 size of the diffuse maps? I can understand aiming high when authoring assets: Games and users expectations grow over time. However 1/1 for aux maps would be excessive on it's own since the diffuse maps are already that high resolution. This is made so much worse however by DCS not being able to efficiently manage VRam resulting in a guaranteed botteneck on when VRAM cap is hit, which for some users can happen very quickly.

 

One could do screenshots and pixel-peeking all day, even with aux map on 1/2 size i highly doubt one would see a difference. For the newer HQ assets that is.

 

And your example with the tree bark is a good one: Imagine how a screenshot would have to look to make good use of that texture size. On a normal monitor it would be tree bark detail and nothing else: Digital Woodworm Simulator!

[Taz1004]

Do those external texturesets of aircrafts also have their aux maps 1/1 size of the diffuse maps?

 

I haven't checked all but everything I've seen so far, yes. They all have normal and roughmet texture of same pixel dimension as diffuse. It's obvious to me they're batch baking these textures from their 3D program... I'm guessing 3DS Max.

They only need to run another automated batch process from Photoshop to resample all of them before packaging them... But seems they have more important things to work on.

 

And the pattern I see is that newer files are worse. Older files are actually better optimized.

[Taz1004]

Just started looking through ground units and I'm finding more and more. Forget what I said about newer ones being worse. This one was compiled in 2014. And just from looking at the normal file, I have doubts about its effectiveness. Looks pretty flat to me.

 

[Minsky]

Looks pretty flat to me.

 

well, technically it's not flat, but yeah, for an AI T-90 a 4K normal is ridiculous

 

 

[stormrider]

 

well, technically it's not flat, but yeah, for an AI T-90 a 4K normal is ridiculous

 

[ATTACH=JSON]{"alt":"Click image for larger version Name:\tUntitled-1.jpg Views:\t0 Size:\t177.6 KB ID:\t7120306","data-align":"none","data-attachmentid":"7120306","data-size":"full"}[/ATTACH]

 

Not only 4k, but at 32bits on top of that.

[Worrazen]

I would just briefly add that this may not seem so bad, even tho I haven't investigated this issue, so this is a general comment: it may be signs of preparation to support HDR/WCG, future proofing, perhaps released with too much bit depth too early, so let's keep that in mind, future GPUs will have a lot more VRAM and DCS overally improvements should optimize VRAM allocation and use it more efficiently and it may all come together nicely in the end.

 

I will most likely be getting AMD Radeon 6800 with 16GB VRAM, I am so impressed they did exactly what I dreamed of, they went full blown to more VRAM rather than less, this will be great for things like DCS and other sims/games that do a lot of stuff that normal RTS/FPS games do not.

[Taz1004]

I would just briefly add that this may not seem so bad, even tho I haven't investigated this issue, so this is a general comment: it may be signs of preparation to support HDR/WCG, future proofing, perhaps released with too much bit depth too early, so let's keep that in mind, future GPUs will have a lot more VRAM and DCS overally improvements should optimize VRAM allocation and use it more efficiently and it may all come together nicely in the end.

 

I will most likely be getting AMD Radeon 6800 with 16GB VRAM, I am so impressed they did exactly what I dreamed of, they went full blown to more VRAM rather than less, this will be great for things like DCS and other sims/games that do a lot of stuff that normal RTS/FPS games do not.

 

Getting Deja vu. Your comment sound familiar so I searched and you made exact same comment on my other thread. Only to realize you were talking about diffuse.

https://forums.eagle.ru/forum/englis...91#post6364491

 

There is no future where 32bit normal map (bump map) will ever be useful.

 

And I did reduce many of the textures including ground units. I don't run MP but on A10C campaigns, I'm getting far less stutter in VR. Almost none actually. Only stutter I get is if I destroy too much and look into all the smokes. VRAM stays between 3-6GB.

 

And my system is fairly old. i7-4790K and GTX-1080.

[Worrazen]

 

Getting Deja vu. Your comment sound familiar so I searched and you made exact same comment on my other thread. Only to realize you were talking about diffuse.

https://forums.eagle.ru/forum/englis...91#post6364491

 

There is no future where 32bit normal map (bump map) will ever be useful.

 

And I did reduce many of the textures including ground units. I don't run MP but on A10C campaigns, I'm getting far less stutter in VR. Almost none actually. Only stutter I get is if I destroy too much and look into all the smokes. VRAM stays between 3-6GB.

 

And my system is fairly old. i7-4790K and GTX-1080.

 

Yes I'm kinda busy with other stuff, I come here sporadically, forget and say the same thing, but yes, I am not completely digging into these subjects right now that's why the disclaimer.

[Aarnoman]

Yeah, definitely an area that has a lot of potential for optimisation with no drawback. I am surprised we still have not had input from any ED staff members, this seems like very low hanging fruit in optimising resource use (even if there is no net fps gain, freeing up VRAM is important to reduce load stutter, as well as opening up VRAM for other textures).

[twistking]

[...] this seems like very low hanging fruit in optimising resource use (even if there is no net fps gain ...)

Haha, tell that to the man who runs DCS on 4GB of Vram!

(Hint: It's me. The man is me!)

[Aarnoman]

Haha, tell that to the man who runs DCS on 4GB of Vram!

(Hint: It's me. The man is me!)

 

As a fellow GTX 1080 user, I can sympathise.