markecc

Rats can easily improve the graphics with an open source upscaler ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks)

205 posts in this topic

AI isn't an artist. It just runs programs, and the programmers usually are engineers, not artists.

The impressively engineered ESRGAN program provides for upscaling and sharpening. That's what it does.

The sharpening is better than such programs have done in the past.

That doesn't make the program an artist. Artists understand contextually how to achieve "realism". I'm sure the ESRGAN folks would agree that they haven't programmed that.

Share this post


Link to post
Share on other sites

I think ground textures are managed via LODs, aren't they? If so, presumably the sharpening would be done only on LOD0 and maybe LOD1. Thus there'd be no effect on distant appearance.

Share this post


Link to post
Share on other sites
14 minutes ago, jwilly said:

The impressively engineered ESRGAN program provides for upscaling and sharpening. That's what it does.

It's doing considerably more than sharpening and upscaling, but you're right that ESRGAN is not an artist. Nevertheless, generative AI is indeed a thing, and in the decades to come, it will become an increasingly widely-used design/development tool.

https://blog.openai.com/generative-models/

 

Here's a primitive example of what's possible now (essentially an AI artist that can quickly add textures to line drawings): https://affinelayer.com/pixsrv/index.html

Imagine what will be state-of-the-art in 20 years....

Edited by xanthus
2 people like this

Share this post


Link to post
Share on other sites

Our guys are currently examining the tool fyi. @xanthus in 20 years from now, maybe even 10 sure. I'd tamper expectations a little bit, but we're no doubt going to take it seriously and give it a true try.

3 people like this

Share this post


Link to post
Share on other sites
4 hours ago, jwilly said:

I think ground textures are managed via LODs, aren't they? If so, presumably the sharpening would be done only on LOD0 and maybe LOD1. Thus there'd be no effect on distant appearance.

2 states best i can tell (not counting object LOD's like buildings)
1) close enough to render the real ground and its polys
2) terrain rendered as the 1st LOD map view tiles

State 1 you might get some improvement using such a tool
State 2 would require a a decent amount of effort because of how the graphics resources are done for that view, is kind of hard to explain and requires use of a calculator

Share this post


Link to post
Share on other sites

Huh. So a player on, say, a high vantage point that can see the ground out 2.5 km, and that sees distant weapons/vehicles at the farthest LOD, sees full resolution terrain textures all the way out?

Share this post


Link to post
Share on other sites
Quote

Our guys are currently examining the tool fyi.

 

Can't wait to see the results  :)

Edited by markec

Share this post


Link to post
Share on other sites
6 minutes ago, jwilly said:

Huh. So a player on, say, a high vantage point that can see the ground out 2.5 km, and that sees distant weapons/vehicles at the farthest LOD, sees full resolution terrain textures all the way out?

yea.
when I was first looking at the terrain art, just to see as a test what kind of resolutions it might actually load, I took a high rez aerial photo
(which did not remotely match or anything nor repeat well at all)
and I cropped out a square dimensioned piece to see if it would display and of so how would it display.
I only did the ground itself, not any of the lower LOD map assets

I still saw my texture like 6km in the air

Share this post


Link to post
Share on other sites

Rats upscaling, small hint: don't forget that you can upscale the already upscaled texture , so a 256x256 tex upscaled to 512x512 can be upscaled further to1024x1024 and then to 2048x2048 and continue and continue......  :) 

I totaly forgot about this at start  myself when I tried the upscaler and had only a 512x512 texture and was thinking what all the buzz is about..... but when I upscaled  something to 16k holy moly.... it almost looked photo realistic.

Edited by markec

Share this post


Link to post
Share on other sites

Interesting. So what you are saying it's more of an iterative process?

So rather than going 256 x 256 > 4K, you should rather go 256 > 512 > 1024 > 1440 > 4K?

What kind of black magic do they have in this algorithm to cut down on the noise generated and propagate through each level? With the one step noise is just a factor of two, but just by doing 5 steps your error is X^32. I am completely amazed and baffled that an automated system can control for signal/noise propagation like that.

EDIT: It reminds me of those parodies in the movies like Super Troopers where the cops just keep saying "enhance" a million times to see the license plate.

Edited by aismov

Share this post


Link to post
Share on other sites

I would love to see a small example of this working WWIIOL. (Inf, building, weapon...anything)  Hopefully a small sample can be posted when someone has a chance to vet this proceedure.

Share this post


Link to post
Share on other sites
33 minutes ago, bmw said:

I would love to see a small example of this working WWIIOL. (Inf, building, weapon...anything)  Hopefully a small sample can be posted when someone has a chance to vet this proceedure.

Hatch said above:

Quote

I'm going to throw some textures together for a couple of models to see if Markec might be interested in passing them through at 4X current res and then post some before after screenies.

 

Share this post


Link to post
Share on other sites

Hatch provided me with some textures, here are the first effects

original

88mCKqf.png

 

after ESRGAN

ETnSJME.jpg

Share this post


Link to post
Share on other sites

Perfect.  Wasnt sure if that was just for internal processing

Share this post


Link to post
Share on other sites
2 minutes ago, markec said:

Hatch provided me with some textures, here are the first effects

original

88mCKqf.png

 

after ESRGAN

ETnSJME.jpg

EEK..........am I seeing this correct? Looks worse than the original??

Share this post


Link to post
Share on other sites

Hopefully @markec can have some samples to upscale models, whether its infantry, terrain or gun models instead. You would think upscaling something smaller is better anyway?

Edited by gretnine

Share this post


Link to post
Share on other sites

A flag design is probably not the best example of what ESRGAN can do (since it doesn't make much sense to add more detail).

I'm looking forward to seeing weapon, infantry, and vehicle textures (where ESRGAN should really shine)!

Share this post


Link to post
Share on other sites

Sharpening can make some kinds of mages look worse by adding gradient edge artifacts where we as viewers already know there shouldn't be an edge, but a subtle gradient was added in an earlier compression or downscaling step.

We've seen on other images in the ESRGAN papers that ESRGAN is improved over previous upscalers/sharpeners on some kinds of images, but fundamentally it still applies sharpening according to a set of fixed rules.

The problem with the flag texture is that we know what it should look like, and the program doesn't.

If a human artist went through that same image, they'd upscale it, then apply sharpening according to their understanding of how we want it to look...so they'd sharpen where edges belong, and smooth where they don't.

Try ESRGAN instead on ground textures, where we all want "realism" but we don't have a precise understanding of what that will look like. Maybe sharpening artifacts in ground textures will appear to be added detail, which might look "realistic" from close up.

Share this post


Link to post
Share on other sites
16 minutes ago, markec said:

Here is 2k original without enhancement

 

aMgPyJZ.png

Ok.....much better way to compare with both at the same size.  The enhanced one is sharper compared to the one above.  How would they look before and after at their actual sizes in game?

Edited by bmw

Share this post


Link to post
Share on other sites

Another apparent problem with ERSGAN in the single example above is that it inserts pixels along the sharpening lines that are darker or lighter than those on either side of the line. Thus the sharpening line is made more distinct. Maybe sometimes that's desirable, but in the flag we don't want to see sharpening lines through the subtle tonal gradients in the solid "white" areas. 

If the sharpening lines could be constrained to be in a color auto-sampled from near the line, say on the darker side, maybe the look would be better for this image type. But, probably for other image types it would seem as if detail was not being added.

How to contextually handle each sharpening line is one of those human-artist issues.

Another is understanding what the theoretical flag design is, when starting from a downscaled/compressed image that has some level of jaggies. We "know" that elements of the flag design are circular curves, but the program has no way to know that, or that it shouldn't sharpen the jaggies.

And, it's sharpened the boundaries of each polygon the original artist used in the solid bars to add tonal gradients to make the flag look "rippled". That, we don't want, but again the program just sees a gradient and has no way to know whether that's good or bad.

Edited by jwilly

Share this post


Link to post
Share on other sites
9 minutes ago, ZEBBEEE said:

Tests should be done with textures not images imo

In the flag example, that was a texture....

 

Only conclusion we can draw is that ESRGAN shouldn't be used on the flag textures. No reason to upscale these anyway.

Share this post


Link to post
Share on other sites
1 hour ago, jwilly said:

Try ESRGAN instead on ground textures, where we all want "realism" but we don't have a precise understanding of what that will look like. Maybe sharpening artifacts in ground textures will appear to be added detail, which might look "realistic" from close up.

Yep. Assuming that tiling isn't an issue, ESRGAN should do wonders for environmental textures (where additional details are a plus, not a minus). Hopefully that also holds for e.g. weapon, uniform, vehicle textures, etc.

Share this post


Link to post
Share on other sites

  • Recently Browsing   0 members

    No registered users viewing this page.