Lighting and fullscreen shaders

[Tiberius]

I mentioned before that it seems to me that light colours are picking up too much diffuse light, or dark colours not enough in comparison. Seems the fault isn't the shaders or lighting, it's in the bloom map - basically, any texture over medium lightness starts to get added bloom, making light colours appear much lighter than they really are and much lighter than they render in just basic HDR. Any surface then which is over mid brightness then ends up one mass of bloom.

Seems to me now, actual TOD light levels are fine, except for ambient being too high at night. HDR is pretty muted before you add bloom, but that means you've got a lot of room to add extra effects to it, if you wanted it left alone it could easily be brightened up to match the light levels you get with bloom, it just wouldn't look as good :smile:

What it really needs are some nice post process shaders, something like colour correction and de-saturation perhaps? I tried shaders with a few Photoshop effects like hard light/screen/overlay, got some nice arty effects but not really sure what adjustments the lighting needs to make it look more realistic. The HDR is nice, it looks way better than it used to, but everything still looks too fake and rendered - the light levels are there but the whole thing just looks too 'clean' to be realistic. In the screenshots I put on the X-Type thread, and the few with the XJS convertible, sure the colours look nice but it doesn't look anywhere near close to photo real, nobody would believe you if you told them it was a photo..the potential is there but it needs some help.

Some screenshots of the same car on Carlswood - first two screenshots at the time at the same spot on the track - one with basic HDR but with bloom etc turned off, the other with HDR, hacked version of bloom and some de-saturation and a hacked version of the old sepia shader. The other shots again with the same set of shaders:



Yes, it's pretty toned down and it still doesn't play nice with light coloured cars and some track objects like the railings (again, the bloom map being added is making the light levels on bright colours too high), but it looks a lot less rendered. It *is* very bright, large parts of the sky do get bloom at some times of day, sun reflections from the car and road white out that part of the screen. Probably it's too bright, but then if you look at the sun reflected in shiny surface such as car windows/chrome/paint on a bright sunny day, it's gonna hurt your eyes. Turn the light down and it cuts some of the bloom but it just doesn't look as convincing (to me at least). Really it all needs to be adjusted, can't really do that until we have a good lighting setup to adjust everything from.

Ideally then we could have different shaders for different effects, some bright, some more muted etc.

This is just a few paragraphs of disjointed thoughts really, I've spent a few hours over the past week or so trying to work out some shaders but not really found an answer yet, seem to be going in circles with it. Time to throw it out to you guys..thoughts, improvements, better shaders, whatever you've got are welcome :smile:

 

[gtpdzbiz]

Racer do have much potential in terms of graphic. Below is some old screenshot when I was playing aroudn with HDR. The first two was using HDR cubmap as environment, which imo looks quite convincing, and they can get event better with more detailed car body. And the third one uses a simple scene without any pre-shot cubemap, and most of the work goes for tweaking the post effects atmosphere and camera settings.

All screens are just what they are without any photoshoping.

 

[luthobu]

@gtpdzbiz: nice.. really..

 

[Tiberius]

That third one especially looks really good, that's the type of look I was thinking of - very stylized and probably not everyone's taste, but much less rendered than it is now.

Really, things would look a lot nicer than they do at present, just with some shader/lighting tweaks and some nice fullscreen shaders - better bloom effects, some kind of colour/contrast correction etc. Hmm :smile:

 

[luthobu]

That third one especially looks really good, that's the type of look I was thinking of - very stylized and probably not everyone's taste, but much less rendered than it is now.

Yeah, I agree.. it's not "photo-realistic", but just.. right.. kind of.. heh..

 

[gtpdzbiz]

Correct. Most of them will look much better if we dig deep enough. I think the next setp is hoping for a more efficent gfx engine and Ruud mentioned that he will improve the HDR some day since some thing is not doing right right now(maybe the automatic exposure control and bloom).

And it would be great that Ruud can include some of the tweaked shaders/material in his offical release so that everything will keep improving and we don't have to do things from the ground up everytime a new version is release.

As somebody mentioned above, stylized shots are good for still frame but not moving ones. Maybe it those eye candies are not for everyone, but why not? Racer could also turn out to be an excellent still shot generater, and many of us enjoy making beautiful shots How about some dynamic post effect manager or event better, a photo mode in Racer?

 

[Ruud]

HDR has the problem that LDR is still there. I'm going to drop LDR I'm afraid to get HDR moving forward. Most HDR stuff currently is too much centered around 0..1 instead of a more dynamic range. That should change, and a range should be determined, since real-life values are too dynamic for the GPU still.

For bloom, in data/renderer/fullscreen_shaders_hdr you have bloom_bright_f.cg where SuppressLDR_v1() currently decides which light levels get bloomed or not. You may want to play with that.
You want levels really above 1.0 to get bloomed; now it's more like 0.6 and up.

In a perfect world, the fullscreen shaders won't add or subtract from the average brightness of the rendered frame; they're more there to accentuate things, not brighten or darken.

@gtpdzbiz : can you post those shaders you used for the screenshots? Esp the 3rd seems very interesting. http://www.racedepartment.com/members/gtpdzbiz.html

 

[gtpdzbiz]

Sure, here it is:

The first two shots didn't use any special post shader, just some HDR back drop.

And the third one use a shader to mimic 'ghost' effect in lens system, but it should look better than what it is now,

The technique is fairy simple, just shrink and stretch the bloom map into different scales and add them together.
But shirking a fbo into a smaller than screen size one will cause stretch artifact for those edge pixels, using some circular mask will fix the problem but full-screen shader won't allow any bitmap input at the moment.

Add some tinting into the 'ghosts' can also help improving some realism.

 

[Tiberius]

This is exactly the sort of thing I was thinking about, nice to see some more input.

I did play around with the bloom shaders, all a matter of taste I suppose but I really preferred the 'alternate' colour shift bloom shader. I also changed the blending of each bloom map, so the highest blur only works on the brightest colours (something like RGB 250) - the lesser colours then still pick up slight bloom but nowhere near as saturated with it as before, you then get some visible colour shift bloom on really bright reflections, mainly sun spots on car paintwork and chrome. Of course, it's still far from perfect, really it seems we need to wait around until the whole HDR/LDR thing is sorted out :smile:

 

[gtpdzbiz]

Tiberius, that's also what I am thinking about.

More RGB intensity=more bloom/bleed. I looked into Ogre's HDR system some day and find out that they will also do a bright adjustment to the bloom maps according to scene brightness, i.e. brighter scene->tune up the brightness of the bloom map. And they also use smaller map size(128*128) to increase bloom radius.

p.s. Ogre's post process managing system is amazing, it's a script which is separated apart from the main program.

 

[Stereo]

, you then get some visible colour shift bloom on really bright reflections,

Reminds me, my glasses have a slight colour shift (red/blue edges on bright objects) so I'm not seeing the world accurately in the first place. Makes it tough to judge which effects should go into the game engine and which can be provided just by the fact that we look at the monitor with our eyes. For screenshots/movies there are effects that aren't needed during gameplay, that are introduced by the camera.

 

[Tiberius]

For bloom, in data/renderer/fullscreen_shaders_hdr you have bloom_bright_f.cg where SuppressLDR_v1() currently decides which light levels get bloomed or not. You may want to play with that.
You want levels really above 1.0 to get bloomed; now it's more like 0.6 and up.

Aha, suddenly things begin to look much better again :smile:

In my humble opinion, 0.6 is indeed too low, kills the nice smooth lighting of bright colours, virtually everything above mid tones before was being obliterated by bloom - between 0.95 and 1 seems to look good here, light colours then play nicely with the bloom again, lighting was far too compressed in the low regions before. The screenshot below is what I'm getting tonight, I know that it needs a lot of improvement but that's the sort of lighting levels we were getting at first, it looked so good back then :tongue:.

Running currently on plain old Carlswood with normal (non VC) shaders, standard except a custom skybox, the only shader I've got is one which desaturates slightly, changes the colour tone (not too sure about the colour tone, really just playing around with ideas), and then returns the scene lightened a bit. Bloom between 0.95 and 1.0, I'm still getting bloom on light colours and sun reflections. Really I'm trying to keep it simple, all basic lerp functions etc to mix in the tone and desat colouring, I realize that all this is only temporary and that it's gonna change again soon but for the first time since the early HDR days, it's starting to look good again :smile:

 

[Tiberius]

More RGB intensity=more bloom/bleed. I looked into Ogre's HDR system some day and find out that they will also do a bright adjustment to the bloom maps according to scene brightness, i.e. brighter scene->tune up the brightness of the bloom map. And they also use smaller map size(128*128) to increase bloom radius.

My mod of the bloom shader isn't very sophisticated, all I did was mixed each bloom map separately, I tried a few different ways but this seemed to work quite well - there's probably a much better way to do it:

//mix each bloom map separately
  bloom1=(bloom1); 
  bloom2=(bloom2*bloom2); 
  bloom3=(bloom3*bloom3*bloom3);

That ghosting shader you posted works nice, yup it all needs a lot of tweaking but I can see where you're coming from, eventually it'd be nice to get effects like flares etc, perhaps some flare effects ONLY appearing on direct sun reflections rather than just light specular etc (not sure if that would be possible, to pass some kind of sun reflection map to the fullscreen shader?).

Reminds me, my glasses have a slight colour shift (red/blue edges on bright objects) so I'm not seeing the world accurately in the first place.

Again, I suppose we're back to the camera vs. eye discussion, everyone sees/wants to see things reproduced differently?

What you really want to do is get UV coated lenses and then leave them on the dashboard on a hot summer's day. I only really wear mine for night driving but it's one helluva way to wreck £300 of lenses :frown:

 

[gtpdzbiz]

My mod of the bloom shader isn't very sophisticated, all I did was mixed each bloom map separately, I tried a few different ways but this seemed to work quite well - there's probably a much better way to do it:

//mix each bloom map separately
  bloom1=(bloom1); 
  bloom2=(bloom2*bloom2); 
  bloom3=(bloom3*bloom3*bloom3);

That ghosting shader you posted works nice, yup it all needs a lot of tweaking but I can see where you're coming from, eventually it'd be nice to get effects like flares etc, perhaps some flare effects ONLY appearing on direct sun reflections rather than just light specular etc (not sure if that would be possible, to pass some kind of sun reflection map to the fullscreen shader?).



Again, I suppose we're back to the camera vs. eye discussion, everyone sees/wants to see things reproduced differently?

What you really want to do is get UV coated lenses and then leave them on the dashboard on a hot summer's day. I only really wear mine for night driving but it's one helluva way to wreck £300 of lenses :frown:

The ghost effect is taken from an old GDC paper, you can get it at:http://www.daionet.gr.jp/~masa/archives/GDC2003_DSTEAL.ppt

And maybe you can even try out the HDR demo at this guy's web site. It's very impressive.

 

[Tiberius]

Well, this is where I got to last night. Still playing with the colour tone idea, bloom seems good starting around 0.95 - at least on the shaders I'm running:



The effect is probably a bit too strong (too warm?) but it does look more natural to my eye, at least on my old Sony CRT - the lighting levels seem much less compressed and the highlights look more realistic here, even though it could probably be improved a great deal more. Really I'm just trying to desaturate strong light reds a bit, while keeping blues relatively untouched in most areas - basically just using different lerp mixes for each colour channel. Also by expanding the lighting levels again, the env map reflections become much more realistic, no way to get decent env map reflections with Racer's default HDR/bloom setup :smile:

gtpdzbiz: Internet has gone down again here (second time since Christmas), I've still got a connection - barely - but only getting dialup speeds..hopefully it'll be fixed Monday morning, until then it's hobbled at 50kb :frown:. The demo sounds good though.

 

[Mr Whippy]

Just FYI, in real life there are no blacks (except black bodies), or whites (except black bodies/lights maybe), so for any look up table/texture, where you have a range of 255 intensities, 220-225 is pretty much white, and 15-20 is as black as things get...

So keep this in mind when generating sky textures and so on. Every texture thus far in Racer is pumped beyond what is realistic for diffuse/ambient channels because of the textures.


I still think we need a good calibrated track. The fact people are STILL using Carlswood despite TONS of problems with it, to setup shaders/lights etc, is worrying me!

Start a new track and calibrate it with matte white objects (230 ish diffuse texture), and then adjust the tracks special.ini sun/sky settings to get it looking how you want (exposure/colouring), THEN use it as a base for deciding on how things should look in it. Ie, photo a matte white ball, and then in the same instance photo a metallic car paint, or a tyre, then you can compare fairly with real life (or at least as best as possible)


Just for instance, imo, NFS Shift blows everything out of the water right now for gfx and audio... stunning! I think they have got the balance really good... if in doubt, look what others do and how they make it work... I believe they make it work by starting with the basic principles and getting those right, before just trying to make a metallic shader look right on a totally messed up track... no offence intended here, just best to direct our efforts with some logic, rather than random tinkerings

When Racer looks good it looks bloody great. It's just all badly setup standards right now imo. The bloom halved looks fine, the HDR works fine here, it's just the values people are bunging in because they KEEP falling back to Carlswood as somehow 'right'

Start from the ground up with some thought and it all works fine imo

Dave

 

[Tiberius]

I still think we need a good calibrated track. The fact people are STILL using Carlswood despite TONS of problems with it, to setup shaders/lights etc, is worrying me!

I give up..

You really think I put that amount of effort into just throwing random numbers at it in the vain hope that something might end up looking right? You're right if you think I didn't use a slide rule to adjust the shaders, but I didn't exactly just throw random values at it either. I set the car up on different surfaces/tracks and the Carlswood in those shots is running non-vc shaders with what I believe to be 'logical' shader and atmosphere settings which look reasonable to me on most any car/track. Carlswood might not be an oil painting by 2010 graphics standards, but it's still only a bunch of textured polygons, the textures might not be detailed but at least they're reasonably lit/coloured, surely?

Even the overall colour tone which you can see on those shots came from a colour sample from a photograph of a diffuse white surface taken at sunset, hence why it's probably too warm for daytime lighting but I just happened to like the effect and left it in for those shots.

 

[Mr Whippy]

Don't give up

Not meaning to be negative at all, your work is essential to Racer in my view, everyone playing with shaders is making for a better Racer visually.

I will admit I am being lazy because what I am good at is being wasted not sorting out some logically calibrated environments to test things with. I bought the gear to do it with, but just haven't found time to show my working and show my results and let others work with what I have come up with if they feel it makes sense (like us all I guess)

So sorry for being negative Tiberius


I'm not sure on how good Carlswood is properly, the textures last time I looked were far from normalised looking which is always a danger (not your fault, just a legacy of where Racer came from and where we are today ), or how you are testing with it, hence my comments. Ie, I don't even know if you are running TOD, because that isn't calibrated well imo yet. Manual settings are better, but do they match the colour tones of the sky because they re-inforce the reflection colouring too (all obvious stuff but I'm just assuming it's the default very 'blue' Carlswood sky image which needs de-saturating and pumping a bit in our LDR texture environment)... we can end up with odd material behaviour because of that as well


It's all a mine field. I don't assume you are silly by any means and are throwing random numbers around... I'm just cautious that all old tracks are just questionable environments for setting up what is ultimately a totally new gfx system
Ie, on my current track I have adjusted specular maps and diffuse intensities so many times trying to find the right behaviour of materials vs photos. I have my sky colour etc variables, the sun specular variables on the materials, the texture variables, and I don't know if ANY of them are even right or wrong...

Hence us (I'll do it eventually I promise ) ideally doing some really nice calibration work with Racer. Re-create some real-life scenes with captured analog data (ie, photos at varying exposures and so on), and then try simulate them as closely as possible and see what settings we need. Then aim to make each one achievable by simply adjusting the primary variables like sky colour/intensity, sun colour/intensity etc... getting materials to then be passive and react properly so they look right...


That is the ultimate end place we need to get I think, and I'm sure you agree too... just in my view Carlswood is pants because no one here even made it, the textures are all a bit random in intensity and colour balance etc (just looked and the are a bit mis-matched)... I think we all want to do the same thing it's just us all finding time

I'll try get my track a bit more done and then you can at least sense check your car shaders on my environment which I hope will 'react' nicely with around four or five different setups for the environment. Ie, dusk, dawn, midday sunny, midday overcast and twilight.
Even in that task I've had a real hard time because the stupid environment shader seems to make everything black after 100m without diffuse lighting... grrrr!

 

[gtpdzbiz]

Hmmm, caliberating lighting is very interesting, and I have already done some experiment. I use hdrshop which is a freeware to sample hdr map for references. Then you can easily set up material and lighting parameters in racer. ( however I wish racer could have some realtime slider for parameter input, but that's another story) And there is also a plugin for hdrshop and what it does is converting a hdrmap into several direct light parameters.( color value, direction vector) so that you can easily recreate the lighting acccording to the sky map in other applications. You can see the results on the previous page. Just feel free to try it.

 

[Mr Whippy]

I've seen those rationalisation tools, so you can make the HDR down in to a few key point lights. Handy for realtime rendering

I can capture very rough HDR's of real environments now, using a mirror ball, but it's not so nice for reflections/mapping, just for the light analysis it is nice (the balls surface isn't great, you need a REALLY good one to get even half-nice results for imagery)

My plan is to picture some matte white balls, some mid-grey, and black (need to calibrate those in racer somehow with the same balls, but i've heard of a grey that photographers use that I need to match as closely as possible), and then my chrome ball, on a chequered board. Then also take picturs of my real car with known exposure settings, maybe at a few different ones. Then I can look to recreate rubbers, metals, metallic paint, material hood etc.

Then try recreate it all in Racer roughly, and then when I load different environments which *should* work right for each one, my car should look as it did in the photos in each of them.



Your HDR results look great. It'd be nice to have that as a default somehow. Not even have to make polygons for the sky etc, just have a slot in the track special.ini where you list a texture, and the x offset (for rotation for north direction), and then let Racer just use it correctly.
I'd still be happier with fixed lighting/time of day and have it look really really good, than have variable TOD and things, but have things not look as nice because of it.
Or at least have the option. A good location HDR these days is relatively easy to get, and if we can just pass in a file and have Racer deal with it all then that would be great

Would also be really nice for the car selection screen, ala GT5 P kinda scenes.

Dave

 

[gtpdzbiz]

In my opinion, real time realistic rendering needs good texture and lighting, specially texture. Static objects needs a good baking engine to get the lighting right. In any good baking engine, the lighting can always use just one HDR backdrop or several point/direct light as light source and leave the GI/photon calculation to the software. Then render those baked maps into HDR textures for using in any realtime apps. Considered that Racer does not support multi-light source at the moment, this could be a good choice.

For dynamic objects, when it's moving around in the environment, the surface lighting will be altered quite a lot by the surroundings. In realtime rendering world this effect is separated into three sub-process. 1. Diffuse lighting 2. Realtime reflection 3. Shadows. Racer already process the later two while the first part is mostly ignored. And that's one reason why cars in Racer sometimes look 'dull'. As the car is moving around the track, the diffuse lighting always stays the same, which gives people a very computer-generated feel.
There are already many solutions, like differed-shading, which looks pretty cool but with critical performance issue. Another solution is to use pre-rendered light source. That means baking scene-lighting information based on position into a texture or a point cloud. This technique is much older but pretty hardware-friendly.

Honestly, it's more difficult to create a full dynamic day-night cycled environment than a static one with full detail and proper lighting.

@Mr Whippy : The lighting calibration part sound very promising, I think it will be good if you start it in some small size static environment(yeah, like those GT5P scene). Thus you can talk full-advantage of static rendering, like Image-based lighting, which I believe could be done easily in Racer's shader system. We just need Racer to support(hopefully) .hdr , .exr or 16bit DDS files.

p.s. I read a paper on real time rendering of realistic vehicles, and saw a m3 on the cover which is identical to the one you created.