Lighting and fullscreen shaders

[Tiberius]

I mentioned before that it seems to me that light colours are picking up too much diffuse light, or dark colours not enough in comparison. Seems the fault isn't the shaders or lighting, it's in the bloom map - basically, any texture over medium lightness starts to get added bloom, making light colours appear much lighter than they really are and much lighter than they render in just basic HDR. Any surface then which is over mid brightness then ends up one mass of bloom.

Seems to me now, actual TOD light levels are fine, except for ambient being too high at night. HDR is pretty muted before you add bloom, but that means you've got a lot of room to add extra effects to it, if you wanted it left alone it could easily be brightened up to match the light levels you get with bloom, it just wouldn't look as good :smile:

What it really needs are some nice post process shaders, something like colour correction and de-saturation perhaps? I tried shaders with a few Photoshop effects like hard light/screen/overlay, got some nice arty effects but not really sure what adjustments the lighting needs to make it look more realistic. The HDR is nice, it looks way better than it used to, but everything still looks too fake and rendered - the light levels are there but the whole thing just looks too 'clean' to be realistic. In the screenshots I put on the X-Type thread, and the few with the XJS convertible, sure the colours look nice but it doesn't look anywhere near close to photo real, nobody would believe you if you told them it was a photo..the potential is there but it needs some help.

Some screenshots of the same car on Carlswood - first two screenshots at the time at the same spot on the track - one with basic HDR but with bloom etc turned off, the other with HDR, hacked version of bloom and some de-saturation and a hacked version of the old sepia shader. The other shots again with the same set of shaders:



Yes, it's pretty toned down and it still doesn't play nice with light coloured cars and some track objects like the railings (again, the bloom map being added is making the light levels on bright colours too high), but it looks a lot less rendered. It *is* very bright, large parts of the sky do get bloom at some times of day, sun reflections from the car and road white out that part of the screen. Probably it's too bright, but then if you look at the sun reflected in shiny surface such as car windows/chrome/paint on a bright sunny day, it's gonna hurt your eyes. Turn the light down and it cuts some of the bloom but it just doesn't look as convincing (to me at least). Really it all needs to be adjusted, can't really do that until we have a good lighting setup to adjust everything from.

Ideally then we could have different shaders for different effects, some bright, some more muted etc.

This is just a few paragraphs of disjointed thoughts really, I've spent a few hours over the past week or so trying to work out some shaders but not really found an answer yet, seem to be going in circles with it. Time to throw it out to you guys..thoughts, improvements, better shaders, whatever you've got are welcome :smile:

 

[Stereo]

I did a fairly simple desaturation code because I thought it was a bit excessive at full levels.

  float3 c2 = color.rgb;
  float ma = .9;
  float mi = (1.0f-ma)*0.5f;
  color.r = ma*c2.r + mi*(c2.g+c2.b);
  color.g = ma*c2.g + mi*(c2.r+c2.b);
  color.b = ma*c2.b + mi*(c2.r+c2.g);

ma=0.33 would be complete desaturation, all 3 channels equal. With .9 it's just a little less intense.

The trouble is writing track special.ini lighting in desaturated or sepia toned mode. Can't really get an accurate idea of how bright things should be when the fullscreen shaders are messing with stuff.

And the mirror and other gui items don't use the fullscreen shaders, which makes the mirror look a little odd.

(this is with 0.4, which is not complete desaturation - the Murcielago is actually purple)

 

[luthobu]

Some screenshots of the same car on Carlswood - first two screenshots at the time at the same spot on the track

The sky color in the second shot is really nice.. is it the sepia shader that does this?

 

[Mr Whippy]

The bloom is indeed too hardcore.

I simply halved it's strength in the fs filter, while also removing vignetting. This seemed to work fairly nicely.


I would be cautious though about using Carlswood for ANY testing. It has baked vert lighting which is of questionable balance vs the tracks special.ini settings. The settings in the tracks special.ini are also questionable. Very sunny and bright.


We really need fs filters that are not 'glitzy' for setting things up. Mine removing vignetting and changing my bloom factor to 0 (from 0.5), means it's a flat render, and makes setting up lighting nicer.
I then put a 230 rgb texture for diffuse/ambient on a sphere (nothing is 255 unless it's an emitter, naturally the whitest a material can get is about 90-95 reflectance), a matte sphere, and then drop it in scene. Now you have a benchmark object to 'expose' for.

Now, if we had a proper HDR system (real values), we would use silly big figures for sun light brightness and sky intensity, but since we don't, we set up our track special.ini sky/sun values to get the look we want on our sphere, and/or appropriate for our sky texture, and vice versa.

If you drop a white matte ball into Carlswood it's just solid white over one half, so way over-exposed, but the track vert lighting is darker. It's just all wrong really.



So, imo, ignore Carlswood as a calibration environment for making shaders and choices on. I quite like the Rudskogen track with a few special.ini tweaks (I dropped in my 'sphere' car with matte white material and set up the lighting nicely)... much better to use a track like that imo!


Dave

 

[Mr Whippy]

Also, the photo vs real point is interesting.

Photos are deceptive because they have a limited dynamic range vs our minds eye... what we see.


I guess it depends what we want, but for 'driving' I want my minds eye looking graphics. For replays and track cams, I'd prefer either minds eye or photo looking imagery/video.


Your eye will look out of a car and 'flatten' the dynamic range out vs what a photo of the same scene might look like. Our eyes do some auto-exposure, but I'd 'feel' that our minds eye has twice the kinda dynamic range as most SLR cameras...


Tis a tough one.

 

[KS95]

Lol yeah. I've seen many arguments about whether a game looks realistic or not. For example, Forza Motorsport 3. At first I thought it looked very realistic like many others however a few people had said to me that it looks like a cartoon. And frankly, I can see that now. It's not really possible to make a game look just like what we see in RL, is it? It's just not technically possible at the moment.

I hope I'm not talking ****e, but I try.

 

[Stereo]

Our eyes do some auto-exposure, but I'd 'feel' that our minds eye has twice the kinda dynamic range as most SLR cameras...

The tricky part, I think, is that our eyes auto-expose the object we're focusing on. Even if the sun's in your peripheral vision, everything else doesn't black out to compensate. If you stare into shadows you can make out some details despite them being surrounded by bright objects. But the computer doesn't know what you're looking at, and the screen can't display full HDR, so it has to make a compromise. I guess it could be coded so auto-expose level weights the centre of the screen higher, or some designated target area (if it's an interior car view, the centre of the windscreen for example).


If you want to drop vertex lighting from Carlswood, I think it's just a matter of changing the shader to one that ignores that information. But I don't know if the unlit Carlswood looks any good.

 

[FiftyOrange]

Geniune screenshot:

:frown:

 

[Some1]

Also, the photo vs real point is interesting.

Photos are deceptive because they have a limited dynamic range vs our minds eye... what we see.

I guess it depends what we want, but for 'driving' I want my minds eye looking graphics. For replays and track cams, I'd prefer either minds eye or photo looking imagery/video.

Your eye will look out of a car and 'flatten' the dynamic range out vs what a photo of the same scene might look like. Our eyes do some auto-exposure, but I'd 'feel' that our minds eye has twice the kinda dynamic range as most SLR cameras...

So the real question is, wheter a game should look photorealistic or eyerealistic? :smile:

I have always wondered how iRacing manages to get so photorealistic graphics (at least they look very photo realistic to me). The screens here look almost like photos, and yet you don't see any fancy effects like bloom or hdr? Though, its unlikely that iRacing has dynamic time of day...

I guess it all comes down to correct lighting and well made textures?

 

[Mr Whippy]

In past Gran Turismo games I've noticed that replays did look clinical, but then watching real life footage at Tsukuba say, it looks just like a GT replay, I almost thought I was watching a GT4 replay once and then realised it was a real Best Motoring challenge.

So there is 'videoRealistic' too... does it look like video footage?

I know that is similar to photorealistic, but many film cameras are still a deep depth of field and not quite so 'arty' with vignetting etc.


I think we need a 'clean' setup environment, and then effects are added over the top by preference. No default blooms, vignetting, blurring etc. All those should be fragments defined in fs_filter, perhaps.
Better still, track cameras and car cameras could have an fs_filter entry point and so different cameras can have different styles, from a bit arty blown out shots, to others with a shallow depth of field, to internal car cameras with a very eyerealistic emphasis etc.


Loads of options, just once again we have had a single fs_filter hard coded for ALL rendering, when it could hopefully be defined per camera...

Dave

 

[Tiberius]

Lots of interesting points here. :smile:

So the real question is, wheter a game should look photorealistic or eyerealistic?

Now, those screens of iRacing, to me they look rendered. Not in a bad way, the lighting and textures are definitely well done, but it looks fake in a photographic sense - very few people would believe you if you showed them that shot and told them it was a real photograph. So I suppose I've opened another can of worms here, much of whether graphics looks good or bad is down to the individual looking at them. I'm more into the arty/photography side, so I'm probably looking for things to be photorealistic rather than true to what my mind's eye might want to see, hence why I probably like stuff like colour shift bloom effects, because that's what you'd get through a lens and see on a photograph, looking at bright surfaces in sunlight would hurt your eyes or send the exposure on a photograph to hell and you'd get visual artifacts from it.

I think that some of the stick that post processing gets is where some games (especially some early ones) did it so wrong, I don't buy many games but one which stands out to me is UT3. They took a good engine, good meshes, good textures, yet to the majority of people (including me), the result looked really bad. I bought that game within a couple of days of it being released, long before I'd read what anyone else thought of it, yet as soon as I started playing it I noticed the greyed/blurry/hazy effect everything had over it, it really looked bad to me and killed any interest I might have had in it otherwise. Like KS says about Forza, what can look really good to one person might look really bad to the next.

The tricky part, I think, is that our eyes auto-expose the object we're focusing on.

Yep, our eyes can adjust to lighting/colours in a subtle way that a camera can't do. Take a photograph in a room lit with tungsten lamps, even with a good camera, and everything will be amber coloured. Because the camera sees the colour temperature of that light for what it is, instead of gradually compensating for the environment around it.

The bloom is indeed too hardcore.

I simply halved it's strength in the fs filter, while also removing vignetting. This seemed to work fairly nicely.

Yep, on those shots I cut it by roughly a third, so that a light value of 1.0 on all three maps would return a 1.0. I never thought about the vert lighting on Carlswood, as I remember the version I had in 086 I did lose the vert colouring on and it looked quite bright but did look better to me, better with higher res textures than it is now.

Again, I'm really trying to stick to TOD, so it's more limited at the moment trying to get a good lighting setup (all we can change is fog and sun intensity, right?)

We're getting back to the point about what the fullscreen shaders can do, the ideal would be to set up the basic HDR (no vignette, bloom etc) on a well lit track and get the defaults from there, then be able to have the option to add extra shaders *just* to add extra effects, without trying to compensate for what's lacking otherwise. Carlswood and the Lambo really need seriously updating now, even if it was just sharper textures/shaders.

The bloom is a tough one, I tried several mixes so far but can't get anything which looks as natural as the default one, even if it is too strong. I'm using a modified version of the 'alternate' bloom in the bloom.cg, again that's what just looks good to me - to the next person it might look wrong.

The sky color in the second shot is really nice.. is it the sepia shader that does this?

Yep, that's a combination of the sepia and the de-saturation. I did tone down the sepia effect quite a bit, really to just give a slight hue..again, it's all down to what you're looking for in the graphics.

Hmmmm.. :tongue:

 

[camsinny]

###EDIT###
Dammit, wrong thread...how do you delete posts again?

 

[EngineOfDarkness]

So the real question is, wheter a game should look photorealistic or eyerealistic? :smile:

I have always wondered how iRacing manages to get so photorealistic graphics (at least they look very photo realistic to me). The screens here look almost like photos, and yet you don't see any fancy effects like bloom or hdr? Though, its unlikely that iRacing has dynamic time of day...

I guess it all comes down to correct lighting and well made textures?

For me they look more like what you would see standing there with a mix of a photograph. There are several things which just disturb me in most games, one of it being DOF. It's not that it's bad, but it just seems too strong for my taste (that might also be caused by the fact that a photo does not adjust the DOF Effect to the place I am looking at ). Nevertheless the screens do look pretty good in my opinion.

However I think we won't get any eye realistic game soon, because that would mean the game would have to adjust to our eye movement e.g how far does the player look, where does he look, how fast do his eyes move, etc. pp...

Anyway it was kind of an "ooh" effect seeing the screens. Opposed to some other game I'm currently playing that looks way nicer to the eyes (the coloring, brightness and stuff), well that's at least what I think of the screens. Don't know how all that looks when you are in motion.

I don't like they way racer currently looks on the other hand (I'm using ATI), maybe that changes once it gets better shaders.

 

[davidiau]

Maybe we need to look at this from another angle, maybe not compare Racer gfx to photo's or what we see. Even 1080i Video or TV doesn't look as good as a photo, but we watch tv because it's a reasonable compromise between the quality of a photo and being able to display movement/motion. So, Racer in the same vein is NEVER going to be as good as a hi-res ray-trace of the scene, but Racer will always have a higher fps, a lot more playable, at least on any gear I can afford this century.
Monitors have limitations, resolution, framerate, contrast compared to photographic paper, video cards can't dispay as full a range of colours as the eye can see, etc etc. Are we asking too much of Ruud, the shaders etc trying to overcome inherant hardware limitations?

So, where do we want the compromises between playability & photo-realism to be?

 

[FiftyOrange]

Maybe we need to look at this from another angle, maybe not compare Racer gfx to photo's or what we see. Even 1080i Video or TV doesn't look as good as a photo, but we watch tv because it's a reasonable compromise between the quality of a photo and being able to display movement/motion. So, Racer in the same vein is NEVER going to be as good as a hi-res ray-trace of the scene, but Racer will always have a higher fps, a lot more playable, at least on any gear I can afford this century.
Monitors have limitations, resolution, framerate, contrast compared to photographic paper, video cards can't dispay as full a range of colours as the eye can see, etc etc. Are we asking too much of Ruud, the shaders etc trying to overcome inherant hardware limitations?

So, where do we want the compromises between playability & photo-realism to be?

This is a fair point.
But
I can watch TV on my PC monitor, and I can tell that it's real. As you say, it's not going to be precisely the same as real life, because it's not real life, it's a flat image on a flat screen made up of tiny squares. But it's not so bad that it doesn't look real. So what you're saying is valid, but I still think it can be pretty realistic.

 

[Tiberius]

Been thinking about this today.

Just my humble opinion, I think you can attempt something which approaches what you might accept for realism, if only at a quick glance and even if you have to make a lot of compromises along the way. Not to say you could get close to having something with the complex lighting and true dynamic range of a photograph, but you can go some way to making it look convincing - at least to the point of thinking how it looks quite realistic even though you know that it's only a rendered image onscreen.

Some games do go way OTT with the post process to my eye, one series which really stands out to me are the current Codemasters games. In the official screens for Dirt 2, there's just so much going on there that it ends up looking a mess. Bloom, haze, colour changing light effects, just too much going on on the screen at the same time. Some of the highlights such as reflections from the sky and objects like the car paintwork look nice, but I really wouldn't want to go that far with the effects.

I think I was wrong about light colours and bloom, light colours DO get lightened by bloom in strong sunlight. Or should that just come under diffuse lighting or even specular?

My eyes see bloom - only a small amount, but look at a bright colour in direct sunlight and you'll probably notice that colour will bleed onto an adjacent dark colour. A camera lens will behave similar, except that you'll probably also see some colour shifting on the edges of that dark object and even some flare if the light coloured surface is bright enough (like looking through tree branches into a bright part of the sky, focused on the nearer object, light bleeds around it).

 

[gtpdzbiz]

All of those cool-gfx effects are developed by specialized graphic programmers. Those games comes with a price 'cause game companies invest lots of money into developing the graphic engine. Like the latest GT5 title, which uses advanced lighting technology from illuminate labs to bake all the light maps. http://www.illuminatelabs.com/


However,racer does not have the luxury of accessing those resources at the moment.
The graphic part is currently under develop with limited resources. Utilizing some third party gfx engine like Ogre would be an alternate path if we really want to see some break through in a foreseeing future.

 

[KS95]

That depends on whether Ruud wants to spend anything from $15k to $90k, doubtful though :tongue:

 

[luthobu]

So the real question is, wheter a game should look photorealistic or eyerealistic? :smile:

Have been pondering this one for a while, and I think I mentioned it before. Photo realism is good for stills, but not for dynamic content like Racer. It would be as if every frame rendered was a still.. and that is really bad, and has nothing to do with the way we observe in reality. In fact, everything we see is blurred, it is a composite (is that the right word?) of all the impressions around us, and it only becomes "photo realistic" when we focus on something. What we see are impressions, not photos. So.. eye-realistic.. whatever that means... seems like a better option..

But then again.. Racer can handle FOV, auto exposure, blurring, fog.. so.. *shrug*

 

[Tiberius]

GT5 is something else, I don't really take much notice of what's going on in the computer/games industry but I realize there's a lot more going on in there than simple HDR shaders.

I've tried some tests using a high res HDR cubemap as env map, no matter how I adjust shaders/lighting I just can't get anywhere near realistic looking materials, especially reflective materials like bodywork - that's been the issue all through. I realize that we're working with a very simplified lighting model (as with most games), in some places it's really going to show - ambient light for instance. So I suppose 'photorealistic' was a bad choice of words, I'm thinking more along the lines of getting a level of lighting and visual quality which would be more convincing than it is now, hence my words in a previous post that Racer has potential - it's just not showing at the moment with the present lighting/post process setup, in my humble opinion it should be able to look better than it does.

Yes, textures also play a large part, I'm using bump maps more and more because apart from adding texture, they add a huge amount of realism to a material, even materials where you wouldn't directly think of using them.

I was looking at Ogre a long time ago, it's a nice engine, even just to mess around with. If you look on the showcase there was a kart sim posted on there about 2 years ago. At that point those were some of the most realistic graphics/lighting I'd seen.

Really, it comes back to the reason I started this thread, improving the look of what's there at the moment (lighting, shaders, post effects etc).