Mipmap envmaps and sky model questions

[Mr Whippy]

Currently I'm trying to get the live envmap to become a reference for general object reflections.

Mipmapping is really powerful because it softens the reflection map as if the reflecting material were rough, when infact it is still perfectly sharp, just the reflection itself is 'rough'

However I currently struggle with the sharp edges where a bright item (ie the sun) rolls onto another face of the cube map. It has a nasty sharp edge.

Also, the filtering for mipping looks pyramidal, so the sharp sun simply errs towards being a bigger sharp square still, rather than a big soft spot.


So first question, is it possible to avoid the sharp edges in the cube maps in any elegant way (shouldn't the mip really look across the edges of the cube map?)

Is there a way to blur the mipped cube map so the result is more pleasing? I've seen a blur map used for a bloom but it was for a 2d tex blur, not a cubemap blur. I'm struggling to make anything work that looks nice.



Lastly, the sky model. Is it technically right by now? I still get a deep orange opposite the sun set or sun rise, despite most real life sunsets and sunrises being opposed by a blue sky.

I was reading about someone implementing the mie/ray system and had this issue and apparently it is wrong, the sky shouldn't be symmetric at sunrise/sunset, but asymmetric.


Hmmmm

Dave

 

[Some1]

As you know, the envmap is generated by rendering the scene multiple times for each side of the cube. So you will get distotions around the edges, since the sides cannot tile perfectly (field of view is not perfect?).

Anyway, I think you could do interpolation from one side to another to avoid sharp edges, however, the sun would still not look right.

To get the sun perfect, isn't there a way to render the cubemap so that the sun alwasy faces one side entirely? (by rotating the cubemap somehow?)

Still, If we also consider that there are more light source than the sun alone (lights?), then its hard to get it all perfect... I wonder how proffessionals do it?

 

[Mr Whippy]

I guess so. I run my envmap at a side per frame vs 6 sides per frame (ie, full cube), so in motion it's really gonna have non-perfect edges!

I guess if you blur it heavily and respect the edges of the cube sides wrapping around then you should get an ok result.
The sun will still stay quite small and sharp but have a huge halo around it that is also quite intense.


I'm basically adding reflection response to all my road surfaces and adjusting the fresnel and alpha control maps... but I can't get a really nice final look because I can't actually get a proper blur of the envmap, and the mip-map although working pretty well, is still not ideal to check the proof of concept.

Ideally I could check with the new envmap system BUT the whole effect I'm after pivots about having a HDR envmap ideally, so the LDR one doesn't work.
I could make a HDR then blur it etc, but then it's not gonna match the specific Racer scene to see if the materials look good in response to the sky.


Hmmm... would be really nice if the envmaps could be HDR. I'd probably be happy enough with static ones if that is the only elegant way forward, but I think it's important for realistic track surfaces.

On an overcast white day you have no spec and tracks look so flat and dull, but IRL you get the specular 'effect' but at a lower intensity from every direction.
Having the envmap being white and bright all over on such a day, and turning off the spec/gloss and replacing it with a reflection with a steep fresnel (0/1/16 say), then the tracks suddenly come to life.
It works well on a sunny blue sky day too, the exposure drops lots, so the blue reflection is much more subtle, and the sun spot becomes the source of specular. Downside is that currently it's not blurred, so it's very small, when in fact a blurred HDR would have a huge apparent white spot across 1/4 of the sky at the exposure we'd be running...


So close but so far with these new envmaps hehe...

Dave

 

[Mr Whippy]

Well I had a play and I think the above method is feasible, but not really going to happen without BIG graphics changes that will delay any possible V0.9 further (ie, live mini envmaps in HDR rendered in real time with variable update rates and blurring etc)

I had a play with LDR envmaps but they just didn't work for this technique generally, never mind the fact they are static and so only allow a fixed TOD to work their best (which wasn't very good)



So I decided to implement the next sensible option which is adding energy to materials that are lit.
Currently we light with an ambient term, then a diffuse term, then mix them with a shadow mask so we get only ambient in the shadows.
We also apply specular lighting like that from the sun.

The problem is, we don't just get specular reflection from the sun, we get it from the sky, but there is no specular from the sky in Racer by default.

I decided to add it in and think it makes the visuals a lot more impressive. It's far from perfect because the sun diff/amb values need to match the sky rendering values closely for everything to come together and feel right. That is hard around sunrise/sunset, but for middle day values it's pretty nice looking.

Better still, overcast days now look lively and interesting where before they looked flat as there was no specular response from materials.


For now I just tweaked the static burn shader which uses alpha channel to determine spec intensity, then you can also use spec values in the shader (to tint them if you wanted etc).

The glossiness value might also be worth using, perhaps in this 'ambient' specular use, the glossiness could be an exponent on the alpha channel so a glossier value makes the alpha map have a higher contrast???


This is a realistic behaviour, but the energies are not really conserved and ultimately what is happening here is reflections and they are linear with view angle in this implement, vs a proper fresnel response or an IOR definition... but hey, we can only do so much it's a realtime engine after all.
There has to be sensible manual adjustment to get the right behaviour but it's not that hard if you use reference materials from real life (ie, light meter, HDR photography, and general imagery at different TOD's)

This is the usual shot in Racer where you are heading away from the sun (this is a bright sunny day)... the image looks much better with the specular response from the ambient 'sky' light, rather than just having no response at all except the default ambient * texture colouring.

I'm having a think now how I can put this code into the ambient term generally, then you can tweak the effect per material via the ambient and spec values etc, and where present an alpha map gives you more control as usual... hmmmm...


The shader I altered was this:
standard_burn_a_f.cg
Line 78:
float3 ambientColor=(baseCol*(ambient+Ke))+(lightAmbient*baseCol.a*Ks);

Just added the lightAmbient (or "ambient" ideally as then a downward facing polygon will get a load less ambient specular) * alpha map * specular term from the shader entry itself.
Using these values in the shader entry:
specular=2 2 2 2
shininess=8


The only slight downside with the whole effect now is that at sunrise/sunset the sun colouring and intensity is dwarfed somewhat by it's impact on the sky itself (ie, the orange bloom it generates)... so there is some compromise at sunrise/sunset on getting a realistic sun value for lighting the scene, and getting a good 'specular' response too...




So yes, best seen as ambient specular basically. Just as the sun gets to put it's colouring influence on everything AND a specular influence, so the sky should get to put a specular influence on things alongside it's colouring influence

Since values are now near real and HDR (intensities), it makes sense to do this... and I think with very basic values it just works and looks right which is a good sign! (ie, spec values are shared with sun/sky influences and balance nicely)

More work to come but I think that this is the right way to go for now...

Cheers

Dave

 

[Ruud]

I had a play with that; notice that in lighting.cg there is already:

ambient=Ka*lightAmbient*normalInfluence;

Now to add ambient in each shader seems wasteful, although the effect of baseCol.a on specular (Ks) could be explicitly stated so that you get back to (in standard_burn_f.cg):

  // Lighting
  float3 ambient,diffuse,specular;
 
  Ks*=baseCol.a;   // specular adjust to specular map LightingSun(Ke,Ka,Kd,Ks,shininess,lightDirection,lightColor,lightAmbient,IN.Position,IN.normal,eyePosW,
    ambient,diffuse,specular);
  float3 sdColor=baseCol*diffuse+specular*baseCol.a;
  float3 ambientColor=baseCol*(ambient+Ke);

Then in lighting.cg, make the change global for all shaders:

  // Ambient only
  //ambient=Ka*lightAmbient*normalInfluence;
  // Ambient with specular ambience
  ambient=(Ka+Ks)*lightAmbient*normalInfluence;

The effect I see is that specular surfaces get softer shadowing (drowned by specular ambience) and surfaces like grass (low specular/Ks) get darker shadows.
I like it, I'll modify shaders with specular map to modify Ks before passing it all to LightingSun().

 

[Mr Whippy]

I'm not sure on the right way to implement it all. My only concerns were that alpha isn't always the specular, sometimes it's the normal map alpha that contains the specular.

That led me to dig into each shader specifically and do the extra multiply on the ambient colour by specular alpha control wherever it was for that specific material... but I guess it makes sense to do this even for stuff without the alpha control too!


Obviously the effect is in the simplest terms multiplying the spec term (wherever it exists) by the pure ambient lighting value.

Sun spec = sun intensity rgb * spec map
Ambient spec = ambient intensity rgb * spec map


Same process, same sources of look up data, so makes sense to do it... I'll leave the way it's finally implemented to the expert


I'm not sure it'll look great on some objects though, but technically it should push realism of the final rendering.

If things don't look right it's probably a function of the input values.

It's looking good here though


Dave

 

[Mr Whippy]

Also just thinking about the locking of the sun spec and amb spec together... generally that works ok where we are dealing with the materials these shaders should be used for, but for other shaders (like cars/glossy ones) this is possibly not ideal.

However in those materials I suppose you would use the superior mipped envmap or a cube map instead


I guess we will have to see if we come across odd results then tweak the shading where necessary.






Just for now I'm interested Ruud, the diffuse appearance is defined by the incident light angle. So the sun straight down and perpendicular will make a surface appear brighter than at a steep parallel angle.
Does the specular thus change from near zero intensity perpendicular to almost 100% at a parallel angle?

Is the "specular =" value we define the specular at a certain angle? Ie, 45 deg?


Hmmm

Dave

 

[Ruud]

Just for now I'm interested Ruud, the diffuse appearance is defined by the incident light angle. So the sun straight down and perpendicular will make a surface appear brighter than at a steep parallel angle.
Does the specular thus change from near zero intensity perpendicular to almost 100% at a parallel angle?

Is the "specular =" value we define the specular at a certain angle? Ie, 45 deg?

Diffuse is not view dependent, specular is. The specular calculation is in lighting.cg:

float specularLight=specFactor*pow(saturate(dot(N,H)),shininess);
Where N=normal, H=halfvector (lightDirection+viewDirection averaged). Not sure if that is Blinn or something else off the top of my head.
See http://en.wikipedia.org/wiki/Blinn–Phong_shading_model , that looks like it (the key is dot(N,H)). As the vectors are normalized, the dot product is at most 1, then it uses the full Ks.

 

[Mr Whippy]

So in theory a material with shininess = 100 and specular = 1 1 1 should generate a specular spot on the the material of equal intensity to the sun diffuse value (kLux) at a 90deg incidence angle?

That is basically what I'm wanting to check... or do we need to go above specular = 1 values to achieve a reflected energy equal to the incoming energy?

I'll probably do some more testing here. My main concern was equalising the specular values across diffuse/ambient so a value of 1 always gives an expected intensity.

I appear to need ~ specular = 14 14 14 to get the sun specular spot equal in intensity to the sun Klux intensity in the sky map (using mouse pointer colour picker in Racer)

Obviously that is on a mirror like surface material. On a tracks tarmac it might be 10x less shiny (shininess = 10 not 100), so a value around 10x less for specular seems about right (1 - 1.5 ish)




The ambient specular works really nicely as amb intensity * spec map * 1 since it's ambiently diffused reflection it should generally be equally reflective from all angles and directions etc.


Just making sure we don't get ourselves into a mess, but I guess it's easily tweaked later if necessary.

And as said, for the majority of materials with old style specular they are generally matt type materials with a subtle gloss and the effect seems to work ok in those cases.


Thanks

Dave

 

[Mr Whippy]

Hi Ruud,

I'm just looking at this again.

Some shaders in 0.9 RC3 seem to have the ambient specular on them, but others don't... ie, speca_f dynamic shader doesn't seem to have it applied... while the standard dynamic shader does.


I'm rethinking the ambient specular based on the last query basically. This effect should be like fresnel reflections really.

Diffuse light is basically diffused reflection so there is no real impact on the apparent intensity of the material based on the lights incident angle.
Specular light however is reflected light, not diffused, so we need to manage it better.

Having the ambient specular makes sense, but having it reflect that ambient specular when we are looking straight at a material seems not ideal, and indeed for car materials (interior plastics) it starts to essentially just add more ambient to the material and make it look like it's overly bright, rather than the intended effect which is to add subtle reflection of the ambient environment... using the ambient light value to approximate that is useful, but technically we should be adding it with a fresnel type property so edges pick up reflections more than the facing surfaces.

Indeed, does the current 'specular' for diffuse light use any kind of falloff or anything? Does the specular spot intensity change depending on viewing angle/light incidence angle? Or is it just fixed?



This also goes back to the issue that ambient/diffuse sun values are not proportional to the rendered sky values much of the time, so we often need to use values above 1 for specular to get a specular that matches the envmap reflections.

The problem then is that specular impacts both ambient and diffuse specular intensities now, but these are probably best being separate.


Is it hard to do something like this?

specular=1 1 1 1
scale=2
fresnel
{
}

So we define specular for diffuse, a scale value scales the ambient diffuse, and then fresnel properties can impact the specular responses from the material?

Obviously this seems quite complex and is probably better for only a few material shader types?!

I'm just trying to do a nice looking car interior shader set right now and everything I use looks a bit pants really. The ambient specular fails to do the job I want it to, and I'm still finding the power of mipped reflections and fresnel better at making plastics and leathers etc look more correct (despite having nasty mip edges and not occluding the sun in the shaded areas etc)


It'd be nice to get Racer having something half decent for faking materials in a realistic way.
As soon as items are in the shade in Racer they just go flat and dull looking so we need to get some life into them!
We are certainly moving in the right direction but without consistency across materials I'm still struggling to make plastic look like plastic with plastic kinda values.




Pie in the sky for now but it'd be nice to move torwards this kinda system eventually.

http://seblagarde.wordpress.com/2011/08/17/feeding-a-physical-based-lighting-mode/


So we just define materials with almost realistic properties and light things with actual environment maps (HDR I guess)

Seems it's the way forward in all the new renderers etc... using static or dynamic LUT's for lighting/reflections around the place....


Hmmmmmm

Cheers

Dave

 

[Mr Whippy]

Hmmm, Ruud...

ambient=(Ka+Ks)*lightAmbient*normalInfluence;
//ambient=Ka*lightAmbient*normalInfluence;

That is in lighting.cg


Is that basically the total of the implement for the ambient specular method?



It seems a really bad way to manage things unless we can scale the Ks per material.

Ie, a dark plastic that is darker than leather in the sun, in ambient the leather with low specular stays quite dark but the plastic stays bright... it just looks bad.

The technique works ok for tracks mostly, but as soon as you get to a car interior then it looks really bad quite often!



I've managed to add a scale variable to a shader and it works nicely, but only when that shader isn't using the new ambient term above.

Is there any way to make it so we either put the term per shader and offer a scale per shader, or something like that? I don't think there is an ideal value for the effect.

Erring away from fresnel and all that stuff for specular, a scale gets us at least a bit further forward but the current implement of ambient specular the way it is isn't good.

I do think ambient specular is a good thing and it adds a lot when balanced properly, but a one-size fits all doesn't work well and is just as much a compromise as not having it at all.





So to sum up, any way to make a scale value for all materials? Perhaps a new global called specScale?

So we get ambient=(Ka+Ks*specScale)*lightAmbient*normalInfluence;

Where specScale is set per shader but is globally defaulted to a value like 0.25 or something?





Also, the normal influence value is quite useful to adjust. It'd be nice to be able to scale this value depending on the difference in values between sun ambient and sun diffuse, so as the sun gets very bright at midday we get stronger contrast, and then on overcast days the contrast will be lower.

Hmmmm

Dave

 

[Mr Whippy]

Just more thoughts on this.

The current ambient specular runs purely off the term for specular this way, rather than being implemented as a function of any spec map present, so it doesn't really run very well in that regard either.

I think ambient specular if you want to use it ideally needs to be done per shader, multiplied by a spec map if present, and then multiplied by the scale factor so the author can tweak the impact of the effect.

A global system just doesn't seem to do anything any favours so is probably best left out imo!
Do it per material and control the effect how you want it


Cheers

Dave

 

[Mr Whippy]

Just another thought on this.

It's sensible to implement something globally to get realism, but since all our materials are combinations of different lighting and reflecting techniques there is no balance possible except what authors do by eye!

Unless of course we move to the suggested new envmap and specular solution which appears to be vogue in current rendering techniques in racing type games which then makes life quite a lot easier for balancing things out.

I'm sure the energy conservation technique can then be implemented a little bit more effectively with the 'scale' for ambient specular being powered through the conservation system too.
Ie, the shinier we set a material the lower the ambient specular moves, the more matte the material, the higher the ambient specular influence gets.


We should either give control to artists to balance effects by eye, or provide a more naturally balanced system and automate things for artists.

But the current automation of specular ambient in an unbalanced system without user control AND applied globally is just not great basically. I've turned it off here because it just makes half the stuff look wrong as much as it makes the other half of stuff look a bit better!

Cheers

Dave