Smooth edged texCUBElod

[Stereo]

This is something Dave's asked for, I finally got enough math together in my head to implement it. A texCUBElod that doesn't have sharp edges. What it basically is, is a drop-in function to be called in place of texCUBElod. In turn, it calls texCUBElod in a couple places, and blends the samples smoothly.

The edge in the top picture's reflection is, of course, what I am referring to.

Well, to be honest, this is cheating slightly - I didn't yet implement the corner case, so the corners of the map are still visible if there are sharp transitions. But once I've added that, I'll post it up.


EDIT: There, it's completed. See lower post for download link, since pasting the code didn't work.

Stick this into renderer\common\utils.cg, probably, since that's included in most shaders already.
Then just replace texCUBElod() with texCUBElod_soft() in the fragment (_f.cg) shader.

#include "../common/utils.cg"

(just stick this with the other #includes if it's missing)
Since normal texCUBE() doesn't go to mipmaps within view there's no point implementing it for that one.

 

[Alex Forbin]

This is something Dave's asked for, I finally got enough math together in my head to implement it. A texCUBElod that doesn't have sharp edges. What it basically is, is a drop-in function to be called in place of texCUBElod. In turn, it calls texCUBElod in a couple places, and blends the samples smoothly.

Interesting, did he say what he wanted it for?

Alex Forbin

 

[Some1]

This blending could be useful for the regular envmap as well, since there seem to be occasionally visible seams.

 

[Stereo]

Interesting, did he say what he wanted it for?

Anything where mipmap_bias is used - soft reflections look funny when there are hard edges on it. I believe he also wanted to replace the specular spot with one based on the reflection map, since then it'd be more accurate to multiple light sources.

 

[Mr Whippy]

Early days but the thing that really hurt my 'basic' image based lighting shader was the pesky hard edges which didn't 'diffuse'
It's all faked but anything that is meant to be diffuse with a sharp edge doesn't work so well.

With a subtle blur and a mask for example this would make lovely interiors that respond to street lights etc... Still no shadows but image based lighting based off a soft envmap seems very sensible for cars on night tracks etc...

Lots of experimenting to come!

Thanks for posting Stereo, will be a good default addition to anything using a mipped envmap. Blurring would make it amazing if that is what you had planned!

Dave

 

[Stereo]

Blurring would make it amazing if that is what you had planned!

Dave

I was hoping to come up with something, most of the techniques I can find online would do a per-pixel blur to the FBO the envmap is sitting in. I'm trying to do it from the fragment shader, since that's what I have to work with, so I need to think things through.

I should also mention that I haven't tested the performance of this yet, it's reasonable that it'd be a bit slower than just plain texCUBE lookups.

As posted in the Picture thread, I've tried substituting a mipmap level 8 lookup for lightAmbient and am, in general, happy with the results, although it should ideally be even farther blurred. It gives the right general character of illumination to the car appropriate to the scene, it's just a bit too sharp, more like specular than ambient. It also really highlights flaws in the general lighting of tracks - some look ok, on others the sky heavily tints things blue, where a 'clear' sky has diffuse light that's much too dim. And an envmap without the sunspot would divide the load between ambient (no shadows) and diffuse (casts shadows) better.

Then again, maybe it's just habit that makes me expect the lack of blue? Obviously not a direct comparison, here, but:

(inset with lightAmbient for ambient, main picture uses the envMap mipmap 8)

EDIT: better representation picture, 'reload shaders' somehow broke shadows the first time. For what it's worth, no measurable difference in framerate.

 

[Mr Whippy]

I guess a big problem with the compare in the last picture is setting an appropriate track/TOD value for ambient lighting.

In theory despite the sky being pretty slit like against the buildings, it's intensity is pretty high and so the overall colouring might need to be bluer.

Also we see many tracks with sky textures that contain blue, despite that being wrong if the values are not balanced right, and so once again we'd get too much ambient blue.

Lastly, we have no white balance in play, so a cool blue lighting which our eyes naturally compensate for (say a winters day with occluded sun light/clouds) looks fairly white.

When you sit indoors in winter it feels normal inside and blue outside. Swap around and it looks very orange inside and normal outside. I guess we don't get that with Racer which is making things difficult.
Maybe we need a white balance AND exposure value hehe...



I like where you are going with this though. I think it makes total sense and have kinda wanted Ruud to go this route for a while.

If you turn sunny=0 the envmap sun disappears, and you get just the HDR sky and surroundings.
Blur those down varying amounts and you get a perfect response alongside fresnel etc. We use mips currently, but with blurring too you'd get a great look.

Then replace the sun-spot with the specular spot. This works quickly and nicely everywhere, and keeps energy consistent from shiny reflective materials to slightly blurry materials through to matte materials.


If we can make specular also adhere to the fresnel properties too, AND scale the envmap blurring/mipping with the shininess variable, then we can start to get some really nice looking car materials.

Ie, glossy paint is sharp envmap and a sharp specular blob. Make it blurry and the sharp specular blob softens, the envmap blurs a bit too, and we basically get a soft looking material like rubber.

Throw in AO maps to multiply down the 'ambient' component and scale those up the 'softer' the reflections get and eventually you end up with an almost flat value for the material response, with an AO map.
Ultimately this is a straight sun ambient value with AO map, but very few materials really are totally like that.

Ie, angle the camera at the car and catch the sun down the side, and the tyres get a nice specular reflection. Great right now.
Spin the camera around and look against the blue sky but we get no blue hue reflecting off the tyre.


This is kinda where the 'ambient specular' thing we tried might also be useful too. Right now tracks look lifeless under cloudy sky settings, what we really need is fresnel applying to specular, and specular being present both in the diffuse pass and the ambient pass.

Not sure on the ideal best implement there really. It makes sense for most track materials that are generally not reflective to the point you can see details in reflections.

Ie, with fresnel impacting both diffuse specular and ambient specular, the balancing of values is probably just a function of specular and shininess values. Ie, if the shiny value is higher then the ambient specular is probably more intense, but if the shiny value is lower then it's less intense.
Maybe even those intensities are balanced with the fresnel properties?

Again, all this can probably be balanced so energy is conserved and appearance is consistent and somewhat automatic.

Ie, should we say a material is just shiny or dull, or somewhere in between. Inverse diffuse = specular response (with a scale value for contrast only?!)
Gloss map then determines the reflections. If we have envmap on then we use blurred envmap/mip envmap. If not then we get an ambient specular response.




All interesting stuff. Maybe a bit over-kill. There really is no need to make it automatic if artists are good they can balance things themselves imo.

Just making specular adhere to fresnel and define a fresnel per material would immediately make things look much nicer imo.
Strip out the sun spot from the envmap, then the envmap can be used for glossy reflections.

Sun spot in the sky can then be rendered in as a post effect?!



All just thoughts, but nice to see you are feeling the same way Stereo. Just look at a car interior with nice IBL for ambient. Throw in AO maps (would be nice on a 2nd UV channel to make life easy!), and even a mask for the effect, and suddenly interiors will be looking seriously realistic as you drive under bridges, under lights, etc etc...

Dave

 

[Mr Whippy]

Hmmm, trying to make this work but hitting problems.

Just copying and pasting the code above into utils.cg and not trying to use it is giving me a huge failure before a track even tries to load.

I think it's formatting related.

Can you post your utils.cg file please Stereo?


Thanks

Dave

 

[Stereo]

Oh yeah, I switched to the full-featured input thingy to get the video syntax and it may have messed with the pasting. Here's the whole file, replaces racer/data/renderer/common/utils.cg.

Here's a test shader that uses it for ambient light - try driving into a tunnel or something. Replaces dyn_standard_bump_reflect_f.cg.

 

[Mr Whippy]

OK, this is looking really nice. Those sharp vertical lines in the reflections at high mip levels are gone!

Just added it to a car and it's nice now, as the TOD changes you get what is essentially automatic sun.ambient colouring on the car, so it does automatically look more natural. Not sure if the sun influence is still gonna be a problem because you add it again in the diffuse pass... hmmm...

On my track where I have a sunset image as the sky and clouds=20 or so (to get kLux values about right), this works perfectly however... my subtle track reflections look great, deep blue road heading away from the sun, a nice orangey yellow into the sun just like real life.


A good use for this shader would be adding it to a cube and a sphere and cone, or some other shapes, use the envmap shader for ambient but have it set up as matte object, and then also add the same objects with a matte material using classical sun amb/diff values.

Then tweak the TOD values in Racer to get them looking the same, so basically balancing the visual look to what is technically accurate from the sky envmap (kinda, just that pesky sun hehe)


In theory for tracks hard-coded values maybe make sense as nothing moves, but for the car that moves THROUGH the scene then dynamic ambient via this technique makes a lot of sense... just that balancing process needs to be done really to get coherent appearance or as close to that as possible.



Just wondering now if it's possible to sample out the sun spot somehow from the envmap mipmap. Not sure how you'd do it. Maybe there is some bodgey maths?!

Ideally we should scrap the sun being rendered in the envmap and have it added over the top via some other process... reflections of it can be done via good old specular then, and generally when you 'see' the sun it can probably be a post effect bloom/glare/flare type effect?!


Hmmmm

Dave

 

[Stereo]

A good use for this shader would be adding it to a cube and a sphere and cone, or some other shapes, use the envmap shader for ambient but have it set up as matte object, and then also add the same objects with a matte material using classical sun amb/diff values.

Then tweak the TOD values in Racer to get them looking the same, so basically balancing the visual look to what is technically accurate from the sky envmap (kinda, just that pesky sun hehe)


In theory for tracks hard-coded values maybe make sense as nothing moves, but for the car that moves THROUGH the scene then dynamic ambient via this technique makes a lot of sense... just that balancing process needs to be done really to get coherent appearance or as close to that as possible.

That's about what I'd figured, too - only use dynamic ambient light in the dyn_ shaders, and tweak the track materials and TOD so everything else behaves approximately right on its own. (shaders that use the envmap obviously benefit from this by nature of it improving cubemap lookups)

Ambient light depends on clouds= to a large degree but isn't affected by the value of it - so you currently can't change clouds= without messing with the match between sky and ambient light. Probably best to leave it at a fixed number for ideal results.

 

[Mr Whippy]

Hmmm been playing with this today.

If you set up a basic track with a chrome ball, matte white balls, mid grey and all that, and then have a few matte materials using the ambient term from the envmap above, you seem to need to pump the sun ambient values quite high, and quite blue, to get the balls matching.

This looks ok, and swapping from sunny = 1 > 0 and values in between seems to give the correct kinds of responses on both balls.

So technically this is accurate.

The main question mark now is the intensity of the sun intensity value which drives the sky intensity, and thus the intensity of the ambient values we set.


Balancing is still hard I feel. I'd really love to be able to load in a HDR image sky that was used properly (real values), and then we could start to really tweak values and KNOW that at the current EV camera exposure the sky *IS* correct, and then we can determine how the chrome ball should look, and then the matte white ball, and start to find the right values for sun ambient/diffuse to light the scene objects with the conventional lighting system!


I think this is where we really need to have a system where we define gloss maps, and spec maps are instead 1-kd (with a scale variable perhaps to adjust the 'curve' or contrast of the resulting 1-kd output).
We then really need to somehow tie the sun ambient/diffuse values to what is happening in the sky, so our specular responding items match in with the chrome reflection type materials.


Here are three pics:



First pic is using envmap ambient for the tarmac I'm parked on and the left hand ball.
The middle tarmac is using the sun ambient I had set all along (from an IES sky model in 3DS Max), as is the right hand ball. Oh, they look dramatically wrong!

The next two pictures are me playing with sun ambient values to balance the right hand ball with the left hand ball, thus bringing them into the same visual look. Interestingly if I set sunny = 0 then things still look natural, both spheres looking about the same intensity etc...


I'm not sure what is right or wrong now. I'm starting to question my IES values and also if Racer is using sun ambient values properly or not.

I think it also highlights the need for a white balance system, since the blue tint is over-powering.
The sky rendering using mie/rayleigh is nice, but if it's not white balanced then are we fundamentally setting bad values because we are using our brain to auto-white balance when setting values, thus building in 'whiteness' where whiteness doesn't exist?


In theory the white balance can be automatic? If we know the mathematical properties of scene values then the white balance point must be known?


I'm generally still not happy with Racer's visuals. Mainly tracks looking so flat and dull when driving out of the sun but still on a bright sunny day.
Yes on an overcast day things look flat, but you still have a myriad of specular response...

I think best thing to do would be to dump sun spot from envmap rendering (but keep it in the sky for direct visual purposes), use true specular for all materials again.
Then apply the fresnel properties to all materials and make specular adhere to them too, so specular and reflection maps both are impacted by fresnel properties!

Then suddenly ambient specular will be much easier to balance, and normal specular will also look a great deal better and more natural too imo!


Hmmmm

Dave

 

[Stereo]

Balancing is still hard I feel. I'd really love to be able to load in a HDR image sky that was used properly (real values), and then we could start to really tweak values and KNOW that at the current EV camera exposure the sky *IS* correct, and then we can determine how the chrome ball should look, and then the matte white ball, and start to find the right values for sun ambient/diffuse to light the scene objects with the conventional lighting system!

Hmm, I don't know if Racer supports any actual HDR formats, but scaling it into LDR and using emission to set the brightest point (ambient/diffuse to 0 to avoid contamination) would probably let the skybox work fine with vf_standard and mode=linear.

 

[Mr Whippy]

Just playing more, could really do with a white balance transform in HDR space before tone mapping.

Would be nice even if just based in shader for now to scale temp up and down to see the effect... I wonder if Racer is currently running some default white balance like 5400k or so built into the tone mapping for HDR... Hmmm... Those balance values of rgb we see, hmmm...

Dave

 

[Stereo]

Currently Racer's using code from
http://imdoingitwrong.wordpress.com/2010/08/19/why-reinhard-desaturates-my-blacks-3/
In particular, it's using c*(1+c/L_white^2)/(1+c), which is sort-of the final code there, which is then gamma corrected (gamma=1/2.2) so that monitors will display it right.

As-is, this is effectively leaving the white balance alone, so as far as I know it's just gonna reflect the color temp of your monitor. Something that's (100,100,100) and is scaled down to (1,1,1) by exposure will become a 255,255,255 pixel on the screen.

As you can see on for example this post there are various definitions of luminance in rgb space, I'm gonna try to stick a few in Racer's tonemapper and see what happens.

edit: here we go.

top: Racer's default.
bottom: treating luminance as Red=0.212395, Green=0.701049, Blue=0.086556
(on this track because it's the only one RC5 can reload shaders on, so I guarantee a direct comparison)

It's obviously up to subjective feelings in something like this:

(note: car & track using default ambient light shaders, so setting balance not necessarily accurate but definitely consistent)
The bottom one (using luminance as above) feels less washed-out.

Playing with a manual white balance adjustment, it becomes obvious just how blue Carlswood is in the shade relative to the 'white' sun... even with what I'd say is a pretty severe change in balance (from 5000K to at least 20000K) the white car still shows up bluish in ambient light.

Here I edited Carlswood's ambient TOD curve to match, otherwise the shadows look almost as orange as the sunlit parts of the track.

I'm not sure how to go about automatic white balance, or if it'd be better to just make that a track setting using the tonemapping adjustment cube.

 

[Mr Whippy]

I wonder if the mie/ray sky is rendered at a preset temp or something... maybe causing balancing issues.
On my SLR 2000k each side of 5500k or so is a heavy orange or blue tint.

IMO the ray/mie sky looks different to the old days, I swear the sun is bluer and bigger... Maybe its just me though

The effect of balancing the sky appearance with the track diffuse/ambient sun settings IS an improvement though for sure!

Dave

 

[Stereo]

The effect of balancing the sky appearance with the track diffuse/ambient sun settings IS an improvement though for sure!

Dave

Yeah, there's a lot of freedom in the TOD setup but without a way to see what it's doing, it's not much practical use. With the skybox contributing a lot to the illumination of the track (in the form of clouds) it's just plain not possible for the game to work out what numbers fit together on its own.

Static objects are one thing, it would be useful to put together a test-car that's just a couple spheres/cubes that can be loaded with any track, drive somewhere and see how things fit together. Really 4 spheres would be useful - a couple regular matte-white with track/envmap for ambient, and then a couple with purely ambient lighting so sampling klux values off the objects for comparison is possible.

I haven't yet worked my way into the bits and bobs of the sky rayleigh/mie, what I do have to offer is an alternate HDR tonemapping function:

// luminance calculation, easy to vary
float Luminance1(float3 color)
{
	return 0.212395*color.r+0.701049*color.g+0.114350*color.b;
	//return 0.33*color.r+0.34*color.g+0.33*color.b;
}

float3 ToneMapHDR_AltModifiedReinhard(float3 color,float exposure)
// Modified Reinhard from http://imdoingitwrong.wordpress.com/2010/08/19/why-reinhard-desaturates-my-blacks-3/
{
  float3 c;
  //const float L_white=2.0;
  const float L_whiteSquared=2.0*2.0;

  // Night time gives blue shift
  //color.rgb=NightDesaturate(color.rgb);

  // Filmic tonemapping
  c=color.rgb;
  c.rgb*=exposure;

  // Reinhard tonemapping
  //c=c/(1+c);
  // Modified Reinhard
  float l=Luminance1(c);
  c=c*(1.0+l/L_whiteSquared)/(1.0+l);

  // Gamma correction so that it comes out linearly to your eye
  const float gamma=1.0/2.2;
  c=pow(c,gamma);
  return float4(c,1);
}

Stick this function somewhere above the bottom of renderer\common\hdr.cg, change ToneMapHDR to call it (instead of _ModifiedReinhard) and hopefully get slightly less desaturation, if I read the blogpost right.
I didn't end up playing with the luminance calculations much because it didn't make a whole lot of difference. The point is more about curving all 3 channels based on the same amount instead of individually.

 

[Mr Whippy]

Hmmm, feeling like this is just altering the gamma of each channel, or something along those lines.

Also noticed that the gamma tweak in constants.cg isn't applied to half the tone-mapping items, so it's not used.

I think the tone mapping per se is kinda not the issue here, it's the colour temperature... changing these values doesn't look like a colour temp adjusted image...

I guess it's part and parcel of the same mechanism but needs to be dealt with in a different way to these coefficients which seem to be balancing for correction for the human eye response, display responses etc.




This envmap blurring is interesting to assess the colourings too. The sun generally comes out as very blue in the envmap reflections, but this is contrary to what we expect.

Or, is it? Is this blue tint merely the result of a blue-tinted colour balance?

If we white balance the sun to be white or a very slight yellow, will the rest of the scene balance better... with these pale blues for ambient becoming deeper and more subtle?

Right now with 'real' values Racer is looking like I'm viewing it a bit through blue tinted glass... but it generally looks really nice from a values point of view.



It's interesting to also see what happens when you add clouds. My spheres that matched nicely with a clear blue sky now don't match, the envmap ambient sphere now looking whiter, and the ambient/diffuse value lit sphere looking blue still... obviously.

This is another reason why, imo, we just need to analyse the envmap (perhaps have a single track location we define where a small envmap is rendered at load of track, or as TOD changes), and Racer auto-renders a small sunny=0 sky, checks intensity, uses that for sun ambient value, and then renders another one with sunny=1, checks peak intensity for the sun diffuse value.

We can also then use that small static envmap for most track reflections (like road surfaces etc), rather than our cars dynamic one.

Technically this is all relatively do-able in an automatic fashion in my view.


Well, if we abandon TOD then suddenly it doesn't matter as the author can set it all up from the start with fixed conditions in mind.


Also generally would be nice to have those extra envmaps we've had given to be HDR not limited to LDR... for a lot of purposes they don't even need to be high res... just small static look ups for reflective stuff.



But any way, back to the matter at hand. This stuff is powerful and cool. It seems to work generally for a static track where you can get really nicely balanced visuals and reflections and everything just coming together really nicely.

Downsides, white balance being static really hurts it. And I wonder generally if trying to tune our values to compensate is even accurate? Ie, artificially setting wrong colours seems broken when the sky itself is rendering with the right colours. It's never gonna look right with the mie/ray sky (probably why Racer often looks best with a fully sky image because it's white balanced more in line with what we expect!)


Hmmmm

Dave

 

[Mr Whippy]

Also, just thinking while playing more.

We need to be careful because my sphere illuminated ambiently with the envmap works ok as it's a sphere, but a flat surface won't pick up as much light from the bounced light from the surroundings.

Ie, right now the envmap is picking up the bright floor and asphalt surroundings so that works nicely on the sphere.

But the track which is just sat on the floor will pick up just the ambient light from the sky dome, NOT the ambient light from the sky dome and reflected from itself hehe.


I'm not sure what that means really. I'm getting a bit confused and running in circles still with intensities and colourings.

Gonna try get some kLux values off HDR imagery, and also use that HDR > LDR (clouds) imagery, then tweak the output value of the sky till it's kLux values match.

THEN, I can start to illuminate a test objects via the ambient envmap method and the sun ambient/diffuse values... thinking a box is more sensible here so I can have a side facing up, down, and one at the sun, one away etc...


I don't think we'll ever get things perfect because of the way we render (no raytracing)... but we can at least get a better idea of balanced values perhaps...



Hmmm, alternatively I could set the spheres up on a black area to remove any interference from the ground, so the only ambient light influence will be the sky... hmmm... that might technically be bad though as the black will reduce the sky ambient influence which is going too far the other way then?!
Maybe I need to make sure I don't mip down too far so I can see the distinct edge and can split the amounts contributed from each bit more easily.


Hrmmm... going in circles indeed.

Time to go read up on getting real values out of HDR images! It must be possible to work out a pixels real intensity value in lux... hmmmm...

Dave

 

[Stereo]

This envmap blurring is interesting to assess the colourings too. The sun generally comes out as very blue in the envmap reflections, but this is contrary to what we expect.

Or, is it? Is this blue tint merely the result of a blue-tinted colour balance?

I was playing around with TOD curves some more, and found that the sun diffuse/ambient curves have a pretty strong effect on the skybox's clouds - which seems to introduce a fair amount of blue, on Carlswood at least.

Setting sunny=0 doesn't change the colour of the sky at all (except at the actual sun-spot), changing the sun diffuse TOD curve does? It's odd.

Sampling from just above the roof (where there's apparently a gap in the clouds), clouds=1 on the right, sun.diffuse.green = ~0 on the bottom.
Top left
Top right
Bottom left
Bottom right

It's tricky to fit everything together when the parts all interact like this. The way sun.diffuse lights up the clouds seems wrong, but not in a way I can clearly define yet. I don't think Carlswood's cloudmap is an accurate depiction of clouds, either. In fact it has its own sunspot somewhere (how did this happen?) I suppose it'd be best to start over on the cloudmap, draw something with more resemblance to what sky.cg is actually expecting, and tune it from there.