Secondary static reflections

[Stereo]

Okay basically the intention is to have a second map that shows bits of the car which aren't being shown on the regular UV map.

First of all.
Blender exports cubemaps in a different order.
http://imgur.com/i3DUn
So I produced this reference for how to translate to the 6 cubemap bits. (right face will have to wait til I generate one that's non-empty - I'd guess it's 180 too though)


Second. There's still a major issue - Racer cubemaps import 3 channels only, no transparency. Alpha is always 1.0. So what I'm going ahead with is not going to be a permanent solution - just working around this problem.

Anyway, ingame, using only the above map (except the _bw texture, which is black) the single reflection map looks like this:

Obviously this method can't do multiple internal reflections, so the window and frame are reflecting as plain grey.

You can also see another issue - the cubemap, or maybe its mipmaps, aren't being clamped right.

So. Next step: swizzle up the channels.

alpha->red, greyscale texture->green

edit the shader to use a second reflection map - this is a snippet from an edited dyn_standard_reflect_f.cg:

  // Reflection
#ifdef USE_CUBEMAP_MIPMAPS
  float3 envColor=texCUBElod(envMap,float4(IN.R,mipmap));
#else
  float3 envColor=texCUBE(envMap,IN.R);
#endif
  float  reflectiveNess=baseCol.a;
  float4 reflColor=texCUBE(reflMap,IN.R).gggr;

  // Lighting
  float3 ambient,diffuse,specular;
  LightingSun(Ke,Ka,Kd,Ks,shininess,lightDirection,lightColor,lightAmbient,IN.Position,N,eyePosW,
    ambient,diffuse,specular);
  float3 litColor=baseCol*diffuse+specular*reflectiveNess;
  float3 shadowColor=baseCol*(ambient+Ke);

  envColor.rgb=envColor.rgb*(1.0-reflColor.a)+reflColor.a*reflColor.rgb*(diffuse+ambient);

1) reflColor = (...).gggr is swizzling the 2 channels back into the expected 4, to make it easier to update later.
2) envColor is updated to use reflColor's alpha channel.

So, after also adding a TEXUNIT that loads reflMap, the car.shd is also updated:

shader_chromemirror~shader_chrome
{
  ;reflect=0.0
  ;ambient=1.0 0.0 0.0
  layer1
  {
    map=cubemaps/mirror_rg*.tga
  }
  layer2
  {
    map=$trackenvmap
  }
  fragment_shader
  {
    file=dyn_stereo_reflect_f.cg
  }
}

And, finally, the static reflection map works!

Since the mirror's just using a regular reflection shader, not the mirror texture, the side of the car shows up there, too.

Okay so I'm cheating a little (which is why the car's parked diagonally - and also why I've probably got the cubemap rotated 90 degrees). The reflection vector's operating in the wrong coordinate space, so when you turn everything goes wrong. But this is a start. To fix that, I'm gonna have to hack up the vertex shader too, so it passes an object-space reflection vector.

 

[Mr Whippy]

Heavy duty on the actual reflecting mirror hack/bodge/fix!

I guess if you move the head around though, you see more or less of the side of the car in it, so the car mirror really starts to feel like it's showing a reflection from the mirror, not just a rendering backwards from the car?

Wouldn't it be better to somehow combine the flat back facing mirror texture and the 3d cube map, rather than using the envmap/cube map combo? I guess mirrors are quite small though so the lower res isn't so important... hmmm... not sure just thinking out loud as usual


Loving the technique on the chrome outside reflecting the car though. I guess you are wanting to be able to paint alpha into the fixed envmap so you can mask out the world and let the usual live env-map be there?
I wanted to do something similar for chromes etc inside the car, so you'd apply the usual live env-map and use a fixed envmap cubemap over the top which was the car interior.


Why not create the car without the mirror head (but leave the stalk), load Racer up in a nice location, move the camera to the mirrors exact location and then render out the cube-map... then you get the shading from the car really nice and from Racer itself, then just use that as the base for your reflection overlay map?

That is how I would create my interior mask... but you are totally right, it's important if referenced that all channels (rgba) can be imported and used as it gives a huge amount more flexibility for things like this. I'd want to see the same feature added/fixed as I'd want to use a similar technique for a similar end effect too (but I'd probably be asking you to help write the shader still hehe )

Thanks

Dave

 

[johnthelemon]

Would it work on glass too? For example, the piece of glass that covers the dials. If given a typical glass shader, it reflects everything, including the road.

 

[Stereo]

Heavy duty on the actual reflecting mirror hack/bodge/fix!

I guess if you move the head around though, you see more or less of the side of the car in it, so the car mirror really starts to feel like it's showing a reflection from the mirror, not just a rendering backwards from the car?

Yeah, I tweaked it so that it reflects 'behind the car' when the camera's inside the driver's head, just geometrically. It cuts down on # materials on the car since it's just using the same chrome as all the rest of the shiny parts of the car. On a more final version I might change it to use the mirror texture. The side mirrors are never very large onscreen though so it might not be necessary.
The one downside is it requires live envmapping, which is way more resource intensive than mirrors (disproportionately so: I don't get any framerate change when I turn off mirrors, I get a 50% increase with static envmaps. Probably because the envmaps total ~2048x3072, the mirror is only 60x200 or something)

The mirror was just the easiest to get the basics down, being a near-spherical chromed object, as you can see I had to do a lot of rotating to match up all the faces from Blender's output. This kind of thing would be good for interiors and stuff of course - well, after it has alpha.

Do rendered out cube-maps include the car itself? I'd figured they don't, maybe it'd be good to set up a "blank environment" that just gives a generic sky/lighting, over pavement, and you can drop your car model in as a prop to get the basic sky-horizon-reflection in place on the car..

I think I have a handle on the math necessary to get this map rotating with the car, should see an update when I get it implemented and debugged.



It shouldn't be a problem per se to use it on transparent materials, just an added degree of complexity.

The simplest version would be a mask saying what directions not to reflect, or rather, scale everything by the appropriate factor - since that would just be greyscale it'd work with the current version's rgb cubemaps. That'd work fine for anything that's proportionally dark - car interiors, for example, are darker than the sky to the point it wouldn't look hugely out of place to make the interior not reflect at all.

For obvious reasons this can't tell which parts of the car are shadowed, I just use the lighting on the object the reflection map is on to determine the map's brightness - as if it was lit by ambient+diffuse normally (ignoring shadows because

 

[Stereo]

Hit the message limit so I have to double post...

I have it completely working now.

This is a better idea of the quality of the mirror, hopefully.

dyn_stereo_reflect_v.cg (need a better name for this)

//
//  reflection on 2 maps - dynamic models (modelMatrix)
// Stereo, June 14 '12
//

#include "atmosphere.cg"
#include "lighting.cg"

// 'inversion' for a transform matrix
float4x4 inverttr(float4x4 m) {
	float4x4 mout = m;
	float3x3 R = transpose(float3x3(mout)); // gets the rotation, on its own. transpose a 3x3
    float3 t = m._m03_m13_m23; // gets the translation, on its own
    mout._m03_m13_m23 = -mul(R,t); // inverted transpose
    mout._m00_m01_m02 = R._m00_m01_m02; // and put the transposed rotation
    mout._m10_m11_m12 = R._m10_m11_m12;
    mout._m20_m21_m22 = R._m20_m21_m22;
    return(mout);
}



// App to vertex
struct a2v
{
  float4 Position : POSITION;
  float3 normal   : NORMAL;
  float2 tc0      : TEXCOORD0;
};

// Vertex to pixel shader structure
struct v2p
{
  float4 Position       : POSITION;
  float4 tc0            : TEXCOORD0; // tc0.zw is R2.xy
  float4 Direction      : TEXCOORD1; // Direction.w is R2.z
  float4 RayleighColor  : TEXCOORD2;
  float4 MieColor       : TEXCOORD3;
  float4 FPosition      : TEXCOORD4;    // For fragment shader
  float3 normal         : TEXCOORD5;
  // Interpolate fresnel in PS
  float3 I              : TEXCOORD6;
  //float  fresnel        : TEXCOORD6;
  float3 R              : TEXCOORD7;    // Reflection vector
  float  extinction     : COLOR;        // 0..1
};

void main(
  // a2v
  in  a2v IN,
  out v2p OUT,

  // Constants
  uniform float4x4 modelViewProj,
  uniform float4x4 modelMatrix,
  uniform float3   lightDirection,
  uniform float3   eyePosW,
  uniform float3   lightColor,
  // Atmosphere
  uniform float    sunIntensity,
  uniform float    atmosRayleigh,
  uniform float    atmosMie,
  uniform float    extinctionFactor
)
{
  // Transform to clipspace
  OUT.Position=mul(modelViewProj,IN.Position);
  float3 posW=mul(modelMatrix,IN.Position).xyz;

  // Atmosphere
  float3 Direction;
  CalculateAtmosphere(atmosRayleigh,atmosMie,sunIntensity,eyePosW,IN.Position,lightDirection,
    OUT.RayleighColor,OUT.MieColor,Direction);
    OUT.Direction.xyz = Direction;

  // Extinction factor of regular color
  OUT.extinction=CalculateAtmosphereExtinction(OUT.Position,atmosRayleigh,atmosMie,extinctionFactor);

  float3 N=mul((float3x3)modelMatrix,IN.normal);

  // Pass texcoords and such
  OUT.tc0.xy=IN.tc0;
  //OUT.normal=IN.normal; //mul((float3x3)modelMatrix, IN.normal);
  OUT.normal=N;
  OUT.FPosition=mul(modelMatrix,IN.Position);

  // Reflection
  float3 I=posW-eyePosW;
  float3 R;
  R=reflect(I,N);
  OUT.R=R;
  // Obj. Reflection
  float3 OI = IN.Position-mul(inverttr(modelMatrix),float4(eyePosW.x,eyePosW.y,eyePosW.z,1)).xyz;
  float3 R2;
  R2 = reflect(OI,IN.normal);
  OUT.tc0.zw = R2.xy;
  OUT.Direction.w = R2.z;

/*
  // Fresnel
  const float fresnelBias=0; //0.2;
  const float fresnelScale=1.0; //0.9
  const float fresnelPower=2.0; //4.0;
  I=normalize(I);
  float dotIN=dot(I,N);
  // Avoid sharp edges
//  dotIN=min(dotIN,0.2);
//  dotIN=max(dotIN,-0.2);
  OUT.fresnel=fresnelBias+fresnelScale*pow(1.0+dotIN,fresnelPower);
//OUT.fresnel=pow(1+dotIN,fresnelPower);
//OUT.fresnel=N; //dotIN;
*/

  // Fresnel in pixel shader (pixely effects when done in VS)
  OUT.I=I;

/*
  // Reflection (dynamic)
  float3 posW=mul(modelMatrix,IN.Position).xyz;
  float3 I=posW-eyePosW;
  float3 Nwc=mul((float3x3)modelMatrix,IN.normal);
  R=reflect(I,Nwc);
*/
}

dyn_stereo_reflect_f.cg

//
// Standard with reflection cubemap
//

#include "atmosphere.cg"
#include "lighting.cg"
#include "fresnel.cg"
#include "shadowmapping.cg"

// Vertex to pixel shader structure
struct v2p
{
  float  extinction     : COLOR;
  float4 tc0            : TEXCOORD0;
  float4 Direction      : TEXCOORD1;
  float4 RayleighColor  : TEXCOORD2;
  float4 MieColor       : TEXCOORD3;
  float4 Position       : TEXCOORD4;    // For fragment shader
  float3 normal         : TEXCOORD5;
  // Fresnel in pixel shader
  float3 I              : TEXCOORD6;
  //float  fresnel        : TEXCOORD6;
  float3 R              : TEXCOORD7;    // Reflection vector
};

void main(
  // In
  in v2p IN,

  // Out
  out float4 out0 : COLOR0,
#ifdef CSM_MRT
  out float4 out1 : COLOR1,
#endif

  // Constants
  uniform sampler2D   baseMap : TEXUNIT0,
  uniform samplerCUBE reflMap  : TEXUNIT1,
  uniform samplerCUBE envMap   : TEXUNIT2,
  uniform float3    lightDirection,
  uniform float3    lightColor,
  uniform float3    lightAmbient,
  uniform float3    eyePosW,
  uniform float     atmosRayleigh,
  uniform float     atmosMie,
  uniform float3    Ke,
  uniform float3    Ka,
  uniform float3    Kd,
  uniform float3    Ks,
  uniform float     Kr,
  uniform float     shininess,
#ifdef USE_CUBEMAP_MIPMAPS
  uniform float     mipmap,       // Reflection mipmap (>0 = blurry)
#endif
  uniform float     sunny,
  uniform float     exposure,
#ifdef CSM
	uniform sampler2D    shadowArray  : TEXUNIT7,
  uniform float4x4     smTexMatArray[SM_MAX_SPLITS],
  uniform float smSplits,
#endif
  uniform float    fresnelBias,
  uniform float    fresnelScale,
  uniform float    fresnelPower
)
{
  float3 skyColor;

  float3 N=normalize(IN.normal);
  float3 R2 = float3(IN.tc0.z,IN.tc0.w,IN.Direction.w);

#ifdef CSM
  // Output shadowing
  float shadow = GetShadowFactor(IN.Position, IN.normal,shadowArray, smTexMatArray, smSplits, lightDirection)*sunny;
  //outShadow.rgb = N;
#else
  const float shadow=sunny;
#endif

  // Get sky gradient color
  skyColor.rgb=GetSkyColor(lightDirection,IN.Direction.xyz,IN.RayleighColor,IN.MieColor,atmosRayleigh,atmosMie,lightColor,lightAmbient);

  // Get base texture color
  float4 baseCol=tex2D(baseMap,IN.tc0.xy);

  // Reflection
#ifdef USE_CUBEMAP_MIPMAPS
  float3 envColor=texCUBElod(envMap,float4(IN.R,mipmap));
#else
  float3 envColor=texCUBE(envMap,IN.R);
#endif
  float  reflectiveNess=baseCol.a;
  float4 reflColor=texCUBE(reflMap,R2).gggr;

  // Lighting
  float3 ambient,diffuse,specular;
  LightingSun(Ke,Ka,Kd,Ks,shininess,lightDirection,lightColor,lightAmbient,IN.Position,N,eyePosW,
    ambient,diffuse,specular);
  float3 litColor=baseCol*diffuse+specular*reflectiveNess;
  float3 shadowColor=baseCol*(ambient+Ke);

  envColor.rgb=envColor.rgb*(1.0-reflColor.a)+reflColor.a*reflColor.rgb*(diffuse+ambient);
  //envColor.rgb=reflColor.rgb*40.0;
  //envColor.rgb = reflColor.aaa*1.0;

  // Add reflection with fresnel and the texture's alpha channel as a reflection factor
  float fresnel=Fresnel(fresnelBias,fresnelScale,fresnelPower,normalize(IN.I),normalize(N));
  shadowColor=FresnelMix(shadowColor,envColor.rgb,Kr*reflectiveNess,fresnel);

#ifdef CSM_MRT
  // Mix sky with texture color based on atmospheric influence
  out0.rgb=lerp(skyColor,shadowColor,IN.extinction);
  //outShadow.rgb=lerp(skyColor,litColor,IN.extinction);
  out1.rgb=litColor*IN.extinction;
  out1.a=shadow;

  // Blending
  out0.a=baseCol.a;
#else
  // Mix sky with texture color based on atmospheric influence
  float3 finalColor=lerp(skyColor,shadowColor+litColor*shadow,IN.extinction);
  // Need to clamp output - my nVidia GTX285 only handles colors upto 255, then makes them +Inf!
  finalColor=min(finalColor,255);
  out0.rgb=finalColor;
  // Blending
  out0.a=baseCol.a;
#endif
}

Using it in a shader just requires:
layer0: regular texture
layer1: red/green map - red channel is a mask - white = use this texture, black = use track envmap, green channel is greyscale 'this texture' that's actually displayed.
layer2: $trackenvmap

 

[Mr Whippy]

Wow that is amazing so far.

I run a 1024 env-map live all the time on my GTX275 @ 1920x1200. I've always run live if I can as it adds so much more imo to the whole visual appearance... I do slow it down to 1 side per frame though, so it's not super smooth but it allows ~ 100fps on a good track which means the update is about 20fps... not so bad


I guess the mirrors look fine like that with the envmap, I'd be more than happy with the added realism of seeing the car and slightly worse mirror reflections to be honest.

So if your head moves outwards towards the glass I assume you see more of the side of the car in the mirror? That is very cool!


Ah yes, good point on the envmap rendering, the car won't be there

Solution, add the car into the track itself, then move the camera to the desired location and envmap render
That should make the cube envmap look a lot better than the Blender one... it's good for development but the Racer generated one will look oodles better since it's all from the same engine/lighting env etc


Agree on the masking option for some reflections, on/off is probably enough for the effect... but in any case using an envmap now is so easy with the new feature you can just paste down the live/main envmap, then overlay your cube map (with alpha) to get any final result you want


Very cool work!


I've been reading about shaders again trying to get my head around a few things... mainly SSAO which I think would be nice to get into Racer soon... it'd remove such a burden from artists and also generally add that extra depth where normally an artists can't implement it to start with

Cheers

Dave

 

[Stereo]

So if your head moves outwards towards the glass I assume you see more of the side of the car in the mirror? That is very cool!

Yes, if the SMD cam was any good (not sure how to set one up)

You can get an idea of how it moves around though.

I agree it should be done with a render from Racer, that does somewhat limit it though - it would need separate textures for every paintjob. The red-green version is 750KB at 512x512 sides, only 100KB compressed, so size isn't a problem, but pre-rendering half a dozen options could get to be a pain., esp. since they individually need to be photoshopped to remove the terrain/sky parts.

It's almost tempting to introduce a 3rd control cube map, with channels for (1) alpha masking, (2) texture colour masking, so that the primary one can be rendered out regularly, then just use the second to switch between the two and use the shader's ambient/diffuse settings for colour.

I think the current results look decent enough within the limitations, if alpha doesn't happen I'd just make the blue channel control colour masking and be done with it - then I can make the windows/door handles stay grey, but the paintjob matching the body.

 

[Mr Whippy]

Rendering out lots of colours would be a pain, but perhaps a last step in the process.

Get the camera in the right location, check everything out, then just take envmap, reload shader (alter it), envmap, reload shader, envmap, and just go through them all.

It's not super elegant, but sometimes stuff that just works is good enough.


Implement details aside that looks really really great!


With the extra work to put a polygon radius lip around the back of the housing so you get a chrome lip highlight, and then sit the glass back inside the mirror housing a bit, it'll look pretty damn real as Racer mirrors have ever gone before


Thanks

Dave

 

[Stereo]

Reflected on the reflection code, realized I don't need a vertex shader, just multiply out the reflection vector in the fragment shader. Should make it more portable since it's now basically 6 lines edited in the fragment shader.


make the following changes to dyn_standard_reflect_f.cg. Or, probably, any other *reflect_f.cg will accept them equally, although bumpmap ones will use 'R' instead of 'IN.R' and use higher TEXUNITs.

    // replace the original samplerCUBE envmap with this. Or change UNIT1 and UNIT2.  Just affects what layer# the shader needs.
  uniform samplerCUBE reflMap  : TEXUNIT1,
  uniform samplerCUBE envMap   : TEXUNIT2,
  uniform float4x4 modelMatrix,

  // put this below the other texCUBE loads.
  float3 R2 = mul(transpose((float3x3)(modelMatrix)),IN.R);
  float4 reflColor=texCUBE(reflMap,R2).gggr;

  // put this below the lighting calculations since it needs diffuse+ambient
  envColor.rgb=envColor.rgb*(1.0-reflColor.a)+reflColor.a*reflColor.rgb*(diffuse+ambient);

The .gggr can be happily deleted once alpha channel works, and it'll run as expected.

 

[Mr Whippy]

I really need to make a nice car for Racer soon too, well finish my current one haha. I want to implement all these sorts of things in that too... it adds so much to the feeling of actually being there when these things which have always plagued the illusion of realism are suddenly fixed!


I'll also be trying to work out how to make a soft-top fold away using the scripting haha, that should be fun

All cool stuff, thanks for sharing the code Stereo

Dave

 

[Stereo]

I was just eying the pictures thread, and thought of a widely applicable use for these - left & right halfspaces for wheels. It doesn't really need to be anything specific, just a generic black/grey so that chrome wheels don't reflect anything from behind the car.

It does require exporting left & right wheels with distinctly named materials, so they can use the correct half-mask... but a half-black cubemap of say 32x32 would be a very light addition to the vehicle.

Which brings me to ask... is there any quick way to rename a .dof's materials without importing/exporting it?

 

[Mr Whippy]

I think modeller renaming is the way to go still. 10sec job.

Distinct materials should be ok as most of the time you can't see both sides so batch count won't go up.

Good idea though, usually I just use a gradient fading out ref strength steeply after the face of the wheel, but you approach is one step on so even the spoke edges don't reflect the horizon from inside the wheel well haha! Another one of those detail tweaks that should hopefully be cheap but make a big difference to realism!



Dave

 

[camsinny]

I was just eying the pictures thread, and thought of a widely applicable use for these - left & right halfspaces for wheels. It doesn't really need to be anything specific, just a generic black/grey so that chrome wheels don't reflect anything from behind the car.

It does require exporting left & right wheels with distinctly named materials, so they can use the correct half-mask... but a half-black cubemap of say 32x32 would be a very light addition to the vehicle.

Which brings me to ask... is there any quick way to rename a .dof's materials without importing/exporting it?

My thought was for exhausts, they should have a similar cubemap...would be nice if we could have a rotation vector per material layer so we could use a generic half-black cubemap for multiple things!

 

[Mr Whippy]

Good call. Lots of these things work well with a grad on the ref strength right now (if used, worked well on my M3 9 yrs ago heh )

Just to get this straight, it's a cube map that masks reflection as well as provides something to reflect, but in LDR and static...
Don't we still need the alpha in envmap? Something ideally we need to pitch to Ruud, then it should get in soon!

Stereo's current wing mirrors sell the concept to me for now so I'm sure they will to Ruud, fingers crossed

I'm sure a generic one makes sense but I'd always like to put something in there to reflect *if* we are going to the lengths to make this tweak for realism in the first place

Dave

 

[Stereo]

You could make the emission float4 into a quaternion if you wanted, those would be easy to multiply by the reflection vector, and have separate values per material, since most materials don't use emission.

Yes, fingers crossed to get alpha in the cubemaps.

Hand-tailored cubemaps rendered to fit the individual object would look nicest, but a half-white one with Ka or something to get the right color would really be good enough for most purposes.

 

[Mr Whippy]

Yep that's a good generic solution in this case!

Emission for direction + Ka for colour = very logical.

That combined with custom would be fantastic set for detailing cars really nicely... Without much real overhead... Just missing alpha on called in envmaps then?


Dave

 

[camsinny]

Good call on the emission - chrome shouldn't be touching that variable at all!

My worry with the colour is if there's some obvious colour change on the side of the car (i.e. a black rubber bit around the window)
I guess that should be black/dark in the rendered cubemap though and shouldn't really make a difference to the Ka modulation. Hm. I guess in some circumstances (liveries) it could throw up some issues but in most cases would be fine. I'll have to try this out with my super secret dev car.

 

[Mr Whippy]

I think on an item like an exhaust for example, just the mere hint of the right colour would do. If on close inspection it fails it's not really an issue for me.
It's all about the illusion really...

From behind the car or a track cam it's gonna be a 5px size object tops... not enough size to see liveries or whatever else through!?

Hmmmm

Dave

 

[Stereo]

Yeah, the primary purpose is to get rid of the reflecting off an object that's obscured, just because it's on the reflection map. Errant sunspots are pretty visible on chrome. Further details, if it matters, can be a specifically rendered map in full colour, once alpha works.

 

[Mr Whippy]

Hmmm, just thinking of other places that this would be useful.

Ie, the Lambo side vent areas that run along the sills look really bad reflecting the sun, and the sky generally in the envmap when the sun is on the opposite side for example.

Having a colour swatch of the car colour and a half-dome mask that you can change the direction of would be really nice for fixing these areas up generally. Maybe not perfect, but way better than currently and in motion should stand up to most passing scrutiny

Dave