Tone mapping

[Mr Whippy]

I currently think the system used is rather limited.

It appears to try give light detail across a massively wide dynamic range, and as such, imo, fails to give sufficient realism.

Has anyone else played with the tone mapping method?

I'm currently trying to implement a 4 stops wide (4 EV) digital camera type system of tone mapping, but I'm a little unsure how it all works right now.

c=colour.rgb*exposure seems to give a linear representation...

...what we might see with an SLR camera, but I'm not sure. Contrast seems much higher and realism feels better. It also shows up issues with specular intensities (they become lots more powerful than may be ideal on most materials)


I'm thinking that we need a nicely calibrated ToneMap method that tries to copy a real cameras dynamic range, to even start to tweak materials in game (diffuse, ambient, specular levels etc)...

The logarithmic method "ToneMapHDR" seemed to be lacking realism. It gave a good presentation of the dynamic range of the scene, but it didn't feel all that real to me...



Hmmmm, just starting to tinker, but I'd like to maybe have a system where we have about 4 linear stops, then about 4 stops at each end that knee out (exponential) to black or white out, ending up close to the ~ 13 stops the human eye is capable of seeing at any given 'glance' of a scene...
The current system appears to be up to 25 stops wide, possibly more, and it just makes the scene feel flat some times! Specular in particular feel to just become very subtle with such a wide dynamic range squished into the 255 levels of intensity we ultimately have...


Hmmmmm

Dave

 

[Mr Whippy]

That mix shader is really nice. I'll probably use it or something like it here for my road > soil/grass/gravel transitions...

What it really needs is bump mapping adding in, somehow.

rgb for diffuse on both maps, with coverage on the alpha of the local mapped version.
Then maybe another two maps for normals on rgb and specular on alpha for them?! Hmmmm



Bump seems important for materials in the latest high-end games... NFS HP clearly uses them all over the roads/edges and it shows with more depth to the imagery

Dave

 

[boomer541]

Agreed, but for some reason I have trouble with bumpmapping as it doesn't look as good in ver0.0821 as it did a long time ago when it first came out. That is something for me to work on.

 

[DavidI]

Agreed, I've wanted bumpmapping for a long long time. It has its place, but I also think we'll be demanding tesselation as well soon. Tesselation combined with bumpmapping where applicable seems to be a good solution.

I haven't checked 0822, but previous standard_bump never looked right for me either.

 

[Ruud]

For v0.8.22, be careful that it sees all images as from a camera, which means they're pre-gammaed using gamma=1.0/2.2 (your monitor then does gamma 2.2, giving back the linear picture). So all images are corrected when loaded to become linear in memory.
For bumpmaps, these are generated so should not be corrected, and will give funny results. So add 'mode=linear', and don't forget those tangents!

shader_road~vf_bump
{
  tangents=1
  layer0
  {
    map=road.tga
  }
  layer1
  {
    map=road_bump.tga
    mode=linear
  }
}

Not sure what tesselation & bump have in common, except for the modeling phase where you generate a bumpmap based on a high-poly model, then use that map on the low-poly ingame model.

 

[DavidI]

Not sure what tesselation & bump have in common, except for the modeling phase where you generate a bumpmap based on a high-poly model, then use that map on the low-poly ingame model.

Nothing except the two effects combined, or seperately can do some wonderful stuff, and tesselation's fairly free thanks to dx10/11 mandating a hardware tesselation engine.
I was thinking along the lines of expanses of grass for instance, bumpmapping can provide the grass blade detail, and tesselation can deform the surface to provide variance in height. Maybe sand too.

 

[Mr Whippy]

Hmmm, that tessellation looks nice on hardware. I was never really keen on normal maps, so now at least we can use a height map again (yay, much more intuitive), and do what high-end renders do and have real extrusion... just quickly

This will make tyres look nice as a starter... Grills on cars would benefit from this too (ie, the honeycomb type)... ummm...

Grass blade shape is a bit ott perhaps... I think there is more scope for a decent grass shader that just works totally differently. Was playing GT5 P earlier and noticed they have a weird parallax effect going on on the flat grass polygons that make up the ground up close, it kinda gives the surface a light fluffy feel as different bits pass by at different speeds. hmmmm... seems to work nice mixed with the usual scatter objects for grass too


Dave

 

[luthobu]

.. don't forget those tangents!

Uhm.. forgive me for being stupid.. but what do the tangents do?
(I googled it but did not understand)

Edit: I applied tangents=1 to Terragen.. and it really looks bad.
At least I now understand what it does, but why does it mess
up the surfaces? (see attach)

 

[DavidI]

I wasn't implying we bumpmap individual blades of grass per-se; better example would be a road, bumpmapping provides the surface texture, but bumpmapping add's micro-detail. We could then use tesselation to add in potholes, cracks etc; the macro details.
Also, tesselation looks to be a good base to provide damage to objects, Racer can alter & deform the tesselation model to apply to the car/track object mesh.

 

[Ruud]

Grass blade shape is a bit ott perhaps... I think there is more scope for a decent grass shader that just works totally differently. Was playing GT5 P earlier and noticed they have a weird parallax effect going on on the flat grass polygons that make up the ground up close, it kinda gives the surface a light fluffy feel as different bits pass by at different speeds. hmmmm... seems to work nice mixed with the usual scatter objects for grass too

I know of a method where you put 2 transparent layers on top of eachother, and get a sort of moving grass optical effect that way. That is, put two horizontal slabs on top and blend.
But these days, a nicer shader-type of grass would be better, let's not be limited to what GT5 does.

 

[Ruud]

Uhm.. forgive me for being stupid.. but what do the tangents do?

Edit: I applied tangents=1 to Terragen.. and it really looks bad.
At least I now understand what it does, but why does it mess
up the surfaces? (see attach)

A bumpmap shows your normals from the surface perspective; that is 'texture space'. A regular blue normal (normal=0 0 1) is defined to point away into the world along the normal. So your bump image is defined relative to that; the surface.
The Cg shader then works in texture space; light/viewer/vertex position is all rotated from world space to texture space to be able to handle the bumpmap image.

As you might know, any transformation from one space to another is done with a 3x3 rotation matrix. Really the 3x3 matrix is a 3x vector, where each vector represent the basis for the transformed space. That is, normally you'd consider X=1,0,0, Y=0,1,0 and Z=0,0,1 to be the basis for a world space. In texture space these are transformed.

So we need a new basis for texture space (to go from world->texture space); we have 'up', which is the normal of the triangle face. Then we need 2 more axes. That is the tangent; it derives the direction of the texture 'flow' from the texture coordinates. So the tangent flows along with the texture.

In the Cg shader, the 3rd axis is calculated based on normal & tangent; do a crossproduct and you get the binormal (=bitangent), which is just the 3rd axis you need for the full transform.

Now you can see what's wrong; the texture coordinates must be flowing into a direction; if you use separate patches that all have UV ranging from 0..1, you never get it to work, since the direction flips every time. Use coords going from 0..100 for example so that there is a direction. I believe Carlswood's road also has this problem, always repeating the same road piece with UV=0..1.
So make sure to not clamp the UV coords to 0..1 but let it flow...

 

[Mr Whippy]

Hmmm, I'm not sure I get the flow thing you are trying to describe Ruud... is there a picture that will help explain it better?


As per the grass, yep, I think GT5 P does use a few layers then actually, or some system. It's got a nice parallax to it and it's hard to make out a solid surface, which is just what grass needs.
BUT, I agree, we have more power and better shaders these days to do more interesting things I think.

I guess you could almost do grass totally procedurally, and just have a few coefficients to tweak colouring and length etc, then just paste it on your surfaces... maybe even use a map to control coverage

Hmmm

No idea how of course hehe, for now I'm still using scatter billboards (any chance of an example of how we should do this type of grass on racer.nl, ie, how the shader is set up and created in the track.shd file... I'm still getting the car rendering over the grass if I move the camera into the grass, hmmmmm)


Cheers

Dave

 

[Ruud]

Hmmm, I'm not sure I get the flow thing you are trying to describe Ruud... is there a picture that will help explain it better?

...
No idea how of course hehe, for now I'm still using scatter billboards (any chance of an example of how we should do this type of grass on racer.nl, ie, how the shader is set up and created in the track.shd file... I'm still getting the car rendering over the grass if I move the camera into the grass, hmmmmm)

I don't have Photoshop with me atm, but for example, Carlswood's road used a loft where the road mapped like this: quad 1: uv coords 0..1, quad 2: uv coords 0..1 etc. So when you consider just the road mesh, the v coordinates (longitudinal) go like this: 0 -> 1 -> 0 -> 1 -> 0 -> 1 etc. And near this:

+-+
|/ | quad 1
+-+ <=
|/ | quad 2
+-+

you get the v coordinate jumping in 1 vertex from 1 to 0 as you go from quad1->quad2. The tangents are like normal smoothing groups: with smoothing you consider all normals from each triangle's vertices that are on the same XYZ position. You just average those and you've got your smoothed normal.
The same with tangents: you take all equal vertices and check their UV coordinates. Really the tangent calculation takes the direction in which U and V are going. But if you have a road like this where I draw the side of the road: (edit: damn, all my spaces are cut down so it looks a bit bad, the layout)

0 q1 1,0 q2 1,0
+---------+--------+----- road1
p1 p2 p3

What direction is the V coordinate going on each '+' (a vertex)? At vertex p1, there is a definite direction, V goes from 0 to 1. At p2 however, the algorithm has a problem with 'v' being both 1 and 0. Really what it wants is:

0 q1 1 q2 2 3
+---------+--------+--------+--- road2
p1 p2 p3 p4

where 'v' continuosly grows instead of 'changing direction'. When looking at a sequence of 0/1/0/1/0/1, the derivate of 'v' is +1, -infinity, +1, -infinity (as you go from 1 to 0 at the same vertex, the same position).
Above in 'road2' you have a monotonously increasing 'v' coordinate (0/1/2/3/4...); the derivate is +1, +1 ,+1 ...

Hope that makes it clearer.

On the billboards, I tried a bit but it needs to be done per object, since you need the center. For example, Carlswood has the first 4 trees packed into 1 object. Here billboarding wouldn't work since it'll rotate all 4 trees, a bit too much.
If you can create a few single objects in your track, I can try to create a shader for such separate objects (using the 'center', which is passed to each vertex shader).

 

[Mr Whippy]

Hmmm, I'm still only just getting that vert/normal issue... BUT, it sounds like the practical solution is to not use 0>1 texture extents, and instead go from 0.1 > 0.9 say, for a repeating normal map texture? Ie, a tyre or a road repeating section?


As for the billboarding, is there no way to ask normals to point at camera, rather than the object? Hmmm, single objects could get a bit messy as there would be thousands.


Anyway, not at that stage yet, still DTM'ing the main terrain... gah, how far can you see haha! Still, early tests show no real FPS drop from 25k view distance to 2.5km view distance... well, about 5fps from 95 > 90fps, so not a big issue yet. LOD may well be the right way to go for mega tracks and long view distances as you said!


Oh, one last thing. The default Carlswood sky clouds value is 10!!! This is making the sky look right, but the Klux value of some clouds is 175kLux... massive intensity!

As said in the past, there is no way to get clouds looking white and fluffy and solid without using big clouds values, so by the time you drop the clouds= value to make the kLux reasonable for clouds (50-100kLux TOPS), they have become very washed out.

Is there no way we can use an alternative method for clouds? I like the idea of a map where rgb is normals, and alpha is cloud density. That way a white alpha means solid cloud no matter what, and clouds=1 should make a white alpha patch be pure white (or whatever colour it should be given the ambient conditions)

Right now it feels like the mie/rayleigh are being applied OVER the clouds, since the clouds are part of the sky, while in reality the clouds are pretty near relative to the massive distance that ray/mie are projecting from... Ie, if clouds were at the edge of space, then I guess the system we use now is slightly accurate...

Hmmm

Dave

 

[Stereo]

A bumpmap shows your normals from the surface perspective; that is 'texture space'. A regular blue normal (normal=0 0 1) is defined to point away into the world along the normal. So your bump image is defined relative to that; the surface.
The Cg shader then works in texture space; light/viewer/vertex position is all rotated from world space to texture space to be able to handle the bumpmap image.

As you might know, any transformation from one space to another is done with a 3x3 rotation matrix. Really the 3x3 matrix is a 3x vector, where each vector represent the basis for the transformed space. That is, normally you'd consider X=1,0,0, Y=0,1,0 and Z=0,0,1 to be the basis for a world space. In texture space these are transformed.

So we need a new basis for texture space (to go from world->texture space); we have 'up', which is the normal of the triangle face. Then we need 2 more axes. That is the tangent; it derives the direction of the texture 'flow' from the texture coordinates. So the tangent flows along with the texture.

In the Cg shader, the 3rd axis is calculated based on normal & tangent; do a crossproduct and you get the binormal (=bitangent), which is just the 3rd axis you need for the full transform.

Now you can see what's wrong; the texture coordinates must be flowing into a direction; if you use separate patches that all have UV ranging from 0..1, you never get it to work, since the direction flips every time. Use coords going from 0..100 for example so that there is a direction. I believe Carlswood's road also has this problem, always repeating the same road piece with UV=0..1.
So make sure to not clamp the UV coords to 0..1 but let it flow...

If the normal is (0,0,1), which I think is rgb (127 127 255), shouldn't it be based entirely on the Z coordinate, which is the normal of the face? So the UV coordinates shouldn't affect it at all. The bumpmap for the most part is near normal, so it shouldn't be too far off this.

I am still seeing problems with the bumpmaps going in the wrong direction, which aren't so obvious on continuous mapped objects - except they're not matching the expected direction.

Of course, if the normal isn't perpendicular to the surface, the coordinate system isn't orthonormal... but I think this is a lesser problem than (0,0,1) pointing in some other wrong direction.

Top is dyn_standard_reflect, bottom is dyn_standard_bump_reflect.
Since most of the map is just normal, it should generate approximately the same reflections - but they're showing stronger with the bumpmap enabled. Reflection direction doesn't use the bumpmap so it's unaffected and at least the same thing is being reflected. The car is also glowing slightly more orange with the bump map.






0->1 0->1 0->1 should be the equivalent of 0->1->2->3 - it's a matter of the direction changing, not the coordinates themselves. When stuff is mapped 0->1 next to 1->0 the coordinates are being flipped around.

 

[luthobu]

Interesting reads. I'll do some tests and try to remember to document along the way.

 

[Ruud]

If the normal is (0,0,1), which I think is rgb (127 127 255), shouldn't it be based entirely on the Z coordinate, which is the normal of the face? So the UV coordinates shouldn't affect it at all. The bumpmap for the most part is near normal, so it shouldn't be too far off this.

Top is dyn_standard_reflect, bottom is dyn_standard_bump_reflect.
...
0->1 0->1 0->1 should be the equivalent of 0->1->2->3 - it's a matter of the direction changing, not the coordinates themselves.

Indeed, I'll have a closer look at the tangent creation algorithm. Here is the code in case you feel lucky ;-) :

//
// Tangents
// Creation code: Lengyel, Eric. “Computing Tangent Space Basis Vectors for an Arbitrary Mesh”.
// Terathon Software 3D Graphics Library, 2001.
// http://www.terathon.com/code/tangent.html
//
struct Point2D
{
  float x,y;
};
struct Point3D
{
  float x,y,z;
};
struct Triangle
{
  unsigned short  index[3];
};
struct Vector3D
{
  float x,y,z;

  const Vector3D &operator +=(const Vector3D &other)
  {
    x+=other.x;
    y+=other.y;
    z+=other.z;
    return *this;
  }
};
struct Vector4D
{
  float x,y,z,w;
};

static void CalculateTangentArray(long vertexCount, const Point3D *vertex, const Vector3D *normal,
        const Point2D *texcoord, long triangleCount, const Triangle *triangle, Vector3D *tangent)
        //const Point2D *texcoord, long triangleCount, const Triangle *triangle, Vector4D *tangent)
{
  Vector3D *tan1 = new Vector3D[vertexCount*2];
  Vector3D *tan2 = tan1 + vertexCount;

  int a;

  memset(tan1,0,vertexCount*sizeof(Vector3D)*2);
  //ZeroMemory(tan1, vertexCount * sizeof(Vector3D) * 2);
  
  for(a = 0; a < triangleCount; a++)
  {
    long i1 = triangle->index[0];
    long i2 = triangle->index[1];
    long i3 = triangle->index[2];

    const Point3D& v1 = vertex[i1];
    const Point3D& v2 = vertex[i2];
    const Point3D& v3 = vertex[i3];

    const Point2D& w1 = texcoord[i1];
    const Point2D& w2 = texcoord[i2];
    const Point2D& w3 = texcoord[i3];

    float x1 = v2.x - v1.x;
    float x2 = v3.x - v1.x;
    float y1 = v2.y - v1.y;
    float y2 = v3.y - v1.y;
    float z1 = v2.z - v1.z;
    float z2 = v3.z - v1.z;

    float s1 = w2.x - w1.x;
    float s2 = w3.x - w1.x;
    float t1 = w2.y - w1.y;
    float t2 = w3.y - w1.y;

    float d=s1*t2-s2*t1;
    const float threshold=0.000001f;
    //const float threshold=0.1f;
    if(fabs(d)<threshold)
    {
      if(d<0)d=-threshold;
      else   d=threshold;
    }
    float r=1.0f/d;

    Vector3D sdir,tdir;
    sdir.x=(t2 * x1 - t1 * x2) * r;
    sdir.y=(t2 * y1 - t1 * y2) * r;
    sdir.z=(t2 * z1 - t1 * z2) * r;
    //Vector3D sdir((t2 * x1 - t1 * x2) * r, (t2 * y1 - t1 * y2) * r, (t2 * z1 - t1 * z2) * r);
    tdir.x=(s1 * x2 - s2 * x1) * r;
    tdir.y=(s1 * y2 - s2 * y1) * r;
    tdir.z=(s1 * z2 - s2 * z1) * r;
    //Vector3D tdir((s1 * x2 - s2 * x1) * r, (s1 * y2 - s2 * y1) * r, (s1 * z2 - s2 * z1) * r);

    tan1[i1] += sdir;
    tan1[i2] += sdir;
    tan1[i3] += sdir;

    tan2[i1] += tdir;
    tan2[i2] += tdir;
    tan2[i3] += tdir;

    triangle++;
  }
  
  for(a = 0; a < vertexCount; a++)
  {
    const Vector3D& n = normal[a];
    const Vector3D& t = tan1[a];

    // Gram-Schmidt orthogonalize
    DVector3 dn,dt;

    dn.x=n.x;
    dn.y=n.y;
    dn.z=n.z;
    dt.x=t.x;
    dt.y=t.y;
    dt.z=t.z;

    //tangent[a] = (t - n * Dot(n, t)).Normalize();
    float dot=dn.Dot(dt);
    DVector3 v;
    v.x = (t.x - n.x*dot);
    v.y = (t.y - n.y*dot);
    v.z = (t.z - n.z*dot);
    v.Normalize();
    tangent[a].x=v.x;
    tangent[a].y=v.y;
    tangent[a].z=v.z;

/*tangent[a].x=1;
tangent[a].y=1;
tangent[a].z=0;
continue;*/

    // Calculate handedness (future! bumpmap use?)
    //tangent[a].w = (Dot(Cross(n, t), tan2[a]) < 0.0F) ? -1.0F : 1.0F;
    if(0)
    {

      DVector3 qn,qt,qc,t2a;
      qn.x=n.x;
      qn.y=n.y;
      qn.z=n.z;
      qn.Normalize();
      qt.x=t.x;
      qt.y=t.y;
      qt.z=t.z;
      qt.Normalize();
      qc.Cross(&qt,&qn);
      t2a.x=tan2[a].x;
      t2a.y=tan2[a].y;
      t2a.z=tan2[a].z;
      //t2a.x=tangent[a].x;
      //t2a.y=tangent[a].y;
      //t2a.z=tangent[a].z;
      float d;
      d=qc.Dot(&t2a);

      //float w = (Dot(Cross(n, t), tan2[a]) < 0.0F) ? -1.0F : 1.0F;
      
      if(d<0)
      {
        float ox,oy,oz;
        ox=tangent[a].x;
        oy=tangent[a].y;
        oz=tangent[a].z;
        tangent[a].x=ox;
        tangent[a].y=oz;
        tangent[a].z=oy;
        tangent[a].x=1;
        tangent[a].y=0;
        tangent[a].z=1;
      } else
      {
        tangent[a].x=0;
        tangent[a].y=1;
        tangent[a].z=0;
      }
      //qdbg("tangent d %f\n",d);
    }
  }
  
  QDELETE_ARRAY(tan1); //delete[] tan1;
}

Then the bump shader, thanks for making such a clear screenshot! It turned out that fresnel was not calculated in the right spaces in the bump shaders; one half in world space, the other in texture space. The fix saves a matrixl mult; in dyn_standard_bump_reflect_v.cg, comment out this line:

I=mul(tbnMatrix,I);

so it becomes this:

//I=mul(tbnMatrix,I);

That fixes it.

 

[Stereo]

Amazing how much difference one line makes.

I'm just sitting here with time=1500, clouds=3, Lamborghini on Carlswood, and it looks really nice. Plus, with all the tweaks & overclocking my graphics card, I'm up to 30fps with live envmapping and shadows.

 

[Mr Whippy]

Amazing how much difference one line makes.

I'm just sitting here with time=1500, clouds=3, Lamborghini on Carlswood, and it looks really nice. Plus, with all the tweaks & overclocking my graphics card, I'm up to 30fps with live envmapping and shadows.

Have you turned live envmap updates to 1 (from 6), it's a bit more jerky on the old envmap, but it doesn't really look so bad and it's a fraction faster too!

Imo it's been my default for a good year now

Dave

 

[Ruud]

v0.8.23 has live_track.frames_per_update. There's a bug still that if you set render_once to 1 that it never renders the first time (still counting down the frames). You can set that to 10 for example and things will be quick & still live.
I find that the amount of AA (samples=xx, coverage_samples=xx) really makes a difference. And ofcourse shadowmapping. I've set that to 1024 mapsize, with a new dot(normal,position) in shadowmapping.cg to reduce the Z correction things look much better. I run around 150fps on Carlswood with the Lambo (csm on) at the start/finish line (Quadcore, nVidia GTX285, 195.62 instrumented drivers).

Also, I modified the sky*.cg shaders to do more of an 'a,1-a' blend for the clouds, so they don't become washed out. Really it wasn't that the clouds were rayleigh/mie'd away, but the atmosphere color was always added to the end result, so to get white, you really had to add a lot of cloud texture. Nicer now, with for example 38klux in the atmosphere, and 70 klux in the clouds. You'd say clouds are darker, but are they? Hm. Darker really gets you gray clouds from my experiments today. In any case, it's back to clouds=1 for v0.8.23.

 

[Mr Whippy]

Clouds are lighter than the sky... point metering with my SLR, by and large anyway. Storm clouds are dark, and heavy ones. Bug generally big fluffy white clouds get bright. I've had ~140k in and around patchy clouds occluding the sun at sunset!

Not messed with AA samples in ages, might give it a try just to see, but I think I stopped playing with it because the value ended up seeming the one I ended up choosing from release to release


Just added a bug in the 0.8.22 release thread Ruud, maybe not a bug, but worth sorting for v0.8.23 imo!

Dave