Racer v0.8.37 released

[Ruud]

Just in time for X-mas.

Get the new version at http://www.mediafire.com/?hc44aiq5izp4iqi
Then get the racer.exe/pdb which fixes reflections at www.racer.nl/temp/racer0838_alpha.7z

At some point we're going to have to call it v0.9rc1, since this one does a little tweaking to the graphics intensities (energy conserving) and sets cast_shadow to 0 by default. Most shadows will disappear; a huge fps boost until you put shadows back in shader files (.shd).

Merry Christmas!
Ruud

The changelist for v0.8.37:
- Added views.ini keep_aspect property to maintain aspect ratio for static images. See http://www.racer.nl/tutorial/dials.htm
- dyn_standard_reflect_window_f.cg now uses alpha as reflectiveness as well; opaque parts get less reflection
- TrackEd's AutoAseToDof has improved error messages on bad submaterial usage.
- TrackEd: some (progress) dialogs never appeared. Copy flags to group didn't work.
- TrackEd now colorizes the Collide and Surface flags in Props mode (F4) for easy tweaking of flags.
- standard_bump_detail_f.cg had a hardcoded 12x scale; now it uses the scale defined in the .shd file
- The garage track is not shown in the 'Select Track' screen anymore.
- Added log.events.host/port settings for remote racer tracking (niche usage).
- Lidar data (track.rld files) can now be sorted and driven on. Pro use only.
- Added car pitch/roll in body AND world coordinates in the Ctrl-9 debug screen.
- Added 'analyse shaders' command to check all shaders (track.shd/car.shd).
- The shaders default 'reflect' value for each layer is now 0 (was 1).
- Final tweaks in the shading system; energy conservation for diffuse and specular. For diffuse colors, this
means diffuse colors are divided by PI (assuming diffuse colors are kept between 0 and 1), and specular
colors are being corrected with (n+6)/8*PI.
See also http://www.rorydriscoll.com/2009/01/25/energy-conservation-in-games/
- Timers are now in microsecond accuracy using QueryPerformanceTimer(). Multicore machine issues?
- Profiling is more accurate due to the better timer accuracy. More graphics divisions added.
- Shader cast_shadow property by default is now 0; it's better to explicitly define shadow-casting materials
since you don't need a lot of shaders to cast shadows to still get a good image.
You *will* need to add cast_shadow=1 lines for shaders that you want to cast shadows.
- Depth texture setup optimized for shadowmapping; quite a big fps increase.
- Added look/left right controls (look_left, look_right) to dynamically look left or right (Q/W by default)
- In your controller file you can add global.left_right_look_velocity to set left/right look velocity (default=250)
- In your controller file you can add global.left_right_look_max to set the max left/right look angle (default=45)

 

[DavidI]

Ruud, can you not test for these GFX features, maybe run a script compile at Racer load, and check it compiles and then flag (&log in qlog) the features available? Might save some of this heartache for the ATI & lowlier nVidia card users.
Or when is openCL viable as an alternative to Cg? Noticed nVidia open sourced CUDA
PS. That link to the pdf doesn't work for me.

 

[Mr Whippy]

Damn ATI and their cheapy OGL implements

Nice work with the mipping of envmaps Ruud! A much needed feature for proper material response.


Perhaps now, time permitting, it's worth looking into a new material system that can take advantage of this... I'm not sure how it would start, but it makes sense to use physically based phenomenon to base the system upon, so things like energy conservation can be built in nicely.

For instance, IBL may now be quite easy to implement? That way we get away from using the sun.diffuse and sun.ambient values which we may generate badly, and simply let the resultant environment map dictate our scene lighting.

We can also look at materials more how they really exist. A car paint has a dull base coat with clear smooth lacquer on top.

So we need a base colour (or colours? maybe if we use a fresnel value and move between two colours for base coats (approaching brdf technique)), and then reflect off that (multiply?) a heavily mipped envmap, which lights it up. Metallics would simply use different base colours and less heavily mipped envmap, to give them their different response (looks more "specular")

Then we can add the old style envmap (add?) over the top?

No idea exactly how it will all work, but it's worth perhaps looking to something like Maxwell Render, or Fry Render, and see how their materials respond fantastically well to real lighting values/techniques, despite relatively simply input parameters!

No specular or ambient/diffuse values, just responses to light.

Maxwell currently uses a system like:

• Colour 0deg, colour 90deg (so we can say at 90deg it is red, and at 0deg it is blue, oooo, weird colours)
• Index of refraction, that then dictates parameters for us in this case, of how we interpolate between 0 > 90deg values. We have fresnel to approximate the same.
• Roughness 0 - lambertian, so either perfectly smooth to totally diffuse. Neither extreme really exists. We can approximate that by choosing the appropriate envmap mipmap to use, either a sharp one for smooth materials, or a blurry one for rough materials. This is then used to light?? the base colours (multiply blend?)

Then you layer materials, like we do in the shader. So we can now apply a 'blend' envmap (sharp) over the top of that base layer, and get the response of the lacquer clear coat. Maxwell Render does that with a 'layer' and thickness and IOR, but that isn't needed for our level of rendering of course
I'm not sure exactly how we get different materials like metallics vs non-metallics etc, I think it's down to the mixing of the 0/90 colours and the IOR/fresnel.

Again, no idea on the actual implement, but the inputs are very basic, the outputs are extremely realistic. All the sub-elements are already there.
Indeed, energy conservation is irrelevant in this situation, since what we see bouncing off materials is driven by what hits them in the first place (the envmap light).
There is no mis-match of using one set of values (sun.specular) that are unlinked from what we see being reflected (envmap), for powering a set of output values (specular/reflection blob on a material)... In theory, as we blur our envmap, energy is conserved. A small sharp spot gets larger and it's intensity reduces. Dark areas get brighter as the values are sampled and mixed. I think in theory, if you blur an HDRi skydome image down to 1px square, you get the average scene intensity (which we use for scene intensity in kLux in Racer!)... thus we automatically get the energy conservation we desire by softer reflections having less power in the 'bright spot'
I think it would be good to dump the old specular terminology altogether moving forward as it is just confusing. Everything is a reflection of varying softness or sharpness, whichever way you want to look at it.

Then you can simply layer up a materials appearance if you want. There may be some overhead on more complex materials, but I'm generally just thinking they are only really needed for car materials up close (ie, paints). Most of the time we will still just use one layer?!
Hmmmm

Dave

 

[ammods]

So is it my understanding, then. That ur latter versions wont support ATI?, if this the case, this is most disappointing, as alot of people use them, i also guess my suggestions have fallen on deaf ears, so on that note, i bid u adew, fairwell, because i only use nvidia or ati gpus, all the leading game developers support ATI.....

 

[Mr Whippy]

ATI is supported, but unfortunately ATI are to blame for your issues, not Ruud.

Racer has always been a cross-platform application, and as such DirectX can't be supported. That leaves OpenGL, which ATI doesn't support as widely as Nvidia do.
Every single advance in graphics Racer makes, ATI's trip up somewhere because of their shoddy support for the OGL standard.

It takes Ruud extra time to code ways around the issues that arise because of ATI's crappy OGL support. Those work-arounds do come, but not always as soon as the logical initial implementation of the same features which Nvidia do support.

Ie, this CubemapLod command, that is just built into the OGL standard, but ATI have chosen to not implement it properly or something. That means hours of time spent by Ruud working around it to do exactly what the OGL one does in one simple built-in command to the OGL specification.

Ruud is stuck between a hard place and a rock because ATI skimp on their software/drivers. Ruud will eventually offer you ATI users a fix, but it won't come instantly like the features do for Nvidia. It's just the nature of discovering as you work along, where ATI have cut corners or missed functionality.



I buy nVidia exclusively now, because I've realised that it's the best card for Racer, but also you get what you pay for... proper support for EVERYTHING you might want to do with your graphics card, not just super fast support in DirectX "games" and tons of missing features you should expect to have working, for exactly these situations, using your graphics card for something not DirectX!


As for the second part, a realtime shader tweaking environment would be nice. However, once you get the hang of it, 'reload shaders' via the console is fairly quick.
It's far from perfect, but I can manage to work with Racer and tweak graphics quite intuitively that way. It's a learning curve for sure. I'd prefer Ruud to develop the graphics and get the engine really good first, then support tools can come later if he has the time I guess.


Don't lose faith so soon

Dave

 

[Ruud]

Ruud, can you not test for these GFX features, maybe run a script compile at Racer load, and check it compiles and then flag (&log in qlog) the features available? Might save some of this heartache for the ATI & lowlier nVidia card users.
Or when is openCL viable as an alternative to Cg? Noticed nVidia open sourced CUDA
PS. That link to the pdf doesn't work for me.

The link explains the vertex texture fetch capability of SM3.0 (Shader Model 3). The PDF is from 2004; I do expect certain features to have been passed to the ATI side by now. Perhaps a side effect of Cg though; in the future I want to move to GLSL.

 

[QuadCoreMax]

ftp://download.nvidia.com/developer/Papers/2004/Vertex_Textures/Vertex_Textures.pdf

@ammods
Racer will always support both as long as they exist, it's just a mess for programmers & developers, none of the stuff can't be done straight away, it's like coding a webpage, none (Mozilla vs Explorer) renders the same with the same code...

Anyways, I'm really happy to see so much improvements & our dreams coming true, I mean that was a feature we all requested 1 year ago

 

[Alex Forbin]

See the attached test_trackir.txt (rename to test_trackir.ini and put it in data/controls). Then set ini.controls to test_trackir.ini. It works like the mouse currently (I should get the official TrackIR SDK, but then I also need a more recent TrackIR device; the one I have is years old, although it still works on XP SP3).
That should get you started.

I had a chance to try trackIR and here's a little feedback...

The system works in mousemode but not native mode. The only real problem here is that it prevents you from using the mouse for other things.
It would be nice if the cursor would disappear when in trackIR mode.

Other than that I see no issues.

Alex Forbin

 

[Alex Forbin]

The link explains the vertex texture fetch capability of SM3.0 (Shader Model 3). The PDF is from 2004; I do expect certain features to have been passed to the ATI side by now. Perhaps a side effect of Cg though; in the future I want to move to GLSL.

Why wait to move to GLSL? I would think that it should be a part of 0.9f since it will affect content.

For what it's worth the current damage system with the canned style of damage would be fine for an arcade racer like NFS but there are much better techniques available, and Racer deserves only the best.

Alex Forbin

 

[Mr Whippy]

For me, damage is an arcade feature anyway.

Every game with 'realistic' damage looks interesting, but the reality is if you damage a car even 10% what you often see in a car game, the drive-shafts would be smashed, the diff smashed, the car bent, the engine mounts smashed, etc etc...

Only really for rally cars might it be useful, since their panels get smashed off but the car is still designed to keep going

Most race cars will be totally out of the race before they get a lot of visible damage anyway, so dents and scuffs would be the most you would ever see.

Ie, car hits tree, damage deformation fitting shape of tree = nice.
However, car would be damaged and un-usable, and/or driver dead.
Ergo, nice to have, but what purpose does it serve except as a glorified and accurate end to your race?

Unless we intend to have flashy visual damage with really poor impact to driving physics, like GT5, but I don't get the point in that. If you hit a wall at 50mph in most cars, it's car written off and driver in hospital. In GT5 you get slightly wobbly steering, and broken bumper


Agree on any big graphics moves to be made for .9 though.

For me, Racer needs to get a system that will stay adaptable/upgradable moving forward, but also employ sensible techniques for moving forward into the future.
Right now the CSM is nice, the HDR is nice. But the reliance on the old spec/diff/amb colours for object lighting is not so ideal. I could live with it if they were automated values from envmap lookups, but they are user defined and as such generate a HUGE issue with realism or time to tweak the values!
Ie, the whole energy conservation thing is irrelevant if the envmap looks one way, and the diffuse/amb value curves you define say another thing... at the very least they need to be tied together (envmap appearance/intensity and the diff/amb curves)
Or at the most, move away from the diff/amb/spec curves, and use the envmap to light from (IBL essentially?!)

Dave

 

[Raphael Buthmann]

I'm a nvidia user , and i can't get env mipmap working =/ .

 

[Mr Whippy]

I've just put 37 on and downloaded Ruud's update again and I'm not even getting envmap working yet hehe.

I'd not worry so much at this stage, probably best to wait for 38 proper and keep using good old 36

Dave

 

[Mr Whippy]

Ah, working now.

Hmmm...

The mipping works great.

I'm now wondering however, if mipping won't offer what we need for IBL anyway.

I think the issue is that as you get to say mip level 8, the cube map is err, about 4px across, which almost totally removes the sun spot... great for what is essentially a pixel pointing in each direction and summing up to be the intensity of the whole environment from that one direction...
However, it has sharp boundary edges, which is obvious since there is no resolution for blending/smoothing now (also it seems when the sun is at a boundary on the cube map, the edge is sharp, hmmm)

I wonder if for IBL we will ultimately need some blurring algorithm to retain pixels for smooth gradients in the lighting but remove the sharpness.
Ie, running mip level 6 and suddenly we have 4x more resolution which means smooth reflections, but the sun spot is really clear in the envmap, so it's not as effective for representing the diffuse illumination.

Anyone know a good bit of code for blurring the envmap in HDR




For reflections though this should work pretty nicely, and for soft/dull specular.

Using the right fresnel parameters can give you really nice and subtle effects.

It's nice for example, to have the soft specular on the road at sunrise orange, despite the sun.diffuse colour being near white (which is technically accurate)... since the sun is actually generating yellow white light. The sky is generating the deeper yellow/orange colours and then emitting them into the scene, at much lower intensity. They are non the less reflected by the scene objects... white light, orange reflections (check next time the sun is setting, take some images of a white card with daylight white balance, and see how grey exposes grey, but the 'specular' off the road and other objects is orangey.

 

[Alex Forbin]

For me, damage is an arcade feature anyway.

Every game with 'realistic' damage looks interesting, but the reality is if you damage a car even 10% what you often see in a car game, the drive-shafts would be smashed, the diff smashed, the car bent, the engine mounts smashed, etc etc...

Only really for rally cars might it be useful, since their panels get smashed off but the car is still designed to keep going

Most race cars will be totally out of the race before they get a lot of visible damage anyway, so dents and scuffs would be the most you would ever see.

Ie, car hits tree, damage deformation fitting shape of tree = nice.
However, car would be damaged and un-usable, and/or driver dead.
Ergo, nice to have, but what purpose does it serve except as a glorified and accurate end to your race?

Unless we intend to have flashy visual damage with really poor impact to driving physics, like GT5, but I don't get the point in that. If you hit a wall at 50mph in most cars, it's car written off and driver in hospital. In GT5 you get slightly wobbly steering, and broken bumper


Agree on any big graphics moves to be made for .9 though.

For me, Racer needs to get a system that will stay adaptable/upgradable moving forward, but also employ sensible techniques for moving forward into the future.
Right now the CSM is nice, the HDR is nice. But the reliance on the old spec/diff/amb colours for object lighting is not so ideal. I could live with it if they were automated values from envmap lookups, but they are user defined and as such generate a HUGE issue with realism or time to tweak the values!
Ie, the whole energy conservation thing is irrelevant if the envmap looks one way, and the diffuse/amb value curves you define say another thing... at the very least they need to be tied together (envmap appearance/intensity and the diff/amb curves)
Or at the most, move away from the diff/amb/spec curves, and use the envmap to light from (IBL essentially?!)

Dave

Actually there are a lot of race cars that run damaged, In Nascar it's not unusual to see a few at the back of the pack that are really beat-up but still running for points, they just don't get much airtime. Also having realistic damage makes you drive more carefully than without (providing it affects performance as well). When we raced Nascar Racing years ago, if you made a mistake (or someone else did) and you were damaged, you could limp around to the pits for some quick repairs that would let you continue the race, but you may have to contend with a hard pull to the right and decreased power. In GTAIV running from the cops gets even more exciting once you've piled into something and have a wheel locked up from contact with a fender.
FWIW I don't care for GT5 damage either, it sucks when you hit a wall at 130mph and it just scratches the paint or something falls off, it really takes away from the idea that it's a simulation. But I lost enthusiasm for the GT series once I realized it was impossible to roll the cars in GT3, that's why I think we can do better. We also got a kick out of seeing the carnage that took the leader out in slow motion replays.
The best overall damage model has to be Rigs of Rods soft body system, but I realize that it's probably very CPU intensive. GTAIV however seems to be a nice compromise between CPU demand and realism.

I agree on the Image Based Lighting too. I have real trouble getting things to look right currently since the TOD curves and the auto exposure seem to be at odds with each other. The energy conservation will be nice, once we have a realistic environment for it to work in. (in other words... ditto)

Alex Forbin

 

[Alex Forbin]

Ruud,
I may have figured out the "shimmy" problem we've had forever. It suddenly appeared today after I made a profile for Racer in Logitechs profiler program, once I deleted it everything went back to normal. It might be that some people are letting it scan for games when it installs and it's making a default profile for Racer. Apparently it conflicts with Racers system somehow.

Alex Forbin

 

[Mr Whippy]

Hmmm, thinking more here.

The specular energy conservation was really not ideal.

I can't put my finger on it exactly, but right now a car reflecting the environment map has the sun spot reflecting out at 255kLux, which is massively bright. Clearly the sun is very bright.

However, even trying to emulate that sun spot (shininess = 200) in the shader, and specular = 1 1 1 1, the intensity of that spot is factors away from the intensity of the spot from the envmap.

Basically, you have to use a specular value near 5 5 5 5 or something to get near 255kLux, which is what the envmap is giving out from the sky dome.

I'm generally happy with the sky dome intensity after checking it vs real life quite a lot, even the sun having intensities up over 200-250kLux!


I think there is some sense/logic checking that needs to be done through the entire lighting/envmapping part of the renderer to make sure it all makes sense.

Ie, deviation of sun.diffuse and sun.ambient values away from what the envmap suggests is not ideal. Checking what realistic sun.intensity values are throughout a clear day too, so we can get our ambient/diffuse AND environment map all matching up really nicely. Right now I think they are a bit out and that is making realism a bit harder to achieve, especially across the sunrise/sunset periods!

Hmmmm, hmmmm...

Dave

 

[Mr Whippy]

The best overall damage model has to be Rigs of Rods soft body system, but I realize that it's probably very CPU intensive. GTAIV however seems to be a nice compromise between CPU demand and realism.

I'll have to try RoR at some time

I just remember in Viper Racing for example, the damage was a great novelty but generally it was so realistic that even a firm touch with another car would mean you had to retire from the race... super realistic hehe.

So, despite it being excellent, it was turned off most of the time when we played properly as it could easily ruin a big race.

Generally I'd say if you can see damage that much you can tell it's not canned, then the chances are the car would have failed and/or broken down from the damage.

Rally/Nascar aside I guess as the main exceptions.


I'm all for a damage system, but there is so much where Racer can be better I'd prefer to use the system of damage we have now simply to imply you've had a rough race, and concentrate on the other elements like audio and renderer optimisations.




Ruud, is there any easy way to batch DOF's using the same shader per frame? Ie, if we have 1000 trees as one big DOF, it is immeasurably faster than 1000 DOF trees as individual DOF.
Should we be batching our shared shader geometry at authoring or would it be better to do it in real time? Or is that too costly?

Just trying to get the ideal balance between loading all the geometry in one DOF even if a lot of it will not be visible that frame (so wasteful to load the polygons at all before they are culled), vs having more graphics calls than needed at other times due to having more than one DOF with that same shader in the view...

Hmmmm

Dave

 

[Pedro_Takumi]

the game keeps getting better and better but let me just put a simple question without any technical detail, isn't the game eating to much resources? i noticed a frame rate improvement with version 8.7, 10 to 15 frames up in the same track

 

[Stereo]

However, it has sharp boundary edges, which is obvious since there is no resolution for blending/smoothing now (also it seems when the sun is at a boundary on the cube map, the edge is sharp, hmmm)

I wonder if for IBL we will ultimately need some blurring algorithm to retain pixels for smooth gradients in the lighting but remove the sharpness.

It would be possible (I say without knowing how) to repeat pixels along the boundaries of the mipmaps, so that it doesn't have to do blending there. I've seen algorithms to work around the issue but that involves going deeper than cg's texCUBE, since you need to blend manually.

mipmaps are nice cause they're fast, and since it does it iteratively the kernel's never more than ~4 pixels. An actual blur algorithm (eg. Gaussian) to get the same radius will end up using a lot more points per pixel in the output image. mipmapping uses 1 + 1/4 + 1/16 + ... = ~4/3 of the original pixels.

@ damage, I didn't see any until l was hitting walls at 100+mph and rolling the car multiple times, I guess the sensitivity needs some tweaking.

 

[DavidI]

Only really for rally cars might it be useful, since their panels get smashed off but the car is still designed to keep going

Most race cars will be totally out of the race before they get a lot of visible damage anyway, so dents and scuffs would be the most you would ever see.

Ie, car hits tree, damage deformation fitting shape of tree = nice.
However, car would be damaged and un-usable, and/or driver dead.
Ergo, nice to have, but what purpose does it serve except as a glorified and accurate end to your race?

Beg to differ, haven't watched V8 Supercars racing have you? One car in the last race of the year two years in a row finished missing a door. You should see what the V8 Utes series is like, they don't mind a bit of biffo between cars.

Damage would be nice when implemented well, but there are features that would be nicer to finish before we work on damage.
Like traffic so I can get the trolley tractor to do it's job...

 

[Mr Whippy]

It would be possible (I say without knowing how) to repeat pixels along the boundaries of the mipmaps, so that it doesn't have to do blending there. I've seen algorithms to work around the issue but that involves going deeper than cg's texCUBE, since you need to blend manually.

mipmaps are nice cause they're fast, and since it does it iteratively the kernel's never more than ~4 pixels. An actual blur algorithm (eg. Gaussian) to get the same radius will end up using a lot more points per pixel in the output image. mipmapping uses 1 + 1/4 + 1/16 + ... = ~4/3 of the original pixels.

@ damage, I didn't see any until l was hitting walls at 100+mph and rolling the car multiple times, I guess the sensitivity needs some tweaking.

Yeah, the mip's work great for the reflections to say mip level 6 or so... they should make chrome and alloy wheel type materials look much better.

There is ultimately a big underlying question over how we do materials I guess. I'd like to see IBL and stuff, but it might not happen for v0.9... not worried either way really... the current system is nice and flexible. It'd just be cool to drive into a tunnel with say green tinted lights, and have the car tint a greeny colour (not the sky ambient colour) for example.

I think we can also still play with shaders ourselves, and if they make sense they will find their way into the Racer defaults and get updated in future...

Ie, the mipmap command would be best called mipmap_bias surely, like the other bias values, and then added under the specific envmap slot you want to use?

The current command is for the entire material, so I assume you can't use the envmap twice, a blurry one and a sharp one?

Just little details like that hurt flexibility. It's details like that I think we need to tidy up for v0.9, so moving forward we are not limited by rushed implementations of features.

I'd also like to see the audio stuff finished up (on/off throttle, maybe better splitting of interior/exterior sounds for wind, road noise etc) for v0.9, as there are just those bits that stop you doing really nice stuff with cars!


All good, but no harm in making it better still

Dave