How to find the right AI level without trying them all?

[Shovas]

Update 2020-04-10: Based on my experience in this thread, I've released an AI mod: "SHO Competition AI is a mod for original content GTR2 cars and tracks that modifies various AI-related configuration parameters to achieve more realistic race performance by AI drivers resulting in more realistic, tighter, and more intense racing." Click here to download the SHO Competition Mod.

How to find the right AI level without trying them all?

I've always had a problem with dialing in AI. I don't know what produces a good race. It seems like I either get blown away or I have to start from way back in the grid (even though my times are better) and work my way up... There's no middle ground like where I might qualify top 6 and have a good battle with the lead pack: If I quality top 6 I always end up killing them in the race by large margins, like 10sec+.

I feel like there's some combination of qualifying grid position that would mean the AI level is correct and that I will have a tough battle to win. Maybe I need to qualify top 6-12th but not higher or lower to have a good fight mid pack? I don't know.

How do you find the right AI level?

Here's some times I pulled qualifying at VLM Sebring in the Ginetta G55:
Ginetta G55 @ VLM Sebring, 20C+, Clear, 2pm QUAL

screen capture tool

1. I notice 80-95 has its own slope linearity vs 95-120. I believe this is due to <95 not being able to use full throttle (it's hard-wired like that into the game, if I understand correctly).

2. My best qualify was a 2:14.420 so that turned out to be AI 93 (which I had times for: 1st 2:14.660 vs 25th 2:16.088). My actual qualify was lower, like 12th or something but I easily got up to 1st within a lap or two and proceeded to lead by 18sec at the end of the 5 lap race.

Ideally, I'd like to figure out the time deviation between each AI level, run a single qualifying session at AI 100, and then run my time through a calculation that would pick the ideal AI level for me. That calculation is eluding me so far, though

My whole problem is I don't want to have to try qualifying in many sessions with different AI levels just to run one race. I'd much rather run one qualifying session and then determine an AI level from that with a calculation and then start another race session with that AI level. I think that's about as streamlined as it can get.

 

[Caribstu]

Ok so opened a random .aiw file and copied/paste it from there into my custom built tracks .aiw, will fiddle about with the amounts.

 

[Andrew C. Clegg]

Won a race from the back yesterday (Amateur and Corner grip usage at 99). Now I'll try Semi Pro and say 95 for Corner grip usage.

Sorry, racing GT Legends at the moment. I answered the alert without checking whether GT Legends or GTR2.

 

[Shovas]

"It's in the track AIW files under GameData\Locations\..."

Nope. It's not. I assumed it would be in the same text grouping as Best, Mid, Worse but it's not. I did a notepad text search for QualiRatio and RaceRatio and it's not there. Checked the .aiw files for four of my custom tracks. None have it.

That's strange. I'm looking at GTR 2 - FIA GT Racing Game\GameData\Locations\Monza\3Monza.AIW and I have QualRatio and RaceRatio under the [Waypoint] section.

I guess some custom tracks don't include it?

 

[Caribstu]

I guess some custom tracks don't include it?

The .aiw is generated by Bob's Track Builder ( the software i use for track building ) so perhaps BTB doesn't generate QualiRation and RaceRatio. Since i added it to the .aiw

Quali R = 0.85
Race R = 0.95

its dramatically improved the game play.

 

[Shovas]

Came back to this last night and made some promising steps. I had to come to accept the need to tune RaceRatio ugh. Also discovered Worst/Mid/BestAdjust settings were probably the root cause of inconsistent Race vs Qualifying AI performance. I'm a glutton for punishment, I know

Keeping notes. Will post back here at some point.

My plan is to have a set of steps anybody can follow to setup their AI, run a practice session, put their time into a formula, and it will determine an AI level and AI Corner Grip/Brake Grip/Brake Power values, and the resulting Qualifying and Race sessions will be tight and competitive racing around your qualifying position.

 

[Andrew C. Clegg]

Thanks for all that you're doing with this.

 

[Shovas]

Good news, I've been having very realistic, competitive, yet clean races with the AI at VLM Sebring 12H even with different cars (still single-make races though). My qualifying position ends up being roughly about where I'm competitive in the Race field, which is what I'm going for, as opposed to stock GTR2 just allowing you to pass the whole field from the back, which drives me crazy.

In these races, I have AI who drive cleanly yet hard, they won't willingly give up their position. If you're not a lot faster than them they'll happily go into a corner side-by-side but without purposely touching you, which can create some amazingly tense but good moments. I've noticed the AI will seem to block as well on long straights before a corner which is very cool to witness while you're racing.

Some examples of cars I've tried and the races they've produced (all races at VLM Sebring 12H so far) ...

In the MRW Ferrari 488 GT3 I qualified first but ended up 3rd which is very plausible to me. AI1 went ahead way too fast (maybe a custom rcd I need to zero out) and AI2 caught up and passed me late in the race. Racing was fast and hard. Surprisingly good result actually not having tuned any AI parameters differently than my original testing with a Ginetta G55 GT4 Supercup at the same track.

In the MRW Porsche 911 GT3 Cup (991-2) I qualified last but competitively (2:06.175 right behind the next guy) and the leader was only at 2:05.232. In the race I quickly got up to 17th and had good battles between 13th and 17th, fell back to 17th for a while, then finished 13th due to an AI crash on the final turn. All the AI around me were seconds apart, not like large amounts of time ahead or behind like stock GTR2 AI can produce.

In the stock Ferrari F575 I qualified 14th, and the race really surprised me. I made up 13th within a lap or two and held it for the rest of the race. The guy behind me was about 1sec behind for most of the race really pressuring me but ended up about 2.4sec behind in the last lap. The guy ahead of me got ahead by 4-6sec. That seemed actually quite realistic for my qualifying position. I routinely have iRacing races (ie. real people) where I usually end up in my qualifying position +/- 1 or 2 spots at the end of the race so this GTR2 result seems pretty realistic.

Note these were all using the same AI parameters from my calculations that I came up with testing with the Ginetta G55 GT4 Supercup. The fact that one car's AI tuning applies so well to other cars, in the sense of qualifying vs race competitiveness as opposed to say always getting a guaranteed qualifying position no matter the car, that really pleases me.

Now, say, in the example above where I qualified last, I really wanted a chance to win so I really want to qualify top 3 - my calculations allow that! I plugin my laptime and, for example, the Q3 AI laptime, and it provides AI Corner Grip/Brake Grip/Brake Power values that will effectively slow down the AI to my pace and allow me to qualify higher (ideally top 3). You can run this calculation from Practice results or Qualifying results, it doesn't really matter, it's all relative. If you base it on Qualifying results, you 'could' run qualifying again or you can just save and exit GTR2, plugin the new AI params in the PLR file, start up GTR2, load your Race Weekend again, and go directly to the Race session and, while you'll still start from your Qualifying position, you will be able to make up places relatively easily and have a chance to win. If you want to qualify higher then yeah you have to run qualifying again.

The calculations assume AI Difficulty at 100 for now. Believe me, you probably can't handle AI 100 when AI Corner Grip/Brake Grip/Brake Power are at 1.0 (default they're much lower). If you can, well wait a bit and I'll have a calculation that will spit out the right AI Difficulty, too. That's all part of this process.

So, I've been testing different cars on the same track to make sure the math logic produces consistent results and so far it looks good.

My next step will be to try different tracks and then different cars at those different tracks to see if my AI parameters produce consistent AI competitiveness.

After that I will try a multi-class race which I think will work because you will tune to your own class and it should behave in an AI performance sense like a single-make race for your class, the other classes will be however fast they want to be (still based on your ai params) but they don't matter since you're not racing against them.

After that, I hope to make some kind of easy spreadsheet and some easy steps to setup your own GTR2 AI so you can do all this. Actually setting up GTR2 for this should be pretty easy. Just figuring out the AI performance param data and math to spit out new AI params is the involved part. I'm keeping notes already so I'll have steps to do all this.

Anyway, let's hope it works out this time. I'm pretty positive since these test races have been really good.

 

[Kenny Paton]

Shovas You're doing some amazing work there and it should make all us gtr2 players very happy.
This goes beyond the older AI mods by a margin.

 

[Andrew C. Clegg]

Sounds good to me too.

 

[Shovas]

Shovas You're doing some amazing work there and it should make all us gtr2 players very happy.
This goes beyond the older AI mods by a margin.

Thank you! I really enjoy GTR2 so I'm happy to to tinker with it and share the results

Regarding older AI mods, yes, I think this is a whole different take on it. While I totally respect the older AI work, I think it's just a fundamental problem in sims with getting the AI Difficulty competitive for your own personal skill level.

Honestly, one of my inspirations is R3E's Auto AI which doesn't work like you would think and doesn't produce the results that I, personally, would want. I believe I actually have a better idea here. Further, I'm pretty sure all this could just be done transparently in any sim for the user. It's just a matter of knowing how fast AI are per some performance unit and bringing them up or down to the user's laptimes.

Here's hoping some devs are keeping an eye on this thread

 

[Kenny Paton]

one of my inspirations is R3E's Auto AI which doesn't work like you would think and doesn't produce the results that I, personally, would want

Personally I think their AAI after training is very good and gives me good close races in most combos. It's not perfect but certainly up there with the best.
It"s really great that you're bringing this older AI model up to a modern level.

 

[Kenny Paton]

Slightly off topic but since I noticed you liking my post GTR233 I would add that the additions you've made to both GTR2 and GTL coupled with what Shovas is doing for the AI it's sometimes hard to think as these as being over 10 years old, and in some ways putting the more modern games to shame.

 

[Shovas]

Totally agree, I wouldn't be here without GTR233's work, and DurgeDriven, and definitely some others :beer:

 

[GTR233]

yes Kenny I keep an eye on Shovas progress and will certainly add his AI mod to both GTL/GTR2 anniversary patches when he thinks it is finished
I already enjoy Shovas ffb tweaks as it is a great improvement over stock game and older ffb mods

 

[Shovas]

Another positive experience, this time with my next step of testing the same car but at a different track: Ginetta G55 GT4 Supercup at HQ Imola GP04.

I qualified 2:02.815, AIQ1 was 2:00.105, so calculated new Corner Grip/Brake Power/Brake Grip values using my formula based off of the same car a VLM Sebring 12H, and then re-qualified 2:02.477, AIQ2 2:02.598, AIQ3 2:02.607, AIQ25 2:03.914. This did pretty much exactly what I wanted: Slow down the AI to my pace all in a one-step calculation.

^ Note I qualified quite a bit faster, 4/10ths faster, then the previous qualifying, so the calculation can't help to correct for that. If this happens, and you're bothered by it, then save your game, exit, recalculate to make the gap, and load your game again. I went ahead with the race and at the least it matched my expectations (ie. predictions in my head if my calculations were working properly).

In the race, the AI were racing hard and clean again, AI2 took me on the first corner but I gained it back and held 1st until the end of the race. The AI behind me were always within 4sec. For the first half of the race they were about 2-4sec behind, sometimes having hero laps and getting up to 1.5-2sec behind, then late in the race the AI really came on and I was being pressured by the AI behind at 0.5sec behind, he really would have passed me if there was a chance. I even had to defend because I was concerned he would try an inside line pass!

^ This AI behaviour isn't the greatest (hero laps and late race sprints) for my goals and, again, I think it's due to custom RCD files I haven't baselined yet for AI behaviour params. Don't worry, though, I don't want to take the personality out of the AI, it's just that for now I need a baseline for a reliable foundation to build the AI calculations on.

So, my takeaway from this is that my calculations, based on a different track, apply quite well to a new track! I even found a mistake in my calculations which actually makes me feel good because it brought the two tracks into sync with AI competitiveness.

There may still be a need to collect AI performance data per-track, I can't exclude that possibility, but it's not the worst thing ever, it's just collecting AIQ1 laptimes for AI Corner Grip/Brake Power/Brake Grip at 1.0 vs 0.5 and the calculations can work off of that.

My next step I think will be to try a stock GTR2 track just as a sanity check to confirm my calculations are now working without making any changes like fixing the mistake I caught above.

After that I'll try some new cars on these different tracks.

Lots of fun!

 

[Kenny Paton]

late race sprints

could that be due to tyre wear I wonder.
Makes the last laps a bit tense then

 

[Shovas]

could that be due to tyre wear I wonder.
Makes the last laps a bit tense then

I would think with tyre wear that would slow them down? Or do you mean they finally get warmed in?

My races are only 5-6laps / 10min long so that I can test quicker. Maybe these late race sprints are just the personality of the AI in RCD files that I haven't baselined yet. I have another experiment to try to test that.

 

[FinFan]

This is an interesting read. I try to come up with a method to determine AI Levels for me and I am curious to see what you come up with. But I don't tweak any game files. I am just trying to predict race laptimes by the AI performances in qualification and warm-up and, by comparing it to my times, setting an AI level for each race that suits my level of driving. HQ tracks and with addon/mod cars. I'm not sure if what I am doing is valid or if I'm just lucky in hitting the right levels but after 10 rather boring races the last two were really good (I have one in progress that really demands my everything). Opposite to you I am doing mostly endurance races depending on how many cars are in a mod. Full grid 100% race length, half grid 50 % and likewise. I like doing endurance races. I made two observations during those races and wanted to share those. Maybe it helps.
First thing is consistency. An AI drivers times can vary dramatically within a race (probably not a factor in a 10 lap race). I discovered that several times during races. While I keep my times within a second over a couple of laps I gained about 5 secs on the car following me. Then, all of a sudden within one lap he made up that five seconds within one lap and overtook me on the next only to be overtaken be me again and falling behind rapidly. And this kept going on (Audi RG GT3 LMS mod). On my actual race I was within a second with another car for 10 laps fighting for position. Upon entering a corner I saw him going sideways in the mirror and the gap rose to 6 seconds. He was back at my rear within a lap. There is a consistency entry in the talent files but I've never touched them. I don't know if a higher number means more consistence laptimes or more variation. I have already won a race where my fastest laptimes was 2 seconds slower than the slowest AI time (on a 36 car grid). That makes it more difficult to set an AI level since only taking one best time from each AI driver doesn't give enough data I think. I am cheating here a little bit by adding a certain percentage to the difficult level. It's my personal performance penalty if you will. But it works for me. I am in the process of a 93 lap race in Dubai with BAC Mono and Ariel Atom 500. I'm in an Atom, started at 13th (random grid position). By the middle of the race I gained 1st due to varying pitstops and fell back to 3rd. But I had several exciting battles with different cars going back and forth. Most fun I had in months. As I mentioned maybe I am just lucky in hitting the right AI Level. I don't think I can win this. The Mono is faster but needs to pit more.
Talking about AI pitstops. Maybe not a factor in determining AI strength but it is unrealistic. I am certain AI always does 4 tires and full load of fuel. It sometimes happens that cars have to pit with only a couple of laps left and I was able to win races by just taking the needed fuel and no tires. AI was always pitting much longer.
Whatever, maybe you can take something out of it, if not I think I have taken enough of your time . I hope you will continue your work.

 

[Andrew C. Clegg]

Interesting read.

 

[Shovas]

Shocking discovery (at least to me) in the last few days: AI Strength doesn't matter at all to my calculations and that's a good thing. Apparently the devs are doing internally exactly what I'm doing manually, that is, adjusting AI Brake Power/Brake Grip/Corner Grip PLR values when you change AI Strength (ie. 80-100). Clever guys, these devs.

To clarify the above, what I mean is AI 100, 101, 102, and all the way up to 120, produce effectively the same laptimes when the AI Brake/Corner values match.

The problem with the way the game does it internally, adjusting AI Brake/Corner values based on AI Strength, is that the 40 increments of the AI Strength (80-120), results in adjusting AI Brake/Corner values by far too much per AI level. So, for example, adjusting AI Strength just one increment, 100 to 101, would adjust AI Brake/Corner values by orders of magnitude more than you might actually want to match your performance. It's the fine granularity that you need to find the right AI performance and the AI Strength lack of granularity kills any chance of finding that.

The good news is I thought I would have to manage two different calculations: (1) Find the right AI Strength, then (2) Find the right AI Brake/Corner values within that AI Strength window. Now I can just leave AI Strength at any value(*), say 100, and focus only on a simpler process for calculating AI Brake/Corner.

With this realization I've experimented further with different cars and tracks and had some good racing experiences. It's still tough to find exactly the AI performance that matches your performance and, so far, it's taking 1-2 extra calculations to get the actual AI Brake/Corner calculation that will produce a good race. That means exiting the sim, editing the PLR, and loading your save back up, 1 or 2 extra times that I would really like to avoid. Would be nice to have a reliable calculation that usually always works.

I'm continuing to refine the process. Ideally, I'd like to get it down to just one round of qualifying, recalculating AI Brake/Corner once, then finish qualifying and go into the race.

There might finally be a light at the end of the tunnel *fingerscrossed*

*AI Strength 80-94 do actually change AI performance for the same AI Brake/Corner values and this is due to 80-94 restricting full throttle usage on the AI whereas with 95 and above they have full throttle and the only limits are AI Brake/Corner values.