Ryzen AM5 ; 7000

[RasmusP]

@RasmusP im noticing a cpu hit ie performance in game when using sim hub. Rf2 is single core, could I perhaps dedicate a core to sim hub? Or have I got cores all wrong lol, and would this mess up non single core sims?

You can definitely try to do this!
It can give a little fps boost, but it might also be the cache/ram here and not just Windows splitting tasks incorrectly across the cores.

Tell us the results!

 

[Rebel_488]

Hmm now I’m thinking process lasso and tightening up my ram timings as well lol, cheers for ideas

 

[RasmusP]

Holy crap, I didn't expect some RAM tuning to have THAT much impact!

I found a nice guide from Igor's lab:

The big Ryzen 7000 Memory and OC Tuning Guide - Infinity Fabric, EXPO, Dual-Rank, Samsung and Hynix DDR5 in Practice test with Benchmarks and Recommendations | igor´sLAB

AMD’s new Ryzen 7000 desktop CPUs, based on the Zen4 micro-architecture, still use the same chiplet design as their predecessors, with a few small but not negligible changes. Igor already gave details…

www.igorslab.de

PPT was 71.8W avg during the benchmark. So just 1.8W more than before.
Same +200 MHz via PBO, -10 Curve Optimizer and RAM set to 6000 in 1:1 mode (inf. fabric 2000, memory controller 3000.

Put in all the timings from this part:
6000 CL30 EXPO Aggr 1:1 5H16M SR, 2000 IF – “aggressive”, pre-optimized subtimings

Here are some comparisons. PurePC.pl didn't bench the 7600X, only the 7700X. Naturally, more cores won't gain many fps in ACC so they should be basically identical.
If I scale the scores of the 10600k with 1.2664 and 1.2281, I very closely match the 7700X results.
But I used the 1.5066 and 1.4935 and wow, great value imo!!

 

[Neilski]

Holy crap, I didn't expect some RAM tuning to have THAT much impact!

Sorry if I'm totally missing the point but I'm struggling to work out the apples vs. apples in the CPU comparison above. The blue and red bars specify the CPU, but the yellow/orange and green don't. If they are both with the 7600X, what were the RAM timings for the 50W run?

 

[RasmusP]

Sorry if I'm totally missing the point but I'm struggling to work out the apples vs. apples in the CPU comparison above. The blue and red bars specify the CPU, but the yellow/orange and green don't. If they are both with the 7600X, what were the RAM timings for the 50W run?

Whoops!
The blue one is my Intel, the other bars are all the 7600X. The red one was the first test with the xmp/expo profile used, which is 5600 CL36.

To make sure it wasn't something else, since I didn't do these 3 tests right after another:
I did an ACC only test, one after another, without power limits and just noting down the avg power from hwinfo64
(which is super consistent in ACC during an AI race start).

I tested CPU OC via PBO (+200 boost clock) with the 6000CL30, then with stock ram at 4800CL40 and then stock CPU with 6000CL30.
Didn't put it into a sheet yet but here's the raw data. Ignore the 0.1%, minimum and maximum. They fluctuate way too much within the 40 seconds of testing.

The interesting part for me was the efficiency so:
—-avg fps—-
PBO+Igor's 6000CL30 = 1.45 fps/W
PBO+Stock 4800CL40 = 1.40 fps/W
PBO_off+6000CL30 = 1.67 fps/W
—-1% lows—-
PBO+Igor's 6000CL30 = 1.28 fps/W
PBO+Stock 4800CL40 = 1.26 fps/W
PBO_off+6000CL30 = 1.47 fps/W

 

[Neilski]

Ah OK that makes sense now, ta!
The takeaway conclusion for me, above all else, is still that ACC deeply sucks with "normal" cache sizes and is thus insanely dependent on RAM speed/latency (or both). As I think I said before, I would like to think that a better optimisation (respecting typical CPU cache sizes) would flip that on its head, but I guess I should instead presume that the Kunos team aren't morons and have probably already done what they can in that regard.

The interesting part for me was the efficiency so:

Yeah, that's fun!

 

[RasmusP]

Ah OK that makes sense now, ta!
The takeaway conclusion for me, above all else, is still that ACC deeply sucks with "normal" cache sizes and is thus insanely dependent on RAM speed/latency (or both). As I think I said before, I would like to think that a better optimisation (respecting typical CPU cache sizes) would flip that on its head, but I guess I should instead presume that the Kunos team aren't morons and have probably already done what they can in that regard.

Absolutely! Stefano said a few times after he left Kunos, that the UE4 is basically made for 30-60 fps and can made to run on potatos at 30 fps by reducing settings.
But it has some parts in the rendering pipeline that take quite a bit of time no matter which CPU you throw at it and the higher the fps get, these "always too long" parts have more and more impact on the final frame time.

 

[RasmusP]

Had some more "fun" more or less accidentally and also forced, lol.

Lot of bla bla. If you just wanna see some efficiency comparisons for OC/UV/RAM settings just look at the 2 charts at the bottom.

Currently, AM5 has issues with sleep mode at more than 5400 MHz on the memory. And also if you load an XMP/Expo profile.
The bug varies from going to sleep but when the PSU clicks, it instantly clicks again, the fans ramp up but the monitor stays black until eternity. Until you press and hold the power button, turn it on again and it will boot right into where you left it for sleep mode...
The other possibility is that Windows simply hides the sleep mode completely.

That's the "forced" part of why I'm using 5400CL30 Tuned now but with only 1.25V instead of 1.35V (with Igor's timings).
I like the sleep mode a lot to save the 100-150W idle power of the full PC when taking a short break without closing and opening everything up again.

I was a bit confused about sleep mode since I read that the RAM stays powered but I could switch off the PSU just fine and it woke up within the normal 3 seconds without issues.

The "accidental" part is that I wanted to watch a replay in ACC and had really low fps.
I tested how much power saving could be done via the energy saving power plan at 10% max CPU speed. All 6 cores were stuck at 3.0 GHz. You can't save much though, it's 30W instead of 32W at idle...
Of course I forgot about that power plan

But the fps weren't THAT terrible so I thought I could do some testing and here are the results:
PBO means +200 MHz allowed and automatically boosting to 5.64 GHz in my case. More isn't available yet.
Stock results in 0.14 GHz auto boost to 5.44 GHz and "-500" leads to 4.84 GHz (5.3 spec).
Decimal values for the power = no limit set, clean values = limited.

I guess I could tweak it further to find the real min/max-sweetspot but I can't be bothered for now...

One thing I didn't mention:
All tests were done with the curve optimizer set to -15 on 5 cores and -10 on core 4.
Underclocking it to only 4.84 GHz and the reduced RAM clocks meant that I could increase the undervolting to -30/-22 on the cores. It's stable since 12 hours. hopefully it stays stable

One last note:
I have a 100 Hz g-sync monitor and got used to 85-90 fps with a limiter to maintain a smooth and clean image with minimal input lag.
These ACC tests are done with 49 AI at Spa, having ALL visible and starting last.
Normally I only race with 20-25 cars and set visible opponents to 20, which meant my 10600k showed an avg FPS of 87 with the limiter set to 85.
40W, PBO -500, 5400CL30 Tuned easily has more than 85 FPS for the 1% lows with these settings, while the 10600k dropped into the 70's.

Dropping the RAM voltage and MHz apparently gives the CPU more room within the 40W limit but the tight timings help a lot.

 

[Neilski]

The bug varies from going to sleep but when the PSU clicks, it instantly clicks again, the fans ramp up but the monitor stays black until eternity. Until you press and hold the power button, turn it on again and it will boot right into where you left it for sleep mode...

Freaky...

I like the sleep mode a lot to save the 100-150W idle power of the full PC.
I was a bit confused about sleep mode but I could switch off the PSU just fine and it woke up within the normal 3 seconds without issues.

Yeah, 100+W of idle power is definitely worth saving. Sounds pretty high too. Does your mention of 32W at idle for the CPU (below) mean that most of it is the rest of the machine, e.g. GPU etc?

NB: when you say "switch off the PSU", I guess you still just mean the press+hold on the power button? If so that's definitely bizarre, but may mean that the PSU never completely powers down (having not quite powered the machine up successfully?), so the sleep state remains in place somehow. If you could get the same result by actually switching off the power at a physical mains switch (back of PSU or wall socket) then I'd be astonished, cos the RAM should have lost its data a fraction of a second after the refreshes stopped (not sure when that would be, but maybe when the voltages sag a little?).

I tested how much power saving could be done via the energy saving power plan at 10% max CPU speed. All 6 cores were stuck at 3.0 GHz. You can't save much though, it's 30W instead of 32W at idle...
Of course I forgot about that power plan

Dropping the RAM voltage and MHz apparently gives the CPU more room within the 40W limit but the tight timings help a lot.

Wow, you've found a weird corner of the envelope there! Your testing showed that this wasn't the case with a 60W limit, if I'm reading it right? I am kinda surprised that it happens at any power level, as I'd have expected that the IMC wouldn't use anywhere near as much power as the cores of the CPU, and also I'd have thought that the RAM voltage should really only affect the RAM power dissipation rather than the power budget for the CPU itself. Likely I'm just way out of date on this stuff.

 

[RasmusP]

Freaky

Normal for new platforms. Alder Lake had some bugs too in early bios versions and big issues with ddr5 stability.
But they had their laptop CPU generation already at the same nanometers iirc so it wasn't a completely fresh platform for Intel.

But one has to admit that new AMD platforms/generations always have some issues like the x570 usb issues, ryzen 1xxx ddr4 issues etc.
Honestly I expected more weird things to happen than just some sleep mode bugs!
Everything else is super stable with zero issues yet and 5400 seems to be completely fine

Yeah, 100+W of idle power is definitely worth saving. Sounds pretty high too. Does your mention of 32W at idle for the CPU (below) mean that most of it is the rest of the machine, e.g. GPU etc?

Yeah it's 30W cpu, 20-30W rtx 3080, 3 fans, 4 sata ssd, 2 nvme ssd, keeping internet connection, Bluetooth and WiFi running.
PSU efficiency at low wattage etc.
I didn't dive into making windows shut off the drives or other parts yet though.
But from what I know 100-150W at idle is normal for desktop PCs!

guess you still just mean the press+hold on the power button? If so that's definitely bizarre, but may mean that the PSU never completely powers down (having not quite powered the machine up successfully?), so the sleep state remains in place somehow. If you could get the same result by actually switching off the power at a physical mains switch (back of PSU or wall socket) then I'd be astonished, cos the RAM should have lost its data a fraction of a second after the refreshes stopped (not sure when that would be, but maybe when the voltages sag a little?).

No, holding the power button would turn it back on first and then kill it during waking up. Not a good idea and can lead to Windows needing to repair itself via a Windows USB stick
I flipped the hardware switch at the back of the PSU and also switched off the power strip!
I need to try switching off the power strip overnight but it seems that there is "ram staying powered up" happening... Same with my i5 10600k!

Wow, you've found a weird corner of the envelope there! Your testing showed that this wasn't the case with a 60W limit, if I'm reading it right? I am kinda surprised that it happens at any power level, as I'd have expected that the IMC wouldn't use anywhere near as much power as the cores of the CPU, and also I'd have thought that the RAM voltage should really only affect the RAM power dissipation rather than the power budget for the CPU itself. Likely I'm just way out of date on this stuff.

Yeah it's "platform power" for AMD so memory controller, pci-e lanes etc. are all adding to it.
The idle wattage of the cores is below 1W according to ryzen Master and hwinfo64!
The SOC, which afaik is the memory controller takes 13-16W at 6000CL30 @ 1.35V and 5-9W at 5400CL30 @ 1.25V.
So the cores can use a few watt more.
The "60.xy" watt runs had no power limit set. It simply didn't use more than that in ACC.
With PBO +200 and 6000CL36 @ 1.35V ram, the platform used 71.8W on average so 70W held it back a tiny bit.
Timings don't seem to use more power, frequency does though.
And voltage ofc too.

At my "sweetspot" the restricted clockspeed stacks with the increased stability at heavier undervolting, which allows for more amps, increasing fps further.

 

[RasmusP]

Little addition, sorry for the double post!
This is how hwinfo64 looks like while typing this and listening to minimized spotify in the background.
Apparently I misread the SoC power... It might've been 6-10W at 1.1V and 4800CL40 and 16-17W with 1.35V and 6000CL30...

And this is during a race start in ACC:

 

[Neilski]

Apparently I misread the SoC power... It might've been 6-10W at 1.1V and 4800CL40 and 16-17W with 1.35V and 6000CL30...

That's a pretty enormous variation, albeit still a modest absolute power difference.

As for the hwinfo64 output, it's confusing as hell! The per-core power figures don't add up to match the "core power (SVI3...)" figure; the MISC power is another non-trivial unexplained chunk (doesn't seem to vary much with load?), and then the PPT and package figures are larger still with no obvious reason

 

[RasmusP]

That's a pretty enormous variation, albeit still a modest absolute power difference.

As for the hwinfo64 output, it's confusing as hell! The per-core power figures don't add up to match the "core power (SVI3...)" figure; the MISC power is another non-trivial unexplained chunk (doesn't seem to vary much with load?), and then the PPT and package figures are larger still with no obvious reason

Yeah no idea about all this.. But the overall power consumption is less than my i5 and the fps are awesome so... I'll take it hehe.

Btw I switched both the PSU and the power strip off overnight and when I pressed the power button of the pc, it showed the boot screen like a normal cold boot but then it took only a second from there to be back on the desktop with everything still running.

Seems to be some way faster hybernation instead of powered up RAM...
Would be interesting to test this on a HDD...
My Intel Windows was on a 970 evo plus nvme, my new Windows is on a WD Black 850X. The wake up times would allow for the full 32GB of RAM being read from the hard drive...

 

[Neilski]

Btw I switched both the PSU and the power strip off overnight and when I pressed the power button of the pc, it showed the boot screen like a normal cold boot but then it took only a second from there to be back on the desktop with everything still running.

OK, so that's 100% the thing that used to be quite unambiguously called hibernation. But you're saying it now happens when you ask Windows to suspend/sleep? (Which word was used?) Is this Windows 11?
Goddamn MS, sounds like they are relabelling things yet again. (Like when you ask vanilla Windows 10 to shut down and it decides to just log you off and hibernate, not bothering with the actual shutdown bit cos reboots are so slow )

Seems to be some way faster hybernation instead of powered up RAM...
Would be interesting to test this on a HDD...
My Intel Windows was on a 970 evo plus nvme, my new Windows is on a WD Black 850X. The wake up times would allow for the full 32GB of RAM being read from the hard drive...

Hmm, I'd say you probably wouldn't want to test on a HDD ever again - you may have forgotten just how awful it was, especially once Windows 10 came along (I had to abandon my last HDD boot drive a couple of years ago because Win10 performs so badly on a mechanical drive, despite it being a supported config.)
I've been hibernating my main box every night for some years, not because reboots are slow but because I want to resume exactly where I left off in all of my apps
From watching how quickly the machine comes back to life (SATA drives + 32 GB) I've concluded that Windows is sensible about which bits of RAM it writes to the hibernation file, typically writing way under half the contents of my RAM (e.g. omitting cached stuff).

 

[RasmusP]

OK, so that's 100% the thing that used to be quite unambiguously called hibernation.

Yeah now the fun bit: you can enable hibernation in the energy options to have sleep and hibernation for selection.
And no, the behavior is not identical, lol.
Hibernation is way slower!

But you're saying it now happens when you ask Windows to suspend/sleep? (Which word was used?) Is this Windows 11?

Yep. It is win11, but it was (and is) the same on my old win 10 both with the i5 and the R5.

Goddamn MS, sounds like they are relabelling things yet again. (Like when you ask vanilla Windows 10 to shut down and it decides to just log you off and hibernate, not bothering with the actual shutdown bit cos reboots are so slow )

Yeah not really since both are still there haha.
On laptops this makes a lot more sense!
My Surface Pro 8 goes to "modern standby", when putting it to "sleep".
Which means instant back on and battery drain.
After 16 hours it goes into hibernation and shows a boot logo etc. and takes about 20 seconds to resume. It drains 5-10% over night in sleep (modern standby) and 0% in hibernation.

About win 10 shut off:
Afaik you can hit shut down while holding down shift to do a real shut down. Similar to advanced restart when holding down shift.
Or you can disable "fast startup" in the energy settings.

Hmm, I'd say you probably wouldn't want to test on a HDD ever again - you may have forgotten just how awful it was, especially once Windows 10 came along (I had to abandon my last HDD boot drive a couple of years ago because Win10 performs so badly on a mechanical drive, despite it being a supported config.)

Haha yeah definitely! I had my fun with win8/10 "micro accesses" slowing down laptops to unusable degrees.
No cpu load, no throughput, enough ram headroom but disk usage showing 100%...

I've been hibernating my main box every night for some years, not because reboots are slow but because I want to resume exactly where I left off in all of my apps

Yep, same reason why I want to use sleep

From watching how quickly the machine comes back to life (SATA drives + 32 GB) I've concluded that Windows is sensible about which bits of RAM it writes to the hibernation file, typically writing way under half the contents of my RAM (e.g. omitting cached stuff).

Yeah it seems to only write what's actually used to the drive and maybe compressed too?
I wouldn't notice anyway though, since my new nvme can read with 7200 MB/s
I needed more space and planned to buy some cheap big ssd or external ssd but there were no deals at all.
However the wd black 850x 2TB dropped from 249€ to 179€ (with the be quiet nvme cooler as an addition).
Too good to buy something slightly cheaper but massively slower...

 

[Neilski]

Hibernation is way slower!

But my understanding is that only hibernation can allow a machine to resume from a fully-powered off state (talking only about standard desktops here , rather than laptops etc. which may do some funky things). This is weird...

Or you can disable "fast startup" in the energy settings.

Yeah, been doing that for the last few years on each new machine I use. (This reminds me: another fun thing that has bitten myself and my colleagues in the last year or two - when our Dell (work) laptops get confused in a certain way, and you decide to reboot to cure it, they can sit there with the whirling dots for several minutes waiting for something or other to respond, give up waiting, and put you right back at the lock screen. You think they have rebooted, but nope, cos Task Manager will immediately show hours/days of uptime. Massive facepalm Not sure if this is just Dell doing something funky or if it can happen on other machines. But the Dell already does some dumb stuff because a 4-second hold on the power button does sweet FA; you have to hold it for around 10 seconds to force a power-off.)

I wouldn't notice anyway though, since my new nvme can read with 7200 MB/s

Nice , but can it write fast enough for you to not notice that it's doing it on the sleep/hibernate? You could deliberately allocate nearly all of the 32 GB (probably loads of tools to do that, but a DIY way would be to allocate a large array in Python or similar), in order to force it to write a shedload.

I just can't get my head around what's going on here, but I'm very curious about it. DRAM needs constant refreshes (D for "dynamic") so without power, it's dead...

 

[Deleted member 1066209]

Hey Rasmus, thanks for sharing the results of your experiments and documenting it in such a clear format. I'm pleased to know that you've gotten a massive improvement from your new upgrades!

 

[RasmusP]

But my understanding is that only hibernation can allow a machine to resume from a fully-powered off state (talking only about standard desktops here , rather than laptops etc. which may do some funky things). This is weird...

Same for me. No idea what's going on, lol.
The only thing I know is that most laptops have the S1 state iirc., which works similar to phones when you switch off the display.
That's the real "sleep" mode since it still drains power, can get notifications but the power drain is massively reduced.

Desktop CPUs mostly only support S3, which is the hibernation state...

Maybe sleep on desktop CPUs with win 10/11 is just an optimized hibernation and therefore a bit quicker?

Nice , but can it write fast enough for you to not notice that it's doing it on the sleep/hibernate? You could deliberately allocate nearly all of the 32 GB (probably loads of tools to do that, but a DIY way would be to allocate a large array in Python or similar), in order to force it to write a shedload.

That's a good idea haha!
Gonna load one of these demo unreal engine projects. Last time it couldn't even run with my 16 GB.

Not sure it will make a difference though. The nvme writes at 6000+ MB/s

I just can't get my head around what's going on here, but I'm very curious about it. DRAM needs constant refreshes (D for "dynamic") so without power, it's dead...

Yep... it's definitely writing something to the drive instead of keeping the DRAM powered.
I'll come back after I did some testing!

 

[RasmusP]

Hey Rasmus, thanks for sharing the results of your experiments and documenting it in such a clear format. I'm pleased to know that you've gotten a massive improvement from your new upgrades!

It's a good way to practice my excel/google sheet skills and make data easy to understand
Happy that it's useful or interesting to others too!

One thing I noticed I forgot to mention is that I got my Thermalright cooler mounted already, without the official AM5 kit (end of December, 10€ shipping).
What might be interesting for some people who have a cooler that doesn't use the original AM4 backplate:

Spoiler: Story about learning to use full-carbide tools

I managed to mount my Thermalright Le Grand Macho. The four #6-32 x 3/4" (19mm) screws arrived within 2 days via padded letter envelope from a German online "inch shop".
I 3D printed new "distance-poles" without threading and wide enough for the #6-32 screws going through. Pretty easy!
The endboss was the Thermalright stainless steel frame though.

One of my best mates bought the same parts I bought but bought his standard Macho Rev. B a year later and got the AM4 screws.
However Thermalright used their own backplates for AM4 and M3 screws from both sides into the distance-poles (bottom screw through backplate+mobo, top screw through mounting frame into the distance-pole).

So same positions for the mounting holes, identical CPU height (since it's AM4 compatible!), different screws!
And 3D printed distance-poles needed!

#6-32 are 3.51mm diameter, the M3 holes were only 3.49mm.
Luckily I have a super cheap Dremel-Clone from Aldi but with a 10€ original Dremel milling bit, which is full carbide with half-round tip.
Widening the holes to 4mm was quite easy. Took 10 minutes total.

However MY frame had NO AM4 holes. I used his frame to trace the layout on paper.
Pinched the AM4 holes, put the paper on my frame and punch marked the positions.
Then I read about drilling into stainless steel and didn't want to wait for a carbide drill from Amazon.
Drove to a hardware store, bought a set of 2 standard HSS 4mm drills for 7€ and stopped at my best mate's dad's house to use his drill press.
Slowest rpm, some random standard oil, medium pressure.
The angled tip of the drill went in super easily, then it stopped cutting and we couldn't even get another 0.001mm deeper.
Took the second drill and veeeeery carefully tried to get it to cut at all. Nope, no chance!

Jumped back into the car, drove to another hardware store and bought a cobalt drill, which is explicitly for stainless steel. 5€.
They didn't have anything better...

I then managed to drill 1 hole with ease, but it stopped after the tip on the second hole......... Like WTF?!
However it managed to get the tip in for the other 2 holes.

I then had to choose between trying to "wolf down?!" the holes with my Dremel full carbide milling bit or spend another 9€ on a carbide drill and wait 3 days for it to arrive.

End of the story: I milled for 1.5h and although the holes are quite ugly, my Macho was mounted before midnight

The milling bit got stuck for a short moment while wanting to mill one hole a bit quicker.
I was quite sure that either the mill would break or my hands would break but to my surprise the milling bit was just a bit deformed but continued to mill just fine, LOL.
The punch through my hands/arms wasn't that strong either tbh. I expected more from 30k rpm

Stainless steel can be really tough to drill. I knew this, but I didn't expect 4mm into only a 2mm thick plate to be THAT hard.
I'm impressed by full carbide tools though. Apart from the deformed area in the middle of the mill bit, it's still super sharp and looks like new Oo

 

[RasmusP]

@Neilski I did some testing in the morning. Spoiler: it doesn't make any sense.

I used TestLimit from sysinternals and allocated 25GB of my ram with it.
When choosing 30GB, Windows decided to shove 25 of it into the page file...

I then took a stopwatch with 5s pre-time to really hit the 0 when starting.
With only 6GB ram used, going to sleep took 4.5s, waking up took 5.2s.
With 31 GB used, going to sleep took 4.2s and waking up took 5.1s.

So within my human error, lol.

However even with my nvme, writing more than 25GB should at least take 4 seconds.
Now the question is if it always writes the full 32GB onto my nvme, which would explain the slightly more than 4s.

I guess I'll have to clone my boot+win partitions to one of the 2.5" laptop HDDs I still have lying around and repeat my testing


Fun fact:
I tested to out my ram to 5600 MHz without changing anything else at all.
Result was the pc going "to sleep" but without the PSU clicking and also my mouse/keyboard would wake the monitor back up instantly.
I disallowed everything to wake my PC up, so I guess it simple shut off the monitor signal.

I'll try to find time to copy the partitions to a hdd. I'm curious about this hehe.

And I also can't wait for gigabyte to release a bios update or AMD to release a new agesa or chipset driver.
The last update already fixed the sleepmode for anything above default speeds and fixed random bsod issues (never buy amd stuff right after release! End of November was a good time though hehe).