Perhaps early Windows 11 was a bit different. Plus I disabled Superfetch and BITS which would otherwise max out CPU, and when still on HDD, that too.
Though I am not entirely sure if I also disabled something else.

It could idle at around 5% I believe.

Also, unused RAM is wasted RAM. Just because it’s used doesn’t mean it’s unusably full. Linux too will eat up entire RAM for caching, but it can be freed when needed.