Comment on Mirror seeing half the write IOPS on one disk than the other, is this normal?

themoonisacheese@sh.itjust.works ⁨4⁩ ⁨months⁩ ago

• lightrush@lemmy.ca ⁨4⁩ ⁨months⁩ ago

  I put the low IOPS disk in a good USB 3 enclosure, hooked to an on-CPU USB controller. Now things are flipped:

                                          capacity     operations     bandwidth 
  pool                                  alloc   free   read  write   read  write
  ------------------------------------  -----  -----  -----  -----  -----  -----
  storage-volume-backup                 12.6T  3.74T      0    563      0   293M
    mirror-0                            12.6T  3.74T      0    563      0   293M
      wwn-0x5000c500e8736faf                -      -      0    406      0   146M
      wwn-0x5000c500e8737337                -      -      0    156      0   146M
  

  You might be right about the link problem.

  Image

  Looking at the B350 diagram, the whole chipset is hooked via PCIe 3.0 x4 link to the CPU. The other pool (the source) is hooked via USB controller on the chipset. The SATA controller is also on the chipset so it also shares the chipset-CPU link. I’m pretty sure I’m also using all the PCIe links the chipset provides for SSDs. So that’s 4GB/s total for the whole chipset. Now I’m probably not saturating the whole link, in this particular workload, but perhaps there’s might be another related bottleneck.

  source

  • themoonisacheese@sh.itjust.works ⁨4⁩ ⁨months⁩ ago

    I’m not fully familiar with the overheads associated with all things going on on a chipset, but it’s not unreasonable to think that this workload, plus whatever the chipset has to do (hardware management tasks mostly), as well as the CPU’s other tasks on similar interfaces that might saturate the IO die/controller, would influence this.

    B350 isn’t a very fast chipset to begin with, and I’m willing to bet the CPU in such a motherboard isn’t exactly current-gen either. Are you sure you’re even running at PCIe 3.0 speeds too? There are 2.0 only CPUs available for AM4.

    source

    • lightrush@lemmy.ca ⁨4⁩ ⁨months⁩ ago

      B350 isn’t a very fast chipset to begin with

      For sure.

      I’m willing to bet the CPU in such a motherboard isn’t exactly current-gen either.

      It’s a Ryzen 9 5950X and I’m pretty proud how far I’ve managed to stretch this board. 😆

      Are you sure you’re even running at PCIe 3.0 speeds too?

      So given the CPU, it should be PCIe 3.0, but that doesn’t remove any of the queues/scheduling suspicions for the chipset.

      source

      • themoonisacheese@sh.itjust.works ⁨4⁩ ⁨months⁩ ago

        Oh wow congrats, I’m currently in the struggle of stretching an ab350m to accept a 4600G and failing.

        You’re right, you should hit PCIe 3 speeds and it’s weird, but the fact that the drives swap speeds depending on how they’re plugged in points to either drivers or the chipset.

        source

        • -> View More Comments