Mirror seeing half the write IOPS on one disk than the other, is this normal?

Comments

• themoonisacheese@sh.itjust.works ⁨2⁩ ⁨months⁩ ago

  It might be that the data to both disks saturates a common link before the second disk reaches full iops capability, and thus the driver then writes at full speed on one disk and at half speed on the other, for twice as long.

  source

  • lightrush@lemmy.ca ⁨2⁩ ⁨months⁩ ago

    I put the low IOPS disk in a good USB 3 enclosure, hooked to an on-CPU USB controller. Now things are flipped:

                                            capacity     operations     bandwidth 
    pool                                  alloc   free   read  write   read  write
    ------------------------------------  -----  -----  -----  -----  -----  -----
    storage-volume-backup                 12.6T  3.74T      0    563      0   293M
      mirror-0                            12.6T  3.74T      0    563      0   293M
        wwn-0x5000c500e8736faf                -      -      0    406      0   146M
        wwn-0x5000c500e8737337                -      -      0    156      0   146M
    

    You might be right about the link problem.

    Image

    Looking at the B350 diagram, the whole chipset is hooked via PCIe 3.0 x4 link to the CPU. The other pool (the source) is hooked via USB controller on the chipset. The SATA controller is also on the chipset so it also shares the chipset-CPU link. I’m pretty sure I’m also using all the PCIe links the chipset provides for SSDs. So that’s 4GB/s total for the whole chipset. Now I’m probably not saturating the whole link, in this particular workload, but perhaps there’s might be another related bottleneck.

    source

    • themoonisacheese@sh.itjust.works ⁨2⁩ ⁨months⁩ ago

      I’m not fully familiar with the overheads associated with all things going on on a chipset, but it’s not unreasonable to think that this workload, plus whatever the chipset has to do (hardware management tasks mostly), as well as the CPU’s other tasks on similar interfaces that might saturate the IO die/controller, would influence this.

      B350 isn’t a very fast chipset to begin with, and I’m willing to bet the CPU in such a motherboard isn’t exactly current-gen either. Are you sure you’re even running at PCIe 3.0 speeds too? There are 2.0 only CPUs available for AM4.

      source

      • -> View More Comments
• Shadow@lemmy.ca ⁨2⁩ ⁨months⁩ ago

  Usually means a failing drive in my experience.

  source

  • lightrush@lemmy.ca ⁨2⁩ ⁨months⁩ ago

    Interesting. SMART looks pristine on both drives. Brand new drives - Exos X22. Doesn’t mean there isn’t an impending problem of course.

    source

    • Shadow@lemmy.ca ⁨2⁩ ⁨months⁩ ago

      You could just swap the two disks and see if it follows the drive or the link.

      If the drive, rma it. I don’t put a lot of faith in smart data.

      source

      • -> View More Comments
• Decronym@lemmy.decronym.xyz ⁨2⁩ ⁨months⁩ ago

  Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

  Fewer Letters	More Letters
  NVMe	Non-Volatile Memory Express interface for mass storage
  PCIe	Peripheral Component Interconnect Express
  SATA	Serial AT Attachment interface for mass storage
  SSD	Solid State Drive mass storage

  ---

  [Thread #958 for this sub, first seen 8th Sep 2024, 16:15] [FAQ] [Full list] [Contact] [Source code]

  source