Comment on Help with understanding memory usage discrepancy

• tal@lemmy.today ⁨1⁩ ⁨day⁩ ago

  I am guessing that htop is in the wrong.

  If you look in /proc/meminfo, you’ll see a MemAvailable field and a MemTotal field.

  It looks to me like this is approximately (I didn’t check top’s source) what top is using: MemTotal - MemAvailable.

  stackoverflow.com/…/difference-between-memfree-an…

  Rik van Riel’s comments when adding MemAvailable to /proc/meminfo:

  /proc/meminfo: MemAvailable: provide estimated available memory

  Many load balancing and workload placing programs check /proc/meminfo to estimate how much free memory is available. They generally do this by adding up “free” and “cached”, which was fine ten years ago, but is pretty much guaranteed to be wrong today.

  It is wrong because Cached includes memory that is not freeable as page cache, for example shared memory segments, tmpfs, and ramfs, and it does not include reclaimable slab memory, which can take up a large fraction of system memory on mostly idle systems with lots of files.

  Currently, the amount of memory that is available for a new workload, without pushing the system into swap, can be estimated from MemFree, Active(file), Inactive(file), and SReclaimable, as well as the “low” watermarks from /proc/zoneinfo.

  However, this may change in the future, and user space really should not be expected to know kernel internals to come up with an estimate for the amount of free memory.

  It is more convenient to provide such an estimate in /proc/meminfo. If things change in the future, we only have to change it in one place.

  Looking at the htop source:

  github.com/htop-dev/htop/blob/main/MemoryMeter.c

     /* we actually want to show "used + shared + compressed" */
     double used = this->values[MEMORY_METER_USED];
     if (isPositive(this->values[MEMORY_METER_SHARED]))
        used += this->values[MEMORY_METER_SHARED];
     if (isPositive(this->values[MEMORY_METER_COMPRESSED]))
        used += this->values[MEMORY_METER_COMPRESSED];
  
     written = Meter_humanUnit(buffer, used, size);
  

  It’s adding used, shared, and compressed memory, to get the amount actually tied up, but disregarding cached memory, which, based on the above comment, is problematic.

  top, on the other hand, is using the kernel’s MemAvailable directly.

  gitlab.com/procps-ng/procps/-/blob/…/free.c

  	printf(" %11s", scale_size(MEMINFO_GET(mem_info, MEMINFO_MEM_AVAILABLE, ul_int), args.exponent, flags & FREE_SI, flags & FREE_HUMANREADABLE));
  

  source

  • a_fancy_kiwi@lemmy.world ⁨1⁩ ⁨day⁩ ago

    Thank you for the detailed explanation

    source

    • tal@lemmy.today ⁨1⁩ ⁨day⁩ ago

      No problem. It was an interesting question that made me curious too.

      source

      • a_fancy_kiwi@lemmy.world ⁨1⁩ ⁨day⁩ ago

        Came across some more info that you might find interesting. If true, htop is ignoring the cache used by ZFS but accounting for everything else.

        link

        source

        • -> View More Comments
• vk6flab@lemmy.radio ⁨1⁩ ⁨day⁩ ago

  This is the job for the OS.

  You can run most Linux systems with stupid amounts of swap and the only thing you’ll notice is that stuff starts slowing down.

  In my experience, only in extremely rare cases are you smarter than the OS, and in 25+ years of using Linux daily I’ve seen it exactly once, where oomkiller killed running mysqld processes, which would have been fine if the developer had used transactions. Suffice to say, they did not.

  I used a 1 minute cron job to reprioritize the process, problem “solved” … for a system that hadn’t been updated for 12 years but was still live while we documented what it was doing and what was required to upgrade it.

  source