M1 Mac Ram Usage Incorrect for Application Memory (System Ram, and M1 GPU Ram)

[sigkill]

Might not be exclusive to the M1.

htop version installed via homebrew:
htop 3.0.5

I'm not sure of the ramifications of this, but it seems as if disabling the hardware acceleration in Google Chrome free'd up ram that was not being shown as in use by htop, but was shown in use by the Apple "Activity Monitor" program.

Upon disabling hardware acceleration, I was able to get some ram back for my Application Memory problem, but then noticed that htop still seems to think the available ram is as follows:

The Application Memory killer popped up, and I looked at htop and found a nearly 4GB inconsistency from what htop had been reporting.

The weird part is that the memory was consumed, and I'm not sure on a hardware level if the GPU shares the system ram, but the Activity Monitor did show the memory being used at the time, I've replicated by opening a huge amount of tabs and re-enabling hardware acceleration here:

I suspect this bug would, if it is a bug, show up on the new iMac systems as well.

Will compare to my Intel based system later today to see if the reporting is off there too.

[sigkill]

Forgot to provide that I am running Big Sur 11.3 and 11.4 Beta (Both have the same issue), and have swap disabled.

[sigkill]

Just compared to an Intel Mac running 11.3 Big Sur:

Seems like the same issue is present here, where activity monitor sees 13GB in use, where htop report 9.17

Swap is enabled on this system, and appears to be reported correctly by htop and Activity Monitor.

[cgzones]

The code in question is:

htop/darwin/Platform.c

Lines 240 to 251 in 69cfaf2

	void Platform_setMemoryValues(Meter* mtr) {
	const DarwinProcessList* dpl = (const DarwinProcessList*)mtr->pl;
	const struct vm_statistics* vm = &dpl->vm_stats;
	double page_K = (double)vm_page_size / (double)1024;
	mtr->total = dpl->host_info.max_mem / 1024;
	mtr->values[0] = (double)(vm->active_count + vm->wire_count) * page_K;
	mtr->values[1] = (double)vm->purgeable_count * page_K;
	// mtr->values[2] = "shared memory, like tmpfs and shm"
	mtr->values[3] = (double)vm->inactive_count * page_K;
	// mtr->values[4] = "available memory"
	}

htop calculates the used memory as active_count + wire_count and the cached memory as inactive_count.
Probably the Activity Monitor treats part of the inactive_count as used memory, so its used memory is higher; cause the total amount of occupied memory (13.00GB + 2.98GB vs 1 space to the right of the yellow bar) seems equal.

[sigkill]

That evaluation makes sense, I will try and research some tidbits about what triggers the Application Memory killer, and post back what I find, if helpful.

It seems that the OS errs on the side of the "higher" value being consumed, and triggers the application termination program, even though as what you have defined above explains the issue.

I do think the memory that was not getting added to the total in htop was indeed in use though, the "compressed" vs "cached" metric in the Activity Monitor program is quite interesting. I'm going to research and find out if there are any man pages or technical documents on how compression related, and where the files are cached, and if that cache is part of the sum, or an additional metric.

Thank you for looking into the code so quickly.

[sigkill]

Pretty interesting explanation here: https://www.reddit.com/r/iPadPro/comments/nijycb/coming_from_the_first_gen_ipad_air_this_129_m1/gz3sln5/

"On the Mach Kernel used for macOS and iOS, Memory falls in tiers free, active, inactive, speculative, cached, wired, and deallocated (momentarily, before returned to free)

Mach offers something called Tiered Access. The Cache piece is a hold-over from the Xsan and Xgrid days where servers could be clustered together into one giant server farm easily. Data and Application Data fragments could be tiered into fast storage while slower, network addressed storage could hold the larger segments that are queried and swapped in as needed. It relied on Network being slow and Hard disk or RAM being really fast to keep the App Pool saturated.

inactive content is memory that an application deems useful, but not actively being queried. It exists on disk, and can be re-read if needed at the expense of load times.

Where things failed here: NAND was the Hard Disk and Cache. Data was being read from disk, stored into RAM, ran out of space in RAM so the system realizes “Oh, I have 256GB of Cache, I will use it too!”, so it starts flushing “inactive” content to cache (which is the disk).

Because the system /thinks/ it has 8GB or 16GB RAM + 256GB Cache, it doesn’t care to put applications to sleep, or deallocate RAM pages. Reading from disk is slow after all, and free memory is wasted memory as far as the system is concerned.

256GB of Cache becomes filled, so it realizes that it has exhausted the Cache space, so the system finally decides “I should finally start managing memory and start deallocating sleeping processes, unused elements, or invoking the Garbage Collection processes.” This frees up a little bit of space, letting Cache reclaim space, letting it start resume writing to NAND."