Proxmox VE rocks, except for this one thing, the memory usage graph.

[orwadira]

Idea: what if the RAM indicator/graph was not available unless you have a guest agent enabled in the VM and the OS-specific probe into "available" memory was built into the agent?

 

[Johannes S]

We are all users, regardless of whether we pay a subscription, or not. Business users are not first-class citizens and homelaber's are not 2nd-class citizens. What some users do not offer in terms of monetary support to the Proxmox project, they make up in volumes in terms of patches being sent and documentation/content being created (as most of Proxmox's wide reach is indeed created by the vast amount of content created for it, mostly by homelab'ers).

I'm not sure about the exact numbers but as far I know about 90% of development is done by staff members of Proxmox Server Solutions GmbH. Now this could mean that it's simply to hard to get patches accepted. Then however it shouldn't be hard to start a fork with all community-patches included (OpenBSD started like that as a fork of NetBSD, X11.org or libreoffice would be other examples). Up to now there is not such a thing so I guess that there isn't actually a demand for it. But my point wasn't that typical homelabber requests are invalid because they don't pay for PVE (there are enough companys who are freeloaders too), my point was that it's not worth the effort to fix a problem which isn't a problem in a professional run environment since PVE design is for usage in corporate or government infrastructure. It's the same reason why I don't think that it would be a good idea to invest large times to reduce wear amplification on consumer SSDs except low-effort changes.

I hence wholeheartedly invite you to reconsider this logic. We are all users and we must be able to consolidate and litigate issues taking our combined interest into mind. Not only that this dichotomy of users unethical, it is also inaccurate, as many people started as homelab'ers before they upgraded their use to start making money off Proxmox.

I sincerly hope that they have a basic understanding how everything works before they are charging money for it. BTW: I'm a homelabber myself, I'm not even part of the virtualization team at my workplace (and they run VMware anyhow sadly). But I'm using Linux as my daily driver on my private and professional infrastucture since I got my first own computer two decades ago so I'm not surprised that ProxmoxVE works exactly the same way like any other Linux distribution I know. I don't want this to change to make live easier for non-Linux-savy homelabbers, sorry, not sorry. There is simply no such thing as "our combined interest" in that regard, why should my wishes less legitimate as any other non-customer of Proxmox Server Solutions GmbH?

I wholeheartedly agree with this definition of what free means. Regardless of what is right to do. Failing to engage and acknowledge this simple perspective to me seems to speak of a problem in our community, which is manifesting in this thread.

Well in my book it's a "problem of our community" that people don't di research before asking questions already answered several times before (often enough even on the same day or week), click-baity videos how to do not-supported stuff like attaching NFS or CIFS networks storages to PBS over WAN or Reddit recommentations for using "hellish scripts" to run docker inside LXC containers despite the Proxmox developers recommend against Docker inside lxc. With other words: I'm not a fan of the way the homelab community at reddit handles things and I like that this forums community strifes for handling things in a professional way. People who start with a homelab before earning their living profit from this "problem of our community" too, because they learn how things actually work and are doing them right instead of "It works somehow/mostly, no idea why".

Some people are clearly prepared to go to incredible lengths to convince everyone that since the VM is using a great part of this RAM for caching, and since not allowing it to use as much cache would effect its function, then this amount of cache is "used memory", and hence, it is "unavailable memory". We disagree. Many people disagree (https://www.linuxatemyram.com/). Please acknowledge that this is not a universal point of view.

Meh, LnxBil actually referenced that page to show, that your "valid concerns" are actually not very valid.

Now that we know that it is misleading, we will start ignoring it, in favour of an internal probing method, but then, what use is this RAM utilisation bar indicator? maybe the right solution would be to remove it entirely.

Like explained by @LnxBil and @BobhWasatch in the thread it's not wrong, it simply not what people expect because they have no previous background in Linux. Now there is no shame in not knowing how Linux modus operandi for RAM and caches works but "ESX/Windows are different thus ProxmoxVE/Linux should behave like them" isn't a good argument.

I did not know what ballooning is until I did some reading. As far as I understood, this is an optional feature that requires an agent running inside the VM which is disabled by default.

Yes because operating systems (especially Windows) don't have them installed by default. Non the less it's best practice is to install them and is something the admin needs to do anyway. The guest tools or ballooning are not a replacement for a monitoring software though. Even the ballooning function to free memory on the fly is more like last resort and shouldn't be taken as the correct approach to deal with over-assigned memory to vms.

Refusing to acknowledge the quite-valid perspective of many people here is a blocker of innovation because it is preventing us from thinking of simple ways to deal with this misleading UI problem. Indeed, there might be very simple fixes but to be able to figure them out, we need to agree that the current situation can benefit from some improvement.

There is indeed a simple fix: Running a monitoring software inside the VM like it's best practice in any professional context. And with that the admins would see that their VM has way more memory assigned than it actually needs and could use the next maintenance window to adjust the assignment appropriately. I won't repeat the arguments, why the "misleading overview" is actually not missleading if you know how it works.

And a monitoring software (together with the guest tools) would be needed to be installed anyway.

I also think that when someone's point of view is acknowledged, they feel more understood which leads to more constructive dialogue. At the end of the day, you might be paying a subscription and you might think that your use of Proxmox more legit but I doubt that you would be prepared to spend a week diving into Proxmox's source code to fix an issue, whereas many homelab'ers (hobby users) are prepared to do such a thing.

If my employer would expect me to fix a problem I would definitively do that and I guess other professionals too. If my boss however thinks that I should ask the support instead of spending my time because we pay for support , I will follow these commands. Actually (not with ProxmoxVE but some commercial software) I did both in the past, depending on the situation and whether support could help us or not.

 

[orwadira]

Meh, LnxBil actually referenced that page to show, that your "valid concerns" are actually not very valid.

I do not have a concern. I am just trying to understand the situation for myself, and the conclusion I am reaching is that:

1. Some users understand that these gauges, when red, do not imply that the VM is short of memory.
2. Some users are misled by these gauges.

From a usability/support point of view, it is likely that new comers will always fall in category 2 before they move to category 1. In the process of transitioning, there will be noise, and with this noise, there will be overhead.

My 2 cents would be that the removal of the gauges is the best since it eliminates this confusion and does not seem to subtract any functionality. Those who know not to look, will not have to look, and those who are confused, won't be confused.

 

[chrcoluk]

Thank you for this. It is true. Yet, from a user standpoint, it would make more sense to display the buffer cache as free. Failing to acknowledge this, in my opinion, can come across (and indeed does come across) as dismissing user input.

I think this would be misleading, whilst the cache can be reallocated as and when its needed, it isnt unutilised memory.
Perhaps what you want is a 3rd metric on the graph for available memory?
Also with things like memory fragmentation, memory paging, all of this RAM is not necessarily available for new virtual machines. For a while I ran proxmox without no swap, and then started getting OOM's whilst proxmox was reporting multiple gigs as unutilised. I fixed it now with a very small zram page, and also a somewhat larger low priority page area backed by SSD, although as of yet, the SSD part has never been utilised, its happy just allocating a few 10s of MB to the zram.

 

[orwadira]

I think this would be misleading, whilst the cache can be reallocated as and when its needed, it isnt unutilised memory.
Perhaps what you want is a 3rd metric on the graph for available memory?
Also with things like memory fragmentation, memory paging, all of this RAM is not necessarily available for new virtual machines. For a while I ran proxmox without no swap, and then started getting OOM's whilst proxmox was reporting multiple gigs as unutilised. I fixed it now with a very small zram page, and also a somewhat larger low priority page area backed by SSD, although as of yet, the SSD part has never been utilised, its happy just allocating a few 10s of MB to the zram.

True. Available, not free. My bad.

It still fascinates me that someone had to make a dedicated page for this (https://www.linuxatemyram.com/). Still grateful for this link.

 

[BobhWasatch]

I think this would be misleading, whilst the cache can be reallocated as and when its needed, it isnt unutilised memory.

Once more, RAM that is owned by a VM is not "available" to the hypervisor. It is only "available" within the VM. Outside of the special case of KSM memory is not shared between VM's. If you want that, use a container.

Why is this so hard for people to grasp?