If you want the actual memory usage of your VM you will have to setup a monitoring software like Zabbix, Icinga2 or prometheus in it's own lxc or vm and install the agent/node exporter in your vms/lxcs. If it turns out that they don't actually use most of their ram you can lower it in the VM/lxc settings.