Nested Virt - Proxmox 6.3.3 rebooting

[goseph]

Hi,

i have a netcup server with VMX enabled (Support can't help me since their host is not rebooting). I don't know the Linux Kernel Version of the Host yet.

Specs:
Proxmox 6.3.3:
Kernel Version Linux 5.4.78-2-pve #1 SMP PVE 5.4.78-2 (Thu, 03 Dec 2020 14:26:17 +0100)
16 GB RAM, 4 x Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz (1 Socket), 200GB SSD

Default Proxmox installation from Iso: lvm (ext4)
Using ifupdown2
About 10 small Linux VMs. No LXC used.

Every few weeks (about 2 weeks) my whole Proxmox is rebooting.
Since syslog will be gone after reboot how do i identify the problem? Please give as detailed information as possible.

Thanks a lot!

 

[Hannes Laimer]

Hey,

how much RAM is assigned to the VMs? Based on your description I'd assume the system runs out of memory, which then causes it to reboot. You can check the RAM usage every few days and see if the RAM is actually filling up.

something like tail -f /var/log/syslog > <some_file> could run in the background to preserve the syslog in case of a crash

 

[goseph]

Hello Hannes,

in average about 1GB of RAM each VM (some have 0,5GB, others 1,5GB). Right now (2 Days up): RAM usage 63.07% (9.86 GiB of 15.64 GiB). I do have swap as well of 8GB available.
Would it really reboot that easy?

Will all needed information really be in that sylog file?
I am trying this right now:

tail -f /var/log/syslog > /root/syslog.txt &

Thanks

 

[XueSheng]

@goseph, Could you resolve your issue? I'm also using proxmox at netcup (but only with lxc containers). It also reboots without any errors in the logs. I tried also the updated kernel 5.11 without any success. Netcup support says it must be an proxmox specific issue and cannot be resolved by them. Really annoying.

 

[goseph]

@goseph, Could you resolve your issue? I'm also using proxmox at netcup (but only with lxc containers). It also reboots without any errors in the logs. I tried also the updated kernel 5.11 without any success. Netcup support says it must be an proxmox specific issue and cannot be resolved by them. Really annoying.

NetCup says: they know of the reboots with nested virt. They told me we can try to move to a host to one with more recent Kernel. So maybe i will move to 5.10 or 5.11 soon. Don't know if they tell me which one.
They also say they don't sell nested virt anymore because of these problems. Neither for Intel nor AMD based KVM-VPS.
@XueSheng you pay for VMX, right?

 

[XueSheng]

@XueSheng you pay for VMX, right?

No, I'm not paying for VMX. For my usecase, LXC is enough (but must be stable, of course).

I don't get the logic of Netcup support, that it must be proxmox/client related, because the logs don't show any errors. My assumption is that their host must be involved somehow.

 

[goseph]

No, I'm not paying for VMX. For my usecase, LXC is enough (but must be stable, of course).

I don't get the logic of Netcup support, that it must be proxmox/client related, because the logs don't show any errors. My assumption is that their host must be involved somehow.

Yes, their Host is involved in this. They don't really see the reboot. That can be a problem.
I could not identify my reboots. They just happen. Every few days lately.
I have another server at PHP-Friends. It was rebooting as well but the support is much better and they really try to help to fix it. They moved me to a different host with more recent kernel. Fine so far.

 

[goseph]

NetCup now switched 2 days ago from Host with Kernel 4.X to Kernel 5.4.
Lets see how it goes.
To recap: Intel CPU and VMX

 

[XueSheng]

A couple of days ago my VPS was also migrated to another server with a more recent kernel (netcup support didn't mention any details) and so far no unexpected restarts. But it's too early to really say that it's running stable.

 

[goseph]

No rebooting here since weeks since moving the Host by Netcup. So far so good.
And phpfriends no problem either.

EDIT (07/07/2021)
Proxmox (6.3.3) rebooted again. Now after 43 Days.
NetCup uses Kernel 5.4.86 on Host.