Search results

  1. S

    Empty root user password allows login with any password

    If the root user doesn't have a password set, or is deleted (using passwd -d root), the Proxmox WebUI allows login with any arbitrary password. This could potentially be a huge vulnerability. For example, a Debian 12 server is delivered with a blank root password and SSH key only auth. And then...
  2. S

    Sudden Bulk stop of all VMs ?

    And yes, exactly the same here, some have uptime upwards of 3 months! However I think the common factor here is that only AMD Ryzen 7000 series appears to be affected. I was having crashes even with the CPU set to max. I assume it must be a problem with nested virtualization then.
  3. S

    Sudden Bulk stop of all VMs ?

    Glad to hear it's not just me! I had initially (wrongly?) suspected that Windows VMs were at fault, but we have other hypervisors on the same hardware which seem to be just fine, only two are affected. I deployed a spare one recently, and it also starts to reboot randomly after I live migrate...
  4. S

    Sudden Bulk stop of all VMs ?

    We have been having this problem on 2 hypervisors running 7950X3D and 7950X CPUs, we run about 50+ in total, 8 of which are 7950X3D's, and one is a 7950X. I am confident this is not a hardware problem, ran memtests, and after migrating all VMs to another server, this kept reoccurring. We use...
  5. S

    VM disks going readonly

    Disabling/enabling discard didn't seem to help much. It has gotten a lot less frequent recently though, not sure what has changed.
  6. S

    VM disks going readonly

    This might actually be unrelated, the timestamp in the VM is 20:03 EST, this server had a running backup and the backup had failed due to a hardware problem with the backup server at around the same time. This behaviour isn't really ideal :p (no iothread, VirtIO SCSI controller) INFO: 69%...
  7. S

    VM disks going readonly

    It happened on another VM, not sure if related. The output of this command above was 107. I grabbed kernel logs from the guest:
  8. S

    VM disks going readonly

    Thanks for your reply. It's not lack of space, there's enough space available on all these servers.
  9. S

    VM disks going readonly

    I'm not seeing anything interesting in the host, I should have grabbed logs from the guest but in an attempt to minimize downtime I didn't think of that. However, I did briefly go through the error messages on the VM console, and they were along the lines of sd hung task I believe. Perhaps this...
  10. S

    VM disks going readonly

    This is actually not from the large single VMs where the issue was rampant, this is from another hypervisor where I didn't notice issues before. iothread isn't enabled as well, and controller is VirtIO SCSI.
  11. S

    VM disks going readonly

    That's great, hopefully it is merged and rolled out ASAP.
  12. S

    VM disks going readonly

    Just noticed another stuck VM: { "QMP": { "version": { "qemu": { "micro": 2, "minor": 1, "major": 8 }, "package": "pve-qemu-kvm_8.1.2-6" }, "capabilities": [] } } { "return": {} } { "return": [ { "io-status": "ok"...
  13. S

    VM disks going readonly

    Interesting, thank you, I know this problem occurs on other applications when things get busy as the defaults are low, never really even thought that this would have been a problem on Proxmox. Maybe Proxmox should ship with higher defaults in future, such as 1048576?
  14. S

    VM disks going readonly

    I believe it is, it's a single VM on this host.
  15. S

    VM disks going readonly

    Thanks for your reply, here's the output from the PVE host: 41368 / 1024 kvm (PID 3721702) 2496 / 1024 pmxcfs (PID 1221) 660 / 1024 rrdcached (PID 1211) 216 / 1024 tuned (PID 1156) 148 / 1024...
  16. S

    VM disks going readonly

    From one such server: df -i / Filesystem Inodes IUsed IFree IUse% Mounted on /dev/sda1 671023104 86758 670936346 1% /
  17. S

    VM disks going readonly

    All are using ext4
  18. S

    VM disks going readonly

    Thank you very much for your reply. Unfortunately in most cases the guest console got spammed with systemd messages stating the disk was readonly. I was able to grab the error right after in one or two instances, I don't have a screenshot but it was something like this: validate_block_bitmap...
  19. S

    VM disks going readonly

    I have been observing VM disks going readonly randomly (every 5-7 days, it's not a regular pattern). This appears to only be happening to VMs with very large disks, such as 4TB or more. I have a fairly large deployment with over 50 hypervisors. The VMs have a local RAID 1/RAID 10 disk, I have...
  20. S

    vzdump appears to be slowing down VM IO

    My storage is regular raw files on a EXT4 partition. I'm primarily curious if disk writes are hijacked to write to both the backup and the actual disk, as limited bandwidth seems to aggravate this.