Recent content by SagnikS

  1. S

    VM disks going readonly

    This might actually be unrelated, the timestamp in the VM is 20:03 EST, this server had a running backup and the backup had failed due to a hardware problem with the backup server at around the same time. This behaviour isn't really ideal :p (no iothread, VirtIO SCSI controller) INFO: 69%...
  2. S

    VM disks going readonly

    It happened on another VM, not sure if related. The output of this command above was 107. I grabbed kernel logs from the guest:
  3. S

    VM disks going readonly

    Thanks for your reply. It's not lack of space, there's enough space available on all these servers.
  4. S

    VM disks going readonly

    I'm not seeing anything interesting in the host, I should have grabbed logs from the guest but in an attempt to minimize downtime I didn't think of that. However, I did briefly go through the error messages on the VM console, and they were along the lines of sd hung task I believe. Perhaps this...
  5. S

    VM disks going readonly

    This is actually not from the large single VMs where the issue was rampant, this is from another hypervisor where I didn't notice issues before. iothread isn't enabled as well, and controller is VirtIO SCSI.
  6. S

    VM disks going readonly

    That's great, hopefully it is merged and rolled out ASAP.
  7. S

    VM disks going readonly

    Just noticed another stuck VM: { "QMP": { "version": { "qemu": { "micro": 2, "minor": 1, "major": 8 }, "package": "pve-qemu-kvm_8.1.2-6" }, "capabilities": [] } } { "return": {} } { "return": [ { "io-status": "ok"...
  8. S

    VM disks going readonly

    Interesting, thank you, I know this problem occurs on other applications when things get busy as the defaults are low, never really even thought that this would have been a problem on Proxmox. Maybe Proxmox should ship with higher defaults in future, such as 1048576?
  9. S

    VM disks going readonly

    I believe it is, it's a single VM on this host.
  10. S

    VM disks going readonly

    Thanks for your reply, here's the output from the PVE host: 41368 / 1024 kvm (PID 3721702) 2496 / 1024 pmxcfs (PID 1221) 660 / 1024 rrdcached (PID 1211) 216 / 1024 tuned (PID 1156) 148 / 1024...
  11. S

    VM disks going readonly

    From one such server: df -i / Filesystem Inodes IUsed IFree IUse% Mounted on /dev/sda1 671023104 86758 670936346 1% /
  12. S

    VM disks going readonly

    All are using ext4
  13. S

    VM disks going readonly

    Thank you very much for your reply. Unfortunately in most cases the guest console got spammed with systemd messages stating the disk was readonly. I was able to grab the error right after in one or two instances, I don't have a screenshot but it was something like this: validate_block_bitmap...
  14. S

    VM disks going readonly

    I have been observing VM disks going readonly randomly (every 5-7 days, it's not a regular pattern). This appears to only be happening to VMs with very large disks, such as 4TB or more. I have a fairly large deployment with over 50 hypervisors. The VMs have a local RAID 1/RAID 10 disk, I have...
  15. S

    vzdump appears to be slowing down VM IO

    My storage is regular raw files on a EXT4 partition. I'm primarily curious if disk writes are hijacked to write to both the backup and the actual disk, as limited bandwidth seems to aggravate this.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!