Hello, I recently upgraded Proxmox from 5.3 to 6.0 and the process went very smoothly. So first of all I wanted to thank the team for making the upgrade so easy. However I may have discovered a bug tonight when rebooting my server to replace a failed disk in my ZFS pool.
Upon coming back up from a reboot, my ZFS pool failed to mount. Upon further investigation, this was because the zfs-mount.service Systemd unit was in a failed state because the mount point that it wanted to mount the pool too already existed.
Curious, I thought. But no worries, I ensured that the directory contained no contents and deleted it, restarted the module, and things started up just find afterwards. However, I later discovered that my nfs-server.service Systemd unit had also entered a failed state for the same reason. I restarted it and it came back up just fine, since I had previously mounted the ZFS filesystem after cleaning up the strange directory.
Anyway, later on that evening I had to restart the server for another, unrelated, reason. And discovered on boot that both of those units had failed once more! And that mysterious directory was back again. However, it contained a subdirectory, `backups/`. Interesting I thought. I spent some time checking out my crontab and enabled Systemd units and could not track down the cause for this directory to be created on boot.
So I set up a `watch ls` on the directory where my mountpoint lived and then deleted the directory. A few moments later it showed up again. I repeated this several times before I realized that Proxmox itself must be creating the directory. I navigated to The Backups section for my Datacenter and disabled my VM backups all together and tried again. But the directory was still showing up. So I navigated to storage and disabled the Backups device entirely. After which the directory was no longer being created.
I did not notice this behavior in Proxmox 5.3, or any previous version of Proxmox that I have ran on my hardware. To me this seems like a bug, but I could be wrong. Is it not kosher to be putting your VM backups on the ZFS pool present within Proxmox? This race condition seems unavoidable if you choose to create the Backups device on your ZFS pool, as Proxmox starts up before the zfs-mount.service Unit attempts to mount the pool.
I'll proceed without VM backups enabled for the time being. But I would greatly appreciate some advice from the Proxmox team on whether or not this is user error or a legitimate bug. Thank you for your time!
Upon coming back up from a reboot, my ZFS pool failed to mount. Upon further investigation, this was because the zfs-mount.service Systemd unit was in a failed state because the mount point that it wanted to mount the pool too already existed.
Curious, I thought. But no worries, I ensured that the directory contained no contents and deleted it, restarted the module, and things started up just find afterwards. However, I later discovered that my nfs-server.service Systemd unit had also entered a failed state for the same reason. I restarted it and it came back up just fine, since I had previously mounted the ZFS filesystem after cleaning up the strange directory.
Anyway, later on that evening I had to restart the server for another, unrelated, reason. And discovered on boot that both of those units had failed once more! And that mysterious directory was back again. However, it contained a subdirectory, `backups/`. Interesting I thought. I spent some time checking out my crontab and enabled Systemd units and could not track down the cause for this directory to be created on boot.
So I set up a `watch ls` on the directory where my mountpoint lived and then deleted the directory. A few moments later it showed up again. I repeated this several times before I realized that Proxmox itself must be creating the directory. I navigated to The Backups section for my Datacenter and disabled my VM backups all together and tried again. But the directory was still showing up. So I navigated to storage and disabled the Backups device entirely. After which the directory was no longer being created.
I did not notice this behavior in Proxmox 5.3, or any previous version of Proxmox that I have ran on my hardware. To me this seems like a bug, but I could be wrong. Is it not kosher to be putting your VM backups on the ZFS pool present within Proxmox? This race condition seems unavoidable if you choose to create the Backups device on your ZFS pool, as Proxmox starts up before the zfs-mount.service Unit attempts to mount the pool.
I'll proceed without VM backups enabled for the time being. But I would greatly appreciate some advice from the Proxmox team on whether or not this is user error or a legitimate bug. Thank you for your time!
