Recent content by robm

  1. robm

    VM live migration issue - api error (status = 400: api error (status = 596: ))

    Thanks for the tip. We added the IPs of the clusters to the firewall, and it worked! Just had to map the server names to the private network in /etc/hosts to force the migration over the PVN, and use detailed mapping, and everything went as expected. :)
  2. robm

    VM live migration issue - api error (status = 400: api error (status = 596: ))

    Tried the live VM migration between clusters, but got the error: api error (status = 400: api error (status = 596: )) What is required to be configured in each cluster to perform these type of migrations? Root ssh keys authorized? Firewall issues? Each server in a cluster uses a Let's Encrypt...
  3. robm

    Migration possible from PVE 7.4 to latest PVE 8?

    Testing out the new (and awesome) PDM. I was able to add two clusters, one which has PVE 7 servers, and one that has PVE 8 servers. Is it possible to live migrate VMs from the PVE 7 cluster to the PVE 8 cluster? Would help us perform upgrades and migrations more smoothly.
  4. robm

    qm remote-migrate hung up on fstrim

    fstrim manually takes a few seconds at most (SSD). Storage is ZFS. Each cluster has standalone VMs, both cluster are running latest Proxmox 7.4 with all packages updated. Same CPU in both servers (e3-1230).
  5. robm

    qm remote-migrate hung up on fstrim

    Tasklog for the migration attached
  6. robm

    qm remote-migrate hung up on fstrim

    Also, just did another test with a test VM, and the same sort of thing happened. It stopped at the fstrim, but this time it locked up the VM at 100% CPU (1 core), timed out after 10 minutes, and the VM was unresponsive on console. Had top stop/start it, at which point it was running on the new...
  7. robm

    qm remote-migrate hung up on fstrim

    Sure, I PM'ed you the full journal entries. Thanks.
  8. robm

    qm remote-migrate hung up on fstrim

    Old: Nov 09 09:07:39 swtrading-pve qm[1298464]: <root@pam> starting task UPID:swtrading-pve:0013D041:00755213:654CF5BB:qmigrate:123:root@pam: Nov 09 09:39:28 swtrading-pve qm[1298497]: migration problems Nov 09 09:39:28 swtrading-pve qm[1298504]: migration aborted Nov 09 09:39:28 swtrading-pve...
  9. robm

    qm remote-migrate hung up on fstrim

    Latest 7.4 proxmox, was moving a VM from one cluster to another using the "qm remote-migrate" command. First VM moved fine, second one got hung up: 2023-11-09 09:29:28 stopping NBD storage migration server on target. tunnel: -> sending command "nbdstop" to remote tunnel: <- got reply tunnel: ->...
  10. robm

    PVE7 / PBS2 - Backup Timeout (qmp command 'cont' failed - got timeout)

    We're seeing the timeouts too with a local PBS backup server for many standalone Proxmox servers (no NFS). We think it's because when the backups run on the PBS, the server can be slow to respond (GUI sometimes takes 15-20 seconds to load a new page), and this is tripping that 3 second timeout...
  11. robm

    VM live migration using lvm-thin with discard results in high I/O

    Thank you for looking into it. I wonder if there is a way to set an "ionice" setting on the disk zeroing to lessen its I/O impact when zeroing the disk. Since it's a live migration, a few extra minutes would not be an issue on the initial zeroing...
  12. robm

    VM live migration using lvm-thin with discard results in high I/O

    Sure, here are the task logs for the same VM being migrated 3 different ways. Also attached is the image of I/O load for these 3 tests (discard on first, discard off second, VM offline migration 3rd). The live migration with discard on really impacts other VMS performance during the zeroing of...
  13. robm

    VM live migration using lvm-thin with discard results in high I/O

    Sure. Task log with discard on: Migrate with discard enabled: ----------------------------- Proxmox Virtual Environment 7.0-11 Logs 2021-10-09 22:14:05 use dedicated network address for sending migration traffic (10.10.23.81) 2021-10-09 22:14:06 starting migration of VM 214 to node 'vps16'...
  14. robm

    VM live migration using lvm-thin with discard results in high I/O

    When doing a live VM migration (latest proxmox enterprise 7) from one server to another where both servers use local disks with lvm-thin (ext4 hardware SSD RAID-10), if the VM hard disk has "Discard" enabled, we find that the migration hammers the I/O of the target node until the first copy...
  15. robm

    VM clone high I/O with hardware RAID-10 SSD drives

    When we perform an offline VM clone, we see a very high I/O for 1-2 minutes, which causes other VMs to slow down considerably. Running the latest Proxmox 7. RAID-10 is using Intel VROC. SSDs are: Crucial/Micron Micron_1100_MTFDDAK2T0TBN Intel(R) Xeon(R) CPU E5-2680 v4 dual CPUs 512 GB RAM ext4...