Cross-cluster migration : PVE version required

YvainP · Thursday at 15:14

Hi,
I upgraded my PDM to version 1.0.1 and I'm looking for a compatibility matrix for cross-cluster migration.
What are the required PVE8 and PVE9 versions?
Is it possible to migrate from a PVE8 cluster to a PVE9 cluster?

Currently, I'm seeing this error message in the Target Storage dropdown box:
"Error: api error (status = 400: api returned unexpected data - failed to parse api response)"

dcsapak · Friday at 08:27

hi,

there is no compatibility matrix currently, but migrations from pve8 to pve9 should be working
just make sure that both are fully up to date (with respect to the major version)

YvainP said:
Currently, I'm seeing this error message in the Target Storage dropdown box:
"Error: api error (status = 400: api returned unexpected data - failed to parse api response)"

can you post the output of

Code:

pvesh get /nodes/NODENAME/storage --enabled 1 --format 1 --content images --output-format json-pretty

?

the error message indicates that pdm cannot parse some values returned by the PVE cluster (which would indicate our types/schema not matching up with reality)

YvainP · Friday at 10:50

Hi,
Here is the output on one node :

YAML:

  WARNING: Cannot find matching striped segment for VG_B1PSR1_L003_PVE_RCT003/vm-346-disk-3.
[
   {
      "active" : 1,
      "avail" : 72026902682927104,
      "content" : "iso,images",
      "enabled" : 1,
      "format" : [
         {
            "qcow2" : 1,
            "raw" : 1,
            "vmdk" : 1
         },
         "raw"
      ],
      "shared" : 1,
      "storage" : "NFS-9K",
      "total" : 7.20575940379279e+16,
      "type" : "nfs",
      "used" : 30691355000832,
      "used_fraction" : 0.000425928112235852
   },
   {
      "active" : 1,
      "avail" : 1775964782592,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L006_PVE_RCT006",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 7020124045312,
      "used_fraction" : 0.798096083686869
   },
   {
      "active" : 1,
      "avail" : 2241968734208,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L001_PVE_RCT001",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6554120093696,
      "used_fraction" : 0.745117542799732
   },
   {
      "active" : 1,
      "avail" : 332371474986,
      "content" : "rootdir,images",
      "enabled" : 1,
      "shared" : 0,
      "storage" : "local-lvm",
      "total" : 343110844416,
      "type" : "lvmthin",
      "used" : 10739369430,
      "used_fraction" : 0.0312999999993565
   },
   {
      "active" : 1,
      "avail" : 7845827313664,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L011_PVE_RCT011",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 950261514240,
      "used_fraction" : 0.108032278076304
   },
   {
      "active" : 1,
      "avail" : 8522284662784,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA3_L252_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 273804165120,
      "used_fraction" : 0.0311279445304606
   },
   {
      "active" : 1,
      "avail" : 2873328926720,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L002_PVE_RCT002",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 5922759901184,
      "used_fraction" : 0.673340164823611
   },
   {
      "active" : 1,
      "avail" : 1815693230080,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L005_PVE_RCT005",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6980395597824,
      "used_fraction" : 0.793579479970684
   },
   {
      "active" : 1,
      "avail" : 2268803891200,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L004_PVE_RCT004",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6527284936704,
      "used_fraction" : 0.742066737206811
   },
   {
      "active" : 1,
      "avail" : 2294582083584,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L008_PVE_RCT008",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6501506744320,
      "used_fraction" : 0.739136094635055
   },
   {
      "active" : 1,
      "avail" : 2744479907840,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L010_PVE_RCT010",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6051608920064,
      "used_fraction" : 0.687988609308533
   },
   {
      "active" : 1,
      "avail" : 8022994714624,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L013_PVE_RCT013",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 773094113280,
      "used_fraction" : 0.0878906669095358
   },
   {
      "active" : 1,
      "avail" : 7555917021184,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA3_L253_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 1240171806720,
      "used_fraction" : 0.14099127816738
   },
   {
      "active" : 1,
      "avail" : 7754559258624,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L014_PVE_RCT014",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 1041529569280,
      "used_fraction" : 0.118408259586458
   },
   {
      "active" : 1,
      "avail" : 6848321159168,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L015_PVE_RCT015",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 1947767668736,
      "used_fraction" : 0.221435652463747
   },
   {
      "active" : 1,
      "avail" : 6343662501888,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L012_PVE_RCT012",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 2452426326016,
      "used_fraction" : 0.278808726696361
   },
   {
      "active" : 1,
      "avail" : 3100962193408,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L007_PVE_RCT007",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 5695126634496,
      "used_fraction" : 0.647461246233581
   },
   {
      "active" : 1,
      "avail" : 2539395219456,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L003_PVE_RCT003",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6256693608448,
      "used_fraction" : 0.711304050113702
   },
   {
      "active" : 1,
      "avail" : 3087003549696,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L009_PVE_RCT009",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 5709085278208,
      "used_fraction" : 0.649048161052781
   }
]

And the output on another node from another cluster :

YAML:

[
   {
      "active" : 1,
      "avail" : 2244116217856,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L252_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6551972610048,
      "used_fraction" : 0.744873402058316
   },
   {
      "active" : 1,
      "avail" : 1947763474432,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L253_RCT",
      "total" : 8589930397696,
      "type" : "lvm",
      "used" : 6642166923264,
      "used_fraction" : 0.773250377563661
   },
   {
      "active" : 1,
      "avail" : 1929509863424,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L251_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6866578964480,
      "used_fraction" : 0.780640020675669
   },
   {
      "active" : 1,
      "avail" : 2093792362496,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L250_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6702296465408,
      "used_fraction" : 0.761963253957393
   },
   {
      "active" : 1,
      "avail" : 1846831742976,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L001_PVE2_RCT001",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6949257084928,
      "used_fraction" : 0.790039439220161
   },
   {
      "active" : 1,
      "avail" : 1990713147392,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L243_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6805375680512,
      "used_fraction" : 0.773682009545331
   },
   {
      "active" : 1,
      "avail" : 4709427445760,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L242_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 4086661382144,
      "used_fraction" : 0.464599830913463
   },
   {
      "active" : 1,
      "avail" : 2049768947712,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L008_PVE2_RCT008",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6746319880192,
      "used_fraction" : 0.766968139156408
   },
   {
      "active" : 1,
      "avail" : 2020777918464,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L006_PVE2_RCT006",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6775310909440,
      "used_fraction" : 0.770264039165516
   },
   {
      "active" : 1,
      "avail" : 1970312052736,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L003_PVE2_RCT003",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6825776775168,
      "used_fraction" : 0.776001346588777
   },
   {
      "active" : 1,
      "avail" : 2039031529472,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L244_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6757057298432,
      "used_fraction" : 0.768188842863485
   },
   {
      "active" : 1,
      "avail" : 1973533278208,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L249_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6822555549696,
      "used_fraction" : 0.775635135476654
   },
   {
      "active" : 1,
      "avail" : 72026902682927104,
      "content" : "images,iso",
      "enabled" : 1,
      "format" : [
         {
            "qcow2" : 1,
            "raw" : 1,
            "vmdk" : 1
         },
         "raw"
      ],
      "shared" : 1,
      "storage" : "NFS-9K",
      "total" : 7.20575940379279e+16,
      "type" : "nfs",
      "used" : 30691355000832,
      "used_fraction" : 0.000425928112235852
   },
   {
      "active" : 1,
      "avail" : 1876896514048,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L246_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6919192313856,
      "used_fraction" : 0.786621468840346
   },
   {
      "active" : 1,
      "avail" : 1816766971904,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L002_PVE2_RCT002",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6979321856000,
      "used_fraction" : 0.793457409599976
   },
   {
      "active" : 1,
      "avail" : 343110844416,
      "content" : "rootdir,images",
      "enabled" : 1,
      "shared" : 0,
      "storage" : "local-lvm",
      "total" : 343110844416,
      "type" : "lvmthin",
      "used" : 0,
      "used_fraction" : 0
   },
   {
      "active" : 1,
      "avail" : 1966017085440,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L247_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6830071742464,
      "used_fraction" : 0.776489628071608
   },
   {
      "active" : 1,
      "avail" : 1986418180096,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L005_PVE2_RCT005",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6809670647808,
      "used_fraction" : 0.774170291028162
   },
   {
      "active" : 1,
      "avail" : 1949910958080,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L007_PVE2_RCT007",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6846177869824,
      "used_fraction" : 0.778320683632223
   },
   {
      "active" : 1,
      "avail" : 1946689732608,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L248_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6849399095296,
      "used_fraction" : 0.778686894744346
   },
   {
      "active" : 1,
      "avail" : 1814619488256,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_L004_PVE2_RCT004",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6981469339648,
      "used_fraction" : 0.793701550341392
   },
   {
      "active" : 1,
      "avail" : 7946460430336,
      "content" : "images",
      "enabled" : 1,
      "format" : [
         {
            "qcow2" : 1,
            "raw" : 1,
            "subvol" : 1,
            "vmdk" : 1
         },
         "raw"
      ],
      "shared" : 0,
      "storage" : "BLCK-MIGRATION",
      "total" : 8793955991552,
      "type" : "dir",
      "used" : 847495561216,
      "used_fraction" : 0.0963725042552129
   },
   {
      "active" : 1,
      "avail" : 1954742796288,
      "content" : "images",
      "enabled" : 1,
      "shared" : 1,
      "storage" : "B1PSR1_PRA4_L245_RCT",
      "total" : 8796088827904,
      "type" : "lvm",
      "used" : 6841346031616,
      "used_fraction" : 0.777771366964038
   }
]

And attached is my version table

dcsapak · 2025-12-15T09:17:14+0100

hi,

thanks for posting the output.

The issue is that when encountering large values (such as the total of your nfs)

perl sometimes converts integers to floating point values like this:

YvainP said:
"total" : 7.20575940379279e+16,

And our rust parser expects an integer and fails.
i sent a patch on the pve side to fix this:

https://lore.proxmox.com/pve-devel/20251215081435.472283-1-d.csapak@proxmox.com/T/#u

but for your case, i guess the only way is to either fix the storage to not report such big numbers (i guess it's not really 64 PIB of storage?)
or to disable this storage for the time being, or to do the remote migration manually via the cli/api (this problem should only affect the drop down, not the actual migration)

YvainP · 2025-12-15T10:56:54+0100

Thanks for your analysis. It's a bug from our Purestorage array.
When creating NFS share we can't define a quota or limit and the array give us this funny size.
We expect it to be solved in the next upgrade.
I will try with CLI and tell you if it works.

Search

Search

Cross-cluster migration : PVE version required

YvainP

New Member

dcsapak

Proxmox Staff Member

YvainP

New Member

Attachments

dcsapak

Proxmox Staff Member

YvainP

New Member

We value your privacy