pct restore "simple" not using the stored rootfs information

[hvisage]

Good day,
Trying pct restore on the CLI today for the time, on PVE 7.0, it seems to only use the "local", not the rootfs config inside the backup, and I seem to need to provide the storage parameter to use the rootfs on the correct storage.

root@prox01-rbx8:/mnt/pve/backup-prox02-rbx8-500G/dump# pct restore 1244011 /mnt/pve/backup-prox01-rbx8-500G/dump/vzdump-lxc-1244011-2021_09_21-20_09_29.tar.zst
400 Parameter verification failed.
storage: storage 'local' does not support container directories
pct restore <vmid> <ostemplate> [OPTIONS]
root@prox01-rbx8:/mnt/pve/backup-prox02-rbx8-500G/dump# pvesm extractconfig backup-prox01-rbx8-500G:backup/vzdump-lxc-1244011-2021_09_21-20_09_29.tar.zst
#LXC for sage caddy01-ovh1
arch: amd64
cores: 2
cpuunits: 1000
features: nesting=1
hostname: sage-caddy01-ovh1
memory: 2048
nameserver: 10.4.220.1
net0: name=eth0,bridge=vmbr0,gw=10.4.220.1,hwaddr=c0:c0:ca:04:DC:0b,ip=10.4.220.11/24,tag=1244,type=veth
onboot: 1
ostype: ubuntu
rootfs: local-zfs:subvol-1244011-disk-0,size=10G
swap: 512

Spoiler

root@prox01-rbx8:/mnt/pve/backup-prox02-rbx8-500G/dump# pveversion -v
proxmox-ve: 7.0-2 (running kernel: 5.11.22-3-pve)
pve-manager: 7.0-11 (running version: 7.0-11/63d82f4e)
pve-kernel-5.11: 7.0-7
pve-kernel-helper: 7.0-7
pve-kernel-5.4: 6.4-4
pve-kernel-5.11.22-4-pve: 5.11.22-8
pve-kernel-5.11.22-3-pve: 5.11.22-7
pve-kernel-5.11.22-1-pve: 5.11.22-2
pve-kernel-5.4.124-1-pve: 5.4.124-1
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
ceph-fuse: 15.2.14-pve1
corosync: 3.1.5-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve1
libproxmox-acme-perl: 1.3.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-6
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-11
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
openvswitch-switch: 2.15.0+ds1-2
proxmox-backup-client: 2.0.9-2
proxmox-backup-file-restore: 2.0.9-2
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-9
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-3
pve-firmware: 3.3-1
pve-ha-manager: 3.3-1
pve-i18n: 2.5-1
pve-qemu-kvm: 6.0.0-4
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-13
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1

 

[oguz]

hi,

it seems to only use the "local", not the rootfs config inside the backup, and I seem to need to provide the storage parameter to use the rootfs on the correct storage.

yes that's the default behavior, if you're restoring a container on a different node where you don't have the same storage then it would also fail when parsing rootfs from the backup configuration.

400 Parameter verification failed. storage: storage 'local' does not support container directories pct restore <vmid> <ostemplate> [OPTIONS]

this error is just about the allowed content type for that storage. that can be configured in /etc/pve/storage.cfg or on the GUI: Datacenter -> Storage -> Edit -> Content to edit allowed content types for a specific storage

 

[hvisage]

hi,

yes that's the default behavior, if you're restoring a container on a different node where you don't have the same storage then it would also fail when parsing rootfs from the backup configuration.

but the problem here is that the "expected" is that it'll parse the rootfs/mpX/etc. and use that exact storage in that config.
I'd rather say that the manuals then need to be specific: "Simple mode will ALWAYS ONLY use local storage, no parsing the config, only the layout/sizes of the storages will be used all on local"

this error is just about the allowed content type for that storage. that can be configured in /etc/pve/storage.cfg or on the GUI: Datacenter -> Storage -> Edit -> Content to edit allowed content types for a specific storage

100% correct, as I do NOT want the local to be containing any containers/VM's storage, that had been disabled, I wanted it on the local-zfs (that is across the cluster) as in the config.


If the behaviour you describe and what I see is the "correct" behaviour, it's not what I'd expected as *I* read the documentation

If neither the rootfs parameter nor any of the optional mpX parameters are explicitly set, the mount point configuration from the backed up configuration file is restored using the following steps:

1. Extract mount points and their options from backup
2. Create volumes for storage backed mount points (on storage provided with the storage parameter, or default local storage if unset)

The assumptions (based on 1. and the start paragraph) is that the storage is part of the configurations being extracted, so even though I now see it's quite hidden in the parentheses that you need to specify the storage...

thus, for ANY container's backup that is not all on local, you have to ALWAYS manually fetch the config, set the storages based on the container's settings, for each and every restore, else it'll be forced into local, not based on the extracted config's storages?

I think that is a point to be enforced in the documentation, as it was a counter-intuitive effect:
1) backup a container that is using (varying) non-local storage
2) Restoring the container on the same host with the same storage
-> I _expected_ a simple restore to the same places
Result: Have to manually specify the storage as it is in the extractconfig to get the same placement/etc. as backed up.

 

[fabian]

the docs explicitly state this though, you even copied it:

2. Create volumes for storage backed mount points (on storage provided with the storage parameter, or default local storage if unset)

if you don't set the storage parameter, 'local' will be used as default..

 

[hvisage]

the docs explicitly state this though, you even copied it:

2. Create volumes for storage backed mount points (on storage provided with the storage parameter, or default local storage if unset)

if you don't set the storage parameter, 'local' will be used as default..

You missed the points:
1. It's counter intuitive
2. The Documentation is NOT clearly EXPLICIT about it... only in parathesis which in the English language is "optional"

https://www.google.com/search?q=parentheses+meaning&oq=paranthesis

a word or phrase inserted as an explanation or afterthought into a passage which is grammatically complete without it, in writing usually marked off by brackets, dashes, or commas.

 

[fabian]

I disagree if you restore a backup on the GUI it forces you to select a storage, there is no option for "same as backup" - this is the "easy to use" interface we offer, which is pretty self-explanatory. if you use the CLI/API, you are expected to read the docs to figure out what's what. the sentence in brackets is obviously a more detailed explanation of HOW the volumes are created and not "optional" - if you choose to ignore it, then you might be surprised when what happens is different from what you expect. the same will be true for lots of other parts of the docs though. the CLI has less safeguards in general to allow advanced usage (e.g., compare how removing guests or their volumes is handled on the GUI, vs. what the CLI/API offer).

I agree that it is unfortunate that the behaviour of container and VM restore is different (not just w.r.t. volumes) - it's mainly for historical reasons, and some parts of it CAN be fixed up. in this case, there is already an easy workaround, so it's not high priority to get this fixed. a patch implementing a "re-use storage from backed-up config" feature/mode would likely be accepted (if it follows the general guidelines we've set for contributions and doesn't break other functionality).

 

[oguz]

You missed the points:
1. It's counter intuitive
2. The Documentation is NOT clearly EXPLICIT about it... only in parathesis which in the English language is "optional"

https://www.google.com/search?q=parentheses+meaning&oq=paranthesis

thanks for your feedback, i've sent a patch on our docs repository to make it clearer [0]

[0]: https://lists.proxmox.com/pipermail/pve-devel/2021-September/050079.html

 

[hvisage]

if you use the CLI/API, you are expected to read the docs to figure out what's what. the sentence in brackets is obviously a more detailed explanation of HOW the volumes are created and not "optional"

I'm 100% with you @fabian and I did read the documentation. I'll refer you to the definition and use of brackets in writing: the sentence should stand on it's OWN... the brackets are OPTIONAL details, and that's why the simple explanation and the results I experienced "surprising".

Now I know you don't follow these writing standards, I'll read your documentations accordingly ;(

parenthesis | pəˈrɛnθɪsɪs | noun (plural parentheses | pəˈrɛnθɪsiːz | )

1 a word or phrase inserted as an explanation or afterthought into a passage which is grammatically complete without it, in writing usually marked off by brackets, dashes, or commas: in a challenging parenthesis, Wordsworth comments on the evil effects of contemporary developments.
• (parentheses) a pair of round brackets ( ) used to mark off a parenthetical word or phrase: the stage number is added in parentheses to the name or formula.

2 an interlude or interval: the three months of coalition government were a lamentable political parenthesis.

PHRASES
in parenthesis
as a digression or afterthought: in parenthesis I should say that I am passing quickly over the significance of these matters.
ORIGIN
mid 16th century: via late Latin from Greek, from parentithenai ‘put in beside’.

 

[fabian]

"grammatically complete" just means that the rest of the sentence is a proper, meaningful sentence on its own. it doesn't mean you should ignore the rest (or that you can without losing information). if there were no meaning/value in the sentence inside the parentheses, it would not be there. but I frankly have better things to do than argue about grammar, like fixing bugs and implementing new features (and I hope you do too )