Problem setting pvesm shared

[greg]

Greeting

I can't figure out how to set/unset the flag parameter with pvesm:

# pvesm set POOL --shared 1
(ok, the word "shared" with nothing behind is added to storage.cfg)

# pvesm set POOL --shared 0
file /etc/pve/storage.cfg line 2 (section 'POOL') - unable to parse value of 'shared': unexpected property 'shared'

The documentation (https://pve.proxmox.com/wiki/Storage#_storage_configuration) is not precise but it seems that there shouldn't a value behind "shared" or "sparse" however the GUI created the config file with "sparse 1".

What is exactly the format of this option? how can I have pvesm to alter the files with no error?

Thanks in advance

Regards

 

[Stoiko Ivanov]

Which versions are you running? (`pveversion -v`) ?

For me `pvesm set <storagename> --shared 1` works (and puts a `shared 1` in <storagename> definition in /etc/pve/storage.cfg

Could you post your /etc/pve/storage.cfg ?

 

[greg]

Thanks for your answer.

$ pveversion -v
proxmox-ve: 5.3-1 (running kernel: 4.15.18-9-pve)
pve-manager: 5.3-6 (running version: 5.3-6/37b3c8df)
pve-kernel-4.15: 5.2-12
pve-kernel-4.15.18-9-pve: 4.15.18-30
pve-kernel-4.15.18-7-pve: 4.15.18-27
pve-kernel-4.15.18-5-pve: 4.15.18-24
pve-kernel-4.10.17-2-pve: 4.10.17-20
pve-kernel-4.10.17-1-pve: 4.10.17-18
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: not correctly installed
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-3
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-43
libpve-guest-common-perl: 2.0-18
libpve-http-server-perl: 2.0-11
libpve-storage-perl: 5.0-34
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.0.2+pve1-5
lxcfs: 3.0.2-2
novnc-pve: 1.0.0-2
proxmox-widget-toolkit: 1.0-22
pve-cluster: 5.0-31
pve-container: 2.0-31
pve-docs: 5.3-1
pve-edk2-firmware: 1.20181023-1
pve-firewall: 3.0-16
pve-firmware: 2.0-6
pve-ha-manager: 2.0-5
pve-i18n: 1.0-9
pve-libspice-server1: 0.14.1-1
pve-qemu-kvm: 2.12.1-1
pve-xtermjs: 1.0-5
pve-zsync: 1.7-2
qemu-server: 5.0-43
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.12-pve1~bpo1
​

I updated everything before asking.

So the format is actually "shared 1"?

Here is the storage.cfg:

zfspool: ct_B
pool ct_B/ct
content images,rootdir
shared 1
sparse 1
dir: local
path /var/lib/vz
content rootdir,vztmpl,backup,iso,images
maxfiles 0
shared 0

zfspool: local-zfs
pool rpool/data
content images,rootdir
shared 1
sparse 1

zfspool: ct_A
pool rpool/ct
content rootdir,images
sparse 1

dir: backup_D
path /ct_D/backup_D
content rootdir,vztmpl,backup,iso,images
maxfiles 0
shared 1

​

$ pvesm set local-zfs --shared 1
file /etc/pve/storage.cfg line 4 (section 'ct_B') - unable to parse value of 'shared': unexpected property 'shared'
update storage failed: error with cfs lock 'file-storage_cfg': section 'ct_B' - missing value for required option 'shared'



Note: identation is wrong in the post, but correct in the config file

 

[Stoiko Ivanov]

Thanks for the config and description!
you cannot set a zfspool as shared storage (a pool is always only on one server). - See https://pve.proxmox.com/pve-docs/chapter-pvesm.html#_storage_types

What do you want to accomplish with the shared storage?

However I could reproduce the error-message you got - we should change the code to prevent users from setting shared for non-shareable storages in the first place - I opened a bug-report for this: https://bugzilla.proxmox.com/show_bug.cgi?id=2042

Thanks for catching it!

P.S.: the indentation should work inside the [ code ] blocks (remove the spaces before/after the brackets)

 

[greg]

OK sorry I thought shared was common to all storage.
I was investigating a problem with replication, I have an error message:


()


Replication Log

2018-12-28 17:13:01 114-0: start replication job
2018-12-28 17:13:01 114-0: guest => CT 114, running => 1
2018-12-28 17:13:01 114-0: volumes => ct_B:subvol-114-disk-1
2018-12-28 17:13:02 114-0: (remote_prepare_local_job) file /etc/pve/storage.cfg line 4 (section 'ct_B') - unable to parse value of 'shared': unexpected property 'shared'
2018-12-28 17:13:02 114-0: freeze guest filesystem
2018-12-28 17:13:03 114-0: create snapshot '__replicate_114-0_1546013581__' on ct_B:subvol-114-disk-1
2018-12-28 17:13:03 114-0: thaw guest filesystem
2018-12-28 17:13:03 114-0: full sync 'ct_B:subvol-114-disk-1' (__replicate_114-0_1546013581__)
2018-12-28 17:13:03 114-0: file /etc/pve/storage.cfg line 4 (section 'ct_B') - unable to parse value of 'shared': unexpected property 'shared'
2018-12-28 17:13:03 114-0: file /etc/pve/storage.cfg line 4 (section 'ct_B') - unable to parse value of 'shared': unexpected property 'shared'
2018-12-28 17:13:03 114-0: ct_B/ct/subvol-114-disk-1 name ct_B/ct/subvol-114-disk-1 -
2018-12-28 17:13:03 114-0: volume 'ct_B/ct/subvol-114-disk-1' already exists
2018-12-28 17:13:03 114-0: cannot send 'ct_B/ct/subvol-114-disk-1': I/O error
2018-12-28 17:13:03 114-0: command 'zfs send -Rpv -- ct_B/ct/subvol-114-disk-1@__replicate_114-0_1546013581__' failed: exit code 1
2018-12-28 17:13:03 114-0: delete previous replication snapshot '__replicate_114-0_1546013581__' on ct_B:subvol-114-disk-1
2018-12-28 17:13:03 114-0: end replication job with error: command 'set -o pipefail && pvesm export ct_B:subvol-114-disk-1 zfs - -with-snapshots 1 -snapshot __replicate_114-0_1546013581__ | /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=sysv5' root@149.202.91.80 -- pvesm import ct_B:subvol-114-disk-1 zfs - -with-snapshots 1' failed: exit code 255

There was another line about this "shared" parameter so I looked that way.

 

[Stoiko Ivanov]

2018-12-28 17:13:03 114-0: file /etc/pve/storage.cfg line 4 (section 'ct_B') - unable to parse value of 'shared': unexpected property 'shared'

These errors are due to the presence of the shared flag in /etc/pve/storage.cfg for a storage which does not support it (line4).
Please remove the `shared` lines in /etc/pve/storage.cfg for storages that do not support it and try the replication job again.


* hmm - the exit code 255 could indicate a problem with ssh - can you ssh between the nodes (from the source of the replication to the target of the replication) without password?
* however I guess this is due to the following lines:

2018-12-28 17:13:03 114-0: cannot send 'ct_B/ct/subvol-114-disk-1': I/O error
2018-12-28 17:13:03 114-0: command 'zfs send -Rpv -- ct_B/ct/subvol-114-disk-1@__replicate_114-0_1546013581__' failed: exit code 1

* do you see any problems with the zpool? (dmesg/journal/zpool status)
* you could try to run the zfs send command above on the command line - it should output the stream to stdout (make sure you have enough space or redirect it to /dev/null) - maybe you'll find the error that way

* also check the receivers journal

 

[spamsam]

pvesm add lvmthin backup --thinpool backup --vgname hdd_pve