No common base to restore the job state

F

Fisk

New Member
Proxmox Subscriber
Apr 28, 2022
7
1
3
Hallo Zusammen,

nachdem wir gestern einiges an Probleme mit unserem Proxmox Cluster hatten, haben wir zwei VMs bei denen die Replikation nicht mehr ganz hin haut.

nocommonbase.png
Code: 
Dec  5 12:21:03 pve-1 pvescheduler[535603]: 109-3: got unexpected replication job error - No common base to restore the job state#012please delete jobid: 109-3 and create the job again

Dec  5 12:20:02 pve-2 pvescheduler[283052]: 114-1: got unexpected replication job error - No common base to restore the job state#012please delete jobid: 114-1 and create the job again

Löschen und neu erstellen der Jobs hat nicht geholfen. Wir hatten noch ein paar mehr VMs mit dem gleichen Fehler, dort hat das neu erstellen des Jobs geholfen.
Hat jemand eine Idee was wir versuchen könnten um die beiden Error zu lösen?

Beste Grüße
Fisk
 
Last edited:
Hi,
könntest Du nach dem Löschen des Jobs schauen, ob am Ziel-Node noch replizierte Disken von der VM übrig sind und die Ausgabe von cat /var/lib/pve-manager/pve-replication-state.json am Quell-Node posten? Bitte auch pveversion -v von beiden Nodes posten.
 
Code: 
root@pve-1:~# zfs list
NAME                    USED  AVAIL     REFER  MOUNTPOINT
ZFS                    2.02T  8.76T      104K  /ZFS
ZFS/subvol-101-disk-0  1.13G  6.87G     1.13G  /ZFS/subvol-101-disk-0
ZFS/subvol-125-disk-0  2.13G  37.9G     2.13G  /ZFS/subvol-125-disk-0
ZFS/subvol-128-disk-0  3.33G  4.67G     3.33G  /ZFS/subvol-128-disk-0
ZFS/vm-100-disk-0      20.0G  8.76T     20.0G  -
ZFS/vm-100-disk-1      18.1G  8.76T     18.1G  -
ZFS/vm-102-disk-0      46.6G  8.76T     46.6G  -
ZFS/vm-103-disk-0      77.9G  8.76T     77.9G  -
ZFS/vm-104-disk-0      49.2G  8.76T     49.2G  -
ZFS/vm-105-disk-0       126G  8.76T      126G  -
ZFS/vm-106-disk-0       193G  8.76T      191G  -
ZFS/vm-106-disk-1      48.2G  8.76T     40.8G  -
ZFS/vm-107-disk-0       128G  8.76T      128G  -
ZFS/vm-108-disk-0      87.0G  8.76T     87.0G  -
ZFS/vm-108-disk-1      22.3G  8.76T     22.3G  -
ZFS/vm-109-disk-0      54.0G  8.76T     54.0G  -
ZFS/vm-110-disk-0      21.8G  8.76T     21.7G  -
ZFS/vm-111-disk-0      66.4G  8.76T     66.4G  -
ZFS/vm-112-disk-0      15.4G  8.76T     14.8G  -
ZFS/vm-113-disk-0      78.1G  8.76T     77.9G  -
ZFS/vm-114-disk-0      18.9G  8.76T     18.9G  -
ZFS/vm-115-disk-0       185G  8.76T      185G  -
ZFS/vm-116-disk-0      68.6G  8.76T     40.5G  -
ZFS/vm-117-disk-0      87.2G  8.76T     87.2G  -
ZFS/vm-118-disk-0      12.3G  8.76T     12.3G  -
ZFS/vm-119-disk-0      58.1G  8.76T     46.7G  -
ZFS/vm-119-disk-1       104G  8.76T     82.0G  -
ZFS/vm-120-disk-0       143G  8.76T      121G  -
ZFS/vm-121-disk-0      37.3G  8.76T     27.3G  -
ZFS/vm-121-disk-1       272K  8.76T      192K  -
ZFS/vm-123-disk-1       592K  8.76T      592K  -
ZFS/vm-123-disk-2       147G  8.76T      147G  -
ZFS/vm-124-disk-0      10.1G  8.76T     10.1G  -
ZFS/vm-124-disk-1       592K  8.76T      592K  -
ZFS/vm-127-disk-0       115G  8.76T      115G  -
ZFS/vm-201-disk-0       592K  8.76T      592K  -
ZFS/vm-201-disk-1      23.9G  8.76T     23.9G  -
ZFS/vm-201-disk-2      4.06M  8.76T     4.06M  -
Code: 
root@pve-2:~# zfs list
NAME                    USED  AVAIL     REFER  MOUNTPOINT
ZFS                    1.72T  9.07T      112K  /ZFS
ZFS/subvol-126-disk-0  3.73G  36.4G     3.59G  /ZFS/subvol-126-disk-0
ZFS/subvol-129-disk-0  1.21G  6.79G     1.21G  /ZFS/subvol-129-disk-0
ZFS/subvol-129-disk-1  1.92G  6.08G     1.92G  /ZFS/subvol-129-disk-1
ZFS/vm-100-disk-0      20.0G  9.07T     20.0G  -
ZFS/vm-100-disk-1      18.1G  9.07T     18.1G  -
ZFS/vm-102-disk-0      46.6G  9.07T     46.6G  -
ZFS/vm-103-disk-0      77.9G  9.07T     77.9G  -
ZFS/vm-104-disk-0      49.2G  9.07T     49.2G  -
ZFS/vm-105-disk-0       126G  9.07T      126G  -
ZFS/vm-106-disk-0       191G  9.07T      191G  -
ZFS/vm-106-disk-1      40.9G  9.07T     40.8G  -
ZFS/vm-107-disk-0       130G  9.07T      128G  -
ZFS/vm-108-disk-0      87.0G  9.07T     87.0G  -
ZFS/vm-108-disk-1      22.3G  9.07T     22.3G  -
ZFS/vm-109-disk-0      54.9G  9.07T     54.0G  -
ZFS/vm-110-disk-0      21.7G  9.07T     21.7G  -
ZFS/vm-111-disk-0      66.4G  9.07T     66.4G  -
ZFS/vm-112-disk-0      15.2G  9.07T     14.8G  -
ZFS/vm-113-disk-0      77.9G  9.07T     77.9G  -
ZFS/vm-114-disk-0      19.1G  9.07T     18.9G  -
ZFS/vm-115-disk-0       185G  9.07T      185G  -
ZFS/vm-116-disk-0      66.1G  9.07T     40.5G  -
ZFS/vm-118-disk-0      12.4G  9.07T     12.3G  -
ZFS/vm-119-disk-0      57.9G  9.07T     46.7G  -
ZFS/vm-119-disk-1       103G  9.07T     82.0G  -
ZFS/vm-120-disk-0       144G  9.07T      121G  -
ZFS/vm-127-disk-0       115G  9.07T      115G  -
Code: 
root@pve-1:~# cat /var/lib/pve-manager/pve-replication-state.json
{"113":{"local/pve-3":{"last_node":"pve-1","duration":10.486406,"storeid_list":["ZFS"],"last_sync":1670203801,"fail_count":0,"last_try":1670203801,"last_iteration":1670203801},"local/pve-2":{"duration":8.92916,"last_node":"pve-1","storeid_list":["ZFS"],"last_sync":1670203921,"fail_count":0,"last_try":1670203921,"last_iteration":1670203921}},"119":{"local/pve-2":{"fail_count":0,"last_try":1670217630,"last_iteration":1670217600,"duration":15.613016,"last_node":"pve-1","storeid_list":["ZFS"],"last_sync":1670217630},"local/pve-3":{"last_sync":1670217615,"duration":14.848124,"last_node":"pve-1","storeid_list":["ZFS"],"last_iteration":1670217600,"fail_count":0,"last_try":1670217615}},"118":{"local/pve-2":{"last_iteration":1670205601,"last_sync":1670205601,"duration":12.113858,"last_node":"pve-1","fail_count":0,"last_try":1670205601,"storeid_list":["ZFS"]},"local/pve-3":{"fail_count":0,"duration":11.594364,"last_node":"pve-1","storeid_list":["ZFS"],"last_try":1670205721,"last_iteration":1670205721,"last_sync":1670205721}},"109":{"local/pve-3":{"duration":14.815107,"last_node":"pve-1","fail_count":0,"last_try":1670248814,"storeid_list":["ZFS"],"last_iteration":1670248803,"last_sync":1670248814}},"106":{"local/pve-2":{"last_try":1670202001,"storeid_list":["ZFS"],"duration":11.798418,"last_node":"pve-1","fail_count":0,"last_iteration":1670202001,"last_sync":1670202001},"local/pve-3":{"last_iteration":1670202001,"fail_count":0,"last_try":1670202013,"last_sync":1670202013,"duration":11.203025,"last_node":"pve-1","storeid_list":["ZFS"]}},"111":{"local/pve-3":{"last_iteration":1670248803,"fail_count":0,"last_try":1670248829,"last_sync":1670248829,"duration":9.913946,"last_node":"pve-1","storeid_list":["ZFS"]},"local/pve-2":{"fail_count":0,"duration":9.171922,"last_node":"pve-1","storeid_list":["ZFS"],"last_try":1670248839,"last_iteration":1670248803,"last_sync":1670248839}},"115":{"local/pve-3":{"last_try":1670248803,"fail_count":0,"last_iteration":1670248803,"storeid_list":["ZFS"],"last_node":"pve-1","duration":10.995752,"last_sync":1670248803},"local/pve-2":{"last_iteration":1670247063,"fail_count":0,"last_try":1670247063,"last_sync":1670247063,"last_node":"pve-1","duration":11.534716,"storeid_list":["ZFS"]}},"104":{"local/pve-2":{"last_sync":1670198401,"last_iteration":1670198401,"storeid_list":["ZFS"],"last_try":1670198401,"fail_count":0,"last_node":"pve-1","duration":4.326136},"local/pve-3":{"fail_count":0,"last_try":1670198521,"last_iteration":1670198521,"last_node":"pve-1","duration":4.506817,"storeid_list":["ZFS"],"last_sync":1670198521}},"105":{"local/pve-2":{"storeid_list":["ZFS"],"last_try":1670200201,"fail_count":0,"last_node":"pve-1","duration":4.479922,"last_iteration":1670200201,"last_sync":1670200201},"local/pve-3":{"last_try":1670200321,"storeid_list":["ZFS"],"last_node":"pve-1","duration":4.466561,"fail_count":0,"last_sync":1670200321,"last_iteration":1670200321}}}root@pve-1:~#
Code: 
root@pve-2:~# cat /var/lib/pve-manager/pve-replication-state.json
{"107":{"local/pve-1":{"last_iteration":1670200923,"fail_count":0,"storeid_list":["ZFS"],"last_node":"pve-2","last_sync":1670200923,"last_try":1670200923,"duration":5.862882},"local/pve-3":{"last_iteration":1670200803,"fail_count":0,"last_node":"pve-2","storeid_list":["ZFS"],"last_try":1670200803,"last_sync":1670200803,"duration":7.024595}},"100":{"local/pve-3":{"fail_count":0,"last_iteration":1670199003,"last_sync":1670199003,"last_try":1670199003,"storeid_list":["ZFS"],"last_node":"pve-2","duration":6.625966},"local/pve-1":{"last_iteration":1670242921,"fail_count":0,"duration":631.566525,"storeid_list":["ZFS"],"last_node":"pve-2","last_sync":1670242921,"last_try":1670242921}},"119":{"local/pve-3":{"last_node":"pve-2","storeid_list":["ZFS"],"last_try":1670146927,"last_sync":1670146927,"last_iteration":1670146921,"fail_count":0,"duration":763.613355}},"110":{"local/pve-1":{"duration":7.58614,"last_node":"pve-2","storeid_list":["ZFS"],"last_try":1670248801,"last_sync":1670248801,"last_iteration":1670248801,"fail_count":0},"local/pve-3":{"duration":5.596342,"last_iteration":1670248801,"fail_count":0,"last_node":"pve-2","storeid_list":["ZFS"],"last_try":1670248809,"last_sync":1670248809}},"120":{"local/pve-3":{"duration":18.785814,"last_iteration":1670206203,"fail_count":0,"storeid_list":["ZFS"],"last_node":"pve-2","last_sync":1670207815,"last_try":1670207815},"local/pve-1":{"duration":1611.847325,"last_sync":1670206203,"last_try":1670206203,"storeid_list":["ZFS"],"last_node":"pve-2","fail_count":0,"last_iteration":1670206203}},"108":{"local/pve-1":{"last_iteration":1670202603,"fail_count":0,"storeid_list":["ZFS"],"last_node":"pve-2","last_sync":1670202610,"last_try":1670202610,"duration":6.452575},"local/pve-3":{"last_sync":1670202603,"last_try":1670202603,"storeid_list":["ZFS"],"last_node":"pve-2","duration":6.948378,"fail_count":0,"last_iteration":1670202603}},"114":{"local/pve-3":{"last_try":1670204405,"last_sync":1670204405,"last_node":"pve-2","storeid_list":["ZFS"],"fail_count":0,"last_iteration":1670204403,"duration":12.51429}},"112":{"local/pve-1":{"duration":26.524742,"fail_count":0,"last_iteration":1670201403,"last_try":1670201403,"last_sync":1670201403,"last_node":"pve-2","storeid_list":["ZFS"]},"local/pve-3":{"duration":215.451941,"fail_count":0,"last_iteration":1670235781,"last_sync":1670235781,"last_try":1670235781,"storeid_list":["ZFS"],"last_node":"pve-2"}}}root@pve-2:~#
Code: 
root@pve-1:~# pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.13.19-6-pve)
pve-manager: 7.1-12 (running version: 7.1-12/b3c09de3)
pve-kernel-helper: 7.1-14
pve-kernel-5.13: 7.1-9
pve-kernel-5.11: 7.0-10
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-5-pve: 5.13.19-13
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.13.19-1-pve: 5.13.19-3
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-5-pve: 5.11.22-10
pve-kernel-5.11.22-4-pve: 5.11.22-9
ceph-fuse: 15.2.14-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.1
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-7
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-5
libpve-guest-common-perl: 4.1-1
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.1-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-2
proxmox-backup-client: 2.1.5-1
proxmox-backup-file-restore: 2.1.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-7
pve-cluster: 7.1-3
pve-container: 4.1-4
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-6
pve-ha-manager: 3.3-3
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.1-2
pve-xtermjs: 4.16.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1
Code: 
root@pve-2:~# pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.13.19-6-pve)
pve-manager: 7.1-12 (running version: 7.1-12/b3c09de3)
pve-kernel-helper: 7.1-14
pve-kernel-5.13: 7.1-9
pve-kernel-5.11: 7.0-10
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-5-pve: 5.13.19-13
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.13.19-1-pve: 5.13.19-3
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-5-pve: 5.11.22-10
pve-kernel-5.11.22-4-pve: 5.11.22-9
ceph-fuse: 15.2.14-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.1
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-7
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-5
libpve-guest-common-perl: 4.1-1
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.1-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-2
proxmox-backup-client: 2.1.5-1
proxmox-backup-file-restore: 2.1.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-7
pve-cluster: 7.1-3
pve-container: 4.1-4
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-6
pve-ha-manager: 3.3-3
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.1-2
pve-xtermjs: 4.16.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1
 
Ist das nach dem Löschen vom Job? Offenbar gibt es die Disk am Target noch und das macht dann wohl Probleme beim Neu-Erstellen.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back