came into work this morning to find on of my nodes was down and unable to connect to the iSCSI but the other node can connect fine.
my setup is that i ahve a readynas 2100 with two 512GB iSCSI shares - IMAGES-SCSI and IMAGES-TWO-SCSI, both with LVM ontop (IMAGES-LVM & IMAGES-TWO-LVM).
On the node where the iSCSI has failed, the system log shows:
any suggestions what i should check?
my setup is that i ahve a readynas 2100 with two 512GB iSCSI shares - IMAGES-SCSI and IMAGES-TWO-SCSI, both with LVM ontop (IMAGES-LVM & IMAGES-TWO-LVM).
On the node where the iSCSI has failed, the system log shows:
Code:
Aug 24 09:21:21 bellerophon kernel: connection16:0: detected conn error (1020)
Aug 24 09:21:21 bellerophon iscsid: Kernel reported iSCSI connection 16:0 error (1020) state (2)
Aug 24 09:21:25 bellerophon pvedaemon[3497]: starting vnc proxy UPID:bellerophon:00000DA9:0002DE3F:50373985:vncproxy:123:root@pam:
Aug 24 09:21:25 bellerophon pvedaemon[2472]: <root@pam> starting task UPID:bellerophon:00000DA9:0002DE3F:50373985:vncproxy:123:root@pam:
Aug 24 09:21:25 bellerophon pvedaemon[2471]: <root@pam> successful auth for user 'root@pam'
Aug 24 09:21:26 bellerophon pvedaemon[3497]: command '/bin/nc -l -p 5900 -w 10 -c '/usr/sbin/qm vncproxy 123 2>/dev/null'' failed: exit code 2
Aug 24 09:21:26 bellerophon pvedaemon[2472]: <root@pam> end task UPID:bellerophon:00000DA9:0002DE3F:50373985:vncproxy:123:root@pam: command '/bin/nc -l -p 5900 -w 10 -c '/usr/sbin/qm vncproxy 123 2>/dev/null'' f$
Aug 24 09:21:31 bellerophon pvedaemon[3500]: start VM 123: UPID:bellerophon:00000DAC:0002E042:5037398B:qmstart:123:root@pam:
Aug 24 09:21:31 bellerophon pvedaemon[2471]: <root@pam> starting task UPID:bellerophon:00000DAC:0002E042:5037398B:qmstart:123:root@pam:
Aug 24 09:21:32 bellerophon pvedaemon[3504]: starting vnc proxy UPID:bellerophon:00000DB0:0002E0B0:5037398C:vncproxy:123:root@pam:
Aug 24 09:21:32 bellerophon pvedaemon[2472]: <root@pam> starting task UPID:bellerophon:00000DB0:0002E0B0:5037398C:vncproxy:123:root@pam:
Aug 24 09:21:32 bellerophon pvedaemon[3504]: command '/bin/nc -l -p 5900 -w 10 -c '/usr/sbin/qm vncproxy 123 2>/dev/null'' failed: exit code 2
Aug 24 09:21:32 bellerophon pvedaemon[2472]: <root@pam> end task UPID:bellerophon:00000DB0:0002E0B0:5037398C:vncproxy:123:root@pam: command '/bin/nc -l -p 5900 -w 10 -c '/usr/sbin/qm vncproxy 123 2>/dev/null'' f$
Aug 24 09:22:44 bellerophon pvedaemon[3500]: WARNING: command '/usr/bin/iscsiadm --mode discovery --type sendtargets --portal 192.168.100.6' failed: exit code 255
Aug 24 09:22:58 bellerophon pvestatd[2681]: WARNING: command '/usr/bin/iscsiadm --mode node --targetname iqn.2012-03.ReadyNAS2:images --login' failed: exit code 255
Aug 24 09:22:58 bellerophon pvestatd[2681]: status update time (272.512 seconds)
Aug 24 09:23:00 bellerophon kernel: scsi23 : iSCSI Initiator over TCP/IP
Aug 24 09:23:00 bellerophon kernel: connection17:0: detected conn error (1020)
Aug 24 09:23:00 bellerophon pvestatd[2681]: WARNING: storage 'IMAGES-TWO-ISCSI' is not online
Aug 24 09:23:01 bellerophon iscsid: Kernel reported iSCSI connection 17:0 error (1020) state (2)
Aug 24 09:23:28 bellerophon pvedaemon[2471]: <root@pam> successful auth for user 'root@pam'
Aug 24 09:24:19 bellerophon pvedaemon[2474]: WARNING: command '/usr/bin/iscsiadm --mode node --targetname iqn.2012-03.ReadyNAS2:images-two --login' failed: exit code 255
Aug 24 09:24:20 bellerophon iscsid: session [iqn.2012-03.ReadyNAS2:images-two,192.168.100.6,3260] already running.
Aug 24 09:24:21 bellerophon pvedaemon[2474]: WARNING: storage 'IMAGES-ISCSI' is not online
Aug 24 09:24:23 bellerophon pvedaemon[2474]: WARNING: storage 'IMAGES-ISCSI' is not online
Aug 24 09:24:48 bellerophon pvestatd[2681]: WARNING: command '/usr/bin/iscsiadm --mode discovery --type sendtargets --portal 192.168.100.6' failed: exit code 255
Aug 24 09:24:50 bellerophon kernel: scsi24 : iSCSI Initiator over TCP/IP
Aug 24 09:24:50 bellerophon kernel: connection18:0: detected conn error (1020)
Aug 24 09:24:51 bellerophon iscsid: Kernel reported iSCSI connection 18:0 error (1020) state (2)
Aug 24 09:24:54 bellerophon pvedaemon[3500]: WARNING: command '/usr/bin/iscsiadm --mode node --targetname iqn.2012-03.ReadyNAS2:images-two --login' failed: exit code 255
Aug 24 09:24:54 bellerophon pvedaemon[3500]: can't activate LV '/dev/IMAGES-TWO-LVM/vm-123-disk-1': Skipping volume group IMAGES-TWO-LVM
Aug 24 09:24:54 bellerophon pvedaemon[2471]: <root@pam> end task UPID:bellerophon:00000DAC:0002E042:5037398B:qmstart:123:root@pam: can't activate LV '/dev/IMAGES-TWO-LVM/vm-123-disk-1': Skipping volume group I$
Aug 24 09:26:03 bellerophon kernel: connection18:0: detected conn error (1020)
Aug 24 09:26:04 bellerophon iscsid: Kernel reported iSCSI connection 18:0 error (1020) state (2)
Aug 24 09:27:03 bellerophon pvestatd[2681]: WARNING: command '/usr/bin/iscsiadm --mode node --targetname iqn.2012-03.ReadyNAS2:images --login' failed: exit code 255
Aug 24 09:27:35 bellerophon kernel: scsi25 : iSCSI Initiator over TCP/IP
Aug 24 09:28:13 bellerophon kernel: connection19:0: detected conn error (1020)
Aug 24 09:28:14 bellerophon iscsid: Kernel reported iSCSI connection 19:0 error (1020) state (2)
Aug 24 09:28:37 bellerophon pvedaemon[2474]: <root@pam> successful auth for user 'root@pam'
Aug 24 09:29:17 bellerophon pvestatd[2681]: WARNING: command '/usr/bin/iscsiadm --mode node --targetname iqn.2012-03.ReadyNAS2:images-two --login' failed: exit code 255
Aug 24 09:29:17 bellerophon pvestatd[2681]: status update time (378.261 seconds)
Aug 24 09:29:36 bellerophon kernel: scsi26 : iSCSI Initiator over TCP/IP
Aug 24 09:31:01 bellerophon pmxcfs[1497]: [dcdb] notice: data verification successful
any suggestions what i should check?
Last edited: