When i am logged into the master i only see vmbr0 and vmbr1 .. Even though the node i am trying to set the VM on has a vmbr2. If i log into this node directly i can setup the vmbr2 but not from the master node.
Master node not showing all VMBR
- Thread starter tylerdurden81
- Start date
No i can't. For one some nodes don't have the same amount of PCI slots as others. I have different networking requirements for the different nodes. So the master dostn poll the node and display that nodes configuration? Also i have been having alot of network problems. I have been trying Balance-rr and Balance-alb. But have had alot of problems with the VM's losing network connectivity. I also have been getting alot of MAC address errors. Either duplicate MAC address's or invalid MAC address's.
You only need to use the same bridge names on all nodes. Hint: there is no need to connect those bridges to any physical network.
Ahhh good idea
Would having different network setups across the nodes cause my networking issues?Also i have been getting messages like this when i create a VM or Migrate a VM. Also when i click on hardware tab sometimes it times out.
/usr/bin/ssh -t -t -n -o BatchMode=yes 192.168.2.80 /usr/sbin/qmigrate --online 192.168.2.81 220
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
May 16 09:37:27 starting migration of VM 220 to host '192.168.2.81'
May 16 09:37:27 copying disk images
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
May 16 09:37:31 starting VM on remote host '192.168.2.81'
May 16 09:37:35 starting migration tunnel
May 16 09:37:36 starting online/live migration
May 16 09:37:38 migration status: active (transferred 67092KB, remaining 455804KB), total 541056KB)
May 16 09:37:40 migration status: active (transferred 136119KB, remaining 359808KB), total 541056KB)
May 16 09:37:42 migration status: completed
May 16 09:37:42 migration speed: 85.33 MB/s
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
May 16 09:37:44 migration finished successfuly (duration 00:00:20)
Connection to 192.168.2.80 closed.
VM 220 migration done
/usr/bin/ssh -t -t -n -o BatchMode=yes 192.168.2.80 /usr/sbin/qmigrate --online 192.168.2.81 220
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
May 16 09:37:27 starting migration of VM 220 to host '192.168.2.81'
May 16 09:37:27 copying disk images
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
May 16 09:37:31 starting VM on remote host '192.168.2.81'
May 16 09:37:35 starting migration tunnel
May 16 09:37:36 starting online/live migration
May 16 09:37:38 migration status: active (transferred 67092KB, remaining 455804KB), total 541056KB)
May 16 09:37:40 migration status: active (transferred 136119KB, remaining 359808KB), total 541056KB)
May 16 09:37:42 migration status: completed
May 16 09:37:42 migration speed: 85.33 MB/s
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
May 16 09:37:44 migration finished successfuly (duration 00:00:20)
Connection to 192.168.2.80 closed.
VM 220 migration done
Hi,
one of your defined storage doesn't excist anymore.
You should delete the right one. Look with "dmsetup -info" and see what storage has the minor numer 65 (dm-65). This isn't available...
Udo
Last edited:
Also.. In a setup with multiple NIC's what nic is used to communicate with the Storage Machine? Can i create a bond that is only used for storage comunication?
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkr5RmVDAJ9iDs4e4T6JpPEjQX7HmjzNbH
Name: openfiler192.168.2.83-vm--192--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 5
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkXDu3fh0Kq7eUSbkfeKsZY5mLKbbcVXxO
Name: OPENFILER-vm--104--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 66
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFML2P7c0edomptbqDoUKLuRxYtapS3tGfkw
Name: openfiler3-vm--181--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 53
Number of targets: 1
UUID: LVM-kPMqBGJjEMGXAYcw8zwjTPMfH5Qlf671xVwCcX70cMiiPCzoFCi8cggclWh26Lsr
Name: openfiler192.168.2.83-vm--205--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 17
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakk7lArdEjZq215G2dY8CcYc4Fu5N2u110X
Name: openfiler3-vm--148--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 44
Number of targets: 1
UUID: LVM-kPMqBGJjEMGXAYcw8zwjTPMfH5Qlf671L3q5elJuYNGFLYzz4Z5hMRxwXKV56cRs
Name: OPENFILER-vm--130--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 89
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLfkiskZc6JRXQ0un3z2jGzeeTRnnGMcEU
Name: OPENFILER-vm--139--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 97
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLpR1tW1EefT29KACbQkmOJlOs5Ed8Sx21
Name: OPENFILER-vm--124--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 84
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLe6qPwLG6V39lF71tp7BD1LsLX1uN3ga2
Name: openfiler192.168.2.83-vm--210--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 25
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkGhMfHyqmcf3XTfSi4IK2dHIUMZ0dW96c
Name: openfiler192.168.2.83-vm--191--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 3
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakk8cjn8a5vfW40XTxsI6TOLUXRG5ZVubl5
Name: OPENFILER-vm--118--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 80
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLZNlL4ZF381hFwcO2vnE3iX1nYrB1OJm6
Name: OPENFILER-vm--103--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 68
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLOmgIhFpdyMqjcW97ckDKWlv4Ul3x5CtL
Name: openfiler192.168.2.83-vm--204--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 18
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkh5Ha4eDl0ogfmLj0m7Oqvl60DZEblxPU
Name: openfiler3-vm--147--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 43
Number of targets: 1
UUID: LVM-kPMqBGJjEMGXAYcw8zwjTPMfH5Qlf671ex2UzswfpbJCBWBURPwrkvU3riouhvKJ
Name: OPENFILER-vm--150--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 100
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLsYtZ0KLr2rtNH60wxSdqZWthIDI6NVpL
Name: openfiler5lvm-vm--220--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 102
Number of targets: 1
UUID: LVM-LiIIZJsV3o4cBLPXNx51a8qw8T2iBXaT1U6WLZW3sqzq69Pyd67icJgY4qdoc8XA
Name: OPENFILER-vm--117--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 79
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFML1TROOi0z818h3fwcfLDtP3AtLudjfjSd
Name: OPENFILER-vm--102--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 73
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLIf5kAKvjIGdMRfTkf7NvNDOBfouKQEB4
Name: openfiler192.168.2.83-vm--218--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 34
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkqYsGuSb2N0fdLd438vnEOpZaKbBBjiAl
Name: openfiler192.168.2.83-vm--199--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 13
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkr85vQwVPfnisiaThfhwHPq5p9dTgcU0U
Name: OPENFILER-vm--101--disk--2
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 63
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLTXVqmupk3RVjOgFvhlBGcYLrP21rzY3Q
Name: openfiler3-vm--146--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 42
Number of targets: 1
UUID: LVM-kPMqBGJjEMGXAYcw8zwjTPMfH5Qlf671DEDPlrphSnizzIYUwZruzGTt9kS5DLCc
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkr5RmVDAJ9iDs4e4T6JpPEjQX7HmjzNbH
Name: openfiler192.168.2.83-vm--192--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 5
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkXDu3fh0Kq7eUSbkfeKsZY5mLKbbcVXxO
Name: OPENFILER-vm--104--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 66
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFML2P7c0edomptbqDoUKLuRxYtapS3tGfkw
Name: openfiler3-vm--181--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 53
Number of targets: 1
UUID: LVM-kPMqBGJjEMGXAYcw8zwjTPMfH5Qlf671xVwCcX70cMiiPCzoFCi8cggclWh26Lsr
Name: openfiler192.168.2.83-vm--205--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 17
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakk7lArdEjZq215G2dY8CcYc4Fu5N2u110X
Name: openfiler3-vm--148--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 44
Number of targets: 1
UUID: LVM-kPMqBGJjEMGXAYcw8zwjTPMfH5Qlf671L3q5elJuYNGFLYzz4Z5hMRxwXKV56cRs
Name: OPENFILER-vm--130--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 89
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLfkiskZc6JRXQ0un3z2jGzeeTRnnGMcEU
Name: OPENFILER-vm--139--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 97
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLpR1tW1EefT29KACbQkmOJlOs5Ed8Sx21
Name: OPENFILER-vm--124--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 84
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLe6qPwLG6V39lF71tp7BD1LsLX1uN3ga2
Name: openfiler192.168.2.83-vm--210--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 25
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkGhMfHyqmcf3XTfSi4IK2dHIUMZ0dW96c
Name: openfiler192.168.2.83-vm--191--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 3
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakk8cjn8a5vfW40XTxsI6TOLUXRG5ZVubl5
Name: OPENFILER-vm--118--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 80
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLZNlL4ZF381hFwcO2vnE3iX1nYrB1OJm6
Name: OPENFILER-vm--103--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 68
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLOmgIhFpdyMqjcW97ckDKWlv4Ul3x5CtL
Name: openfiler192.168.2.83-vm--204--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 18
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkh5Ha4eDl0ogfmLj0m7Oqvl60DZEblxPU
Name: openfiler3-vm--147--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 43
Number of targets: 1
UUID: LVM-kPMqBGJjEMGXAYcw8zwjTPMfH5Qlf671ex2UzswfpbJCBWBURPwrkvU3riouhvKJ
Name: OPENFILER-vm--150--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 100
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLsYtZ0KLr2rtNH60wxSdqZWthIDI6NVpL
Name: openfiler5lvm-vm--220--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 102
Number of targets: 1
UUID: LVM-LiIIZJsV3o4cBLPXNx51a8qw8T2iBXaT1U6WLZW3sqzq69Pyd67icJgY4qdoc8XA
Name: OPENFILER-vm--117--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 79
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFML1TROOi0z818h3fwcfLDtP3AtLudjfjSd
Name: OPENFILER-vm--102--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 73
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLIf5kAKvjIGdMRfTkf7NvNDOBfouKQEB4
Name: openfiler192.168.2.83-vm--218--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 34
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkqYsGuSb2N0fdLd438vnEOpZaKbBBjiAl
Name: openfiler192.168.2.83-vm--199--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 13
Number of targets: 1
UUID: LVM-B3SHjCEAE6dcFMqS71it6BdR4IdbXakkr85vQwVPfnisiaThfhwHPq5p9dTgcU0U
Name: OPENFILER-vm--101--disk--2
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 63
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLTXVqmupk3RVjOgFvhlBGcYLrP21rzY3Q
Name: openfiler3-vm--146--disk--1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 42
Number of targets: 1
UUID: LVM-kPMqBGJjEMGXAYcw8zwjTPMfH5Qlf671DEDPlrphSnizzIYUwZruzGTt9kS5DLCc
proxmox1:/dev# pvscan
/dev/dm-7: read failed after 0 of 4096 at 214748299264: Input/output error
/dev/dm-7: read failed after 0 of 4096 at 214748356608: Input/output error
/dev/dm-7: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-7: read failed after 0 of 4096 at 4096: Input/output error
/dev/dm-7: read failed after 0 of 4096 at 0: Input/output error
PV /dev/sde VG openfiler5lvm lvm2 [1.82 TB / 1.76 TB free]
PV /dev/sdd VG openfiler3 lvm2 [1.16 TB / 262.34 GB free]
PV /dev/sdc VG openfiler192.168.2.83 lvm2 [1.16 TB / 48.34 GB free]
PV /dev/sdb VG OPENFILER lvm2 [1.80 TB / 308.18 GB free]
PV /dev/sda2 VG pve lvm2 [74.00 GB / 4.00 GB free]
Total: 5 [6.02 TB] / in use: 5 [6.02 TB] / in no VG: 0 [0 ]
/dev/dm-7: read failed after 0 of 4096 at 214748299264: Input/output error
/dev/dm-7: read failed after 0 of 4096 at 214748356608: Input/output error
/dev/dm-7: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-7: read failed after 0 of 4096 at 4096: Input/output error
/dev/dm-7: read failed after 0 of 4096 at 0: Input/output error
PV /dev/sde VG openfiler5lvm lvm2 [1.82 TB / 1.76 TB free]
PV /dev/sdd VG openfiler3 lvm2 [1.16 TB / 262.34 GB free]
PV /dev/sdc VG openfiler192.168.2.83 lvm2 [1.16 TB / 48.34 GB free]
PV /dev/sdb VG OPENFILER lvm2 [1.80 TB / 308.18 GB free]
PV /dev/sda2 VG pve lvm2 [74.00 GB / 4.00 GB free]
Total: 5 [6.02 TB] / in use: 5 [6.02 TB] / in no VG: 0 [0 ]
proxmox10:~# pvscan
/dev/dm-65: read failed after 0 of 4096 at 214748299264: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 214748356608: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 4096: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
PV /dev/sde VG openfiler5lvm lvm2 [1.82 TB / 1.76 TB free]
PV /dev/sdd VG OPENFILER lvm2 [1.80 TB / 308.18 GB free]
PV /dev/sdc VG openfiler3 lvm2 [1.16 TB / 262.34 GB free]
PV /dev/sdb VG openfiler192.168.2.83 lvm2 [1.16 TB / 48.34 GB free]
PV /dev/sda2 VG pve lvm2 [297.59 GB / 3.99 GB free]
Total: 5 [6.24 TB] / in use: 5 [6.24 TB] / in no VG: 0 [0 ]
proxmox9:~# pvscan
PV /dev/sde VG openfiler5lvm lvm2 [1.82 TB / 1.76 TB free]
PV /dev/sdd VG OPENFILER lvm2 [1.80 TB / 308.18 GB free]
PV /dev/sdc VG openfiler192.168.2.83 lvm2 [1.16 TB / 48.34 GB free]
PV /dev/sdb VG openfiler3 lvm2 [1.16 TB / 262.34 GB free]
PV /dev/sda2 VG pve lvm2 [297.59 GB / 3.99 GB free]
Total: 5 [6.24 TB] / in use: 5 [6.24 TB] / in no VG: 0 [0 ]
proxmox8:~# pvscan
PV /dev/sdd VG openfiler192.168.2.83 lvm2 [1.16 TB / 48.34 GB free]
PV /dev/sdc VG OPENFILER lvm2 [1.80 TB / 308.18 GB free]
PV /dev/sdb VG openfiler3 lvm2 [1.16 TB / 262.34 GB free]
PV /dev/sda2 VG pve lvm2 [297.59 GB / 3.99 GB free]
Total: 4 [4.42 TB] / in use: 4 [4.42 TB] / in no VG: 0 [0 ]
/dev/dm-65: read failed after 0 of 4096 at 214748299264: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 214748356608: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 4096: Input/output error
/dev/dm-65: read failed after 0 of 4096 at 0: Input/output error
PV /dev/sde VG openfiler5lvm lvm2 [1.82 TB / 1.76 TB free]
PV /dev/sdd VG OPENFILER lvm2 [1.80 TB / 308.18 GB free]
PV /dev/sdc VG openfiler3 lvm2 [1.16 TB / 262.34 GB free]
PV /dev/sdb VG openfiler192.168.2.83 lvm2 [1.16 TB / 48.34 GB free]
PV /dev/sda2 VG pve lvm2 [297.59 GB / 3.99 GB free]
Total: 5 [6.24 TB] / in use: 5 [6.24 TB] / in no VG: 0 [0 ]
proxmox9:~# pvscan
PV /dev/sde VG openfiler5lvm lvm2 [1.82 TB / 1.76 TB free]
PV /dev/sdd VG OPENFILER lvm2 [1.80 TB / 308.18 GB free]
PV /dev/sdc VG openfiler192.168.2.83 lvm2 [1.16 TB / 48.34 GB free]
PV /dev/sdb VG openfiler3 lvm2 [1.16 TB / 262.34 GB free]
PV /dev/sda2 VG pve lvm2 [297.59 GB / 3.99 GB free]
Total: 5 [6.24 TB] / in use: 5 [6.24 TB] / in no VG: 0 [0 ]
proxmox8:~# pvscan
PV /dev/sdd VG openfiler192.168.2.83 lvm2 [1.16 TB / 48.34 GB free]
PV /dev/sdc VG OPENFILER lvm2 [1.80 TB / 308.18 GB free]
PV /dev/sdb VG openfiler3 lvm2 [1.16 TB / 262.34 GB free]
PV /dev/sda2 VG pve lvm2 [297.59 GB / 3.99 GB free]
Total: 4 [4.42 TB] / in use: 4 [4.42 TB] / in no VG: 0 [0 ]
It looks like different nodes are seeing different things? Should all the storage configuration be the same?
you need to make sure that all nodes have access to all storages defined on the master. if this is not the case, you will run into timeouts and other issues. start at the master to check all storages, then continue with the nodes.
I have done that ... They all can access all storages. Some of my nodes are 1.7 and some are 1.8 could this be the problem?
HEY!!! After reading more about DM i was able to find 65
proxmox10:~# dmsetup info -j 254 -m 65
Name: OPENFILER-vzsnap--proxmox1--0
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 65
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLLP868D60sD4v2vqpifyvHr1JccSoko3E
proxmox10:~# dmsetup info -j 254 -m 65
Name: OPENFILER-vzsnap--proxmox1--0
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 0
Major, minor: 254, 65
Number of targets: 1
UUID: LVM-0L0c3whaj8P9nMRsEqeOV9xUcdirFFMLLP868D60sD4v2vqpifyvHr1JccSoko3E