iSCSI storage, VM crashing

omegared77

New Member
Feb 4, 2019
16
0
1
45
Hey all,

I noticed that I have sometimes VM with read only FS, or crashed.
A VM reset solves the issue, still I'd like to know why it happens.

I have a 4 node cluster, running v5.4.6 (the 5th node has a hardware issue)

The SAN storage is a iSCSI HPE MSA1040

Then, the LUN is mounted as a bare metal LVM :
upload_2019-7-4_10-18-32.png

Each cluster member herits the datacenter settings, and is correctly connected to the SAN and LUN.
upload_2019-7-4_10-20-53.png

upload_2019-7-4_10-21-34.png

upload_2019-7-4_10-22-25.png

The MSA is configured on 4 ethernet ports in a dedicated VLAN
upload_2019-7-4_10-24-19.png

upload_2019-7-4_10-25-49.png

On monday, I added a new node in the cluster.
Fresh install, no VM.
The iSCSI storage has been mounted automatically, no issue... expect I had many VMs unreachable, displaying this king of messages :

upload_2019-7-4_10-27-12.png

upload_2019-7-4_10-27-28.png

I had to reset some of them, because I couldn't login anymore (no password prompt after the login one).
On some tohers, the / FS was read only, a restart solved the issue.

Any idea about how to fix this issue ?
 

Attachments

  • upload_2019-7-4_10-17-30.png
    upload_2019-7-4_10-17-30.png
    3.4 KB · Views: 8
Did you check the syslog/journal of the nodes to see if anything pops up?
 
Alwin,

In journalctl -xe I can see this :

Jul 05 13:03:48 btz-pve2301 kernel: sd 2:0:0:0: alua: device naa.600c0ff0003be5490000000000000000 port group 0 rel port 2
Jul 05 13:03:48 btz-pve2301 kernel: sd 2:0:0:0: [sdd] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Jul 05 13:03:48 btz-pve2301 kernel: sd 2:0:0:0: [sdd] Sense Key : Illegal Request [current]
Jul 05 13:03:48 btz-pve2301 kernel: sd 2:0:0:0: [sdd] Add. Sense: Invalid command operation code
Jul 05 13:03:48 btz-pve2301 kernel: sd 2:0:0:1: alua: supports implicit TPGS
Jul 05 13:03:48 btz-pve2301 kernel: sd 2:0:0:1: alua: device naa.600c0ff0003be549f1358a5c01000000 port group 0 rel port 2
Jul 05 13:03:48 btz-pve2301 kernel: sd 2:0:0:0: alua: port group 00 state A preferred supports tolusNA
Jul 05 13:03:48 btz-pve2301 kernel: sd 3:0:0:0: alua: supports implicit TPGS
Jul 05 13:03:48 btz-pve2301 kernel: sd 3:0:0:0: alua: device naa.600c0ff0003be4db0000000000000000 port group 1 rel port 6
Jul 05 13:03:48 btz-pve2301 kernel: sd 3:0:0:0: [sde] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Jul 05 13:03:48 btz-pve2301 kernel: sd 3:0:0:0: [sde] Sense Key : Illegal Request [current]
Jul 05 13:03:48 btz-pve2301 kernel: sd 3:0:0:0: [sde] Add. Sense: Invalid command operation code
 
I also got his :

[9659567.693226] sd 2:0:0:0: alua: supports implicit TPGS
[9659567.693232] sd 2:0:0:0: alua: device naa.600c0ff0003be5490000000000000000 port group 0 rel port 2
[9659567.694547] sd 2:0:0:0: [sdd] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[9659567.694550] sd 2:0:0:0: [sdd] Sense Key : Illegal Request [current]
[9659567.694552] sd 2:0:0:0: [sdd] Add. Sense: Invalid command operation code
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!