Hello Proxmox Community!
We've got a 6 node Proxmox cluster running the current version of proxmox 4.4. The disks of our VMs are located on one of our 2 multipathed iSCSI SANs. We got a new one this year and rebuilt our network with 10 GbE DAC Ethernet, a physically seperated 10 GbE LAN and Jumbo Frames activated - for iSCSI traffic only.
Before this we only had 10 GbE on one of our servers and the old SAN without active Jumbo Frames.
Since this change in May this year our older SAN (NEXSAN E18) is having serious problems. We're in contact with the technical support of NEXSAN and are currently in the process of pinning down the problem. One recommendation of the support is to change the "max_sectors_kb" to a lower value. They are suspecting that large writes to the SAN make it crash and setting "max_sectors_kb" to a lower value might fix the problem.
When we try to set the value with udev the system reverts the max_sectors_kb to what I believe is a calculated value. Echoing a number into /sys/block/.../max_sectors_kb also resets to the value after some time.
We tried to find the process that resets that value with auditd - without success.
I'd like to ask the community if anybody is able to set max_sectors_kb for /dev/dm-X and /dev/sdX devices that persist and how one can achieve that.
Furthermore I'd like to ask if there are any other recommendations to decrease the IO size written to a specific SAN.
On a side note: The new SAN is working without any problems whatsover and the "max_sectors_kb" is "automagically" set to the correct value recommended by the other vendor.
Kind regards,
igot2b
We've got a 6 node Proxmox cluster running the current version of proxmox 4.4. The disks of our VMs are located on one of our 2 multipathed iSCSI SANs. We got a new one this year and rebuilt our network with 10 GbE DAC Ethernet, a physically seperated 10 GbE LAN and Jumbo Frames activated - for iSCSI traffic only.
Before this we only had 10 GbE on one of our servers and the old SAN without active Jumbo Frames.
Since this change in May this year our older SAN (NEXSAN E18) is having serious problems. We're in contact with the technical support of NEXSAN and are currently in the process of pinning down the problem. One recommendation of the support is to change the "max_sectors_kb" to a lower value. They are suspecting that large writes to the SAN make it crash and setting "max_sectors_kb" to a lower value might fix the problem.
When we try to set the value with udev the system reverts the max_sectors_kb to what I believe is a calculated value. Echoing a number into /sys/block/.../max_sectors_kb also resets to the value after some time.
We tried to find the process that resets that value with auditd - without success.
I'd like to ask the community if anybody is able to set max_sectors_kb for /dev/dm-X and /dev/sdX devices that persist and how one can achieve that.
Furthermore I'd like to ask if there are any other recommendations to decrease the IO size written to a specific SAN.
On a side note: The new SAN is working without any problems whatsover and the "max_sectors_kb" is "automagically" set to the correct value recommended by the other vendor.
Kind regards,
igot2b