Proxmox NFS crash

[mehdi]

Hello everyone,

I have a problem with my cluster running proxmox 1.8 (192.168.103.20X) and nexenta for storage( 192.168.103.139):

access to NFS server works fine and after a while connection is impossible:

CT#0: nfs: server 192.168.103.139 not responding, still trying
INFO: task cp:10930 blocked for more than 300 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
cp D ffff81022c35eb60 0 10930 10904 (NOTLB)
ffff81021b76bd48 0000000000000082 ffff810108c5a3c0 ffffffff80017c46
ffff81022c35eb60 ffff81022f3f2220 000001745596c1a7 000002e73791ba0b
ffff81022c35ed68 ffff81022f3ee000 ffffffff80343b80 0000000000000000
Call Trace:
[<ffffffff80017c46>] cache_grow+0x37a/0x3c0
[<ffffffff884c045d>] :nfs:nfs_wait_bit_uninterruptible+0x0/0xd
[<ffffffff80064c66>] io_schedule+0x59/0x8a
[<ffffffff884c0466>] :nfs:nfs_wait_bit_uninterruptible+0x9/0xd
[<ffffffff80064e9b>] __wait_on_bit+0x40/0x6f
[<ffffffff8009a2d1>] recalc_sigpending+0xe/0x25
[<ffffffff884c045d>] :nfs:nfs_wait_bit_uninterruptible+0x0/0xd
[<ffffffff80064f36>] out_of_line_wait_on_bit+0x6c/0x78
[<ffffffff800a32cd>] wake_bit_function+0x0/0x23
[<ffffffff884c3a5d>] :nfs:nfs_wait_on_requests_locked+0x70/0xca
[<ffffffff884c4bfb>] :nfs:nfs_sync_inode_wait+0x60/0x1db
[<ffffffff884ba6e4>] :nfs:nfs_do_fsync+0x1f/0x3f
[<ffffffff884baec9>] :nfs:nfs_file_flush+0x84/0xac
[<ffffffff8000b1d5>] fget_light+0x25/0x8f
[<ffffffff8002478f>] filp_close+0x36/0x64
[<ffffffff8001e3d4>] sys_close+0x88/0xbd
[<ffffffff80060166>] system_call+0x7e/0x83

However i still can ping 192.168.103.139
I've tried a lot of things, changing kernel to 2.6.18, verified that nexenta is using nfs version 3...
The only solution is to reboot all nodes.

An idea ?
Thank you for your help

 

[mehdi]

we solved the problem,
it was a conflict between vlan and link aggragation on nexenta. Now everything works fine

 

[macday]

we solved the problem,
it was a conflict between vlan and link aggragation on nexenta. Now everything works fine

could you tell me what the problem was in detail and how you solved it? I doing many servers with link-agreggation.
thank you

 

[mehdi]

We have 2 clusters, one for production and one for testing. Both are connected to one nas with 3 nics. production cluster is on vlan102 (192.168.102.X) whereas testing cluster is on vlan103 (192.168.103.X).
The nas is connected in lacp with 3 nics, in native vlan102 (192.168.102.139) and vlan103 (192.168.103.139) trunk allowed. But nfs crashes after a while only in testing cluster.

To solve the problem, i configured link aggregation with 2 nics in vlan 102 and the other nic directly to vlan103. I separated physically the vlans.
However, I dont know if it was a problem of nexenta or proxmox. It made me crazy.