Can't kill backup task

[gosha]

Hi All!

About 1 hour started backup task on each from 3 nodes.
But backups are hanging:

and not going on...
Stop button from GUI don't work...
Fro cli:
ps aux | grep vzdump
Identify process numbers and:

# kill -9 22609 22610 22614

again:
ps ax | grep vzdump
22614 ? Ds 0:04 task UPID:cn1:00005856:00ED7C5E:5910765F:vzdump::root@pam:

Repeat kill don't work:
root@cn1:~# kill -9 22614
root@cn1:~# kill -9 22614
root@cn1:~#

The state of the cluster is this:


What can I do?

Best regards,
Gosha

 

[fabian]

seems like your NFS server is down? bring it back up and wait for the NFS connections to recover.. the non-NFS parts of your system should be fine, but pvestatd hangs while checking the status of your NFS storage (hence no status)

 

[tom]

And please post your full network configuration.

 

[gosha]

Hi, Fabian!

seems like your NFS server is down? bring it back up and wait for the NFS connections to recover.. the non-NFS parts of your system should be fine, but pvestatd hangs while checking the status of your NFS storage (hence no status)

I know about this, but there was some other case. Backup NFS-storage was available. I checked it right away.
I can't found the reason and just restarted nodes.

Best regards,
Gosha

 

[gosha]

Hi, Tom!

And please post your full network configuration.

Ok. Backups passed through the interface vmbr2 (10Gbit) to another cluster with NFS-storage.

root@cn1:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual
iface eth2 inet manual
iface eth3 inet manual
iface eth4 inet manual
iface eth5 inet manual

iface bond0 inet manual
    slaves eth0 eth1
    bond_miimon 100
    bond_mode active-backup
    post-up ifconfig eth0 mtu 9000
    post-up ifconfig eth1 mtu 9000
    post-up ifconfig bond0 mtu 9000

auto vmbr0
iface vmbr0 inet static
    address  192.168.0.250
    netmask  255.255.255.0
    gateway  192.168.0.254
    bridge_ports bond0
    bridge_stp off
    bridge_fd 0
    pre-up ifup bond0
    post-down ifdown bond0

auto vmbr1
iface vmbr1 inet static
    address  10.104.249.250
    netmask  255.255.255.0
    bridge_ports eth2
    bridge_stp off
    bridge_fd 0

auto vmbr2
iface vmbr2 inet static
    address  192.168.110.17
    netmask  255.255.255.240
    bridge_ports eth5
    bridge_stp off
    bridge_fd 0
    post-up ifconfig eth5 mtu 9000

auto vmbr3
iface vmbr3 inet static
    address  10.74.152.11
    netmask  255.255.255.128
    bridge_ports eth3
    bridge_stp off
    bridge_fd 0

auto vmbr4
iface vmbr4 inet static
    address  192.168.110.1
    netmask  255.255.255.240
    bridge_ports eth4
    bridge_stp off
    bridge_fd 0
    post-up ifconfig eth4 mtu 9000

Best regards,
Gosha