VM cant start with second nic

[jsterr]

Hello, recently I created a seperate private LAN see here: https://forum.proxmox.com/threads/private-lan-between-servers-3-node-ceph.97109/

VM starts only with one nic (vmbr0):

root@pve03:~# cat /etc/pve/qemu-server/101.conf
agent: 1,fstrim_cloned_disks=1
balloon: 0
bios: ovmf
boot: order=scsi0;net0
cores: 4
efidisk0: vm_nvme:vm-101-disk-0,size=128K
machine: q35
memory: 16384
name: js-croit-mgmt
net0: virtio=7A:A4:6F:E6:5A:76,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: vm_nvme:vm-101-disk-1,discard=on,iothread=1,size=132G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=40295918-31de-4a87-b50e-e5eda004f121
sockets: 1
vmgenid: d3f58317-2987-447b-b0ef-5fa7ccd9c111

VM does not start with vmbr0 and vmbr1 in it:

root@pve03:~# cat /etc/pve/qemu-server/101.conf
agent: 1,fstrim_cloned_disks=1
balloon: 0
bios: ovmf
boot: order=scsi0;net0
cores: 4
efidisk0: vm_nvme:vm-101-disk-0,size=128K
machine: q35
memory: 16384
name: js-croit-mgmt
net0: virtio=7A:A4:6F:E6:5A:76,bridge=vmbr0,firewall=1
net1: virtio=0A:B0:48:DB:AC:CE,bridge=vmbr1,firewall=1,tag=33
numa: 0
ostype: l26
scsi0: vm_nvme:vm-101-disk-1,discard=on,iothread=1,size=132G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=40295918-31de-4a87-b50e-e5eda004f121
sockets: 1
vmgenid: d3f58317-2987-447b-b0ef-5fa7ccd9c111

VM start times out with that:

'type=tap,id=net1,ifname=tap101i1,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=F6:D6:EE:A1:C4:40,netdev=net1,bus=pci.0,addr=0x13,id=net1' -machine 'type=q35+pve0'' failed: got timeout

Does someone know why that happens?


Edit:

• Seems like its host related, because on pve02 I can start the vm.
• If I start the vm on different host and migrate it to pve03 (where I could not start the vm) it does work
• Allthough stopping and starting again does not work (vm cant power on on pve03)
• Network settings /etc/network/interfaces for vmbridge1 are the same ...
• If I remove the second nic from the vm, the vm can start ...

Edit 2:

• the only difference I noticed
• rebooting server, restarting bonds or slaves in bond did not change anything.
  
  
  

  root@pve01:~# ip a | grep perm
      link/ether 04:3f:72:c6:ea:a0 brd ff:ff:ff:ff:ff:ff permaddr 04:3f:72:c6:ea:a1
      link/ether 0c:9d:92:c5:78:df brd ff:ff:ff:ff:ff:ff permaddr 0c:9d:92:c5:78:e0
   
  root@pve02:~# ip a | grep perm
      link/ether 04:3f:72:c6:ea:a4 brd ff:ff:ff:ff:ff:ff permaddr 04:3f:72:c6:ea:a5
      link/ether 04:d4:c4:5c:d2:fa brd ff:ff:ff:ff:ff:ff permaddr 04:d4:c4:5c:d2:fb
   
  root@pve03:~# ip a | grep perm
      link/ether a2:aa:3e:ee:07:9d brd ff:ff:ff:ff:ff:ff permaddr d4:5d:64:be:25:5e
      link/ether d6:a9:71:09:95:c8 brd ff:ff:ff:ff:ff:ff permaddr 04:3f:72:c6:f0:8c
      link/ether d6:a9:71:09:95:c8 brd ff:ff:ff:ff:ff:ff permaddr 04:3f:72:c6:f0:8d
      link/ether a2:aa:3e:ee:07:9d brd ff:ff:ff:ff:ff:ff permaddr d4:5d:64:be:25:5f

 

[jsterr]

I deleted vmbr1 and recreated it. It did not work. Then suddenly it started working also after powering on a vm that has only vmbr0 configured. after that I started a vm that has both vmbr0 and vmbr1 now its suddenly working out of nowhere:

root@pve03:~# ip a | grep perm
    link/ether 04:3f:72:c6:f0:8c brd ff:ff:ff:ff:ff:ff permaddr 04:3f:72:c6:f0:8d
    link/ether d4:5d:64:be:25:5e brd ff:ff:ff:ff:ff:ff permaddr d4:5d:64:be:25:5f