3.0, KVM, and "TAP" device

[dlasher]

upgraded one of my lab nodes to 3.0 as a test, from 2.3.. so far so... not good? none of my KVM's still start, saying the following:

/var/lib/qemu-server/pve-bridge: could not launch network script
kvm: -netdev type=tap,id=net0,ifname=tap502i0,script=/var/lib/qemu-server/pve-bridge: Device 'tap' could not be initialized

appropriate snip from /etc/network/interfaces

auto eth0
iface eth0 inet manual


auto eth1
iface eth1 inet manual


auto bond0
iface bond0 inet manual
        slaves eth0 eth1
        bond_miimon 100
        bond_mode 802.3ad

auto vlan66
iface vlan66 inet manual
        vlan-raw-device bond0


iface vmbr66 inet manual
        bridge_ports vlan66
        bridge_stp off
        bridge_fd 0
        bridge_fd 0

help?

 

[dlasher]

I even created a brand new KVM, telling it to boot from CD, etc, but with same network parameters (two interfaces, one on VMBR0 the other on VMBR66). Here's the entire CLI command it's using to launch (and fail):

/usr/bin/kvm -id 502 -chardev 'socket,id=qmp,path=/var/run/qemu-server/502.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/502.vnc,x509,password -pidfile /var/run/qemu-server/502.pid -daemonize -name monowall2 -smp 'sockets=1,cores=1' -nodefaults -boot 'menu=on' -vga cirrus -k en-us -m 512 -cpuunits 2000 -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -drive 'file=/var/lib/vz/template/iso/monowall-cdrom-1.33.iso,if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=/var/lib/vz/images/502/vm-502-disk-1.raw,if=none,id=drive-ide0,format=raw,cache=writeback,aio=native' -device 'ide-hd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap502i0,script=/var/lib/qemu-server/pve-bridge' -device 'e1000,mac=7A:9E:01:CA:E2:2D,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -netdev 'type=tap,id=net1,ifname=tap502i1,script=/var/lib/qemu-server/pve-bridge' -device 'rtl8139,mac=BA:29:B1:17:63:46,netdev=net1,bus=pci.0,addr=0x13,id=net1,bootindex=301'

 

[dietmar]

What is the output of

# pveversion -v

You rebooted the node after the upgrade (new kernel)?

 

[dlasher]

What is the output of

# pveversion -v

You rebooted the node after the upgrade (new kernel)?

yes. should be on latest 3.x version.. did upgrade, then did apt-get update && apt-get upgrade.

Sadly, I can't get back into the lab node right now. I went to the network configuration, made a couple of configuration additions via the GUI, and it broke the 802.3ad/vlan configuration.. the GUI still doesn't handle that correctly.

I'll have to go back, log into the console, and copy interfaces.good back over, and reboot the box, which may not get done till later today.

 

[dlasher]

Sadly, I can't get back into the lab node right now. I went to the network configuration, made a couple of configuration additions via the GUI, and it broke the 802.3ad/vlan configuration.. the GUI still doesn't handle that correctly.

not to muddy the waters, but what I mean by "doesn't handle correctly" is doing something as simple as setting vmbr66 to be active on boot, then rebooting. After reboot, nothing is reachable,pingable, and the Catalyst switch it's attached to says

.May 24 2013 17:23:09 UTC: %EC-5-L3DONTBNDL2: Gi3/41 suspended: LACP currently not enabled on the remote port.
.May 24 2013 17:23:10 UTC: %EC-5-L3DONTBNDL2: Gi3/42 suspended: LACP currently not enabled on the remote port.

 

[dlasher]

What is the output of

# pveversion -v

root@pmx3:~# pveversion -v
pve-manager: 3.0-20 (pve-manager/3.0/0428106c)
running kernel: 2.6.32-20-pve
proxmox-ve-2.6.32: 3.0-100
pve-kernel-2.6.32-11-pve: 2.6.32-66
pve-kernel-2.6.32-20-pve: 2.6.32-100
pve-kernel-2.6.32-19-pve: 2.6.32-95
lvm2: 2.02.95-pve3
clvm: 2.02.95-pve3
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-1
pve-cluster: 3.0-4
qemu-server: 3.0-15
pve-firmware: 1.0-22
libpve-common-perl: 3.0-4
libpve-access-control: 3.0-4
libpve-storage-perl: 3.0-6
vncterm: 1.1-3
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-12
ksm-control-daemon: 1.1-1

 

[spirit]

root@pmx3:~# pveversion -v
pve-manager: 3.0-20 (pve-manager/3.0/0428106c)
running kernel: 2.6.32-20-pve
proxmox-ve-2.6.32: 3.0-100
pve-kernel-2.6.32-11-pve: 2.6.32-66
pve-kernel-2.6.32-20-pve: 2.6.32-100
pve-kernel-2.6.32-19-pve: 2.6.32-95
lvm2: 2.02.95-pve3
clvm: 2.02.95-pve3
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-1
pve-cluster: 3.0-4
qemu-server: 3.0-15
pve-firmware: 1.0-22
libpve-common-perl: 3.0-4
libpve-access-control: 3.0-4
libpve-storage-perl: 3.0-6
vncterm: 1.1-3
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-12
ksm-control-daemon: 1.1-1

Seem that your bridge is not up.
(can do post the result of "btcrl show"


also, does it work better with this setup ?

auto bond0
iface bond0 inet manual
        slaves eth0 eth1
        bond_miimon 100
        bond_mode 802.3ad


iface vmbr66 inet manual
        bridge_ports bond0.66
        bridge_stp off
        bridge_fd 0
        bridge_fd 0

 

[dlasher]

Seem that your bridge is not up.
(can do post the result of "btcrl show"

Because 802.3ad is broken, I had to decouple eth0/eth1, and assign eth0/bond0/vmbr0 to management so I could get back into the box while I fight with the vlans not working.

root@pmx3:/etc/network# brctl show
bridge name     bridge id               STP enabled     interfaces
vmbr0           8000.001e8c5161f5       no              bond0
vmbr100         8000.022d31b73b94       no              tap502i0
                                                        veth401.0
vmbr66          8000.001e8c5162eb       no              vlan66
vmbr67          8000.000000000000       no
vmbr8           8000.000000000000       no
vmbr9           8000.000000000000       no

So shared vmbr100 works (back side of firewall, front side of a vm) but vmbr66/vlan66 doesn't work. Lots of RCV packets, nothing sent.

vmbr66    Link encap:Ethernet  HWaddr 00:1e:8c:51:62:eb  
          inet6 addr: fe80::21e:8cff:fe51:62eb/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:22193 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1103121 (1.0 MiB)  TX bytes:468 (468.0 B)

I'll try the subinterface method (bond.66) later.

 

[dlasher]

also, does it work better with this setup ?

Yes, thank you, that works. It appears something changed from debian6 to debian7, breaking the old way of doing dot1q/vlans. Now if I can just get 802.3ad working again..


WORKING CONFIG:

<snip>

auto eth1
iface eth1 inet manual

auto bond1
iface bond1 inet manual
        slaves eth1


auto vmbr66
iface vmbr66 inet manual
        bridge_ports bond1.66
        bridge_stp off
        bridge_fd 0


auto vmbr67
iface vmbr67 inet manual
        bridge_ports bond1.67
        bridge_stp off
        bridge_fd 0

<snip>

 

[mir]

I think your original setup would have worked if you had this option:
bond_xmit_hash_policy layer3+4

My setup is identical to your original setup and here it works.