Dedicate physical eth to KVM

A

anubis

New Member
Apr 29, 2013
27
0
1
Hi All,

I have a KVM which needs to have a physical eth dedicated to it, so that it can see/work with all VLAN's. I cannot use a fixed numbered solution (specifying the VLAN's in the host and then creating bridges up to the KVM); as the VLAN's can change regularly.

How can I get the KVM to link directly to the physical eth (no other KVM is using it) so that it can work with VLAN's.

Thanks
Anubis
 
Put a vmbr on the ETH device and attach to your KVM.
Make sure the port you are plugged into on the switch is a trunk port. Done.

Your vlans will pass through to your KVM. This is how I have my virtual router setup

Sent from my Nexus 5
 
Hi Pirateghost,

Thanks for your reply. That's exactly what I've done, but I'm not getting any traffic in or out of that interface. I also tried on another eth interface, same thing. Other VM's (that don't rely on VLAN's) work normally on the host.

eth3 > vmbr3 > kvm

Is there a command I can run on the host to see traffic coming out of the KVM to check if it's tagged properly?

Thanks
Anubis.
 
Hi anubis,

you need two things to get this working:
1. The swtich port that eth3 is connected to must be configured to allow tagged ethernet frames only, attach the port to all VLANs you want to use in the KVM
2. In Proxmox' hardware settings tab of your KVM make sure that you enter the correct VLAN ID for you virtual network device.


Hth,

alex
 
Last edited:
tuxmin said:
Hi anubis,

you need two things to get this working:
1. The swtich port that eth3 is connected to must be configured to allow tagged ethernet frames only, attach the port to all VLANs you want to use in the KVM
2. In Proxmox' hardware settings tab of your KVM make sure that you enter the correct VLAN ID for you virtual network device.


Hth,

alex
Click to expand...

Neither of those are necessary. I have one vmbr being used as a vlan trunk carrying 6 vlans

Sent from my Nexus 5
 
Hi All,

@tuxmin
The switch port is configured correctly (as a trunk) as this same port was used with the previous physical server that the proxmox KVM is replacing. This switchport is connected directly to eth3 on the host which is the only port mapped to vmbr3.

The solution you posted however would only work where the VLAN ID's are fixed and not regularly changing, in this case, my KVM needs to be able to see 'all' possible VLAN's as when we require one, we create it on the KVM, but I don't want to add additional complexity by also needing to configure it on the host.

@pirateghost
That's exactly my same config, so not sure why it's not working, I did expect it to.

@udo
I did try that, but instead listening to 'all' data on the interface, the only data I saw was the ARP requests from the KVM going to the physical switch, but no answers which led me to believe they weren't being tagged properly.

I shall investigate further, clearly it should be working as it is for pirateghost, so I need to re-check everything.

Thanks
Anubis.
 
I run all my vlans over the vmbr that I have dedicated as my proxmox 'trunk'

Sent from my Nexus 5
 
Hi All,

After further testing here are the results...

With the setup as:

Cisco Switch Port (trunk mode) > Host eth3 > vmbr3 > Guest KVM eth1 > eth1.100

When attempting to ping a host on the vlan #100, I can see the ping getting to the remote host from the guest KVM and I can see the remote host sending the reply. I can see the reply exiting the cisco switch port and if I configure vlan #100 on the host, I can see the replies getting there.

BUT, I can't see the reply getting across vmbr3 to the guest.

Something must be missing here as outbound all seems fine, but return inbound seems to be blocked somewhere between the host eth3 and vmbr3.

Thanks
Anubis.
 
Hi All,

Does anyone have any idea why this issue might be occurring? Outbound VLAN traffic seems to be getting applied correctly, but inbound appears to keep breaking.

Thanks.
Anubis.
 
Hi All,

I've added some additional information below. I can see from the remote host that pings are reaching it and they're indeed coming back, but for some reason, despite the fact the replies are getting to the proxmox interface, they're not being pushed across the bridge.


ifconfig of all applicable interfaces:
--------------------------------------------
root@vs1:~# ifconfig eth3
eth3 Link encap:Ethernet HWaddr a0:36:9f:20:aa:9d
inet6 addr: fe80::a236:9fff:fe20:aa9d/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:898129 errors:0 dropped:0 overruns:0 frame:0
TX packets:164682 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:105904338 (100.9 MiB) TX bytes:8236790 (7.8 MiB)


root@vs1:~# ifconfig vmbr3
vmbr3 Link encap:Ethernet HWaddr 16:f0:a8:4c:35:fc
inet6 addr: fe80::a236:9fff:fe20:aa9d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:838780 errors:0 dropped:0 overruns:0 frame:0
TX packets:1625 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:84707095 (80.7 MiB) TX bytes:154914 (151.2 KiB)


root@vs1:~# ifconfig tap2000i1
tap2000i1 Link encap:Ethernet HWaddr 16:f0:a8:4c:35:fc
inet6 addr: fe80::14f0:a8ff:fe4c:35fc/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:44248 errors:0 dropped:0 overruns:0 frame:0
TX packets:697785 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:2036480 (1.9 MiB) TX bytes:89501044 (85.3 MiB)



vmbr config for the specified interface and kvm guest:
----------------------------------------------------------------
root@vs1:~# brctl show vmbr3
bridge name bridge id STP enabled interfaces
vmbr3 8000.16f0a84c35fc no eth3
tap2000i1


Pings (requests and replies) on remote host showing the request are arriving (on the correct VLAN 300) and the answer also exiting via VL300:
-------------------------------------------------------------------------------------------------------------------------
root@host1:~# tcpdump -nne -i eth1 host 10.10.255.240
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
12:43:48.170030 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 46
12:43:48.170034 68:05:ca:07:de:2c > 4a:30:f6:34:c9:3b, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Reply 10.10.255.253 is-at 68:05:ca:07:de:2c, length 28
12:43:49.168589 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 46
12:43:49.168598 68:05:ca:07:de:2c > 4a:30:f6:34:c9:3b, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Reply 10.10.255.253 is-at 68:05:ca:07:de:2c, length 28
12:43:50.168585 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 46
12:43:50.168589 68:05:ca:07:de:2c > 4a:30:f6:34:c9:3b, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Reply 10.10.255.253 is-at 68:05:ca:07:de:2c, length 28
12:43:51.186072 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 46
12:43:51.186082 68:05:ca:07:de:2c > 4a:30:f6:34:c9:3b, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Reply 10.10.255.253 is-at 68:05:ca:07:de:2c, length 28



tcpdump directly on 'eth3' showing request exiting via VL300 but no sign of replies:
------------------------------------------------------------------------------
root@vs1:~# tcpdump -nne -i eth3
tcpdump: WARNING: eth3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 65535 bytes
12:37:46.883871 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:37:47.675904 00:18:18:58:71:15 > 00:18:18:58:71:15, ethertype Loopback (0x9000), length 60:
12:37:47.763303 00:18:18:58:71:15 > 01:80:c2:00:00:00, 802.3, length 135: LLC, dsap STP (0x42) Individual, ssap STP (0x42) Command, ctrl 0x03: STP 802.1s, Rapid STP, CIST Flags [Learn, Forward, Agreement]
12:37:47.883803 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:37:48.901316 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:37:49.776666 00:18:18:58:71:15 > 01:80:c2:00:00:00, 802.3, length 135: LLC, dsap STP (0x42) Individual, ssap STP (0x42) Command, ctrl 0x03: STP 802.1s, Rapid STP, CIST Flags [Learn, Forward, Agreement]
12:37:49.899844 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:37:50.899853 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:37:51.789814 00:18:18:58:71:15 > 01:80:c2:00:00:00, 802.3, length 135: LLC, dsap STP (0x42) Individual, ssap STP (0x42) Command, ctrl 0x03: STP 802.1s, Rapid STP, CIST Flags [Learn, Forward, Agreement]
12:37:51.917313 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:37:52.915840 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:37:53.803494 00:18:18:58:71:15 > 01:80:c2:00:00:00, 802.3, length 135: LLC, dsap STP (0x42) Individual, ssap STP (0x42) Command, ctrl 0x03: STP 802.1s, Rapid STP, CIST Flags [Learn, Forward, Agreement]
12:37:53.915862 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 300, p 0, ethertype ARP, Request who-has 10.10.255.253 tell 10.10.255.240, length 28


If I configure VL300 directly on eth3 I can see the replies hitting the interface:
--------------------------------------------------------
root@vs1:~# tcpdump -nne -i eth3.300 host 10.10.255.240
tcpdump: WARNING: eth3.300: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3.300, link-type EN10MB (Ethernet), capture size 65535 bytes
12:41:11.971885 68:05:ca:07:de:2c > 4a:30:f6:34:c9:3b, ethertype ARP (0x0806), length 60: Reply 10.10.255.253 is-at 68:05:ca:07:de:2c, length 46
12:41:12.972009 68:05:ca:07:de:2c > 4a:30:f6:34:c9:3b, ethertype ARP (0x0806), length 60: Reply 10.10.255.253 is-at 68:05:ca:07:de:2c, length 46
12:41:13.989488 68:05:ca:07:de:2c > 4a:30:f6:34:c9:3b, ethertype ARP (0x0806), length 60: Reply 10.10.255.253 is-at 68:05:ca:07:de:2c, length 46



However, if I do the same on vmbr3 (firstly removing eth3.300) I again only see the requests:
-------------------------------------------------------------
root@vs1:~# tcpdump -nne -i vmbr3.300
tcpdump: WARNING: vmbr3.300: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr3.300, link-type EN10MB (Ethernet), capture size 65535 bytes
12:40:18.683813 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:40:19.701773 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:40:20.699815 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.10.255.253 tell 10.10.255.240, length 28
12:40:21.699804 4a:30:f6:34:c9:3b > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.10.255.253 tell 10.10.255.240, length 28



So clearly, the VM is allowing packets to exit using the correct VLAN, but for some reason, the return packets are getting discarded. This is driving me nuts, don't know where else to look? I've got other linux based (not proxmox) devices which act as a bridge between two interfaces and these forward all VLAN's as a trunk perfectly, so no idea what's going on here.

Thanks
Anubis.
 
Hi Pirateghost,

Can you tell me the version of Proxmox you're using where this is working? It's still not happening in our version and I need to get this resolved?

Thanks
Anubis.
 
I am running the latest 3.2

Code: 
proxmox-ve-2.6.32: 3.2-121 (running kernel: 2.6.32-27-pve)pve-manager: 3.2-1 (running version: 3.2-1/1933730b)
pve-kernel-2.6.32-27-pve: 2.6.32-121
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.5-1
pve-cluster: 3.0-12
qemu-server: 3.1-15
pve-firmware: 1.1-2
libpve-common-perl: 3.0-14
libpve-access-control: 3.0-11
libpve-storage-perl: 3.0-19
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-6
vzctl: 4.0-1pve4
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.7-4
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.2-1
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back