Hello. This week I bought some fresh hardware for my home lab, made a fresh Proxmox installation, and am now facing a weird issue.
I have a 802.1Q VLAN-separated network at home. The server's NICs are plugged directly into the router.
The server has 2 NICs:
- one integrated in the mobo which I connect to an access port, i.e. no tagging on Proxmox side (added to
- another is a PCIE card, for a VLAN-aware bridge to tag VMs' traffic (added to
Everything is good on the first NIC. The second one sends a DHCP broadcast, the router offers an address (I can see the offered address in the router GUI), but the server doesn't get the response. Found a post with a similar issue and tried to do similar troubleshooting by listening to the traffic while the VM is making DHCP requests:
Then I tried to do the same thing, but inside Proxmox itself by running
NMAP DHCP discovery script:
It's even worse here. The guy could get the response on Proxmox side, I don't even get that.
Here is
Test VM config:
I use non-subscription repository. Proxmox/PVE packages from
NICs are some Realteks. Tried to use them both to tag traffic, same results.
The firewall is at default settings and is disabled on both VM and cluster level. Nothing else fancy in the configuration I can think of. Tell me if I'm missing something. I've deployed a cluster with very similar dual NIC configuration this month and it works just as expected, no issues of this kind whatsoever, the VMs get their addresses just fine and sit in the VLANs they should. The only difference I can think of is that at home I don't use managed switches between the router and the server, i.e. I plug the server directly into the router.
Network configuration outside Proxmox doesn't seem to be an issue. Plugging the same network cable (the one that goes into the interface designated for tagged traffic) into a managed switch with access ports works as expected: connected devices get their IP addresses just fine.
The relevant part of the server specs:
- Motherboard: ASRock N100M
- CPU: Intel N100 (soldered)
- NICs: integrated 1Gbit/s + noname 1Gbit/s Chinese NIC with Realtek chip
Any ideas what might cause this? How do I troubleshoot this further?
I have a 802.1Q VLAN-separated network at home. The server's NICs are plugged directly into the router.
The server has 2 NICs:
- one integrated in the mobo which I connect to an access port, i.e. no tagging on Proxmox side (added to
vmbr0)- another is a PCIE card, for a VLAN-aware bridge to tag VMs' traffic (added to
vmbr1)Everything is good on the first NIC. The second one sends a DHCP broadcast, the router offers an address (I can see the offered address in the router GUI), but the server doesn't get the response. Found a post with a similar issue and tried to do similar troubleshooting by listening to the traffic while the VM is making DHCP requests:
Code:
root@node-01:~# tcpdump -i vmbr1.250 -pvn port 67 and port 68
tcpdump: listening on vmbr1.250, link-type EN10MB (Ethernet), snapshot length 262144 bytes
22:35:21.609088 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from <CUT>, length 300, xid 0x67971a2a, secs 11, Flags [none]
Client-Ethernet-Address <CUT>
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53), length 1: Discover
Hostname (12), length 7: "node-01"
Parameter-Request (55), length 13:
Subnet-Mask (1), BR (28), Time-Zone (2), Default-Gateway (3)
Domain-Name (15), Domain-Name-Server (6), Unknown (119), Hostname (12)
Netbios-Name-Server (44), Netbios-Scope (47), MTU (26), Classless-Static-Route (121)
NTP (42)Then I tried to do the same thing, but inside Proxmox itself by running
dhclient vmbr1.250 in a second terminal. Still get the same thing:
Code:
root@node-01:~# tcpdump -i vmbr1.250 -pvn port 67 and port 68
tcpdump: listening on vmbr1.250, link-type EN10MB (Ethernet), snapshot length 262144 bytes
23:10:59.076756 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from <CUT>, length 300, xid 0x6aa76264, Flags [none]
Client-Ethernet-Address <CUT>
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53), length 1: Discover
Hostname (12), length 7: "node-01"
Parameter-Request (55), length 13:
Subnet-Mask (1), BR (28), Time-Zone (2), Default-Gateway (3)
Domain-Name (15), Domain-Name-Server (6), Unknown (119), Hostname (12)
Netbios-Name-Server (44), Netbios-Scope (47), MTU (26), Classless-Static-Route (121)
NTP (42)NMAP DHCP discovery script:
Code:
root@node-01:~# nmap --script broadcast-dhcp-discover -e vmbr1.250
Starting Nmap 7.93 ( https://nmap.org ) at 2024-05-26 23:13
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 0.18 secondsIt's even worse here. The guy could get the response on Proxmox side, I don't even get that.
Here is
/etc/network/interfaces:
Code:
auto lo
iface lo inet loopback
iface enp2s0 inet manual
iface enp1s0 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.100.10/24
gateway 192.168.100.1
bridge-ports enp2s0
bridge-stp off
bridge-fd 0
auto vmbr1
iface vmbr1 inet manual
bridge-ports enp1s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
auto vmbr1.250
iface vmbr1.250 inet manual
source /etc/network/interfaces.d/*Test VM config:
Code:
agent: 1
balloon: 0
boot: order=scsi0;net0
cores: 1
cpu: host
memory: 1024
meta: creation-qemu=8.1.5,ctime=1716678160
name: caddy
net0: virtio=<CUT>,bridge=vmbr1,tag=250
numa: 0
onboot: 1
ostype: l26
scsi0: local-zfs:vm-110-disk-0,discard=on,iothread=1,size=50G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=69080dba-f6e4-4ba6-be1f-84361ea92684
sockets: 1
vmgenid: 44b885b2-81d5-4cb3-aaf8-4399b1a53c89I use non-subscription repository. Proxmox/PVE packages from
apt list --installed:
Code:
proxmox-archive-keyring/stable,now 3.0 all [installed]
proxmox-backup-client/stable,now 3.2.2-1 amd64 [installed]
proxmox-backup-file-restore/stable,now 3.2.2-1 amd64 [installed]
proxmox-backup-restore-image/stable,now 0.6.1 amd64 [installed]
proxmox-default-kernel/stable,now 1.1.0 all [installed]
proxmox-firewall/stable,now 0.4.1 amd64 [installed,upgradable to: 0.4.2]
proxmox-grub/stable,now 2.06-13+pmx2 amd64 [installed]
proxmox-kernel-6.8.4-2-pve-signed/stable,now 6.8.4-2 amd64 [installed]
proxmox-kernel-6.8.4-3-pve-signed/stable,now 6.8.4-3 amd64 [installed,automatic]
proxmox-kernel-6.8/stable,now 6.8.4-3 all [installed]
proxmox-kernel-helper/stable,now 8.1.0 all [installed]
proxmox-mail-forward/stable,now 0.2.3 amd64 [installed]
proxmox-mini-journalreader/stable,now 1.4.0 amd64 [installed]
proxmox-offline-mirror-docs/stable,now 0.6.6 all [installed]
proxmox-offline-mirror-helper/stable,now 0.6.6 amd64 [installed]
proxmox-termproxy/stable,now 1.0.1 amd64 [installed]
proxmox-ve/stable,now 8.2.0 all [installed]
proxmox-websocket-tunnel/stable,now 0.2.0-1 amd64 [installed]
proxmox-widget-toolkit/stable,now 4.2.3 all [installed]
pve-cluster/stable,now 8.0.6 amd64 [installed]
pve-container/stable,now 5.1.10 all [installed]
pve-docs/stable,now 8.2.2 all [installed]
pve-edk2-firmware-legacy/stable,now 4.2023.08-4 all [installed]
pve-edk2-firmware-ovmf/stable,now 4.2023.08-4 all [installed]
pve-edk2-firmware/stable,now 4.2023.08-4 all [installed]
pve-esxi-import-tools/stable,now 0.7.0 amd64 [installed]
pve-firewall/stable,now 5.0.7 amd64 [installed]
pve-firmware/stable,now 3.11-1 all [installed]
pve-ha-manager/stable,now 4.0.4 amd64 [installed]
pve-i18n/stable,now 3.2.2 all [installed]
pve-lxc-syscalld/stable,now 1.3.0 amd64 [installed]
pve-manager/stable,now 8.2.2 amd64 [installed]
pve-qemu-kvm/stable,now 8.1.5-6 amd64 [installed]
pve-xtermjs/stable,now 5.3.0-3 all [installed]NICs are some Realteks. Tried to use them both to tag traffic, same results.
Code:
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 09)
Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
Flags: bus master, fast devsel, latency 0, IRQ 18, IOMMU group 10
I/O ports at 4000 [size=256]
Memory at 80a00000 (64-bit, non-prefetchable) [size=4K]
Memory at 6000000000 (64-bit, prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
Capabilities: [d0] Vital Product Data
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Virtual Channel
Capabilities: [160] Device Serial Number <CUT>
Kernel driver in use: r8169
Kernel modules: r8169
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
Subsystem: ASRock Incorporation Motherboard (one of many)
Flags: bus master, fast devsel, latency 0, IRQ 18, IOMMU group 11
I/O ports at 3000 [size=256]
Memory at 80904000 (64-bit, non-prefetchable) [size=4K]
Memory at 80900000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Virtual Channel
Capabilities: [160] Device Serial Number <CUT>
Capabilities: [170] Latency Tolerance Reporting
Capabilities: [178] L1 PM Substates
Kernel driver in use: r8169
Kernel modules: r8169The firewall is at default settings and is disabled on both VM and cluster level. Nothing else fancy in the configuration I can think of. Tell me if I'm missing something. I've deployed a cluster with very similar dual NIC configuration this month and it works just as expected, no issues of this kind whatsoever, the VMs get their addresses just fine and sit in the VLANs they should. The only difference I can think of is that at home I don't use managed switches between the router and the server, i.e. I plug the server directly into the router.
Network configuration outside Proxmox doesn't seem to be an issue. Plugging the same network cable (the one that goes into the interface designated for tagged traffic) into a managed switch with access ports works as expected: connected devices get their IP addresses just fine.
The relevant part of the server specs:
- Motherboard: ASRock N100M
- CPU: Intel N100 (soldered)
- NICs: integrated 1Gbit/s + noname 1Gbit/s Chinese NIC with Realtek chip
Any ideas what might cause this? How do I troubleshoot this further?
Last edited: