I created a Proxmox server on VMWare as a sort of development environment/ for experimenting to test if my idea works or not before moving it to the actual physical device. I am also using GNS3 to simulate connecting to a switch and then to another VM (external device).
Some terms I use for the thread:
1.
2.
3.
4.
This is the network topology:
My goal is to create a
TROUBLESHOOTING TESTS:
I have tried troubleshooting by capturing packets between:
CAPTURE 1: Between the
CAPTURE 2: Between the
Test A - Proxmox to Switch/External Device:
For connectivity testing, I tried to ping:
1.
2.
3.
4.
In all 4 tests in test A, both CAPTURE 1 and CAPTURE 2 shows ARP requests and ARP responses but the ping status is Destination Host Unreachable. The ARP requests and response in CAPTURE 2 are seen to be tagged as VLAN 10 (but not for CAPTURE 1 which is expected since it is an access port).
The ARP tables for the
Test B - External device to Switch/Proxmox:
For connectivity testing, I tried to ping:
1.
2.
3.
In tests 1 and 2 of test B, both CAPTURE 1 and CAPTURE 2 show ARP requests sent to the
The ARP table for the External device also shows the ARP as failed/incomplete, but this time it is understandable since there is no response.
In test 3 of test B, the ping between the
Based on these tests, I feel that there is some issue with my
CONFIGURATIONS OF EACH MACHINE:
- Port G0/0 is configured as a trunk port, allowed VLANs are currently only 1 and 10
- Port G1/0 is configured as an access port, accessing VLAN 10.
To simulate the existing Proxmox server which only has 1 physical network interface card (and adding a second one via an adapter or PCIe lane is not possible), I configured a subinterface of the physical network interface, ens33.10 (ChatGPT said I was supposed to do this)
This is the network tab on the
This is the network interfaces config in the
This is the
In the web ui, I configured the network device to be on bridge vmbr1 and VLAN tag set to 10
For the network configuration, I used netplan to set the IPs
This is the
The
Thanks for reading through, any help troubleshooting this is appreciated. Thanks in advance!
Some terms I use for the thread:
1.
Proxmox Server - Proxmox OS hosted on VMWare with a single interface, ens332.
Proxmox VM - A VM that is hosted on the Proxmox server, VMID 1003.
Switch - A Layer 2 Cisco switch 4.
External Device - A ubuntu machine that is separated from the Proxmox server by a Layer 2 switchThis is the network topology:
| Device | Interface | IP Address | Remarks |
|---|---|---|---|
Switch | G0/0 | - | Trunk Port, allowed VLANs 1 and 10 |
Switch | G1/0 | 192.168.1.10 | Access port, access VLAN 10 |
Proxmox Server | ens33 | - | Physical network interface |
Proxmox Server | ens33.10 | - | Used to bridge network from vmbr1 to ens33 |
Proxmox Server | vmbr0 | 192.168.81.2 | IP given to Proxmox server by VMWare (Shouldn't affect anything), untagged VLAN |
Proxmox Server | vmbr1 | - | Used to bridge network from vmbr1.10 to ens33.10 |
Proxmox Server | vmbr1.10 | 192.168.1.1 | VLAN 10 |
Proxmox VM | ens18 | 192.168.1.2 | VLAN 10 |
External Device | ens33 | 192.168.1.100 | VLAN 10 |
My goal is to create a
Proxmox VM in the Proxmox server in VLAN 10 and have it communicate with the External device which is also in VLAN 10 but not communicate anywhere else, including other VLANs and untagged VLANs. For now, I am focused on trying to establish connection between the two before going to isolate the VLAN from the rest of the networkTROUBLESHOOTING TESTS:
I have tried troubleshooting by capturing packets between:
CAPTURE 1: Between the
Switch and the Proxmox serverCAPTURE 2: Between the
Switch and the External deviceTest A - Proxmox to Switch/External Device:
For connectivity testing, I tried to ping:
1.
Proxmox VM (192.168.1.2) to External device (192.168.1.100)2.
Proxmox VM (192.168.1.2) to configured Switch IP on VLAN 10 (192.168.1.10)3.
Proxmox Server (192.168.1.1) to External device (192.168.1.100)4.
Proxmox Server (192.168.1.1) to configured Switch IP on VLAN 10 (192.168.1.10)In all 4 tests in test A, both CAPTURE 1 and CAPTURE 2 shows ARP requests and ARP responses but the ping status is Destination Host Unreachable. The ARP requests and response in CAPTURE 2 are seen to be tagged as VLAN 10 (but not for CAPTURE 1 which is expected since it is an access port).
The ARP tables for the
Proxmox server and Proxmox VM shows the ARP as failed, even though there is a response going from the switch to the Proxmox server/VM. Proxmox server ARP table:
Code:
192.168.1.100 dev vmbr1.10 FAILED
192.168.81.1 dev vmbr0 lladdr 00:50:56:c0:00:03 REACHABLE
192.168.1.2 dev vmbr1.10 lladdr da:ee:fa:95:34:69 STALE
192.168.1.10 dev vmbr1.10 FAILEDProxmox VM ARP table:
Code:
192.168.1.100 dev ens18 FAILED
192.168.1.1 dev ens18 lladdr 00:0c:29:97:38:5b REACHABLE
192.168.1.10 dev ens18 FAILEDTest B - External device to Switch/Proxmox:
For connectivity testing, I tried to ping:
1.
External Device (192.168.1.100) to Proxmox Server (192.168.1.1)2.
External Device (192.168.1.100) to Proxmox VM (192.168.1.2)3.
External Device (192.168.1.100) to configured Switch IP on VLAN 10 (192.168.1.10)In tests 1 and 2 of test B, both CAPTURE 1 and CAPTURE 2 show ARP requests sent to the
Proxmox Server and Proxmox VM but no response is sent back. Same as above, the ARP requests in CAPTURE 2 are seen to be tagged as VLAN 10 (but not for CAPTURE 1 which is expected since it is an access port)The ARP table for the External device also shows the ARP as failed/incomplete, but this time it is understandable since there is no response.
External device ARP table:
Code:
192.168.1.2 dev ens33 FAILED
192.168.1.1 dev ens33 INCOMPLETE
192.168.1.10 dev ens33 lladdr 0c:55:46:ee:80:0a REACHABLEIn test 3 of test B, the ping between the
External device and the Switch is successfulBased on these tests, I feel that there is some issue with my
Proxmox Server which is preventing it from receiving packets from the switchCONFIGURATIONS OF EACH MACHINE:
Switch configuration:- Port G0/0 is configured as a trunk port, allowed VLANs are currently only 1 and 10
- Port G1/0 is configured as an access port, accessing VLAN 10.
Code:
interface GigabitEthernet0/0
switchport trunk allowed vlan 1,10
switchport trunk encapsulation dot1q
switchport mode trunk
media-type rj45
negotiation auto
interface GigabitEthernet1/0
switchport access vlan 10
switchport mode access
media-type rj45
negotiation auto
interface Vlan10
ip address 192.168.1.10 255.255.255.0Proxmox server configuration:To simulate the existing Proxmox server which only has 1 physical network interface card (and adding a second one via an adapter or PCIe lane is not possible), I configured a subinterface of the physical network interface, ens33.10 (ChatGPT said I was supposed to do this)
This is the network tab on the
Proxmox server web uiThis is the network interfaces config in the
Proxmox Server:
Code:
auto lo
iface lo inet loopback
iface ens33 inet manual
auto ens33.10
iface ens33.10 inet manual
vlan-raw-device ens33
auto vmbr0
iface vmbr0 inet static
address 192.168.81.2/24
gateway 192.168.81.1
bridge-ports ens33
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
auto vmbr1
iface vmbr1 inet manual
bridge-ports ens33.10
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 10
auto vmbr1.10
iface vmbr1.10 inet static
address 192.168.1.1/24This is the
ip a and ip route for the Proxmox server:
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
link/ether 00:0c:29:97:38:5b brd ff:ff:ff:ff:ff:ff
altname enp2s1
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:0c:29:97:38:5b brd ff:ff:ff:ff:ff:ff
inet 192.168.81.2/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe97:385b/64 scope link
valid_lft forever preferred_lft forever
4: ens33.10@ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr1 state UP group default qlen 1000
link/ether 00:0c:29:97:38:5b brd ff:ff:ff:ff:ff:ff
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:0c:29:97:38:5b brd ff:ff:ff:ff:ff:ff
inet6 fe80::20c:29ff:fe97:385b/64 scope link
valid_lft forever preferred_lft forever
6: vmbr1.10@vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:0c:29:97:38:5b brd ff:ff:ff:ff:ff:ff
inet 192.168.1.1/24 scope global vmbr1.10
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe97:385b/64 scope link
valid_lft forever preferred_lft forever
7: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr1 state UNKNOWN group default qlen 1000
link/ether fa:85:ac:61:10:80 brd ff:ff:ff:ff:ff:ff
Code:
default via 192.168.81.1 dev vmbr0 proto kernel onlink
192.168.1.0/24 dev vmbr1.10 proto kernel scope link src 192.168.1.1
192.168.81.0/24 dev vmbr0 proto kernel scope link src 192.168.81.2Proxmox VM configuration:In the web ui, I configured the network device to be on bridge vmbr1 and VLAN tag set to 10
For the network configuration, I used netplan to set the IPs
YAML:
network:
ethernets:
ens18:
dhcp4: false
addresses:
- 192.168.1.2/24
routes:
- to: default
via: 192.168.1.1
nameservers:
addresses: [8.8.8.8]
version: 2This is the
ip a and ip route for the proxmox VM:
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether da:ee:fa:95:34:69 brd ff:ff:ff:ff:ff:ff
altname enp0s18
inet 192.168.1.2/24 brd 192.168.1.255 scope global ens18
valid_lft forever preferred_lft forever
inet6 fe80::d8ee:faff:fe95:3469/64 scope link
valid_lft forever preferred_lft forever
Code:
default via 192.168.1.1 dev ens18 proto static
192.168.1.0/24 dev ens18 proto kernel scope link src 192.168.1.2External device configuration:The
External device is just a ubuntu machine running Ubuntu 22.04 desktop. This is the ip a and ip route:
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:48:c2:db brd ff:ff:ff:ff:ff:ff
altname enp2s1
inet 192.168.1.100/24 brd 192.168.1.255 scope global ens33
valid_lft forever preferred_lft forever
Code:
192.168.1.0/24 dev ens33 proto kernel scope link src 192.168.1.100Thanks for reading through, any help troubleshooting this is appreciated. Thanks in advance!