Hey guys,
I have been trying to get this to work for several days now, for the life of me i cant figure out whats going wrong.
To summaries before details, in Proxmox I have a PFsense VPN client VM that is only for a specific network, If i connect to that network with an AP my phone/laptop can use the VPN just fine; and pfsense can connect through the VPN. If i connect a Proxmox VM/LXC it can send traffic out but never receives (except Pings). Note, these VM/LXC can connect on the normal network fine.
Proxmox interfaces:
Container Interfaces
Pfsense
If i ping our of the LXC container I can follow the packets all the way out of my router to the VPN server, however it just never receives anything back. At first i thought it was a DNS issue, after using PFsense as the DNS and he upstreams to the VPN the LXC container can resolve hostnames correctly, and even ping the IP addresses it just fails to make any other connection to the addresses.
From LXC Container over PFsense VPN Client
What is so confusing is I know the PFsense router works correctly since when connected to an AP connected to PFsense using a physical device it works as expected, it only fails when it's a proxmox VM/LXC thats connected to the PFsense VM. However its more confusing since is it appears that the network is fine since it's sent all the way through correctly out my router. Just never see anything coming back in IF it's for a proxmox LXC/VM I see messages being responded too if its for a physical device.
From Laptop over PFsense VPN Client
I have been trying to get this to work for several days now, for the life of me i cant figure out whats going wrong.
To summaries before details, in Proxmox I have a PFsense VPN client VM that is only for a specific network, If i connect to that network with an AP my phone/laptop can use the VPN just fine; and pfsense can connect through the VPN. If i connect a Proxmox VM/LXC it can send traffic out but never receives (except Pings). Note, these VM/LXC can connect on the normal network fine.
Proxmox interfaces:
Code:
auto lo
iface lo inet loopback
iface enp3s0 inet manual
iface enp2s0 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.8.8
netmask 24
gateway 192.168.8.1
bridge-ports enp3s0
bridge-stp off
bridge-fd 0
#Normal network
auto vmbr5
iface vmbr5 inet static
address 192.168.9.0
netmask 24
bridge-ports enp2s0
bridge-stp off
bridge-fd 0
#VPN NetworkContainer Interfaces
Code:
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
iface eth0 inet6 autoPfsense
Code:
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether fe:7d:cc:a9:f7:ab
hwaddr fe:7d:cc:a9:f7:ab
inet6 fe80::fc7d:ccff:fea9:f7ab%vtnet0 prefixlen 64 scopeid 0x1
inet 192.168.8.3 netmask 0xffffff00 broadcast 192.168.8.255
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-T <full-duplex>
status: active
vtnet1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether c2:5d:9a:fc:91:e6
hwaddr c2:5d:9a:fc:91:e6
inet6 fe80::c05d:9aff:fefc:91e6%vtnet1 prefixlen 64 scopeid 0x2
inet 192.168.9.1 netmask 0xffffff00 broadcast 192.168.9.255
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-T <full-duplex>
status: active
enc0: flags=0<> metric 0 mtu 1536
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: enc
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
pflog0: flags=100<PROMISC> metric 0 mtu 33160
groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
groups: pfsync
syncpeer: 224.0.0.240 maxupd: 128 defer: on
syncok: 1
ovpnc1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
inet6 fe80::9814:5b1d:7bbd:2479%ovpnc1 prefixlen 64 scopeid 0x7
inet6 fdda:d0d0:cafe:1301::1009 prefixlen 64
inet 10.15.0.11 --> 10.15.0.1 netmask 0xffff0000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: tun openvpn
Opened by PID 26607If i ping our of the LXC container I can follow the packets all the way out of my router to the VPN server, however it just never receives anything back. At first i thought it was a DNS issue, after using PFsense as the DNS and he upstreams to the VPN the LXC container can resolve hostnames correctly, and even ping the IP addresses it just fails to make any other connection to the addresses.
From LXC Container over PFsense VPN Client
Code:
root@lxc-transmission-02:~# dig reddit.com
; <<>> DiG 9.11.5-P4-5.1-Debian <<>> reddit.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11803
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;reddit.com. IN A
;; ANSWER SECTION:
reddit.com. 300 IN A 151.101.65.140
reddit.com. 300 IN A 151.101.1.140
reddit.com. 300 IN A 151.101.129.140
reddit.com. 300 IN A 151.101.193.140
;; Query time: 95 msec
;; SERVER: 192.168.9.1#53(192.168.9.1)
;; WHEN: Sat Nov 16 15:59:36 UTC 2019
;; MSG SIZE rcvd: 103
root@lxc-transmission-02:~# ping 151.101.65.140
PING 151.101.65.140 (151.101.65.140) 56(84) bytes of data.
64 bytes from 151.101.65.140: icmp_seq=1 ttl=60 time=11.2 ms
64 bytes from 151.101.65.140: icmp_seq=2 ttl=60 time=11.1 ms
64 bytes from 151.101.65.140: icmp_seq=3 ttl=60 time=11.3 ms
^C
--- 151.101.65.140 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 11.067/11.193/11.288/0.153 ms
root@lxc-transmission-02:~# wget reddit.com
--2019-11-16 16:01:24-- http://reddit.com/
Resolving reddit.com (reddit.com)... 151.101.65.140, 151.101.1.140, 151.101.129.140, ...
Connecting to reddit.com (reddit.com)|151.101.65.140|:80... failed: Connection timed out.
Connecting to reddit.com (reddit.com)|151.101.1.140|:80... failed: Connection timed out.
Connecting to reddit.com (reddit.com)|151.101.129.140|:80...^C
root@lxc-transmission-02:~#What is so confusing is I know the PFsense router works correctly since when connected to an AP connected to PFsense using a physical device it works as expected, it only fails when it's a proxmox VM/LXC thats connected to the PFsense VM. However its more confusing since is it appears that the network is fine since it's sent all the way through correctly out my router. Just never see anything coming back in IF it's for a proxmox LXC/VM I see messages being responded too if its for a physical device.
From Laptop over PFsense VPN Client
Code:
~ »» ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 98:fa:9b:21:d3:4d brd ff:ff:ff:ff:ff:ff
3: wlp0s20f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether d0:c6:37:d4:43:88 brd ff:ff:ff:ff:ff:ff
inet 192.168.9.57/24 brd 192.168.9.255 scope global dynamic noprefixroute wlp0s20f3
valid_lft 4309sec preferred_lft 4309sec
inet6 fe80::898a:7c39:527a:4728/64 scope link noprefixroute
valid_lft forever preferred_lft forever
~ »» dig reddit.com
; <<>> DiG 9.14.7 <<>> reddit.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52637
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;reddit.com. IN A
;; ANSWER SECTION:
reddit.com. 300 IN A 151.101.193.140
reddit.com. 300 IN A 151.101.129.140
reddit.com. 300 IN A 151.101.1.140
reddit.com. 300 IN A 151.101.65.140
;; Query time: 14 msec
;; SERVER: 192.168.9.1#53(192.168.9.1)
;; WHEN: Sat Nov 16 16:09:06 GMT 2019
;; MSG SIZE rcvd: 103
~ »» ip r g 151.101.193.140
151.101.193.140 via 192.168.9.1 dev wlp0s20f3 src 192.168.9.57 uid 1000
cache
~ »» ping 151.101.193.140
PING 151.101.193.140 (151.101.193.140) 56(84) bytes of data.
64 bytes from 151.101.193.140: icmp_seq=1 ttl=60 time=13.0 ms
64 bytes from 151.101.193.140: icmp_seq=2 ttl=60 time=13.1 ms
^C
--- 151.101.193.140 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 12.980/13.019/13.059/0.039 ms
~ »» wget reddit.com
--2019-11-16 16:09:32-- http://reddit.com/
Resolving reddit.com (reddit.com)... 151.101.193.140, 151.101.129.140, 151.101.1.140, ...
Connecting to reddit.com (reddit.com)|151.101.193.140|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.reddit.com/ [following]
--2019-11-16 16:09:32-- https://www.reddit.com/
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving www.reddit.com (www.reddit.com)... 151.101.17.140
Connecting to www.reddit.com (www.reddit.com)|151.101.17.140|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 654021 (639K) [text/html]
Saving to: ‘index.html’
index.html 100%[===========================================================>] 638.69K 3.66MB/s in 0.2s
2019-11-16 16:09:34 (3.66 MB/s) - ‘index.html’ saved [654021/654021]
Last edited: