Pfsense as default Gateway

[Ahmed_F]

Hello,

Here is what I am trying to do:

Pfsense in front to handle all incomoing / outgoing traffic of my HA Cluster
3 Nodes in Cluster mode that will contains many VMs with different Vlans.

Pfsense is NOT installed in the Proxmox Cluster, it is in separate physical machine.


My Pfsense got 4 NICS and need to handle arround 5 Vlans or more
Each node proxmox got 5 NICS and will contains VMs with different Vlan.

I assigned IP 10.167.20.254 to eth2 to the Pfsense which will act as gateway for the VLan20
Then I assigned IP 10.167.20.1 to eth2 in node One in Proxmox

They ping each other with no problem.


I created a vmbr0 to bridge on eth2 and assigned my VM with 10.167.20.2 with 10.167.20.254 as Gateway but I am unable to ping anything, not even the eth2 of the Proxmox.


Any idea why can't I ping any of those 2 IPs? Or maybe this is not the right way to do it?

Thank you,






EDIT*******

I made some tests by putting the default gateway in the eth2 on the proxmox node and I am able to ping outside and see the traffic passing through the pfsense but this is not what I am looking for.
Assigning my VM to vmbro0 and adding 10.167.20.254 as gw in the VM doesn't work, something else I need to do?

 

[Ahmed_F]

I am still looking for some help please

 

[manu]

I would suggest to configure your VLAN a bit simpler:
* on your switch/ router, defines a trunk port, who will get all your Ethernet packets, with or without VLAN headers
* on the PVE side, you do not need to configure any VLAN on the host, just make sure vmbr0 has a physical port connected to this trunk port
* when you define a VM nic, choose the appropriate VLAN tag, this VLAN tag should match a VLAN that you defined on the switch

 

[Ahmed_F]

Hi Manu,

Thank you for your help.
The thing is that I don't have any switch.
At the moment I am using this solution in my lab with virtualbox.
I have successfully installed 3 nodes on HA and now all I need is to create an external gateway to be sure that when a node is down and the VM has beed migrated, the network still works.
That's why I thought that and external PFsense would work.
Actually I have in my VirtualBox 4 machines :

3 Proxmox nodes with 5 nics
1 Pfsense with 4 nics

So I setup the lan on my pfsense with IP 10.167.20.254 and one of my proxmox's nic with 10.167.20.12. They ping each other.
But what I need to do now is to get my vm in that lan.

I am new with proxmox, so maybe this is not the right way to do that things?

thank you

 

[Ahmed_F]

If there any solution to make this work please?

 

[manu]

please put the output of /etc/network/interfaces of the host and the config of the guest VM (qm config <my_vmid>)

 

[Ahmed_F]

Hello,

Here is my hosts network file

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address  192.168.1.31
    netmask  255.255.255.0
    gateway  192.168.1.254
    post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

auto eth1
iface eth1 inet static
    address  10.0.0.11
    netmask  255.255.255.0

auto eth2
iface eth2 inet manual

auto vmbr0
iface vmbr0 inet static
    address  10.167.20.2
    netmask  255.255.255.0
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward

And the VM config

bootdisk: scsi0
cores: 1
ide2: none,media=cdrom
kvm: 0
memory: 1024
name: Debian-1
net0: e1000=F2:32:7B:CB:18:B5,bridge=vmbr0
numa: 0
ostype: l26
scsi0: storage_vm:vm-100-disk-1,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=1e75ac35-8399-4b2a-95e4-144f75772755
sockets: 1

My Pfsense Lan IP is 10.167.20.254
My VM IP is 10.167.20.12 with gateway 10.167.20.254

Thanks for your help

 

[manu]

since you're using bridging mode your VM and your gateway are all connected to the same LAN,
and you do no need
post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

also you need to add eth2 to the vmbr0 as a bridge port

then
* restart the network

* test if pinging from the proxmox host to the gateway with IP 10.167.20.254 works
and if pinging from the proxmox host to the VM with IP 10.167.20.2 works

 

[Ahmed_F]

Thanks manu for the help.
I adjusted my config as you said.

Ping from proxmox host to the gateway work :

 ping -I vmbr0 10.167.20.254
PING 10.167.20.254 (10.167.20.254) from 10.167.20.2 vmbr0: 56(84) bytes of data.
64 bytes from 10.167.20.254: icmp_seq=1 ttl=64 time=0.580 ms
64 bytes from 10.167.20.254: icmp_seq=2 ttl=64 time=0.275 ms
64 bytes from 10.167.20.254: icmp_seq=5 ttl=64 time=0.330 ms

Ping from proxmox host to the VM work :

ping -I vmbr0 10.167.20.12
PING 10.167.20.12 (10.167.20.12) from 10.167.20.2 vmbr0: 56(84) bytes of data.
64 bytes from 10.167.20.12: icmp_seq=1 ttl=64 time=1.00 ms
64 bytes from 10.167.20.12: icmp_seq=2 ttl=64 time=0.326 ms

But ping from vm to gateway, doesn't work :
It says from 10.167.20.12 icmp Destination Host unreachable

 

[Ahmed_F]

Any idea how I can get this working please?