[SOLVED_somehow :)] Proxmox / pfSense / 1 NIC : how to let physical computers join pfsense LAN network ?

[cryonie]

Hello all,

So let's say i'm new to proxmox, linux, pfsense and ... new to everything linked to network and virtualization.
Yeah ... a great start isn't it ?

I'm trying to make a server out of my old computer.
Installed proxmox. Working good
Created a second vmbr (vmbr1), installed pfSense, put all my VM on vmbr1 (pfSense LAN) and everything is working fine.

My network is :


Now lets say i want to let my physical computer (actuelly on 192.168.0.x) be "behind" the pfSense (so in 192.168.42.x).
It's been 2 weeks of searching on the internet and testing and ... i wasn't able to make it work.

What I think i've learned :
- vmbr1 is not "listening" to the network because it's not linked to eth0.
I tried many things ... last thing was to "iptables -A FORWARD -i vmbr0 -o vmbr1 -j ACCEPT" to try to let vmbr1 be able to "listen" to requests from my computer but when i "tcpdump -n -i vmbr1" it seems nothing reaches this interface.

What I tried also is to create (in pfSense this time) a virtual IP (192.168.42.x) on the WAN (vmbr0 linked to eth0) interface so it could "listen" to the requests and make them follow the good route ... but I didn't manage to make it work also.

I've read many guides or questions about this subject but most of them were not really linked to what I'm trying to do right now (have my physical machine (not the proxmox) behind the pfSense).
I think the right thing to do it make vmbr1 able to "listen" to the network the same way vmbr0 is doing it ... but I may be wrong.

Also, just to be clear, on most of the questions on this kind of subject the answer were :
- Buy a second NIC
But this won't help me because i thought of doing multiple different network to be able to isolate different things (IoT, Guests, trusted things, medium-trusted things ...) so ... well buying 1 NIC per usage is not really what I want.
- Use VLAN
This seems to be super interesting but ... I'm 3 weeks into this subject and I don't understand all I'm doing ... going further and using a more advanced networking technology seems not a good idea (again I may be wrong but I don't understand anything at all in VLAN, didn't find time to learn things on it for now)

Ok ... that's it for my explanations ... so now if anybody have an idea or better, if anybody have already dont it I'll be super happy of any help

Thanks in advance
Cryonie

PS :
My actual /etc/network/interfaces


And here is my proxmox VM config :


PS2 : Proxmox and pfSense are at the last version
PS3 : I don't have any firewall rules on my proxmox for now

 

[Bruno Garcia]

Assuming i understood what you want to do , ive made this diagram for you. Your "physical" network should be like this.

1 - Buy one mini-Switch like this: https://www.amazon.com.br/D-link-1008C-Switch-Portas-Fast-Ethernet/dp/B0752MBK6Q/
2 - Conect proxmox, your pc and the ISP in the switch
3 - Be sure to disable DHCP in the ISP router, pfSense will be your new DHCP server.
4 - Configure Proxmox and get internet access. Do the same to pfSense.
5 - Add a second pfSense interface, and set the DHCP/NAT configs (there must be a wizard tool to do it)
6 - The DHCP offer will travel to your network switch and arrive at your PC DHCP client.

Done , your PC is in a pfSense controlled network. This will work to any device conected to network switch.

Hope it helps

 

[cryonie]

Hello,

Thanks for your answer.
My setup is actually like this except 2 things :
- The DHCP. I'm trying to do it with fixed IP for now.
- The pfSense is linked to 2 proxmox interfaces.
-- vmbr0 (linked to eth0) for the WAN
-- vmbr1 (linked to nothing) for the LAN.

So yeah this is nearly my setup but even if i put a manual IP on my computer like 192.168.42.10 netmask 255.255.255.0 gateway 192.168.42.1 ... i can't reach anything and this is my problem

PS : I checked.
I already have a DHCP server on my LAN (192.168.42.x) that is active. If I create a VM and let it on DHCP, everything works.
The thing is that in pfSense you activate a DHCP on an interface.
Here it is active on my LAN interface (vmbr1) and if i put my physical machine in DHCP (disabling dhcp 192.168.0.x) then ... my machine do not join 192.168.42.x network.
This is for me linked to my problem that vmbr1 on proxmox is not "listening" to things coming from eth0

 

[cryonie]

OK ... I've re-read what you proposed and ...
I thought this was not something possible to do / that we should not do : link everything on one vmbr.
But apparently it's possible ... What I wanted to do was to have a vmbrx for each lan and then connect them to pfSense ... and here what will be possible is to have one vmbr for All (proxmox side), declare many network interfaces on the pfSense machine in proxmox and then make the different lan in pfSense.
I'll try it and come back with the setup and if it worked or not

Thanks again for your answer.

 

[Bruno Garcia]

Thats great ! Send pictures =]

 

[cryonie]

Hello,

So i think it's working
In my mind it seems bad (should be VLAN and not what i'l doing) but it's working so ...

Here is my PVE :

I only have 1 NIC and on this NIC only 1 virtual interface. Pretty simple.
All the VM in the pve will be linked to this vmbr0 also.

Here is the configuration of the pfsense VM :

So i made 5 network devices all are coming from vmbr0.

And here is what i see on my pfsense console :


After that everything is pretty simple.
You configure manually IP address of each VM to match the network and gateway you want and it works.

Some things to keep in mind :
- You will be able to have only 1 DHCP on one network (because they are all linked and thus if you put a DHCP server on 2 lan ... you'll have trouble
- Be carefull ! if you put no restriction rules, all network (except WAN of course) will see the others ! So machines in 192.168.169.x will be able to ping machines in 192.168.42.x ! This is not good ... you make different networks to isolate things and here, because i'm doing like this, it's not isolated at all! So you'll have to make rules on each network in pfsense to block things (haven't made this for now).

But hey ... it works. I'm able to direct trafic like i want, able to publish things on the internet, able to restrict access between network outside/inside and even from inside/inside so ... yeah
Trying to make multiple vmbrX in pve was always a failure ... but this solution works.

Thanks for you help Bruno, my first read of your diagram was like "He didn't understant what I want ..." but reading it again with a friend gave the solution so I think that it was more a "I'm not able to understand your hint"

Have fun all
Cryonie

 

[Bruno Garcia]

Cool! Glad i could help !

Remember, Windows desktops are not vlan aware. So the use of vlan in your current infrastructure could be useless anyway.

Best regards,