CT firewall rules with source ip

G

Golhum

New Member
Jan 2, 2020
10
0
1
44
Hello !
I've a fresh install of proxmox 6.
I've create a CT debian 10, with external FO ip
I want to use the firewall to limit access to the CT, and filtering on source ips.
but If I set source IP in the container, the traffic was not allowed. I need to set the source as the ip of the promox host !
A rule without source is OK.
A rule with source different than the host is bocking.

As is proxmox was a router and the only visible source ip was the proxmox.

But I want to use my external ip directly.

Maybe a config of the bridge interface ?
2020-01-02_14-13.png
2020-01-02_14-14.png
2020-01-02_14-16.png
 

Attachments

  • 2020-01-02_14-14.png
    2020-01-02_14-14.png
    47.1 KB · Views: 19
Last edited:
Hi,

are the firewall on datacenter level and Container,NIC level turned on?
 
Hi,
Yes, all turned ON (datacenter, host, container and on network interface).

I just want to filtering ex : port 22 to my ips only.
I've reinstall 2 times, always the same pb.
I'm on a dedicated server by OVH, with proxmox preconfigured, and range of failover ips.
 
The only source I can use in the rule in the container was the ip of the host. All other source will not work.
 
can you please send the fw files of the Container
they are located at.

/etc/pve/firewall/<GuestID>.fw
/etc/pve/firewall//cluster.fw
 
I've similar issue, but with disabled all firewalls and noticed that proxmox overwrite source IP to own public IP address. Couldn't find solution yet. Will paste dumps here from tcpdump later
 
My files ...
I've make somes tests. here, an iplist +group, but not working too.
In the container, no rule keep connection pass from ip in list or ip in source
I need to create a rule with ip of the host in source (so no ip filtration).
I've try the 'ip_filter' to yes, but not changing anything


root@xxxx:/# cat /etc/pve/firewall/cluster.fw
[OPTIONS]

enable: 1

[IPSET nico]

xx.xx.xx.140 # vpn
xx.xx.xx.50 # freebox
xx.xx.xx.221 # orange
xx.xx.xx.30 # bbox
xx.xx.xx.241 # az

[group grnico]

IN ACCEPT -source +nico -log nolog

root@xxx:/# cat /etc/pve/firewall/101.fw
[OPTIONS]

ipfilter: 1
enable: 1

[RULES]

IN ACCEPT -source xx.xx.xx.221 -log nolog
|GROUP grnico
 
tcpdumps:
From proxmox:
17:38:01.689454 IP 89.XX.XX.41 > 51.ZZ.AAA.88: ICMP echo request, id 1, seq 1, length 64
17:38:01.689479 IP 51.ZZ.AAA.156 > 51.BB.YYY.88: ICMP echo request, id 1, seq 1, length 64
17:38:01.689515 IP 51.BB.YYY.88 > 51.ZZ.AAA.156: ICMP echo reply, id 1, seq 1, length 64
17:38:01.689522 IP 51.BB.YYY.88 > 89.XX.XX.41: ICMP echo reply, id 1, seq 1, length 64
Click to expand...
From VM(doesn't matter if LXC or KVM):
17:43:45.549781 IP 51.ZZ.AAA.156 > 51.BB.YYY.88: ICMP echo request, id 2846, seq 1, length 12
17:43:45.549796 IP 51.BB.YYY.88 > 51.ZZ.AAA.156: ICMP echo reply, id 2846, seq 1, length 12
Click to expand...

Where:
89.XX.XX.41 - my public IP addr
51.BB.YYY.88 - VM public IP addr
51.ZZ.AAA.156 - proxmox public IP addr
 
Yes, I think we have the same problem.
I don't have dump the packets, but all traffic in the VM like to get from the proxmox host.
I try some configuration, and flags ... but no result
 
@Golhum I found the cause of the problem, but unfortunately I didn't find the correct solution. Solution(temporary) which works for me is just disable apparmor:
systemctl stop apparmor
systemctl disable apparmor
Click to expand...

I also restart the server to make sure apparmor not impact to any existing process.

Hope it will work for you too
 
  • Like
Reactions: Golhum
jozwior said:
@Golhum I found the cause of the problem, but unfortunately I didn't find the correct solution. Solution(temporary) which works for me is just disable apparmor:


I also restart the server to make sure apparmor not impact to any existing process.

Hope it will work for you too
Click to expand...

Yes !!!!!
It works like a charm.
I've reboot too, and no problem.
Some tests, and sources in the firewall works perfectly. Tomorrow I'll test more.

As I'm the only user of the system and the containers, I have no really risk.
But if anyone know what the disable of apparmor can perturb proxmox or create significant security breach
Is it safe to production environment ?

Thanks jozwior
 
I can't reproduce this behavior here.
Is there any special setting on this host or do you install extra packages?
 
Hi !
No, install from OVH template on dedicated server.
If you want, I can send you an access as it was not on prod now and can easily reset.
 
No this is not necessary.

please send me a complete list of installed packages.
and the running network config.

Code: 
dpkg -l
ip a
ip r

for the ip command, the container must run.
You can send me the IP output private or mask the public IP's.
But if you mask them, keep in mind I have to know how the network works.
 
OK ...
So, the config, with 2 container running. and apparmor disabled
The only package I've manually install on the host was curlftpfs (but I've install it after)

dpkg -l :
see attached file


ip a :
Code: 
root@toto:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
    link/ether 08:_____ brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 08:_____ brd ff:ff:ff:ff:ff:ff
    inet 46._._._/24 brd 46._._._ scope global dynamic vmbr0
       valid_lft 69369sec preferred_lft 69369sec
    inet6 fe80::__/64 scope link
       valid_lft forever preferred_lft forever
5: veth100i0@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether fe:8d:c8:ec:32:ec brd ff:ff:ff:ff:ff:ff link-netnsid 0
6: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ea:92:ba:b1:f4:29 brd ff:ff:ff:ff:ff:ff
7: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether 4e:eb:bd:c4:ed:66 brd ff:ff:ff:ff:ff:ff
8: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether ea:92:ba:b1:f4:29 brd ff:ff:ff:ff:ff:ff
10: veth101i0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000
    link/ether fe:d2:dc:7a:0b:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 1
11: fwbr101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 76:18:01:72:85:fa brd ff:ff:ff:ff:ff:ff
12: fwpr101p0@fwln101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether be:a9:b4:d9:74:11 brd ff:ff:ff:ff:ff:ff
13: fwln101i0@fwpr101p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000
    link/ether 76:18:01:72:85:fa brd ff:ff:ff:ff:ff:ff


and ip r :
Code: 
default via 46._._._ dev vmbr0
46._._._/24 dev vmbr0 proto kernel scope link src 46._._._


And in the container 100 :
Code: 
root@iii:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
4: eth0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:00:00:____ brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 51._._.80/32 brd 51._._.80 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::ff:_:_/64 scope link
       valid_lft forever preferred_lft forever


root@iii:~# ip r
default via 51._._.254 dev eth0
51._._.254 dev eth0 scope link

container 101 :
Code: 
root@test:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
14: eth0@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:00:00:___ brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 51._._.81/32 brd 51._._.81 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::ff:_:_/64 scope link
       valid_lft forever preferred_lft forever

root@test:~# ip r
default via 51._._.254 dev eth0
51._._.254 dev eth0 scope link
 

Attachments

  • dpkg.txt
    57.3 KB · Views: 2
There are additional AppArmor packages (profiles) at this installation.
Please try to remove the following packages.
But be careful, please check which packages will automatically be removed too.

Code: 
apparmor-easyprof
apparmor-profiles 
apparmor-profiles-extra
apparmor-utils
reboot the system
 
Hi !
I've remove package you suggest, enable apparmor and reboot ! And I cannot access to the server. Try to disable firewall, apparmor ... no way to retrive the service working. :mad: (I've no KVM on the hardware, so I've try with netboot rescue system to change settings)

So,
I reinstall the server with a fresh system.
Remove the 4 packages, and reboot.
Create a container.

And now, it works perfectly.
apparmor is active on the host.
firewall filtering was OK for me (ip source works in the container). :D

Are this 4 packages important for proxmox ?

Maybe I can now use the server in prod now. Need to try with real container and services before.

Big Thanks Wolfgang !
 
These 4 packages are not part of Proxmox VE.
Why OVH decide to install them I don't know.
This are extra AppArmor profiles that are not compatible with or set.
 
OK !
Very thanks for your help. :D:D:D

I'll try to send the information to OVH ...

I need some times to check all before mark this thread solved.
 
I will check if we set these packages as conflicted with our proxmox-ve firewall packages.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back