Unrecognized traffic 169.254.169.254:80 TCP:S

[Model_SM9208]

Hello, been using proxmox for about a year now and I am trying to improve the security of my cluster. Treating all traffic as a problem until proven otherwise. the Block IPv4 link-local (1000000102) rule on my PFsense firewall keeps blocking traffic from my proxmox VMs coming out of the cluster to 169.254.169.254:80 TCP:S (TCP Sync request). All my VMs regardless of what they are running try to reach out here. I am wondering if it is the qemu-guest-agent as that is a common thread accross the servers making this traffic.

 

[UdoB]

That's an automatically configured IP address (https://en.wikipedia.org/wiki/Apipa) of one of your devices.

And no, it has nothing do to with that Qemu agent.

(( I for myself prefer to configure all my devices semi-statically (MAC-specific, manually crafted lease in the DHCP-Server), including an entry in a Nameserver - this way I always know which device is which... ))

Best regards

 

[Model_SM9208]

Thanks, any advice for tracking down which device is getting that automatically configured IP? I set up every host (Proxmox 8.1) and VM (Ubuntu 22.04 LSTS) with a static IP. The PFsense arp table doesn't have anything with that IP. Traceroute doesn't find it. DHCP is disabled.

EDIT: I've continued to investigate and every time my puppet agent checks in is when the traffic shows up.

 

[UdoB]

Brute force / low level method:

• take one of the linux boxes in that network and be "root"
• get yourself a random IP address in 169.254.0.0/16 via "ip address add ... ..."
• verify that this step has been done correctly by just running "ping 169.254.169.254" in a terminal with endless output each second
• turn off one device after another (or remove connectivity aka "the network cable" for a few seconds) until that "ping" stops

When that "ping" does not work any more you've located the guilty device... ;-)

----
Are you "in the cloud"? Actually that address seems to have an established meaning - ask a random search machine and you'll find (for example) https://serverfault.com/questions/427018/what-is-this-ip-address-169-254-169-254
Maybe PFsense tries to be clever and imitates some mechanism? Disclaimer: I have NO idea...

 

[Model_SM9208]

No cloud only basement. Have a separate network behind a pfesne box for this cluster. I'm starting to think this is a question better asked on puppet forums. https://www.going-flying.com/blog/what-linux-process-just-tried-to-connect-to-where.html and the fact that it's coming out of boxes with the puppet agent whenever I run agent -t or the 30m interval. Thanks for your help all.