Thank you very much @jun3280, you are a star!
I've spent days trying different things and I did everything even following this Guide
https://github.com/isc30/ryzen-7000-series-proxmox?tab=readme-ov-file
Which is almost identical to yours, but it still didn't work. I think my issue was with the AMDGopDriver_5800H.rom file.
I've used this one: https://github.com/isc30/ryzen-7000-series-proxmox/blob/main/AMDGopDriver_5800H.rom
Another thing I did, was install the AMD drivers while using the Windows RDP app instead of using the Console VNC (I am not sure if this makes a difference)
And it didn't work for me.
@thebigbo Pardon my comprehension, but did you end up getting everything working in the end? If so - congrats!