You don't mention it, but you have to add to HA every VM you want to be in high availability.
If you did, and you did issue a "reboot" of the PVE host, remember that there is a setting in Datacenter -> Options called "HA settings" where you can set the "Shutdown policy". By default it will...