A few year ago, we ordered 8 servers for a new vmware cluster.
One of the servers exposted unstability: Memtest failed after a few hours.
It took me weeks with dell support to rule out memory stick and mainboard defect to finaly discover one...
I would upgrade one node after the other:
* evict vm/lxc to other nodes before upgrading
* conduct the upgrade like you said (pv8to8 checks...) (and doing it on an empty node)
* migrate back a few "ginea pigs" vm ( to verify network...)
* once...
you might try to migrate vm one by one to a node in the new cluster ( qm remote-migrate command ). shared storage ( nfs) can help for this task
* move volume to nfs storage
* qm remote migrate
* re migrate to local storage