I have 2 cars; a bad car & a terrible car, how should we plan this trip?
Whatever you do, have ON A DIFFERENT DEVICE stored somewhere (USB ext. disk, cloud storage etc.) full & restorable backups of EVERYTHING; VMs, LXCs & also your PVE host. - YOU WILL NEED THESE IN THE FUTURE! -
Host backup is not currently completely possible - you could (easily on another PC) make an image of the host drive with dd
, but at the least copy the relevant directories for future reference / restore, & document all changes you make on the PVE host (only when really necessary), plus all the configurations you have made (Host names, NW, storages etc.) so that you can very easily reinstall & start again. (That will probably happen).
Looking at your main question, that eMMC I guess when it dies will not be replaceable (unless you are a real tinkerer). That is something to think about now, & for that reason I would rather just use the SD card which is easily replaceable.
If it is not possible to run this node from an external USB hdd/sdd/m.2 (you should really research if this is not possible), then this is what I would do:
1. Make a normal PVE install to the SD-card - with NO ZFS (ZFS will only cause more unnecessary disk-writing to that SD-card without redundancy).
2. Do not use this device in a cluster. (Again less disk-writing).
3. All backups should only be performed to a different device. (Less disk-writing & anyway your biggest point-of-failure is that SD-card, so backups are almost pointless on that medium).
4. MAKE REGULAR BACKUPS OF EVRYTHING - as above.
5. Try to commit all logging to RAM - you can search this forum & others on the subject.
6. Probably don't use this server in a production environment.
7. Don't go running 30 VMs & 50 LXCs on this supercomputer.
8. Try & purchase a decent & reliable SD-card (probably 2 of them initially for easy replacement) - you can research them online.
Good luck.