HDD configuration

[whytewolves]

I am new to Proxmox and all Virtual Environments, I am also fairly new to RAID configurations. I had a server which previously was a Windows 11Pro, running 4 20TB HDD's in a RAID 5 serving movies and the like to PLEX and Jellyfin. Wasn't an actual server other than this.
As of yesterday, I got my Proxmox installed and all 12 drives installed, the 4 20TB mentioned above, 2 10TB, 1 8TB, 1 4TB HDD's and 4 4TB SSD's. the plan is this: I need to set up PiHole and an Adblocker running a VPN of some sort for all connections through each VM, container, etc... And I am looking to take those 4 20TB HDD's and reuse them for the movie server as they already have the files RAIDed on them. How do I save that information and reconfigure them into this environment without losing the information on them???
Everything else I'd like to do is being put to end after this is addressed as my granddaughter needs her Bluey. LOL

 

[Daniel_Dog]

If I was you I would just straight up give (passthrough) the 4 20TB HDD's to the VM.
To do this, you would want to read this wiki:
https://pve.proxmox.com/wiki/Passthrough_Physical_Disk_to_Virtual_Machine_(VM)

 

[whytewolves]

If I was you I would just straight up give (passthrough) the 4 20TB HDD's to the VM.
To do this, you would want to read this wiki:
https://pve.proxmox.com/wiki/Passthrough_Physical_Disk_to_Virtual_Machine_(VM)

Thank you for this information, I appreciate it. I have a couple issues with this though. LOL, I don't know any of this. I have never used linux based anything and do not know "apt install lshw" or whatever. I don't know how to SSH or access anything from where I am on this Windows based PC to the new Proxmox server. Any and all information with a detailed explanation of how to do whatever it may be, would be greatly appreciated.

 

[Daniel_Dog]

Unfortunately there are only two ways do this: (That I can think of.)

Know Debian/Proxmox VE terminal/CLI and do passthrough. (Passthrough can only be done via CLI)
Or wipe the disks, create a new datastore (LVM) and create a new virtual disk that is just a bit smaller as the datastore. (If you do not have a raid card, ZFS might be a good option too but ZFS has it advantaged and disadvantages as well. (But is definitely not for users new to Linux.)

 

[Dunuin]

Having no clue what you are doing is also not great for running a server. I would highly recommend you try to learn the linux administration basics first and only switch from Windows to PVE once you are sure you know enough to not screw everything up. Not losing data and not leaking your identity is a difficult task that might need years of learning and practice. Basically all more advanced stuff has to be done via the CLI (like you see when trying to passthrough a disk) and you won't have much fun with PVE without being familiar with the console.

Tinkering around with PVE is a great way to learn new stuff but then I wouldn't try to run stuff you actually care about.

 

[whytewolves]

Unfortunately there are only two ways do this: (That I can think of.)

Know Debian/Proxmox VE terminal/CLI and do passthrough. (Passthrough can only be done via CLI)
Or wipe the disks, create a new datastore (LVM) and create a new virtual disk that is just a bit smaller as the datastore. (If you do not have a raid card, ZFS might be a good option too but ZFS has it advantaged and disadvantages as well. (But is definitely not for users new to Linux.)

If I am understanding you correctly, Proxmox is a Debian based system; therefore, Linux if I am correct? I know nothing about this type of OS and do believe that these drives would not be readily available through the file system?
I do have a RAID card and my Motherboard also offers a RAID solution. The MB option is what I had used in the previous iteration of the Windows based server.
Okay, so these 4 drives were previously in a RAID 5 configuration and already have thousands of videos on them. I would prefer not to redo all that converting again from disc. Not even sure I still have all them. What would you suggest?

 

[whytewolves]

Having no clue what you are doing is also not great for running a server. I would highly recommend you try to learn the linux administration basics first and only switch from Windows to PVE once you are sure you know enough to not screw everything up. Not losing data and not leaking your identity is a difficult task that might need years of learning and practice. Basically all more advanced stuff has to be done via the CLI (like you see when trying to passthrough a disk) and you won't have much fun with PVE without being familiar with the console.

Tinkering around with PVE is a great way to learn new stuff but then I wouldn't try to run stuff you actually care about.

Well, I am 53 and late to the game. I also wish I'd heard this information before installing Proxmox on the system which was once the Windows system I was using; so, all that is lost as of now, minus the information on each of the drives, as they weren't the OS.
How would you go about retrieving this information and possibly getting it set up here on PVE???

 

[Daniel_Dog]

Stick to Windows.
Like @Dunuin said, Proxmox VE is NOT for Linux beginners.
Proxmox VE is a type 1 hypervisor and runs on Debian.

Hyper-V is also a type 1 hypervisor but with Windows as OS.

 

[whytewolves]

Stick to Windows.
Like @Dunuin said, Proxmox VE is NOT for Linux beginners.
Proxmox VE is a type 1 hypervisor and runs on Debian.

Hyper-V is also a type 1 hypervisor but with Windows as OS.

Too late

 

[Daniel_Dog]

You can always just reinstall windows and reconfigure Hyper-V.
I get that you want to switch but you first need the learn Linux/Debian before you can even think about running Proxmox VE.

 

[whytewolves]

You can always just reinstall windows and reconfigure Hyper-V.
I get that you want to switch but you first need the learn Linux/Debian before you can even think about running Proxmox VE.

I wasn't using Hyper-V. I merely used it as a Plex server mainly, but I would like to use it to serve my personal and business files. Provide cloud back up for our devices. Home Assistant and the list goes on. So, I'm not sure how to do any of that in Windows either, so I might as well start somewhere and here I am. LOL. In all its glory. If nothing else, could I set up a Windows VM and mount those drives for it and then serve them up through the plex interface or not?

 

[Dunuin]

If nothing else, could I set up a Windows VM and mount those drives for it and then serve them up through the plex interface or not?

Yes, thats what we explained before where you told us you can't follow the tutorial in the wiki because of the CLI usage

 

[whytewolves]

Yes, thats what we explained before where you told us you can't follow the tutorial in the wiki because of the CLI usage

Okay, so setup a Windows VM in Proxmox and then what? How would I go about setting those drives up in only that VM?
Can I install the Plex Container and still have it access those files?
I apologize that I have no idea what it is I am doing, but I am here to learn and this is a do or lose situation I would rather not lose.
So again, any and all information and help would be greatly appreciated.

 

[Dunuin]

Okay, so setup a Windows VM in Proxmox and then what? How would I go about setting those drives up in only that VM?

VMs can only use virtual hardware and not the real physical hardware of the host unless you make use of USB/PCI passthrough. And then neither any other VM nor the PVE host can use them. So a Windows VM can`t access those existing disks unless you pass them through. The linked wiki article describes how to "disk passthrough". With this the VM is still working with virtual disks but those virtual disks are mapped to the physical disks. Without that your Windows VM wouldn't be able to access the already existing proprietary filesystem on that raid array of your old Windows installation. Again, you have to follow the wiki article and passthrough those disks. But not sure how well this will work because of the use of the fake HW raid of the mainboard. Usually you would passthrough individual disks and use software raid inside the guest OS.

Can I install the Plex Container and still have it access those files?

You would need to use SMB network shares. So your Windows VM as a SMB server and then mounting those to the LXC. But the more secure unprivileged LXCs can't mount SMB shares directly. For that, you would need to mount those SMB shares on the PVE host and then bind-mount them from the PVE host into the Plex LXC. Then fight with user/group remapping. And you might run into problems like that PVE can't mount the SMB share at boot because the Windows VM providing the SMB share isn't running at that point, so bind-mounting it from the host to the unprivileged LXC might fail.
An easier option would be to use a privileged LXC (less secure) or a VM (more secure) for your Plex as only those could directly mount a SMB share.
But with Plex in a VM you then got the problem that Plex wouldn`t be able to use any hardware accelerated transcoding as the VM couldn`t use the hosts GPU unless you PCI passthrough that GPU into the Plex VM.

I would prefer not to redo all that converting again from disc.

And keep the popular "Raid is not a backup!" in mind. If you care about data, you should have proper backups of it anyway. Snapshots and raid won't count.

 

[whytewolves]

VMs can only use virtual hardware and not the real physical hardware of the host unless you make use of USB/PCI passthrough. And then neither any other VM nor the PVE host can use them. So a Windows VM can`t access those existing disks unless you pass them through. The linked wiki article describes how to "disk passthrough". With this the VM is still working with virtual disks but those virtual disks are mapped to the physical disks. Without that your Windows VM wouldn't be able to access the already existing proprietary filesystem on that raid array of your old Windows installation.

You would need to use SMB network shares. So your Windows VM as a SMB server and then mounting those for the LXC. But he more secure unprivileged LXCs can't mount SMB shares directly. For that you would need to mount those SMB shares on the PVE host and then bind-mount them from the PVE host into the Plex LXC. Then fight with user/group remapping. And you might run into problems like that PVE can't mount the SMB share at boot because the Windows VM providing the SMB share isn't running at that point so bind-mounting it from the host to the unprivileged LXC might fail.
Easier option would be to use an privileged LXC or VM for your Plex as only those could directly mount a SMB share.

Ok, so hypothetically, if I had another SSD, I could install Windows on it and get my Plex server back up.
So, could I then take this SSD and install it into another PC I have laying around and install my PiHole, AdBlocker, and VPN on it creating a router for all devices in my home to access through and then install my HA (Home Assistant) onto this same PC?
Then could I install some VM's to get learning Linux/Debian?
Eventually getting it all into one system?
One last question, how would I take a RAID and unRAID it for this type of situation in the future?
I lied, LOL, one more question.
Is there anyway for me to currently move files from this PC to that Proxmox SSD where the VM's are currently sitting? I have a bunch of ISO's downloaded so that I could use for the VM's to learn.

 

[Dunuin]

One last question, how would I take a RAID and unRAID it for this type of situation in the future?

I would prefer not to redo all that converting again from disc.

Keep the popular "Raid is not a backup!" in mind. If you care about your data, you should have proper backups of it anyway. Snapshots and raid won't count. So with proper backups you simply destroy that raid, create a new one and restore all data from your backups. The "3-2-1 backup rule" would be a good start.

So, could I then take this SSD and install it into another PC I have laying around and install my PiHole, AdBlocker, and VPN on it creating a router for all devices in my home to access through and then install my HA (Home Assistant) onto this same PC?
Then could I install some VM's to get learning Linux/Debian?
Eventually getting it all into one system?

Yes, sounds like a better plan to run existing stuff like before and then tinker around with PVE on another machine. When you are confident enough with it, you could backup those new VMs/LXCs, install PVE on the big server and restore the VMs/LXCs there.

 

[whytewolves]

Keep the popular "Raid is not a backup!" in mind. If you care about your data, you should have proper backups of it anyway. Snapshots and raid won't count. So with proper backups you simply destroy that raid, create a new one and restore all data from your backups.


Yes, sounds like a better plan to run existing stuff like before and then tinker around with PVE on another machine. When you are confident enough with it, you could backup those new VMs/LXCs, install PVE on the big server and restore the VMs/LXCs there.

So I understand RAID is not a backup, but to backup 50TB of information would mean an expensive array for the backup process. I think. I am unsure of any of this. As I stated at the beginning, I am new to all this. I have worked with Microsoft since the 90's, but as far as doing anything other than basic stuff, I have done very little. I have been lucky up until this past year when I was hacked and this is the main reason I am trying to migrate away from Microsoft. I mean there is more to it than that, but an important reason as of now. So, how do I backup 50TB without having 50 extra TB?
Ok, I am downloading the OS i had before and installing it on the SSD I had laying around.
Should I install the PiHole and other things now while this is up and running before taking it out and moving it to another device, or will I need ot do that anyway? The other device will NOT have any other drives associated with it, so all 512GB of the PVE will be used for all this stuff.
I also asked as you were responding, if there was a way for me to currently move ISO's from this PC to the current PVE VM space?

 

[Dunuin]

So I understand RAID is not a backup, but to backup 50TB of information would mean an expensive array for the backup process.

Sure. Thats why you usually buy 150TB of storage when you want to store 50TB of data. Still cheaper than buying 100TB of storage + renting 50TB of cloud storage unless you use Backblaze and that won't work with Linux nor Windows VMs.
And yes, backups are expensive. But as you got hacked already, you probably see why those are important. 3 copies (on two different media and one of them stored offsite) of everything you really care about. 2 copies of stuff that would annoy you to rip/download/redo again, but where it wouldn't be the end of the world to lose it. 1 copy for stuff you don't mind losing like temporary files, new downloads and so on. A snapshot and raid (like raid1) won't count as an additional copy.

Should I install the PiHole and other things now while this is up and running before taking it out and moving it to another device, or will I need ot do that anyway? The other device will NOT have any other drives associated with it, so all 512GB of the PVE will be used for all this stuff.

I would create them on the other machine so you don't risk losing those 40TB of data while tinkering around. Linux will do what you tell it to do. If you tell it to wipe your disks, it will do it without warning or asking for confirmation. It won't treat you like a kid and won't stop you from doing stupid mistakes.

I also asked as you were responding, if there was a way for me to currently move ISO's from this PC to the current PVE VM space?

What do you mean by "PVE VM space"? You can't store ISOs on "local-lvm" in case you mean that. For that you got "local".

 

[whytewolves]

Sure. Thats why you usually buy 150TB of storage when you want to store 50TB of data. Still cheaper than buying 100TB of storage + renting 50TB of cloud storage unless you use Backblaze and that won't work with Linux nor Windows VMs.
And yes, backups are expensive. But as you got hacked already, you probably see why those are important. 3 copies (on two different media and one of them stored offsite) of everything you really care about. 2 copies of stuff that would annoy you to rip/download/redo again. 1 copy for stuff you don't mind losing like temporary files, new downloads and so on.


I would create them on the other machine so you don't risk losing those 40TB of data while tinkering around. Linux will do what you tell it to do. If you tell it to wipe your disks, it will do it without warning or asking for confirmation. It won't treat you like a kid and won't stop you from doing stupid mistakes.


What do you mean by "PVE VM space"? You can't store ISOs on "local-lvm" in case you mean that. For that you got "local".

Ok, so more money for more HDD's, yeah. LOL
Ok, I will move that drive later this evening to the other PC
Yes, I was meaning for storage. So the ISO's I have downloaded to the PVE I am going to lose when I move the drive over?

 

[Dunuin]

So the ISO's I have downloaded to the PVE I am going to lose when I move the drive over?

You can move the drive without reinstalling PVE. Then you won't lose those ISOs. IF you wipe that disk to install PVE again, yes, then those ISOs on "local" will be wiped too.