M.2 Upgrade

[Smash_Override]

1. I am such an FNG and I apologize in advance for I know not what I do...or say.

2. Config: I bought the components from a coworker and added the storage.
Proxmox VE 7.2-3
CPU: AMD Ryzen Threadripper 2990WX 32-Core Processor
MB: ASROCK X399
RAM: 64GB
Storage: 1x 4TB Spinning Rust for Proxmox and guest VMs labeled /sda
3x 12TB Spinning Rust TrueNAS Storage
/sdb & /sdc are passthrough to TrueNAS.
/sdd is a recent add to move vms to while expanding /sda, then will be passed through to TrueNAS.


3. Problem: Local (home) storage is maxed out at a little over 100 gigs and it's a 4TB drive. I had Local-LVM which was the remainder of the 4TB drive that I deleted in an attempt to expand the Local home with the following command " lvresize -l +100%FREE /dev/pve/root " without the quotes of course. It gave me about 12 gigs back. I did have a few VMs running on it that I migrated over to one of my three 12 TB drives before attempting the expansion. There was one snapshot that was locked on the Local-LVM that I deleted after editing the .conf and removing the locked snapshot line. Ran the command again and same same.

4. Task: Transfer all data from 4TB spinning rust to a new 4TB M.2 and utilize the full 4TB. Then reutilize the 4TB Spinning rust as a mirror for the M.2.

So have I made a complete mess of all this and I'm better off wiping it all and starting from scratch? Or is there a way to accomplish #4 Task and avoid starting over? I have attached as many screenshots as I thought might be useful. If anyone wants more just say the word. I appreciate everyone's help and apologize if I should have looked elsewhere in the forum first. I just didn't know where to start. My vocabulary is lacking. This is my home lab I built in an effort to learn through experience. Assuming I make it past this problem, my next goals are migrating TrueNAS Core to Scale, install docker and containers for various things, and miror all data to an offsite TruNAS Scale system I built and connected with TailScale.

Cheers
Brad

 

[Smash_Override]

 

[Dunuin]

2. Config: I bought the components from a coworker and added the storage.
Proxmox VE 7.2-3
CPU: AMD Ryzen Threadripper 2990WX 32-Core Processor
MB: ASROCK X399
RAM: 64GB
Storage: 1x 4TB Spinning Rust for Proxmox and guest VMs labeled /sda
3x 12TB Spinning Rust TrueNAS Storage
/sdb & /sdc are passthrough to TrueNAS.
/sdd is a recent add to move vms to while expanding /sda, then will be passed through to TrueNAS.

Thats not a lot RAM for all those horsepowers of the CPU.
My bet would be that all will be bottlenecked by that 4TB HDD you use as a VM storage.

4. Task: Transfer all data from 4TB spinning rust to a new 4TB M.2 and utilize the full 4TB. Then reutilize the 4TB Spinning rust as a mirror for the M.2.

Bad idea. This will slow down writes to HDD performance.

3. Problem: Local (home) storage is maxed out at a little over 100 gigs and it's a 4TB drive. I had Local-LVM which was the remainder of the 4TB drive that I deleted in an attempt to expand the Local home with the following command " lvresize -l +100%FREE /dev/pve/root " without the quotes of course. It gave me about 12 gigs back. I did have a few VMs running on it that I migrated over to one of my three 12 TB drives before attempting the expansion. There was one snapshot that was locked on the Local-LVM that I deleted after editing the .conf and removing the locked snapshot line. Ran the command again and same same.

Why do you want to remove "local-lvm" in the first place? Thats the default for storing virtual disks as it offers the best mix of performance, features and low overhead. Moving virtual disks to "local" you add additional overhead.

According to your screenshot you only deleted the "local-lvm" storage but not the actual LVM thin pool. So that pool is still there and consuming space...you just can't use it anymore without a LVM-Thin type storage pointing to it.

 

[Smash_Override]

Thanks for the response. Been a min since I could get back to my keyboard and work on this project.

So the big picture is replace the 4TB HDD with a 4TB NVME M.2 and run all OS's from the NVME. The 4TB SATA HDD would just be used for backing up the M.2 NVME. Why? Well really just because it's already installed and why not? Yeah it's old and not as fast as the NVME, but (honest question here), would it really cause a problem since it just going to be used as a backup? If the NVME takes the place of the spinning rust, why is write speed to the HDD important? In my mind I would be doing snapshops/backups once a day during non peak hours.

64 Gig of RAM...yeah that could be more but you know what...okay now I am shopping for more ram. I have four more slots and the system will support up to 128GB. Looks like I can snag 64GB for $109 (DDR4 3200. Just matching whats already in there). If I am going to pull this machine out of the closet to open it up and add the M.2, why not throw a little more RAM in it? Thanks for that . My primary use for this machine in general is a Plex server (aside from learning how to VM, container, docker, etc etc). Just did my first load test yesterday streaming between two local network devices and one offsite device (Florida > Texas) and it appears that I could probably handle an additional two streams without issue. Granted, not a lot of transcoding but it works. Pretty sure it would fall on it's face if I was writing to the disks and streaming though. Meh, that problem for another day perhaps. ..and I digress.

In regards to removing local LVM, this is where I show my ignorance and I apologize if any of this sounds like me talking out of my rear end. The ultimate problem was that my local (home) only had 100GB and was full. I found this when I went to upload an ISO (migrating TrueNAS Core to Scale) and found out I was out of space. So knowing that the HDD is 4TB, I was confused as to why it would only show 100GB? Beyond that, the ISOs previously uploaded only take up 12 GB so nothing close to 100 GB. Why is Local (home) showing its full? What am I missing here? I did some googling, found a video with some setup instructions that said to run the commands listed in my previous post in order to allocate more space to Local (home). Technically that method would have alocated all the available space to local (home). I was thinking about allocating 1TB to local (home) and the rest to the VMs. this proceedure was also supposed to be run during Proxmox initial setup and not after having all these things running. Well, now I have the VMs running on a third 12TB disk LVM-Thin in my storage array while I try to fix my mess. Once I had the local and local-LVM allocation spaces to where the HDD could be utilized fully and I can load more ISOs, I would move the VMs back to the HDD. Then, mirror everything over to the M.2. and add my 12TB HDD previously used for VMs to my TrueNAS VM.

So....nuke it and start over? Or.....type some voodoo shell commands and automagically put it all back together? If there are links to guides or other posts where this has already been addressed, please send me that way. No need to rehash something that was already accomplished.

Thanks again for all the help and understanding my ignorance. As frustrating as technology can be, I am enjoying everything that I am learning.