Not understanding scsi passthrough

[djzer0g]

I think I'm not understanding something that's probably pretty basic here.
I've got an ssd at /sda on proxmox that I want to use in OpenMediaVault

Formatted sda directly with btrfs with no partition.
qm set <omv vm> /dev/disk/by-id/<my 500gb Samsung EVO>

spin up my omv vm, and mount the device.
in ovm terminal, device shows as mounted at /srv/diskby-id-yadayada-it's my evo
create share, and nano hello.txt.

open hello.txt from a diff machine via the smb share and edit it.
back to omv .... edits show up in hello.txt.

but now back in proxmox terminal ......
mount /dev/sda /mnt/data2
ls -ahls
all i have is . and .. with no /share01/ and no hello.txt file.

how do i have a share directory and file on the disk in omv that isn't on the disk in proxmox?

 

[leesteken]

You cannot share a (virtual or physical) disk/drive between VMs or between a VM and the host. Like a physical disk/drive cannot be connected to two physical computers at the same time. No filesystem is intended to work like that and you will get corruption issues because of technical reasons.
You can share directories and files between VMs (and the host) using the same methods as you use for physical computers in a network: using network shares like SMB and NFS. I kindly refer you to the documentation about them on the internet and various posts this forum.

 

[djzer0g]

You cannot share a (virtual or physical) disk/drive between VMs or between a VM and the host. Like a physical disk/drive cannot be connected to two physical computers at the same time. No filesystem is intended to work like that and you will get corruption issues because of technical reasons.
You can share directories and files between VMs (and the host) using the same methods as you use for physical computers in a network: using network shares like SMB and NFS. I kindly refer you to the documentation about them on the internet and various posts this forum.

Thanks avw, that makes total sense and multiple access really wasn't my intent but is probably muddying the waters a little.

My intent was only to have the filesystem connected to one system (the omv vm).
However, what I was trying to accomplish was to have a filesystem that I could directly access from prox or any other linux baremetal if I move the drive to another machine or screw up my vm.
SMB share is indeed what I'm trying to get set up here.

Mounting on the host was just me trying to verify that it was working as I intended, and no surprise - it wasn't.

I will note here that I powered down the omv vm before mounting on the host. (maybe I'm missing a step here?)

I had previously set the disk as storage and had it working in omv, but when I came back out to the host, i saw that the drive was now a 500GB .raw file and I don't know what to do with that from the host or another box when I inevitably screw up my vm during this learning process.

Apologies if that's confusing. If I knew all the correct language, I'd probably already have it running and move forward to jellyfin and transmission.

Thanks for the reply.

 

[leesteken]

Ah right, using the drive with one VM (or physical machine) at any one time should work (but don't enable unsafe caching).
If you passthrough the whole drive by using /dev/disk/by-id/your-drive, then I would expect the OS inside the VM to partition the drive into one or more disks (C:, D: or /dev/sda1, /dev/sda2). If you shutdown the VM and want to access the drive paritions from the Proxmox host, you might need to run partprobe first (which you might need to install first using apt install parted) or reboot. Otherwise the Proxmox host won't detect the new partitions on the drive. Then you would mount the filesystems on those partitions using /dev/sda1 or /dev/sda2 and not the whole drive (which is /dev/sda). Maybe this is what went wrong?

 

[djzer0g]

Ah right, using the drive with one VM (or physical machine) at any one time should work (but don't enable unsafe caching).
If you passthrough the whole drive by using /dev/disk/by-id/your-drive, then I would expect the OS inside the VM to partition the drive into one or more disks (C:, D: or /dev/sda1, /dev/sda2). If you shutdown the VM and want to access the drive paritions from the Proxmox host, you might need to run partprobe first (which you might need to install first using apt install parted) or reboot. Otherwise the Proxmox host won't detect the new partitions on the drive. Then you would mount the filesystems on those partitions using /dev/sda1 or /dev/sda2 and not the whole drive (which is /dev/sda). Maybe this is what went wrong?

Okay.

I'm doing a loooot of learning here from an actual configurable router, ddns, vpn, proxmox, omv etc, so I do appreciate the help.
Before I go rebuilding this ... would I be better served to create GPT in prox rather than having a btrfs disk before I even pass to proxmox?

Alternatively, am I redesigning a wheel here and there's a simpler standard way to accomplish passing a "plug and play" filesystem to a vm?

 

[djzer0g]

Okay.

I'm doing a loooot of learning here from an actual configurable router, ddns, vpn, proxmox, omv etc, so I do appreciate the help.
Before I go rebuilding this ... would I be better served to create GPT in prox rather than having a btrfs disk before I even pass to proxmox?

Alternatively, am I redesigning a wheel here and there's a simpler standard way to accomplish passing a "plug and play" filesystem to a vm?

Update: Got it working!
Steps taken:

• after wiping the disk in proxmox,
• created GPT from proxmox "disks"
• prox terminal: fdisk to create primary partition sda1, linux filesystem
• prox terminal: mkfs.btrfs on sda1
  • temporary -- terminal: mount filesystem and drop file01 in it. Unmount the system.
• prox terminal: set qm <vm id> -scsi2 /dev/disk/by-id/<my 500gb Samsung EVO>
• spin up omv vm and open up admin interface
• mount the partition corresponding to proxmox sda1
• create SMB share01
• open SMB share01 from a different machine, and drop a file02 in it.
• omv terminal: verify file02 is in share.
• shutdown omv
• proxmox terminal:
  • temporary -- terminal: mount filesystem. file01 still where it was and now share01 folder exists with file02 in it.
  • temporary -- terminal: unmount filesystem.

Cool beans.

Thanks for your help avw!
Looks like btrfs needs to be on a partition for omv and prox to understand what's going on.
Dunno if this could be accomplished with a subvolume instead .... maybe try that another time when I add raid to btrfs.
For now it works and moving on with life.

 

[djzer0g]

Update 2: Minor setback resolved!

The forums never have your exact problem or your exact errors and the more info out there, the more info for troubleshooting. so I thought I'd share the hell I just went through.

In case anyone is following my trials and tribulations as a guide:

In the process of going through the steps posted above, I never thought to go back and edit /etc/fstab.
Two VM's later, I needed to reboot my system for a stability verification --- failed!
When I performed a full system reboot to check and make sure that all my vms would start up properly and in order of dependence, ie.

• OpenVPN
• Open Media Vault
• Nginx Web 1
• Web 2

I suddenly could not get proxmox to boot up properly.
Had to pull out a monitor and keyboard and sling cables across the room to get the error message:
blah blah ata2 blah COMRESET failed (erno= 32)
and no options but to work in terminal in a recovery mode, so no gui and no ssh.
now, ata2 is my sda / sda1 that I've been working on this btrfs ovm share.

Most internet searches led to "your ssd or controller is failing" or "your SATA cable isn't seated or cable is bad"
So I tried reseating the cables. No luck. Swapped to a fresh cable. No luck.

However, one page I landed on mentioned partitions maybe being corrupt, so got me thinking about the btrfs I just installed on sda1.

Solution:
in the recovery mode offered after the prompts on the ad-hoc monitor keyboard (wires all over the house) emergency mode:
nano /etc/fstab
remove the line mounting this drive by prepending with a # ... it's not the proxmox, so who cares if I can get back to gui / ssh term and get this cabled rat nest out of my room. CTRL-O Enter CTRL-X
reboot now

Machine should now boot up and you're back in the biz.

SSH in and check out those mounts.
Term 1:
fdisk -x
Term 2:
nano /etc/fstab

The UUID's are different because the disk was wiped and partitioned. (and I moved the mount point as well, so double trouble.
Mont he device as you wish with the new UUID and see the data is all there as it should be.
Unmount the drive and edit your fstab with the new UUID.

My fstab entries for /dev/sda1/:
# set up for btrfs .... failed after reboot. changing entry accordingly. #UUID=212f2906-59ad-4e9a-8222-ce7aa0ee2482 /mnt/data2 btrfs defaults 0 0 UUID=1841e1c5-998b-4b15-b76e-ac5481ded5da /mnt/omv1 btrfs defaults 0 0

see if the thing mounts by running the fstab mounts:
mount -a

if you're up and running, then reboot.
My install gave half an error on ata1 first boot, but made it.
Rebooted again and no error.

Golden!
Total waste of 3 hours because my n00b-a$$ forgot all about my fstab.

Mind the /etc/fstab and safe travels.



safe travels