[TUTORIAL] Virtual Pendrive on VM

heringer said:
I spent the day trying to solve this and did it
Click to expand...
Good for you. However your solution involves quite a serious host modification, & in fact is not a Proxmox/HV directly linked solution. This could be done purely in the Host (without passthrough to VM), or it could be directly done Guest/Client side, as others have done in non HV environments with the dummy_hcd module, for which it was designed. However doing it within PVE host & then using passthrough to VM probably is the only possible solution to make it bootable. (Have you tested booting from it?).

BTW this has been done in Proxmox before as shown here.
 
gfngfn256 said:
Good for you. However your solution involves quite a serious host modification, & in fact is not a Proxmox/HV directly linked solution. This could be done purely in the Host (without passthrough to VM), or it could be directly done Guest/Client side, as others have done in non HV environments with the dummy_hcd module, for which it was designed. However doing it within PVE host & then using passthrough to VM probably is the only possible solution to make it bootable. (Have you tested booting from it?).

BTW this has been done in Proxmox before as shown here.
Click to expand...
Ah, it's true, someone have done this before!
The problem of other solutions, like the one an this Thread, in my case, is that I need to specify the ProductId and VendorId, not only the serial. By using the qemu to create the virtual USB I can't set the PID and VID (it uses a QEMU one), and it doesnt work for me.
And yes, the USB is bootable!
Now I need to find a way to load the module g_mass_storage file=/mnt/backing_file idVendor=.... on proxmox boot. dummy_hcd module load on boot after editing /etc/modules, g_mass_storage dont, and I think that the problem is the arguments

edit: did it! now it's ready for my VM after boot
 
Last edited:
heringer said:
Ah, it's true, someone have done this before!
The problem of other solutions, like the one an this Thread, in my case, is that I need to specify the ProductId and VendorId, not only the serial. By using the qemu to create the virtual USB I can't set the PID and VID (it uses a QEMU one), and it doesnt work for me.
And yes, the USB is bootable!
Now I need to find a way to load the module g_mass_storage file=/mnt/backing_file idVendor=.... on proxmox boot. dummy_hcd module load on boot after editing /etc/modules, g_mass_storage dont, and I think that the problem is the arguments

edit: did it! now it's ready for my VM after boot
Click to expand...
How did you get the module to load? Can you share the instructions for what you did? Thanks!
 
eclipxe said:
How did you get the module to load? Can you share the instructions for what you did? Thanks!
Click to expand...
on /etc/modprobe.d I made a file 1-mass_storage.conf with the content: options g_mass_storage file=xxxxx idVendor=0x0781 idProduct=xxxxxx iManufacturer=xxxxxxxx iProduct=xxxxxx iSerialNumber=xxxxxxx

and on /etc/modules-load.d:
0-usbvirtual.conf with the content dummy_hcd
and 1-mass_storage.conf with the content g_mass_storage
 
I just wanted to say Huge Thanks in particular to @PanWaclaw, @gfngfn256 and @Mirmanium.

Indeed without using XHCI it took AGES to install Debian 12 on the USB Image.

After switching to XHCI, it's super Fast ;).

I tried to collect the Information and put it into a somewhat easy to use Script to attach/detach Virtual USB Flashdrive from a VM.

You can find it here:
https://github.com/luckylinux/pve-scripts/blob/main/configure_virtual_usb_flashdrive.sh

The "Trick" is to add bootindex=0 at the end of the -args Command Line is what makes the System boot from USB, since it's NOT possible to configure boot the "normal" Way (by setting boot: order=usb0;sata0;ide0 or something along these Lines)

If you do NOT add bootindex=0 to the -args, it's still possible to boot to USB, but that will NOT occur automatically.
You must in such Case act fast in the Console by continuosly pressing ESC as soon as the VM is started, and then selecting the right Device (which IMHO is not always so easy to recognize, although with XHCI Driver it's correctly recognized as USB MSC Drive QEMU QEMU HARDDISK 2.5+):

1743943200599.png

It's also possible to add a ,serial=0123456789 (or whatever Serial Number you desire) before the ,bootindex=0 Directive in case you want to easily recognize the Device after Boot:

Code: 
root@liveusb-proxmox:~# ls -l /dev/disk/by-id/
total 0
lrwxrwxrwx 1 root root  9 Apr  6 14:30 ata-QEMU_DVD-ROM_QM00003 -> ../../sr0
lrwxrwxrwx 1 root root  9 Apr  6 14:30 scsi-0QEMU_QEMU_HARDDISK_drive-scsi0 -> ../../sda
lrwxrwxrwx 1 root root  9 Apr  6 14:30 usb-QEMU_QEMU_HARDDISK_0123456789-0:0 -> ../../sdb
lrwxrwxrwx 1 root root 10 Apr  6 14:30 usb-QEMU_QEMU_HARDDISK_0123456789-0:0-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Apr  6 14:30 usb-QEMU_QEMU_HARDDISK_0123456789-0:0-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Apr  6 14:30 usb-QEMU_QEMU_HARDDISK_0123456789-0:0-part3 -> ../../sdb3


And this is how lsusb Output looks:
Code: 
root@liveusb-proxmox:~# lsusb
Bus 003 Device 002: ID 46f4:0001 QEMU QEMU USB HARDDRIVE
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU Tablet
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

root@liveusb-proxmox:~# lsusb -d 46f4:0001 -vvv

Bus 003 Device 002: ID 46f4:0001 QEMU QEMU USB HARDDRIVE
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.00
  bDeviceClass            0
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0         9
  idVendor           0x46f4 QEMU
  idProduct          0x0001
  bcdDevice            0.00
  iManufacturer           1 QEMU
  iProduct                2 QEMU USB HARDDRIVE
  iSerial                 3 0123456789
  bNumConfigurations      1

  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x002c
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          6 Super speed config (usb 3.0)
    bmAttributes         0xc0
      Self Powered
    MaxPower                0mA

    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0

     Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst              15

      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst              15

Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength       0x0016
  bNumDeviceCaps          2

  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x00000002
      HIRD Link Power Management (LPM) Supported

  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000e
      Device can operate at Full Speed (12Mbps)
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   1
      Lowest fully-functional device speed is Full Speed (12Mbps)
    bU1DevExitLat          10 micro seconds
    bU2DevExitLat          32 micro seconds

can't get debug descriptor: Resource temporarily unavailable

Device Status:     0x0001
  Self Powered

EDIT 1:
Usage Instructions:
Code: 
./configure_virtual_usb_flashdrive.sh attach <vmid> <path_to_img_file>
./configure_virtual_usb_flashdrive.sh detach <vmid> <path_to_img_file>

Usage Example:
Code: 
./configure_virtual_usb_flashdrive.sh attach 99999004 /var/lib/usbimages/Debian_From_Scratch.img
./configure_virtual_usb_flashdrive.sh detach 99999004 /var/lib/usbimages/Debian_From_Scratch.img

I had Issues booting my existing Physical LiveUSB that I imaged using dd, probably because back then (6 Years ago ?) when it was created, it was probably NOT propertly aligned to 4MiB etc.

That's what I could understand at least from this Post here, which matched the Error I was getting: https://forum.proxmox.com/threads/l...ot-get-write-permission-without-resize.89891/.

Hence why the Image is named Debian_From_Scratch.img ;).

EDIT 2: fixed Formatting. Backtick habit from GitHub not working here for Code Snippets o_O.

EDIT 3: Note that I only briefly tested this. I added a check to shutdown the VM if trying to detach the Virtual USB Flashdrive (because I am assuming that you booted from that Drive and NOT the HDD/CD-ROM) as well as a Convenience Do you want to [re]start the VM and boot into LiveUSB ? Message + Action depending on User Input.

I did NOT cover all edge Cases etc. If you have existing settings in args, it should preserve them, and only add (when attaching) or strip away (when detaching) the Settings defined in the drive_str Line.

EDIT 4: to be clear, if you are also getting the Error that I was getting described at the end of (EDIT 1), you cannot use the Physical USB Drive and Image it to build a Virtual USB Drive. Or at least you must fix the Alignment first.

I thought it would take less Time to just create an Empty Virtual Disk with dd, mount a Debian 12 AMD64 ISO as CD-ROM and the (currently empty) USB Virtual Drive Image in a VM, then just perform the Installation as Usual (I personally make sure that I setup BOTH a BIOS/GRUB Partition + EFI FAT32 Partition + "/" EXT4 Partition and Optionally a separate /boot and Swap, although the latter 2 shouldn't be needed in normal Circumstances).

So I just created a 32GB empty Virtual Drive and run the Installer from within the VM :):
Code: 
dd bs=1G count=32 if=/dev/zero of=/var/lib/usbimages/Debian_From_Scratch.img
 
Last edited:
  • Like
Reactions: crypted and gfngfn256
You must log in or register to reply here.
Top Bottom