sshfs shared storage

bread-baker

Member
Mar 6, 2010
432
0
16
I think it would be good to allow sshfs for shared storage.

For us sshfs is more reliable then nfs . We use sshfs for user documents - with $HOME/Documents targeted to a kvm sshfs . We've done that for 3 years and have had very few issues.
 
never tried but just mount via /etc/fstab and then configure a shared directory storage, does that work for you?
 
the mount and share work, thanks for the idea.

however restoring an openvz template does not work. If I remember correctly, sshfs and links do wot get along. here is some of the error message at pve :
Code:
Creating container private area (/mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz)
tar: ./lib/libnl-3.so.200: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/terminfo/r/rxvt-m: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/libxtables.so.7: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libnsl.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/librt.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libuuid.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libudev.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libpng12.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libncurses.so.5: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libpamc.so.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libbz2.so.1.0: Cannot change ownership to uid 0, gid 0: No such file or directory
tar: ./lib/x86_64-linux-gnu/libbz2.so.1: Cannot change ownership to uid 0, gid 0: No such file or directory
...
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/SOAPpy-0.12.0.egg-info: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/python_debianbts-1.10.egg-info: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/SOURCES.txt: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/top_level.txt: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/PKG-INFO: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/reportbug-6.3.1.egg-info/dependency_links.txt: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/pymodules/python2.7/debianbts.py: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./usr/lib/python2.7/sitecustomize.py: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: ./lib64/ld-linux-x86-64.so.2: Cannot change ownership to uid 0, gid 0: No such file or directory[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]tar: Exiting with failure status due to previous errors[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]vps-create ERROR: Unpack /mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz failed[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]Creation of container private area failed[/FONT][/COLOR] 

TASK ERROR: command 'vzctl --skiplock create 8010 --ostemplate /mnt/pve/fbc1-nfs/template/cache/debian-7.0-FBC_7.0_amd64.tar.gz --private /fbc246-data/private/8010' failed: exit code 48

I think the error lines are all due to symlinks ...

Next I'll try a kvm on sshfs.
 
Last edited:
Code:
[COLOR=#000000][FONT=tahoma]May 30 12:59:58 starting migration of VM 8020 to node 'fbc240' (10.100.100.240)[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 12:59:58 copying disk images[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 12:59:58 starting VM 8020 on remote node 'fbc240'[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 12:59:59 starting migration tunnel[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 13:00:00 starting online/live migration on port 60000[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 13:00:02 migration status: completed[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 13:00:02 migration speed: 256.00 MB/s[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]May 30 13:00:04 migration finished successfuly (duration 00:00:06)[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]TASK OK

so on line migration works . but I'll use the kvm a while and check for problems.[/FONT][/COLOR]
 
I had too many errors with rsync and the kvm on sshfs .

the issues may have had nothing to do with sshfs, b ut the system i restored works without issues on drbd and local storage.

So I give up on sshfs for now. Will use nfs instead.

here were some of the issues:
Code:
[mntent]: warning: no final newline at the end of /etc/mtab                                                                                    
[mntent]: warning: no final newline at the end of /etc/mtab                                                                                    
[mntent]: warning: no final newline at the end of /etc/mtab


inflate returned -3 (0 bytes)                                                                                                                  
rsync error: error in rsync protocol data stream (code 12) at token.c(547) [receiver=3.0.3]                                                    
rsync: connection unexpectedly closed (10290 bytes received so far) [generator]                                                                
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [generator=3.0.3]     


rsync: readlink "/bkup/u/hadjx1.pro" failed: Input/output error (5)                                                                            
rsync: recv_generator: failed to stat "/bkup/u/hadjx1.pro": Input/output error (5)                                                             
rsync error: some files could not be transferred (code 23) at main.c(1524) [generator=3.0.3]
 
Last edited:
Using sshfs for VM storage is certainly an intriguing idea.

The main issue I see is that SSH uses a lot of CPU unless you have hardware accelerated crypto.
Many CPU's have AES-NI acceleration but squeeze does not have AES-NI enabled for SSH.

The fastest cipher, without hardware acceleration, is arcfour.
If you are using sshfs on a trusted network using arcfour cipher for the sshfs connection should help with performance and reduce your CPU usage.

Just pass the option when mounting sshfs:
Code:
-o Ciphers=arcfour
 
hi there e100

i have this in our global ssh_config. do you know if arcfour weaknesses are fixed? In any case It would be great id sshfs issue could get fixed. I've an extra server that I can use for testing if any one has suggestions for retesting sshfs.
Code:
# Changing the ciphers used by SSH to less cpu-demanding ones can improve speed. In this aspect, the best choices are arcfour and blowfish-cbc. Please do not do this unless you know what you are doing; arcfour has a number of known weaknesses. To use them, run SSH with the "c" flag, like this: 
# 2012-04-21 put blowfish-cbc 1-st as 'arcfour has a number of known weaknesses.'
Ciphers blowfish-cbc,arcfour
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!