I'm unable to upload files to my proxmox server

[fleabeard]

Hello,
Strange problem I'm having. I'm trying to upload an .IMG file via the proxmox pve webui under Datacenter > pve > local (pve) > ISO Images but it's only xferring a very small amount before it just stops doing anything. Same thing happens when I try to upload the .IMG from proxmox pve webui under Datacenter > pve > zfs_mp (pve) > ISO Images.




Screenshot of `local (pve)` Summary:


Screenshot of `zfs_mp (pve)` Summary:




Results of pveversion:

Linux pve 5.13.19-1-pve #1 SMP PVE 5.13.19-2 (Tue, 09 Nov 2021 12:59:38 +0100) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Sep  2 13:00:58 2022 from 192.168.40.122
root@pve:~# pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.13.19-1-pve)
pve-manager: 7.1-4 (running version: 7.1-4/ca457116)
pve-kernel-5.13: 7.1-4
pve-kernel-helper: 7.1-4
pve-kernel-5.13.19-1-pve: 5.13.19-2
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-1
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-3
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.14-1
proxmox-backup-file-restore: 2.0.14-1
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.4-2
pve-cluster: 7.1-2
pve-container: 4.1-2
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-1
pve-qemu-kvm: 6.1.0-2
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-3
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3

Results of me trying to SCP an ISO to my proxmox server in /tmp (note that the transfer never goes past 0%):

scp -v /home/fleabeard/Downloads/elementaryos-6.1-stable.20211218-rc.iso root@192.168.69.2:/tmp
Executing: program /usr/bin/ssh host 192.168.69.2, user root, command scp -v -t /tmp
OpenSSH_8.9p1 Ubuntu-3, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to 192.168.69.2 [192.168.69.2] port 22.
debug1: Connection established.
debug1: identity file /home/fleabeard/.ssh/id_rsa type 0
debug1: identity file /home/fleabeard/.ssh/id_rsa-cert type -1
debug1: identity file /home/fleabeard/.ssh/id_ecdsa type -1
debug1: identity file /home/fleabeard/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/fleabeard/.ssh/id_ecdsa_sk type -1
debug1: identity file /home/fleabeard/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /home/fleabeard/.ssh/id_ed25519 type -1
debug1: identity file /home/fleabeard/.ssh/id_ed25519-cert type -1
debug1: identity file /home/fleabeard/.ssh/id_ed25519_sk type -1
debug1: identity file /home/fleabeard/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /home/fleabeard/.ssh/id_xmss type -1
debug1: identity file /home/fleabeard/.ssh/id_xmss-cert type -1
debug1: identity file /home/fleabeard/.ssh/id_dsa type -1
debug1: identity file /home/fleabeard/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4p1 Debian-5+deb11u1
debug1: compat_banner: match: OpenSSH_8.4p1 Debian-5+deb11u1 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.69.2:22 as 'root'
debug1: load_hostkeys: fopen /home/fleabeard/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:d7/c71x9GeugN9jfTYVlKSob4K2qCyAarmyCwJp22sY
debug1: load_hostkeys: fopen /home/fleabeard/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '192.168.69.2' is known and matches the ED25519 host key.
debug1: Found key in /home/fleabeard/.ssh/known_hosts:7
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: agent returned 1 keys
debug1: Will attempt key: /home/fleabeard/.ssh/id_rsa RSA SHA256:qXCS673vALAn+0IuRybbf16H741/NgfcpeLjVFwUalE agent
debug1: Will attempt key: /home/fleabeard/.ssh/id_ecdsa
debug1: Will attempt key: /home/fleabeard/.ssh/id_ecdsa_sk
debug1: Will attempt key: /home/fleabeard/.ssh/id_ed25519
debug1: Will attempt key: /home/fleabeard/.ssh/id_ed25519_sk
debug1: Will attempt key: /home/fleabeard/.ssh/id_xmss
debug1: Will attempt key: /home/fleabeard/.ssh/id_dsa
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering public key: /home/fleabeard/.ssh/id_rsa RSA SHA256:qXCS673vALAn+0IuRybbf16H741/NgfcpeLjVFwUalE agent
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/fleabeard/.ssh/id_ecdsa
debug1: Trying private key: /home/fleabeard/.ssh/id_ecdsa_sk
debug1: Trying private key: /home/fleabeard/.ssh/id_ed25519
debug1: Trying private key: /home/fleabeard/.ssh/id_ed25519_sk
debug1: Trying private key: /home/fleabeard/.ssh/id_xmss
debug1: Trying private key: /home/fleabeard/.ssh/id_dsa
debug1: Next authentication method: password
root@192.168.69.2's password:
Authenticated to 192.168.69.2 ([192.168.69.2]:22) using "password".
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: filesystem
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: client_input_hostkeys: searching /home/fleabeard/.ssh/known_hosts for 192.168.69.2 / (none)
debug1: client_input_hostkeys: searching /home/fleabeard/.ssh/known_hosts2 for 192.168.69.2 / (none)
debug1: client_input_hostkeys: hostkeys file /home/fleabeard/.ssh/known_hosts2 does not exist
debug1: Sending environment.
debug1: channel 0: setting env LANG = "en_US.UTF-8"
debug1: Sending command: scp -v -t /tmp
debug1: client_global_hostkeys_private_confirm: server used untrusted RSA signature algorithm ssh-rsa for key 0, disregarding
debug1: update_known_hosts: known hosts file /home/fleabeard/.ssh/known_hosts2 does not exist
scp: debug1: fd 3 clearing O_NONBLOCK
Sending file modes: C0644 2662367232 elementaryos-6.1-stable.20211218-rc.iso
Sink: C0644 2662367232 elementaryos-6.1-stable.20211218-rc.iso
elementaryos-6.1-stable.20211218-rc.iso         0%    0     0.0KB/s   --:-- ETA

As you can see, the filename gets written to /tmp but it has a size of 32K:

root@pve:~# ls -la /tmp
total 68
drwxrwxrwt  9 root root  4096 Sep  5 10:16 .
drwxr-xr-x 19 root root  4096 Dec  2  2021 ..
-rw-r--r--  1 root root 32768 Sep  5 10:16 elementaryos-6.1-stable.20211218-rc.iso
drwxrwxrwt  2 root root  4096 Sep  3 16:43 .font-unix
drwxrwxrwt  2 root root  4096 Sep  3 16:43 .ICE-unix
drwx------  3 root root  4096 Sep  3 16:43 systemd-private-6f44b0aa71004a368c6926c6668ba49d-chrony.service-wmxMej
drwx------  3 root root  4096 Sep  3 16:43 systemd-private-6f44b0aa71004a368c6926c6668ba49d-systemd-logind.service-s2ofrg
drwxrwxrwt  2 root root  4096 Sep  3 16:43 .Test-unix
drwxrwxrwt  2 root root  4096 Sep  3 16:43 .X11-unix
drwxrwxrwt  2 root root  4096 Sep  3 16:43 .XIM-unix

root@pve:~# du -sh /tmp/elementaryos-6.1-stable.20211218-rc.iso
32K    /tmp/elementaryos-6.1-stable.20211218-rc.iso

Attached results from journalctl to post.

I've also looked at the javascript console in Chrome but there is no errors at all when attempting to upload an ISO.

I'm unsure where to start as to why this is happening and would appreciate any help you can provide.
Thank you!

 

[Dunuin]

No idea but PVE will upload ISOs to "/var/tmp" and then move them to the final destination. Did you maybe change something with /var/tmp like moving it to RAM disk or something similar?

 

[fleabeard]

No idea but PVE will upload ISOs to "/var/tmp" and then move them to the final destination. Did you maybe change something with /var/tmp like moving it to RAM disk or something similar?

I have not made any configuration changes to my pve instance since I first installed it over a year ago except from doing the updates that are available on it.

 

[Neobin]

I have not made any configuration changes to my pve instance since I first installed it over a year ago except from doing the updates that are available on it.

proxmox-ve: 7.1-1 (running kernel: 5.13.19-1-pve)
pve-manager: 7.1-4 (running version: 7.1-4/ca457116)

Current version is: 7.2

If you do not have a active subscription [1], you need to disable/comment out the pve-enterprise repository [2] (see the note there) and add the pve-no-subscription repository [3].
Once this is done, update [4] the PVE-host and reboot him afterwards.

[1] https://proxmox.com/en/proxmox-ve/pricing
[2] https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_enterprise_repo
[3] https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_no_subscription_repo
[4] https://pve.proxmox.com/wiki/System_Software_Updates

 

[fleabeard]

Current version is: 7.2

If you do not have a active subscription [1], you need to disable/comment out the pve-enterprise repository [2] (see the note there) and add the pve-no-subscription repository [3].
Once this is done, update [4] the PVE-host and reboot him afterwards.

[1] https://proxmox.com/en/proxmox-ve/pricing
[2] https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_enterprise_repo
[3] https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_no_subscription_repo
[4] https://pve.proxmox.com/wiki/System_Software_Updates

I've removed the enterprise repo, and have since updated my installation to the latest: pve-manager/7.2-7/d0dd0e85 (running kernel: 5.13.19-1-pve)

Rebooted my proxmox server for good measure and tried to upload an .IMG again but sadly, same issue

Something I did notice while being connected via SSH was that my terminal would stop accepting keyboard input but the connection didn't drop (or at least it never notified me the connection dropped) - Not sure what that's on about.

 

[Neobin]

I've removed the enterprise repo, and have since updated my installation to the latest: pve-manager/7.2-7/d0dd0e85 (running kernel: 5.13.19-1-pve)

Rebooted my proxmox server for good measure

Not going to say that this has something to do with your problem or that it will fix it, but is the output from the running kernel before or after you rebooted?
A reboot in this case is not only for good measure, but needed to boot into the newly installed/updated kernel. Without a reboot, the old kernel is still in use. This would explain your output about the running kernel, if it was taken before the reboot.
After the reboot it should be: 5.15.39-4-pve.
If it is not and you want to investigate further, please provide the output in code-tags of:

• pveversion -v
• proxmox-boot-tool status
• proxmox-boot-tool kernel list

BTT: Here: [1] has someone currently a similar problem, but unfortunately no solution yet. (Maybe/Hopefully after the weekend.)

[1] https://forum.proxmox.com/threads/no-upload-to-proxmox-possible.113695

 

[fleabeard]

Not going to say that this has something to do with your problem or that it will fix it, but is the output from the running kernel before or after you rebooted?
A reboot in this case is not only for good measure, but needed to boot into the newly installed/updated kernel. Without a reboot, the old kernel is still in use. This would explain your output about the running kernel, if it was taken before the reboot.
After the reboot it should be: 5.15.39-4-pve.
If it is not and you want to investigate further, please provide the output in code-tags of:

• pveversion -v
• proxmox-boot-tool status
• proxmox-boot-tool kernel list

BTT: Here: [1] has someone currently a similar problem, but unfortunately no solution yet. (Maybe/Hopefully after the weekend.)

[1] https://forum.proxmox.com/threads/no-upload-to-proxmox-possible.113695

My output was prior to rebooting. I'm actually on 5.15.39-4 pve.

Linux pve 5.15.39-4-pve #1 SMP PVE 5.15.39-4 (Mon, 08 Aug 2022 15:11:15 +0200) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat Sep  3 17:02:28 2022 from 192.168.40.122
root@pve:~# pveversion -v
proxmox-ve: 7.2-1 (running kernel: 5.15.39-4-pve)
pve-manager: 7.2-7 (running version: 7.2-7/d0dd0e85)
pve-kernel-5.15: 7.2-9
pve-kernel-helper: 7.2-9
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.39-4-pve: 5.15.39-4
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-1-pve: 5.13.19-3
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-2
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-3
libpve-storage-perl: 7.2-8
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.5-1
proxmox-backup-file-restore: 2.2.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-2
pve-container: 4.2-2
pve-docs: 7.2-2
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-5
pve-firmware: 3.5-1
pve-ha-manager: 3.4.0
pve-i18n: 2.7-2
pve-qemu-kvm: 7.0.0-2
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.5-pve1

 

[fleabeard]

I want to add that I've updated the OP with additional info, like results from me trying to SCP an ISO to the server (output of scp -v) and the results from that.

Additionally, I've noticed that if I'm on the same VLAN as the proxmox server, I can happily upload files with no issues via the webui or scp. I'm not entirely sure if I'm experiencing a networking issue or if it's still a proxmox configuration issue.

 

[ledufakademy]

same issue here with 7.2.4 ... (i'm trying to update ... :-( )
WEB UI always crashing.
Test with filezilla sftp ... same : iso rockylinux 8 GB ... so test with minimal 2.9 GB ... same ....

Note :
rpool/ROOT/pve-1 108G 29G 80G 27% /

Edit : Update done but can't still do 8GB iso xfer .... this is weird.
SFTP , crash then restart.

it's kill all my IOPS then nic too ! (ssh is broken too)

But all VM are running fine !

 

[fleabeard]

I found out about the Syslog today and tried to upload an ISO again to see if it would provide any useful info. This is what I found.
Sep 15 10:25:41 pve pveproxy[4154633]: problem with client ::ffff:192.168.40.122; Connection timed out

Additionally, I noticed my SSH sessions stop responding at random times when I'm SSH'd into my pve server. Here's the Syslog for that too.

Sep 15 10:27:59 pve pveproxy[3909]: worker 4154633 finished
Sep 15 10:27:59 pve pveproxy[3909]: starting 1 worker(s)
Sep 15 10:27:59 pve pveproxy[3909]: worker 47145 started
Sep 15 10:28:00 pve pveproxy[47141]: got inotify poll request in wrong process - disabling inotify
Sep 15 10:28:01 pve pveproxy[47141]: worker exit

 

[fleabeard]

It appears both threads reporting this issue have died. Still having these issues, if someone could be so kind as to help. Thanks

 

[ledufakademy]

hell fleabeard, not dead for me ... always the same issue testing with Windows 2k22 iso ... 5GB : always the same symptoms .... ;-)
and webgui or scp command (ssh) same punishment : must wait 5 or 10 mn to recover access ... (from webgui : "Connection error" after displaying "loading", for ssh : blinking cursor and stuck to 73% of uploading)

Edit : After 8mn , recover ssh access, then webgui , i achieved to move *.iso of 5GB to /var/lib/vz/template/iso/
But again new freeze ...

My PVE 7.2 (os part) is installed on ZFS raid1. on two SSD ==> i'm strongly suspect Proxmox OS install on a ZFS RAID1 pool to be the root cause ???
(i have also weird delay when just cloning a vm on a standard RAID megaraid 16TB logical volume presented to proxmox, but no ZFS here)

Je constate cela dans mes log :

Nov 06 12:11:09 labsrv zed[2376]: Bumping queue length to 2147483647
Nov 06 12:11:09 labsrv zed[2376]: Missed 22 events
Nov 06 12:11:09 labsrv zed[2376]: Bumping queue length to 2147483647
Nov 06 12:11:09 labsrv zed[2376]: Missed 56 events
Nov 06 12:11:09 labsrv zed[2376]: Bumping queue length to 2147483647
Nov 06 12:11:09 labsrv zed[2376]: Missed 24 events
Nov 06 12:11:09 labsrv zed[496660]: eid=169 class=deadman pool='rpool' vdev=ata-KINGSTON_SA400S37120G_50026B73804CA118-part3
size=131072 offset=111320797184 priority=3 err=0 flags=0x180880 bookmark=276:103225:0:32701

 

[Neobin]

i'm strongly suspect Proxmox OS install on a ZFS RAID1 pool to be the root cause ???

pool='rpool' vdev=ata-KINGSTON_SA400S37120G_50026B73804CA118-part3

Not PVE on ZFS is the problem, but I guess your absolutely low-end consumer SSDs in combination with ZFS are...
Seems those are produced (without changing the model number) with either TLC- or QLC-nand. If you got the QLC-version, it is even worse.

 

[ledufakademy]

Not PVE on ZFS is the problem, but I guess your absolutely low-end consumer SSDs in combination with ZFS are...
Seems those are produced (without changing the model number) with either TLC- or QLC-nand. If you got the QLC-version, it is even worse.

ok, thank you for answering : what is your suggestion ? (model )

 

[Dunuin]

Either use HW/onboard raid with LVM on top or get some enterprise SSD for ZFS. What model really depends on your needs. The more durable and write optimized the enterprise SSD is, the more expensive they will get. For entry enterprise grade SSD get something with a powerloss protection, 1 DWPD and TLC NAND. Something like a Intel S4510/S4520, Samsung PM883/PM893 or Micron 5300 Pro/5300 Pro which are for read-intense workloads. If you need more durability and write performance a mixed-use enterprise SSD might be a better choice, like a Intel S4600, Samsung PM897 or a Micron 5200 MAX/5300 MAX/5400 MAX.

 

[fleabeard]

So, I have an update. I created a ubuntu server virtual machine and put it on the same VLAN as my proxmox server and am able to scp files to my virtual machine just fine while on another VLAN, so the issue appears to be with proxmox and not my network/VLAN's.

Still looking for help if anyone cares to chime in. Thanks

 

[ledufakademy]

So, I have an update. I created a ubuntu server virtual machine and put it on the same VLAN as my proxmox server and am able to scp files to my virtual machine just fine while on another VLAN, so the issue appears to be with proxmox and not my network/VLAN's.

Still looking for help if anyone cares to chime in. Thanks

strange for me too, my ssd are kingston .
i suspect that doing webgui (or ssh ) on the same NIC (admin nic at 1GB) and xfer large ISO is the main cause for us.

 

[Docop2]

I always be able to upload any kind of file via web browser with failure and corruption. The only way was quite always to used winscp, filezilla, mobaxtrm... is that because of cache or so.. not sure, but on different system, no zfs stuff no vlan stuff : that give error.

 

[fleabeard]

Just updated my PVE to the latest and decided to see if it would allow me to upload an ISO and same thing. No idea why but this just isn't working for me. It's kind of important to have functional upload/ssh capabilities in a PVE. I guess I'm going to have to migrate things back over to Hyper-V at this point. I really enjoyed my time in Proxmox and learned a ton. But I can't have these little issues that can't seem to be resolved ever prevalent on my system.

Thanks for the ride!