PVE to PBS point to point 10G Fiber high I/O delay

[mare19]

Hi

I tried to configure a point to point connection with a 10G Fiber between PVE Host and PBS.

PBS IP: 10.10.10.2/24
PVE Host IP: 10.10.10.1/24

Adding the PBS to PVE Host worked without problems, but if I want to get the Backup then it comes to high I/O delay on the PBS.

Any suggestions?

Thanks

 

[mare19]

Big problem now, it seems it destroyed the whole ZFS RAID:


All Disks are connected, but now the ZFS pool show that some disk are removed???
What happened?

 

[hvisage]

Big problem now, it seems it destroyed the whole ZFS RAID:
View attachment 57333

All Disks are connected, but now the ZFS pool show that some disk are removed???
What happened?

1) check the disk models whether they are CMR or shingled(sp?)/SMR - the Shingle drives (typically those with like 128MB-256MB or more RAM/cache) are known to break horribly with ZFS
2) personally, I'd rather setup a DRAID2 or a RaidZ3 given your number of disks *AND* have SSD/NVMe SLOG and Cache drives configured as minimum, though I'd advise on a nice special device setup

 

[hvisage]

comes to high I/O delay on the PBS.

Any suggestions?

seems like that might be the clue that you are using SMR and not CMR drives.

 

[mare19]

1) check the disk models whether they are CMR or shingled(sp?)/SMR - the Shingle drives (typically those with like 128MB-256MB or more RAM/cache) are known to break horribly with ZFS
2) personally, I'd rather setup a DRAID2 or a RaidZ3 given your number of disks *AND* have SSD/NVMe SLOG and Cache drives configured as minimum, though I'd advise on a nice special device setup

Okey, I tried with zpool clear and I saw almost all Drives online again, but afterwards it turned back to same state.
Is there anyway to safe the data out? Now when I restart the Server it's stuck and this position:


What can I do now? I have some important Backups there

 

[hvisage]

replace with CMR drives?

Hendrik's rules of computing:
1) Make a backup
2) Make *ANOTHER* backup
2b) At least one backup *off* provider (ie. a totally different DC/owner/etc.)
3) *CHECK* those backups

Sorry, but it looks like those drives aren't in a good state w.r.t. ZFS.

 

[mare19]

I started Proxmox in Recovery Mode and now they are resilvering the pool show online! I hope it will work, and this is caused because I wanted a point to point connection? Very special

 

[hvisage]

No, it's the drives

 

[mare19]

No, it's the drives

I have Exos X10 these should be CMR drives..