Lots of ZFS @___replicate__ snapshots remaining

[AFrenchCompany]

Hello,

After removing all snapshots of all VM/LXC using the GUI, we find the following snapshots that remain.

Some timestamps are almost 6 month old.
Are those required by PVE's replication system? Or could they be removed and how ?

If that's relevant, replications are from 15 minutes appart to 1 day appart.

Thanks

pve-manager/8.1.3/b46aac3b42da5d15 (running kernel: 6.5.11-7-pve)

Output of zfs list -t snapshot on one node.

NAME                                                                                     USED  AVAIL  REFER  MOUNTPOINT
local-vms/subvol-102-disk-1@__replicate_102-1_1693227601__                              1.97G      -  7.12G  -
local-vms/subvol-102-disk-1@__replicate_102-0_1702304118__                               728K      -  5.46G  -
local-vms/subvol-103-disk-2@__replicate_103-1_1693227600__                               883M      -  1.21G  -
local-vms/subvol-103-disk-2@__replicate_103-0_1702302303__                                 0B      -   985M  -
local-vms/subvol-103-disk-3@__replicate_103-1_1693227600__                               133G      -   775G  -
local-vms/subvol-103-disk-3@__replicate_103-0_1702302303__                              11.8M      -   700G  -
local-vms/subvol-105-disk-0@__replicate_105-1_1693227619__                              1.50G      -  2.23G  -
local-vms/subvol-105-disk-0@__replicate_105-0_1702303203__                              1.40M      -  3.09G  -
local-vms/subvol-105-disk-1@__replicate_105-1_1693227619__                               826G      -  8.62T  -
local-vms/subvol-105-disk-1@__replicate_105-0_1702303203__                                 0B      -  9.12T  -
local-vms/subvol-106-disk-2@__replicate_106-1_1693227629__                              1.64G      -  2.08G  -
local-vms/subvol-106-disk-2@__replicate_106-0_1702304103__                              5.32M      -  2.69G  -
local-vms/subvol-106-disk-3@__replicate_106-1_1693227629__                              2.49G      -  35.7G  -
local-vms/subvol-106-disk-3@__replicate_106-0_1702304103__                               268K      -  36.5G  -
local-vms/subvol-107-disk-0@__replicate_107-1_1693227638__                              2.26G      -  3.83G  -
local-vms/subvol-107-disk-0@__replicate_107-0_1702304109__                               174K      -  3.77G  -
local-vms/subvol-108-disk-0@__replicate_108-1_1693173637__                              1.35G      -  2.19G  -
local-vms/subvol-108-disk-0@__replicate_108-0_1702249217__                                 0B      -  2.98G  -
local-vms/subvol-110-disk-0@__replicate_110-2_1675294205__                               118G      -   297G  -
local-vms/subvol-110-disk-0@__replicate_110-3_1675726204__                               116G      -   294G  -
local-vms/subvol-110-disk-0@__replicate_110-1_1693175436__                              45.1G      -   222G  -
local-vms/subvol-110-disk-0@__replicate_110-0_1702251000__                              68.5M      -   253G  -
local-vms/subvol-111-disk-0@__replicate_111-1_1693227652__                              2.68G      -  5.90G  -
local-vms/subvol-111-disk-0@__replicate_111-0_1702304122__                              12.9M      -  6.02G  -
local-vms/subvol-111-disk-1@__replicate_111-1_1693227652__                               630M      -   328G  -
local-vms/subvol-111-disk-1@__replicate_111-0_1702304122__                                 0B      -   357G  -
local-vms/subvol-112-disk-0@__replicate_112-2_1675762247__                              1.26G      -  4.50G  -
local-vms/subvol-112-disk-0@__replicate_112-1_1693173653__                              1.32G      -  4.64G  -
local-vms/subvol-112-disk-0@__replicate_112-0_1702249200__                               752M      -  14.0G  -
local-vms/subvol-113-disk-0@__replicate_113-1_1693173609__                              1.57G      -  2.17G  -
local-vms/subvol-113-disk-0@__replicate_113-0_1702249215__                               120M      -  3.57G  -
local-vms/subvol-114-disk-1@__replicate_114-0_1702249221__                                 0B      -  10.6G  -
local-vms/subvol-115-disk-0@__replicate_115-1_1693090807__                               665M      -   912M  -
local-vms/subvol-115-disk-0@__replicate_115-0_1702166402__                              4.29M      -  1.72G  -
local-vms/subvol-115-disk-1@__replicate_115-1_1693090807__                              6.80M      -  1.12T  -
local-vms/subvol-115-disk-1@__replicate_115-0_1702166402__                               734K      -  1.12T  -
local-vms/subvol-116-disk-1@__replicate_116-1_1693227666__                              2.74G      -  10.3G  -
local-vms/subvol-116-disk-1@__replicate_116-0_1702303218__                               181K      -  13.8G  -
local-vms/subvol-118-disk-0@__replicate_118-1_1693227605__                               998M      -  3.23G  -
local-vms/subvol-118-disk-0@__replicate_118-0_1702302328__                              4.21M      -  4.09G  -
local-vms/subvol-118-disk-1@__replicate_118-1_1693227605__                              1.20G      -  1.20G  -
local-vms/subvol-118-disk-1@__replicate_118-0_1702302328__                              2.00M      -  1.24G  -
local-vms/subvol-120-disk-1@__replicate_120-1_1693173637__                               816M      -  35.4G  -
local-vms/subvol-120-disk-1@__replicate_120-0_1702249221__                              64.7M      -  35.8G  -
local-vms/subvol-121-disk-0@__replicate_121-0_1702249226__                              2.74M      -  1.68G  -
local-vms/subvol-122-disk-0@__replicate_122-1_1693173645__                              7.64M      -   834M  -
local-vms/subvol-122-disk-0@__replicate_122-0_1702249229__                              1.04M      -   849M  -
local-vms/subvol-124-disk-0@__replicate_124-1_1693173706__                               612M      -  3.15G  -
local-vms/subvol-124-disk-0@__replicate_124-0_1702249225__                               134K      -  3.41G  -
local-vms/subvol-124-disk-1@__replicate_124-1_1693173706__                              3.94G      -   540G  -
local-vms/subvol-124-disk-1@__replicate_124-0_1702249225__                                 0B      -   632G  -
local-vms/subvol-125-disk-1@__replicate_125-0_1702304128__                               359K      -  8.24G  -
local-vms/subvol-126-disk-0@__replicate_126-0_1702302307__                              4.63M      -  19.9G  -
local-vms/subvol-127-disk-0@__replicate_127-0_1702249300__                              1.51M      -  21.5G  -
local-vms/subvol-127-disk-1@__replicate_127-0_1702249300__                              6.81M      -  12.9M  -
local-vms/subvol-128-disk-0@__replicate_128-0_1693173710__                              44.8M      -   678M  -
local-vms/subvol-128-disk-0@__replicate_128-1_1702249305__                              1.28M      -   884M  -
local-vms/subvol-129-disk-0@__replicate_129-0_1702249308__                              1.37M      -  2.74G  -
local-vms/vm-101-disk-0@__replicate_101-1_1693228861__                                     0B      -  41.6G  -
local-vms/vm-101-disk-0@__replicate_101-0_1702304107__                                     0B      -  41.6G  -
local-vms/vm-101-disk-1@__replicate_101-1_1693228861__                                  7.55G      -  53.2G  -
local-vms/vm-101-disk-1@__replicate_101-0_1702304107__                                  12.8M      -  55.3G  -
local-vms/vm-104-disk-0@__replicate_104-1_1693173615__                                  1.44G      -  4.64G  -
local-vms/vm-104-disk-0@__replicate_104-0_1702249203__                                     0B      -  3.64G  -
local-vms/vm-109-disk-0@__replicate_109-1_1693228861__                                  8.29G      -  25.6G  -
local-vms/vm-109-disk-0@__replicate_109-0_1702304115__                                     0B      -  24.3G  -
local-vms/vm-119-disk-0@__replicate_119-1_1680785150__                                     0B      -   571K  -
local-vms/vm-119-disk-0@__replicate_119-0_1702302328__                                     0B      -   571K  -
local-vms/vm-119-disk-1@__replicate_119-0_1702302328__                                     0B      -   538K  -
local-vms/vm-119-disk-2@__replicate_119-0_1702302328__                                     0B      -  86.7G  -
local-vms/vm-123-disk-0@__replicate_123-1_1693173676__                                  60.4G      -   125G  -
local-vms/vm-123-disk-0@__replicate_123-0_1702249233__                                  1.17G      -   110G  -
local-vms/vm-130-disk-0@__replicate_130-0_1702249311__                                     0B      -  4.87G  -

 

[AFrenchCompany]

Upping the thread (impressed by the volume of posts in this forum!)

 

[fabian]

yes, those are used by replication. if the timestamps are old, maybe that guest is no longer replicated?

 

[AFrenchCompany]

They are all still replicated.
There are multiple snapshots for the same disks.

For example

103 disk 2 has :
1693227600 -> August 28th 2023
and
1702302303 -> December 11th 2023


119 disk 0 has :
1680785150 -> April 6th 023
and
1702302328 -> December 11th 2023

 

[fabian]

these are for two different replication jobs though (103-0 and 103-1). each replication target requires one snapshot to be kept on the source side (representing the last replicated state for that target).