Windows VSS Backups Error 12298

Mar 23, 2022
10
2
3
Hi, I have sometimes errors with the VSS Writer making Backups. Windows Event Log Summary:

Code:
Jul 05 12:11:29 0.12298 VSS Volume Shadow Copy Service error: 
The I/O writes cannot be held during the shadow copy creation period on volume \?\Volume{8f7aaeb9-08b6-42dc-8866-e03d56525d84}\. 
The volume index in the shadow copy set is 0. 
Error details: 
Open[0x00000000, The operation completed successfully. ], 
Flush[0x00000000, The operation completed successfully. ], 

Release[0x80042314, The shadow copy provider timed out while holding writes to the volume being shadow copied. This is probably due to excessive activity on the volume by an application or a system service. Try again later when activity on the volume is reduced. ], 

OnRun[0x00000000, The operation completed successfully. ]. 
Operation: Executing Asynchronous Operation Context: Current State: DoSnapshotSet

The Drive always mentioned is C:
Does someone have the same errors and did you do something against it?

Does the Error mean that the Backup is not succesful and that there is a chance that some Data is Missing?

The VSSADMIN LIST WRITERS command lists the following Writers with an error:

Code:
Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {54557dbb-b3e9-462f-9988-a65397299f3b}
   State: [9] Failed
   Last error: Timed out

Writer name: 'Sentinel Agent Log VSS Writer'
   Writer Id: {90e12c7e-b9c2-4649-81a2-947d06dd5eba}
   Writer Instance Id: {cd666d16-dab9-4f4c-937f-a5bda2da3496}
   State: [9] Failed
   Last error: Timed out

Writer name: 'Sentinel Agent DFI Research Data VSS Writer'
   Writer Id: {dcf8278a-dc0f-4b35-afd0-37a07e92fe90}
   Writer Instance Id: {a39e2d7a-3139-4c0c-b9bf-d6d81ed71162}
   State: [9] Failed
   Last error: Timed out

Writer name: 'Sentinel Agent Database VSS Writer'
   Writer Id: {abd005ca-6bfc-44c9-a439-8eb665a676b3}
   Writer Instance Id: {e645a2e2-9685-4fed-8149-dd391711117c}
   State: [9] Failed
   Last error: Timed out

I hope that someone faces simmilar errors and can help me.


System Info:

Windows Server 2022
QEMU GA from iso virtio-win-0.1.215
 
Please post Hardware, Disk-Setup, PVE-Versions, PBS-Versions and VM-Config.
Then we can rub our glass sphere to get an idea what is wrong....
 
Hi Thanks for your Reply,
Server Setup:
PVE:
All-Flash Ceph
AMD 24-Core

PVE Versions:
Code:
proxmox-ve: 7.2-1 (running kernel: 5.15.35-2-pve)
pve-manager: 7.2-4 (running version: 7.2-4/ca9d43cc)
pve-kernel-5.15: 7.2-4
pve-kernel-helper: 7.2-4
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.35-2-pve: 5.15.35-5
pve-kernel-5.15.35-1-pve: 5.15.35-3
pve-kernel-5.15.30-2-pve: 5.15.30-3
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph: 16.2.7
ceph-fuse: 16.2.7
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-2
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-2
libpve-storage-perl: 7.2-4
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.12-1
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
openvswitch-switch: 2.15.0+ds1-2+deb11u1
proxmox-backup-client: 2.2.3-1
proxmox-backup-file-restore: 2.2.3-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-1
pve-container: 4.2-1
pve-docs: 7.2-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.4-2
pve-ha-manager: 3.3-4
pve-i18n: 2.7-2
pve-qemu-kvm: 6.2.0-10
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1

PBS Setup:
2x 8 TB HDD ZFS Mirror with SSD Special Devices

PBS Versions:
Code:
()
proxmox-backup: 2.2-1 (running kernel: 5.15.35-1-pve)
proxmox-backup-server: 2.2.3-2 (running version: 2.2.3)
pve-kernel-5.15: 7.2-4
pve-kernel-helper: 7.2-4
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.35-2-pve: 5.15.35-5
pve-kernel-5.15.35-1-pve: 5.15.35-3
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-1-pve: 5.13.19-3
ifupdown2: 3.1.0-1+pmx3
libjs-extjs: 7.0.0-1
proxmox-backup-docs: 2.2.3-1
proxmox-backup-client: 2.2.3-1
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.5.1
pve-xtermjs: 4.16.0-1
smartmontools: 7.2-pve3
zfsutils-linux: 2.1.4-pve1

VM Config:

Code:
agent: 1
balloon: 8192
bios: ovmf
boot: order=scsi0
cores: 8
cpu: host
efidisk0: shared-ceph:vm-107-disk-0,efitype=4m,pre-enrolled-keys=1,size=528K
hotplug: disk,network,usb,memory,cpu
machine: pc-q35-6.1
memory: 32768
meta: creation-qemu=6.1.1,ctime=1647957577
name: CS-FS01
net0: virtio=2A:DF:DA:0D:CA:CB,bridge=vmbr0
numa: 1
onboot: 1
ostype: win11
scsi0: shared-ceph:vm-107-disk-4,discard=on,size=100G,ssd=1
scsi1: shared-ceph:vm-107-disk-5,discard=on,size=3075G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=e2881a26-667c-4176-a490-2a36b4e1928f
sockets: 1
tpmstate0: shared-ceph:vm-107-disk-1,size=4M,version=v2.0
vga: virtio
vmgenid: 53edd22b-751c-484c-a888-f6675c13d20d


If there is anything else i can do tell me,
 
Can you please "shutdown" not "reboot" the VM and test again after fresh boot?
 
Good Morning, Yesterday I did shutdown the VM and booted it up again unfortunately the error comes back:

Code:
Jul 08 00:12:22 0.12298 VSS Volume Shadow Copy Service error: The I/O writes cannot be held during the shadow copy creation period on volume \\?\Volume{8f7aaeb9-08b6-42dc-8866-e03d56525d84}\. The volume index in the shadow copy set is 0. Error details: Open[0x00000000, The operation completed successfully. ], Flush[0x00000000, The operation completed successfully. ], Release[0x80042314, The shadow copy provider timed out while holding writes to the volume being shadow copied. This is probably due to excessive activity on the volume by an application or a system service. Try again later when activity on the volume is reduced. ], OnRun[0x00000000, The operation completed successfully. ]. Operation: Executing Asynchronous Operation Context: Current State: DoSnapshotSet

Is there someone else with the same problem ?

Are there any logs that I can provide which maybe show the problem?
 
There are ongoing issues with vzdump and latest PVE-Version.
Also in combination with Proxmox Backup Server but not only there.

We have some timeouts during Backup here and sometimes the VSS-Error when in-guest snapshot is created.
As far as I know, Proxmox is aware of this and actively searching for a repro where they can fix it.
 
Hi, thanks for your answer, the error 8194 did also occur but has been fixed with the same steps provided from msp360.

Kind regards

EDIT:

I do use Windows Server Backup but the 12298 Event occured even before.
 
Last edited:
  • Like
Reactions: itNGO
One more Information for everybody facing this EventID 12298 in the future.

in the meantime I replaced all SAS Drives with SSD, now that the Storage is way faster than before the issue does not appear anymore, I hope someone is helped with this Info.

Kind regards
 
HI _gabriel, as I wrote the last reply I didnt check the actuall configuration I wrote, my current production setup includes 2x HPE DL380 with each having PBS on a SAS mirror and 7x8TB SATA SSDs in RaidZ, the 7 SSDs in each Server had been upgraded from SAS Drives last month and after that I didnt notice the Event ID 12298 any more, I suspect that the RaidZ is just to slow for some VMs that they throw a error message, if you use something like a Raid 1 equivalent then I think the issue woulndt appear because the Raid is fast enough.

Kind Regards
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!