I might have found out why "stale file handle". Maybe because previous job hasnt finished bulk transfer despite GUI showing transfer is done. See example below:
INFO: starting new backup job: vzdump 101 --notes-template '{{guestname}}' --mode snapshot --compress zstd --node pve1 --storage pve-backups-REMOTE --remove 0 --notification-mode auto
INFO: Starting Backup of VM 101 (qemu)
INFO: Backup started at 2025-08-16 17:08:38
INFO: status = running
INFO: VM Name: vm-desktop
INFO: include disk 'scsi0' 'local-zfs:vm-101-disk-0' 24G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: pending configuration changes found (not included into backup)
INFO: creating vzdump archive '/mnt/pve/pve-backups-REMOTE/dump/vzdump-qemu-101-2025_08_16-17_08_38.vma.zst'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '12c93aa0-ff45-4598-b178-d8b7ccba57f8'
INFO: resuming VM again
INFO: 7% (1.7 GiB of 24.0 GiB) in 3s, read: 574.0 MiB/s, write: 421.9 MiB/s
INFO: 12% (3.1 GiB of 24.0 GiB) in 6s, read: 471.2 MiB/s, write: 410.2 MiB/s
INFO: 22% (5.5 GiB of 24.0 GiB) in 9s, read: 832.5 MiB/s, write: 413.8 MiB/s
INFO: 28% (6.9 GiB of 24.0 GiB) in 12s, read: 461.6 MiB/s, write: 407.8 MiB/s
INFO: 33% (8.1 GiB of 24.0 GiB) in 15s, read: 429.0 MiB/s, write: 376.1 MiB/s
INFO: 41% (9.9 GiB of 24.0 GiB) in 18s, read: 613.8 MiB/s, write: 455.5 MiB/s
INFO: 46% (11.2 GiB of 24.0 GiB) in 21s, read: 433.8 MiB/s, write: 409.4 MiB/s
INFO: 51% (12.3 GiB of 24.0 GiB) in 24s, read: 367.3 MiB/s, write: 340.1 MiB/s
INFO: 57% (13.8 GiB of 24.0 GiB) in 27s, read: 516.0 MiB/s, write: 458.8 MiB/s
INFO: 62% (15.0 GiB of 24.0 GiB) in 30s, read: 416.9 MiB/s, write: 346.0 MiB/s
INFO: 77% (18.6 GiB of 24.0 GiB) in 33s, read: 1.2 GiB/s, write: 338.7 MiB/s
INFO: 94% (22.8 GiB of 24.0 GiB) in 36s, read: 1.4 GiB/s, write: 306.4 MiB/s
INFO: 100% (24.0 GiB of 24.0 GiB) in 38s, read: 630.0 MiB/s, write: 162.0 MiB/s
INFO: backup is sparse: 9.96 GiB (41%) total zero data
INFO: transferred 24.00 GiB in 38 seconds (646.7 MiB/s)
INFO: archive file size: 5.02GB
INFO: adding notes to backup
INFO: Finished Backup of VM 101 (00:04:01)
INFO: Backup finished at 2025-08-16 17:12:39
INFO: Backup job finished successfully
INFO: notified via target `mail-to-root`
TASK OK
Actuall write was still in progress (i observed packet flow with tcpdump). I got suspicious because actual transfer speed to remote CIFS location was below 100mbps for that particular flow. However INFO stated it was 646MiB/s.
When transfer was finished I started new job and got not error about stale file.
Therefore I suspect the slow response from CIFS was causing "stale file handle" error because previous transfer wasnt really finished.