Search results

  1. grin

    [SOLVED] Missing chunks, how to fix - update atime failed for chunk - ENOENT: No such file or directory

    When you create a new, incremental backup, and you have a chunk with H1 hash, which is already known to the system, do you actually verify its existence in the disk or just reference it (based on some hatabase of known chunks)? I suspect that H1 was in the old backup and the file is missing...
  2. grin

    [SOLVED] Missing chunks, how to fix - update atime failed for chunk - ENOENT: No such file or directory

    So, there was the successful GC on 08-19 00:02. One minute before that there was a 2020-08-19T00:00:00+02:00: starting garbage collection on store pub...
  3. grin

    [SOLVED] Missing chunks, how to fix - update atime failed for chunk - ENOENT: No such file or directory

    * what disks are used for the LVM? are they healthy? Dell PERC 740p hardware RAID-6 array, 2 PVs, 1 VG/LV, 18TB in size. Everyone's healthy and happy. * how big is the CT (rough estimate) 1.81 TB, give or take a few gigs. * what big files are in there, or are there just so many small...
  4. grin

    [SOLVED] Missing chunks, how to fix - update atime failed for chunk - ENOENT: No such file or directory

    2020-08-25T10:21:57+02:00: WARN: warning: unable to access chunk xx, required by "/mnt/datastore/pub/ct/666/2020-08-21T02:33:47Z/root.pxar.didx" - update atime failed for chunk "/mnt/datastore/pub/.chunks/fa39/xxx" - ENOENT: No such file or directory This have been killing GC (which is fixed in...
  5. grin

    GC failed with TASK ERROR

    No, GC haven't been able to run the last week or so. Only backups are running continuously (I yet to see why some of the packups run 20+ hours). However right now proxmox-backup-manager task list says only one backup job is running, and indeed now GC was able to run. So far it have generated...
  6. grin

    GC failed with TASK ERROR

    This was not intentional, and if you look at them you may have noticed that progressively more and more information was revealed about the problem and the reason while the same time I have tried to figure out _where_ it should have been posted, and the same time realised that posts cannot be...
  7. grin

    GC failed with TASK ERROR

    Same here: a chunk is missing (disappeared) and it's killing GC. I am still looking for how to fix, apart from extremely radical measures.
  8. grin

    [SOLVED] garbage collection spam logs by repeatedly failing due to running backup

    There is a backup taking 12h+, which is okay. GC is daily. Now, GC isn't happy when trying to clean up files of an active backup: 2020-08-23T18:34:00+02:00: starting garbage collection on store pub 2020-08-23T18:34:00+02:00: task triggered by schedule 'daily' 2020-08-23T18:34:00+02:00: Start...
  9. grin

    corosync 3.0.2-pve4 memory leak

    557b49a8b000-557fb2d9a000 rw-p 00000000 00:00 0 [heap] Size: 18500668 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Rss: 18500668 kB Pss: 18500668 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean...
  10. grin

    corosync 3.0.2-pve4 memory leak

    I have a (possibly unrelated) problem where one cluster member's TOTEM keep generateing retransmits from other nodes or local timeouts. When I checked nodeC kept joining and leaving the cluster, about 50 per seconds. Restarting corosync resulted the same. It turned out however that on other...
  11. grin

    [pve6] ceph lumi to nautilus guide problem: Required devices (block and data) not present for bluest

    # ceph-volume simple scan stderr: lsblk: /var/lib/ceph/osd/ceph-2: not a block device stderr: Bad argument "/var/lib/ceph/osd/ceph-2", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument Running...
  12. grin

    serios problems with lxc containers

    https://github.com/lxc/lxc/issues/2141 Namespace clone results in hang: uninterruptible sleep #2141 Seems to be fixed in kernel 4.14.16 - 4.14.20
  13. grin

    FYI Firefox nightly (68.0a1 (2019-03-25)) won't display ui anymore

    Indeed it works in a private. Thanks! However it'd not be the ideal solution, so I went on looking, and it doesn't seem to be really happy; I document it here if anyone would get bitten by it later. So, I have several extensions (mostly Conex, uBlock Origin, uMatrix and privacyBadger) which...
  14. grin

    FYI Firefox nightly (68.0a1 (2019-03-25)) won't display ui anymore

    I will walk through my extensions, and report back if I find anyithing.
  15. grin

    FYI Firefox nightly (68.0a1 (2019-03-25)) won't display ui anymore

    Well, there's really not much room for that: UI doesn't seem to use cookies before auth (FF doesn't see cookie for the host at all), and all pages are retrieved directly (no page caching). It was working 3 days ago, so clearly forefox have changed something but the error message is really not...
  16. grin

    FYI Firefox nightly (68.0a1 (2019-03-25)) won't display ui anymore

    NS_ERROR_UNEXPECTED: ext-all.js:22 ExtJS 4 getKeys readLocalStorage constructor callParent constructor https://x:8006/pve2/js/pvemanagerlib.js?ver=5.2-8:2238 ExtJS 3 g anonymous create initComponent...
  17. grin

    Upgrade breaks pvestatd against CTs

    This was a rather impolite response. First, one don't restart the system unless it is absolutely necessary, and as far as I am aware there weren't critical and incompatible kernel issues which would have make reboot compulsory and immediate. Often reboots do not happen the same time as...
  18. grin

    Upgrade breaks pvestatd against CTs

    Actually the (somewhat verbose) "fix": --- LXC.pm.orig 2019-03-19 01:29:09.176628691 +0100 +++ LXC.pm 2019-03-19 01:39:05.055046924 +0100 @@ -334,6 +334,12 @@ my $nsdir = $unprivileged ? '' : 'ns/'; my $path = "/sys/fs/cgroup/$group/lxc/$vmid/${nsdir}$name"; +if( ! -r $path &&...
  19. grin

    Upgrade breaks pvestatd against CTs

    Mar 19 01:10:30 freddy pvestatd[2324627]: lxc status update error: can't open '/sys/fs/cgroup/blkio/lxc/117/ns/blkio.throttle.io_service_bytes' - No such file or directory Mar 19 01:10:34 freddy pvestatd[2324627]: lxc console cleanup error: can't open...