"Fun" with .pxarexclude

[t0mc@]

Hi everybody,

for hours now I try to achieve a - for me - simple backup use case using .pxarexclude:
-> Backup just one or two selected, named folders and all its subfolders, ignore every other.

Assume this tree:

/opt/backuptest# tree
.
├── backupme
│   └── file0
├── backupmetoo
│   ├── file0
│   ├── subfolder0
│   │   └── file0
│   └── subfolder1
├── notme1
│   └── file0
├── notme2
│   ├── file0
│   └── subfolder0
├── notme3
│   └── file0
├── notme4
│   └── file0
├── notme5
│   └── file0
├── notme6
│   └── file0
├── notme7
│   └── file0
├── notme8
│   └── file0
└── notme9
    └── file0

Here simply "backupme" and "backupmetoo" should be backed up including all files and subfolders, every other folder should be excluded without explicitly naming it. So the resulting backup should be:

├── backupme
│   └── file0
├── backupmetoo
│   ├── file0
│   ├── subfolder0
│   │   └── file0
│   └── subfolder1

Following this doc: https://pbs.proxmox.com/docs/backup-client.html#excluding-files-directories-from-a-backup I tried this with a .pxarexclude in /opt/backuptest.
The first try was:

*
!backupme
!backupmetoo

But this backed up just empty folders:



Next try:

*
!backupme/*
!backupmetoo/*

leads to nothing be backed up:



Another one:

*/
!backupme
!backupmetoo

Now only files in the first level, but no subfolders are backed up:


Now with this:

*/
!backupme/**/*
!backupmetoo/**/*

Again, nothing backed up except .pxarexclude itself:



I tried much more different combinations of globs, trailing slashes a.s.o. in the exclude and include lines but never found a working solution.

Any hints how to write the correct .pxarexclude for achieving this simple use case?

It would be so much simpler if it would be possible to have some sort of ".pxarinclude", just listing files/folders which should be included in the backup, ignoring everything else.

Thx in advance!
T0mc@

 

[dcsapak]

from my quick test you need to include the dirs and their content too (this config seemed to work here for your example):

*
!backupme
!backupme/**
!backupmetoo
!backupmetoo/**

 

[fiona]

Hi,
it should also work if you use leading slashes. With a leading slash, the pattern will match only at the top level. Without, the match will be recursive i.e. each pattern is applied in each sub-directory again.

EDIT: As @dcsapak pointed out to me, there is currently a bug when using negated patterns with a leading slash, but the following should work too:

/*
!backupme
!backupmetoo

 

[t0mc@]

Hi,
it should also work if you use leading slashes. With a leading slash, the pattern will match only at the top level. Without, the match will be recursive i.e. each pattern is applied in each sub-directory again.

EDIT: As @dcsapak pointed out to me, there is currently a bug when using negated patterns with a leading slash, but the following should work too:

/*
!backupme
!backupmetoo

@Fabian_E : Thx a lot.. indeed this seems to work at a first glance... with this content in .pxarexclude this is the resulting backup and it looks like what should be achieved:

 

[t0mc@]

Hi,
it should also work if you use leading slashes. With a leading slash, the pattern will match only at the top level. Without, the match will be recursive i.e. each pattern is applied in each sub-directory again.

EDIT: As @dcsapak pointed out to me, there is currently a bug when using negated patterns with a leading slash, but the following should work too:

/*
!backupme
!backupmetoo

@Fabian_E :
Next challenge: when using your solution, including lines containing slashes doesn't seem to work at all, not just leading ones. Look at this example:

/opt/backuptest# tree
.
├── backupme
│   └── file0
├── backupmetoo
│   ├── file0
│   ├── subfolder0
│   │   └── file0
│   └── subfolder1
├── backuppartly
│   ├── backupme
│   │   └── file0
│   ├── file0
│   └── notme
├── notme1
│   └── file0
├── notme2
│   ├── file0
│   └── subfolder0

Notice the "backuppartly" folder... in it only "backupme" and its contents should be backed up, so the backup should look like this:

.
├── backupme
│   └── file0
├── backupmetoo
│   ├── file0
│   ├── subfolder0
│   │   └── file0
│   └── subfolder1
├── backuppartly
│   ├── backupme
│   │   └── file0

In .pxarexclude I entered:

/*
!backupme
!backupmetoo
!backuppartly/backupme

But with this "backuppartly/backupme" is completely ignored:


Any idea?

 

[fiona]

@Fabian_E :
Next challenge: when using your solution, including lines containing slashes doesn't seem to work at all, not just leading ones. Look at this example:

/opt/backuptest# tree
.
├── backupme
│   └── file0
├── backupmetoo
│   ├── file0
│   ├── subfolder0
│   │   └── file0
│   └── subfolder1
├── backuppartly
│   ├── backupme
│   │   └── file0
│   ├── file0
│   └── notme
├── notme1
│   └── file0
├── notme2
│   ├── file0
│   └── subfolder0

Notice the "backuppartly" folder... in it only "backupme" and its contents should be backed up, so the backup should look like this:

.
├── backupme
│   └── file0
├── backupmetoo
│   ├── file0
│   ├── subfolder0
│   │   └── file0
│   └── subfolder1
├── backuppartly
│   ├── backupme
│   │   └── file0

In .pxarexclude I entered:

/*
!backupme
!backupmetoo
!backuppartly/backupme

But with this "backuppartly/backupme" is completely ignored:

Yes, because backuppartly itself is already excluded by /* and so the inclusion for backuppartly/backupme never matches. You need to include backuppartly itself too.

EDIT: If you only want to include part of backuppartly, I'd suggest creating another .pxarexclude file with the desired rules within that directory, or alternatively have another rule to exclude backuppartly/* at the top-level.

View attachment 35504

Any idea?

 

[t0mc@]

Yes, because backuppartly itself is already excluded by /* and so the inclusion for backuppartly/backupme never matches. You need to include backuppartly itself too.

Indeed it now works with this content:

/*
/backuppartly/*
!backupme
!backupmetoo
!backuppartly
!backuppartly/backupme

As you said correctly one has first to exclude everything under backuppartly (/backuppartly/*) and then include what is needed in two lines: (!backuppartly, !backuppartly/backupme).
This is the result:

EDIT: If you only want to include part of backuppartly, I'd suggest creating another .pxarexclude file with the desired rules within that directory, or alternatively have another rule to exclude backuppartly/* at the top-level.

I'm not really a friend of spreading such things out to many files and - even worse - to files in (many) different (sub-)dirs.. that would be very confusing. Having just one file at "root" level is more clear as you see at a first glance what will be backed up.

Nevertherless.... I'm still wondering, if it would be worth thinking about some sort of ".pxar_IN_clude" file, where you just write down a list of files/folders/globs, which should be INcluded in the backup and everything else is automatically excluded. Could imagine there are plenty of situations where one has a filesystem and just want to backup a handfull of (sub-)dirs/files, where in this case a small INclude file is more readable than a larger EXclude file which will mainly be double-negated at the end of the day ("an exclude file, mainly containing entries not to be excluded").

Have to mention that in the past I was working a lot with "bacula", where this is possible since many many years like this:

FileSet {
  Name = "MyBackupFiles"
  Include {
    File = /opt/backuptest/backupme
    File = /opt/backuptest/backupmetoo
    [...]
  }
  Exclude {
    File = /opt/backuptest/notme
    [...]
  }
}