npbackup icon indicating copy to clipboard operation
npbackup copied to clipboard

NPBackup sets wrong (and incorrect) path

Open Gaibhne opened this issue 5 months ago • 9 comments

I set up my first repo to back up C:\ through Folder / file list, which NPBackup shows as C:/. Backing up results in a snapshot with metadata like this:

  {
    "time": "2025-07-24T12:28:34.7196972+02:00",
    "parent": "f0eedeb27d2fed97fd141ec16f1d7fecd2c7e9443a7f84e828d79c3f10692853",
    "tree": "8c6ea5924ff5b71f938fb7222bace1cb434d7206891595abae9a8a4afee174d1",
    "paths": [
      "C:\\Users\\me\\Downloads\\npbackup-windows-x64-gui-public\\npbackup-gui"
    ],
    "hostname": "myhost",
    "username": "me",
    "tags": [
      "C:\\"
    ],
    "program_version": "restic 0.18.0",
    "summary": {
      "backup_start": "2025-07-24T12:28:34.7196972+02:00",
      "backup_end": "2025-07-24T12:32:31.0690101+02:00",
      "files_new": 92,
      "files_changed": 192,
      "files_unmodified": 272332,
      "dirs_new": 49,
      "dirs_changed": 138,
      "dirs_unmodified": 38984,
      "data_blobs": 1100,
      "tree_blobs": 165,
      "data_added": 557524614,
      "data_added_packed": 104849415,
      "total_files_processed": 272616,
      "total_bytes_processed": 125928401789
    },
    "id": "8b996a2b678edae8cdfa00602bbdc01125e3856b25f17995fff78ee108bcd507",
    "short_id": "8b996a2b"
  }

The snapshot does in fact contain my C:\ as it should, but the path incorrectly claims it contains only the folder from which I started NPBackup.

Gaibhne avatar Jul 24 '25 10:07 Gaibhne

Could you share your anonymized configuration file ?

deajan avatar Jul 24 '25 10:07 deajan

Sure (I am backing up to a local repo on my other HDD anyways):

conf_version: 3.0.3
audience: public
repos:
  default:
    repo_uri: 
      __NPBACKUP__ble__NPBACKUP__
    repo_group: default_group
    backup_opts:
      exclude_files: []
      tags:
      - C:\
      additional_backup_only_parameters: ''
    repo_opts:
      repo_password: 
        __NPBACKUP__ble__NPBACKUP__
      random_delay_before_backup: 20
      retention_policy: {}
    prometheus: {}
    env: {}
    is_protected: false
groups:
  default_group:
    backup_opts:
      paths: []
      source_type: folder_list
      stdin_from_command:
      stdin_filename:
      tags: []
      compression: max
      use_fs_snapshot: true
      ignore_cloud_files: true
      one_file_system: false
      priority: low
      exclude_caches: true
      excludes_case_ignore: true
      exclude_files:
      - excludes/generic_excluded_extensions
      - excludes/generic_excludes
      - excludes/windows_excludes
      - excludes/linux_excludes
      - C:/restic/local.exclude
      - C:/restic/windows.exclude
      exclude_patterns: []
      exclude_files_larger_than: 0.0
      additional_parameters:
      additional_backup_only_parameters:
      additional_restore_only_parameters:
      minimum_backup_size_error: 10.0 MiB
      pre_exec_commands: []
      pre_exec_per_command_timeout: 3600
      pre_exec_failure_is_fatal: false
      post_exec_commands: []
      post_exec_per_command_timeout: 3600
      post_exec_failure_is_fatal: false
      post_exec_execute_even_on_backup_error: true
      post_backup_housekeeping_percent_chance: 0
      post_backup_housekeeping_interval: 1
    repo_opts:
      repo_password: 
        __NPBACKUP__ble__NPBACKUP__
      repo_password_command:
      minimum_backup_age: 300
      random_delay_before_backup: 15
      upload_speed: 0.0
      download_speed: 0.0
      backend_connections: 4
      retention_policy:
        last: 10
        hourly: 72
        daily: 60
        weekly: 26
        monthly: 24
        yearly: 99
        keep_tags: []
        apply_on_tags: []
        keep_within: true
        group_by_host: true
        group_by_tags: true
        group_by_paths: false
        ntp_server:
      prune_max_unused: 0
      prune_max_repack_size: 10 GB
    prometheus:
      backup_job: ${MACHINE_ID}
      group: ${MACHINE_GROUP}
    env:
      env_variables: {}
      encrypted_env_variables: {}
    repo_uri: 
      __NPBACKUP__ble__NPBACKUP__
    repo_group: default_group
    is_protected: false
identity:
  machine_id: ${HOSTNAME}
  machine_group: ourgroup
global_prometheus:
  metrics: true
  instance: ${MACHINE_ID}
  destination: http://ourprometheus
  http_username:
  http_password:
  additional_labels: {}
  no_cert_verify: true
global_email:
  enable: false
  instance: ${MACHINE_ID}
  smtp_server:
  smtp_port: 587
  smtp_username:
  smtp_password:
  sender:
  recipients:
  on_backup_success: true
  on_backup_failure: true
  on_operations_success: false
  on_operations_failure: true
global_options:
  auto_upgrade: false
  auto_upgrade_percent_chance: 5
  auto_upgrade_interval: 15
  auto_upgrade_server_url:
  auto_upgrade_server_username:
  auto_upgrade_server_password:
  auto_upgrade_host_identity: ${MACHINE_ID}
  auto_upgrade_group: ${MACHINE_GROUP}
  full_concurrency: false
  repo_aware_concurrency: false

Gaibhne avatar Jul 24 '25 12:07 Gaibhne

It looks like you've setup your backup local in tag instead of paths variable. Your config should look like this:

conf_version: 3.0.3
audience: public
repos:
  default:
    repo_uri: 
      __NPBACKUP__ble__NPBACKUP__
    repo_group: default_group
    backup_opts:
      paths:
      - C:\
      tags:
      - your favorite tag
[...]

deajan avatar Jul 24 '25 18:07 deajan

@Gaibhne Any news on this ?

deajan avatar Jul 29 '25 09:07 deajan

Sorry, I'm away on vacation without access to the PC in question, will updaten when I return in about two weeks. I did set the tag, which I do on all my snapshots for organization, but I also used the 'Add Folder' to add C:\

Sent with Shortwave https://www.shortwave.com?utm_medium=email&utm_content=signature&utm_source=am9lcmcuc3RvZXZlckBnbWFpbC5jb20=

On Tue Jul 29, 2025, 09:25 AM GMT, Orsiris de Jong @.***> wrote: [image]deajan left a comment (netinvent/npbackup#167) https://github.com/netinvent/npbackup/issues/167#issuecomment-3131504367

@Gaibhne https://github.com/Gaibhne Any news on this ? — Reply to this email directly, view it on GitHub https://github.com/netinvent/npbackup/issues/167#issuecomment-3131504367, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF4WAQGA6NXE2JYVHVCJXGL3K44XPAVCNFSM6AAAAACCIN7ANOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCMZRGUYDIMZWG4. You are receiving this because you were mentioned.Message ID: @.***>

Gaibhne avatar Jul 29 '25 09:07 Gaibhne

Hmmm... I redid the test with NPBackup v3.0.3 just to be sure, and it indeed added C:\ to the path variable. Keep me updated once you get back. Have nice holidays.

deajan avatar Jul 29 '25 09:07 deajan

I used the UI to create the above config, but I am now uncertain whether I didn't during debugging removed the actual path - sorry if that sent anyone down the wrong path. I have opened the config and added C:\ through the Add folder button, which resulted in a config file looking like this:

repos:
  default:
    repo_uri: 
      __NPBACKUP__ble__NPBACKUP__
    repo_group: default_group
    backup_opts:
      exclude_files: []
      tags:
      - C:\
      additional_backup_only_parameters: ''
      paths:
      - C:/
    repo_opts:
      repo_password: 
        __NPBACKUP__ble__NPBACKUP__
      random_delay_before_backup: 20
      retention_policy: {}
    prometheus: {}
    env: {}
    is_protected: false

My config UI looks like this:

Image

As expected, this results in the following NPB GUI log messages:

Running backup of ['C:/'] to repo default
Using VSS snapshot to backup
using parent snapshot 8b996a2b
creating VSS snapshot for [c:\]
successfully created snapshot for [c:\]

and the following lines in the log file:

2025-08-11 10:03:21,208 :: INFO :: Running backup of ['C:/'] to repo default
2025-08-11 10:03:21,209 :: INFO :: Using VSS snapshot to backup

From the data in the produced snapshot, I can tell that C:\ is correctly backed up in the snapshot. However, the snapshots metadata looks like this:

{
"time": "2025-08-11T10:03:21.3125062+02:00",
"parent": "8b996a2b678edae8cdfa00602bbdc01125e3856b25f17995fff78ee108bcd507",
"tree": "ecc27c20a38c594d9bf6b678cccf0386ed959411a688914a2887590f45c2fd22",
"paths": [
  "C:\\Users\\me\\Downloads\\npbackup-windows-x64-gui-public\\npbackup-gui"
],
"hostname": "myhost",
"username": "me",
"tags": [
  "C:\\"
],
"program_version": "restic 0.18.0",
"summary": {
  "backup_start": "2025-08-11T10:03:21.3125062+02:00",
  "backup_end": "2025-08-11T10:07:25.5453721+02:00",
  "files_new": 650,
  "files_changed": 3499,
  "files_unmodified": 268422,
  "dirs_new": 37,
  "dirs_changed": 1406,
  "dirs_unmodified": 37734,
  "data_blobs": 2568,
  "tree_blobs": 1374,
  "data_added": 1420233164,
  "data_added_packed": 763748518,
  "total_files_processed": 272571,
  "total_bytes_processed": 126403552311
},
"id": "3e7a7d2e1ff80da5691a4e6cc43d56d9002542558a8d07a7b18c04d618bf59dc",
"short_id": "3e7a7d2e"
}

Because of this, path filtering / grouping does not work, neither in restic nor NPBackup. As my tag usage is unusual, I suspected that maybe that was somehow triggering a bug and removed the tag. That made no difference, same incorrect path entry in the resulting snapshot.

Gaibhne avatar Aug 11 '25 08:08 Gaibhne

I've just redone a test by backing up my whole drive and checking the snapshot metadata with restic cat snapshot <ID>. For me, everything looks fine, paths are what they're supposed to be.

Are you sure you checked the metadata for the last snapshot ? Could you perhaps remake a try ? Also, what npbackup version are you running ?

deajan avatar Aug 21 '25 12:08 deajan

Any news on this ?

deajan avatar Sep 09 '25 21:09 deajan