snapcraft icon indicating copy to clipboard operation
snapcraft copied to clipboard
verified publisher icon canonical

File filtering handled incorrectly

Open dilyn-corner opened this issue 1 year ago • 4 comments

Bug Description

When attempting to filter out files using the stage and prime keywords, the prime step will fail due to a missing file which was filtered out during the stage step.

Specifically, when filtering a file out in the stage step and then explicitly priming the parent directory of that file, the build fails due to a missing file (the one which was removed in the stage step).

When filtering the file out during the prime step, the error is averted.

To Reproduce

Project directory tree:

dilyn@Ares:~/test -> tree
.
├── snap
│   └── snapcraft.yaml
└── src
    ├── foo
    │   └── bar
    │       ├── baz
    │       ├── buzz
    │       └── fizz
    └── test
        └── ttest

6 directories, 5 files

Environment

Ubuntu 23.10, building with LXD

snapcraft.yaml

name: test-core2{2,4} # you probably want to 'snapcraft register <name>'
base: core2{2,4} # the base snap is the execution environment for this snap
version: '0.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Single-line elevator pitch for your amazing snap # 79 char long summary
description: |
  This is my-snap's description. You have a paragraph or two to tell the
  most important story about your snap. Keep it under 100 words though,
  we live in tweetspace and your description wants to look good in the snap
  store.

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: devmode # use 'strict' once you have the right plugs and slots

parts:
  my-part:
    # See 'snapcraft plugins'
    plugin: dump
    source: src/
    stage:
      - -foo/bar/baz # To see success, move this line to the prime step
    prime:
      - foo/bar

Relevant log output

Just posting one log :)

2024-06-03 10:33:14.389 Starting snapcraft, version 8.2.9.post42+gite6a0f5c3
2024-06-03 10:33:14.389 Log verbosity level set to VERBOSE
2024-06-03 10:33:14.391 Falling back from craft-application to snapcraft.
2024-06-03 10:33:14.392 lifecycle command: 'pack', arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=True, manifest_image_information=None, bind_ssh=False, build_for=None, http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, directory=None, output=None)
2024-06-03 10:33:14.392 command: pack, arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=True, manifest_image_information=None, bind_ssh=False, build_for=None, http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, directory=None, output=None)
2024-06-03 10:33:14.400 CPU count (from process affinity): 24
2024-06-03 10:33:14.400 Invalid SNAPCRAFT_MAX_PARALLEL_BUILD_COUNT ''
2024-06-03 10:33:14.400 Running on amd64 for amd64
2024-06-03 10:33:14.402 Checking build provider availability
2024-06-03 10:33:14.406 Retrieved snap config: {}
2024-06-03 10:33:14.406 Using default provider 'lxd' on linux system.
2024-06-03 10:33:14.531 Executing on host: lxc --project default profile show local:default
2024-06-03 10:33:14.624 '--enable-manifest' is deprecated, and will be removed in core24.
2024-06-03 10:33:14.624 Using hostname 'snapcraft-test-core22-on-amd64-for-amd64-2730288'
2024-06-03 10:33:14.624 Launching instance...
2024-06-03 10:33:14.624 Executing on host: lxc remote list --format=yaml
2024-06-03 10:33:14.724 Remote 'craft-com.ubuntu.cloud-buildd' already exists.
2024-06-03 10:33:14.724 Executing on host: lxc project list local: --format=yaml
2024-06-03 10:33:14.818 Set LXD instance name to 'snapcraft-test-core22-on-amd64-for-amd64-2730288'
2024-06-03 10:33:14.818 Checking for instance 'snapcraft-test-core22-on-amd64-for-amd64-2730288' in project 'snapcraft' in remote 'local'
2024-06-03 10:33:14.818 Executing on host: lxc --project snapcraft list local: --format=yaml
2024-06-03 10:33:14.990 Instance 'snapcraft-test-core22-on-amd64-for-amd64-2730288' does not exist.
2024-06-03 10:33:14.991 Set LXD instance name to 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861'
2024-06-03 10:33:14.991 Checking for base instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861' in project 'snapcraft' in remote 'local'
2024-06-03 10:33:14.991 Executing on host: lxc --project snapcraft list local: --format=yaml
2024-06-03 10:33:15.125 Checking validity of instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861'.
2024-06-03 10:33:15.125 Executing on host: lxc --project snapcraft info local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861
2024-06-03 10:33:15.228 Executing on host: lxc --project snapcraft info local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861
2024-06-03 10:33:15.325 Executing on host: lxc --project snapcraft config get local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.status
2024-06-03 10:33:15.425 Instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861' is ready.
2024-06-03 10:33:15.425 Instance is valid.
2024-06-03 10:33:15.425 Creating instance from base instance
2024-06-03 10:33:15.425 Creating instance from base instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861'.
2024-06-03 10:33:15.425 Executing on host: lxc --project snapcraft list local: --format=yaml
2024-06-03 10:33:15.543 Executing on host: lxc --project snapcraft copy local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 local:snapcraft-test-core22-on-amd64-for-amd64-2730288
2024-06-03 10:33:15.856 Executing on host: lxc --project snapcraft list local: --format=yaml
2024-06-03 10:33:15.986 Executing on host: lxc --project snapcraft config set local:snapcraft-test-core22-on-amd64-for-amd64-2730288 raw.idmap 'both 1000 0'
2024-06-03 10:33:16.145 Executing on host: lxc --project snapcraft config get local:snapcraft-test-core22-on-amd64-for-amd64-2730288 raw.idmap
2024-06-03 10:33:16.231 Got LXD idmap for instance 'snapcraft-test-core22-on-amd64-for-amd64-2730288': 'both 1000 0'
2024-06-03 10:33:16.231 Starting instance
2024-06-03 10:33:16.231 Executing on host: lxc --project snapcraft start local:snapcraft-test-core22-on-amd64-for-amd64-2730288
2024-06-03 10:33:16.795 Executing on host: lxc --project snapcraft file push /tmp/tmpbv5t6pke local:snapcraft-test-core22-on-amd64-for-amd64-2730288/etc/hostname --mode=0644
2024-06-03 10:33:16.911 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/hostname
2024-06-03 10:33:16.983 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical hostname -F /etc/hostname
2024-06-03 10:33:17.056 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-06-03 10:33:17.170 Executing on host: lxc --project snapcraft file pull local:snapcraft-test-core22-on-amd64-for-amd64-2730288/etc/craft-instance.conf /home/dilyn/tmp4obrp6f4.tmp-craft/tmp80ujgqg2
2024-06-03 10:33:17.259 Instance has already been setup.
2024-06-03 10:33:17.260 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical cat /etc/os-release
2024-06-03 10:33:17.400 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-06-03 10:33:17.614 Executing on host: lxc --project snapcraft file pull local:snapcraft-test-core22-on-amd64-for-amd64-2730288/etc/craft-instance.conf /home/dilyn/tmpd6ssydpb.tmp-craft/tmpz2wpbdm3
2024-06-03 10:33:17.736 Instance is compatible with compatibility tag 'snapcraft-buildd-base-v7.1'
2024-06-03 10:33:17.736 No cache path set, not mounting cache directories.
2024-06-03 10:33:17.736 Waiting for environment to be ready...
2024-06-03 10:33:17.736 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-06-03 10:33:17.877 systemctl is-system-running status: starting
2024-06-03 10:33:18.128 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-06-03 10:33:18.277 Waiting for networking to be ready...
2024-06-03 10:33:18.277 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical getent hosts snapcraft.io
2024-06-03 10:33:18.398 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical snap unset system proxy.http
2024-06-03 10:33:18.678 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical snap unset system proxy.https
2024-06-03 10:33:18.924 Installing snap 'snapcraft' with channel=None and classic=True
2024-06-03 10:33:18.924 Installing snap 'snapcraft' from host (classic=True)
2024-06-03 10:33:18.929 Installing base snap 'core22' for 'snapcraft' from host
2024-06-03 10:33:18.929 Installing snap 'core22' from host (classic=False)
2024-06-03 10:33:18.931 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-06-03 10:33:19.095 Executing on host: lxc --project snapcraft file pull local:snapcraft-test-core22-on-amd64-for-amd64-2730288/etc/craft-instance.conf /home/dilyn/tmpn3_c6_tj.tmp-craft/tmpjdgbudfd
2024-06-03 10:33:19.150 Revisions found: host='1380', target='1380'
2024-06-03 10:33:19.151 Skipping snap injection: target is already up-to-date with revision on host
2024-06-03 10:33:19.151 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-06-03 10:33:19.265 Executing on host: lxc --project snapcraft file pull local:snapcraft-test-core22-on-amd64-for-amd64-2730288/etc/craft-instance.conf /home/dilyn/tmpehhyl5wm.tmp-craft/tmpjphx3ds6
2024-06-03 10:33:19.352 Revisions found: host='11843', target='11810'
2024-06-03 10:33:19.355 Creating an assert file for snap 'snapcraft'
2024-06-03 10:33:19.355 Executing command on host: ['snap', 'known', 'account-key', 'public-key-sha3-384=BWDEoaqyr25nF5SNCvEv2v7QnM9QsfCc0PBMYD_i2NGSQ32EF2d4D0hqUel3m8ul']
2024-06-03 10:33:19.375 Executing command on host: ['snap', 'known', 'snap-declaration', 'snap-name=snapcraft']
2024-06-03 10:33:19.399 Executing command on host: ['snap', 'known', 'snap-revision', 'snap-revision=11843', 'snap-id=vMTKRaLjnOJQetI78HjntT37VuoyssFE']
2024-06-03 10:33:19.462 Executing command on host: ['snap', 'known', 'account', 'account-id=canonical']
2024-06-03 10:33:19.484 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -d /tmp
2024-06-03 10:33:19.648 Executing on host: lxc --project snapcraft file push /home/dilyn/tmpysjs799j.tmp-craft/tmp0pkzovqc local:snapcraft-test-core22-on-amd64-for-amd64-2730288/tmp/snapcraft.assert --gid=0 --uid=0
2024-06-03 10:33:19.746 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical snap ack /tmp/snapcraft.assert
2024-06-03 10:33:20.034 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -d /tmp
2024-06-03 10:33:20.199 Executing on host: lxc --project snapcraft file push /home/dilyn/tmp1v1smg2a.tmp-craft/snapcraft.snap local:snapcraft-test-core22-on-amd64-for-amd64-2730288/tmp/snapcraft.snap --gid=0 --uid=0
2024-06-03 10:33:20.531 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical snap install /tmp/snapcraft.snap --classic
2024-06-03 10:33:35.316 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-06-03 10:33:35.480 Executing on host: lxc --project snapcraft file pull local:snapcraft-test-core22-on-amd64-for-amd64-2730288/etc/craft-instance.conf /home/dilyn/tmpx_oxghdz.tmp-craft/tmpolg6m5w5
2024-06-03 10:33:35.607 Executing on host: lxc --project snapcraft file push /tmp/tmp7h9ahlsu local:snapcraft-test-core22-on-amd64-for-amd64-2730288/etc/craft-instance.conf --mode=0644
2024-06-03 10:33:35.719 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/craft-instance.conf
2024-06-03 10:33:36.058 Setting instance timezone to match host timezone 'America/New_York'.
2024-06-03 10:33:36.059 Executing on host: lxc --project snapcraft config set local:snapcraft-test-core22-on-amd64-for-amd64-2730288 environment.TZ America/New_York
2024-06-03 10:33:36.206 Executing on host: lxc --project snapcraft config device show local:snapcraft-test-core22-on-amd64-for-amd64-2730288
2024-06-03 10:33:36.311 Executing on host: lxc --project snapcraft config device add local:snapcraft-test-core22-on-amd64-for-amd64-2730288 disk-/root/project disk source=/home/dilyn/test path=/root/project
2024-06-03 10:33:36.453 Executing on host: lxc --project snapcraft file push /tmp/tmp97oj2yd3 local:snapcraft-test-core22-on-amd64-for-amd64-2730288/root/.bashrc --mode=644
2024-06-03 10:33:36.544 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /root/.bashrc
2024-06-03 10:33:36.745 Emitter: Pausing control of the terminal
2024-06-03 10:33:40.755 Emitter: Resuming control of the terminal
2024-06-03 10:33:40.756 Executing in container: lxc --project snapcraft exec local:snapcraft-test-core22-on-amd64-for-amd64-2730288 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 SNAPCRAFT_BUILD_INFO=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /tmp/snapcraft.log
2024-06-03 10:33:40.970 Executing on host: lxc --project snapcraft file pull local:snapcraft-test-core22-on-amd64-for-amd64-2730288/tmp/snapcraft.log /home/dilyn/tmp7e_ih_ve.tmp-craft/tmp8ov3vf8u
2024-06-03 10:33:41.059 Logs retrieved from managed instance:
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.558 Starting snapcraft, version 8.2.9.post42+gite6a0f5c3
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.558 Log verbosity level set to BRIEF
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.560 Falling back from craft-application to snapcraft.
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.562 lifecycle command: 'pack', arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=True, manifest_image_information=None, bind_ssh=False, build_for='amd64', http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, directory=None, output=None)
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.562 command: pack, arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=True, manifest_image_information=None, bind_ssh=False, build_for='amd64', http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, directory=None, output=None)
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.566 CPU count (from process affinity): 24
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.566 Invalid SNAPCRAFT_MAX_PARALLEL_BUILD_COUNT ''
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.566 Running on amd64 for amd64
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.568 Initializing parts lifecycle
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.568 is_snap: True, SNAP_NAME set to snapcraft
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.569 process my-part:Step.PULL
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.569 add action my-part:Step.PULL(ActionType.RUN)
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.569 process my-part:Step.BUILD
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.569 add action my-part:Step.BUILD(ActionType.RUN)
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.569 process my-part:Step.STAGE
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.569 add action my-part:Step.STAGE(ActionType.RUN)
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.570 process my-part:Step.PRIME
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.570 add action my-part:Step.PRIME(ActionType.RUN)
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.570 ignore patterns: ['*.snap']
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.570 Installing build-packages
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:38.570 Installing build-snaps
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.037 verify plugin environment for part 'my-part'
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.037 Pulling my-part
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.038 execute action my-part:Action(part_name='my-part', step=Step.PULL, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.060 Building my-part
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.060 execute action my-part:Action(part_name='my-part', step=Step.BUILD, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.061 load state file: /root/parts/my-part/state/pull
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.067 remove directory /root/parts/my-part/build
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.070 Executing PosixPath('/root/parts/my-part/run/build.sh')
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:39.071 :: + cp --archive --link --no-dereference . /root/parts/my-part/install
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.586 Staging my-part
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.586 execute action my-part:Action(part_name='my-part', step=Step.STAGE, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.605 Priming my-part
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.606 execute action my-part:Action(part_name='my-part', step=Step.PRIME, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.608 Failed to copy '/root/stage/foo/bar/baz': no such file or directory.
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.615 Traceback (most recent call last):
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/snapcraft/application.py", line 385, in main
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.616     return app.run()
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_application/application.py", line 488, in run
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.616     dispatcher = self._get_dispatcher()
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/snapcraft/application.py", line 244, in _get_dispatcher
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.616     raise errors.ClassicFallback()
2024-06-03 10:33:41.059 :: 2024-06-03 10:33:40.616 snapcraft.errors.ClassicFallback
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616 During handling of the above exception, another exception occurred:
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616 Traceback (most recent call last):
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/utils/file_utils.py", line 141, in link
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     os.link(source_path, destination, follow_symlinks=False)
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616 FileNotFoundError: [Errno 2] No such file or directory: '/root/stage/foo/bar/baz' -> '/root/prime/foo/bar/baz'
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616 The above exception was the direct cause of the following exception:
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616 Traceback (most recent call last):
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/snapcraft/parts/parts.py", line 222, in run
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     aex.execute(action, stdout=stream, stderr=stream)
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 327, in execute
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     self._executor.execute(actions, stdout=stdout, stderr=stderr)
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 137, in execute
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     self._run_action(act, stdout=stdout, stderr=stderr)
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 214, in _run_action
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     handler.run_action(action, stdout=stdout, stderr=stderr)
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 170, in run_action
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     state = handler(step_info, stdout=stdout, stderr=stderr)
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 407, in _run_prime
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     contents = self._run_step(
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 491, in _run_step
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     return step_handler.run_builtin()
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/step_handler.py", line 108, in run_builtin
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     return handler()
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/step_handler.py", line 245, in _builtin_prime
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     files, dirs = migrate_files(
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/executor/migration.py", line 123, in migrate_files
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     file_utils.link_or_copy(
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/utils/file_utils.py", line 98, in link_or_copy
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     link(source, destination, follow_symlinks=follow_symlinks)
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_parts/utils/file_utils.py", line 143, in link
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616     raise errors.CopyFileNotFound(source) from err
2024-06-03 10:33:41.060 :: 2024-06-03 10:33:40.616 craft_parts.errors.CopyFileNotFound: Failed to copy '/root/stage/foo/bar/baz': no such file or directory.
2024-06-03 10:33:41.060 Executing on host: lxc --project snapcraft config device show local:snapcraft-test-core22-on-amd64-for-amd64-2730288
2024-06-03 10:33:41.155 Executing on host: lxc --project snapcraft config device remove local:snapcraft-test-core22-on-amd64-for-amd64-2730288 disk-/root/project
2024-06-03 10:33:41.458 Executing on host: lxc --project snapcraft stop local:snapcraft-test-core22-on-amd64-for-amd64-2730288
2024-06-03 10:33:45.701 Failed to execute pack in instance.
2024-06-03 10:33:45.710 Traceback (most recent call last):
2024-06-03 10:33:45.710   File "/snap/snapcraft/11843/lib/python3.10/site-packages/snapcraft/application.py", line 385, in main
2024-06-03 10:33:45.710     return app.run()
2024-06-03 10:33:45.710   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_application/application.py", line 488, in run
2024-06-03 10:33:45.710     dispatcher = self._get_dispatcher()
2024-06-03 10:33:45.710   File "/snap/snapcraft/11843/lib/python3.10/site-packages/snapcraft/application.py", line 244, in _get_dispatcher
2024-06-03 10:33:45.710     raise errors.ClassicFallback()
2024-06-03 10:33:45.710 snapcraft.errors.ClassicFallback
2024-06-03 10:33:45.710 
2024-06-03 10:33:45.710 During handling of the above exception, another exception occurred:
2024-06-03 10:33:45.710 Traceback (most recent call last):
2024-06-03 10:33:45.711   File "/snap/snapcraft/11843/lib/python3.10/site-packages/snapcraft/parts/lifecycle.py", line 545, in _run_in_provider
2024-06-03 10:33:45.711     instance.execute_run(cmd, check=True, cwd=output_dir)
2024-06-03 10:33:45.711   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_providers/lxd/lxd_instance.py", line 293, in execute_run
2024-06-03 10:33:45.711     return self.lxc.exec(
2024-06-03 10:33:45.711   File "/snap/snapcraft/11843/lib/python3.10/site-packages/craft_providers/lxd/lxc.py", line 387, in exec
2024-06-03 10:33:45.711     return runner(final_cmd, timeout=timeout, check=check, **kwargs)
2024-06-03 10:33:45.711   File "/snap/snapcraft/11843/usr/lib/python3.10/subprocess.py", line 526, in run
2024-06-03 10:33:45.711     raise CalledProcessError(retcode, process.args,
2024-06-03 10:33:45.711 subprocess.CalledProcessError: Command '['lxc', '--project', 'snapcraft', 'exec', 'local:snapcraft-test-core22-on-amd64-for-amd64-2730288', '--cwd', '/root/project', '--', 'env', 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin', 'CRAFT_MANAGED_MODE=1', 'SNAPCRAFT_BUILD_INFO=1', 'DEBIAN_FRONTEND=noninteractive', 'DEBCONF_NONINTERACTIVE_SEEN=true', 'DEBIAN_PRIORITY=critical', 'snapcraft', 'pack', '--verbosity=verbose', '--enable-manifest', '--build-for', 'amd64']' returned non-zero exit status 1.
2024-06-03 10:33:45.711 Recommended resolution: Run the same command again with --debug to shell into the environment if you wish to introspect this failure.
2024-06-03 10:33:45.711 Full execution log: '/home/dilyn/.local/state/snapcraft/log/snapcraft-20240603-103314.389189.log'

Additional context

Seems like snapcraft is using the same file list for both steps, but this feels like incorrect behavior - for instance, it's theoretically possible that the existence of some file interferes with some other step I perform during a snap build. While what I'm attempting to do isn't the most "efficient", filtering in this way provides more explanatory power.

dilyn-corner avatar Jun 03 '24 14:06 dilyn-corner

I'm able to reproduce this directly with craft-parts. I proposed a solution in https://github.com/canonical/craft-parts/issues/766 but I'm not sure what side-effects it will have.

mr-cal avatar Jul 02 '24 14:07 mr-cal

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-3085.

This message was autogenerated

syncronize-issues-to-jira[bot] avatar Jul 02 '24 14:07 syncronize-issues-to-jira[bot]

Any update on this? It's affecting snap builds we are about to share with our partner and if we implement workaround we will need to plan for cleanup

kubiko avatar May 21 '25 18:05 kubiko

Making this fix has some concerning side effects with overlays in craft-parts. We have some upstream documentation and design work to do before this can be completed (https://github.com/canonical/craft-parts/issues/766).

mr-cal avatar Jun 23 '25 13:06 mr-cal