Error inclusion pattern: Error reading 'zstd'

[suhaibmujahid]

2023-04-11 16:31:44,329:INFO:bugbug.utils:Downloading https://community-tc.services.mozilla.com/api/index/v1/task/project.bugbug.train_testlabelselect.v0.0.499/artifacts/public/testlabelselectmodel.zst...
2023-04-11 16:31:44,841:INFO:bugbug.db:Downloading https://community-tc.services.mozilla.com/api/index/v1/task/project.bugbug.data_test_label_scheduling_history.latest/artifacts/public/past_failures_label.lmdb.tar.zst to data/past_failures_label.lmdb.tar.zst
tar: Error inclusion pattern: Error reading 'zstd'
Traceback (most recent call last):
  File "/Users/smujahid/repos/mozilla/bugbug/scripts/commit_classifier.py", line 865, in <module>
    main()
  File "/Users/smujahid/repos/mozilla/bugbug/scripts/commit_classifier.py", line 851, in main
    classifier = CommitClassifier(
  File "/Users/smujahid/repos/mozilla/bugbug/scripts/commit_classifier.py", line 198, in __init__
    assert db.download_support_file(
  File "/Users/smujahid/repos/mozilla/bugbug/bugbug/db.py", line 74, in download_support_file
    utils.extract_file(path)
  File "/Users/smujahid/repos/mozilla/bugbug/bugbug/utils.py", line 289, in extract_file
    extract_tar_zst(path)
  File "/Users/smujahid/repos/mozilla/bugbug/bugbug/utils.py", line 282, in extract_tar_zst
    subprocess.run(["tar", "-I", "zstd", "-xf", path], check=True)
  File "/opt/homebrew/Cellar/[email protected]/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['tar', '-I', 'zstd', '-xf', 'data/past_failures_label.lmdb.tar.zst']' returned non-zero exit status 1.

Steps to reproduce:

• Remove the data directory
• Run scripts/commit_classifier.py

The tar command works when dropping the -I zstd part.

[jpangas]

Hey @suhaibmujahid , for some reason. I cannot reproduce the error. On my side, the data folder is downloaded without any errors coming up.

[suhaibmujahid]

I can reproduce it on macOS, example:

$ tar --version
bsdtar 3.5.3 - libarchive 3.5.3 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.8

$ tar -I zstd -xf ./data/commit_experiences.lmdb.tar.zst
tar: Error inclusion pattern: Error reading 'zstd'

There is no error if I strip -I zstd, example:

$ tar -xf ./data/commit_experiences.lmdb.tar.zst

Also, there is no error if I use the GNU version of tar, example:

$ brew install gnu-tar
...

$ gtar --version
tar (GNU tar) 1.34
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ gtar -I zstd -xf ./data/commit_experiences.lmdb.tar.zst

[j-enny]

@suhaibmujahid I am unable to move past this error. This is the error I am getting currently. Kindly assist

nafeesah@DESKTOP-1N5TMF8:~/bugbug$ tar -xf ./data/commit_experiences.lmdb.tar.zst tar: ./data/commit_experiences.lmdb.tar.zst: Cannot open: No such file or directory tar: Error is not recoverable: exiting now

[suhaibmujahid]

I am unable to move past this error. This is the error I am getting currently. Kindly assist

It seems that you do not have the ./data/commit_experiences.lmdb.tar.zst file installed. It will be installed when you run the scripts/commit_classifier.py script.

[suhaibmujahid]

Resolving #3755 will help to mitigate the problem, but will not solve it.