west init completes but leaves everything in zephyr/manifest-tmp

[jefffhaynes]

I have no idea what causes this or how to reliably reproduce it, but running west init on windows occasionally leaves everything in the zephyr/manifest-tmp folder upon "successful" completion. west update subsequently fails because everything is in the wrong place.

[marc-hb]

Can you remember any error message, even vaguely?

Going forward, can you use west -v -v init?

[jefffhaynes]

There is no error message until I run update as the init command doesn't fail, it just leaves everything in the wrong place. I'll try the new command.

[marc-hb]

It could fail without a visible error message. It's extremely rare with Python and even rarer with west but it's theoretically possible. Next time this happens please use echo %errorlevel% immediately

[jefffhaynes]

Running west -v -v init seems to have worked, although I don't know if that was the intention or if it just happened to work that time. If it happens again I'll try to get the error level. Thanks.

[mbolivar-nordic]

-v -v should just increase the verbosity level without otherwise changing behavior (other than precise timing details of when things happen, of course), but the verbose logs would be very useful for diagnosing this. I haven't seen this happen myself, but I believe you, and it'd be good to get more details from the verbose logs.

[congnguyenhuu]

Hi everyone, I have same problem when run west init on windows. All everything in the folder manifest-tmp moved folder zephyr, but The folder manifest-tmp is not removed. That make can not run west update. This is error display, Anyone have solution ? Thanks !

west.manifest: loaded C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
Traceback (most recent call last):
  File "c:\python38\lib\shutil.py", line 780, in move
    os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\congnh\\zephyrproject\\.west\\manifest-tmp' -> 'C:\\Users\\congnh\\zephyrproject\\zephyr'

[marc-hb]

Can you reproduce every time? If yes then please follow the instructions above.

[congnguyenhuu]

yes, it can reproduce. you mean use west -v -v init but I try and can not

[jefffhaynes]

I think he's asking you to post what it says when you do.

[congnguyenhuu]

This is all display when you did, it relates permission access folder when run os.rename

$ west -v -v init -m https://bitbucket.com/xx/xx/zephyr.git --mr xxxx.xx.xx.x --mf xxxx.yml
=== Initializing in C:\Users\congnh\zephyrproject
--- Cloning manifest repository from https://bitbucket.com/xx/xx/zephyr.git
running 'git clone --branch  https://bitbucket.com/'C:\Users\congnh\zephyrproject\.west\manifest-tmp'' in C:\Users\congnh\zephyrproject
Cloning into 'C:\Users\congnh\zephyrproject\.west\manifest-tmp'...
remote: Enumerating objects: 578417, done.
remote: Counting objects: 100% (578417/578417), done.
remote: Compressing objects: 100% (126962/126962), done.
remote: Total 578417 (delta 440454), reused 577231 (delta 439607)
Receiving objects: 100% (578417/578417), 383.54 MiB | 2.33 MiB/s, done.
Resolving deltas: 100% (440454/440454), done.
Note: switching to 'khsvt32h567586t8k'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Updating files: 100% (13543/13543), done.
west.manifest: loading C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
west.manifest: group-filter: unset
west.manifest: added project cmsis path modules/hal/cmsis revision from C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
west.manifest: added project hal_nxp path modules/hal/nxp revision from C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
west.manifest: loaded C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
Traceback (most recent call last):
  File "c:\python38\lib\shutil.py", line 780, in move
    os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\congnh\\zephyrproject\\.west\\manifest-tmp' -> 'C:\\Users\\congnh\\zephyrproject\\zephyr'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python38\lib\runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python38\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python38\Scripts\west.exe\__main__.py", line 9, in <module>
  File "c:\python38\lib\site-packages\west\app\main.py", line 781, in main
    app.run(argv or sys.argv[1:])
  File "c:\python38\lib\site-packages\west\app\main.py", line 106, in run
    self.run_command(argv)
  File "c:\python38\lib\site-packages\west\app\main.py", line 336, in run_command
    cmd.run(args, unknown, self.topdir, manifest=self.manifest)
  File "c:\python38\lib\site-packages\west\commands.py", line 140, in run
    self.do_run(args, unknown)
  File "c:\python38\lib\site-packages\west\app\project.py", line 198, in do_run
    topdir = self.bootstrap(args)
  File "c:\python38\lib\site-packages\west\app\project.py", line 305, in bootstrap
    shutil.move(os.fspath(tempdir), os.fspath(manifest_abspath))
  File "c:\python38\lib\shutil.py", line 792, in move
    rmtree(src)
  File "c:\python38\lib\shutil.py", line 730, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\python38\lib\shutil.py", line 603, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\python38\lib\shutil.py", line 603, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\python38\lib\shutil.py", line 603, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\python38\lib\shutil.py", line 608, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "c:\python38\lib\shutil.py", line 606, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\congnh\\zephyrproject\\.west\\manifest-tmp\\.git\\objects\\pack\\pack-2535647586gfhf56757.idx'

[marc-hb]

PermissionError: [WinError 5] Access is denied: 'C:\Users\congnh\zephyrproject\.west\manifest-tmp' -> 'C:\Users\congnh\zephyrproject\zephyr'

What happens if you try to rename this yourself from a command window?

west.manifest: loaded C:\Users\congnh8\Desktop\New folder.west\manifest-tmp\zephyrbsp.yml

Even stranger: is this imported from another manifest?

This is not the same problem as originally reported in the description BTW

[congnguyenhuu]

west.manifest: loaded C:\Users\congnh8\Desktop\New folder.west\manifest-tmp\zephyrbsp.yml sorry, I rename it , it must be this west.manifest: loaded C:\Users\congnh\zephyrproject\manifest-tmp\zephyrbsp.yml

[mbolivar-nordic]

sorry, I rename it , it must be this

I don't understand what you mean, sorry. It looks like you have edited the logs. This makes it hard to understand them since I cannot know for sure how the output comes from the input, or even if the output is accurate.

Maybe you are using a private repository. I understand if you cannot share that.

But do you maybe have some way to make a manifest you can share which reproduces the issue for you? That would be really helpful, thank you.

[congnguyenhuu]

yep. because some private information, i cannot post, so edited somethings in logs. but basically it is also same

[marc-hb]

If you cannot reproduce with the open-source Zephyr manifest (did you try?), it means something in your manifest(s) is triggering the problem. To identify what triggers the issue, remove the private elements one by one until the problem disappears. Now you found what triggers the issue. Finally, to reproduce the issue without any confidential information, add something similar (and not confidential) to the open source manifest. Now you have a bug report that you can fully share and something in common we can all work on.

[congnguyenhuu]

With community source Zephyr, I have same problem. I try with other machines, some can work but some cannt

C:\Users\datnd14>west init -m https://github.com/zephyrproject-rtos/zephyr.git --mr zephyr-v2.7.0 --mf west.yml
=== Initializing in C:\Users\datnd14
--- Cloning manifest repository from https://github.com/zephyrproject-rtos/zephyr.git, rev. zephyr-v2.7.0
Initialized empty Git repository in C:/Users/datnd14/.west/manifest-tmp/.git/
remote: Enumerating objects: 642618, done.
remote: Counting objects: 100% (2/2), done.
remote: Total 642618 (delta 1), reused 1 (delta 1), pack-reused 642616 eceiving objects: 100% (642618/642618), 4
Receiving objects: 100% (642618/642618), 433.69 MiB | 25.00 KiB/s, done.
Resolving deltas: 100% (488975/488975), done.

* [new tag] zephyr-v2.7.0 -> zephyr-v2.7.0
503c7138df2b7b0afa0f649c783b33e8d971f4d0 refs/tags/zephyr-v2.7.0
Updating files: 100% (17693/17693), done.
Note: switching to 'zephyr-v2.7.0'.



You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.


If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:


git switch -c <new-branch-name>


Or undo this operation with:


git switch -


Turn off this advice by setting config variable advice.detachedHead to false


HEAD is now at 3f826560aa release: Zephyr 2.7.0
Traceback (most recent call last):
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 806, in move
os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\datnd14\\.west\\manifest-tmp' -> 'C:\\Users\\datnd14\\zephyr'


During handling of the above exception, another exception occurred:


Traceback (most recent call last):
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\datnd14\AppData\Local\Programs\Python\Python39\Scripts\west.exe\__main__.py", line 7, in <module>
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\main.py", line 779, in main
app.run(argv or sys.argv[1:])
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\main.py", line 106, in run
self.run_command(argv)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\main.py", line 336, in run_command
cmd.run(args, unknown, self.topdir, manifest=self.manifest)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\commands.py", line 132, in run
self.do_run(args, unknown)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\project.py", line 198, in do_run
topdir = self.bootstrap(args)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\project.py", line 295, in bootstrap
shutil.move(os.fspath(tempdir), os.fspath(manifest_abspath))
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 824, in move
rmtree(src)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 740, in rmtree
return _rmtree_unsafe(path, onerror)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 613, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 613, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 613, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 618, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 616, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\datnd14\\.west\\manifest-tmp\\.git\\objects\\pack\\pack-92b9de99850183d98ff55ccee114517db6eeae5c.idx'

[marc-hb]

What happens when you try this after the failure:

del C:\Users\datnd14\.west\manifest-tmp\.git\objects\pack\pack-92b9de99850183d98ff55ccee114517db6eeae5c.idx

Please also share the output of west init -v -v... and edit your comment above.

[congnguyenhuu]

I tried delete that, but have access error although I grant permission

del C:\Users\datnd14\.west\manifest-tmp\.git\objects\pack\pack-5fe6ccedee5ad97c70bfdbc8b9476c8c056f7a98.idx
C:\Users\datnd14\.west\manifest-tmp\.git\objects\pack\pack-5fe6ccedee5ad97c70bfdbc8b9476c8c056f7a98.idx
Access is denied.

when run west init -v -v, log is quite long, so you can read in this file log.txt

[marc-hb]

Thanks! Can you please try this and report any error?

cd C:\Users\datnd14\
git init w558
cd w558
git remote add origin -- https://github.com/zephyrproject-rtos/zephyr.git
git fetch origin --tags -- zephyr-v2.7.0 'refs/heads/*:refs/remotes/origin/*'
del .git\objects\pack\*

[congnguyenhuu]

here is result when I run that

C:\Users\datnd14\w588>git remote add origin -- https://github.com/zephyrproject-rtos/zephyr.git

C:\Users\datnd14\w588>git fetch origin --tags -- zephyr-v2.7.0 'refs/heads/*:refs/remotes/origin/*'
remote: Enumerating objects: 607149, done.
remote: Counting objects: 100% (2/2), done.
remote: Total 607149 (delta 1), reused 1 (delta 1), pack-reused 607147
Receiving objects: 100% (607149/607149), 417.88 MiB | 596.00 KiB/s, done.
Resolving deltas: 100% (461118/461118), done.
From https://github.com/zephyrproject-rtos/zephyr
* tag zephyr-v2.7.0 -> FETCH_HEAD
* [new tag] v1.0.0 -> v1.0.0
* [new tag] v1.1.0 -> v1.1.0
* [new tag] v1.1.0-rc1 -> v1.1.0-rc1
* [new tag] v1.10.0 -> v1.10.0
* [new tag] v1.10.0-rc1 -> v1.10.0-rc1
* [new tag] v1.10.0-rc2 -> v1.10.0-rc2
* [new tag] v1.10.0-rc3 -> v1.10.0-rc3

    [snip]

* [new tag] zephyr-v2.2.0 -> zephyr-v2.2.0
* [new tag] zephyr-v2.2.1 -> zephyr-v2.2.1
* [new tag] zephyr-v2.3.0 -> zephyr-v2.3.0
* [new tag] zephyr-v2.4.0 -> zephyr-v2.4.0
* [new tag] zephyr-v2.5.0 -> zephyr-v2.5.0
* [new tag] zephyr-v2.6.0 -> zephyr-v2.6.0
* [new tag] zephyr-v2.7.0 -> zephyr-v2.7.0

C:\Users\datnd14\w588>del .git\objects\pack\*
C:\Users\datnd14\w588\.git\objects\pack\*, Are you sure (Y/N)? Y
C:\Users\datnd14\w588\.git\objects\pack\pack-9554992bbd0bdd6f346a8acf37ca89822648585f.idx
Access is denied.
C:\Users\datnd14\w588\.git\objects\pack\pack-9554992bbd0bdd6f346a8acf37ca89822648585f.pack
Access is denied.

[marc-hb]

Great, so now we know you don't have a west problem. I think you have a problem with your git client and/or the way Windows permissions are set up in your home directory. Can you try the same thing with a USB stick or just in a different directory?

It seems normal for git to create read only pack files but somehow this is not a problem for most people. Windows permissions can be very complicated and I'm no expert https://www.ntfs.com/ntfs-permissions-file-effective.htm

[congnguyenhuu]

yes, I think too and I granted permissions, reinstall git with chocolatey many times. but it can not work

[jaredwolff]

I'm getting the same error invoking west via cmd.exe though a VSCode extension.

Full command is:

C:\Windows\System32\cmd.exe /d /c west -v -v init -m https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git

Here's the output.

> Executing task: west -v -v init -m https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git <

=== Initializing in C:\Zephyr
--- Cloning manifest repository from https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git
running 'git clone https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git 'C:\Zephyr\.west\manifest-tmp'' in C:\Zephyr
Cloning into 'C:\Zephyr\.west\manifest-tmp'...
remote: Enumerating objects: 899, done.
remote: Counting objects: 100% (899/899), done.
remote: Compressing objects: 100% (521/521), done.
remote: Total 899 (delta 472), reused 750 (delta 327), pack-reused 0
Receiving objects: 100% (899/899), 6.71 MiB | 9.22 MiB/s, done.     
Resolving deltas: 100% (472/472), done.
west.manifest: loading C:\Zephyr\.west\manifest-tmp\west.yml
west.manifest: group-filter: unset
west.manifest: added project nrf path nrf revision v1.7.1 from C:\Zephyr\.west\manifest-tmp\west.yml
west.manifest: project <Project nrf ('C:\\Zephyr\\nrf') at v1.7.1>: ignored import (True) 
west.manifest: added project golioth path modules/lib/golioth revision main from C:\Zephyr\.west\manifest-tmp\west.yml
west.manifest: project <Project golioth ('C:\\Zephyr\\modules\\lib\\golioth') at main>: ignored import ({'name-allowlist': ['qcbor']})
west.manifest: added project pyrinas path pyrinas revision v1.7.x from C:\Zephyr\.west\manifest-tmp\west.yml
west.manifest: project <Project pyrinas ('C:\\Zephyr\\pyrinas') at v1.7.x>: ignored import (True)
west.manifest: loaded C:\Zephyr\.west\manifest-tmp\west.yml
Traceback (most recent call last):
  File "C:\Python39\lib\shutil.py", line 803, in move
    os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'C:\\Zephyr\\.west\\manifest-tmp' -> 'C:\\Zephyr\\nfed'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Jared Wolff\.zephyrtools\env\Scripts\west.exe\__main__.py", line 7, in <module>
  File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\main.py", line 781, in main
    app.run(argv or sys.argv[1:])
  File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\main.py", line 106, in run
    self.run_command(argv)
  File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\main.py", line 336, in run_command
    cmd.run(args, unknown, self.topdir, manifest=self.manifest)
  File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\commands.py", line 140, in run
    self.do_run(args, unknown)
  File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\project.py", line 198, in do_run
    topdir = self.bootstrap(args)
  File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\project.py", line 305, in bootstrap
    shutil.move(os.fspath(tempdir), os.fspath(manifest_abspath))
  File "C:\Python39\lib\shutil.py", line 815, in move
    rmtree(src)
  File "C:\Python39\lib\shutil.py", line 737, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Python39\lib\shutil.py", line 610, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Python39\lib\shutil.py", line 610, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Python39\lib\shutil.py", line 610, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Python39\lib\shutil.py", line 615, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Python39\lib\shutil.py", line 613, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Zephyr\\.west\\manifest-tmp\\.git\\objects\\pack\\pack-ae179110ddff9c4a119793934f0f44c69e9e0976.idx'
The terminal process "C:\Windows\System32\cmd.exe /d /c west -v -v init -m https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git" terminated with exit code: 1.

Terminal will be reused by tasks, press any key to close it.

Quite bizarre since west creates the files that it's copying to. Seems to be mostly related to the os.unlink function. I'm starting from a completely clean directory.

For context, I was using git to clone before but west init had an added benefit to initialize the application repo as the correct name as corresponding to self.path in west.yml

[jaredwolff]

Diff of the environment. Works in a stand alone cmd but not within VSCode

1d0
< !::=::\
3d1
< !ExitCode=00000000
9a8
> CHROME_CRASHPAD_PIPE_NAME=\\.\pipe\crashpad_9232_XCMYRGDVZKPZBDFN
24a24
> ORIGINAL_XDG_CURRENT_DESKTOP=undefined
26c26,27
< PATH=/cygdrive/c/Python39/Scripts:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:/usr/bin:/cygdrive/c/Windows/System32/OpenSSH:/cygdrive/c/Program Files/Wireshark:/cygdrive/c/Program Files/PuTTY:/cygdrive/c/ProgramData/chocolatey/bin:/cygdrive/c/Program Files/Git/cmd:/cygdrive/c/bin:/cygdrive/c/Python39:/cygdrive/c/Program Files (x86)/Nordic Semiconductor/nrf-command-line-tools/bin:/cygdrive/c/Program Files (x86)/SEGGER/JLink:/cygdrive/c/Program Files (x86)/ZeroTier/One:/cygdrive/c/Program Files/Microsoft SQL Server/120/Tools/Binn:/cygdrive/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/110/Tools/Binn:/cygdrive/c/Program Files (x86)/Microsoft SQL Server/120/Tools/Binn:/cygdrive/c/Program Files/Microsoft SQL Server/120/DTS/Binn:/cygdrive/c/Program Files (x86)/nodejs:/cygdrive/c/Users/Jared Wolff/AppData/Local/Microsoft/WindowsApps:/cygdrive/c/Users/Jared Wolff/AppData/Local/Programs/Microsoft VS Code/bin:/cygdrive/c/bin:/cygdrive/c/Python39:/cygdrive/c/Users/Jared Wolff/AppData/Roaming/npm
---
> PATH=/cygdrive/c/Python39/Scripts:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:/usr/bin:/cygdrive/c/Windows/System32/OpenSSH:/cygdrive/c/Program Files/Wireshark:/cygdrive/c/Program Files/PuTTY:/cygdrive/c/ProgramData/chocolatey/bin:/cygdrive/c/Program Files/Git/cmd:/cygdrive/c/bin:/cygdrive/c/Python39:/cygdrive/c/Program Files (x86)/Nordic Semiconductor/nrf-command-line-tools/bin:/cygdrive/c/Program Files (x86)/SEGGER/JLink:/cygdrive/c/Program Files (x86)/ZeroTier/One:/cygdrive/c/Program Files/Microsoft SQL Server/120/Tools/Binn:/cygdrive/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/110/Tools/Binn:/cygdrive/c/Program Files (x86)/Microsoft SQL 
> Server/120/Tools/Binn:/cygdrive/c/Program Files/Microsoft SQL Server/120/DTS/Binn:/cygdrive/c/Program Files (x86)/nodejs:/cygdrive/c/Users/Jared Wolff/AppData/Local/Microsoft/WindowsApps:/cygdrive/c/Users/Jared Wolff/AppData/Local/Programs/Microsoft VS Code/bin:/cygdrive/c/bin:/cygdrive/c/Python39:/cygdrive/c/Users/Jared Wolff/AppData/Roaming/npm
49a51,59
> TERM_PROGRAM=vscode
> TERM_PROGRAM_VERSION=1.63.2
> LANG=en_US.UTF-8
> COLORTERM=truecolor
> VSCODE_GIT_IPC_HANDLE=\\.\pipe\vscode-git-35da2a6a1e-sock
> GIT_ASKPASS=c:\Users\Jared Wolff\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass.sh
> VSCODE_GIT_ASKPASS_NODE=C:\Users\Jared Wolff\AppData\Local\Programs\Microsoft VS Code\Code.exe
> VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
> VSCODE_GIT_ASKPASS_MAIN=c:\Users\Jared Wolff\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass-main.js

~~Cygwin looks like the culprit~~

[jaredwolff]

~~Looks like it's because I'm running west in a virtual environment. Without the virtual environment, works ok. With the path to the virtual environment set, it breaks.~~

The plot thickens.

After lots of testing. I've narrowed it down to whether or not VSCode has the folder open. It appears to be scanning the files inside the .west folder. When west is ready to transfer the repo to it's final resting place, it's prevented from doing so because VSCode is indexing the files.

When I close VSCode and run it in a terminal with same environment, it runs perfectly fine.

Hope this helps people that may come across the same problem!

[mbolivar-nordic]

After lots of testing. I've narrowed it down to whether or not VSCode has the folder open. It appears to be scanning the files inside the .west folder. When west is ready to transfer the repo to it's final resting place, it's prevented from doing so because VSCode is indexing the files.

@wbober @trond-snekvik I was thinking about the VS code extension and I wanted to let you know about this to see if you've ever observed random issues like this.

Some of the above discussion looks like random permissions problems that unfortunately I have no idea how to triage right now, but this particular issue reported by @jaredwolff might be something you are running into more often.

[marc-hb]

it's prevented from doing so because VSCode is indexing the files.

It would help if Windows could delete files in use like Linux filesystems can (it would also save a lot of Windows reboots, I digress)

However similar issues could happen on Linux too: a few years ago, it came as a big surprise to me that some operations like "git diff" appear to be read-only but in fact they have some side effects because they can cause git to update some of its internal data: https://public-inbox.org/git/[email protected]/ So in theory we could have a similar race where some Linux IDE triggers some not-so-read-only git commands on a half-moved repo and ends up corrupting it. Very unlikely but possible?

So, @mbolivar-nordic should west switch to some OS-dependent and "standard" /tmp locations to escape this sort of concurrency? Even if the interactions with IDEs ends up being harmless, it's not "nice" to confuse them.

[mbolivar-nordic]

So, @mbolivar-nordic should west switch to some OS-dependent and "standard" /tmp locations to escape this sort of concurrency? Even if the interactions with IDEs ends up being harmless, it's not "nice" to confuse them.

I thought about that, but then you're inviting issues with cross-filesystem moves, which are a can of worms of their own. If you have a strategy for accomplishing this ask, I'm open to ideas, though.

[marc-hb]

but then you're inviting issues with cross-filesystem moves,

OMG, I referenced something from 2016 but now you're bringing back memories from 2010!! https://bugs.eclipse.org/bugs/show_bug.cgi?id=319233 "Cloning repository fails if destination is on network drive because File.renameTo doesn't succeed"

How about zephyr/.manifest-tmp instead of zephyr/manifest-tmp? Or some "hidden" Windows file attribute? Both? @jaredwolff would you be able to test these ~on Windows~ with VS code?

[jaredwolff]

My work around with this, from what I remember, was disabling the Git functionality in VSCode temporarily while west did its thing. It was trying to cache or open the repository in the manifest-tmp folder and thus locking up the files. Next time I'm working on my VSCode plugin I'll have to see if changing the manifest-tmp directory name fixes it.