sfdx-git-delta Getting `Could not find` error when generating delta source

Issue verification check:

[X] is the current repository fully deployable at the commit SHA provided with the 'from' parameter of the command?

What is the problem?

We are getting an error when trying to generate delta source with sfdx-git-delta versions above 5.34.0.

What is the parameter and the value you used with it?

sgd source delta --to HEAD --from stash/master --source C:<REPO_PATH> --api-version 59.0 --output C:<REPO_PATH>\delta-src/ --generate-delta

What is the expected result?

Delta source generation succeeds.

What is the actual result?

Delta source generation fails. Json Response: {"success":false,"error":"Could not find b2c43ed8d669af3b0af194a54362238c0ed66de5.","output":"C:\<REPO_PATH>\delta-src/","warnings":[]}

Steps to reproduce

We are unable to reproduce the issue locally (within our machines) the issue is occurring within our build servers.

Execution context

Within our build servers we use Bamboo and Windows 2012. Operating System: Windows 2012

yarn version: N/A

node version: v18.12.0

git version: git version 2.29.2.windows.3

sfdx version: @salesforce/cli/2.40.7 win32-x64 node-v20.12.2

sgd plugin version: sfdx-git-delta 5.39.2 (5.39.2)

May 14 '24 17:05 kez-sophos

To clarify, sfdx-git-delta version 2.34.0 works as expected for us. We encounter this issue with later versions.

May 14 '24 18:05 kez-sophos

Hi @kez-sophos !

Thanks for raising this issue and thanks for contributing in making this project better!

We are aware of this issue already, it has been discovered with the #762 But we are not able to reproduce it locally so we cannot fix it yet. I tried a lot of things (VM, etc) without success.

You are right, the issue is not there with the v5.34.0 (current stable channel) because after this point we have refactored the way we interact with git. After that point we are using isomorphic-git to abstract and speed up the interaction with git. Unfortunately it has some issues with windows, especially with very large and aged repository...

Do you have a way to reproduce the issue I could use with a VM ?

May 14 '24 18:05 scolladon

Hi @kez-sophos, do you have any way to help me reproduce the issue ?

May 28 '24 17:05 scolladon

Hi @kez-sophos I need your help to reproduce please, could you send us the minimal steps to follow to reproduce it please ?

Jun 04 '24 12:06 scolladon

we also get the issue with sfdx-git-delte version 5.42.0 on our build servers and if we downgrade to the vertion 5.43.0 then the package creation is working again but with the older API version 59.0

OS: MS Windows Server 2019 node version: 20.11.1 git vertion: git version 2.44.0.windows.1 sf version: @salesforce/cli/2.53.6 win32-x64 node-v20.11.1

on my local machine Windows 11 and pretty similar versoins the issue is not appearing

any ideas what could be the cause of the problem?

Aug 14 '24 13:08 ALesc21

Thanks for those information. It really helps

What api version are you using ? 61 ? And what version of sgd is working ? (5.43.0 does not exist yet)

Aug 14 '24 14:08 scolladon

hi @scolladon,

we try or have to use the api vesion 61 because some implementations already requires it.

sorry that was a number mix-up. So the sgd version 5.42.0 is not working then we downgraded to 5.34.0 like "[kez-sophos]" wrote and it start to work without issue

Aug 15 '24 05:08 ALesc21

This issue is very hard because it is windows dependent and also requires a very big project. If you can give me access to your project where you reproduce the issue it will greatly help reproducing and see what happens. I suspect the issue comes from a library we are using that need to be patched, but no customers so far have been able to provide a complete reproduction scenario working on my machine 😪 I can sign NDA, or anything if necessary

Aug 15 '24 10:08 scolladon

hi @scolladon, I will not be available for some days but afterwards maybe we can plan a session for screen sharing and check together.

Aug 15 '24 12:08 ALesc21

That's nice of you @ALesc21 to give a bit of your time but it is very hard to investigate with a screen share and a limited amount of time. We tried already with another person and we did not manage to find the root cause. I would really feel more confident if I can reproduce the issue on my laptop

Aug 15 '24 13:08 scolladon

HI @scolladon like I wrote you in the email it make sense to look in our environment because the execution on my personal client is working and just on the Windows Servers the issue is happening.

Oct 23 '24 14:10 ALesc21

HI @scolladon like I wrote you in the email it make sense to look in our environment because the execution on my personal client is working and just on the Windows Servers the issue is happening.

Do you know if it is possible to debug in your environment ? Or do you know how I could reproduce the issue so I can debug ? (Maybe create a virtual machine and use your repository, etc)

Oct 23 '24 15:10 scolladon

I have full access on this servers so we can do what ever it needs. If you have some ideas what I have to set then let me know.

For reproducing you need the windows server + the installed tools such: git, node, sf and sfdx-git-delta. On this server I run the command "sf sgd:source:delta --to HEAD --from HEAD^ --output " and with the versions higher than 5.34.0 I receive the following outcome (in this case I tryed to install the newest sfdx-git-delta version 5.49):

C:\BuildAgent01\work\86775e496ec9c505>sf sgd:source:delta --to HEAD --from HEAD^ --output deployment/ONECRM-0000 (node:9488) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead. (Use node --trace-deprecation ... to show where the warning was created) (node:9488) Warning: Deprecated config name: apiVersion. Please use org-api-version instead. { "error": "Could not find 9746fb6b644a0a615e7de4902ed68c916e551f2f.", "output": "deployment/ONECRM-0000", "success": false, "warnings": [ "[INFORMATION] sfdx-git-delta v6 is coming soon! Read more and plan your migration: https://github.com/scolladon/sfdx-git-delta/issues/936" ] }

So maybe you find time for screensharing and we are going through and compare.

Oct 24 '24 05:10 ALesc21

Ok @ALesc21, let's setup a screen share. But first you'll need to setup the windows server so we can debug and extract the most of this session.

What I want is to be able to debug the code. So you need to clone this repository and set it up for debug Make sure you have git bash and chrome installed on this server.

cd <folder of your choice>
git clone [email protected]:scolladon/sfdx-git-delta.git
cd sfdx-git-delta
yarn
yarn pack
mkdir output
node --inspect-brk ./bin/run -f HEAD~1 -d
# it should wait for the debugger to be attached, open chrome and the developer console and click on the node icon on the left.

If you get this far then it will be easier to understand what is happening in this case. And from here we will be able to setup a screen share :)

Let me know when ready !

Oct 24 '24 09:10 scolladon

Sounds great @scolladon I will prepare it and let you know. Just to be 100 % sure I would like know if we need the internet access or something else because the servers are behind the firewall and has restricted access to specific sites and downloads.

Oct 24 '24 12:10 ALesc21

Internet access is necessary only to clone the repository @ALesc21. If it is not possible to clone but you can pass files to the server then we will clone on your machine and pass the repo to the server via your machine

Then we just need to be able to access the virtual machine and run the code.

Oct 24 '24 13:10 scolladon

I am getting similar error with version 5.49.1

error	04-Nov-2024 17:56:35	(node:9680) Warning: Deprecated config name: apiVersion. Please use org-api-version instead.
error	04-Nov-2024 17:56:35	(Use `node --trace-warnings ...` to show where the warning was created)
build	04-Nov-2024 17:56:35	{
build	04-Nov-2024 17:56:35	  "error": "Could not find d11e5347d09f3b50be5d8556b5213589bfa21bf0.",
build	04-Nov-2024 17:56:35	  "output": ".",
build	04-Nov-2024 17:56:35	  "success": false,
build	04-Nov-2024 17:56:35	  "warnings": [
build	04-Nov-2024 17:56:35	    "[INFORMATION] sfdx-git-delta v6 is coming soon! Read more and plan your migration: https://github.com/scolladon/sfdx-git-delta/issues/936"
build	04-Nov-2024 17:56:35	  ]
build	04-Nov-2024 17:56:35	}

Nov 04 '24 20:11 dsweetmanusda

I am getting similar error with version v5.49.1

What is the environment you are running in ? (Mac / Windows ? Node version ? Git version ?) Could you help me reproduce the issue by giving me access to the repository ? (I can sign NDA)

Nov 04 '24 20:11 scolladon

the environment is run by another organization, I am "just" a user. Running on Linux. sfdx-cli/7.209.6 linux-x64 node-v18.20.4 sfdx-git-delta 5.49.1