Paket icon indicating copy to clipboard operation
Paket copied to clipboard

Paket install failure on GitHub file dependencies when file has been removed

Open Thorium opened this issue 3 years ago • 4 comments

If you use GitHub dependencies with specific files:

github janl/mustache.js mustache.min.js

...and the file is there, at the time you reference it (https://github.com/janl/mustache.js/commit/8e52a4ac6cf4ed86d0fedb3c4dae643fd7b56998)... but then the author removes it from the repository in later https://github.com/janl/mustache.js/commit/2061046686c46fd5b2b54611f4eb55f56991a210 and then when you run install (not even update!), the Paket fails, throws the towel in, and stops the whole process:

A:\> .paket\paket.exe install

Downloading janl/mustache.js:550d1da9e3f322649d04b4795f5356914f6fd7e8 mustache.min.js to C:\git\welendus\paket-files\javascript\janl\mustache.js\mustache.min.js
  - Download failed. Trying again. (4 trials left)
  - Download failed. Trying again. (3 trials left)
  - Download failed. Trying again. (2 trials left)
  - Download failed. Trying again. (1 trials left)
  - Download failed. Trying again. (0 trials left)
Performance:
 - Average Download Time: 48 milliseconds
 - Number of downloads: 6
 - Average Request Time: 215 milliseconds
 - Number of Requests: 13
 - Runtime: 3 seconds
Paket failed with
-> Could not download from 'https://raw.githubusercontent.com/janl/mustache.js/550d1da9e3f322649d04b4795f5356914f6fd7e8/mustache.min.js'

A:\>

...after trying to get only the last commit in that repo https://github.com/janl/mustache.js/commit/550d1da9e3f322649d04b4795f5356914f6fd7e8

That is clearly incorrect. After that, trying to trace every file history manually, what happened and where should I get it now is slow and time consuming.

Expected behavior

  1. Either keep the existing commit as I was not trying to upgrade the file.
  2. Or trace where the file was removed and use one before that commit (which may hammer GitHub API too much)
  3. Or try to take the most recent commit and if not found, then fall-back to the current version.

Just please don't fail. On every file. It's very typical the authors change the files in Github repositories, and if you have 15 file dependencies, you basically fail the install command every time.

Thorium avatar Jul 01 '21 15:07 Thorium

Repro steps

Ok, here is another one...

paket.dependencies:

source https://api.nuget.org/v3/index.json

github shutterstock/rickshaw vendor/d3.v3.js

paket.lock:

GITHUB
  remote: shutterstock/rickshaw
    vendor/d3.v3.js (382dfa1eae051c73071fbc015b32f88f9c34ac1c)

then run .paket\paket.exe install

Thorium avatar Jul 01 '21 15:07 Thorium

It seems that the Paket.exe install is not using the paket.lock file but "the latest commit of that repository". Which is incorrect behaviour. The lock-file would have the correct commit id.

Thorium avatar Jul 12 '21 09:07 Thorium

if no commit ref is specified, paket defaults to the latest commit hash on the master branch. you can control this, though:

github janl/mustache.js:8e52a4ac6cf4ed86d0fedb3c4dae643fd7b56998 mustache.min.js

this should do what you want. you can also use the short hash, or a branch name, or a tag.

This is based on reading the following from the github dependencies doc page:

By default the master branch is used to determine the commit to reference, you can specify the desired branch or commit in the paket.dependencies file:

github fsharp/fsfoundation:gh-pages img/logo/fsharp.svg
github forki/FsUnit:7623fc13439f0e60bd05c1ed3b5f6dcb937fe468 FsUnit.fs

And the implementation backs that up from what I've been reading.

baronfel avatar Jul 12 '21 20:07 baronfel

Thanks for the workaround. That will do for now.

However the default behaviour:

  • Is not very useful as the repo-owners often change their source filenames. Especially in master after major merges.

  • Is logically incorrect, a build should be atomic, not breaking if I don't paket update.

Thorium avatar Jul 12 '21 21:07 Thorium