E3SM-Project
Git submodule update fails with latest FATES on Mac
When trying to update to the latest version of master using a clone on my Mac laptop, I'm seeing this error when updating the submodules:
jacob@Roberts-MacAirM2-2 E3SM % git submodule update --init --recursive
git-lfs filter-process: git-lfs: command not found
fatal: the remote end hung up unexpectedly
fatal: Unable to checkout '1afb393de4c4d3c74c2f95ca714b9f34996abbdf' in submodule path 'components/elm/src/external_models/fates'
If I immediately run it again, I get this error:
jacob@Roberts-MacAirM2-2 E3SM % git submodule update --init --recursive
error: The following untracked working tree files would be overwritten by checkout:
.gitattributes
.github/ISSUE_TEMPLATE/bug.yml
.github/ISSUE_TEMPLATE/feature.yml
.github/ISSUE_TEMPLATE/task.yml
.github/PULL_REQUEST_TEMPLATE.md
.github/images/logo_fates_large.png
.github/images/logo_fates_medium.png
.github/images/logo_fates_small.png
.github/workflows/add-to-gh_projects.yml
.gitignore
.gitmodules
.zenodo.json
CMakeLists.txt
CODE_OF_CONDUCT.md
CONTRIBUTING.md
(more files listed)
main/FatesHistoryInterfaceMod.F90
main/FatesHistoryVariableType.F90
main/Fat
Aborting
fatal: Unable to checkout '1afb393de4c4d3c74c2f95ca714b9f34996abbdf' in submodule path 'components/elm/src/external_models/fates'
jacob@Roberts-MacAirM2-2 E3SM % git --version git version 2.50.1 (Apple Git-155)
This seemed to start with merge of https://github.com/E3SM-Project/E3SM/pull/7665.
Doesn't happen on Chrysalis so possibly related to Mac filesystem case insensitivity.
I ran into the same error on Chrysalis this morning.
git-lfs filter-process: git-lfs: command not found fatal: the remote end hung up unexpectedly fatal: Unable to checkout '5bb36cba29b95295aca5aedc13f348ef1e001b99' in submodule path 'components/elm/src/external_models/fates'
And I also got the same error on Compy.
Unable to checkout '5bb36cba29b95295aca5aedc13f348ef1e001b99' in submodule path 'components/elm/src/external_models/fates'
@mahf708
On both machines, I get the error:
git: 'lfs' is not a git command. See 'git --help'.
The most similar command is
log
I think I need to download the git-lfs-X.X.X and then try to install it. Wouldn't it be optimum to have them installed on both machines by the admins instead of each user installing them on their own?
I don't know how Rob managed to get it working on chrysalis and the fact we are not hitting this with testing, so it could be something in the user space causing this, idk...
You can also do some filtering/exclusion of the specific lfs settings. From a quick look, these appear to be the offending settings:
testing/test_data/*.nc filter=lfs diff=lfs merge=lfs -text
A quick workaround is to have git-lfs inside a conda env (conda install git-lfs) and do git lfs install which results in the following in global gitconftg
$ tail -n5 ~/.gitconfig
[filter "lfs"]
clean = git-lfs clean -- %f
smudge = git-lfs smudge -- %f
process = git-lfs filter-process
required = true
then proceed as usual with git clone ..., git submodule ...
Thanks @mahf708 for finding the cause of the error and suggesting a workaround.
@evasinha, from a quick test, it seem simply deleting these lines from ~/.gitconfig is a valid workaround:
[filter "lfs"]
clean = git-lfs clean -- %f
smudge = git-lfs smudge -- %f
process = git-lfs filter-process
required = true
Deleting the lines from .gitconfig is the best workaround.
But I'd like to know why FATES needs git-lfs? We thought about using it a while ago and decided not to because its not generally available on every platform with git. @rgknox @glemieux
What I also don't understand is why this just happened now (for me on my laptop) considering it first happened way back in October.
@mahf708 Modifying ~/.gitconfig to remove the following lines worked and all modules were successfully updated.
[filter "lfs"]
clean = git-lfs clean -- %f
smudge = git-lfs smudge -- %f
process = git-lfs filter-process
required = true