elasticdog
yadm — Initialization with imported key failing ( Issues with yadm 2/2)
Hello, I've integrated transcrypt into my yadm repository and stumbled over a few things. No critical errors. I just wanna post some notes about the issues.
Version: 2.3.1-pre Commit: 016b2e4b31951be5ea96233d8d2badef9c9836b
cd .local/share/yadm/repo.git
transcrypt --import-gpg="$CREDENTIALS_PATH_GPG"
Shows:
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
Repository metadata:
GIT_WORK_TREE: /home/username
GIT_DIR: /home/username/.local/share/yadm/repo.git
GIT_ATTRIBUTES: /home/username/.gitattributes
The following configuration will be saved:
CONTEXT: default
CIPHER: aes-256-cbc
PASSWORD: XXXX
Does this look correct? [Y/n]
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Maybe, transcrypt stumbled about the GIT_WORK_TREE ?! It aborts during the decryption of the first file.
yadm status
[…]
Changes not staged for commit:
(use "git add/rm <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
deleted: ../alt/.i3/lock_screens##class.Private/Info.txt
So it stops at the decryption of the first file of yadm ls-crypt.
After yadm checkout -f the decryption/encryption with transcrypt works, so the importing itself works.
Regards, Olaf
It looks like it might be the force_checkout function that is failing, it calls git ls-crypt-${CONTEXT} (CONTEXT is "default" by default) then git checkout --force HEAD -- "$file" for each file listed.
I'm not familiar with yadm. Do you know what is different about how it invokes git, or the environment variables it sets, that could cause these command to fail?
Similar to issue 189 the explicit setting of GIT_DIR will work around the issue:
cd /tmp/tmp.jK4dyZgEQw/.local/share/yadm/repo.git/
export GIT_DIR=$(pwd)
transcrypt --import-gpg=[SomeKey.asc]
During the debugging I've found a typo in export_gpg. Missing '.' before cipher:
@@ -1072,7 +1072,7 @@ export_gpg() {
fi
local current_cipher
- current_cipher=$(git config --get --local "transcrypt${CONTEXT_CONFIG_GROUP}cipher")
+ current_cipher=$(git config --get --local "transcrypt${CONTEXT_CONFIG_GROUP}.cipher")
local current_password
current_password=$(load_password "$CONTEXT_CONFIG_GROUP")
mkdir -p "${CRYPT_DIR}"
Regards Olaf
I've wrote a little test script for the error, see below. My conclusion: The force_checkout() function assumes that the repo directory and the worktree directory are the same. Using "$GIT_DIR" instead of "$REPO" and fixing relative paths in the for-loop resolving the issue.
diff --git a/transcrypt b/transcrypt
index e75ae7a..db81628 100755
--- a/transcrypt
+++ b/transcrypt
@@ -781,11 +781,13 @@ force_checkout() {
# but we already made sure the repo was clean during the safety checks
local encrypted_files
encrypted_files=$(git "ls-crypt-${CONTEXT}")
- cd "$REPO" >/dev/null || die 1 'could not change into the "%s" directory' "$REPO"
+ # Fix for 'yadm': If 'git config --local core.worktree' is set, the paths '$REPO' and '$GIT_DIR' can differ
+ # Using GIT_DIR here and in for-loop prefix '${REPO:-.}/'
+ cd "$GIT_DIR" >/dev/null || die 1 'could not change into the "%s" directory' "$GIT_DIR"
IFS=$'\n'
for file in $encrypted_files; do
- rm -f "$file"
- git checkout --force HEAD -- "$file" >/dev/null
+ rm -f "${REPO:-.}/${file}"
+ git checkout --force HEAD -- "${REPO:-.}/${file}" >/dev/null
done
unset IFS
fi
Test script for --import-gpg, etc:
#!/bin/bash
# Test script to check 'transcrypt --import-gpg' inside of yadm repo.
#
# It creates an yadm repo in a temporary folder, encrypting a file and then tries to run --upgrade.
#
# Script assumes that yadm is installed.
# Debian: sudo apt install yadm
#
set -euo pipefail
#set -x
## Create temp dir and delete it at exit of script.
YADM_TMP_DIR=$(mktemp -d)
test "$?" -ne 0 && exit -1
function introspect {
echo -e "\n\n\tWrite 'bash' to open interactive shell for introspection." \
"\n\tWrite [Enter] to clear temp directory '$YADM_TMP_DIR'\n"
read SHELL_OR_QUIT
if [ "$SHELL_OR_QUIT" = "bash" ] ; then
#bash --rcfile <(echo "export PS1='$PS1 |> '")
bash --rcfile <(echo "export PS1='|> '")
fi
}
function cleanup {
introspect
rm -rf "$YADM_TMP_DIR"
}
trap cleanup EXIT
trap cleanup SIGINT
## Download current transcrypt version
cd "$YADM_TMP_DIR"
wget "https://raw.githubusercontent.com/elasticdog/transcrypt/refs/heads/main/transcrypt"
#cp $HOME/transcrypt transcrypt
# Make it available for the other tools
chmod +x transcrypt
PATH=$(realpath "$YADM_TMP_DIR"):$PATH
## Init yadm repo
# Using temp dir instead of $HOME
WORKTREE_ROOT="$YADM_TMP_DIR"
# To avoid writing yadm config into $HOME/.config/yadm, relocate home dir…
export HOME="$YADM_TMP_DIR"
# Note that 'git commit' still complains about missing global variables even
# if the local variables are set. Thus, we have to set .gitconfig
echo "[user]
email = [email protected]
name = Test Test" > "${YADM_TMP_DIR}/.gitconfig"
yadm init -w "$WORKTREE_ROOT"
# Add a file
cd "$WORKTREE_ROOT"
echo "Public content" > "public_file"
yadm add "public_file"
yadm commit -m "Adding unencrypted file"
echo "Yadm is placing bare repo files here: $(yadm introspect repo)"
## Setup transcrypt in repo…
yadm transcrypt -y
## Add info about file to encrypt
echo 'sensitive_file filter=crypt diff=crypt merge=crypt' > \
"$(yadm introspect repo)/info/attributes"
echo 'Secret content' > "sensitive_file"
yadm add sensitive_file
yadm commit -m "Adding encrypted file"
## Test --export-gpg and --import-gpg
# Prevent update of users keys (even if $HOME is not redirected)
export GNUPGHOME="$YADM_TMP_DIR/.gnupg"
# Generate Test Key
gpg --batch --passphrase '' --quick-gen-key Transcrypt
#gpg --list-keys
# Export key
cd "$(yadm introspect repo)"
transcrypt --export-gpg=Transcrypt
test -f "crypt/Transcrypt.asc" && echo "Export ok"
# Uninstall transcrypt as preparation for --import-gpg
mv "crypt/Transcrypt.asc" "$YADM_TMP_DIR/." # otherwise uninstall is failing
#transcrypt --uninstall -y # Again, GIT_DIR is missing.
yadm transcrypt --uninstall -y
yadm checkout -f # Because trancrypt leaving repo in dirty state
# Import key
transcrypt --import-gpg="$YADM_TMP_DIR/Transcrypt.asc" -y
Thanks for digging into this issue and providing the great test scripts.
I have run some more tests and used your test scripts and I'm coming to the unfortunate conclusion that this probably isn't an issue that can be fixed inside transcrypt. The root issue seems to be that in your yadm worktrees setup, git itself needs to have the $GIT_DIR environment variable set correctly to be able to provide even basic settings required by transcrypt.
In other words, you need to run yadm transcrypt instead of just transcrypt or to set GIT_DIR=$(yadm introspect repo) before running transcrypt.
For example in a working tree directory set up by yadm per your test script, these key config lookup commands required by transcrypt fail with different error messages: git rev-parse --show-toplevel and git config --get --local transcrypt.version
As soon as you set GIT_DIR=$(yadm introspect repo) those same commands work as expected in the working tree directory.
But unless GIT_DIR is set somehow by yadm, as far as I can tell there isn't a way for transcrypt to learn what that GIT_DIR path should be. Any attempt to look up basic Git config settings fails, per the above. So unless I'm missing something, transcrypt (and most likely any git commands or tools) need to be run inside the yadm environment to work correctly.
[…] In other words, you need to run yadm transcrypt instead of just transcrypt or to set
GIT_DIR=$(yadm introspect repo)before running transcrypt.
Yes, during my tests in the last days this solution works. In my memory, this wasn't enough for an unknown transcrypt command, but I can no longer reproduce this.
[…] But unless
GIT_DIRis set somehow by yadm, as far as I can tell there isn't a way for transcrypt to learn what thatGIT_DIRpath should be. Any attempt to look up basic Git config settings fails, per the above. So unless I'm missing something, transcrypt (and most likely any git commands or tools) need to be run inside the yadm environment to work correctly.
Nesting transcrypt commands with yadm is probably a sufficient solution, but I want note that's there is no magic learning about the path.
The changes made by yadm just affects GIT_DIR and GIT_WORK_TREE, see yadm enter env | grep -i git . And in transcrypt you fetching the same values: GIT_DIR and REPO.
# the current git repository's top-level directory
readonly REPO=$(git rev-parse --show-toplevel 2>/dev/null)
# the current git repository's .git directory
readonly RELATIVE_GIT_DIR=$(git rev-parse --git-dir 2>/dev/null || printf '')
readonly GIT_DIR=$(realpath "$RELATIVE_GIT_DIR" 2>/dev/null)
Just the usage of git config --local after cd "$REPO" is problematic because you could not rely on "$GIT_DIR" == "$REPO/.git".
I found just the one occasion of cd "$REPO" (mentioned in the previous post) With this change I can use transcrypt without nesting by yadm.
You are right that there is nothing magic about $GIT_DIR because Git itself understands and uses this environment variable, but when the git repo/working-copy and metadata directories are in unusual places (i.e .git/ directory is not in the repo) that environment variable is required for git config --local lookups to work.
Prefixing commands with yadm does this job of providing the required GIT_DIR environment variable. For non-standard setups there is no way for transcrypt to figure out what GIT_DIR should be if it's not provided.