AGWA
'git-crypt status' with the '-f' option to stage an encrypted version fails for me?
git-crypt status tells me about some "staged/comited" files that are NOT encrypted, but should be.
And then tells me to run "Run 'git-crypt status' with the '-f' option to stage an encrypted version."
not encrypted: .git-crypt/.gitattributes
not encrypted: .gitattributes
not encrypted: .gitignore
<cut some lines>
not encrypted: etc/config/uhttpd
encrypted: etc/config/wireless *** WARNING: staged/committed version is NOT ENCRYPTED! ***
not encrypted: etc/dnsmasq.conf
encrypted: etc/dropbear/dropbear_dss_host_key *** WARNING: staged/committed version is NOT ENCRYPTED! ***
encrypted: etc/dropbear/dropbear_rsa_host_key *** WARNING: staged/committed version is NOT ENCRYPTED! ***
not encrypted: etc/firewall.user
<cut some lines>
encrypted: etc/shadow *** WARNING: staged/committed version is NOT ENCRYPTED! ***
not encrypted: etc/shells
not encrypted: etc/sysctl.conf
not encrypted: etc/sysupgrade.conf
encrypted: etc/uhttpd.crt *** WARNING: staged/committed version is NOT ENCRYPTED! ***
encrypted: etc/uhttpd.key *** WARNING: staged/committed version is NOT ENCRYPTED! ***
Warning: one or more files is marked for encryption via .gitattributes but
was staged and/or committed before the .gitattributes file was in effect.
Run 'git-crypt status' with the '-f' option to stage an encrypted version.
git-crypt status -f
it returns with:
Error: etc/config/wireless: still unencrypted even after staging
Error: etc/dropbear/dropbear_dss_host_key: still unencrypted even after staging
Error: etc/dropbear/dropbear_rsa_host_key: still unencrypted even after staging
Error: etc/shadow: still unencrypted even after staging
Error: etc/uhttpd.crt: still unencrypted even after staging
Error: etc/uhttpd.key: still unencrypted even after staging
Unable to stage 6 files.
What now? How do I fix this?
OK, I got past that now. But now can't remember what I did in the past to avoid this.
OK, so first how I got 'past it'? If I do a git-crypt init in this repo I can then do a `git-crypt status -fand that now does encrypt the staged files that it should encrypt according to the .gitattributes file. So far so good, but this now created a new symmetric key in this repo in.git/git-crypt/keys/default``, which is not wrong per se, but not what I have for my other encrypted git repos.
for all the other git-crypt repos the 'default' key is same as on the upstream master.
I can't remember what I used to do to create a new clone, but I do remember what I did this time.
I used simple git clone myupstream-master-repo-with-git-crypt my-new-slave-repo.
So for some reason all my other 'slave' repos do have the same 'default' key but the two I created today do not? This is likely NOT a issue with the prodcut but an issue with MY MEMORY. I jsut can't remember what I used to do differently, to clone rpeos and have them use the same git-crypt key as the upstream master? I used to be bback on 0.4 and have upgraded to 0.5 before doing the two repos today, but surely that would not have this effect, right?
my upstream master git-crypt repo
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Configs/.git/git-crypt/keys/default
shasum -a 256 OpenWrt-AP-Config*/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_2Lv-dn/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_2Lv-up/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_3Lv/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_4Lv/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_Bungalow/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_Flat1/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_Flat5/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_Flat6/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_Flat7/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551 OpenWrt-AP-Config_Test_AP/.git/git-crypt/keys/default
are all using the same git-crypt default key but todays repos are not, because I jsut typed git-crypt init in them.
ba7079c92061c44ee1e919cf210d217abfdb2b37bd3ca09f042afdb0bffb5e7e OpenWrt-AP-Config_Flat2b/.git/git-crypt/keys/default
8cff463861522bd2023adc604c661c03864bca75db380b4d591ea7153f3f90cf OpenWrt-AP-Config_Flat4/.git/git-crypt/keys/default
@AGWA sorry the hashing of .git/git-crypt/keys/default above makes absolutely no sense. ignore it, I hadn't noticed that the default file is a directory.
But I now have a problem and don't know how to reverse things back to a stable git repo in my upstream master git repo.
If I try any git reset -hard HEAD or a git reset -hard some-commit-sha
then it gives me a series of:
git-crypt: error: encrypted file has been tampered with!
error: external filter "git-crypt" smudge failed 1
error: external filter "git-crypt" smudge failed
fatal: etc/config/wireless: smudge filter git-crypt failed
Help, how do I get back and resume working using git-crypt ?
Same problem. Users key was added with git-crypt add-user, user cloned repo, added file which matched the filter, the file was pushed up to remote unencrypted.
git-crypt-status says:
$ git-crypt status
not encrypted: .git-crypt/.gitattributes
not encrypted: .git-crypt/keys/default/0/6171E0D3EEFAAB1253838C9A883985B9345C8EE2.gpg
not encrypted: .git-crypt/keys/default/0/78211BF6AA2FAA2AC717FADA846CB77A2F46D007.gpg
not encrypted: .git-crypt/keys/default/0/ACBA87D39C7E935A0858F7DF6538A56FC91FAD09.gpg
not encrypted: .git-crypt/keys/default/0/B035C493C480B1B523D301C67B3549A7371F5324.gpg
not encrypted: .gitattributes
not encrypted: TEST.md
encrypted: a_secret.sls
not encrypted: contributors.txt
encrypted: foobar-secret.sls *** WARNING: staged/committed version is NOT ENCRYPTED! ***
encrypted: foo/secret_things.sls
encrypted: phony.key
Warning: one or more files is marked for encryption via .gitattributes but
was staged and/or committed before the .gitattributes file was in effect.
Run 'git-crypt status' with the '-f' option to stage an encrypted version.
git version 2.7.4 git-crypt .5
Additional +1 to this problem. Starting investigation, will edit this comment as I figure out what's what. To start,
$ git version
git version 2.11.0
$ git-crypt version
git-crypt 0.5.0
$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.0 (stretch)
Release: 9.0
Codename: stretch
Update 12:06pm: It's affecting multiple files. Unsure why. Update 12.08pm: It's the same new file in a new checkout. Update: 12.11pm: Looks like in the new checkout, it fails when git-crypt hasn't' been unlocked yet. Update: 12.14pm: Yeah, looks like that was the problem in the old repo also. Running the command
$ git-crypt unlock
restored the previous transparent behaviour of encrypting resources.
Suggest as a resolution you refuse to commit encrypted files if git-crypt is locked? Also, suggest that you warn somehow during the commit process if you cannot encrypt files? Both @figtrap and I managed to push an unencrypted resource accidentally (doesn't matter for me, as I just burn the credentials and it gets caught in review, but would be an easy one for a new person not to check)
I've also run into this exact issue and it resulted in accidentally pushing unencrypted secrets.
hmm for me steps that are suggested by @andrewhowdencom don not work, can not get one file staged and encrypted.
What I've done:
- git crypt init
- git crypt add-gpg-user
- vim .gitattributes (add 3 files there)
- git add (those 3 files)
- git commit
- git crypt status (shows 2 files encrypted and 1 not encrypted)
- tried reverting the commit with git reset HEAD~1 and git crypt status -f broken-file didn't help
- tried applying that in a separate commit - didn't help
- tried locking and unlocking the repo and then committing again - didn't help
git version 2.7.4
git-crypt 0.5.0
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial
gpg --list-keys
/home/aboritskiy/.gnupg/pubring.gpg
-----------------------------------
pub 4096R/26179899 2017-04-12 [expires: 2033-04-XX]
uid Anton Bortitskiy <[email protected]>
uid Anton Bortitskiy <[email protected]>
sub 4096R/E5D8E6FD 2017-04-12 [expires: 2033-04-XX]
ok figured it out, the file which git crypt was unable to encrypt was referenced in two .gitattributes files on different levels of the folder structure. as soon as I fixed that - git crypt happily encrypted the file.
@AGWA, Not sure why this issue is still open, and I noticed there hasn't been any code activity in this repo since ~ November of 2017. That said, I am firm believer of "if ain't broke don't fix it™️".
@TheBigBear, I've only recently started using git-crypt in some of my public repos on GitHub so I am no expert by any stretch, but I'd like to think I have a working "workflow" for how to maintain sanity 👩🏾⚕️ when using git-crypt with a public repo.
@aboritskiy, for you guys n gals the git-crypt two step 🕺 goes as follows,
- Obviously the working repo needs to be initialized using git-crypt with the below command.
cd /path/to/secret/sauce/git/repo
git-crypt init
- Create a
.gitattributesfile within the repo root.
touch .gitattributes
🚨 Now this is where things become highly opinionated.
- Open the
.gitattributesfile with preferred text editor.
nvim .gitattributes
- Add the below lines to the
.gitattribuesfile
*.shu filter=git-crypt diff=git-crypt
From my experience the above line should recursively encrypt all files throughout the repo that have an extension with
.shu.
-
Save and close the
.gitattribuesfile. -
Add, move, copy files with
.shuextension to repo -
Verify files are encrypted
git-crypt status | grep -i "shu"
7b. Files can be force encrypted with the below command
git-crypt status -f
The above command will print error messages if there is any issue with staging or committing the encrypted files.
- Stage, commit, push files as required
git add --all
git commit -m "🤫"
git push
⚠️ See gotcha section if and when running into issues.
Gotchas
Staging a whitelisted file in a locked repo state
If a file is created, while git-crypt has a repo in a locked state, and even if the file is whitelisted in the .gitattribues to be encrypted, the file will not be encrypted when staging an "untracked" file in a git repo. From my personal experience I run into the below issue.
To get around such mentioned issue, make certain the repo isn't in a "locked state" when adding a new file that conforms to the .gitattributes file for files being encrypted with git-crypt. So before adding a new .shu file to the repo make certain the repo is in an unlocked state.
git-crypt unlock
Then stage the file to be committed.
git add --all
git commit -m "add some secret sauce 🍔 to the repo
Optional, verify the newly added and committed file has been encrypted
git-crypt status | grep -i "shu"
⚠ The repo still should probably be in a locked state before pushing files to a public repo, so one should lock the repo. The present fish prompt I'm using will display dirty working tree even if all files have been added and committed, that said to alleviate the dirty working tree prompt, lock the repo using git-crypt.
git-crypt lock
Now one can push the encrypted files to a public git repository.
git push
Optional one can verify the files have been encrypted by trying to view the file on the public repo.
Updating white listed encrypted files
The repo will need to be placed back in an unencrypted state in order to modify / update file that has been encrypted using git-crypt because the file locally will be in an encrypted state, which can be verified by opening a encrypted file in a text editor.
To put the repo back in a state where files can be updated
git-crypt unlock
Then, update white listed encrypted files accordingly.
🚨 To the best of my knowledge, the repo can remain in an unlocked state if the .gitattribues file has not be modified, and the updates to the white listed encrypted files can be added, committed, and pushed to a public repo, and should remain encrypted on the repo, while the repo is in unlocked state locally. Thus, my reasoning for adding a .shu extension to files that should be white listed for encryption, ie. only one rule needs to be added to the .gitattribues file.
Homework
- [ ] write a git hook to unencrypt
.shufiles, copy thesecrect.sause.shufile tosecret.sause, ie. without the.shuextension within the same directory and add and verify thesecret.saucefile is in the global.gitignorefile for the repo.
I ran into a similar issue. How it happened was a certain type of file was pushed first, then .gitattributes was updated for that file type (should have been in reverse order). However, after I did "git-crypt lock" the repo and then unlocked it, the file was no longer listed as "*** WARNING: staged/committed version is NOT ENCRYPTED! ***" file by "git crypt status".
@ipatch following your instructions to the letter. Still not working for me. I either get:
$ git-crypt status -f
Error: a9docs-pybackend/.env.integr: still unencrypted even after staging
Unable to stage 1 file.
When its too darn late and it's game over. or....
$ git-crypt status -f; echo $?
0
and does nothing.
@mkmuto i see this same behavior, do you have any updates?
@kingbuzzman
Yeah the workflow is clunky to say the least and could definitely be improved with using some form a git hook to auto encrypt / decrypt secret sauce 🍔 files. That said, I'd create a completely new repo all follow the steps I outlined in a test / experiment repo and see if you're still experiencing the same issues.
Solved for me:
git filter-branch
git-crypt unlock <dir for your key>
git-crypt lock -k <dir for your key>
git-crypt status -e
