Version 2.2.0 is not compatible with MacOS Ventura's `/usr/bin/openssl`

[jmurty]

In release 2.2.0 we included a fix for #133 to work around a compatibility problem with OpenSSL versions 3+, which changed how salt values are included in encrypted output (they aren't)

Unfortunately our work-around relied on a check of the openssl version's major version: if the major version is 3 or greater it forcibly includes the salt values in the encrypted output.

The recent release of MacOS Ventura includes the LibreSSL project's openssl command which also reports a major version of 3, but our work around is not necessary for the LibreSSL project's openssl because that project didn't make the breaking change to remove salt values from encrypted output.

The upshot of all this is transcrypt release 2.2.0 is currently broken on MacOS Ventura when using the default built-in /usr/bin/openssl binary, because our work-around will end up prefixing the salt value unnecessarily.

[jmurty]

This should be fixed on the main branch as of commit ca6a4675fbd8693c0f214382a5651a405a1c6843

[stherold]

Same issue here. We installed the main branch instead which contains the merged branch.

[alb777]

Still failing even doing the workaround above.

[jmurty]

Hi @alb777 can you describe how the latest main branch version of Transcrypt is failing for you?

And are you sure you have installed/upgraded to the in-progress version? E.g:

repo$ ./.git/crypt/transcrypt --version
transcrypt 2.3.0-pre

I'd like to release an official 2.2.1 version soon with the MacOS 13 Ventura fix, but I'd also like to be sure the latest version on main actually fixes the problem(s).

[alb777]

Hi @jmurty

Sure, follows my current transcrypt state that fails during decrypt/git push to stash:

transcrypt 2.3.0-pre ca6a4675fbd8693c0f214382a5651a405a1c6843
LibreSSL 3.3.6

My workaround to fix were change LibreSSL 3.3.6 to OpenSSL 1.1.1s on git default configurations forcing transcrypt to work with the version above instead of the default LibreSSL that comes with Ventura, just added the following line to my local .git config file:

[transcrypt] openssl-path = /usr/local/Cellar/[email protected]/1.1.1s/bin/openssl

Guess that shouldn't be the definitive solution since default packaging combination may cause the same issue.

[ronangaillard]

Same problem here on Ventura :(

I tried the version on main branch andI have the same issue. I tried the solution by @alb777 but even if file is decrypted I have weird characters added to the file.

Makes team work nearly impossible.

Let me know if you need more info or help.

[jmurty]

Hi @ronangaillard sorry you're having trouble.

I've done some more testing of the latest version from main with encrypted files between MacOS 13 Ventura (openssl version : LibreSSL 3.3.6) and Ubuntu (openssl version : OpenSSL 1.1.1f 31 Mar 2020) but I haven't been able to reproduce any problems.

I'd appreciate your help in testing this and pinning down the issue.

Can you try each of the following things and let me know for each whether or not it worked properly?

• Do a full --uninstall then re-install the latest version of transcrypt on main
• Configure the main transcrypt to use an OpenSSL-provided version of openssl with a command like transcrypt --set-openssl-path=PATH_TO_OPENSSL (as recommended by @alb777)
• Do a full --uninstall then downgrade to transcrypt version 2.1.0, which was prior to the OpenSSL v3 special-handling work-around

Finding out which of these steps helps, if any, might help narrow down the issue.

FWIW I am now running and testing against Ventura version 13.0.1 (22A400) though I don't think the included openssl command version has changed since the initial 13 MacOS release.

[Minishlink]

FYI, to downgrade brew installation of transcrypt, use the following:

brew tap-new $USER/local-transcrypt
brew extract --version=2.1.0 transcrypt $USER/local-transcrypt
brew uninstall transcrypt
brew install [email protected]

Make sure to have a clean repo without transcrypt v2.2.0 configured, otherwise uninstall transcrypt configuration with transcrypt -u beforehand.

[joaooliveiramc]

Hello,

Me and my coworker are having the exact same issue. I'm using Ventura and he is using an older version of MacOS. When I push things into the repo and he pulls he has weird characters at the start. We tried different approaches. Including updating to the latest 2.2.0 transcript, updating and defining the default openssl to 1.1.1s or LibreSSL.

When doing openssl version it gives OpenSSL 1.1.1s 1 Nov 2022 doing Which openssl returns /opt/homebrew/opt/[email protected]/bin/openssl

Although the only way to make it work is doing @alb777 said and manually setting the git config file to /opt/homebrew/opt/[email protected]/bin/openssl

[jmurty]

Thanks for the feedback @joaooliveiramc I have been having trouble isolating the exact problem so this info is useful.

Did you try using the version of Transcrypt on the main branch (commit https://github.com/elasticdog/transcrypt/commit/ca6a4675fbd8693c0f214382a5651a405a1c6843)?

I ask because that branch has a change which I think fixes compatibility with the LibreSSL 3.3.6 openssl binary on MacOS Ventura but I haven't released it officially yet because the fix doesn't seem to work for everyone. If it does work for you, you wouldn't need to set the path to the Homebrew OpenSSL binary.

By the way, the reason you need to override the path to the openssl binary using transcrypt --set-openssl-path=PATH_TO_OPENSSL, or setting it in the git config yourself, is because Transcrypt may not inherit your PATH environment variable when run in the context of Git's clean & smudge operations.

[joaooliveiramc]

Hey,

So I just installed the main version by cloning and following the commands stated there on my MacOS Ventura. I also set the OpenSSL to LibreSSL and kept the git config openssl-path = openssl.

After cloning our work and configuring it with transcript now I'm having all sort of a weird string looking like a random generated password, which before was not happening.

[jmurty]

Ugh, so it's definitely not working for you. Thanks for trying, and sorry about that.

I will need to get to the bottom of this and find a proper fix. In the meantime your best options are probably to go back to using the Homebrew openssl path, or downgrading from 2.2.0

[jmurty]

Hi, could those of you experiencing incompatibility with MacOS Venture please help me to isolate the ongoing bug by running some tests on your Mac, ideally on both MacOS 13 (Ventura) and prior versions like 12?

I have pushed a new version of the transcrypt script in bea18476 with nothing but the released version 2.2.0 plus my prior attempted fix for MacOS / LibreSSL compatibility.

I'd very much appreciate feedback on your results of running some low-level encrypt and decrypt commands to test what results you get using this version of the script with the different versions of OpenSSL and LibreSSL you have on your system.

To get ready to run the tests:

1. Create and cd into a new directory for testing, separate from any existing code directory, e.g: mkdir test-transcypt && cd test-transcypt
2. Download and save this transcrypt version to test: wget https://raw.githubusercontent.com/elasticdog/transcrypt/bea184767608d0c24d20dce01381d3ea3ff50a62/transcrypt
3. Make the transcrypt script executable: chmod u+x transcrypt
4. Initialise git in the test directory: git init
5. Set the cipher to use: git config --local transcrypt.cipher aes-256-cbc
6. Set a fake password, use this exact command: git config --local transcrypt.password abc123
7. Create this plaintext file to test against, use this exact command: echo "Plaintext for testing" > test-secret-file

Once the testing directory is set up, run these commands to test encrypting and decrypting the example file:

1. Configure transcrypt to use your system default openssl: git config --local transcrypt.openssl-path /usr/bin/openssl

• Check the version reported by that openssl path for cross-checking, e.g: /usr/bin/openssl version

3. Output the encrypted version of the plaintext file: cat test-secret-file | ./transcrypt clean test-secret-file

• The output should be exactly U2FsdGVkX18HgDaYdwMdMtRxglvIjDqKGpAP66vzqy4QHyqEzouDb0YweqfVm5+/
• You can ignore output lines like *** WARNING : deprecated key derivation used. and Using -iter or -pbkdf2 would be better.

5. Do a roundtrip encrypt-then-decrypt: cat test-secret-file | ./transcrypt clean test-secret-file | ./transcrypt smudge

• The output should be exactly Plaintext for testing

Repeat the above commands, but substitute the path to openssl for step 1. to use different versions you have from the OpenSSL or LibreSSL projects, for example:

• OpenSSL version 1.1 installed with Homebrew: git config --local transcrypt.openssl-path /opt/homebrew/opt/[email protected]/bin/openssl
• OpenSSL version 3 installed with Homebrew: git config --local transcrypt.openssl-path /opt/homebrew/opt/openssl@3/bin/openssl
• The default openssl binary used on your system (no explicit path): git config --local transcrypt.openssl-path openssl

If the script is working correctly you should get exactly the same outputs each time for the 2. encrypt (clean) command and the 3. encrypt-then-decrypt (clean and smudge) commands.

If you see anything different, such as a different encrypted text blob or garbage characters after the round-trip, that could give me a vital clue to figure out what problems remain. Any anomalies you can report would be much appreciated, I need the help to figure this out!

For what it's worth, when I run the test commands on my system (Ventura 13.1) I get the same results for all versions of OpenSSL and LibreSSL openssl paths. So it "works on my machine" which is making it very difficult to debug.

[jmurty]

For advanced testing, you can run the low-level encrypt (clean) command in an existing repository with decrypted files, to check whether the encrypted output exactly matches the encrypted file stored in git.

If there is any difference between the encrypted data stored in git versus the output of your test clean command, this will cause git to report phantom changes.

As an example, if you're in a repo with transcrypt installed (2.1.0 or later) and your file my-secrets is already decrypted you can generate the encrypted version with:

cat my-secrets | .git/crypt/transcrypt clean my-secrets 2> /dev/null

# NOTE: If you're using transcrypt from the `main` branch you will need to run this instead
cat my-secrets | .git/crypt/transcrypt clean context=default my-secrets 2> /dev/null

The encrypted version generated by this command should exactly match the encrypted data stored in git, which you can print with:

.git/crypt/transcrypt --show-raw=my-secrets 2> /dev/null

[Fausto95]

Hi @jmurty, thanks to the above instructions I was able to fix the issue. All of them worked for me.

[Z7oussem]

Hello @jmurty, thanks a lot you saved my day :)

[emcniece]

Using these two packages on OSX 13.1:

$ openssl version
LibreSSL 3.3.6

$ /opt/homebrew/opt/[email protected]/bin/openssl version
OpenSSL 1.1.1t  7 Feb 2023

Test steps execute exactly as described: encrypted and decrypted content match the examples. Thank you!!

[jmurty]

Thanks very much for your feedback @Fausto95 @Z7oussem @emcniece, it gives me more confidence I've finally fixed the bug using the LibreSSL 3+ flavour of the openssl binary.

I have released a new version 2.2.1 of transcrypt with a fix for the original bug.

I also added a work-around which should properly decrypt files committed on systems that were affected by the bug, so these should now give you the original content instead of showing binary junk at the top of the file. See details in b380f2c9 Note that incorrectly encrypted files will remain in your repository even with this work-around – manifesting as phantom changed files that look no different in diffs – but if you commit the seemingly unchanged file transcrypt will fix the encrypted version stored in Git.

Please grab the 2.2.1 version of transcrypt, make sure it's updated on all systems by running transcrypt --upgrade within every repository, and check whether it finally resolves this problem.

[joaooliveiramc]

How do I catch the 2.2.1? doing upgrade is only trying to upgrade to 2.2.0

[jmurty]

Hi @joaooliveiramc it may take a while for the latest version to be made available by package managers. In the meantime, transcrypt itself is just a bash shell script so you can download version 2.2.1 and run it directly from within your repository like so:

# Download transcrypt script at version/tag 2.2.1
wget https://raw.githubusercontent.com/elasticdog/transcrypt/v2.2.1/transcrypt

# Make it executable
chmod u+x transcrypt

# Confirm the version, and upgrade it in the repository
./transcrypt --version
./transcrypt --upgrade

FWIW, I like to keep a copy of the transcrypt script in shared repositories that use it, at a path like bin/transcrypt. That way anyone using the repo can always run the latest version without relying on package managers to be up-to-date. It's easier to tell colleagues to run ./bin/transcrypt --upgrade in the repo, than to make sure they have the latest version available on their system.