Add the ability to import SHA-1 pre-hashed passwords

[veryCrunchy]

zitadel/zitadel#6196

The implementation and test do not properly handle version differences. The test itself is flawed due to not creating a new hash instead using a static hash.

This causes issues when trying to log in after the account was imported with an old password hash on a separate version.

Lets say user tries to log in using $2y, the verifier will first use passwap to extract the cost and salt, not the version Then the verifier creates a new hash using the extracted paramaters, because no version is passed it will create a $2a. Then these two are compared and ultimately fail the check even though the password is valid, causing the user to be unable to log in, and no passwap ends up happening.

_{This issue is self assigned to @verycrunchy}

[Larzous]

The passwords were migrated into the database correctly. Login fails due to verification of the password on login does not work:

Current Password Manager

• PHP - php8.3-fpm
• Hashing - password_hash($password, PASSWORD_DEFAULT)
  • https://www.php.net/manual/en/function.password-hash.php
• Per documentation - PASSWORD_DEFAULT is bcrypt salt 10, generating a $2y$10 password hash.

Zitadel Server

• Zitadel - v4.1.3
• Python-Client SDK - zitadel-client 4.1.0b4

[fforootd]

@muhlemmer I guess you might want to check this

[muhlemmer]

I don't think the version is the problem. The minor version is actually insignificant in Go's implementation. We simply wrap the golang.org/x/crypto/bcrypt implementation and only check the identifier, so we make sure it is bcrypt. Besides that the encoded hash and password are passed as-is to the bcrypt package.

We even have tests that try all minor versions of bcrypt (2a, 2b and 2y), and they work fine.

Perhaps it is an encoding issue, we experienced that with other PHP hashes in the past as well. Can you perhaps share a couple of example hash strings, together with a password that is known to work?

[Larzous]

@muhlemmer -- Here you are, password & associated hash stored in our DB. -- Only the first password here was specifically attempted to migrate, the other 4 are from random test accounts we have:

Pass -- 2DKozCyVxQPtncV2bSWZ Hash -- $2y$10$/cmuqrai7Hy1qe9B2NJs1eW6e7CsQwx4RwsmekbxGuyMcZzWsAJvq Hass Shown when exporting user via /admin/v1/export endpoint confirming it was imported correctly on user creation when using UserServiceHashedPassword() from the python-sdk -- hashedPassword":{"value":"$2y$10$/cmuqrai7Hy1qe9B2NJs1eW6e7CsQwx4RwsmekbxGuyMcZzWsAJvq"}

Pass -- CVP3ckw0ybf3nyk_rqw Hash -- $2y$10$Qit8VSJQEG6C5CiUfa63ZexnNXiYx/zl8SLqxv01fuKD/30s5YRXG

Pass -- aRAqzsaT190afKwspGYs Hash -- $2y$10$vVRa1cRQjFr9MAKh7y87len/AHD6ZOJ34S.jf07ceWaZlQQeH9hhG

Pass -- bgp2KBD0dgb.nrt@kaf Hash -- $2y$10$tS67bZN5HcMLObB0ltOD9Oy/dwxaP9MTirr6iS8oYI7vwr7on6x3e

[Larzous]

@muhlemmer -- Any update on this? I'm running zitadel v4.2.2 -- We have our project release date and if we can get our password imports working so users don't have to run a reset it would be appreciated.

[muhlemmer]

@Larzous I had another look. The problem isn't zitadel nor passwap. The passwords are truly not matching the hashes.

• Calling Go's bcrypt directly: https://go.dev/play/p/rd9NbGDb-Wy?v=goprev
• Same thing goes for Python's passlib: https://python-fiddle.com/saved/f2a6a65f-2dd9-4cb0-91bf-867397062aec
• And PHP 8.2: https://onlinephp.io/c/9bc8e

Perhaps old application does a conversion or encoding of the password string before it is passed to the hasher. For example some people think it's wise to apply "pepper" before feeding the password into bcrypt.

[Larzous]

@muhlemmer Thanks for double checking. I'm not a frontend guy, so I just know where the passwords are stored in the DB and that is does update when I change my password, so those are the passwords in the database.

As per your suspecting the software is doing something, you possibly right. I checked with a College to confirm since i just assumed that's the password so it should work; and he also cannot reproduce the hash.

I've never heard of "peppering" before, so ty for the info. We currently use the following CMS + Addon to allow for user management).

• Redaxo CMS - https://redaxo.org/
• Addon - Ycom - https://github.com/yakamara/ycom

Just to confirm -- If it's applying a "pepper" before its hashed, I will never be able to migrate the passwords correct?

[Larzous]

@muhlemmer @veryCrunchy I can confirm the passwords are being peppered before encryption so we will not be able to migrate them: https://github.com/redaxo/redaxo/blob/7bc8552a43850597d268eb128732e18aaa949c03/redaxo/src/core/lib/login/login.php#L611

This ticket can be closed as resolved.

[muhlemmer]

Oh dear, what they do there is executing a sha-1 pre-hash, not pepper. Well then they did something custom and actually break the 2y identifier.

Perhaps you can patch the application to use pre-hash for verification of current passwords and then rehash with vanilla bcrypt. After all the users are rotated you can import them into zitadel.

Another option is we add some type of custom identifier and execute a pre-hash in passwap. Before import you can string-replace the identifier to the custom one and import should work.

For the latter option, you'll probably need to open a PR yourself, we don't have much time available on our side.

I adjusted the Go play example so it works with the PHP passwords. Off course PHP doesn't do a binary SHA-1, but instead hex-encodes the output of sha1() by default. I adjusted the example accordingly. This verifies all the passwords correctly:

package main

import (
	"crypto/sha1"
	"encoding/hex"
	"fmt"

	"golang.org/x/crypto/bcrypt"
)

var cases = []struct {
	hash     string
	password string
}{
	{
		"$2y$10$/cmuqrai7Hy1qe9B2NJs1eW6e7CsQwx4RwsmekbxGuyMcZzWsAJvq",
		"2DKozCyVxQPtncV2bSWZ",
	},
	{
		"$2y$10$Qit8VSJQEG6C5CiUfa63ZexnNXiYx/zl8SLqxv01fuKD/30s5YRXG",
		"CVP3ckw0ybf3nyk_rqw",
	},
	{
		"$2y$10$vVRa1cRQjFr9MAKh7y87len/AHD6ZOJ34S.jf07ceWaZlQQeH9hhG",
		"aRAqzsaT190afKwspGYs",
	},
	{
		"$2y$10$tS67bZN5HcMLObB0ltOD9Oy/dwxaP9MTirr6iS8oYI7vwr7on6x3e",
		"bgp2KBD0dgb.nrt@kaf",
	},
}

func main() {
	for i, c := range cases {
		sum := sha1.Sum([]byte(c.password))
		sumHex := make([]byte, hex.EncodedLen(len(sum)))
		hex.Encode(sumHex, sum[:])

		err := bcrypt.CompareHashAndPassword([]byte(c.hash), sumHex)
		if err != nil {
			fmt.Printf("case %d: %v\n", i, err)
		} else {
			fmt.Printf("case %d: OK\n", i)
		}
	}
}

My suggested steps to get to a PR:

1. Define and parse a custom identifier. Perhaps a prefix sha1-pre$2y$10....
2. When the prefix is found, hash the password with sha1.Sum and hex encode it according to example above.
3. Make sure you cut the prefix before proceeding, because bcrypt won't accept it.
4. Call bcrypt to verify the password

In any case I would recommend only implementing the Verifier for this, so that the passwords are verified and then re-hashed to something portable inside Zitadel.

[Larzous]

Perhaps you can patch the application to use pre-hash for verification of current passwords and then rehash with vanilla bcrypt. After all the users are rotated you can import them into zitadel. We aren't using passwap, we are moving directly from the CMS to Zitadel. And with 600k users and a migration date of Oct 13th with a go live of Oct 16th, its impossible to rotate user passwords.

I see your code works, and that's awesome, I'm just not sure if its viable to get a PR in, tested, and deployed by Oct 13th.

We are unifying and switching from the two systems we have now over to Zitadel:

1. Redaxo users with the pre-sha bcrypt passwords.
2. KeyCloak users with pbkdf2 passwords. -- These users have migrated without issue.

If we use a flag for the Verifier, it would have to be done in a way that it only does that for bcrypt passwords. -- My current config.conf is this:

Log:
  Level: 'info'

Port: 3128
ExternalPort: 443
ExternalDomain: '<redacted>'
ExternalSecure: true
TLS:
  Enabled: false

Database:
  postgres:
    Host: 'localhost'
    Port: 5432
    Database: zitadel
    User:
      Username: '<redacted>'
      Password: '<redacted>'
      SSL:
        Mode: 'disable'
    Admin:
      Username: 'postgres'
      Password: '<redacted>'
      SSL:
        Mode: 'disable'

Machine:
  Identification:
    PrivateIp:
      Enabled: false
    Hostname:
      Enabled: true

SystemDefaults:
  PasswordHasher:
    Verifiers:
    #   - "argon2"   # verifier for both argon2i and argon2id.
    #   - "bcrypt"
    #   - "md5"      # md5Crypt with salt and password shuffling.
    #   - "md5plain" # md5 digest of a password without salt
    #   - "md5salted" # md5 digest of a salted password
    #   - "phpass"
    #   - "sha2" # crypt(3) SHA-256 and SHA-512
    #   - "scrypt"
       - "pbkdf2"   # verifier for all pbkdf2 hash modes.