Importing a hash longer than 255 characters will fail.

[edmhar]

Migration failed when using passport-local-mongoose due to "password" field length

(Previous title: Migration failed using passport-local-mongoose due to "password" length is short. )

Description

We would like to migrate from passport authentication to FusionAuth. The library that we are using is this passport-local-mongoose which uses pbkdf2-hmac-sha256-512. Apparently the hashed password stored on our database is at least 1000+ characters in length which causes the migration API to fail since the maximum character length for password field is only varchar 255. This issue could be a big potential blocker on us moving forward in using FusionAuth.

The length of the password field we are trying to import is 1024. Can the FusionAuth schema be modified to support longer password hashes?

Additional Details

I appears as though passport-local-mongoose defaults to a 512 byte key length which is 4,096 bits.

https://www.npmjs.com/package/passport-local-mongoose

keylen: specifies the length in byte of the generated key. Default: 512

The FusionAuth default encryptionScheme values of salted-pbkdf2-hmac-sha256 and pbkdf2-hmac-sha256-512 use a 256 bit and 512 bit key respectively.

A custom plugin would need to be written to use a use a 4,096 bit key, and the password column would need to be modified to be of type TEXT to support a larger hash.

For example, to support PBKDF2 using a 4,096 bit key length with a HMAC SHA-256 hash, the hash will be 512 bytes, so the hex encoded value would be 1024 characters long, or 683 characters base64 encoded. All of these sizes and much larger would be ok if we just bump the column size to TEXT.

Community guidelines

All issues filed in this repository must abide by the FusionAuth community guidelines.

[mooreds]

@edmhar thanks for filing this issue. Can you please let us know what the maximum length of the password hash in your system is?

[edmhar]

Hello @mooreds, its 1024 in character length.

[mooreds]

Thanks @edmhar . I modified the issue to clarify the problem. I can't commit to any timeline to make this change, but any progress will be tracked in this issue.

[robotdan]

Apparently the hashed password stored on our database is at least 1000+ characters in length which causes the migration API to fail since the maximum character length for password field is only varchar 255.

I highly doubt they have created a hash that exceeds 1000 characters in length. Their doc says they use PBKDF2 256 w/ a 512 bit key lenfth which means the hash should not exceed 88 characters.

If you want to post an example value perhaps we can assist. Perhaps they are storing a bunch of values all combined in some sort of schema.

[edmhar]

Hi @robotdan, Sure! no worries, it kinda weird also to have a hashed value of more than 255+ characters. I do some digging on how they hashed the password and here's what I found on passport-local-mongoose;

For their hashing they have this function

module.exports = function pbkdf2(password, salt, options, callback) {
  crypto.pbkdf2(password, salt, options.iterations, options.keylen, options.digestAlgorithm, callback);
};

and since we import the package without passing custom properties, here are the default values for the fields required by the function above

  options = options || {};
  options.saltlen = options.saltlen || 32;
  options.iterations = options.iterations || 25000;
  options.keylen = options.keylen || 512;
  options.encoding = options.encoding || 'hex';
  options.digestAlgorithm = options.digestAlgorithm || 'sha256'; // To get a list of supported hashes use crypto.getHashes()

Also I will add a sample value of the hashed password based on the options above:

118063111607fc1a241b93e8eb32fbb0a415a16a6f7e7b79c270e419221dc36ed67d09fdd63061d5839f11e6c457a18166ab3cf60d8756b870b0355e04f85dd511aa4731025cfc0b3b3a2fda71ed34ecc1e0c958d0f0998de0833d223a6e2bd874b66dbaf9bb5adc919825b079fe48f51ed76cd578ca38dd22926bdb4dd051a767f800ddad1433ec0a785c5bf9ca4a6821d2dfa1bb2843b16341a71f147cce74afe7824aa9329fd828ccb2a2f1bf8028f1506af4e84ffe1afa322f8d90f9e10a04e3ed8dd4fd83fbc801234963e498152e4c7064d5576abb5758d3edd33e4ad7406dabfad166c5d2516924db341e71a5a1986fdf8191309568adca01c20b885913c516297299b115081b8c211fc95f52d0f7b6137939758c8963d5c23bf4e35ec87b0e979dfccf7f6ee7815066ac2214cffa0200f19157eefd6f4360e73f07fed772c249113bf63371b07b052dbea367dccf14f760df0dad88755d527749041ffd1b40fb09315d30f5a7d60c6370d1bc6ebc8dcf03d46a57f3045370cf2146fa867a2855f156989055b0bfabde49d7e06c4419f1a03a351bfddd0d08253408fb5ac27caabb8220bced4f180c7b5c8eb1a4fbe414ac6c583752a945dd1e33be6d087e38fa4697264b9866555850001ecb9386d4396cc000cb5182afa0e380248547418370c7181ef9389166c42da0cf6a115267c9eadc8f5394f4479562b85b2e

Generated it using this helpful online tool: https://neurotechnics.com/tools/pbkdf2-test

[robotdan]

The difference in output length from this is example is due to the key length unit being in bytes instead of bits.

In your example you have set the keylen = 512 so this is 4,096 bytes. This would cause the hash to be a 1024 length hex encoded character string as you're seeing.

The two PBKDF2 hashing algorithms supported out of the box by FusionAuth are PBKDF2 HMAC SHA256 w/ a 256 bit key, or a 512 bit key. These are identified by using the salted-pbkdf2-hmac-sha256 or salted-pbkdf2-hmac-sha256-512 values for the encryptionScheme property during import.

It is possible you are using a 4,096 bit key in your current passport-local-mongoose configuration. If that is the case, then there would be a problem with the current maximum size of the db column which is capped at 255 characters.

We could explore changing this data type to be much larger, but this would require a db schema change. If we made this change, you would also need to write a custom password hashing plugin to utilize use your current hashes as our default options don't account for the use of a 4,096 bit key length.

[robotdan]

Internal notes:

We should probably change the data type of this column, we'll want to do a bunch of migration test to see how long it will take to make this schema change when there are 1-10M users in that table.