JWT.decode broken in v2.4: `unpack1': invalid base64 (ArgumentError)

[collimarco]

The update from v2.3 to v2.4 of this gem breaks our gem.

This is the error:

<internal:pack>:281:in `unpack1': invalid base64 (ArgumentError)
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/base64.rb:74:in `strict_decode64'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/base64.rb:106:in `urlsafe_decode64'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/jwt-2.4.0/lib/jwt/decode.rb:120:in `decode_crypto'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/jwt-2.4.0/lib/jwt/decode.rb:27:in `decode_segments'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/jwt-2.4.0/lib/jwt.rb:28:in `decode'
	from /Users/collimarco/Sites/cuber-gem/lib/cuber/cuberfile_validator.rb:132:in `validate_key'
	from /Users/collimarco/Sites/cuber-gem/lib/cuber/cuberfile_validator.rb:26:in `validate'
	from /Users/collimarco/Sites/cuber-gem/lib/cuber/cli.rb:43:in `validate_cuberfile'
	from /Users/collimarco/Sites/cuber-gem/lib/cuber/cli.rb:20:in `initialize'
	from /Users/collimarco/Sites/cuber-gem/bin/cuber:3:in `new'
	from /Users/collimarco/Sites/cuber-gem/bin/cuber:3:in `<top (required)>'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/bin/cuber:25:in `load'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/bin/cuber:25:in `<top (required)>'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/cli/exec.rb:58:in `load'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/cli/exec.rb:58:in `kernel_load'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/cli/exec.rb:23:in `run'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/cli.rb:479:in `exec'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/cli.rb:31:in `dispatch'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/cli.rb:25:in `start'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/bundler-2.2.33/libexec/bundle:49:in `block in <top (required)>'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/3.0.0/bundler/friendly_errors.rb:103:in `with_friendly_errors'
	from /Users/collimarco/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/bundler-2.2.33/libexec/bundle:37:in `<top (required)>'
	from /Users/collimarco/.rbenv/versions/3.0.4/bin/bundle:23:in `load'
	from /Users/collimarco/.rbenv/versions/3.0.4/bin/bundle:23:in `<main>'

Here's the source code that raise the error: https://github.com/cuber-cloud/cuber-gem/blob/master/lib/cuber/cuberfile_validator.rb#L132

This seems a bug in the latest version of jwt.

[excpt]

Can you check out the current master branch and test again, please?

PR #484 applies a fix.

[collimarco]

@excpt Our token is valid, it's a license key made with a jwt token and it was working in v2.3. The same jwt token should be also valid in v2.4. That PR seems to only catch errors for invalid tokens, but our token is valid.

[anakinj]

I have a feeling that the more strict base64 decoding is an issue. I'll take a look at this.

[excpt]

I have a feeling that the more strict base64 decoding is an issue. I'll take a look at this.

I think the same.

@collimarco Do you use base64 with or without padding?

[collimarco]

Do you use base64 with or without padding?

This is how we generate the token initially:

token = JWT.encode payload, ecdsa_key, 'ES256'
puts token

[excpt]

Thanks for the fast response.

[excpt]

@collimarco Please test again with the master branch. @anakinj Brought back the old base64 implementation.

[anakinj]

Still a little puzzled what kind of tokens the ::Base64.urlsafe_decode64 is not able to process. Just ran this simple test between 2.3 and 2.4 and it worked pretty nicely. Was thinking of some compatibility tests between versions, but i'm not able to produce a token that will fail in the same way.

Is the payload (without the signature) of the token something secret or are you able to share it @collimarco?

This was the test with what I tried to simulate your situation:

git checkout v2.3.0
bin/console.rb

key = OpenSSL::PKey.read(File.read('spec/fixtures/certs/ec256-private.pem'))
token = JWT.encode({pay: 'load'}, key, 'ES256')
File.open('/tmp/test.token', 'w') { |file| file.write(token) }

git checkout v2.4.1
bin/console.rb

key = OpenSSL::PKey.read(File.read('spec/fixtures/certs/ec256-public.pem'))
puts JWT.decode(File.read('/tmp/test.token'), key, true, algorithm: 'ES256')

[anakinj]

Could the reason for this particular issue be a newline char in the end of the token that is passed to the encode method?

How the non-alphabetic chars are handled seems to be the a significant difference between RFC 2045 and RFC 4648.
(https://datatracker.ietf.org/doc/html/rfc4648#section-3.3)

[anakinj]

Im curious if this is such a big issue anyway. Im tempted to bring back the built-in decoding methods with some custom handling like stripping the payload on both ends or something

[anakinj]

Going to close this as the fix is released. Still guessing the reason for this is some extra chars in the token that are just ignored by the base64 decoding.