karaf icon indicating copy to clipboard operation
karaf copied to clipboard

ssh authentication doesn't work

Open jbonofre opened this issue 4 months ago • 12 comments

Using a fresh Karaf runtime, sshd is starting but the authentication doesn't work:

ssh -p 8101 karaf@localhost
The authenticity of host '[localhost]:8101 ([::1]:8101)' can't be established.
RSA key fingerprint is SHA256:tcNqGFCIAs8KVNOWS1VDJuB4tvyjEwEeAVi3lgvbaUA.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:8101' (RSA) to the list of known hosts.
Password authentication
(karaf@localhost) Password:
Password authentication
(karaf@localhost) Password:
Password authentication
(karaf@localhost) Password:
karaf@localhost's password:

jbonofre avatar Aug 25 '25 21:08 jbonofre

Note: Vanilla karaf v4.4.8 works using the Java-based ssh client as does macOS ssh clinet

Karaf Java-ssh:

% ./bin/client -u karaf -p karaf
Logging in as karaf
        __ __                  ____      
       / //_/____ __________ _/ __/      
      / ,<  / __ `/ ___/ __ `/ /_        
     / /| |/ /_/ / /  / /_/ / __/        
    /_/ |_|\__,_/_/   \__,_/_/         

  Apache Karaf (4.4.8)

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit 'system:shutdown' to shutdown Karaf.
Hit '<ctrl-d>' or type 'logout' to disconnect shell from current session.

karaf@root()>                                                                                                                                                                                                                                           

macOS ssh client

apache-karaf-4.4.8 % ssh -p 8101 karaf@localhost
Password authentication
(karaf@localhost) Password: 
        __ __                  ____      
       / //_/____ __________ _/ __/      
      / ,<  / __ `/ ___/ __ `/ /_        
     / /| |/ /_/ / /  / /_/ / __/        
    /_/ |_|\__,_/_/   \__,_/_/         

  Apache Karaf (4.4.8)

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit 'system:shutdown' to shutdown Karaf.
Hit '<ctrl-d>' or type 'logout' to disconnect shell from current session.

karaf@root()>                                    

mattrpav avatar Aug 28 '25 14:08 mattrpav

With plain apache-karaf-4.4.8.tar.gz from Maven Central:

nite@nitebug : ~$ tar xf apache-karaf-4.4.8.tar.gz 
nite@nitebug : ~$ cd apache-karaf-4.4.8
nite@nitebug : ~/apache-karaf-4.4.8$ vim etc/users.properties 
nite@nitebug : ~/apache-karaf-4.4.8$ ./bin/karaf 
        __ __                  ____      
       / //_/____ __________ _/ __/      
      / ,<  / __ `/ ___/ __ `/ /_        
     / /| |/ /_/ / /  / /_/ / __/        
    /_/ |_|\__,_/_/   \__,_/_/         

  Apache Karaf (4.4.8)

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown Karaf.

karaf@root()>

and in other window:

nite@nitebug : ~$ ssh -V
OpenSSH_9.9p1, OpenSSL 3.2.4 11 Feb 2025
nite@nitebug : ~$ ssh -p 8101 karaf@localhost 
The authenticity of host '[localhost]:8101 ([::1]:8101)' can't be established.
RSA key fingerprint is SHA256:pnNq5HjPo+UQC3qh86zsZZkuugqX8zE2DoVtLRuJr4o.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:8101' (RSA) to the list of known hosts.
Password authentication
(karaf@localhost) Password: 
        __ __                  ____      
       / //_/____ __________ _/ __/      
      / ,<  / __ `/ ___/ __ `/ /_        
     / /| |/ /_/ / /  / /_/ / __/        
    /_/ |_|\__,_/_/   \__,_/_/         

  Apache Karaf (4.4.8)

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit 'system:shutdown' to shutdown Karaf.
Hit '<ctrl-d>' or type 'logout' to disconnect shell from current session.

karaf@root()>

and logs show:

nite@nitebug : ~/apache-karaf-4.4.8$ tail -n 6 data/log/karaf.log 
2025-08-28T17:26:17,470 | INFO  | sshd-SshServer[134e79a6](port=8101)-nio2-thread-1 | OpenSSHKeyPairProvider           | 46 - org.apache.karaf.shell.ssh - 4.4.8 | Creating ssh server private key at /home/nite/apache-karaf-4.4.8/etc/host.key
2025-08-28T17:26:17,472 | INFO  | sshd-SshServer[134e79a6](port=8101)-nio2-thread-1 | OpenSSHKeyPairGenerator          | 46 - org.apache.karaf.shell.ssh - 4.4.8 | generateKeyPair(RSA) generating host key - size=2048
2025-08-28T17:26:19,511 | WARN  | sshd-SshServer[134e79a6](port=8101)-nio2-thread-2 | ServerUserAuthService            | 48 - org.apache.sshd.osgi - 2.15.0 | handleUserAuthRequestMessage(ServerSessionImpl[null@/[0:0:0:0:0:0:0:1]:44964]) Failed (IllegalArgumentException) to authenticate using factory method=publickey: No verifier located for algorithm=ssh-ed25519
2025-08-28T17:26:20,806 | INFO  | sshd-SshServer[134e79a6](port=8101)-nio2-thread-2 | ServerSessionImpl                | 48 - org.apache.sshd.osgi - 2.15.0 | Session karaf@/[0:0:0:0:0:0:0:1]:44964 authenticated
2025-08-28T17:26:44,533 | INFO  | CM Configuration Updater (Update: pid=org.ops4j.pax.logging) | EventAdminConfigurationNotifier  | 6 - org.ops4j.pax.logging.pax-logging-log4j2 - 2.3.0 | Sending Event Admin notification (configuration successful) to org/ops4j/pax/logging/Configuration
2025-08-28T17:26:45,241 | INFO  | CM Configuration Updater (Update: pid=org.ops4j.pax.logging) | EventAdminConfigurationNotifier  | 6 - org.ops4j.pax.logging.pax-logging-log4j2 - 2.3.0 | Sending Event Admin notification (configuration successful) to org/ops4j/pax/logging/Configuration

rovarga avatar Aug 28 '25 15:08 rovarga

The problem is depending of the ssh key on the client. If the client uses "old" key with not supported anymore algorithm, it fails. I think it's acceptable if we log it.

jbonofre avatar Aug 28 '25 15:08 jbonofre

@rovarga Are you using 'ed25519' exclusively? Are you adding the bouncy castle libraries in your install?

EDIT: @rovarga your local .ssh/config may be configured to disable password authentication method, if it is falling back to key-based.

Related:

This change of mine needs to be reverted: https://github.com/apache/karaf/pull/1703/commits/03fdb2de1b150ebcdb9a4bc933a300c44d677402

Related JIRA: ref: https://issues.apache.org/jira/browse/KARAF-7601

Background: The WARN message about those algorithms was being logged, and I removed the support for ed25519 (incorrectly, since my installation did not have the net.i2p.crypto 3rd-party installed at the time). Apache MINA has since removed the need for the 3rd-party library net.i2p.crypto in favor of using bouncy castle.

mattrpav avatar Aug 28 '25 16:08 mattrpav

I don't think we should revert the deprecated algorithms, more documents for users the "constraints".

Thoughts ?

jbonofre avatar Aug 28 '25 16:08 jbonofre

@jbonofre the algorithm is not deprecated. Just the opposite. ed25519 is still considered strong.

The usage of that ed25519 by mina-sshd WITH the old 3rd-party library (net.i2p.crypto) had a weak signature -- this combination is what was problematic. Apache Mina refactored to use bouncy castle now, so we should re-enable the algorithm.

Apache Mina refactor: ref: https://github.com/apache/mina-sshd/issues/767

Timeline: 2022 Oct: I patched karaf to remove ed25519 support (when using net.i2p.crypto) 2024 Jun: Apache MINA refactored to use bouncy castle

2025 Aug: ed25519 still considered strong and recommended

  • https://developer.chrome.com/release-notes/137
  • https://forums.freebsd.org/threads/ssh-encryption-type-in-2025-which-is-better.98206/
  • https://www.brandonchecketts.com/archives/ssh-ed25519-key-best-practices-for-2025

Update:

Note that net.i2p.crypto was a transition library that hasn't had a release since 2018. ref: https://mvnrepository.com/artifact/net.i2p.crypto/eddsa

mattrpav avatar Aug 28 '25 16:08 mattrpav

@mattrpav this was just a quick test, and my first-listed IdentityFilehappens to be an ed25519, so I am quite used to seeing ed25519-related warnings :)

rovarga avatar Aug 28 '25 16:08 rovarga

@mattrpav yes sorry I wasn't clear. I meant the "old" lib supporter is deprecated. That said I think we should just warn users and don't consider as a bug (I was thinking about re-adding algorithms by default but not sure it's actually a good idea).

jbonofre avatar Aug 28 '25 17:08 jbonofre

@jbonofre we should most definitely add ed25519 support back in using the new mina-sshd and bouncy castle as the ed25519 provider.

mattrpav avatar Aug 28 '25 17:08 mattrpav

@mattrpav yeah, let me do that: my first proposal was to add back using the deprecated provider, but you are right, it's better to add bc ed25519 provider.

jbonofre avatar Aug 28 '25 17:08 jbonofre

@jbonofre that sounds good!

@rovarga the ed25519 solution will not address your issue. Please provide a client-side debug log for review. The only error on the server-side indicates it is looking for a public key, not finding a match and closing the session.

I don't see a karaf problem for your issue at this point.

mattrpav avatar Aug 28 '25 17:08 mattrpav

I confirm, nothing in Karaf, more an improvement 😄

jbonofre avatar Sep 29 '25 08:09 jbonofre