MeshCentral icon indicating copy to clipboard operation
MeshCentral copied to clipboard

Invalid PKCS/Bad Signature cause?

Open kevinh-csalabs opened this issue 5 years ago • 33 comments

I am seeing "Invalid PKCS signature" and "Bad Signature" in the server info, but don't see any documentation on what would cause this or what needs to be done to resolve it. Any suggestions?

kevinh-csalabs avatar Jan 21 '21 16:01 kevinh-csalabs

Same here... After latest update to 0.7.48

vitko-bg avatar Jan 24 '21 21:01 vitko-bg

@kevinh-csalabs I am using currently this line in config.json to get agents connected and will try to update all agents, will see what will be the result "ignoreagenthashcheck": [ "86.134.X.X", "86.128.X.X", "86.8.X.X" ],

To get the IPs - run systemctl status meshcentral - it will show you which IPs are problematic.

vitko-bg avatar Jan 25 '21 10:01 vitko-bg

By crazy chance, if you are using MongoDB, what version are you using? Also, try updating to v0.7.55 and let me know if it fixes it. I had MongoDB performance improvements starting in v0.7.46 and just pulled them all out in v0.7.55. I would love to know if it makes a difference.

As for documentation of this issue, I had never planned to see this error except if a hacker is purposefully trying to connect and inject junk into the websocket. In that case, the agent authentication signature would fail. This is certainly not something that should be happening.

Ylianst avatar Jan 27 '21 23:01 Ylianst

By crazy chance, if you are using MongoDB, what version are you using? Also, try updating to v0.7.55 and let me know if it fixes it. I had MongoDB performance improvements starting in v0.7.46 and just pulled them all out in v0.7.55. I would love to know if it makes a difference.

As for documentation of this issue, I had never planned to see this error except if a hacker is purposefully trying to connect and inject junk into the websocket. In that case, the agent authentication signature would fail. This is certainly not something that should be happening.

@Ylianst I agree. Man in the middle attacks could produce this as well as certain firewalls or compromised firewalls.

LPJon avatar Jan 27 '21 23:01 LPJon

Interesting. I run everything on the same port, could it be port scanners or other tools trying to break in?

are there settings so I can log this and figure out what is happening?

Allowee avatar Jan 28 '21 09:01 Allowee

I am running mongo, but still seeing entries after 0.7.55 upgrade:

Invalid PKCS signature: 9

Could this be due to running behind cloudflare?

kevinh-csalabs avatar Jan 28 '21 15:01 kevinh-csalabs

I am running mongo, but still seeing entries after 0.7.55 upgrade:

Invalid PKCS signature: 9

Could this be due to running behind cloudflare?

It's possible. I was originally using Lets Encrypt with nginx (reverse proxy) but switched to cloudflare certs with nginx and started noticing the errors.

black3dynamite avatar Jan 29 '21 01:01 black3dynamite

I’m using cloudflare but still using lets encrypt on the backend (confirmed cert are working and renewing as expected before turning on cloudflare)


From: black3dynamite [email protected] Sent: Thursday, January 28, 2021 8:45:43 PM To: Ylianst/MeshCentral [email protected] Cc: Kevin Hart [email protected]; Mention [email protected] Subject: Re: [Ylianst/MeshCentral] Invalid PKCS/Bad Signature cause? (#2197)

I am running mongo, but still seeing entries after 0.7.55 upgrade:

Invalid PKCS signature: 9

Could this be due to running behind cloudflare?

It's possible. I was originally using Lets Encrypt with nginx (reverse proxy) but switched to cloudflare certs with nginx and started noticing the errors.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Ylianst/MeshCentral/issues/2197#issuecomment-769516635, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARHBLPMSWAXZVOUDTAZULTDS4IHMPANCNFSM4WNDKXYA.

kevinh-csalabs avatar Jan 29 '21 13:01 kevinh-csalabs

I have the same problem on the latest version 1.1.22

Is it a bug or a bug in my configuration . ? (I am using meshcentral behind reverse proxy nginx)

Smiley-k avatar Mar 29 '24 21:03 Smiley-k

I solved the problem by changing mesh.doman.com to remote.domain.com and the certificate problem disappeared...

Smiley-k avatar Apr 01 '24 09:04 Smiley-k

Nope, didn't end up resolving the issue @Ylianst We need your help...

Smiley-k avatar Apr 19 '24 11:04 Smiley-k

@Smiley-k what message are you actually seeing? as the messages all mean different things.

si458 avatar Apr 19 '24 12:04 si458

attached the image below

image

Smiley-k avatar Apr 19 '24 12:04 Smiley-k

part of the code is below but basically its taking the signature an agent has given it, and verifying it against its certificate. you can read about it here https://ylianst.github.io/MeshCentral/design/#agent-to-server-handshake

const p7 = forge.pkcs7.messageFromAsn1(msgDer);
const sig = p7.rawCapture.signature;
// Verify with key hash
var buf = Buffer.from(obj.agentSeenCerthash + obj.nonce + obj.agentnonce, 'binary');
var verifier = parent.crypto.createVerify('RSA-SHA384');
verifier.update(buf);
verified = verifier.verify(obj.unauth.nodeCertPem, sig, 'binary');
if (verified !== true) {
  // Not a valid signature
  parent.agentStats.invalidPkcsSignatureCount++;
  parent.setAgentIssue(obj, "invalidPkcsSignature");
  return false;
}

si458 avatar Apr 19 '24 12:04 si458

Dear @si458 I didn't quite understand what I need to do?

Smiley-k avatar Apr 19 '24 14:04 Smiley-k

@Smiley-k nothing. Basically meshcentral is just telling u remote devices are trying to connect BUT something is wrong with the agents, so u need to uninstall the meshagent on the remote devices, redownload the agents and then reinstall

si458 avatar Apr 19 '24 14:04 si458

this is a problem to do on 2000 devices) but I have the same situation on another installation where there are 10 PCs, ... how to understand why this is so...

is there any functionality in the mesh itself to force agents to upgrade to the same version by simply reinstalling?

Smiley-k avatar Apr 19 '24 14:04 Smiley-k

@Smiley-k the only thing u COULD try (not 100% sure if it will work but case of trying it). Set ignoreagenthashcheck: true to allow all ur devices to connect. Then go into a device then the console tab and run agentupdate. This should force download a copy of the agent and then restart it! Then once that machine has restarted, try setting ignoreagenthashcheck: false and see if the device is online?

si458 avatar Apr 19 '24 15:04 si458

@Smiley-k but it could also be a problem with ur reverse proxy/meshcentral setup if u use this, as the webserver ssl cert is different the meshagent ssl cert. So without knowing a little more about ur setup/config.json it's hard to diagnose the problem

si458 avatar Apr 19 '24 15:04 si458

@Smiley-k but it could also be a problem with ur reverse proxy/meshcentral setup if u use this, as the webserver ssl cert is different the meshagent ssl cert. So without knowing a little more about ur setup/config.json it's hard to diagnose the problem

Yes, you are right to point out , I use reverse proxy nginx

Smiley-k avatar Apr 19 '24 17:04 Smiley-k

@Smiley-k so it could just be a config issue with nginx or ur meshcentral, again, plz share ur config.json and ur nginx config and th3 community can belp

si458 avatar Apr 19 '24 17:04 si458

@si458 I'm sorry for the slight delay, I attached the configuration files mesh.json nginx-domain.conf.txt

Smiley-k avatar Apr 20 '24 09:04 Smiley-k

@Smiley-k

your tlsoffload should be the IP address of your reverse nginx server, not the website url 192.168.1.123

agentidletimeout shouldn't really be set as its default is 150 seconds anyway before it disconnects remote devices as they didn't send any web socket traffic

Only 2 issues I can find from a quick read, try those, restart meshcentral and see if it helps

si458 avatar Apr 20 '24 09:04 si458

yes tlsoffload set IP address nginx server

agentidletimeout - i disable

but alas, it did not help

Smiley-k avatar Apr 20 '24 09:04 Smiley-k

Try the following,

Backup then delete the agents, signedagents folders from inside meshcentral-data Restart meshcentral and Make sure it says its resigning the exes. then completely uninstall the agent from a device, then redownload the agent from the web ui on the remote device, Then reinstall

It might be corrupt exes that have bee signed?

si458 avatar Apr 20 '24 10:04 si458

Currently having an "Invalid PKCS signature" and a "bad signature" along with it. Is there a way to tell which machine has these errors?

I'm aware of agentstats and agentissues, but they don't help me determine which machine is having the errors. At least not with my knowledge. Maybe it is not possible in the first place?

I'm going to assume simply reinstalling the Agent will fix it once the machine having the issue is found.

Xevichr avatar Jun 26 '24 08:06 Xevichr

sadly the code only simply logs the remote ipaddress and port of the connection that had the Invalid PKCS signature we would need to add a method into the code to store this extra info

si458 avatar Jun 26 '24 09:06 si458

When running agentissues I have the following result: 6/26/2024, 4:56:00 AM, , invalidPkcsSignature 6/26/2024, 4:56:00 AM, , BadSignature2 6/26/2024, 8:17:10 AM, , invalidPkcsSignature 6/26/2024, 8:17:10 AM, , BadSignature2

So this information is the only thing that will be send to MeshCentral, or is there a way I can find in MeshCentral, which IP had this issue? Because we are currently running MeshCentral LAN only, IP address would be more than enough to find the machine having the issue.

Besides that, considering I haven't even found the machines that have the issue yet, what are the consquences? I'm assuming simply no MeshAgent to MeshCentral connection?

Xevichr avatar Jun 26 '24 09:06 Xevichr

@Xevichr yes this has been notied in the past ive spotted with a few issue, i have never been able to find out WHY it does this?

its meant to insert the ip address, similar to my production system image

one thing you can try is run meshcentral in debug mode with agent set, then watch its output and see if it inserts the ip/port into the logs node node_modules/meshcentral --debug agent

as the code shows its meant to do this, then after insert the badsignature2 into the db for the agentissues

parent.setAgentIssue(obj, "BadSignature2");
parent.parent.debug('agent', 'Agent connected with bad signature, holding connection (' + obj.remoteaddrport + ').');

si458 avatar Jun 26 '24 09:06 si458

With "duplicateAgent" agentissues, it will show the IP address: 6/27/2024, 9:44:00 AM, 000.000.000.000:55632, duplicateAgent. It is pretty much 100% accurate when the IP is sent with this as note.

I ran node node_modules/meshcentral --debug <agentMachineName> in PowerShell (MeshCentral on Windows Server). I am wondering if I did it correctly, because as a test I abruptly disconnected myself. In the MeshCentral console, on the webportal I can see the duplicateAgent message displaying for my laptop, my IP address, but I don't see it in the terminal where I ran the debugging.

PS C:\Program Files\Open Source\MeshCentral> node node_modules/meshcentral --debug <agentMachineName>
MeshCentral HTTP redirection server running on port 81.
MeshCentral v1.1.24, Hybrid (LAN + WAN) mode.
ERR: ERROR: MeshCentral Intel(R) AMT server port 4433 is not available. Check if the MeshCentral is already running.
MeshCentral HTTPS server running on mesh.___.___:444.

Xevichr avatar Jun 27 '24 07:06 Xevichr