sliver icon indicating copy to clipboard operation
sliver copied to clipboard
verified publisher icon BishopFox

Beacons and sessions suddenly stopped working properly

Open su20000101 opened this issue 4 months ago • 7 comments

Describe the bug Sliver has been running stably since last year. Until August 19th, after I executed the rm command on one of the beacons to delete a file, all the beacons suddenly lost contact. At first I thought there was a problem with the machine where the beacon was located. Until today, I generated a new beacon, but I couldn't see any information in the console anyway. I used Wireshak to see the beacon's request to connect to the server (proving that there was no problem with the beacon, at least it requested the server). I generated the session implant again, with the same result. Finally, I generated the session implant using --debug mode, which showed that the server rejected all connections. But I made sure that the server had port 443 open, and I could see in the jobs on the server that mtls was listening on port 443.

To Reproduce Steps to reproduce the behavior:

  1. Execute the rm command on the beacon
  2. Beacons all go offline at the same time
  3. Newly generated beacon implants and session implants are no longer online.
  4. I checked the server and found that the disk was full, so I cleared several log files in the .sliver/logs/ directory.
  5. Through debugging, I found that the session implant reported an error and the server refused the connection.
  6. Restarting the server multiple times, listening to 443, generating new beacon implants and session implants did not work.

Screenshots Use 2.2.2.2 instead of IP

generate --skip-symbols --mtls 2.2.2.2:443 --wg 2.2.2.2:5353 --debug --format exe --name test [] Generated unique ip for wg peer tun interface: 100.64.0.18 [] Generating new windows/amd64 implant binary [] Build completed in 43s [] Implant saved to /root/test.exe

C:\Users\Administrator\Desktop>test.exe 2025/08/22 17:38:15 sliver.go:97: Hello my name is test 2025/08/22 17:38:15 limits.go:58: Limit checks completed 2025/08/22 17:38:15 sliver.go:115: Running in session mode 2025/08/22 17:38:15 session.go:78: Starting interactive session connection loop ... 2025/08/22 17:38:15 transports.go:41: Starting c2 url generator () ... 2025/08/22 17:38:15 transports.go:108: Return generator: (chan *url.URL)(0xc0001d3500) 2025/08/22 17:38:15 transports.go:96: Yield c2 uri = 'mtls://2.2.2.2:443' 2025/08/22 17:38:15 transports.go:96: Yield c2 uri = 'wg://2.2.2.2:5353' 2025/08/22 17:38:15 session.go:95: Next CC = mtls://2.2.2.2:443 2025/08/22 17:38:15 session.go:95: Next CC = wg://2.2.2.2:5353 2025/08/22 17:38:15 session.go:193: Connecting -> 2.2.2.2:443 2025/08/22 17:38:15 transports.go:96: Yield c2 uri = 'mtls://2.2.2.2:443' 2025/08/22 17:38:15 sliver.go:296: Host Uuid: xxx-xxx-xxx-xxx-xxx 2025/08/22 17:38:15 tun-handlers.go:45: [tunnel] Tunnel handlers map[20:0x175b7c0 22:0x17591e0 23:0x17583c0 80:0x1759d80 82:0x175d3c0] 2025/08/22 17:38:15 mtls.go:138: Socket error (read msg-length): read tcp 192.168.73.128:15478->2.2.2.2:443: wsarecv: An existing connection was forcibly closed by the remote host. 2025/08/22 17:38:15 session.go:176: [mtls] lost connection, cleanup... 2025/08/22 17:38:15 session.go:185: [mtls] Stop() 2025/08/22 17:38:15 sliver.go:138: Reconnect sleep: 1m0s 2025/08/22 17:39:15 session.go:294: Connecting -> 2.2.2.2:5353 2025/08/22 17:39:15 session.go:95: Next CC = mtls://2.2.2.2:443 2025/08/22 17:39:15 transports.go:96: Yield c2 uri = 'wg://2.2.2.2:5353' 2025/08/22 17:39:15 wireguard.go:268: Configuring wg device with: private_key=xxx public_key=xxx endpoint=2.2.2.2:5353 allowed_ip=0.0.0.0/0 DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: handshake worker 4 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: encryption worker 7 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: encryption worker 5 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: decryption worker 5 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: handshake worker 5 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: encryption worker 6 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: decryption worker 6 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: handshake worker 6 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: encryption worker 8 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: handshake worker 8 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: decryption worker 8 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: decryption worker 2 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: encryption worker 1 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: decryption worker 1 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: handshake worker 1 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: encryption worker 2 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: TUN reader - started DEBUG: [c2/wg] 2025/08/22 17:39:15 UAPI: Updating private key DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: decryption worker 7 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: decryption worker 3 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: handshake worker 3 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: encryption worker 4 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: decryption worker 4 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: event worker - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Interface up requested DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: handshake worker 2 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: encryption worker 3 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: handshake worker 7 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 peer(Ysvj…+vBs) - UAPI: Created DEBUG: [c2/wg] 2025/08/22 17:39:15 UDP bind has been updated DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: receive incoming v6 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Routine: receive incoming v4 - started DEBUG: [c2/wg] 2025/08/22 17:39:15 peer(Ysvj…+vBs) - UAPI: Updating endpoint DEBUG: [c2/wg] 2025/08/22 17:39:15 peer(Ysvj…+vBs) - UAPI: Adding allowedip DEBUG: [c2/wg] 2025/08/22 17:39:15 peer(Ysvj…+vBs) - Starting 2025/08/22 17:39:15 wireguard.go:278: Successfully set wg device config DEBUG: [c2/wg] 2025/08/22 17:39:15 peer(Ysvj…+vBs) - Routine: sequential sender - started DEBUG: [c2/wg] 2025/08/22 17:39:15 peer(Ysvj…+vBs) - Routine: sequential receiver - started DEBUG: [c2/wg] 2025/08/22 17:39:15 Interface state was Down, requested Up, now Up 2025/08/22 17:39:15 wireguard.go:178: Initial wg connection. Attempting to connect to wg key exchange listener DEBUG: [c2/wg] 2025/08/22 17:39:15 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:39:20 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 2) DEBUG: [c2/wg] 2025/08/22 17:39:20 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:39:26 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 2) DEBUG: [c2/wg] 2025/08/22 17:39:26 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:39:31 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 2) DEBUG: [c2/wg] 2025/08/22 17:39:31 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:39:36 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 3) DEBUG: [c2/wg] 2025/08/22 17:39:36 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:39:41 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 4) DEBUG: [c2/wg] 2025/08/22 17:39:41 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:39:46 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:39:51 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 2) DEBUG: [c2/wg] 2025/08/22 17:39:51 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:39:57 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 3) DEBUG: [c2/wg] 2025/08/22 17:39:57 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:02 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 4) DEBUG: [c2/wg] 2025/08/22 17:40:02 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:07 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 5) DEBUG: [c2/wg] 2025/08/22 17:40:07 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:12 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 6) DEBUG: [c2/wg] 2025/08/22 17:40:12 peer(Ysvj…+vBs) - Sending handshake initiation 2025/08/22 17:40:15 mtls.go:117: Socket ping 2025/08/22 17:40:15 mtls.go:101: Socket error (write msg-length): use of closed network connection DEBUG: [c2/wg] 2025/08/22 17:40:17 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 7) DEBUG: [c2/wg] 2025/08/22 17:40:17 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:22 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 2) DEBUG: [c2/wg] 2025/08/22 17:40:22 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:28 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 3) DEBUG: [c2/wg] 2025/08/22 17:40:28 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:33 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 4) DEBUG: [c2/wg] 2025/08/22 17:40:33 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:38 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 5) DEBUG: [c2/wg] 2025/08/22 17:40:38 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:43 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 6) DEBUG: [c2/wg] 2025/08/22 17:40:43 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:49 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 7) DEBUG: [c2/wg] 2025/08/22 17:40:49 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:54 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 8) DEBUG: [c2/wg] 2025/08/22 17:40:54 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:40:59 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 9) DEBUG: [c2/wg] 2025/08/22 17:40:59 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:41:04 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 10) DEBUG: [c2/wg] 2025/08/22 17:41:04 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:41:09 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 11) DEBUG: [c2/wg] 2025/08/22 17:41:09 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:41:15 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 12) DEBUG: [c2/wg] 2025/08/22 17:41:15 peer(Ysvj…+vBs) - Sending handshake initiation DEBUG: [c2/wg] 2025/08/22 17:41:20 peer(Ysvj…+vBs) - Handshake did not complete after 5 seconds, retrying (try 13) DEBUG: [c2/wg] 2025/08/22 17:41:20 peer(Ysvj…+vBs) - Sending handshake initiation 2025/08/22 17:41:22 wireguard.go:184: Unable to connect to wg key exchange listener: connect tcp 100.64.0.1:1337: operation timed out 2025/08/22 17:41:22 sliver.go:158: [session] failed to establish connection: [wireguard] Cannot connect to empty IP address 2025/08/22 17:41:22 sliver.go:138: Reconnect sleep: 1m0s

Additional context

version [] Client v1.5.42 - 85b0e870d05ec47184958dbcb871ddee2eb9e3df - linux/amd64 Compiled at 2024-02-28 19:46:53 +0000 UTC Compiled with go version go1.20.7 linux/amd64 [] Server v1.5.42 - 85b0e870d05ec47184958dbcb871ddee2eb9e3df - linux/amd64 Compiled at 2024-02-28 19:46:53 +0000 UTC

How can I modify and repair the service? If repair is not possible, can I back up the contents such as keys or certificates and reinstall the recovery beacon?

su20000101 avatar Aug 22 '25 10:08 su20000101

I see an expired certificate error in the server log. How can I generate a new certificate that is compatible with the old beacon and ensures that the beacon is not lost? Server Logs INFO[2025-08-22T10:21:22Z] [sliver/server/c2/mtls.go:85] Accepted incoming connection: 3.3.3.3:54644 ERRO[2025-08-22T10:21:22Z] [sliver/server/c2/mtls.go:165] Socket error (read msg-length): tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-08-22T10:21:22Z is after 2025-08-18T08:29:08Z ERRO[2025-08-22T10:21:22Z] [sliver/server/c2/mtls.go:103] Socket read error tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-08-22T10:21:22Z is after 2025-08-18T08:29:08Z INFO[2025-08-22T10:21:23Z] [sliver/server/c2/mtls.go:85] Accepted incoming connection: 3.3.3.3:54645 ERRO[2025-08-22T10:21:24Z] [sliver/server/c2/mtls.go:165] Socket error (read msg-length): tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-08-22T10:21:24Z is after 2025-08-18T08:29:08Z ERRO[2025-08-22T10:21:24Z] [sliver/server/c2/mtls.go:103] Socket read error tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-08-22T10:21:24Z is after 2025-08-18T08:29:08Z INFO[2025-08-22T10:21:38Z] [sliver/server/c2/mtls.go:85] Accepted incoming connection: 3.3.3.3:54648 ERRO[2025-08-22T10:21:38Z] [sliver/server/c2/mtls.go:165] Socket error (read msg-length): tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-08-22T10:21:38Z is after 2025-08-18T08:29:08Z ERRO[2025-08-22T10:21:38Z] [sliver/server/c2/mtls.go:103] Socket read error tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-08-22T10:21:38Z is after 2025-08-18T08:29:08Z INFO[2025-08-22T10:25:46Z] [sliver/server/c2/mtls.go:85] Accepted incoming connection: 3.3.3.3:54669 ERRO[2025-08-22T10:25:46Z] [sliver/server/c2/mtls.go:165] Socket error (read msg-length): tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-08-22T10:25:46Z is after 2025-08-18T08:29:08Z ERRO[2025-08-22T10:25:46Z] [sliver/server/c2/mtls.go:103] Socket read error tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-08-22T10:25:46Z is after 2025-08-18T08:29:08Z

su20000101 avatar Aug 22 '25 10:08 su20000101

The certificate expires in 2 years. You can check the expiration date of the certificate on the certificate page. The certificate in the implant in the db needs to be checked.

pythonSB avatar Aug 22 '25 11:08 pythonSB

I discovered a similar issue yesterday. I dug deeper and discovered that certificates are time-limited by default, which is a very tricky operation. If your beacon uses the HTTPS protocol, it might be salvageable, but MTLS is a two-way certificate, and the certificate is hard-coded into the implant, which leads to irreversible consequences.

pythonSB avatar Aug 22 '25 11:08 pythonSB

This result is very bad.

su20000101 avatar Aug 25 '25 02:08 su20000101

I discovered a similar issue yesterday. I dug deeper and discovered that certificates are time-limited by default, which is a very tricky operation. If your beacon uses the HTTPS protocol, it might be salvageable, but MTLS is a two-way certificate, and the certificate is hard-coded into the implant, which leads to irreversible consequences.

After switching servers, how can I recover my previous HTTPS connection?

xiaohaofei999 avatar Nov 24 '25 14:11 xiaohaofei999

I discovered a similar issue yesterday. I dug deeper and discovered that certificates are time-limited by default, which is a very tricky operation. If your beacon uses the HTTPS protocol, it might be salvageable, but MTLS is a two-way certificate, and the certificate is hard-coded into the implant, which leads to irreversible consequences.

After switching servers, how can I recover my previous HTTPS connection?

You need to back up all files in the database and the .sliver directory, as losing them will result in the loss of your data

pythonSB avatar Dec 02 '25 05:12 pythonSB

I discovered a similar issue yesterday. I dug deeper and discovered that certificates are time-limited by default, which is a very tricky operation. If your beacon uses the HTTPS protocol, it might be salvageable, but MTLS is a two-way certificate, and the certificate is hard-coded into the implant, which leads to irreversible consequences.

After switching servers, how can I recover my previous HTTPS connection?

You need to back up all files in the database and the .sliver directory, as losing them will result in the loss of your data

So, once the configuration folder .sliver is lost, it can never be recovered, right?

xiaohaofei999 avatar Dec 03 '25 12:12 xiaohaofei999