raix-push icon indicating copy to clipboard operation
raix-push copied to clipboard

Socket hang up, have tested all certificates and device tokens for APN

Open ChrisLangel opened this issue 7 years ago • 29 comments

Ok, I feel like we are so close here, but end up with socket hang up. I have tested the certificates using openssl on the command line, and manually sent a notification to my phone using the device token obtained looking at the output of Push.debug = true. On the logs, it is claiming Push: APN configured. It seems like everything should be in the clear, but when I call a test push from the server with the code:

Push.send({ 
      from: 'push', 
      title: 'Hello', 
      text: 'World',
      query: {},
  }); 

Debug output seems promising

Push: Sent message "Hello" to 1 ios apps 0 android apps

Then there is a socket hang up,

Error: socket hang up
    at TLSSocket.onHangUp (_tls_wrap.js:1054:19)
    at TLSSocket.g (events.js:260:16)
    at emitNone (events.js:72:20)
    at TLSSocket.emit (events.js:166:7)
    at endReadableNT (_stream_readable.js:923:12)
    at nextTickCallbackWith2Args (node.js:458:9)
    at process._tickCallback (node.js:372:17) 

It would seem there is an issue calling the APN server, I've double checked to ensure the production/development mismatch is not an issue as well. The app is deployed on a Digital Ocean droplet using mup. This has been royally frustrating, and I greatly appreciate any insight.

ChrisLangel avatar Nov 25 '16 20:11 ChrisLangel

Hi, I just started getting socket hangup too... wondering if you solved this... my system just started producing this on Friday and I had not changed the code Wednesday... so it had been working for a couple days before that. Now I see these constant socket hangup issues.

Did Apple change anything recently?

BTW, found this issue, not sure if related - https://github.com/node-apn/node-apn/issues/272

adamgins avatar Nov 27 '16 01:11 adamgins

@raix wondering if you've experienced anything similar. Again, seems like everything has been working for ages... not Push seems to be broken. Any thoughts welcome?

adamgins avatar Nov 27 '16 01:11 adamgins

Ok, so looks like I found my issue ... the certificate had expired... phew no warning form Apple and I thought I'd checked/renewed. I could not even find my certs in the apple dev console, so I am guessing the expired them and just removed them.

Also looks like they have a new non-Expiring option that uses a p8 file time (instead of p12)... no sure if this could work with the pushplugin... http://stackoverflow.com/questions/39672297/how-to-use-apples-new-p8-certificate-for-apns-in-firebase-console ?

adamgins avatar Nov 27 '16 03:11 adamgins

@raix anyway not to take the whole server down if a certificate expires? Maybe use try/catch I have not looked at the code or if this was even possible yet.. sorry was frantically trying to get my server back in business.

adamgins avatar Nov 27 '16 04:11 adamgins

We could send an emaill

raix avatar Nov 27 '16 06:11 raix

I was meaning a way that does not crash the whole server, if there's something wrong with the certificates.

adamgins avatar Nov 27 '16 08:11 adamgins

Yeah, the fact it crashes everything is a little disconcerting. I am still stuck but have a few updates. I was testing the certificates locally on my macbook, and not the server it was deployed on, so I ssh'd into the server itself, ran the openssl command, and got the error Verify return code: 20 (unable to get local issuer certificate) Upon downloading the entrust certificate and explicitly pointing to it:

openssl s_client -connect gateway.push.apple.com:2195 -CAfile entrust_2048_ca.cer -cert pushCer.pem -key pushKey.pem

was able to successfully connect. I am not entirely sure how to "install" this certificate on ubuntu from the command line, I tried just copying it into the /etc/ssl/certs folder but this still resulted in the socket hang up. Anyway, does anyone know of how to make sure this certificate is pointed to when the push package calls apn?

ChrisLangel avatar Nov 27 '16 18:11 ChrisLangel

Can you add your config here pls. Are you using dev or production certificates?

adamgins avatar Nov 27 '16 18:11 adamgins

Using production certificates, the client config is

Push.Configure({
  gcm: {
    projectNumber: '***********',
  },
  badge: true,
  sound: true,
  alert: true,
});

the server:

  Push.Configure({
    gcm: {
        apiKey: '***********'
    },
    apn: {
        passphrase: '***********',
        // pem files are placed in the app private folder
        certData: Assets.getText('pushCer.pem'),
        keyData: Assets.getText('pushKey.pem'),
    },
    production: true,
  });

This is the config you’re referring to right? Thanks for your help

ChrisLangel avatar Nov 27 '16 18:11 ChrisLangel

Thanks. And the last few lines of the openssl call.

I'm just confirming that your certificates are correct. I had the issue as one of mine had expired. I also saw the issue when I had miscreated a cert, which is kinda easy to do, especially as the creation is complex and the iTunes Connect interface has changed a little since the guide was written.

You now need to create your p12 files from a .cer file using this option screen shot 2016-11-28 at 6 24 33 am

Which is different to what's mentioned here https://www.raywenderlich.com/123862/push-notifications-tutorial

So after importing the .cer file into your Keychain you need to go through the "sister" steps to generate the Production Certificates.p12 file screen shot 2016-11-28 at 6 28 11 am

adamgins avatar Nov 27 '16 19:11 adamgins

With the command openssl s_client -connect gateway.push.apple.com:2195 -CAfile entrust_2048_ca.cer -cert pushCer.pem -key pushKey.pem

the last few lines of output are

---
SSL handshake has read 3296 bytes and written 2071 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-SHA
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 2A75E50A760AF9545D374F1220C03F0DB318B8823F1FC5BF11913A0A7D8AA4E7E218F370AE9A2812427E6E32252B0BC8
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1480274310
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

ChrisLangel avatar Nov 27 '16 19:11 ChrisLangel

OK, that looks good.

When you run that command, does it prompt you for the passcode before running.. something like

Enter pass phrase for pushKey.pem:

and double checking that's the same passcode you're using in your config file?

adamgins avatar Nov 27 '16 19:11 adamgins

Indeed, it prompts for a passcode, and it is the same as in the config file

ChrisLangel avatar Nov 27 '16 19:11 ChrisLangel

mmm so that sound like you have it all configured and perhaps is a DigitalOcean/entrust_2048_ca.cer config thing. Do you have SSL enabled on your server? I'm using Galaxy and imported my SSL certs to Galaxy and beyond that I did not have do anything, it just worked when I deployed.

I do see articles like https://www.digitalocean.com/community/tutorials/how-to-install-an-ssl-certificate-from-a-commercial-certificate-authority but not sure if that's relevant here.

adamgins avatar Nov 27 '16 19:11 adamgins

That was the conclusion I was thinking too, there is something wrong with the DigitalOcean/entrust_2048_ca.cer. I don't not explicitly set up SSL in the mup config file, but that was going to be my next move. I was sort of hoping there would be an easy way to just move the entrust cert somewhere. Do you have an SSL config in your deployment?

ChrisLangel avatar Nov 27 '16 20:11 ChrisLangel

It's been a while since I setup on Galaxy, but I remember i had to combine/concat a bunch of certs and then upload that certificate to the Galaxy config. It's one of the nice things about Galaxy, they make the setup relatively easy. Previously I was running AWS EC2 with Docker, NGINX config and it was complex.

adamgins avatar Nov 27 '16 20:11 adamgins

Yeah, I was trying to avoid setting up all that manually, but it may have to happen, I'll post the steps here if successful. There's got to be people out there hitting the same issue

ChrisLangel avatar Nov 27 '16 20:11 ChrisLangel

Same problem here. Certificates still valid and was working without any problems. From one day to another, it broke. Using DigitalOcean too.

philipthomasme avatar Feb 17 '17 09:02 philipthomasme

Same issues here.

levinunnink avatar Apr 04 '17 10:04 levinunnink

anyone figured this one out? At the least it shouldn't take down the whole app.

ianpogi5 avatar May 24 '17 05:05 ianpogi5

is it possible to use the p8 one that doesn't expire sounds pretty good. then the issue won't cause the error.

Wade-BuildOtto avatar Jun 22 '17 17:06 Wade-BuildOtto

Same here I started seeing all these crashes out of nowhere...

Will put an agenda warning for the next expiration date on my calendar this time, for next certificate changes ;)

mcoenca avatar Nov 19 '17 22:11 mcoenca

I can add a try catch statement - but it might be nice to be able to track down the issue - would it be enough to log to console? Also it could be an issue triggered by the apn package if NODE_ENV is not set to production? (I'm thinking it should be when deployed right?)

raix avatar Nov 24 '17 15:11 raix

We saw this error today, and it turned out to be an expired Apple Push Notification certificate. Thx @adamgins for leading us in the right direction! My only concern/request is that this type of error fails more gracefully, as it brought down our mobile app.

robsecord avatar Dec 07 '17 16:12 robsecord

@robsecord phew mine expired again... (I thought I was non-expiring one) and I can only generate the new p8 format... how did you setup / config?

adamgins avatar Dec 28 '17 02:12 adamgins

For me it was expired certificates too. But could you add the try catch statement? Its really scary if the whole server crashes because of expired certificates ...

andreasseiler avatar Mar 09 '18 18:03 andreasseiler

Same here. Server crashes when trying to send out messages via APN with "socket hangup". I tested my certificates, they all seem to be ok (not expired). Pretty strange that this crashes the whole server.

derwaldgeist avatar May 24 '18 10:05 derwaldgeist

Hi, guys. Did anyone find the solution for this? Thanks.

lcfm avatar Mar 15 '19 00:03 lcfm

This happened very long ago. IIRC, in the end it was a problem with the certificate. But I'm not sure.

derwaldgeist avatar Mar 24 '19 22:03 derwaldgeist