Debugging federation issues

[bremensaki]

I've updated to current and having issues with interacting with events via Mastodon instances (tried a few on the same event - chinwag.org, aus.social and hackers.town all the issue).

I can view the events on Mastodon fine via their address, but signature verification always fails on the follow requests when they get back to the instance. The verify function always returns false.

Is there an undocumented extra step in the setup somewhere or any hints on debugging this further?

[dariusk]

Hmm, when you say

signature verification always fails on the follow requests when they get back to the instance.

do you mean that the Mastodon instance does not correctly verify the signed Accept message sent back to the instance once the Follow is processed by Gathio?

[dariusk]

Actually I think I am having this exact issue attempting to follow events I create on gath.io itself (Follows aren't verified, etc). I'm going to try and debug this...

[dariusk]

Hmm, I just propped a brand new instance of Gathio to test and it interacts just fine with vanilla Mastodon... let's see.

What version of Node.js are you running?

[bremensaki]

What appears to happen is the follow request is sent back from Mastodon to gathio, and the signature verification fails.

This call always returns false.

Node is v12.14.1, Debian 10 underneath. Pretty fresh install of everything.

[dariusk]

I wish I could reproduce this bug but I can't...

Can you edit the running code such that the block highlighted here is replaced with the following code, and then tell me what the logs say?

    const verifier = crypto.createVerify('RSA-SHA256')
    verifier.update(comparison_string, 'ascii')
    const publicKeyBuf = new Buffer(publicKey, 'ascii')
    const signatureBuf = new Buffer(signature_header.signature, 'base64')
    console.log('public key:', publicKey)
    try {
      const result = verifier.verify(publicKeyBuf, signatureBuf)

I'm having it print the public key which is safe to include in a response here in this issue -- mostly I'm worried that publicKey is coming up as null or undefined.

Also can you point me to an existing event that I can see? I should be able to directly query its ActivityPub actor to see if anything is amiss.

[bremensaki]

I just created a new event a year off as a stable debugging target: https://deffo.com.au/YDBV7BLF

Public key definitely seems to be defined during the call.

Jan 28 02:26:57 deffo npm[1920]: Mongoose connection open!
Jan 28 02:27:36 deffo npm[1920]: public key: -----BEGIN PUBLIC KEY-----
Jan 28 02:27:36 deffo npm[1920]: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAz1IGsqIaQCnNJhjFyhZN
Jan 28 02:27:36 deffo npm[1920]: vwMSjccBCwl934gTW9+A+3IPR6zat5FxeEXfeHMnpWK7VXd0TbuyJSvmnrntll7j
Jan 28 02:27:36 deffo npm[1920]: rsoDQbJv1JEdD9hngbTtuOPDG7PfXAvoMMgUAIg18KMuWnGccw8EAlE8srPoBOvA
Jan 28 02:27:36 deffo npm[1920]: T+GgEXLdR9XgddbHzbSNNCki/TrXKc6bJKcpRxEA2Ij0ZP5zljfco+8dHBK7XgVz
Jan 28 02:27:36 deffo npm[1920]: iNhfgEX9cUgP3Fg4rNE9l4QZ/NhTA4w7r2a4J8aQBXS+celC6xQzJddRff6XAevC
Jan 28 02:27:36 deffo npm[1920]: KVydcelbE6QCMLKjzzhde2HVzU1xtw/CMCwy6yuKviFr9CQCrcrzE988ze2xkGkc
Jan 28 02:27:36 deffo npm[1920]: fwIDAQAB
Jan 28 02:27:36 deffo npm[1920]: -----END PUBLIC KEY-----
Jan 28 02:27:36 deffo npm[1920]: 401 here!

The "401 Here" is my own log point to double check the 401 comes from the if (result) / else condition, not the catch after that block.

[koyaga]

I guess I have the same issue. I created an event on gath.io and when I try to follow the event's Mastodon account, the follow request never gets accepted. I tried to follow the event's account with two different accounts on two diffferent mastodon instances (mamot.fr and mastodon.social)

[kousu]

Hello! Federation is still wonky. I'm on dc165eb935308a05272f5ed1b20e2d1527ef9eb9. I made an event and me and my friend tried to follow it from:

Mastodon: could find it and render the description

Screenshot

but clicking Follow gave us this in the logs:

Nov 12 05:02:48 public npm[94586]: sent Accept to https://indieweb.social/users/evan
Nov 12 05:06:06 public npm[94586]: sent Accept to https://coq.social/users/tambourine

and no other response. No Poll, no DM.

Pleroma: trying to look up the event on Pleroma just says "Not Found":

Screenshot

and nothing in the logs.

That's as far as I've debugged so far. If I find out more I'll keep posting.

[dariusk]

Hi @kousu , I fixed this today in this PR: https://github.com/lowercasename/gathio/pull/79

[kousu]

Hi @dariusk! Your PR just got merged, and I've updated to master and I'm testing now.

Here's a test event: https://events.public.garden/1igF40AIfCVPW_gBzcvOz

Here's how it renders via Mastodon: https://coq.social/web/@[email protected]

Clicking follow

shows me this server-side in

root@public:~# journalctl -f -u gathio
Nov 13 19:22:18 public.garden npm[23662]: sent Accept to https://coq.social/users/tambourine

and it took about 20 seconds, but I got a pingback on Mastodon:

I was also able to make a comment: https://coq.social/web/@tambourine/109338135278523385

(and the "a few seconds ago" link correctly goes back to https://coq.social/web/@tambourine/109338135278523385)

All these interactions were logged on Gathio's side like:

Nov 13 19:25:08 public.garden npm[23662]: sent to https://coq.social/users/tambourine
Nov 13 19:25:08 public.garden npm[23662]: sent to https://coq.social/users/tambourine
Nov 13 19:25:08 public.garden npm[23662]: sent to https://coq.social/users/tambourine
Nov 13 19:25:43 public.garden npm[23662]: sent to https://coq.social/users/tambourine

[lowercasename]

@dariusk's PR is merged into main and live on the gath.io site, but I'm going to keep this issue open while @kousu is debugging Pleroma issues.

[kousu]

we crossed over each other again :P

I opened a sub-issue just for Pleroma. But I can continue here, if you'd prefer. Just let me know what you prefer :)

[lowercasename]

No, go for it in the sub-issue thread!