gun icon indicating copy to clipboard operation
gun copied to clipboard
verified publisher icon amark

Relay peer crash breaks all browser clients until they refresh the page

Open Joncom opened this issue 5 years ago • 8 comments

TL;DR: 2 clients connect to a relay peer. They subscribe to data using gun.on(), and everything works fine. However, if the relay peer crashes and comes back, clients do not resume receiving updates and must refresh the page to resolve the issue.

Chat from Discord:

Joncom:

  1. ClientA and ClientB are both connected to PeerServer1, the only peer server.
  2. Both clients are listening to changes on object foo like so: let callback = function(data) { console.log(data); }; gun.get('foo').on(callback);
  3. PeerServer1 goes offline, and clients cannot communicate
  4. ClientA writes some offline data to foo like so: gun.get('foo').put({a: 1});
  5. callback fires on ClientA because subscribed to foo
  6. ClientB writes some offline data to foo like so: gun.get('foo').put({b: 1});
  7. callback fires on ClientB because subscribed to foo
  8. PeerServer1 comes back online and both peers connect to it

Question: Should callback fire again on ClientA as it receives the offline update put by ClientB, and vice versa?

I would expect the data from ClientA to sync over to ClientB (and vice versa) and trigger the on callback to fire, when the clients reconnect to the peer server. However, in practice, despite reconnecting to the peer server, clients don't receive updates from each other, and a refresh is needed before all updates show up on both clients. :(

Mark Nadal:

Yes Your 1 ~ 8 is exactly what the PANIC holy-grail test does/checks, yes you should get called with those offline updates. If you're having to refresh, then this is a regression/bug.

Steps to reproduce:

  1. Clone Gun repo
  2. cd test/panic/
  3. mocha holy-grail.js
  4. The test will not complete (hangs on "Browsers re-initialized gun!")

Joncom avatar Jun 25 '20 04:06 Joncom

I observed this behavior of GUN too.

sclee15 avatar Jul 08 '20 06:07 sclee15

Ugh, I wish I knew which version this regressed in. Tho I'm in the middle of an experimental migration/rewrite anyways, so probably better I finish off these changes & get this fixed in the new system so don't lose it again.

I apologize for this regression, that is super annoying as it is part of GUN's bread & butter. You deserve better! Please have patience while I upgrade everything, and rollback to older versions for now (tho be aware they have some other bugs - but hopefully easier to workaround).

amark avatar Jul 09 '20 12:07 amark

I wrote a PANIC test specifically for this bug. https://github.com/Joncom/gun/commit/2ddb5263237b94beca01e9b1a0e97f391e770cb8

Joncom avatar Jul 15 '20 20:07 Joncom

@Joncom YESSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS can you PR this into my master? :D 👍

amark avatar Jul 16 '20 00:07 amark

@amark Opened pull-request #991

Joncom avatar Jul 19 '20 07:07 Joncom

@amark this problem still exists in gun 0.2020.1240 from npm

you can replicate it with a modified todo app example (see gist gun-todo-reconnect.html) and a local gunjs server:

  1. add the local gunsjs replicator server to the code
  2. add a button which reconnects by adding the peers config

Repeat this steps to reproduce the problem:

  1. start the gunjs local replication server
  2. open the todo app in one browser A (e.g. Chrome)
  3. open the todo app in second browser B (e.g. Firefox)
  4. add a todo "1A" in browser A
  5. shutdown gunjs local replication server
  6. add a todo "2A" in browser A - offline
  7. add a todo "1B" in browser B - offline
  8. start the gunjs local replication server
  9. click on button reconnect in browser A
  10. click on button reconnect in browser B

Result:

  • offline data "2A", "1B" from browser A and browser B is synced to the gunjs local replication server
  • new offline data "2A" from browser A is synced to browser B

Error:

  • new offline data "1B" from browser B is not synced to browser A

A refresh of browser B will load all data. The order of clicking the reconnect button will define which data is missed. The browser reconnects first will always miss any offline updates from browsers connecting later.

These are the console log entries from the listener function to map().on() which also reflect the entries in the locals storage gun/ key:

Browser A:

todo {_: {…}, title: '1A'} makxnyu9M9bSaJgJDzha - gun-client.html:32 
todo {_: {…}, title: '1A'} makxnyu9M9bSaJgJDzha - gun-client.html:32
WebSocket connection to 'ws://localhost:8888/gun' failed: open - gun.js:1755
todo {_: {…}, title: '2A'} makxodyqvSEHrbMuZ6qQ - gun-client.html:32 
Reconnected to Gun - gun-client.html:27

Browser B:

todo {_: {…}, title: '1A'} makxnyu9M9bSaJgJDzha - gun-client.html:32 
todo {_: {…}, title: '1A'} makxnyu9M9bSaJgJDzha - gun-client.html:32 
WebSocket connection to 'ws://localhost:8888/gun' failed:  open - gun.js:1755
todo {_: {…}, title: '1B'} makxoi1upzjx4Sq9SzRV - gun-client.html:32 
Reconnected to Gun - gun-client.html:27
todo {_: {…}, title: '2A'} makxodyqvSEHrbMuZ6qQ - gun-client.html:32 
todo {_: {…}, title: '2A'} makxodyqvSEHrbMuZ6qQ - gun-client.html:32

Browser B will receive the offline updates from Browser B

aheissenberger avatar May 12 '25 10:05 aheissenberger

@aheissenberger thanks for the extensive checking. To help me assess criticality:

(A) does using latest GitHub fix the issue? And: (B) Is there a worse bug in latest GitHub that prevents upgrading to it?

Please let me know, thanks!

amark avatar Jun 16 '25 05:06 amark

@aheissenberger thanks for the extensive checking. To help me assess criticality:

(A) does using latest GitHub fix the issue? And: (B) Is there a worse bug in latest GitHub that prevents upgrading to it?

Please let me know, thanks!

(A) YES :-) latest GitHub fixes the issue (B) no problems to upgrade :-)

BROKEN OUTPUT with npm package 0.2020.1240

t1 off offline data from Alice not synced to Bob and Fred

% node src/server.js
Debugger attached.
Hello wonderful person! :) Thanks for using GUN, please ask for help on http://chat.gun.eco if anything takes you longer than 5min to figure out!
Directory "radata" and all its subdirectories have been removed.
Directory "peer_cache" and all its subdirectories have been removed.
AXE relay enabled!
Gun server started: http://localhost:8889/gun
Multicast on 233.255.255.255:8765
Hello wonderful person! :) Thanks for using GUN, please ask for help on http://chat.gun.eco if anything takes you longer than 5min to figure out!
Hello wonderful person! :) Thanks for using GUN, please ask for help on http://chat.gun.eco if anything takes you longer than 5min to figure out!
Hello wonderful person! :) Thanks for using GUN, please ask for help on http://chat.gun.eco if anything takes you longer than 5min to figure out!
AXE relay enabled!
bob: Gun client started with options: {"peers":["http://localhost:8889/gun"]}
Multicast on 233.255.255.255:8765
Message from worker Bob: Gun client Bob connected and data set.
bob: Received command: bob1 with options: undefined
Message from worker Bob: received data: {"_":{"#":"mamcw7h001aAeMbPpbiMucr",">":{"title":1747131347412.002}},"title":"t2"} with key: mamcw7h001aAeMbPpbiMucr
AXE relay enabled!
fred: Gun client started with options: {"peers":["http://localhost:8889/gun"]}
Multicast on 233.255.255.255:8765
AXE relay enabled!
alice: Gun client started with options: {"peers":[]}
Multicast on 233.255.255.255:8765
Message from worker Alice: Gun client Alice initialized and data set.
Message from worker Alice: Gun client Alice connected and data set.
Message from worker Alice: received data: {"_":{"#":"mc0jtshy01ahqR26P12BVEP",">":{"title":1750166300807}},"title":"t1 off"} with key: mc0jtshy01ahqR26P12BVEP
Message from worker Alice: received data: {"_":{"#":"mc0jtshy01ahqR26P12BVEP",">":{"title":1750166300807}},"title":"t1 off"} with key: mc0jtshy01ahqR26P12BVEP
alice: Received command: alice1 with options: undefined
Gun client Alice started without peers (offline mode)
alice: Received command: alice2 with options: undefined
Warning: reusing same fs store and options as 1st.
Message from worker Alice: received data: {"_":{"#":"mc0jtshy01ahqR26P12BVEP",">":{"title":1750166300807}},"title":"t1 off"} with key: mc0jtshy01ahqR26P12BVEP
Message from worker Alice: received data: {"_":{"#":"mamcw7gyF9zymdUXX51e",">":{"title":1747131347410.001}},"title":"t1 off"} with key: mamcw7gyF9zymdUXX51e
Message from worker Alice: received data: {"_":{"#":"mamcw7h001aAeMbPpbiMucr",">":{"title":1747131347412.002}},"title":"t2"} with key: mamcw7h001aAeMbPpbiMucr
Message from worker Bob: received data: {"_":{"#":"mc0jtshz03tIcKEa6evDVyh",">":{"title":1750166300807.004}},"title":"t2"} with key: mc0jtshz03tIcKEa6evDVyh
Message from worker Alice: received data: {"_":{"#":"mc0jtshz03tIcKEa6evDVyh",">":{"title":1750166300807.004}},"title":"t2"} with key: mc0jtshz03tIcKEa6evDVyh
Message from worker Bob: received data: {"_":{"#":"mc0jtshz03tIcKEa6evDVyh",">":{"title":1750166300807.004}},"title":"t2"} with key: mc0jtshz03tIcKEa6evDVyh

WORKING with latest code from GitHub

% node src/server.js
Debugger attached.
Hello wonderful person! :) Thanks for using GUN, please ask for help on http://chat.gun.eco if anything takes you longer than 5min to figure out!
Directory "radata" and all its subdirectories have been removed.
Directory "peer_cache" and all its subdirectories have been removed.
AXE relay enabled!
Gun server started: http://localhost:8889/gun
Multicast on 233.255.255.255:8765
Hello wonderful person! :) Thanks for using GUN, please ask for help on http://chat.gun.eco if anything takes you longer than 5min to figure out!
Hello wonderful person! :) Thanks for using GUN, please ask for help on http://chat.gun.eco if anything takes you longer than 5min to figure out!
Hello wonderful person! :) Thanks for using GUN, please ask for help on http://chat.gun.eco if anything takes you longer than 5min to figure out!
AXE relay enabled!
bob: Gun client started with options: {"peers":["http://localhost:8889/gun"]}
Multicast on 233.255.255.255:8765
AXE relay enabled!
alice: Gun client started with options: {"peers":[]}
Multicast on 233.255.255.255:8765
AXE relay enabled!
fred: Gun client started with options: {"peers":["http://localhost:8889/gun"]}
Multicast on 233.255.255.255:8765
Message from worker Bob: Gun client Bob connected and data set.
bob: Received command: bob1 with options: undefined
Message from worker Bob: received data: {"_":{"#":"mamcw7h001aAeMbPpbiMucr",">":{"title":1747131347412.002}},"title":"t2"} with key: mamcw7h001aAeMbPpbiMucr
Message from worker Bob: received data: {"_":{"#":"mc0jtshz03tIcKEa6evDVyh",">":{"title":1750166300807.004}},"title":"t2"} with key: mc0jtshz03tIcKEa6evDVyh
alice: Received command: alice1 with options: undefined
Message from worker Alice: Gun client Alice initialized and data set.
Message from worker Alice: Gun client Alice connected and data set.
Message from worker Alice: received data: {"_":{"#":"mc0k7ne901a2nw2z9xjWwtF",">":{"title":1750166947378}},"title":"t1 off"} with key: mc0k7ne901a2nw2z9xjWwtF
Message from worker Alice: received data: {"_":{"#":"mc0k7ne901a2nw2z9xjWwtF",">":{"title":1750166947378}},"title":"t1 off"} with key: mc0k7ne901a2nw2z9xjWwtF
Gun client Alice started without peers (offline mode)
alice: Received command: alice2 with options: undefined
Warning: reusing same fs store and options as 1st.
Message from worker Alice: received data: {"_":{"#":"mc0k7ne901a2nw2z9xjWwtF",">":{"title":1750166947378}},"title":"t1 off"} with key: mc0k7ne901a2nw2z9xjWwtF
Message from worker Alice: received data: {"_":{"#":"mamcw7gyF9zymdUXX51e",">":{"title":1747131347410.001}},"title":"t1 off"} with key: mamcw7gyF9zymdUXX51e
Message from worker Alice: received data: {"_":{"#":"mamcw7h001aAeMbPpbiMucr",">":{"title":1747131347412.002}},"title":"t2"} with key: mamcw7h001aAeMbPpbiMucr
Message from worker Alice: received data: {"_":{"#":"mc0jtshy01ahqR26P12BVEP",">":{"title":1750166300807}},"title":"t1 off"} with key: mc0jtshy01ahqR26P12BVEP
Message from worker Alice: received data: {"_":{"#":"mc0jtshz03tIcKEa6evDVyh",">":{"title":1750166300807.004}},"title":"t2"} with key: mc0jtshz03tIcKEa6evDVyh
Message from worker Bob: received data: {"_":{"#":"mc0k7ned01aF1cfUS5TmfTO",">":{"title":1750166947381.002}},"title":"t2"} with key: mc0k7ned01aF1cfUS5TmfTO
Message from worker Alice: received data: {"_":{"#":"mc0k7ned01aF1cfUS5TmfTO",">":{"title":1750166947381.002}},"title":"t2"} with key: mc0k7ned01aF1cfUS5TmfTO
Message from worker Alice: received data: {"_":{"#":"mamcw7h001aAeMbPpbiMucr",">":{"title":1747131347412.002}},"title":"t2"} with key: mamcw7h001aAeMbPpbiMucr
Message from worker Alice: received data: {"_":{"#":"mc0jtshz03tIcKEa6evDVyh",">":{"title":1750166300807.004}},"title":"t2"} with key: mc0jtshz03tIcKEa6evDVyh
Message from worker Alice: received data: {"_":{"#":"mc0k7ned01aF1cfUS5TmfTO",">":{"title":1750166947381.002}},"title":"t2"} with key: mc0k7ned01aF1cfUS5TmfTO
Message from worker Bob: received data: {"_":{"#":"mc0k7ned01aF1cfUS5TmfTO",">":{"title":1750166947381.002}},"title":"t2"} with key: mc0k7ned01aF1cfUS5TmfTO
Message from worker Bob: received data: {"_":{"#":"mamcw7gyF9zymdUXX51e",">":{"title":1747131347410.001}},"title":"t1 off"} with key: mamcw7gyF9zymdUXX51e
Message from worker Bob: received data: {"_":{"#":"mc0jtshy01ahqR26P12BVEP",">":{"title":1750166300807}},"title":"t1 off"} with key: mc0jtshy01ahqR26P12BVEP
Message from worker Alice: received data: {"_":{"#":"mamcw7gyF9zymdUXX51e",">":{"title":1747131347410.001}},"title":"t1 off"} with key: mamcw7gyF9zymdUXX51e
Message from worker Alice: received data: {"_":{"#":"mc0jtshy01ahqR26P12BVEP",">":{"title":1750166300807}},"title":"t1 off"} with key: mc0jtshy01ahqR26P12BVEP
Message from worker Bob: received data: {"_":{"#":"mamcw7gyF9zymdUXX51e",">":{"title":1747131347410.001}},"title":"t1 off"} with key: mamcw7gyF9zymdUXX51e
Message from worker Bob: received data: {"_":{"#":"mc0jtshy01ahqR26P12BVEP",">":{"title":1750166300807}},"title":"t1 off"} with key: mc0jtshy01ahqR26P12BVEP

Here is my testcode - maybe something which could be added as a test:

import GUN from 'gun';
import http from 'node:http';
import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads';
import { rm, mkdir } from 'node:fs/promises';

const port = Number(process.env?.HTTP_PORT ?? "8889");

if (isMainThread) {

    try {
        // cleanup the relay cache
        await rm('./radata', { recursive: true, force: true });
        console.info('Directory "radata" and all its subdirectories have been removed.');
        // cleanup the peers cache
        await rm('./peer_cache', { recursive: true, force: true });
        console.info('Directory "peer_cache" and all its subdirectories have been removed.');
    } catch (error) {
        console.error('Error removing directory:', error);
    }

    await mkdir('./peer_cache', { recursive: true });

    if (isNaN(port) || port < 1 || port > 65535) {
        console.error(`Invalid port number "${port}". Please provide a number between 1 and 65535.`);
        process.exit(1);
    }
    const server = http.createServer(GUN.serve('./')).listen(port, () => {
        const gunServerUrl = `http://localhost:${port}/gun`;
        console.info(
            'Gun server started: ' + gunServerUrl
        );

        const workerBob = new Worker(new URL(import.meta.url), { workerData: { name: 'bob', options: { gunOptions: { peers: [gunServerUrl] } } } });
        workerBob.on('message', (message) => {
            console.log('Message from worker Bob:', message);
        });
        const workerFred = new Worker(new URL(import.meta.url), { workerData: { name: 'fred', options: { gunOptions: { peers: [gunServerUrl] } } } });
        workerFred.on('message', (message) => {
            console.log('Message from worker Fred:', message);
        });
        const workerAlice = new Worker(new URL(import.meta.url), { workerData: { name: 'alice', options: { gunOptions: { peers: [] } } } });
        workerAlice.on('message', (message) => {
            console.log('Message from worker Alice:', message);
        });
        workerBob.postMessage({ port, cmd: 'bob1' });
        setTimeout(() => {
            workerAlice.postMessage({ port, cmd: 'alice1' });
            workerAlice.postMessage({ port, cmd: 'alice2' });
            //workerFred.postMessage({ port, cmd: 'fred1' });
        }, 3000);


    });
    const gun = GUN({ web: server });


} else {
    const { name, options } = workerData;
    const debugOptions = JSON.stringify(options.gunOptions);
    let gun = GUN(options.gunOptions);

    console.info(`${name}: Gun client started with options: ${debugOptions}`);
    parentPort.on('message', ({ cmd, msgoptions }) => {
        console.info(`${name}: Received command: ${cmd} with options: ${JSON.stringify(msgoptions)}`);
        gun = GUN({ peers: (cmd.substr(0, 3) !== 'ali' ? [`http://localhost:${port}/gun`] : []), file: `db/${cmd.substr(0, 3)}.json`, axe: false });
        switch (cmd) {
            case 'init':
                gun = GUN(msgoptions.gunOptions);
                console.log(`Gun client ${type} initialized with options: ${JSON.stringify(msgoptions.gunOptions)}`);
                parentPort.postMessage(cmd);
                break;


            case 'alice1':
                console.info('Gun client Alice started without peers (offline mode)');
                gun.get('myset').map().on((data, key) => {
                    parentPort.postMessage(`received data: ${JSON.stringify(data)} with key: ${key}`);
                });
                gun.get('myset').set({ title: 't1 off' });
                parentPort.postMessage(`Gun client Alice initialized and data set.`);
                break;
            case 'bob1':
                gun.get('myset').map().on((data, key) => {
                    parentPort.postMessage(`received data: ${JSON.stringify(data)} with key: ${key}`);
                });
                parentPort.postMessage(`Gun client Bob connected and data set.`);
                break;
            case 'alice2':
                gun.opt({ peers: [`http://localhost:${port}/gun`] });
                gun.get('key').get('heartbeat').put(new Date().getTime());
                gun.get('myset').set({ title: 't2' });
                parentPort.postMessage(`Gun client Alice connected and data set.`);
                break;
            case 'fred1':
                gun.get('myset').map().on((data, key) => {
                    parentPort.postMessage(`Fred received data: ${JSON.stringify(data)} with key: ${key}`);
                });
                break
            default:
                console.error(`Unknown type "${type}"`);
        }
    })

}

aheissenberger avatar Jun 17 '25 13:06 aheissenberger