field-manual icon indicating copy to clipboard operation
field-manual copied to clipboard

Replicating a database

Open maroodb opened this issue 6 years ago • 9 comments

Hello World!

I'm trying replicating my public database in another computer using the address "/orbitdb/QmPmh5G7kUVTCAAeRCaZmdN8zeDbMByf3oU41xA1Cx5cw9/my-db". db = await orbitdb.docs('/orbitdb/QmPmh5G7kUVTCAAeRCaZmdN8zeDbMByf3oU41xA1Cx5cw9/my-db');

and when I try to get an existing object I got empty document .

maroodb avatar Mar 28 '18 16:03 maroodb

I believe you need to call load() on the database before you try to get something. In other words you need to tell it to replicate all the data, before you will have any data locally that you can get.

(also, it's await orbitdb.docstore, I think)

Something like:

db = await orbitdb.docstore('/orbitdb/QmPmh5G7kUVTCAAeRCaZmdN8zeDbMByf3oU41xA1Cx5cw9/my-db');
db.load();
console.log(db.get('mydoc'));

coyotespike avatar Mar 30 '18 15:03 coyotespike

I believe what @coyotespike writes above is the solution here.

@maroodb any updates? Did you manage to get it working?

haadcode avatar Apr 10 '18 05:04 haadcode

Maybe it's the same problem I'm experiencing here.

I set orbitdb in 3 different machines.

  • One run via nodejs (simple-server.js):
    • There is where I create the db: const db = await orbitdb.docstore('sample-db', access);
  • On the other two machines I just open the simple-client.js on the browser and everything looks fine:
    • I used the db address provided by the nodejs script.
    • When I add a new doc in one browser, it gets replicated in both the other browser and the nodejs.

All nodes receive the data as expected when I add a new doc in any of the browsers. The problem happens when I create a new doc on the nodejs script. It is populated locally but it isn't replicated to the browsers.

I put both files here.

(I know the script naming I gave is not the best one as the idea here is to get rid of the server-client model...)

adrprado avatar Apr 11 '18 21:04 adrprado

@adrprado You may need to more directly connect the nodejs databases to the browser database. You can do this by putting the nodejs machines' IPFS multiaddress into the browser's IPFS bootstrap list. There is a blog post in orbitdb/orbit-db#314 that will be helpful.

coyotespike avatar Apr 13 '18 18:04 coyotespike

@coyotespike, thanks for the tip. I talked with Mark and he also couldn't figure out why the browser node won't sync with the nodejs script.

He suggested a few things, but it seems everything is in place. I'm running a local signal server (rendezvous), I checked the swarm.peers(), pubsub.peers(), and the nodes are populated with each other id, as expected.

I also ran tcpdump and see activity on both sides as well (all traffic routed by the signal server).

And I have the same version of IPFS and OrbitDB on both nodes.

To sum up:

  • Nodejs is where the database were created: const db = await orbitdb.docstore('sample-db', access);
  • Browser 1 opens the database id: const orbitdb = new OrbitDB('/orbitdb/Qm..bR/sample-db');
  • Browser 2 does the same;
  • When I add a new doc on Browser 1, Browser 2 and Nodejs replicate the data locally.
  • When I add a new doc on Nodejs, both browsers won't replicate the new doc.

adrprado avatar Apr 14 '18 18:04 adrprado

thanks @coyotespike it works in CLI .. @haadcode Sorry for being late to give update

maroodb avatar Apr 16 '18 08:04 maroodb

@maroodb great to hear it's now working for you!

@adrprado sound like you're experiencing a different problem and I'd love to understand why it's not working. Specifically this part sounds like there's some glitch somewhere:

When I add a new doc on Browser 1, Browser 2 and Nodejs replicate the data locally. When I add a new doc on Nodejs, both browsers won't replicate the new doc.

If the browser-added doc gets replicated to the nodejs node, it sounds like they're talking. But since the nodejs-added doc doesn't get to the browsers, it sounds like there's something off in the way they're talking. This could be on OrbitDB side or on IPFS side. The browser-nodejs part hasn't been used and tested much so I'd love to understand what we need to fix here in order to get it working smoothly.

@adrprado if you have any insight or more details of the behaviour you're seeing, please do share your notes. I'd be curios to hear if (in the same example script ^), if the nodejs and browser nodes can, for example, get each other's messages over pubsub (ipfs.pubsub.subscribe('something', ...), ipfs.pubsub.publish('something', 'hello?') and if it works two ways (from browser to nodejs and nodejs to browser).

haadcode avatar Apr 17 '18 05:04 haadcode

Hi @haadcode, I'm enabling the debug on your code (locally) so I can check all the messaging exchange and I will let you know if I see something that's preventing this sync from nodejs to the browsers.

adrprado avatar Apr 17 '18 13:04 adrprado

Moving to the Field Manual for deeper discussion

aphelionz avatar Sep 27 '19 16:09 aphelionz