dat-js

dat-js copied to clipboard

Not sure how crucial this is (everything still seems to work), but when I create a new Dat

const dat = new Dat({
          persist: true, 
          gateway:  'ws://gateway.mauve.moe:3000'
})

I keep getting a websocket error in the browser Dev Console:

WebSocket connection to 'ws://localhost:3472/' failed: 
Error in connection establishment: 
net::ERR_CONNECTION_REFUSED

Tried to dig through the code to find where this is called, but can't seem to find it. I'm using Glitch.com, but not sure why the Dat-JS is calling localhost for a websocket.

Oh jeeze, I really need to give a better message there....

Basically, this is intended behavior. When dat-js boots up, it'll attempt to connect to localhost in case you have a discovery swarm server running there.

The idea was that if you have a local server for this, like dat-store, then you could use it's local peer discovery capabilities and get stuff to work offline.

The error isn't an actual error, it's more like an informational message. I'll update that next time I release dat-js. 😅

Was just about to inform about this as well. Basically have set up the example but took me a long time to realize I need to have a discovery swarm server running locally.

I'll take advantage of this thread to add my results :)

So here goes:

I started playing around with dat-js (I'd like to create a distributed link storage service mostly for myself and friends).

I created the following app which right now just basically reproduces the example more or less:

https://gitlab.com/saimonmoore/dlynx/blob/master/src/App.js#L73

I ran into the above error and since the documentation wasn't very helpful it took me a long while to discover: https://github.com/RangerMauve/discovery-swarm-web but once I was running a local version it started to work sort of:

Firefox (60.7.0esr (64-bit) OSX 10.14.5 (18F132))

I consistently get the following error:

On Chrome ( 74.0.3729.169 (Official Build) (64-bit) OSX 10.14.5 (18F132))

Seems to work on 1st load but after hard refresh, the discovery swarm web service dies every time:

...consistently on Chrome...

Safari (Version 12.1.1 (14607.2.6.1.1) OSX 10.14.5 (18F132))

Seems to work but 1st time doesn't seem to be able to read the stored data.

After a hard refresh it does find the data:

And it doesn't seem to break the discovery-swarm-web service.

Looking forward to getting stuck into this and understanding all the bits and pieces. I'd also be very interested in helping out with getting the docs up to scratch so it's a better experience for the next dev.

let me know if you want any additional experiments done...

cc @RangerMauve

Hi @saimonmoore! Thanks for checking this out! :)

You actually shouldn't need the local discovery service, since it should be trying to connect to the public one at https://discoveryswarm.mauve.moe/ after the local connection fails.

The error in the service is a little concerning, it might be happening due to failed incoming or outgoing TCP connections, but I'm not sure. If you could inspect the process with the --inspect flag and set it to pause execution on errors, it might give you a better idea of where in the code it's erroring out.

Regarding the data, it's likely a side effect of how the replication works. Basically, if you use readdir before the metadata is replicated, it'll return an empty directory. I've been using this trick to wait for metadata to load before doing anything, but I think it's a been a little buggy.

It's really exciting to see that somebody is actually using this stuff, though! :D

Cool thanks. I'll do a bit of digging around. I also expected it to just connect to the public service. I'll try and investigate.

Are there any new developments on this? I have also tried to use dat-js via the example code in https://docs.datproject.org/docs/browser-intro and while it seems to connect to mauve.moe I was not able to read any Dats. (The Dats I tried to read were shared from the same computer, in a different browser tab or via the CLI.)

corwin-of-amber: The discovery swarm server has a bit of trouble connecting to the cli running on people's home computer. Try loading your dat inside gateway.mauve.moe as a temporary way to bridge it to the network.

I'm hoping to fix this by integrating our new hyperswarm networking stack which should make it easier to get past the NAT.

Thanks - visiting the dat url through the gateway does in fact make it available to the browser-based client. But then isn't the gateway acting as a relay between the two peers? So it's not really p2p...

At any rate, the warning LOCALHOST_WARNING is never emitted, at least in Chrome, Firefox, and Safari where I tested it (all on Mac). Most probably because WebSocket is asynchronous? So the try .. catch block is not effective. That also means that the default discovery is not used.

Yeah, the gateway is a temporary thing since we haven't incorporated hyperswarm with discovery-swarm-web yet.

I'm not sure what you mean with regards to the localhost warning. Are you trying to connect to a discovery swarm server running on your local computer? You can check the "Network" tab in the dev tools and see which domain the websocket connects to or if it has any errors.

Sorry for being unclear in my previous comment, I was referring to this line in discovery-swarm-web.

Meanwhile I was able to connect to peers using discovery-swarm-web in the browser and it was very cool! I replicated a hypercore feed one-way. Still figuring out how to have multiple writers like in the Dat shopping list demo.

And it is connecting to discoveryswarm.mauve.moe, because it calls _reconnect on error, so that one is fine. Only the error message is still that WebSocket connection to 'ws://localhost:3472/' failed and not your nicely crafted message.

@corwin-of-amber for multi writer, See https://github.com/beakerbrowser/beaker/issues/815

Regarding multiwriter, I suggest waiting a little bit for the dat sdk to land. Then it'd be easiest to make something using kappa-core as a database.

Awesome, thanks @DougAnderson444 @RangerMauve !