go-libp2p
go-libp2p copied to clipboard
libp2p
mDNS discovery interoperability between Go and Javascript
mDNS discovery protocol does not work between implementations of Go and Javascript.
I'm trying to set up basic chat example where nodes discover each other through mDNS, looking at the examples provided in go-libp2p and js-libp2p (which leverage the mdns libraries "github.com/libp2p/go-libp2p/p2p/discovery/mdns" and @libp2p/mdns). The network packets sent by both implementations are, apparently, in the same format.
Both Go <-> Go and Js <-> Js work. I've also tried Rust examples and no cross-implementation discovery seems to work.
Connecting to the other node directly (bootstrap mode) works fine after making sure Js is using the yamux multiplexer.
Edit: I add captures of packets sent from Go and Js.
Go:
Js:
Although there are some small differences in the format, from what I understand Go implementation only looks at the TXT record, and expects to find all information there (full multiaddress prefixed with "dnsaddr="), which is exactly what Js implementation is providing. But please correct me if I'm wrong.
Version Information
github.com/libp2p/go-libp2p/examples cloud.google.com/go v0.65.0 cloud.google.com/go/bigquery v1.8.0 cloud.google.com/go/datastore v1.1.0 cloud.google.com/go/pubsub v1.3.1 cloud.google.com/go/storage v1.10.0 dmitri.shuralyov.com/app/changes v0.0.0-20180602232624-0a106ad413e3 dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9 dmitri.shuralyov.com/html/belt v0.0.0-20180602232347-f7d459c86be0 dmitri.shuralyov.com/service/change v0.0.0-20181023043359-a85b471d5412 dmitri.shuralyov.com/state v0.0.0-20180228185332-28bcc343414c git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999 github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 github.com/BurntSushi/toml v0.3.1 github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802 github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239 github.com/benbjohnson/clock v1.3.0 github.com/beorn7/perks v1.0.1 github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625 github.com/buger/jsonparser v0.0.0-20181115193947-bf1c66bbce23 github.com/census-instrumentation/opencensus-proto v0.2.1 github.com/cespare/xxhash v1.1.0 github.com/cespare/xxhash/v2 v2.2.0 github.com/chzyer/logex v1.1.10 github.com/chzyer/readline v1.5.0 github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 github.com/cilium/ebpf v0.4.0 github.com/client9/misspell v0.3.4 github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f github.com/containerd/cgroups v1.0.4 github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d github.com/coreos/go-systemd/v22 v22.5.0 github.com/cpuguy83/go-md2man/v2 v2.0.0 github.com/creack/pty v1.1.9 github.com/davecgh/go-spew v1.1.1 github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c github.com/decred/dcrd/crypto/blake256 v1.0.0 github.com/decred/dcrd/dcrec/secp256k1/v4 v4.1.0 github.com/dgraph-io/badger v1.6.2 github.com/dgraph-io/ristretto v0.0.2 github.com/docker/go-units v0.5.0 github.com/dustin/go-humanize v1.0.0 github.com/elastic/gosigar v0.14.2 github.com/envoyproxy/go-control-plane v0.9.4 github.com/envoyproxy/protoc-gen-validate v0.1.0 github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568 github.com/flynn/noise v1.0.0 github.com/francoispqt/gojay v1.2.13 github.com/frankban/quicktest v1.14.2 github.com/fsnotify/fsnotify v1.5.4 github.com/ghodss/yaml v1.0.0 github.com/gin-contrib/sse v0.1.0 github.com/gin-gonic/gin v1.6.3 github.com/gliderlabs/ssh v0.1.1 github.com/go-errors/errors v1.0.1 github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4 github.com/go-kit/kit v0.9.0 github.com/go-kit/log v0.2.0 github.com/go-logfmt/logfmt v0.5.1 github.com/go-logr/logr v1.2.3 github.com/go-logr/stdr v1.2.2 github.com/go-playground/assert/v2 v2.0.1 github.com/go-playground/locales v0.13.0 github.com/go-playground/universal-translator v0.17.0 github.com/go-playground/validator/v10 v10.2.0 github.com/go-stack/stack v1.8.0 github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0 github.com/gobwas/httphead v0.0.0-20180130184737-2c6c146eadee github.com/gobwas/pool v0.2.0 github.com/gobwas/ws v1.0.2 github.com/godbus/dbus/v5 v5.1.0 github.com/gogo/protobuf v1.3.2 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e github.com/golang/lint v0.0.0-20180702182130-06c8688daad7 github.com/golang/mock v1.6.0 github.com/golang/protobuf v1.5.2 github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db github.com/google/btree v1.0.0 github.com/google/go-cmp v0.5.9 github.com/google/go-github v17.0.0+incompatible github.com/google/go-querystring v1.0.0 github.com/google/gofuzz v1.0.0 github.com/google/gopacket v1.1.19 github.com/google/martian v2.1.0+incompatible github.com/google/martian/v3 v3.0.0 github.com/google/pprof v0.0.0-20221203041831-ce31453925ec github.com/google/renameio v0.1.0 github.com/google/uuid v1.3.0 github.com/googleapis/gax-go v2.0.0+incompatible github.com/googleapis/gax-go/v2 v2.0.5 github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 github.com/gorilla/websocket v1.5.0 github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 github.com/grpc-ecosystem/grpc-gateway v1.5.0 github.com/hashicorp/errwrap v1.1.0 github.com/hashicorp/go-multierror v1.1.1 github.com/hashicorp/golang-lru v0.5.4 github.com/hashicorp/golang-lru/v2 v2.0.1 github.com/huin/goupnp v1.0.3 github.com/huin/goutil v0.0.0-20170803182201-1ca381bf3150 github.com/ianlancetaylor/demangle v0.0.0-20220319035150-800ac71e25c2 github.com/ipfs/go-cid v0.3.2 github.com/ipfs/go-datastore v0.6.0 github.com/ipfs/go-detect-race v0.0.1 github.com/ipfs/go-ds-badger v0.3.0 github.com/ipfs/go-ds-leveldb v0.5.0 github.com/ipfs/go-ipfs-delay v0.0.0-20181109222059-70721b86a9a8 github.com/ipfs/go-ipfs-util v0.0.2 github.com/ipfs/go-ipns v0.2.0 github.com/ipfs/go-log v1.0.5 github.com/ipfs/go-log/v2 v2.5.1 github.com/ipld/go-ipld-prime v0.16.0 github.com/jackpal/go-nat-pmp v1.0.2 github.com/jbenet/go-cienv v0.1.0 github.com/jbenet/go-temp-err-catcher v0.1.0 github.com/jbenet/goprocess v0.1.4 github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1 github.com/jpillora/backoff v1.0.0 github.com/json-iterator/go v1.1.12 github.com/jstemmer/go-junit-report v0.9.1 github.com/jtolds/gls v4.20.0+incompatible github.com/julienschmidt/httprouter v1.3.0 github.com/kisielk/errcheck v1.5.0 github.com/kisielk/gotool v1.0.0 github.com/klauspost/compress v1.15.12 github.com/klauspost/cpuid/v2 v2.2.1 github.com/konsorten/go-windows-terminal-sequences v1.0.3 github.com/koron/go-ssdp v0.0.3 github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515 github.com/kr/pretty v0.3.0 github.com/kr/pty v1.1.3 github.com/kr/text v0.2.0 github.com/leodido/go-urn v1.2.0 github.com/libp2p/go-buffer-pool v0.1.0 github.com/libp2p/go-cidranger v1.1.0 github.com/libp2p/go-flow-metrics v0.1.0 github.com/libp2p/go-libp2p v0.26.0 github.com/libp2p/go-libp2p-asn-util v0.2.0 github.com/libp2p/go-libp2p-kad-dht v0.20.1-0.20230209220319-6c2045abad23 github.com/libp2p/go-libp2p-kbucket v0.5.0 github.com/libp2p/go-libp2p-record v0.2.0 github.com/libp2p/go-libp2p-routing-helpers v0.4.0 github.com/libp2p/go-libp2p-testing v0.12.0 github.com/libp2p/go-libp2p-xor v0.1.0 github.com/libp2p/go-mplex v0.7.0 github.com/libp2p/go-msgio v0.3.0 github.com/libp2p/go-nat v0.1.0 github.com/libp2p/go-netroute v0.2.1 github.com/libp2p/go-openssl v0.1.0 github.com/libp2p/go-reuseport v0.2.0 github.com/libp2p/go-sockaddr v0.0.2 github.com/libp2p/go-yamux/v4 v4.0.0 github.com/libp2p/zeroconf/v2 v2.2.0 github.com/lunixbochs/vtclean v1.0.0 github.com/mailru/easyjson v0.0.0-20190312143242-1de009706dbe github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd github.com/mattn/go-isatty v0.0.16 github.com/mattn/go-pointer v0.0.1 github.com/matttproud/golang_protobuf_extensions v1.0.4 github.com/microcosm-cc/bluemonday v1.0.1 github.com/miekg/dns v1.1.50 github.com/mikioh/tcp v0.0.0-20190314235350-803a9b46060c github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1 github.com/minio/sha256-simd v1.0.0 github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd github.com/modern-go/reflect2 v1.0.2 github.com/mr-tron/base58 v1.2.0 github.com/multiformats/go-base32 v0.1.0 github.com/multiformats/go-base36 v0.2.0 github.com/multiformats/go-multiaddr v0.8.0 github.com/multiformats/go-multiaddr-dns v0.3.1 github.com/multiformats/go-multiaddr-fmt v0.1.0 github.com/multiformats/go-multibase v0.1.1 github.com/multiformats/go-multicodec v0.7.0 github.com/multiformats/go-multihash v0.2.1 github.com/multiformats/go-multistream v0.4.1 github.com/multiformats/go-varint v0.0.7 github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86 github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab github.com/onsi/ginkgo v1.16.5 github.com/onsi/ginkgo/v2 v2.5.1 github.com/onsi/gomega v1.24.0 github.com/opencontainers/runtime-spec v1.0.2 github.com/opentracing/opentracing-go v1.2.0 github.com/openzipkin/zipkin-go v0.1.1 github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 github.com/pkg/errors v0.9.1 github.com/pmezard/go-difflib v1.0.0 github.com/polydawn/refmt v0.0.0-20201211092308-30ac6d18308e github.com/prometheus/client_golang v1.14.0 github.com/prometheus/client_model v0.3.0 github.com/prometheus/common v0.37.0 github.com/prometheus/procfs v0.8.0 github.com/quic-go/qpack v0.4.0 github.com/quic-go/qtls-go1-18 v0.2.0 github.com/quic-go/qtls-go1-19 v0.2.1 github.com/quic-go/qtls-go1-20 v0.1.1 github.com/quic-go/quic-go v0.33.0 github.com/quic-go/webtransport-go v0.5.1 github.com/raulk/go-watchdog v1.3.0 github.com/rogpeppe/go-internal v1.6.1 github.com/russross/blackfriday v1.5.2 github.com/russross/blackfriday/v2 v2.0.1 github.com/sergi/go-diff v1.0.0 github.com/shurcooL/component v0.0.0-20170202220835-f88ec8f54cc4 github.com/shurcooL/events v0.0.0-20181021180414-410e4ca65f48 github.com/shurcooL/github_flavored_markdown v0.0.0-20181002035957-2122de532470 github.com/shurcooL/go v0.0.0-20180423040247-9e1955d9fb6e github.com/shurcooL/go-goon v0.0.0-20170922171312-37c2f522c041 github.com/shurcooL/gofontwoff v0.0.0-20180329035133-29b52fc0a18d github.com/shurcooL/gopherjslib v0.0.0-20160914041154-feb6d3990c2c github.com/shurcooL/highlight_diff v0.0.0-20170515013008-09bb4053de1b github.com/shurcooL/highlight_go v0.0.0-20181028180052-98c3abbbae20 github.com/shurcooL/home v0.0.0-20181020052607-80b7ffcb30f9 github.com/shurcooL/htmlg v0.0.0-20170918183704-d01228ac9e50 github.com/shurcooL/httperror v0.0.0-20170206035902-86b7830d14cc github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371 github.com/shurcooL/httpgzip v0.0.0-20180522190206-b1c53ac65af9 github.com/shurcooL/issues v0.0.0-20181008053335-6292fdc1e191 github.com/shurcooL/issuesapp v0.0.0-20180602232740-048589ce2241 github.com/shurcooL/notifications v0.0.0-20181007000457-627ab5aea122 github.com/shurcooL/octicon v0.0.0-20181028054416-fa4f57f9efb2 github.com/shurcooL/reactions v0.0.0-20181006231557-f2e0b4ca5b82 github.com/shurcooL/sanitized_anchor_name v1.0.0 github.com/shurcooL/users v0.0.0-20180125191416-49c67e49c537 github.com/shurcooL/webdavfs v0.0.0-20170829043945-18c3829fa133 github.com/sirupsen/logrus v1.8.1 github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d github.com/smartystreets/goconvey v1.6.4 github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572 github.com/spaolacci/murmur3 v1.1.0 github.com/stretchr/objx v0.5.0 github.com/stretchr/testify v1.8.1 github.com/syndtr/goleveldb v1.0.0 github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07 github.com/ugorji/go v1.1.7 github.com/ugorji/go/codec v1.1.7 github.com/urfave/cli v1.22.2 github.com/viant/assertly v0.4.8 github.com/viant/toolbox v0.24.0 github.com/warpfork/go-testmark v0.3.0 github.com/warpfork/go-wish v0.0.0-20200122115046-b9ea61034e4a github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 github.com/yuin/goldmark v1.4.13 go.opencensus.io v0.24.0 go.opentelemetry.io/otel v1.7.0 go.opentelemetry.io/otel/trace v1.7.0 go.uber.org/atomic v1.10.0 go.uber.org/dig v1.15.0 go.uber.org/fx v1.18.2 go.uber.org/goleak v1.1.12 go.uber.org/multierr v1.8.0 go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee go.uber.org/zap v1.24.0 go4.org v0.0.0-20180809161055-417644f6feb5 golang.org/x/build v0.0.0-20190111050920-041ab4dc3f9d golang.org/x/crypto v0.4.0 golang.org/x/exp v0.0.0-20221205204356-47842c84f3db golang.org/x/image v0.0.0-20190802002840-cff245a6509b golang.org/x/lint v0.0.0-20200302205851-738671d3881b golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028 golang.org/x/mod v0.7.0 golang.org/x/net v0.4.0 golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b golang.org/x/perf v0.0.0-20180704124530-6e6d33e29852 golang.org/x/sync v0.1.0 golang.org/x/sys v0.3.0 golang.org/x/term v0.3.0 golang.org/x/text v0.5.0 golang.org/x/time v0.0.0-20191024005414-555d28b269f0 golang.org/x/tools v0.3.0 golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 google.golang.org/api v0.30.0 google.golang.org/appengine v1.6.6 google.golang.org/genproto v0.0.0-20200825200019-8632dd797987 google.golang.org/grpc v1.33.2 google.golang.org/protobuf v1.28.1 gopkg.in/alecthomas/kingpin.v2 v2.2.6 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c gopkg.in/errgo.v2 v2.1.0 gopkg.in/inf.v0 v0.9.1 gopkg.in/yaml.v2 v2.4.0 gopkg.in/yaml.v3 v3.0.1 grpc.go4.org v0.0.0-20170609214715-11d0a25b4919 honnef.co/go/tools v0.0.1-2020.1.4 lukechampine.com/blake3 v1.1.7 nhooyr.io/websocket v1.8.7 rsc.io/binaryregexp v0.2.0 rsc.io/quote/v3 v3.1.0 rsc.io/sampler v1.3.0 sourcegraph.com/sourcegraph/go-diff v0.5.0 sourcegraph.com/sqs/pbtypes v0.0.0-20180604144634-d3ebe8f20ae4
Thanks for creating this ticket, there's been an issue created to add interoperability tests for mDNS to ensure this works across implementations: https://github.com/libp2p/test-plans/issues/179
It might be worth recording a packet trace using Wireshark, and inspect the mDNS messages exchanged and check if they comply with the mDNS spec. zeroconf is the mDNS implementation we're using, it might also be helpful to add some debug statements there. @djav35, want to take a stab at debugging this?
It might be worth recording a packet trace using Wireshark, and inspect the mDNS messages exchanged and check if they comply with the mDNS spec. zeroconf is the mDNS implementation we're using, it might also be helpful to add some debug statements there. @djav35, want to take a stab at debugging this?
Thanks. I just edited the issue providing packet captures.
To close this issue we should document what causes the incompatibility and suggestions for fixes (either in go-libp2p and other libp2p implementations).
For testing cross implementation interop, let's use this issue: https://github.com/libp2p/test-plans/issues/179.
I found in the test that js-libp2p can discover the nodes of go-libp2p, but go-libp2p cannot discover the nodes of js-libp2p.
I tracked the problem and found that it was mainly caused by this piece of logic: https://github.com/libp2p/zeroconf/blob/9ff955d44bca57ab14942285552c0b34e9abbad8/client.go#L285
Since serviceTypeName and serviceName must be different, go back and determine whether there is an A or AAAA record in the mdns response answer.
There is no A record in the response of js-libp2p, so the package will be discarded:
There is an A record in go-libp2p, so it can be processed correctly.
There are many ways to solve this problem:
- Change the libp2p/zeroconf library and set serviceTypeName and serviceName to the same (I think the easiest way)
- Shield this verification logic (may cause some unknown problems)
- js-libp2p adds A record answer
If possible please let me provide pr❤.
