IPFS slows down dramatically with default config

[julienmalard]

• Version: { version: '0.16.0', commit: '89aeaf8e25320276391653104981e37a73f29de9', repo: '12', 'ipfs-core': '0.16.0', 'interface-ipfs-core': '^0.156.0' }

• Platform: Darwin MacBook-Pro-222.local 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:25:27 PDT 2022; root:xnu-8020.141.5~2/RELEASE_X86_64 x86_64

• Subsystem: Running on Node.js v16.13.0

Severity:

Medium - A non-essential functionality does not work, performance issues, etc.

Description:

When running an IPFS node with a default config, the node progressively slows down until calls such as ipfs.block.put() take up to 10 seconds or more to complete.

Steps to reproduce the error:

The following code produces the error. Interestingly, disabling bootstrap (set BOOTSTRAP = false avoids the problem (calls to ipfs.block.put() take 10-50 ms as expected). Disabling the internet connection on the computer also avoids the problem. However, running while normally connected to the internet (and also to a work network whose settings do not allow connecting to any peers apart from dns bootstrap nodes) results in the code slowing down dramatically after a few cycles, easily taking 7-10 seconds per call. I discovered this issue while working with OrbitDB, which makes quite a few calls to ipfs.block.put() in the creation of a database, which leads to over several minutes to create a single OrbitDB database.

#!/usr/bin/env node --experimental-specifier-resolution=node
import { create } from "ipfs";
import { Options } from "ipfs-core/types"
import rm from "rimraf";

const DIR_TEST_REPO = "./testrepo";
rm.sync(DIR_TEST_REPO);

const BOOTSTRAP = true

const configSFIP: Options = {
  repo: DIR_TEST_REPO,
}

if (!BOOTSTRAP) {
  //@ts-ignore
  configSFIP.config = {
    Bootstrap: []
  }
}

const sfip = await create(configSFIP);
console.log(await sfip.version())

while (true) {
  const avant = (new Date()).getTime()
  await sfip.block.put(Buffer.from(avant.toString()))
  console.log("time", (new Date()).getTime() - avant)
  // @ts-ignore
  console.log("peers", (await sfip.swarm.peers()).map(p=>p.addr.toString()))
}

[welcome[bot]]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review. In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment. Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[achingbrain]

Can you try disabling libp2p autodial and see if it still happens?

import { create } from 'ipfs-core'

const node = await create({
  libp2p: {
    connectionManager: {
      autodial: false
    }
  }
})

[julienmalard]

Thanks @achingbrain ! I just updated the script to test as you asked:

#!/usr/bin/env node --experimental-specifier-resolution=node
import { create } from "ipfs";
import { Options } from "ipfs-core/types"
import rm from "rimraf";

const DIR_TEST_REPO = "./testrepo";
rm.sync(DIR_TEST_REPO);

const BOOTSTRAP = true
const AUTODIAL = false

const configSFIP: Options = {
  repo: DIR_TEST_REPO,
}

if (!BOOTSTRAP) {
  //@ts-ignore
  configSFIP.config = {
    Bootstrap: []
  }
}

if (!AUTODIAL) {
  configSFIP.libp2p = {
    connectionManager: {
      autoDial: false
    }
  }
}

const sfip = await create(configSFIP);
console.log(await sfip.version())

const temps = [];
for (const i of [...Array(15).keys()]) {
  console.log(i)
  const avant = (new Date()).getTime()
  await sfip.block.put(Buffer.from(avant.toString()))
  temps.push((new Date()).getTime() - avant)
  // @ts-ignore
  // console.log("peers", (await sfip.swarm.peers()).map(p=>p.addr.toString()))
}
console.log("Temps moyen :", temps.reduce((total, current) => {
        return total + current;
    }, 0)/temps.length)

[julienmalard]

And here are the results:

BOOTSTRAP = true; AUTODIAL = true : Mean time: 2394 ms BOOTSTRAP = false; AUTODIAL = true : Mean time: 53 ms BOOTSTRAP = true; AUTODIAL = false : Mean time: 53 ms

[achingbrain]

This will be fixed by https://github.com/libp2p/js-libp2p/pull/1397 (likely in the next libp2p release, 0.40.x) so you may want to leave autoDial disabled until that ships. When it does you should turn it back on though:

The autoDial setting controls two behaviours (I know, it should only control one 😢 ) - the first is one you want, which is if the libp2p Connection Manager gets below minConnections, it'll start trying to connect to peers from the peer store to ensure it's still connected to the network.

The second behaviour is that when new peers are discovered it'll try to dial them immediately - this is expensive in terms of CPU and causes the slowdown you've observed. It was needed when peer discoverability was more of a challenge but now DHT client mode is enabled by default we don't really need it any more.

[julienmalard]

Thanks! That fixes it.

[whizzzkid]

closing this as fixed.