[Bug]: `drive.get(path)` get stuck

[vforvilela]

Hyperdrive is a wrapper around two Hypercores: one for storing file metadata [A], and the other to store file contents [B], as described here. To access a specific file content we call drive.get(path).

1. If it can access [A] AND [B], it returns the file content.
2. If it cannot access neither [A] NOR [B], it returns null.
3. But if it can access [A] but cannot access [B] it get stuck.

In the code bellow, DRIVE_PUBKEY is a drive with a ./test.txt file and SEEDER_PUBKEY is a peer seeding this drive.

const store = new Corestore(RAM)
const drive = new Hyperdrive(store, DRIVE_PUBKEY)
const dht = new DHT()

const conn = dht.connect(SEEDER_PUBKEY)
store.replicate(conn)

---

Now, if I try to get the content without waiting for a connection, it returns null (as expected in scenery 2)

const result = await drive.get('./test.txt').then((buf) => buf && b4a.toString(buf))
console.log(`Content: ${result}`)

Content: null

---

If I wait the connection, it returns the file content (as expected in scenery 1)

conn.once('open', async () => {
  const result = await drive.get('./test.txt').then((buf) => buf && b4a.toString(buf))
  console.log(`Entry content: ${result}`)
})

Content: { testing: 1 }

---

If I disconnect before I try get the content, it returns null again (as expected in scenery 2)

conn.once('open', async () => {
  await dht.destroy()
  
  const result = await drive.get('./test.txt').then((buf) => buf && b4a.toString(buf))
  console.log(`Entry content: ${result}`)
})

Content: null

---

And finally: If I first get the file metadata, then disconnect, then try to get the file content... it get stuck forever (as described in scenery 2)

conn.once('open', async () => {
  for await (const entry of drive.entries()) {
    console.log(`Entry path: ${entry.key}`)
  }

  await dht.destroy()

  // Get stuck here forever 👇
  const result = await drive.get('./test.txt').then((buf) => buf && b4a.toString(buf))
  console.log(`Entry content: ${result}`)
})

This issue happen to me not because I lost the connection right after getting the metadata and right before de content as shown in the example above (almost impossible situation). I had the metadata cached for days, and when I tried to access the content it found no peers serving the data.

[LuKks]

Can you make a full gist that I just run and reproduces the issue?

Btw, probably this doesn't fixes your issue but still a good practice to use findingPeers, something like this:

const socket = dht.connect(pk)
socket.on('error', console.error)

const done = store.findingPeers()
waitForSocketFlush(socket).then(done, done)
store.replicate(socket)

function waitForSocketFlush (socket) {
  return new Promise((resolve, reject) => {
    socket.on('open', done)
    socket.on('close', done)
    socket.on('error', done)

    function done (error) {
      socket.off('open', done)
      socket.off('close', done)
      socket.off('error', done)

      if (error) reject(error)
      else resolve()
    }
  })
}

[LuKks]

Hyperdrive also has findingPeers() as well

[LuKks]

Try installing these, and try again:

npm i holepunchto/hyperdrive-next#add-get-options
npm i holepunchto/hyperblobs#add-wait-opt

const buffer = await drive.get('./test.txt', { wait: false })

[vforvilela]

Can you make a full gist that I just run and reproduces the issue?

import DHT from '@hyperswarm/dht'
import Corestore from 'corestore'
import Hyperdrive from 'hyperdrive'
import b4a from 'b4a'
import RAM from 'random-access-memory'


// Alice
const alice_store = new Corestore(RAM)
const alice_drive = new Hyperdrive(alice_store)
await alice_drive.ready()

if (alice_drive.version == 1) {
  await alice_drive.put('./test.txt', Buffer.from('{ "testing": 1 }'))
}

const alice_keyPair = DHT.keyPair()
const alice_dht = new DHT()
const alice_server = alice_dht.createServer((conn) => {
  alice_store.replicate(conn)
})
await alice_server.listen(alice_keyPair)


// Bob
const bob_store = new Corestore(RAM)
const bob_drive = new Hyperdrive(bob_store, alice_drive.key)
const bob_dht = new DHT()

const bob_conn = bob_dht.connect(alice_keyPair.publicKey)
bob_store.replicate(bob_conn)

bob_conn.once('open', async () => {
  // Get the metadata
  for await (const entry of bob_drive.entries()) {
    console.log(`Entry path: ${entry.key}`)
  }

  // This line simulates the lost of connection with the seeder
  // Both local or remote failures generate the same issue
  // await alice_dht.destroy() also works 
  await bob_dht.destroy()

  // Get stuck here for ever 👇
  const result = await bob_drive.get('./test.txt').then((buf) => buf && b4a.toString(buf))
  console.log(`Entry content: ${result}`)
})