Unified Provide Interface for Content Routers

[guillaumemichel]

Checklist

• [X] My issue is specific & actionable.
• [X] I am not suggesting a protocol enhancement.
• [X] I have searched on the issue tracker for my issue.

Description

Problem Statement:

Currently, Kubo is responsible for managing the DHT's provide and reprovide operations. However, with the evolution of Content Routers beyond just the DHT, it's evident that the existing mechanism is not optimal. The reasons are:

1. The reprovide strategy Kubo uses was mainly designed for DHTs and is not always suitable for the newer Content Routers such as IPNI which is using a different advertising mechanism.
2. The DHT cannot optimize its reprovide strategy as it doesn't have a direct insight into the content that needs to be republished.

Proposed Solution

To better streamline the providing mechanism across different content routers, we propose a unified interface that shifts the responsibility from Kubo to the individual content routers. The proposed interface includes:

• StartProvide(CIDs): Instructs the content router to begin advertising that the Kubo node is storing the specified CIDs. This advertisement (or republishing) should continue until a StopProvide is invoked for these CIDs.
• StopProvide(CIDs): Commands the content router to cease the advertisement for the given CIDs.
• ListProvides: Returns the list of CIDs currently being advertised by the content router.

Benefits

1. Flexibility: With a generic interface, different content routers can easily integrate with Kubo without being tied to a DHT-specific strategy.
2. Optimization Opportunities: Allows the DHT and other content routers to implement their own specific provide strategies, optimized for their use cases. In the DHT, this change of interface is necessary to implement Reprovide Sweep (IPFS Thing 2023 presentation), allowing a resource efficient reprovide strategy, enabling large content providers to advertise content to the DHT.
3. Clarity of Responsibilities: Removing the responsibility from Kubo makes the system modular, allowing each component to focus on its core functionality.

Feedback and Collaboration

The proposed interface is just a draft for now. The goal of this issue is to gather feedback and start a public discussion about specific interface needs for different content routers, especially IPNI and the DHT. This issue will probably be followed up by an IPIP in ipfs/specs, once we have listed the requirements of all (known) Content Routers.

References

• IPFS Thing 2023 Unconf session notes

cc: @masih @ischasny @aschmahmann @Jorropo @dennis-tra @iand