Implement IPFS multiple gateway support with automatic fallback
Overview
This PR implements support for multiple IPFS gateway URLs with automatic fallback logic in the DAppNode Manager toolkit. The system now tries multiple gateways in order until content is successfully fetched, providing better resilience against gateway failures.
Problem
Previously, the IPFS content fetching relied on a single gateway URL. If that gateway was down or slow, the entire IPFS functionality would fail. This created a single point of failure for package installation and content retrieval.
Solution
The implementation adds multiple gateway support with the following features:
🔄 Automatic Fallback Logic
- Multiple IPFS gateways are tried in sequence until one succeeds
- Local IPFS gateway is prioritized first when client target is "local", with remote as fallback
- Remote IPFS gateway is prioritized first when client target is "remote", with local as fallback
- Comprehensive error collection and reporting from all gateway attempts
🔧 API Changes
DappnodeRepository Constructor:
// Multiple gateways (new)
const repo = new DappnodeRepository([
"http://ipfs.dappnode:8080", // Local gateway (tried first)
"https://gateway.ipfs.dappnode.io" // Remote gateway (fallback)
], provider);
// Single gateway (backward compatible)
const repo = new DappnodeRepository("https://gateway.ipfs.dappnode.io", provider);
New Gateway URL Management:
// Returns array of URLs with proper priority
const urls = getIpfsUrls(); // ["http://ipfs.dappnode:8080", "https://gateway.ipfs.dappnode.io"]
// Backward compatibility maintained
const singleUrl = getIpfsUrl(); // "http://ipfs.dappnode:8080" (first URL)
🛡️ Error Handling
When all gateways fail, detailed error information is provided:
Failed to fetch content from all gateways. Errors:
Gateway http://nonexistent.example.com: fetch failed;
Gateway https://gateway.ipfs.dappnode.io: 404 Not Found
✅ Backward Compatibility
- All existing code continues to work unchanged
- Single URL parameters are automatically converted to single-element arrays
- Existing
getIpfsUrl()function maintained for compatibility
Key Changes
-
Modified
DappnodeRepositoryto accept both single URLs and arrays of URLs -
Added fallback logic in
getAndVerifyContentFromGateway()andlist()methods -
Created
getIpfsUrls()function that returns prioritized array of gateway URLs - Updated main initialization to use multiple gateways
- Added comprehensive unit tests for the new multi-gateway functionality
Benefits
- 🔄 Better Resilience: System continues working if one gateway fails
- ⚡ Improved Performance: Can prioritize faster/local gateways first
- 🔧 Enhanced Flexibility: Easy to add/remove/reorder gateways
- 🔒 Full Compatibility: No breaking changes to existing code
- 👁️ Transparent Operation: Fallback logic is automatic and invisible to calling code
Testing
- ✅ All existing tests pass
- ✅ New unit tests validate multi-gateway API
- ✅ Full project builds successfully
- ✅ Backward compatibility verified
- ✅ Linting passes without issues
This implementation provides a robust foundation for IPFS content fetching that gracefully handles gateway failures while maintaining full backward compatibility with existing DAppNode installations.
[!WARNING]
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
binaries.soliditylang.org
- Triggering command:
/usr/local/bin/node ./build/cli.bundled.js obtain --solc=0.5.16(dns block)ethereum.github.io
- Triggering command:
/usr/local/bin/node ./build/cli.bundled.js obtain --solc=0.5.16(dns block)gateway.ipfs.dappnode.io
- Triggering command:
node /tmp/demo-simple.js(dns block)- Triggering command:
node /tmp/demo-single.js(dns block)nonexistent.example.com
- Triggering command:
node /tmp/demo-simple.js(dns block)ns.dappnode.io
- Triggering command:
/usr/local/bin/node --experimental-specifier-resolution=node --import=tsx/esm /home/REDACTED/work/DNP_DAPPMANAGER/DNP_DAPPMANAGER/packages/utils/node_modules/mocha/lib/cli/cli.js ./test/unit --no-config --recursive --passWithNoTests --no-package --color --exit --extension ts --require dotenv/config --diff --reporter spec --slow 75 --timeout 2000 --ui bdd --watch-ignore node_modules --watch-ignore .git(dns block)relay.trufflesuite.com
- Triggering command:
/usr/local/bin/node ./build/cli.bundled.js obtain --solc=0.5.16(dns block)solc-bin.ethereum.org
- Triggering command:
/usr/local/bin/node ./build/cli.bundled.js obtain --solc=0.5.16(dns block)If you need me to access, download, or install something from one of these locations, you can either:
- Configure Actions setup steps to set up my environment, which run before the firewall is enabled
- Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)
Original prompt
This section details on the original issue you should resolve
<issue_title>Implement IPFS default to local and fallback on remote</issue_title> <issue_description>In the toolkit submodule, allow defining multiple IPFS gateway URLs. Always use the first one provided to start fetching the content and keep moving to the next gateway url if there is an error.
Since the URL are from IPFS gateway, use the exposed enpoint to verify first if in any of the gateway URL is present the content of the ipfs cid to be retrieved. Check official documentation from https://docs.ipfs.tech/reference/http/gateway/
Implement in the index in the dappmanager module the initialization of the instance with the ipfs gateway URLs </issue_description>
Comments on the Issue (you are @copilot in this section)
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.
🚀 Deployed on https://69281206a72b21d0ed945e72--objective-borg-9eeae5.netlify.app