filecoin-plus-large-datasets [DataCap Application] DAG House

Data Owner Name

DAG House and the users of our tools

What is your role related to the dataset

Data onramp entity that provides data onboarding services to multiple clients

Data Owner Country/Region

United States

Data Owner Industry

Web3 / Crypto

Website

https://web3.storage/

Social Media

https://twitter.com/web3storage
https://www.linkedin.com/company/dag-house

Total amount of DataCap being requested

15PiB

Expected size of single dataset (one copy)

500TiB and growing

Number of replicas to store

10

Weekly allocation of DataCap requested

250TiB

On-chain address for first allocation

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Data Type of Application

Public, Open Commercial/Enterprise

Custom multisig

[ ] Use Custom Multisig

Identifier

No response

Share a brief history of your project and organization

DAG House was founded in 2021 as a team inside Protocol Labs to develop tools to make it easy for developers and end users to host content addressed data and store the data on Filecoin. Since then our two flagship products, NFT.Storage (Internet Archive of NFTs) and Web3.Storage (developer storage platform) are used by many prominent projects and companies in Web3 and outside of it.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

This project is associated with Protocol Labs. Projects have plans for independence.

Describe the data being stored onto Filecoin

User uploads that meet or adhere to web3.storage's terms of service (https://web3.storage/terms).

Where was the data currently stored in this dataset sourced from

Other

If you answered "Other" in the previous question, enter the details here

User uploads (generally for their web3 apps)

If you are a data preparer. What is your location (Country/Region)

None

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

No response

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

The goal is to be able to use the Filecoin copies as the only available copies on the network (rather than also storing the data on centralized infra), which requires things like retrieval to have high performance and global availability. As a result, some parts of the dataset have already been stored but not to the replication limit with other Datacap apps (https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1838, https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/2110).

Please share a sample of the data

This platform serves all different kinds of media, including images, files, and videos. Some examples:

https://ipfs.io/ipfs/bafybeid5jpdqzlb4tqsd6peoa7qstoxat3ovsg62wutyp4gnzqbqsggfsq
https://ipfs.io/ipfs/bafybeihity6bx24npzvvkzopjbat25ekefjwmnshe7rvldy72dxngzf644
https://ipfs.io/ipfs/bafybeicvcevx3ktiqjsfwnjguu4lnzejlhgb35brayuod5xdtn7demfdhe

Confirm that this is a public dataset that can be retrieved by anyone on the Network

[X] I confirm

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Daily

For how long do you plan to keep this dataset stored on Filecoin

Permanently

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, Africa, North America, South America, Europe, Australia (continent)

How will you be distributing your data to storage providers

HTTP or FTP server

How do you plan to choose storage providers

Others

If you answered "Others" in the previous question, what is the tool or platform you plan to use

We plan to use Spade for SP selection and deal execution. Spade is being developed by the Data Programs team at Protocol Labs (https://github.com/data-preservation-programs/spade) and supports storage client to SP matching based on requirements like geography, size, retrievability, etc.

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

Others/custom tool

If you answered "Others/custom tool" in the previous question, enter the details here

We plan to use Spade for deal execution to onboard data to Filecoin. Spade is being developed by the Data Programs team at Protocol Labs (https://github.com/data-preservation-programs/spade).

Spade was initially servicing Slingshot deals and was referred to as the Evergreen Dealer.

Can you confirm that you will follow the Fil+ guideline

Yes

Sep 01 '23 13:09 dchoi27

Datacap Request Trigger

Total DataCap requested

15PiB

Expected weekly DataCap usage rate

250TiB

Client address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Sep 01 '23 14:09 simonkim0515

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

DataCap allocation requested

125TiB

Id

1a683d47-7859-4670-9afc-b28fb0611030

Sep 01 '23 14:09 large-datacap-requests[bot]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceawtxdf2sqbes7dp3s56jqt7qyabxxz5tkpzlncsghtqz3hvlfdq2

Address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Datacap Allocated

125.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

1a683d47-7859-4670-9afc-b28fb0611030

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceawtxdf2sqbes7dp3s56jqt7qyabxxz5tkpzlncsghtqz3hvlfdq2

Sep 02 '23 19:09 cryptowhizzard

I have been following and supporting the program before. I can continue to support it this time as well. Looking forward to excellent performance afterwards.

Sep 04 '23 01:09 liyunzhi-666

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacec7gqldor3rgjulokq63qrmvytxiu45sblysrmijrodkybmyu2usu

Address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Datacap Allocated

125.00TiB

Signer Address

f1pszcrsciyixyuxxukkvtazcokexbn54amf7gvoq

Id

1a683d47-7859-4670-9afc-b28fb0611030

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec7gqldor3rgjulokq63qrmvytxiu45sblysrmijrodkybmyu2usu

Sep 04 '23 01:09 liyunzhi-666

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

DataCap allocation requested

250TiB

Id

735ec2b8-7292-48e0-8bef-5f41eb534dff

Sep 08 '23 18:09 large-datacap-requests[bot]

checker:manualTrigger

Sep 09 '23 02:09 Patapon0702

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate: 2.22%
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 97.75% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Sep 09 '23 02:09 filplus-checker-app[bot]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecptph7iozs3gjlsgwde4vfsqikihobcf6sbnayvatdykicmwwgj4

Address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Datacap Allocated

250.00TiB

Signer Address

f1ho2liobpznr7llma6xcl7jtififsfvhdnudn4yy

Id

735ec2b8-7292-48e0-8bef-5f41eb534dff

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecptph7iozs3gjlsgwde4vfsqikihobcf6sbnayvatdykicmwwgj4

Sep 09 '23 02:09 Patapon0702

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecbnk3mre6r7obslof4tmextmcgjfimkxhh5bpmymt5ig7pvqbooa

Address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Datacap Allocated

250.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

735ec2b8-7292-48e0-8bef-5f41eb534dff

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecbnk3mre6r7obslof4tmextmcgjfimkxhh5bpmymt5ig7pvqbooa

Sep 10 '23 16:09 cryptowhizzard

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

DataCap allocation requested

500TiB

Id

2bf30b49-1406-48e8-889d-2bae95955d8d

Sep 17 '23 06:09 large-datacap-requests[bot]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecy5dojl23opqqhgey76fs5nxa45znrarwtzpnad55iyng5m3muxs

Address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Datacap Allocated

500.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

2bf30b49-1406-48e8-889d-2bae95955d8d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecy5dojl23opqqhgey76fs5nxa45znrarwtzpnad55iyng5m3muxs

Sep 18 '23 20:09 cryptowhizzard

checker:manualTrigger

Sep 18 '23 22:09 jamerduhgamer

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate: 0.74%
Overall Bitswap retrieval success rate:

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 50% of total datacap - f010202: 59.16%

Deal Data Replication

⚠️ 97.45% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Sep 18 '23 22:09 filplus-checker-app[bot]

The # of SPs sealing the dataset has increased. f010202 has more than 50% datacap but it is only ~46 TiBs.

Willing to support this next tranche of datacap.

Sep 18 '23 22:09 jamerduhgamer

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecmucctg7dnihxwcu7w2t73y6gmriixijhqz2zh64b6dopj3wnpko

Address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Datacap Allocated

500.00TiB

Signer Address

f1kqdiokoeubyse4qpihf7yrpl7czx4qgupx3eyzi

Id

2bf30b49-1406-48e8-889d-2bae95955d8d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecmucctg7dnihxwcu7w2t73y6gmriixijhqz2zh64b6dopj3wnpko

Sep 18 '23 22:09 jamerduhgamer

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedg7jwfsjetslzyboturqjrkerzikspcsgal6efhr5djjfozteqcw

Address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Datacap Allocated

500.00TiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

Id

2bf30b49-1406-48e8-889d-2bae95955d8d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedg7jwfsjetslzyboturqjrkerzikspcsgal6efhr5djjfozteqcw

Sep 22 '23 20:09 dannyob

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceb7ncd6iws2h53bjwzw226ezm2jgv4l6afkvmo5ox3n7rj26laihu

Address

f3se4aw3p3tlvggkhsymdg7qmbjnxu4uudjvar6c3scfwsge6voesbxff4lkoiqyjcl4yot7hxqmukhusxgurq

Datacap Allocated

500.00TiB

Signer Address

f1ho2liobpznr7llma6xcl7jtififsfvhdnudn4yy