filecoin-plus-large-datasets [DataCap Application] NASA/USGS

Data Owner Name

NASA/USGS

What is your role related to the dataset

Data Preparer

Data Owner Country/Region

United States

Data Owner Industry

Environment

Website

https://www.usgs.gov/landsat-missions/landsat-9

Social Media

[at]USGSLandsat (twitter)

Total amount of DataCap being requested

1500 TiB

Expected size of single dataset (one copy)

275 TiB

Number of replicas to store

5

Weekly allocation of DataCap requested

250 TiB

On-chain address for first allocation

f1uwzfw6hghqf6js4773p62onzvqnupcqdxbkhhvq

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

[ ] Use Custom Multisig

Identifier

n/a

Share a brief history of your project and organization

The EASIER Data initiative kicked off late this summer and is a two year project in collaboration with the Filecoin Foundation for the Decentralized Web to build pipelines for storing and extracting geospatial data on Filecoin and IPFS. These pipelines will be prototyped and demonstrated using one year of Landsat 9 satellite data, which is estimated at about 275TB per replication. We originally opened a request, but there was an issue and it was suggested that we open a new one.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

n/a

Describe the data being stored onto Filecoin

Landsat9 satellite remote sensing data for the year 2019

Where was the data currently stored in this dataset sourced from

Other

If you answered "Other" in the previous question, enter the details here

n/a

If you are a data preparer, what is your location (City and Country)

n/a

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

n/a

If you are not preparing the data, who will prepare the data? (Provide name and business)

James Hoang - Piknik

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

One copy of the data is currently stored with Piknik, but the initial data cap request has become stale and we have not been able to store the additional replications.

Please share a sample of the data

https://www.usgs.gov/landsat-missions/landsat-9

Confirm that this is a public dataset that can be retrieved by anyone on the Network

Yes

If you chose not to confirm, what was the reason

n/a

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

Permanently

In which geographies do you plan on making storage deals

North America, Europe, Asia other than Greater China, Australia (continent)

How will you be distributing your data to storage providers

HTTP or FTP server

How do you plan to choose SP

Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

n/a

If you already have a list of storage providers to work with, fill out their names and provider IDs below

n/a

How do you plan to make deals to your storage providers

Others/custom tool

If you answered "Others/custom tool" in the previous question, enter the details here

n/a

Can you confirm that you will follow the Fil+ guideline

Yes

Application created via filplus.storage

Aug 17 '23 16:08 TaylorOshan

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!

Aug 17 '23 16:08 large-datacap-requests[bot]

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Aug 17 '23 16:08 large-datacap-requests[bot]

The Social Impact team at Filecoin Foundation works closely with this team to support and enable them develop a decentralized cyber infrastructure for efficiently, accessibly, and sustainably onloading, analyzing, and extracting large amounts of spatial data on the filecoin storage network.

Aug 17 '23 17:08 Sankara-Jefferson

Previous application at #995 had a technical issue which required opening a new application.

Aug 18 '23 00:08 jamerduhgamer

Have you prepared enough token for sector pledge?
Best practice for storing large datasets includes ideally, storing it in 3 or more regions, with 4 or more storage provider operators or owners.You should list Miner ID, Business Entity, Location of sps you will cooperate with.
Per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be triggered for notary review. Let us know if you have any questions.

Aug 18 '23 14:08 Sunnyiscoming

Hi @Sunnyiscoming!

Yes we should have enough tokens for sector pledge.
We are looking for more SPs at the moment. As shown above we have:
- f01851060 - PiKNiK - Las Vegas
- f01392893 - NexGen - Amsterdam
- Potential SP in Australia
- 2nd SP in Europe
- SP in Asia
I will work with @TaylorOshan to get KYC verified.

Aug 18 '23 23:08 jamerduhgamer

You should list nodes of more than 4 sps here. Have you completed the following Fil+ registration form?

Aug 23 '23 13:08 Sunnyiscoming

checker:manualTrigger

Aug 24 '23 08:08 zcfil

DataCap and CID Checker Report[^1]

There is no previous allocation for this issue.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

Aug 24 '23 08:08 filplus-checker-app[bot]

DataCap and CID Checker Report[^1]

There is no previous allocation for this issue.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

Aug 24 '23 08:08 filplus-checker-app[bot]

Can you provide more detailed information on other storage vendors participating in this program, such as a list of the SPs currently in contact with? Does the SPS you have chosen support data retrieval?

Aug 24 '23 08:08 zcfil

Any update here?

Aug 28 '23 13:08 Sunnyiscoming

Any update here?

Aug 30 '23 14:08 Sunnyiscoming

We only have the SPs above participating in the program so far. The SPs do support data retrieval.

We are currently looking for more SPs to onboard the data.

Aug 31 '23 22:08 jamerduhgamer

4 or more storage providers should be provided.

Sep 01 '23 16:09 Sunnyiscoming

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1uwzfw6hghqf6js4773p62onzvqnupcqdxbkhhvq

DataCap allocation requested

250TiB

Id

cfbc26f0-b122-44ed-b9c9-7e331c1fe43b

Sep 03 '23 18:09 large-datacap-requests[bot]

Please list the SPs you are pre-collaborating with and the regions.

Sep 05 '23 03:09 zcfil

Hi, Filecoin Foundation notary here.

This is a request from a known Filecoin/IPFS user, which is the EASIER project at the University of Maryland (the applicant is the project lead there). While the data is open data from NASA, the project is to work out the best way to make the data usefully accessible from Filecoin for geodata purposes.

I'm willing to support starting up the initial data allocation while UMD sort out their uploading process and attract other SPs.

Happy to walk through the background with other notaries here. To give an example of the work UMD is doing, here's their work on distributing Intelsat 9 data via the public IPFS network.

Sep 06 '23 01:09 dannyob

Almost none of the report are passable, this is a poor result of sealing. Can't support it.

Sep 09 '23 01:09 AthSmith

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacednpoz4lllj43wm4a6pilb7zmfvem6c4tuds5ak4pk7x2tjfdhbem

Address

f1uwzfw6hghqf6js4773p62onzvqnupcqdxbkhhvq

Datacap Allocated

250.00TiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

Id

cfbc26f0-b122-44ed-b9c9-7e331c1fe43b

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacednpoz4lllj43wm4a6pilb7zmfvem6c4tuds5ak4pk7x2tjfdhbem

Sep 12 '23 12:09 dannyob

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebgmc6pwc3wzgdft2ckcmkmdbr6nmiyc2mgwhahtxmt3utqknifqe

Address

f1uwzfw6hghqf6js4773p62onzvqnupcqdxbkhhvq

Datacap Allocated

250.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

cfbc26f0-b122-44ed-b9c9-7e331c1fe43b

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebgmc6pwc3wzgdft2ckcmkmdbr6nmiyc2mgwhahtxmt3utqknifqe

Sep 12 '23 17:09 cryptowhizzard

Hi, Filecoin Foundation notary here.

This is a request from a known Filecoin/IPFS user, which is the EASIER project at the University of Maryland (the applicant is the project lead there). While the data is open data from NASA, the project is to work out the best way to make the data usefully accessible from Filecoin for geodata purposes.

I'm willing to support starting up the initial data allocation while UMD sort out their uploading process and attract other SPs.

Happy to walk through the background with other notaries here. To give an example of the work UMD is doing, here's their work on distributing Intelsat 9 data via the public IPFS network.

I checked their application and it looks good. Same for me as i am willing to see them start up and have usefull and real data onboarded to the network.

Sep 12 '23 17:09 cryptowhizzard

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Sep 23 '23 01:09 github-actions[bot]

Comment to keep LDN open. Sealing has been on-going.

Sep 26 '23 17:09 jamerduhgamer

DataCap Allocation requested