filecoin-plus-large-datasets icon indicating copy to clipboard operation
filecoin-plus-large-datasets copied to clipboard

[DataCap Application] KyFromAbove on AWS

Open tmzxi opened this issue 1 year ago • 41 comments

Data Owner Name

Kentucky Division of Geographic Information

What is your role related to the dataset

Data Preparer

Data Owner Country/Region

United States

Data Owner Industry

Utilities

Website

https://kyfromabove.ky.gov

Social Media

[email protected]

Total amount of DataCap being requested

2PiB

Expected size of single dataset (one copy)

130T

Number of replicas to store

10

Weekly allocation of DataCap requested

100TiB

On-chain address for first allocation

f1rmyhpf2do3dcl4w7vz273rm2mkitdrzfzi27zhy

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

  • [ ] Use Custom Multisig

Identifier

No response

Share a brief history of your project and organization

Founded in 2021, Digital Future Platform is committed to empowering the digital culture and art industry with technology, while promoting the transformation of traditional artists, boosting new artists to create value, and making digital art collection a trend. The artworks and goods displayed on the Digital collection platform are specific works uniquely identified and digitized by NFT blockchain technology, including but not limited to quadratic, video, music, pictures, digital paintings, 2D drawings, 3D models and other types of art works

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

https://registry.opendata.aws/kyfromabove/

The KyFromAbove initiative is focused on building and maintaining a current basemap for Kentucky that can meet the needs of its users at the state, federal, local, and regional level. A common basemap, including current color leaf-off aerial photography and elevation data (LiDAR), reduces the cost of developing GIS applications, promotes data sharing, and add efficiencies to many business processes. All basemap data acquired through this effort is being made available in the public domain. KyFromAbove acquires aerial imagery and LiDAR during leaf-off conditions in the Commonwealth. The imagery typically ranges from 6-inches to 3-inches in resolution and is available from the kyfromabove Amazon S3 bucket in a Cloud Optimized GeoTiff format. LiDAR data acquired by the program is USGS Quality Level 2 (QL2). Digital Elevation Models (DEMs) at a 2-foot resolution, point cloud data in an LAS format, spot elevations in a geopackage format, and contours in a geopackage format are also available from the kyfromabove Amazon S3 bucket. KyFromAbove LiDAR and imagery data resources are managed in a Kentucky-specific 5000x5000 foot grid (FIPS:1600) (EPSG:3089). More details about the program can be found at (https://kyfromabove.ky.gov/)

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (Country/Region)

China

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

Singularity

singularity prep create
singularity prep  append
singularity repl start

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

According to my inquiries, there are currently no organizations or individuals that store this data on the filecoin network

Please share a sample of the data

aws s3 ls --no-sign-request s3://kyfromabove/ --recursive --human-readable --summarize

Total Objects: 3158843
   Total Size: 130.9 TiB

Confirm that this is a public dataset that can be retrieved by anyone on the Network

  • [X] I confirm

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America

How will you be distributing your data to storage providers

HTTP or FTP server, Shipping hard drives

How do you plan to choose storage providers

Slack, Filmine

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

Hainan future cloud Technology Co., LTD	f02519323 	ningxia(CN)
topblocks	                        f02301	        New Jersey(US)
topblocks	                        f03223	        New Jersey(US)
topblocks	                        f0143858	New Jersey(US)
nick	                                f01969779 	New Jersey(US)
STRAITDEER PTE. LTD.	                f02093396	Singapore
STRAITDEER PTE. LTD.	                f02259777 	Singapore

We will be looking for more sp

How do you plan to make deals to your storage providers

Boost client, Lotus client, Singularity

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

tmzxi avatar Nov 17 '23 06:11 tmzxi

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

  • Have you prepared enough token for sector pledge?
  • Per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

Sunnyiscoming avatar Nov 17 '23 14:11 Sunnyiscoming

@Sunnyiscoming -We are not sp, and the sp we contacted assured us that there were enough tokens for the pledge. - I Submitted, please check, thank you

tmzxi avatar Nov 17 '23 19:11 tmzxi

@Sunnyiscoming I have submitted Fil+ registration form,If you need any more information, please let me know. Thank you

tmzxi avatar Nov 21 '23 01:11 tmzxi

@tmzxi can you send me the contact information of your SP's ?

herrehesse avatar Nov 21 '23 14:11 herrehesse

@herrehesse Hi ,Are you an official?, I have submitted Fil+ registration form,There's contact information in there,You can check it out,Thank you

tmzxi avatar Nov 22 '23 03:11 tmzxi

@tmzxi No, we want to see the list in here so I can check the reputation and history of the SPs.

herrehesse avatar Nov 22 '23 07:11 herrehesse

@herrehesse this is sp list, please check,If there is an sp that does not comply with the rules, we will find a new sp to replace it thank you

Hainan future cloud Technology Co., LTD f02519323 ningxia(CN) topblocks f02301 New Jersey(US) topblocks f03223 New Jersey(US) topblocks f0143858 New Jersey(US) nick f01969779 New Jersey(US) STRAITDEER PTE. LTD. f02093396 Singapore STRAITDEER PTE. LTD. f02259777 Singapore

tmzxi avatar Nov 22 '23 07:11 tmzxi

@tmzxi the Sps should come from at least four different continents.

cryptoAmandaL avatar Nov 22 '23 15:11 cryptoAmandaL

@cryptoAmandaL Hi,I see the Fil+ registration form, the requirement is 3 different regions, and we meet it now. If sp of more regions is needed, I will continue to look for more sp @Sunnyiscoming Can you help with the approval? thank you

fil+ form

tmzxi avatar Nov 23 '23 08:11 tmzxi

@tmzxi the SP list has extremely bad reputation. Not supportive.

herrehesse avatar Nov 23 '23 13:11 herrehesse

@Sunnyiscoming Hi,Sunnyiscoming,Can we continue? thank you

tmzxi avatar Nov 28 '23 10:11 tmzxi

I will follow up this application. Please wait for a moment.

Sunnyiscoming avatar Nov 28 '23 13:11 Sunnyiscoming

@tmzxi have you found new SP's to replace? Current ones have a bad track record.

herrehesse avatar Nov 30 '23 08:11 herrehesse

@Sunnyiscoming thank you

tmzxi avatar Nov 30 '23 10:11 tmzxi

Datacap Request Trigger

Total DataCap requested

2PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1rmyhpf2do3dcl4w7vz273rm2mkitdrzfzi27zhy

Sunnyiscoming avatar Nov 30 '23 13:11 Sunnyiscoming

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1rmyhpf2do3dcl4w7vz273rm2mkitdrzfzi27zhy

DataCap allocation requested

50TiB

Id

c5e43779-509c-4d7d-89ba-d69022b32bda

SP List provided: [{"providerID": "f02519323", "City": "ningxia", "Country": "CN", "SPOrg","Hainan future cloud Technology Co., LTD"}, {"providerID": "f02301", "City": "New Jersey", "Country": "US","SPOrg","topblocks"}, {"providerID": "f03223", "City": "New Jersey", "Country": "US","SPOrg","topblocks"}, {"providerID": "f0143858", "City": "New Jersey", "Country": "US","SPOrg","topblocks"}, {"providerID": "f01969779", "City": "New Jersey", "Country": "US","SPOrg","nick"}, {"providerID": "f02093396", "City": "N/A", "Country": "Singapore","SPOrg","STRAITDEER PTE. LTD. "}, {"providerID": "f02259777", "City": "N/A", "Country": "Singapore","SPOrg","STRAITDEER PTE. LTD. "},]

Sunnyiscoming avatar Nov 30 '23 14:11 Sunnyiscoming

Looks good, willing to provide support in the first round, note subsequent data distribution

zcfil avatar Dec 05 '23 10:12 zcfil

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecp3dnumvm6fvv37ez2juj4esoksmzfodug3rr2s7hyipgiwqeuva

Address

f1rmyhpf2do3dcl4w7vz273rm2mkitdrzfzi27zhy

Datacap Allocated

50.00TiB

Signer Address

f1cjzbiy5xd4ehera4wmbz63pd5ku4oo7g52cldga

Id

c5e43779-509c-4d7d-89ba-d69022b32bda

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecp3dnumvm6fvv37ez2juj4esoksmzfodug3rr2s7hyipgiwqeuva

zcfil avatar Dec 05 '23 10:12 zcfil

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacearvuijawynml6dfylzpepiuppbl253bvvymkq4scollazvmlq3rw

Address

f1rmyhpf2do3dcl4w7vz273rm2mkitdrzfzi27zhy

Datacap Allocated

50.00TiB

Signer Address

f1ui5iy4mmkxjbw7752omiwp2ols2fzv4thrayagi

Id

c5e43779-509c-4d7d-89ba-d69022b32bda

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacearvuijawynml6dfylzpepiuppbl253bvvymkq4scollazvmlq3rw

maxvint avatar Dec 07 '23 00:12 maxvint

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

github-actions[bot] avatar Dec 17 '23 01:12 github-actions[bot]

Please keep it open,Thank you

tmzxi avatar Dec 17 '23 02:12 tmzxi

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

github-actions[bot] avatar Dec 28 '23 01:12 github-actions[bot]

Please keep it open,Thank you

tmzxi avatar Dec 28 '23 01:12 tmzxi

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

github-actions[bot] avatar Jan 08 '24 01:01 github-actions[bot]

Please keep it open,Thank you

tmzxi avatar Jan 08 '24 01:01 tmzxi

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

github-actions[bot] avatar Jan 19 '24 01:01 github-actions[bot]

Please keep it open,Thank you

tmzxi avatar Jan 19 '24 02:01 tmzxi

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

github-actions[bot] avatar Jan 30 '24 01:01 github-actions[bot]