filecoin-plus-large-datasets
filecoin-plus-large-datasets copied to clipboard
filecoin-project
[DataCap Application] <CAO> <NOAA GFS-HRRR-GEFS>
Data Owner Name
National Oceanic and Atmospheric Administration
What is your role related to the dataset
Dataset Owner
Data Owner Country/Region
Afghanistan
Data Owner Industry
Life Science / Healthcare
Website
https://registry.opendata.aws/noaa-gefs/ https://registry.opendata.aws/noaa-hrrr-pds https://registry.opendata.aws/noaa-gefs
Social Media
Website: https://www.noaa.gov/
Twitter: https://twitter.com/NOAA
Total amount of DataCap being requested
10PiB
Expected size of single dataset (one copy)
32g
Number of replicas to store
5
Weekly allocation of DataCap requested
1PiB
On-chain address for first allocation
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
Data Type of Application
Slingshot
Custom multisig
- [ ] Use Custom Multisig
Identifier
No response
Share a brief history of your project and organization
NOAA is an agency that enriches life through science. Their reach goes from the surface of the sun to the depths of the ocean floor as we work to keep the public informed of the changing environment around them.
The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3km grids with 3km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-h period adding further detail to that provided by the hourly data assimilation from the 13km radar-enhanced Rapid Refresh.
The Global Ensemble Forecast System (GEFS), previously known as the GFS Global ENSemble (GENS), is a weather forecast model made up of 21 separate forecasts, or ensemble members. The National Centers for Environmental Prediction (NCEP) started the GEFS to address the nature of uncertainty in weather observations, which is used to initialize weather forecast models. The GEFS attempts to quantify the amount of uncertainty in a forecast by generating an ensemble of multiple forecasts, each minutely different, or perturbed, from the original observations. With global coverage, GEFS is produced four times a day with weather forecasts going out to 16 days.
Is this project associated with other projects/ecosystem stakeholders?
Yes
If answered yes, what are the other projects/ecosystem stakeholders
No response
Describe the data being stored onto Filecoin
NOAA is an agency that enriches life through science. Their reach goes from the surface of the sun to the depths of the ocean floor as we work to keep the public informed of the changing environment around them.
The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3km grids with 3km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-h period adding further detail to that provided by the hourly data assimilation from the 13km radar-enhanced Rapid Refresh.
The Global Ensemble Forecast System (GEFS), previously known as the GFS Global ENSemble (GENS), is a weather forecast model made up of 21 separate forecasts, or ensemble members. The National Centers for Environmental Prediction (NCEP) started the GEFS to address the nature of uncertainty in weather observations, which is used to initialize weather forecast models. The GEFS attempts to quantify the amount of uncertainty in a forecast by generating an ensemble of multiple forecasts, each minutely different, or perturbed, from the original observations. With global coverage, GEFS is produced four times a day with weather forecasts going out to 16 days.
Where was the data currently stored in this dataset sourced from
AWS Cloud
If you answered "Other" in the previous question, enter the details here
No response
If you are a data preparer. What is your location (Country/Region)
None
If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?
No response
If you are not preparing the data, who will prepare the data? (Provide name and business)
No response
Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.
No response
Please share a sample of the data
350-resule
2022-06-27 08:00:43 34 Bytes sst.20220626/rtgssthr_grb_0.5.grib2.idx
2022-06-28 08:00:22 2.2 MiB sst.20220627/rtgssthr_grb_0.083.grib2
2022-06-28 08:00:21 34 Bytes sst.20220627/rtgssthr_grb_0.083.grib2.idx
2022-06-28 08:00:21 3.7 MiB sst.20220627/rtgssthr_grb_0.083_awips.grib2
2022-06-28 08:00:21 34 Bytes sst.20220627/rtgssthr_grb_0.083_awips.grib2.idx
2022-06-28 08:00:21 122.8 KiB sst.20220627/rtgssthr_grb_0.5.grib2
2022-06-28 08:00:21 34 Bytes sst.20220627/rtgssthr_grb_0.5.grib2.idx
Total Objects: 64802983
Total Size: 1.7 PiB
357-resule
2020-08-25 17:22:47 398.4 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf13.grib2
2020-08-25 17:22:52 396.0 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf14.grib2
2020-08-25 17:22:52 394.6 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf15.grib2
2020-08-25 17:22:52 390.2 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf16.grib2
2020-08-25 17:23:08 387.1 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf17.grib2
2020-08-25 17:23:05 384.8 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf18.grib2
2021-09-28 03:48:22 31.6 KiB index.html
Total Objects: 43282174
Total Size: 2.1 PiB
347-resule
2023-02-15 13:35:33 13.5 MiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f258.grib2
2023-02-15 13:35:31 1.2 KiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f258.grib2.idx
2023-02-15 13:38:42 13.5 MiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f264.grib2
2023-02-15 13:37:38 1.2 KiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f264.grib2.idx
2023-02-15 13:40:59 13.5 MiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f270.grib2
2023-02-15 13:40:57 1.2 KiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f270.grib2.idx
2021-09-28 03:47:57 31.6 KiB index.html
Total Objects: 116639180
Total Size: 2.2 PiB
Confirm that this is a public dataset that can be retrieved by anyone on the Network
- [ ] I confirm
If you chose not to confirm, what was the reason
No response
What is the expected retrieval frequency for this data
Yearly
For how long do you plan to keep this dataset stored on Filecoin
1.5 to 2 years
In which geographies do you plan on making storage deals
Greater China, Asia other than Greater China, Africa, North America, South America
How will you be distributing your data to storage providers
Cloud storage (i.e. S3)
How do you plan to choose storage providers
Slack, Big Data Exchange
If you answered "Others" in the previous question, what is the tool or platform you plan to use
No response
If you already have a list of storage providers to work with, fill out their names and provider IDs below
f02050599 Charlie Thailand
f02063409 wangkunpeg Singapore
f02041085 TopValue Japan
f02096851 EdgeIPC Indonesia
f02047843 John Japan
How do you plan to make deals to your storage providers
Boost client, Lotus client
If you answered "Others/custom tool" in the previous question, enter the details here
No response
Can you confirm that you will follow the Fil+ guideline
Yes
Thanks for your request! Everything looks good. :ok_hand:
A Governance Team member will review the information provided and contact you back pretty soon.
![large-datacap-requests[bot] avatar](https://avatars.githubusercontent.com/in/116297?v=4 "Published by a pub.dev verified publisher"){kind=small}
- Per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.
Have you completed KYC? Contact email information and sp organization name has not been all completed in the form. Please resubmit the form with complete information. The Miner IDs and entity name and locaiton should be posted here.
yes f02050599 Charlie Thailand f02063409 wangkunpeg Singapore f02041085 TopValue Japan f02096851 EdgeIPC Indonesia f02047843 John Japan
This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.
-- Commented by Stale Bot.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.
-- Commented by Stale Bot.
@caoxinhe0108 no SP entities listed on your registration form. https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/2227#issuecomment-1772201357
Please update to confirm SP distribution
This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.
-- Commented by Stale Bot.
SP List provided: [{"providerID": "f02050599", "City": "XYZ", "Country": "Thailand", "SPOrg","Charlie"}, {"providerID": "f02063409", "City": "XYZ", "Country": "Singapore","SPOrg","wangkunpeg"}, {"providerID": "f02041085", "City": "XYZ", "Country": "Japan","SPOrg","TopValue"}, {"providerID": "f02096851", "City": "XYZ", "Country": "Indonesia","SPOrg","EdgeIPC"}, {"providerID": "f02047843", "City": "XYZ", "Country": "Japan","SPOrg","XYZ"},]
DataCap Allocation requested
Request number 5
Multisig Notary address
f02049625
Client address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
DataCap allocation requested
2PiB
Id
a621f3bf-015c-4b97-913c-b6f16272ae9d
Request Proposed
Your Datacap Allocation Request has been proposed by the Notary
Message sent to Filecoin Network
bafy2bzacecxpbc42ey4zrewls3uakuezceb4ucvev44gc3s5gcoh2wocqnimm
Address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
Datacap Allocated
2.00PiB
Signer Address
f12mckci3omexgzoeosjvstcfxfe4vqw7owdia3da
Id
a621f3bf-015c-4b97-913c-b6f16272ae9d
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecxpbc42ey4zrewls3uakuezceb4ucvev44gc3s5gcoh2wocqnimm
Request Approved
Your Datacap Allocation Request has been approved by the Notary
Message sent to Filecoin Network
bafy2bzacebt5yrchtdulnmdhbcgqs6frxq4rwlqqdicyamv5ml6qqsdvzpxgc
Address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
Datacap Allocated
2.00PiB
Signer Address
f1tbd632f6w62glfaf7wjpimacbnjiz26poyoes2q
Id
a621f3bf-015c-4b97-913c-b6f16272ae9d
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebt5yrchtdulnmdhbcgqs6frxq4rwlqqdicyamv5ml6qqsdvzpxgc
DataCap Allocation requested
Request number 5
Multisig Notary address
f02049625
Client address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
DataCap allocation requested
2PiB
Id
aaa0c954-15fb-44f4-9321-bbe5d711c72a
DataCap and CID Checker Report Summary[^1]
Storage Provider Distribution
✔️ Storage provider distribution looks healthy.
Deal Data Replication
✔️ Data replication looks healthy.
Deal Data Shared with other Clients[^3]
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Full report
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.
![filplus-checker-app[bot] avatar](https://avatars.githubusercontent.com/in/267472?v=4 "Published by a pub.dev verified publisher"){kind=small}
Request Proposed
Your Datacap Allocation Request has been proposed by the Notary
Message sent to Filecoin Network
bafy2bzacea5amo2bzg5khwpcuyz7eser7n6bcm776qekygo5jabrk3wilvzwm
Address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
Datacap Allocated
2.00PiB
Signer Address
f1jvvltduw35u6inn5tr4nfualyd42bh3vjtylgci
Id
aaa0c954-15fb-44f4-9321-bbe5d711c72a
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea5amo2bzg5khwpcuyz7eser7n6bcm776qekygo5jabrk3wilvzwm
Request Approved
Your Datacap Allocation Request has been approved by the Notary
Message sent to Filecoin Network
bafy2bzaceaviwttox2cej3zri3obbmfjcmwzy2ullv3rbe3wd7mpumixeen2u
Address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
Datacap Allocated
2.00PiB
Signer Address
f1q6bpjlqia6iemqbrdaxr2uehrhpvoju3qh4lpga
Id
aaa0c954-15fb-44f4-9321-bbe5d711c72a
You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaviwttox2cej3zri3obbmfjcmwzy2ullv3rbe3wd7mpumixeen2u
DataCap and CID Checker Report Summary[^1]
Storage Provider Distribution
✔️ Storage provider distribution looks healthy.
Deal Data Replication
✔️ Data replication looks healthy.
Deal Data Shared with other Clients[^3]
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Full report
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.
DataCap Allocation requested
Request number 7
Multisig Notary address
f02049625
Client address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
DataCap allocation requested
2PiB
Id
255d352e-7251-4e93-b3ce-06d9ae9415d8
DataCap and CID Checker Report Summary[^1]
Storage Provider Distribution
✔️ Storage provider distribution looks healthy.
Deal Data Replication
✔️ Data replication looks healthy.
Deal Data Shared with other Clients[^3]
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Full report
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.
Request Proposed
Your Datacap Allocation Request has been proposed by the Notary
Message sent to Filecoin Network
bafy2bzacedzdeo6eqty7oiccvdcfrvj2dashbqiyo3osovxqyjdircyog2u3g
Address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
Datacap Allocated
2.00PiB
Signer Address
f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri
Id
255d352e-7251-4e93-b3ce-06d9ae9415d8
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedzdeo6eqty7oiccvdcfrvj2dashbqiyo3osovxqyjdircyog2u3g
Request Approved
Your Datacap Allocation Request has been approved by the Notary
Message sent to Filecoin Network
bafy2bzacec22gqpdgyul3b5yid2wzajsyhtd4ywmvmaysbiwyhupuiivbmum4
Address
f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy
Datacap Allocated
2.00PiB
Signer Address
f1xx6555qijma7igpnjspyvdunc4vfxkawnpqy5ii
Id
255d352e-7251-4e93-b3ce-06d9ae9415d8
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec22gqpdgyul3b5yid2wzajsyhtd4ywmvmaysbiwyhupuiivbmum4
This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.
-- Commented by Stale Bot.