filecoin-plus-large-datasets icon indicating copy to clipboard operation
filecoin-plus-large-datasets copied to clipboard

KIM JIN HYUK Gongzakso <Organization> - <Project Name>

Open tonylee9988 opened this issue 3 years ago • 94 comments

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

  • Organization Name: KIM JIN HYUK Gongzakso(meaning: Production)
  • Website / Social Media: https://www.facebook.com/gonngzakso
  • Total amount of DataCap being requested (between 500 TiB and 5 PiB): 1.95 PiB
  • Weekly allocation of DataCap requested (usually between 1-100TiB): 100TiB
  • On-chain address for first allocation: f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

Please answer here.

This project had been approved as the issue #209 last Jan but closed due to the 1st 50TiB tranche stored in a single SP while testing. With the full understanding of multi-SP storage rule, we are applying a new issue as guided but with the larger DataCap since the needs of documentaries storage are expanding. After 10years of producing director career from the major Korean national broadcasting companies like MBC and SBS, my name was strong enough to establish a production firm with the market top professionals. Over the 14years of projects, KIMJINHYUK Gongzakso became Korea's #1 documentary production company in quality & quantity; 98 60-minute, 96 30-minute, and 723 10-minute films. A total of 20 contests were awarded, including the largest number of planned contests, the Production Content & Broadcasting Agency, and the Presidential Award at the 2018 Broadcasting Content Awards. As our eyes turn to the world's paradigms on blockchain and WEB 3.0 to BE FREE from the media dinosaurs and the data security in order to promote new global online services with our 14year-of-works, we are facing the next flow of service infrastructure and data management strategy. Projects with National Geographic and Biography Channel, especially, the current projects by Samsung as well as the future potential requirement of 8K UHD document filming along with all our past documents need the next epoch-marking data treatment methodology to provide many angles of online-based service. Breaking out of the media influence blanket and preparing for the world stream is our motto of 3year planning. What is the primary source of funding for this project?

Please answer here.

Two major sources by Family fund and T2B partners (accelerator) and 20% by sub-participating producer group. What other projects/ecosystem stakeholders is this project associated with?

Please answer here.

We are partnering with VOGO Digital Lab (primary research lab for IPFS, Filecoin & Web3) and VOGO Networks (Filecoin SP with a long relationship and good credit) to co-develop the hub for documentary video and photo from the independent documentary film producers and turn their products into NFTs.

Use-case details

Describe the data being stored onto Filecoin

Please answer here.

Publicly distributable video cuts and scene photos and full archives of the original films Where was the data in this dataset sourced from?

Please answer here.

KIMJINHYUK Gongzakso data production storages and tapes Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

Please answer here.

Facebook Main Page: https://www.facebook.com/gongzakso Documentary Samples: https://www.facebook.com/gongzakso/videos/731750654345023 https://www.facebook.com/gongzakso/videos/304153544075615

Youtube Channel : https://www.youtube.com/channel/UCGhsFusjgbmp1hZzW0MBlng

Please answer here.

Yes, the videos and photos are publicly available (yet, the retrieval fees may apply for download later on), but the original archive film file may be too large to view without special tool. What is the expected retrieval frequency for this data?

Please answer here.

We will store the original and the TV edition films for 2 years and start the public service with social media partners with the full preparation in 3 years. For how long do you plan to keep this dataset stored on Filecoin?

Please answer here.

Expectedly 4 to 5 years and it’s likely to be extended afterwards.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Please answer here.

We plan to make deals in South Korea, Singapore, Japan or China if possible. How will you be distributing your data to storage providers? Is there an offline data transfer process?

Please answer here.

Both ways in online and offline transfer processes How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

Please answer here.

We get good advice from the storage provider like VOGO Networks and some other SPs from the community. How will you be distributing deals across storage providers?

Please answer here.

We have learned from the last experience and abosolutely distribute less than 30% per each SPs. Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Please answer here.

Yes, we are all ready to go.

tonylee9988 avatar Oct 07 '22 02:10 tonylee9988

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Datacap Request Trigger

Total DataCap requested

1.95PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

raghavrmadya avatar Oct 07 '22 15:10 raghavrmadya

DataCap Allocation requested

Multisig Notary address

f01858410

Client address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

DataCap allocation requested

50TiB

Hi, based on @raghavrmadya 's conversation, I'm willing to see the first allocation go through. Supporting this for now.

dannyob avatar Oct 08 '22 00:10 dannyob

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedndj3ds4guygxw7mxysm6z4uuthnmo4kvy5mcs4awua37ydmmhga

Address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Datacap Allocated

50.00TiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedndj3ds4guygxw7mxysm6z4uuthnmo4kvy5mcs4awua37ydmmhga

dannyob avatar Oct 08 '22 00:10 dannyob

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedswblul6muthea7w7cuhwaubhbqq4hb4rkkksxa6wlrbfmu5ybma

Address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Datacap Allocated

50.00TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedswblul6muthea7w7cuhwaubhbqq4hb4rkkksxa6wlrbfmu5ybma

psh0691 avatar Oct 11 '22 05:10 psh0691

@tonylee9988 Hi! Great to see you have gotten approval for DataCap and advancing the mission of preserving humanity’s most important information. If you are looking for more storage providers to store these data or have any questions, please visit #bigdata-exchange on Filecoin Slack or reply here.

We have strong demand from a diverse group of SPs, who are actively looking to onboard more data.

BDE-io avatar Oct 11 '22 16:10 BDE-io

DataCap Allocation requested

Request number 2

Multisig Notary address

f01858410

Client address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

DataCap allocation requested

100TiB

Id

81fa2d03-d948-4db0-8091-230e68041e36

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Last two approvers

psh0691 & dannyob

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

100TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (1.95 PiB)

1.90PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
1344 2 50TiB 66.47 12.01TiB

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceahh367geyucqp5yxltd5tybdpicioaytasglhwhfqzzfwuwhylkk

Address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Datacap Allocated

100.00TiB

Signer Address

f1tfg54zzscugttejv336vivknmsnzzmyudp3t7wi

Id

81fa2d03-d948-4db0-8091-230e68041e36

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceahh367geyucqp5yxltd5tybdpicioaytasglhwhfqzzfwuwhylkk

Joss-Hua avatar Nov 14 '22 10:11 Joss-Hua

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecz3gvoug7df6qnigh4hhvpumq6s46tgingyps35xwpbnp33ik3ue

Address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Datacap Allocated

100.00TiB

Signer Address

f1yayfsv6whu3rheviucvventj3y6t542xfpb47ei

Id

81fa2d03-d948-4db0-8091-230e68041e36

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecz3gvoug7df6qnigh4hhvpumq6s46tgingyps35xwpbnp33ik3ue

NDLABS-Leo avatar Nov 22 '22 03:11 NDLABS-Leo

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Last two approvers

not found & Joss-Hua

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

200TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (1.95 PiB)

1.90PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
1700 2 100TiB 52.55 10GiB

DataCap Allocation requested

Request number 3

Multisig Notary address

f01858410

Client address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

DataCap allocation requested

200TiB

raghavrmadya avatar Nov 24 '22 19:11 raghavrmadya

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecgzszq67cfzo4s3zdms5quvxdllru432qddilyur3uaiminqnus6

Address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Datacap Allocated

200.00TiB

Signer Address

f1jvvltduw35u6inn5tr4nfualyd42bh3vjtylgci

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecgzszq67cfzo4s3zdms5quvxdllru432qddilyur3uaiminqnus6

stcloudlisa avatar Nov 25 '22 01:11 stcloudlisa

The first signature did not take effect, re-sign

NDLABS-Leo avatar Nov 25 '22 03:11 NDLABS-Leo

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedc63qv2ytgna4hqqglnl25d3auuhty4ticbxsmttnrhu6a6p7unw

Address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Datacap Allocated

200.00TiB

Signer Address

f1yayfsv6whu3rheviucvventj3y6t542xfpb47ei

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedc63qv2ytgna4hqqglnl25d3auuhty4ticbxsmttnrhu6a6p7unw

NDLABS-Leo avatar Nov 25 '22 03:11 NDLABS-Leo

DataCap and CID Checker Report[^1]

  • Organization: KIM JIN HYUK Gongzakso(meaning: Production)
  • Client: f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

  • Storage provider should not exceed 25% of total datacap.
  • Storage provider should not be storing duplicate data for more than 20%.
  • Storage provider should have published its public IP address.
  • All storage providers should be located in different regions.

⚠️ f01624021 has sealed 38.38% of total datacap.

⚠️ f01918123 has sealed 31.24% of total datacap.

⚠️ f0521569 has sealed 26.66% of total datacap.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01624021 Seoul, Seoul, KR 50.43 TiB 38.38% 46.55 TiB 7.70%
f01918123 Seoul, Seoul, KR 41.04 TiB 31.24% 41.04 TiB 0.00%
f0521569 Seoul, Seoul, KR 35.03 TiB 26.66% 30.72 TiB 12.29%
f01982557new Seoul, Seoul, KR 4.90 TiB 3.73% 4.87 TiB 0.64%
f01715688 Tokyo, Tokyo, JP 4.00 GiB 0.00% 4.00 GiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

  • No more than 25% of unique data are stored with less than 4 providers.

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
53.46 TiB 59.63 TiB 1 45.38%
28.79 TiB 59.55 TiB 2 45.32%
4.05 TiB 12.22 TiB 3 9.30%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f3simkqmjbjnbpifr3pbtdmcfjz35sisrehkq3gcm
wn7ghtussajcw6wiypzzprxdvt5pj6y2dzrxjpvn6
tz2q
Blockchain World(BCW) 126.62 TiB 1,993 LDN v3 multisig

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

filplus-checker avatar Dec 15 '22 09:12 filplus-checker

Hi, please explain the abnormal information.

Sunnyiscoming avatar Feb 02 '23 05:02 Sunnyiscoming

Hi Sunnyiscoming, We have 4 SPs for this storage deal and one of them is going through some issues with their center relocation. Our deal managing staff has excessively allocated the documentary data without balancing monitoring. Since this fact was discovered, we have ceased the sealing process until that SP resolves its problem and finds more valuable SPs simultaneously. Keeping the filecoin ecosystem healthy and valuable is our major concern as well. Thank you.

All the Best

2023년 2월 2일 (목) 오후 2:19, Sunnyiscoming @.***>님이 작성:

Hi, please explain the abnormal information.

— Reply to this email directly, view it on GitHub https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1038#issuecomment-1413167600, or unsubscribe https://github.com/notifications/unsubscribe-auth/AXE2CVN5ZKW6RUAM767QL3LWVM7VZANCNFSM6AAAAAAQ7FWMZA . You are receiving this because you were mentioned.Message ID: <filecoin-project/filecoin-plus-large-datasets/issues/1038/1413167600@ github.com>

tonylee9988 avatar Feb 02 '23 06:02 tonylee9988

Hi @tonylee9988

There are 2 problems:

CID sharing. The data you stored is not the data you said you would be storing. You have been storing data of BCW.

2 questions: Are you the data preparer yourself? Are you doing the distribution?. If yes , why did you use data of someone else?

Distribution

Everything is stored on miners in KR. These are with 1 organization. What is your relationship to this organization? Do you have stake in it?

Are you willing to provide KYC to regain trust? If so -> i would like you to fill out this form to provide us with the necessary information to make a educated decision on your LDN request if we would like to support it.

cryptowhizzard avatar Feb 02 '23 15:02 cryptowhizzard

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

DataCap allocation requested

400TiB

Id

1b90384b-3344-4fa4-91e5-4c33900813a5

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

400TiB

Total DataCap granted for client so far

1818989403545861120.0YiB

Datacap to be granted to reach the total amount requested by the client (1.95 PiB)

1818989403545861120.0YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
6560 7 200TiB 35.75 49.37TiB

Heavy allocation to 2 SPs was recognized. As such, we stopped sending deals to them since Feb. We developed a new SP and started sending deals to them, but since they are also in the same region, we are currently discussing with a few SPs in Europe and the States. So far, it wasn't very successful since they were not in a position to receive deals due to a lack of pledge at their ends. It will be very helpful if anyone in the community can introduce us to any decent SPs in any non-Asian region who can receive deals from us. With the above requested DataCap limit, we will be striving to find new SPs in non-Asian region and evenly distribute our client's data among regions.

Bryan9498 avatar Jun 19 '23 06:06 Bryan9498

checker:manualTrigger

ipollo00 avatar Jun 19 '23 07:06 ipollo00

checker:manualTrigger

psh0691 avatar Jun 20 '23 01:06 psh0691

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

  • Overall Graphsync retrieval success rate: 61.05%
  • Overall HTTP retrieval success rate: 0.00%
  • Overall Bitswap retrieval success rate: 0.00%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 82.45% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacea7ejvnmjbwcjkt6mucdlhtzfgvgxcpla4sncuqkkw3mz3fxtj422

Address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Datacap Allocated

400.00TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

Id

1b90384b-3344-4fa4-91e5-4c33900813a5

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea7ejvnmjbwcjkt6mucdlhtzfgvgxcpla4sncuqkkw3mz3fxtj422

psh0691 avatar Jun 20 '23 01:06 psh0691

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceahxszbewcyvrwdsk4hgdcunrfxu5jlu4p5tg6dl4dqcrbhklgnry

Address

f1lzkycqjpx4nmvznhxkl5fd7rct26tpo56qx2ura

Datacap Allocated

400.00TiB

Signer Address

f1n5wlrrhoxpkgwij25xrtt7w7g2k3fhbthmdn6ri

Id

1b90384b-3344-4fa4-91e5-4c33900813a5

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceahxszbewcyvrwdsk4hgdcunrfxu5jlu4p5tg6dl4dqcrbhklgnry

ipollo00 avatar Jun 20 '23 03:06 ipollo00

@tonylee9988 contacted me, and the packaging process is constantly optimized, which can give this application order opportunity, In the future will continue to pay attention to the updated situation.

ipollo00 avatar Jun 20 '23 03:06 ipollo00

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] avatar Jul 21 '23 08:07 github-actions[bot]