Skip to content
This repository has been archived by the owner on Jul 18, 2024. It is now read-only.

[DataCap Application] <The New Ads> #464

Closed
Gonzalo1987 opened this issue Jul 1, 2022 · 84 comments
Closed

[DataCap Application] <The New Ads> #464

Gonzalo1987 opened this issue Jul 1, 2022 · 84 comments

Comments

@Gonzalo1987
Copy link

Gonzalo1987 commented Jul 1, 2022

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

  • Organization Name: The New Ads
  • Website / Social Media: https://thenewads.com
  • Total amount of DataCap being requested (between 500 TiB and 5 PiB): 5 PiB
  • Weekly allocation of DataCap requested (usually between 1-100TiB): 100 TiB
  • On-chain address for first allocation: f1tza7y76ta5zogx5hdifsfdlyjh354xzb2awr33y
  • Type: Custom Notary

Project details

Share a brief history of your project and organization

The New Ads is an advertising agency specializing in digital signage, app development, social media, and audiovisual content creation. 

What is the primary source of funding for this project?

Our revenue flows are the services provided to our clients.

What other projects/ecosystem stakeholders is this project associated with?

The New Ads is part of the Workanda Group. Workanda is made up of a group of companies from La Rioja, Spain, to offer a comprehensive service through IT services and 360º Communication. Custom programming and software, web development and training, audiovisual and design services.

Use-case details

Describe the data being stored onto Filecoin

The dataset provided consists of different forms of data (mainly video) from our client list which extends to more than 1,400. From the Spanish football team Barca F.C, Deutsche Bank, Volkswagen, and Opel, to The Parliament of “La Rioja” which is a public institution in Spain.

Where was the data in this dataset sourced from?

Own Servers. 

Can you share a sample of the data? A link to file would work.

http://gofile.me/3XZGR/lZ79X3MRp
http://gofile.me/3XZGR/Qft0AGwzU
http://gofile.me/3XZGR/qLrjekXYk
http://gofile.me/3XZGR/0gbiaYqRW
http://gofile.me/3XZGR/qYXd5MWX5

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Yes, since the dataset will be encrypted.

What is the expected retrieval frequency for this data?

There will not be much retrieval frequency of the data. Only if needed to do a revision of a given file. 

For how long do you plan to keep this dataset stored on Filecoin?

One and a half years with possible renewal.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

US and other countries as well

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Files will be hosted online and able to be pulled by storage providers over the internet.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

Choosing reputable providers in the community via Filecoin slack or ranking sites. Also looking for SPs that are able to comply with GDPR.



How will you be distributing deals across storage providers?

Hosted online (like the sample links provided) for the SPs to download


Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

We have adequate resources and support.

@large-datacap-requests
Copy link

Thanks for your request!
Everything looks good. 👌

A Governance Team member will review the information provided and contact you back pretty soon.

@Kevin-PiKNiK
Copy link

@Gonzalo1987 : This is very cool and thanks for your interest in the Filecoin network. You say that your company is part of a group of companies from Spain. However, your request mentions storing in the United States among "other countries". I would like to confirm that this is intentional and without sovereignty restrictions. Thanks.

@Gonzalo1987
Copy link
Author

Hello @Kevin-PiKNiK! We confirm that our intention is to store the data across different places without sovereignty restrictions. In our opinion, one of the main advantages of the Filecoin Network is to diversify risk choosing differents regions and countries. Thanks.

@Kakkouii
Copy link

Kakkouii commented Jul 5, 2022

@Gonzalo1987 Cool idea and I've taken a look at your application. There's one thing I'd like to figure out. You confirmed that it's a public dataset since it's encrypted. How can encrypted data be public?

@Fenbushi-Filecoin
Copy link

We can support the application. Just curious, if the data is already encrypted on your side, is it necessary for SPs to comply with GDPR?

@Gonzalo1987
Copy link
Author

@Gonzalo1987 Cool idea and I've taken a look at your application. There's one thing I'd like to figure out. You confirmed that it's a public dataset since it's encrypted. How can encrypted data be public?

Hello @Kakkouii! We should've been clearer. What we mean is that the data is publicly retrievable but encrypted. By storing on the Filecoin network, it is inherently public for anyone to retrieve but will be encrypted in order to comply with our company's privacy requirements and GDPR.

We can support the application. Just curious, if the data is already encrypted on your side, is it necessary for SPs to comply with GDPR?

Hi @Fenbushi-Filecoin! Yes, every organization (SPs in this case) that stores data from European institutions or individuals will need to comply with GDPR. But encryption is called out in Article 32 as an “appropriate technical and organizational measure” of security. For this reason, we will keep the keys for the encrypted data. This means the SPs does not need to be too concerned with GDPR compliance as we will have control of the data. Here is a link for more info! https://gdpr.eu/faq/.

@Gonzalo1987
Copy link
Author

Hello @galen-mcandrew! What are the next steps now that we have support from one notary (@Fenbushi-Filecoin)?

@Sunnyiscoming
Copy link
Collaborator

Hi, @Gonzalo1987 Hi,
As Deep mentioned filecoin-project/notary-governance#514, you will need to find notaries that can support your application since it includes private enterprise data.

You will need to follow a process similar to filecoin-project/notary-governance#489. This means:

Find notaries that can support your application. You can do this best in Slack, #fil-plus or #fil-plus-application-review
Once you have them, make an issue in the Notary Governance repo, similar to filecoin-project/notary-governance#489, stating your plan and the notaries that can support your request
FIL+ governance can then set up a custom LDN for this with those notaries and proceed from there
These additional steps are required to ensure that notaries can establish a relationship with you and back you as a trustworthy client, since the usual deal data will be encrypted and not open to access for the community to validate your actions on chain after DataCap has been used.

@raghavrmadya
Copy link
Collaborator

any updates?

@Gonzalo1987
Copy link
Author

any updates?

Yes! Yesterday we posted the following message on the notary governance issue:

filecoin-project/notary-governance#573 (comment)

We got the notaries backing our application, and a data distribution plan complying with geographic diversification.

@raghavrmadya
Copy link
Collaborator

@Gonzalo1987, thanks. We will go ahead and create a custom multisig between the notaries listed in #573.

@Gonzalo1987
Copy link
Author

@Gonzalo1987, thanks. We will go ahead and create a custom multisig between the notaries listed in #573.

Hello @raghavrmadya. I see that the issue has the CustomLDN label. What are the next steps to keep moving forward in the process?

@Gonzalo1987
Copy link
Author

Hello @raghavrmadya @Sunnyiscoming @galen-mcandrew! Any updates on the creation of the custom multisig? Is there anything that we can do to accelerate the process?

@Carohere
Copy link

Hi @Gonzalo1987, I understand the need to encrypt data for client privacy. But as @kernelogic mentioned in your proposal, it's hard to verify their willingness if it's already impossible to get them each authorized for data samples. Is there any way you can prove that the companies in your client list authorized New Ads to store their data here?

@jamerduhgamer
Copy link

Hi @Carohere, I believe a few of the notaries besides Kernelogic have actually met with the client themselves to perform the KYC process as mentioned in the corresponding proposal filecoin-project/notary-governance#573.

@Carohere
Copy link

@jamerduhgamer yes james. i'm aware that the project has already gotten support from five notaries. my point here is whether the applicant needs to address queries from other community members after that. cc @Kevin-FF-USA, could you help me on this? thanks.

@Gonzalo1987
Copy link
Author

Hello @Carohere! Totally willing to address queries from the Filecoin community. As you must know, The New Ads is not a data-storage company, it’s an advertisement company with more than 1,400 clients looking to store their whole dataset in the cheapest and most reliable way. We must comply with GDPR and european regulation as mentioned in filecoin-project/notary-governance#573 , and that’s the only requisite while storing our data and the motive for encryption. Therefore, our clients do not have to approve where we are storing our data as long as we comply with european regulation. Each notary made their proper KYC (we had several meetings with them) and we prove our case that we are authorized to store this dataset in Filecoin.

@large-datacap-requests
Copy link

Thanks for your request!
Everything looks good. 👌

A Governance Team member will review the information provided and contact you back pretty soon.

@PluskitOfficial
Copy link

checker:manualTrigger

@filplus-checker-app
Copy link

DataCap and CID Checker Report Summary1

Retrieval Statistics

  • Overall Graphsync retrieval success rate: 42.77%
  • Overall HTTP retrieval success rate: 0.52%
  • Overall Bitswap retrieval success rate: 0.79%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 68.95% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients2

✔️ No CID sharing has been observed.

Full report

Click here to view the CID Checker report.
Click here to view the Retrieval Dashboard.
Click here to view the Retrieval report.

Footnotes

  1. To manually trigger this report, add a comment with text checker:manualTrigger

  2. To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

@PluskitOfficial
Copy link

Deal Data ReplicationOverall HTTP retrieval success rate: 0.52%
Overall Bitswap retrieval success rate: 0.79%

Hi, @Gonzalo1987 could you please explain why these two parts have such a low success rate?

@jamerduhgamer
Copy link

Hi @PluskitOfficial, I took a look at all of the miner IDs to see what the retrieval issues were.

f01998317 - Reached out to the SP, they are looking into it.
f0763337 - Had an issue with its peer ID on-chain, I reached out to the SP and I believe this has been resolved.
f01857342 - Has decent success rate
f01900409 - SP is no longer active

f01165233, f01154023 and f01165179 all belong to one SP that is no longer sealing and have their Boost nodes turned off.

For f01927134, f01884351 and f02022254, I see in the retrieval report that the main error is "protocol not supported", any idea what could be the main cause for these errors so I can relate that to the SPs?

@jamerduhgamer
Copy link

checker:manualTrigger

@filplus-checker-app
Copy link

DataCap and CID Checker Report Summary1

Retrieval Statistics

  • Overall Graphsync retrieval success rate: 39.15%
  • Overall HTTP retrieval success rate: 0.63%
  • Overall Bitswap retrieval success rate: 1.11%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 68.95% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients2

✔️ No CID sharing has been observed.

Full report

Click here to view the CID Checker report.
Click here to view the Retrieval Dashboard.
Click here to view the Retrieval report.

Footnotes

  1. To manually trigger this report, add a comment with text checker:manualTrigger

  2. To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

@github-actions
Copy link

github-actions bot commented Sep 9, 2023

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

--
Commented by Stale Bot.

@jamerduhgamer
Copy link

Hello! Still looking for notaries to sign datacap.

Copy link

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacea4rasrqds4qprk34y5v33ewir2n22d7doqzos5ehsojeyzuc57pw

Address

f1tza7y76ta5zogx5hdifsfdlyjh354xzb2awr33y

Datacap Allocated

400.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

cdfa39a5-ec88-413c-a45b-f08f2ddb7128

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea4rasrqds4qprk34y5v33ewir2n22d7doqzos5ehsojeyzuc57pw

@github-actions
Copy link

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

--
Commented by Stale Bot.

@jamerduhgamer
Copy link

Comment to keep LDN open. Sealing has been on-going.

@github-actions
Copy link

github-actions bot commented Oct 7, 2023

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

--
Commented by Stale Bot.

@jamerduhgamer
Copy link

Comment to keep LDN open. Sealing has been on-going.

@github-actions
Copy link

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

--
Commented by Stale Bot.

@jamerduhgamer
Copy link

Comment to keep LDN open.

@Sunnyiscoming
Copy link
Collaborator

Hello, @Gonzalo1987 per the filecoin-project/notary-governance#922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be allowed to move forward for additional notary review.

Copy link

github-actions bot commented Nov 9, 2023

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

--
Commented by Stale Bot.

@jamerduhgamer
Copy link

Comment to keep LDN open.

Copy link

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

--
Commented by Stale Bot.

Copy link

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

--
Commented by Stale Bot.

@jamerduhgamer
Copy link

Hello @dkkapur, can we get this LDN re-opened?

@jamerduhgamer
Copy link

@simonkim0515 or @Kevin-FF-USA, are you also able to help with getting this LDN re-opened?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests