Investigation of Fil+Notary Public's Failure to Fulfill Duties

[Notary-Labs]

Issue Description

We've compiled a record of recently approved applications exhibiting a retrievability rate of less than 10%. In the list below, we have selected LDNs with a retrievability rate of less than 1%, even though they were granted datacap by notaries last week.

The following notaries colluded with the client, disregarded DC report and retrieval data:

• NationalUniversity
• NDLABS-OFFICE
• Lunaluca
• Sunkistn
• NDLABS-OFFICE
• Joss-Hua
• Gitel-chu
• TechPieOfficial
• shliming
• Ottow8
• Sinsoteam
• Joss-Hua
• ane-1
• NDLABS-OFFICE
• BobStaring
• yhkp
• Joss-Hua
• dchoi27
• Flora-Moody
• Ottow8
• AgeeWeb3
• fildata
• Inayatullah199
• NDLABS-OFFICE
• QodeNu
• Lennon0
• Xavierr4
• NDLABS-OFFICE
• hengdingy
• Joss-Hua
• Jingana
• Janicelu1
• hengdingy
• NetCloud91Wan
• linice
• datalove2
• Joss-Hua
• Acalyt
• laurarenpanda
• NDLABS-OFFICE
• laurarenpanda
• Corerae
• UriahSee
• CreamPiPi

Proposed Solution(s)

Conduct an investigation with the associated notary public to confirm the existence of the problem, delete the notary public's identity, and prohibit the review of DC data during the investigation period.

Timeline

Discussion at June 27th 2023 WG Call
Community review until June 30th 2023 at 5 pm ET
Decision announced by the T&T WG Lead after

Technical dependencies

N/A

Related Issues

LDN ISSUE # | RETRIEVAL SUCCES % | USER NAME
- 1264 0.00%  NationalUniversity
- 1964 0.00%  NDLABS-OFFICE
- 1700 0.00%  Lunaluca
- 1095 0.00%  Sunkistn
- 1721 0.00%  NDLABS-OFFICE
- 1726 0.00%  Joss-Hua
- 1618 0.00%  Gitel-chu
- 1802 0.00%  TechPieOfficial
- 1201 0.00%  shliming
- 1866 0.00%  Ottow8
- 961   0.00%  Sinsoteam
- 1729 0.00%  Joss-Hua
- 260   0.00%  1ane-1
- 1723 0.00%  NDLABS-OFFICE
- 1890 0.00%  BobStaring
- 1731 0.00%  yhkp
- 1725 0.00%  Joss-Hua
- 1838 0.00%  choi27
- 1860 0.00%  Flora-Moody
- 1867 0.00%  Ottow8
- 1898 0.00%  AgeeWeb3
- 311   0.00%  fildata
- 1997 0.00%  Inayatullah199
- 1965 0.00%  NDLABS-OFFICE
- 2026 0.00%  QodeNu
- 1584 0.00%  Lennon00
- 1498 0.00%  Xavierr4
- 1722 0.00%  NDLABS-OFFICE
- 1999 0.00%  hengdingy
- 1728 0.00%  Joss-Hua
- 1679 0.00%  Jingana
- 1938 0.00%  Janicelu1
- 1648 0.00%  hengdingy
- 1066 0.00%  NetCloud91Wan
- 1000 0.00%  linice
- 1579 0,01%  datalove2
- 1580 0,02%  datalove2
- 1444 0,03%  Joss-Hua
- 1627 0,08%  laurarenpanda
- 1779 0,14%  Acalyt
- 1626 0,14%  laurarenpanda
- 1720 0,20%  NDLABS-OFFICE
- 1688 0,23%  laurarenpanda
- 1219 0,69%  Corerae
- 1432 0,72%  UriahSee
- 1431 0,75%  CreamPiPi

[herrehesse]

@Notary-Labs thank you for posting this. It is indeed highly questionable why these signatures where made and I would like to see public responses from all of the involved notaries as soon as possible. Agreeing with proposed solutions.

(we will update this list by end of week with new applications and signatures made)

[herrehesse]

Let's address the issue of zero retrievability applications during the next T&T call, shall we? It's about time we acknowledge a few simple, undeniable facts:

• Retrievability for FIL+ deals has always been and continues to be an absolute requirement. No more excuses. (ex. FIL-E)
• Prior to the launch of the retrieval bot, notaries should have manually tested retrievability. However, almost all of them failed to do their job properly and neglected this crucial step.
• Since the retrieval bot was introduced, graphsync retrievals should function properly (assuming the miner has enabled retrievals), and HTTP/BITSWAP should reach 100% when installed and enabled. If all three metrics show a big fat zero percent, it's clear that the service provider is deliberately obstructing retrievals.
• Storage Prodivders who knowingly refuse to support retrieval should not be granted any datacap, and in my opinion, they should even lose their current multiplier. It's high time we held them accountable.
• Notaries mindlessly approve 0/0/0% LDNs without conducting proper due diligence or asking meaningful questions should be indeed removed from the multisig.

This behaviour needs to come to an end, and it needs to happen now.

@raghavrmadya @dkkapur @simonkim0515

[raghavrmadya]

As discussed, it's too soon to remove notaries solely on the basis of signing for 0% retrievability. If you have further evidence of abuse or collusion against the aforementioned notaries, kindly share

[herrehesse]

Agreed @raghavrmadya. Pausing them would be acceptable?

[spaceT9]

It's advisable to set a clear standard for retrieval rate to avoid any dispute

[NDLABS-Leo]

The ND application was initially based on the combination of payroad_cid and picec_cid for retrieval purposes. We explained this in the GitHub application and provided the data's CID to notaries for retrieval testing during the signature process. Currently, our technology has undergone iterative updates, and the retrieval rate is functioning normally.

[herrehesse]

@NDLABS-OFFICE Can you give me examples of retrieval going from 0>100% last week? As we mentioned multiple times the retrieval bot is still undergoing updates and improvements thus we can not fully base desicions on it yet. That does not mean that zero-retrievable applications should be blindly signed without question or improvement.

I am still fully against the above notaries signing on zero-retrievability LDN's and would love to get a response for all of them. I would also love to see that @NDLABS-OFFICE signatures from now on are only on applications with proper or improving retrieval rates.

[NDLABS-Leo]

@herrehesse
That does not mean that zero-retrievable applications should be blindly signed without question or improvement.
Regarding this matter, we agree, and ND's audit signatures have consistently maintained high standards recently.

In addition, I would like to add that after the search bot comes out, the current minimum standard should be "Graphsync" search rate of more than 1%, and would prefer to focus the search method on "HTTP".

And, for example, for application #2055, ND has improved the search rate after adjusting the procedure of the search method. Our next optimization will also be to support http, but this technical adjustment will take some time and we are working on it.

[herrehesse]

@NDLABS-OFFICE Please keep me informed. I'm eager to monitor any progress on those LDN's. I can personally attest that once a minerID is updated with the latest BOOST/HTTP retrieval, you can achieve an immediate 100% success rate.

I recommend gradually increasing the current minimum, on a weekly basis, starting from 10% and progressively moving up to 20, 30, 50%, and so forth. This adjustment is necessary to ensure that clients and storage providers prioritise the retrievability of data.

[NDLABS-Leo]

@herrehesse Okey.
If Boost is being used, then indeed that is the case because Boost includes comprehensive indexing functionality and integrates HTTP retrieval. However, there are still many SPs using Lotus. They have to supplement indexing because the check bot's timeout is set at 15 seconds, and without indexing, the check bot will not be able to retrieve the data. However, over time, the retrieval rate will definitely improve, so the 1% retrieval rate serves as a foundation, and we hope the community will gradually raise this standard. Additionally, more emphasis should be placed on HTTP retrieval because supporting HTTP retrieval is what truly enables FIL to conveniently facilitate the use of ecological applications.

[herrehesse]

@NDLABS-OFFICE Retrieval has always been a requirement, even prior to the recent updates and bots. Achieving a retrieval rate of 50% to 80% with Graphsync indicates that miners are cooperative and open to random client retrieval requests, which is a positive development. Now, I propose that we enhance the bot and allow clients and notaries sufficient time to upgrade retrievability to acceptable levels through HTTP.

Here are my suggestions:

• Notaries refrain from signing on any application with less than 10% Graphsync retrievals.
• Notaries urge clients to contact their storage providers to enable retrievals (this is mandatory!).
• Monitor the progress of improvement of willing and reputable participants in the ecosystem. Flag them for community vision.
• Cease signing on applications with less than 50% graphsync/HTTP by early August.
• Cease signing on applications with less than 80% graphsync/HTTP by early September.

Zero retrievability directly indicates abusive behaviour by storage providers, as retrievability has always been a requirement for open datasets. No retrievability means no value added to the ecosystem.

[NDLABS-Leo]

@herrehesse Having these clear execution rules and timelines is excellent. I hope they can be shared in proposals for more people to see and discuss. Once approved, we will follow the rules accordingly.

[raghavrmadya]

As there were issues with the retrieval bot, it is most reasonable to provide the listed notaries the benefit of the doubt. Kindly refer and contribute to the V2 spec if you would like to help the T&T WG gather objective evidence to take action - https://github.com/data-preservation-programs/RetrievalBot/blob/main/filplus.md#http-v2

[raghavrmadya]

Request for retrievability standards is taken into consideration and will be part of V2 release