Post-Judging QA

[liveactionllama]

1. Please post feedback about specific findings on the relevant issue in this repo, ❗️but also be sure to flag them via comment on this discussion page so that the judge can easily review.❗️
2. More general feedback for the judge can be shared here.

The judge for this contest is @alex-ppg.

Reminders

• Please keep your comments as fact-based as possible.
• C4 judges have final authority in determining validity and severity.
• You are welcome to start new discussions here!
• Please link to specific issues wherever relevant.

Thank you!

[alex-ppg]

Intro

Greetings everyone, I am the present judge of the NextGen C4 contest and would like to provide some additional information to aid you in your PJQA process.

Glossary

• PJQA: Post-Judging Quality Assurance
• QA: Quality Assurance
• SC: Supreme Court
• NC: Non-Critical
• L: Low-Risk
• CEI: Checks-Effects-Interactions
• AWE: Altered World Event

Response Per Submission

I have provided a detailed response to each individual primary exhibit that details why it has been judged as such. Some wardens made repetitive invalid submissions and their findings may have been invalidated (after review, of course) without providing a detailed rationale and grouped under a single of their invalid submissions.

There have been certain albeit few times whereby the primary issue was changed after a response was given. In such cases, it is useful to inspect the submission's original duplicate issue tag and navigate to the original issue checking if I have provided further insight.

Centralization

Centralization-related issues have been grouped under #303 as they fall under the relevant SC verdict described in the issue and it was time effective to do so given that none of them would be considered valid.

Overinflated Severity

All findings that have been marked as overinflated severity contain a response that details what their perceived severity is. Even if I did not invalidate them, their QA score would be C in all circumstances due to comparatively better QA reports being present in the contest. Based on this, I advise against arguing re-opening such issues as QA given that they would be marked as C and thus yield no reward.

Finding Penalization

A tricky situation arose for #1513 and #1323 as they both pertained to the same functional vulnerability albeit with different severity categorization rationales. Based on the relevant C4 guidelines, I have at-minimum penalized duplicates of #1513 by 50% given that they weaponized the same vulnerability with a medium-risk rather than high-risk impact. These findings and the relevant rationale can be observed here.

In general, any submission that contains invalid statements, an insufficient recommendation, and/or a completely absent PoC may be penalized.

Certain Wardens made repetitive invalid submissions and were correspondingly grouped under a single submission without providing a detailed rationale as to why their findings were invalidated. Rest assured that their contents were reviewed, I just did not provide a detailed response in contrast to all other exhibits in the contest.

Finding Categorization (Re-Entrancy Attacks)

For re-entrancy attacks, the root cause of the vulnerability is considered to be the state which is not updated per the CEI pattern rather than the re-entrancy origin itself. As such, #1517 and #1597 have been considered as being distinct submissions further reinforced by the fact that #1597 would not be remediated by a simple re-entrancy guard.

Gas Reports

To ensure that the contest is judged in a reasonable timeline, I have refrained from providing an intricate response per Gas Report submission. Instead, I reply with what points of the submissions may be invalid/incorrect to aid in Wardens understanding why their submission was given its grade.

Any revert scenario optimizations are considered "Out-of-Scope" as they do not lead to any real-world savings (i.e. a user can dry-run a transaction and simply not execute it).

QA Reports

Any gas optimization submitted as part of a QA report was disregarded in the interest of fairness and should have been submitted as a Gas Optimization report.

Any medium-risk (or higher) vulnerability that was submitted as part of a QA report will be credited an L rating.

While the bar for justification in QA reports is lower, the statements need to be valid in order to be considered rewardable.

For more information, please consult the relevant secret gist accessible in this link.

To note, the above simply serves as a guideline and is not a rulebook, every QA was "post-processed" and manually graded. For example, bot findings that were invalidated are not present in the guideline and will be added in a future version.

In any case, feel free to ping me in the relevant issue if you believe your QA report has been misjudged as I am more than happy to discuss!

Analysis Reports

Analysis reports were judged based on the relevant SC verdict and, based on the contest's project, the following traits were evaluated:

• Does the Analysis report outline the several centralization risks the NextGen protocol poses?
• Does the Analysis report make mention of the systemic risks associated with the oracles integrated?
• Does the Analysis report contain original content rather than re-hashed data from publicly available sources?
• Does the Analysis report contribute something new that cannot be gathered from simply reading information about the project?

Analysis reports that included any of the following (non-exhaustive) were deemed inadequate or were reduced in rating:

• Rudimentary listing of contracts and a textual description of them
• Rehash of data present in the documentation of the project as well as its website, and any other such resource
• Incorrect information or information that is very generic/vague and potentially inapplicable to the NextGen project
• Minimal new data presented/reiterating data of the Warden's submissions
• Findings better suited as part of a QA submission

Closing Thoughts

Please make sure to adhere to the PJQA guidelines shared by @liveactionllama above. Apart from that, I look forward to everyone's contribution to the PJQA process!

[ihtisham-sudo]

Kudos @alex-ppg for detailed judging.
Just left a comment on #1237

[ihtisham-sudo]

#1538 also

[alex-ppg]

Hey @ihtisham-sudo, thanks for contributing. I have replied to both of your submissions.

[zhaojio]

Thanks for judging the contest!
Left comment on #1398

[alex-ppg]

Thanks for contributing, I have replied to your comment in the relevant issue. Do not hesitate to let me know if you require further clarification.

[ZanyBonzy]

[alex-ppg]

Thanks for your feedback, I believe I have addressed both of your concerns. Do let me know if you require further clarification!

[kazantseff]

Thank you for your work!
Left a comment on #1534.

[alex-ppg]

Hey @kazantseff, thanks a lot for your extensive feedback! I have attempted to address your concerns in the relevant issue, and will also ping the Sponsor to provide their own feedback if they wish so.

[Nabeel-javaid]

Thankyou for everything
left a comment in #2028 #1365 #1777 #1917 #2007

[alex-ppg]

Hey @Nabeel-javaid, I believe I have adequately responded to all your queries. Thanks for the due diligence!

[cartlex]

Hello, @alex-ppg
I left comments on my issues #578, #894.
Could you please take a look.

[alex-ppg]

Hey @cartlex, thanks a lot for your feedback! I responded to #578 and corrected the #894 comment as it is a bug with GitHub. Let me know if you require any further clarification!

[xuwinnie]

Hi @alex-ppg , thanks for judging.

1. Two tokens can be minted on first Periodic Sale Model period #1148 seems invalid. After first mint lastMintDate is set to allowlistStartTime. The second one can only be minted after allowlistStartTime + timePeriod. The system works as expected imo.
2. For Feathers can pretend to be eggs hatching into geese #1259, I agree with your comment. "All these pre-conditions delve very deep into the realm of hypotheticals, rendering the described attack very unlikely to occur in a production environment" However, all of the preconditions are possible, which makes me believe this edge case has reached med severity. Similarly, On a Linear or Exponential Descending Sale Model, a user that mint on the last block.timestamp mint at an unexpected price. #1275 needs the mint to happen precisely at a certain timestamp. It's hard to tell which one is more likely to happen, and 1259 has a much higher potential impact.

[alex-ppg]

Hey @xuwinnie, thanks a lot for your contribution!

1148 - Two Tokens for First Periodic Sale

After revisiting the submission and the project's documentation, I am inclined to agree with your statement. The Sponsor confirmed it, however, the documentation specifies that a mint should occur in a mint period rather than after each mint period. As such, I consider the submission to be invalid regardless of the Sponsor's confirmation based on the principle of reasonable assumptions.

1259 - Mint Overlap

I will maintain my original judgment of the submission as the statements concerning probability do not align. The probability of a block.timestamp being equal to the project's end is 1 / 12, or roughly $8.333...\%$. The probability of a collection having minted 10_000_000_000 units in addition to the other set of conditions is way smaller, especially measuring the total gas required to even achieve 10 billion minted NFTs.

Please do make sure to flag anything else you see amiss, this is exactly why the PJQA process is needed!

[xuwinnie]

Thanks for reviewing! If user can take advantage of 1275, the probability should be 1/12, but considering 1275 overcharges user, I think no user would mint at the last moment intentionally. So I guess a better estimation (mint lasts for 7 days) would be 1 / 7 * 24 * 3600 = 1 / 600000. I still can't prove probability of 1259 is larger than 1 / 600000😂, but it's also hard to say it's smaller than 1 / 600000. Anyway, I respect your final decision. @alex-ppg

[Henrychang26]

Thanks for judging @alex-ppg,

I left comments on #170 and #176

[alex-ppg]

Hey @Henrychang26, thanks a lot for your comments! I believe I have replied to each one, let me know if you wish to discuss anything further.

[Henrychang26]

Hey @alex-ppg ,
I replied to your comment on #176

[alex-ppg]

Hey @Henrychang26, thanks for following up. I will revisit it one last time after PJQA but no promises.

[EperezOk]

Hey @alex-ppg, I left a comment on #373.

Thanks in advance!

[alex-ppg]

Hey @EperezOk, thanks a lot for your contribution and making sure to read the parent issue! I responded to your comment in the relevant thread, do not hesitate to let me know if you have further questions.

[aslanbekaibimov]

@alex-ppg

Thank you for judging!

I've left comments on #663 #1611 #1688

[alex-ppg]

Hey @aslanbekaibimov, thanks a lot for your contributions. I have replied to each of your exhibits and will potentially revisit #663 but I make no promises in relation to whether it will ultimately be rewarded or not.

[aslanbekaibimov]

@alex-ppg I've left one more message on #663

[k4zanmalay]

Hi @alex-ppg, thanks for judging! I left a comment on #830

[alex-ppg]

Thanks for flagging this! I left an update on the relevant issue.

[SovaSlava]

Hi @alex-ppg, thanks for judging! I I left a comment on #1382

[alex-ppg]

Hey @SovaSlava, thanks for your feedback! I responded to your issue's thread.

[hungdoo]

Hi @alex-ppg , thanks for the judging.
I left a comment at #2014

[alex-ppg]

Thanks for your feedback @hungdoo! I have replied to the relevant issue's thread.

[sashik-eth]

Hi @alex-ppg, please check my comment in #1819

[alex-ppg]

Hey @sashik-eth, thanks for contributing to the PJQA process! Please make sure to read my top response in this discussion thread before requesting feedback on your individual issues. I have also replied to your issue's thread; do not hesitate to ask for any further clarifications!

[SovaSlava]

Hi @alex-ppg, thanks for judging! I left a comment on #1355

[alex-ppg]

Hey @SovaSlava, thanks for contributing! I have addressed your concerns in the relevant issue thread.

[trachevgeorgi]

Hello @alex-ppg!

Left a comment at #1122 and #1498.

Best wishes!

[alex-ppg]

Hey @trachevgeorgi, I have replied to both of your concerns. Do let me know if there is anything else you require clarification on.

[mcgrathcoutinho]

Hi @alex-ppg , thank you for the quick and clean judging on this one.

I have left my comments on #1045, #381 , #1123, #941, #745, #741, #971, #1686

I appreciate you taking the time to read this.

[alex-ppg]

Hey @mcgrathcoutinho, thanks a lot for contributing! I have gotten back to each of your points/concerns. Do let me know if there is anything else you wish to discuss.

[mcgrathcoutinho]

Hey @alex-ppg, thanks for the responses.

I have left my comments on #1045, #941, #745

[alex-ppg]

Hey @mcgrathcoutinho, thanks for providing more information. I have responded to each of your concerns.

[mcgrathcoutinho]

Hi @alex-ppg, thanks for the responses.

I have left comments on #745 and #743.

[alex-ppg]

Hey @mcgrathcoutinho, I have responded to #743 while #745 will be revisited later today. Thanks for the ping.

[marcotnunes]

[alex-ppg]

Hey @ciphermarco, thanks for your response! I have replied in the relevant thread.

[EperezOk]

Hey @alex-ppg, I left one more (final) comment on #373.

Thanks!

[alex-ppg]

Hey @EperezOk, thanks for providing additional background and being diligent! I have responded as to why I believe that your example misses a crucial step that I hope clarifies things.

[marcotnunes]

[alex-ppg]

Hey @ciphermarco, I have followed up on your response to ensure you understand the judgment of this particular submission. Thank you for the due diligence!

[aktech297]

Hi @alex-ppg

we left comment on #981

kindly check.

[alex-ppg]

Hey @aktech297, thanks for the follow-up! I responded to the respective issue but to sum up, your submission's recommended mitigation is invalid and thus the submission was penalized.

[aktech297]

Hi @alex-ppg

we have replied for your feedback and we see that the mitigation is sufficient and no need to worry about reentrancy.

kindly check

[alex-ppg]

Hey @aktech297, I have replied to your follow-up comment.

[RunSoul22]

Hey @alex-ppg, kudos on judging this contest.

Left a comment at #685

Thank you!!!

[alex-ppg]

Hey @RunSoul22, thanks for requesting more information in relation to another of your submissions. I have responded in the thread and linked an issue that should shed some light into your own's judgment.

[RunSoul22]

Hey @alex-ppg, thanks for the responses.

I have left my comments on #685

[alex-ppg]

Hey @RunSoul22, thanks again for contributing to the PJQA process! I responded to your submission.

[RunSoul22]

hi @alex-ppg - I left a reply on #685, thanks!

[alex-ppg]

Hey @RunSoul22, I have replied to the latest submission you contributed.

[cryptostaker2]

Hi @alex-ppg, thanks for judging this contest.

Commented on #185, #216, #348

Thanks for your time!

[alex-ppg]

Hey @cryptostaker2, I have responded to each of your requested submissions. Do let me know if there is anything else I can assist with!

[thangtranth]

Hi @alex-ppg , I left a comment on #1599 .
Thank you for very comprehensive judging.

[alex-ppg]

Hey @thangtranth, thanks for contributing! I have responded to your relevant submission.

[BhHaipls]

@alex-ppg Thank you for your judging. I have a few notes to share:

• I've left a comment in The RandomizerVRF uses a keyHash specific to the Goerli test network, not the Ethereum Mainnet. #1611
• In my view, issue The approach of issuing non-winning bids during claimAuction() is a hotbed of problems. #1381 should not be considered a duplicate of AuctionDemo opens itself several DoS attack vectors  #2038, as they differ significantly in substance. The nature of 1381 seems more akin to Adversary can block claimAuction() due to push-strategy to transfer assets to multiple bidders #734, where 1381 discusses a range of problems caused by the chosen approach, similarly impactful as in 734.

[alex-ppg]

Hey @BhHaipls, thanks for bringing these to my attention!

I responded to #1611 and will retain my judgment as the precedence is insufficient in upgrading the submission. You can find more information on the issue itself.

In #1381, you did not put sufficient focus on the gas-related attack vector as it is solely mentioned in point 3 and the main focus was an unbounded participation array. Given that this is OOS being marked high-risk by the bot, it would have been your responsibility to not refer to it. The PoC does illustrate the issue, and based on this I am inclined to award it a partial reward of 25%. For future reference, kindly make sure that your submission does not contain multiple problems and makes no mention of bot reports that have been marked at an appropriate severity level.

A PoC alone should never be considered to demonstrate the issue as the code's test succeeding does not necessarily mean that the vulnerability lies at or is caused by a specific point.

[alexxander77]

Hello @alex-ppg, thanks for the judging effort.

I have two requests for my finding #1925 that is duplicated with #739, kindly consider them

• Malicious winner can hold an auction hostage #1925 describes the Dos attack and also provides PoC code, please consider removing the partial-50 tag

• Malicious winner can hold an auction hostage #1925 also describes issue Adversary can block claimAuction() due to push-strategy to transfer assets to multiple bidders #734, I have put it under "Additional notes" since I assumed it won't be valid due to the bot report but is important for sponsors to know. Now that 734 is accepted as a valid finding, please consider splitting my issue 1925 into two and duplicate it with 734

Thanks !!

[alex-ppg]

Hey @alexxander77, thanks for contributing to the PJQA phase! I will address both of your points:

Penalization of #1925

The finding was penalized because its lines of code simply highlight the full function block, and the impact chapter is a one-liner that does not accurately depict the impact of this behavior. I would personally award it a partial-75, but no such option exists and I am left to decide if it is worthy of a full reward or a partial-50. Based on the comparative quality of the submission, I am inclined to award it a partial-50 reward.

Duplicate of #734

While the submission indeed specifies that the behavior described in #734 may occur, the PoC and overall submission do not focus on it. Instead, it was dismissed as being "correctly" labeled low-risk by the bot report which is incorrect as multiple wardens have showcased.

I understand that you may have been aware of this misbehavior, but the C4 guidelines do specify an escalation of a bot report finding is acceptable as illustrated by multiple other wardens submitting this as an independent issue.

Based on the fact that we can presume you were aware of this guideline and the bot report's labeling of the issue as low-risk, I consider #1925 to not be a duplicate of #734 as it does not accurately establish its validity or impact.

Closing Thoughts

I believe all your submissions as a Warden were of merit and you had a high accuracy in the issues you submitted, however, I will have to maintain my stance on the above ruling.

[udsene]

Hi @alex-ppg,

Thanks for judging this contest!

I left a comment on #1771

Thank you.

[alex-ppg]

Thanks for contributing to the PJQA phase! I have re-reviewed your submission and considered the initial ruling correct.

[marcotnunes]

[alex-ppg]

Hey @ciphermarco, thanks for ensuring a high-quality PJQA period! I have responded to the relevant submission with an in-depth comment that should provide my viewpoint, address the concerns that were raised in the various comments of the thread, and solidify the submission's ruling.

[jorgect207]

Hey @alex-ppg, I left a comment on #626

Thanks in advance!

[alex-ppg]

Hey @jorgect207, thanks for bringing this to my attention. I have responded to the relevant submission as to why it was invalidated.

[jorgect207]

Thanks so much, I respond you back

[alex-ppg]

Thanks for providing further feedback! I responded as well, the getPrice value is correct during a re-entrancy so no such attack vector exists and I also elaborated on your submission as to why it would not have been considered despite this. The PJQA period is over and as such I advise refraining from supplying further comments.

[liveactionllama]

As a reminder, the post-judging QA feedback period has ended. The judge will be reviewing all comments before finalizing judging. Please do not make any further comments on this discussion unless specifically requested by the judge. Thank you!

Edit: Just to clarify, no further comments should be made on Github issues either. This phase has ended.