fix some data issues in parallel/parallel multiple answers #423
Conversation
There was a problem hiding this comment.
Hi @vandyxiaowei,
Thanks for contributing!
One minor request: Could we exclude the linting changes from this PR? This would help better highlight the changes to the possible answers. Right now the "Files changed" section indicates over 200 additions and deletions, which are predominantly whitespace alterations. These do not impact the evaluation pipeline's correctness but could obscure the actual modifications to possible answers when other people look at this PR.
There was a problem hiding this comment.
Hey @vandyxiaowei,
I want to sincerely apologize for the delay in reviewing your pull request. The past few months have been particularly busy, and unfortunately, I let this slip through the cracks. Really sorry about the long wait!
I’ve now had a chance to review your changes, and everything looks great. Thank you very much for your contribution!
For other people looking at this PR: Apart from the whitespace adjustments (which are quite a lot, but don't affect evaluation correctness at all), this PR contains 37 possible_answer entry fixes. All of them are due to the missing "" for the optional parameter. Here's a breakdown of the changes:
- parallel function category: 13 entries affected.
- Indices:
3, 73, 103, 105, 113, 115, 116, 127, 129, 146, 189, 195, 200
- Indices:
- parallel_multiple function category: 24 entries affected
- Indices:
29, 61, 64, 67, 72, 75, 76, 85, 89, 94, 95, 97, 100, 101, 117, 119, 126, 129, 132, 156, 157, 184, 185, 195
- Indices:
We will update the leaderboard soon to reflect this change.
…for AST Parallel and Parallel_Multiple Category) (#507) This PR updates the leaderboard to reflect the changes in score due to #423 and #503. Only the score for AST Parallel and AST Parallel_Multiple Categories are affected. We want to thank @vandyxiaowei for helping fix the possible answers.
…til#423) some parameters have default value, so it should be acceptable if the model doesn't predict the parameter. add "" in the possible answers for those cases. Please take a look. Thanks! Co-authored-by: Xiaowei Li <[email protected]> Co-authored-by: Huanzhi (Hans) Mao <[email protected]>
…til#423) some parameters have default value, so it should be acceptable if the model doesn't predict the parameter. add "" in the possible answers for those cases. Please take a look. Thanks! Co-authored-by: Xiaowei Li <[email protected]> Co-authored-by: Huanzhi (Hans) Mao <[email protected]>
some parameters have default value, so it should be acceptable if the model doesn't predict the parameter. add "" in the possible answers for those cases.
Please take a look. Thanks!