Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid breaking apart testcase name when they contain spaces #279

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

MarcCote
Copy link

@MarcCote MarcCote commented Jan 9, 2025

Reference Issues/PRs

This could be related to what @WuYff observed in #275. In any case, this impacts the current HuggingFace dataset, eg. https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified/viewer/default/test?row=7

What does this implement/fix? Explain your changes.

The function parse_log_pytest_v2(log: str) produces truncated testcase names when they contain spaces in them, eg.

PASSED astropy/coordinates/tests/test_sky_coord.py::test_passing_inconsistent_coordinates_and_units_raises_helpful_error[kwargs0-Unit 'deg' \\(angle\\) could not be applied to 'distance'. ]

This PR fixes that issue.

Minimal reproducible example

from swebench.harness.log_parsers import parse_log_pytest_v2
result = parse_log_pytest_v2("PASSED astropy/coordinates/tests/test_sky_coord.py::test_passing_inconsistent_coordinates_and_units_raises_helpful_error[kwargs0-Unit 'deg' \\(angle\\) could not be applied to 'distance'. ]")
print(result)

{'astropy/coordinates/tests/test_sky_coord.py::test_passing_inconsistent_coordinates_and_units_raises_helpful_error[kwargs0-Unit': 'PASSED'}

after the fix

{"astropy/coordinates/tests/test_sky_coord.py::test_passing_inconsistent_coordinates_and_units_raises_helpful_error[kwargs0-Unit 'deg' \(angle\) could not be applied to 'distance'. ]": 'PASSED'}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant