Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

t_decode_utf8_lenient fails on Windows without simdutf #529

Closed
Bodigrim opened this issue Jul 10, 2023 · 1 comment · Fixed by #531
Closed

t_decode_utf8_lenient fails on Windows without simdutf #529

Bodigrim opened this issue Jul 10, 2023 · 1 comment · Fixed by #531
Labels

Comments

@Bodigrim
Copy link
Contributor

(Extracted from #528 (comment))

As witnessed in https://github.com/haskell/text/actions/runs/5507568869/jobs/10037691564?pr=528#step:6:228, cabal test -f-simdutf can fail on Windows with

      error recovery
        t_decode_utf8_lenient:                                FAIL (0.01s)
          *** Failed! Falsified (after 99 tests and 55 shrinks):
          ["!\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL","\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\128\NUL"]
          "!\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\65533\NUL" /= "!\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\128\NUL"
          Use --quickcheck-replay=572156 to reproduce.
          Use -p '/t_decode_utf8_lenient/' to rerun this test only.
@Bodigrim Bodigrim changed the title cabal test -f-simdutf t_decode_utf8_lenient fails on Windows without simdutf Jul 10, 2023
@Bodigrim Bodigrim added the bug label Jul 10, 2023
@Bodigrim
Copy link
Contributor Author

Looking at the structure of the offending string, it's most likely haskell/bytestring#575, fixed in bytestring-0.11.5.0. We should raise the bound at which we switch between native validation and bytestring:

#if defined(SIMDUTF) || MIN_VERSION_bytestring(0,11,2)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant