Assertion failure in Perl__utf8n_to_uvchr_msgs_helper (utf8.c:1912)

[p5pRT]

Migrated from rt.perl.org#133880 (status was 'resolved')

Searchable as RT133880$

[p5pRT]

From @dur-randir

Created by @dur-randir

While fuzzing perl v5.29.8-21-gde59f38ed9 built with afl and run
under libdislocator, I found the following program

y/\x{a00}0-\N{}//

to cause an assertion failure​:

perl​: utf8.c​:1912​: UV Perl__utf8n_to_uvchr_msgs_helper(const U8 *,
STRLEN, STRLEN *, const U32, U32 *, AV **)​: Assertion `0' failed.

This resembled https://rt.perl.org/Public/Bug/Display.html?id=130675
and is revealed by the same commit d1f8d42. GDB stack trace is
following​:

#0 __GI_raise (sig=sig@​entry=6) at ../sysdeps/unix/sysv/linux/raise.c​:50
#1 0x00007ff018eea535 in __GI_abort () at abort.c​:79
#2 0x00007ff018eea40f in __assert_fail_base (fmt=0x7ff01904cee0
"%s%s%s​:%u​: %s%sAssertion `%s' failed.\n%n", assertion=0x55a8ad987d67
"0",
  file=0x55a8ad987465 "utf8.c", line=1912, function=<optimized out>)
at assert.c​:92
#3 0x00007ff018ef80f2 in __GI___assert_fail (assertion=0x55a8ad987d67
"0", file=0x55a8ad987465 "utf8.c", line=1912,
  function=0x55a8ad990500 <__PRETTY_FUNCTION__.18655>
"Perl__utf8n_to_uvchr_msgs_helper") at assert.c​:101
#4 0x000055a8ad765c71 in Perl__utf8n_to_uvchr_msgs_helper
(s=0x7ff015df8fff "", curlen=0, retlen=0x7ffe210dc1a0, flags=0,
errors=0x7ffe210dbff4, msgs=0x0)
  at utf8.c​:1912
#5 0x000055a8ad43d558 in S_utf8n_to_uvchr_msgs (s=0x7ff015df8fff "",
curlen=0, retlen=0x7ffe210dc1a0, flags=0, errors=0x0, msgs=0x0) at
inline.h​:1842
#6 0x000055a8ad45233f in S_pmtrans (o=0x7ff015e20fc8,
expr=0x7ff015e20f88, repl=0x7ff015e20f48) at op.c​:6577
#7 0x000055a8ad45387f in Perl_pmruntime (o=0x7ff015e20fc8,
expr=0x7ff015e20f88, repl=0x7ff015e20f48, flags=1, floor=0) at
op.c​:6941
#8 0x000055a8ad50a51c in Perl_yyparse (gramtype=258) at perly.y​:1228
#9 0x000055a8ad6d14cd in S_doeval_compile (gimme=1 '\001',
outside=0x0, seq=4294967263, hh=0x0) at pp_ctl.c​:3501
#10 0x000055a8ad6d7fe9 in S_require_file (sv=0x7ff0186b1298) at pp_ctl.c​:4321
#11 0x000055a8ad6d8112 in Perl_pp_require () at pp_ctl.c​:4345
#12 0x000055a8ad5a392d in Perl_runops_debug () at dump.c​:2537
#13 0x000055a8ad487124 in S_run_body (oldscope=1) at perl.c​:2692
#14 0x000055a8ad4866a2 in perl_run (my_perl=0x7ff019291fff) at perl.c​:2615
#15 0x000055a8ad43b12e in main (argc=4, argv=0x7ffe210dcfe8,
env=0x7ffe210dd010) at perlmain.c​:127

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.29.9:

Configured by dur-randir at Wed Feb 27 14:51:01 MSK 2019.

Summary of my perl5 (revision 5 version 29 subversion 9) configuration:
  Commit id: c1e47bad34ce1d9c84ed57c9b8978bcbd5a02e98
  Platform:
    osname=darwin
    osvers=13.4.0
    archname=darwin-thread-multi-2level
    uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0:
mon jan 11 18:17:34 pst 2016; root:xnu-2422.115.15~1release_x86_64
x86_64 '
    config_args='-de -Dusedevel -DDEBUGGING -Dusethreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -DPERL_USE_SAFE_PUTENV'
    optimize='-O3 -g'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=10.9 -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
    libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined
dynamic_lookup -L/usr/local/lib -fstack-protector'



@INC for perl 5.29.9:
    lib
    /usr/local/lib/perl5/site_perl/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/site_perl/5.29.9
    /usr/local/lib/perl5/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/5.29.9


Environment for perl 5.29.9:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/dur-randir
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin
    PERLBREW_HOME=/Users/dur-randir/.perlbrew
    PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/Users/dur-randir/perlbrew
    PERLBREW_SHELLRC_VERSION=0.84
    PERLBREW_VERSION=0.84
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

[p5pRT]

@khwilliamson - Status changed from 'new' to 'open'

[p5pRT]

From @khwilliamson

Fixed by
commit ef65a74
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Mar 21 09​:35​:49 2019 -0600

  PATCH​: [perl #133880] assertion failure
 
  This was caused by attempting to continue parsing after an error is
  found, and later assuming that what came before was valid. The fix is
  to put in something valid that's usable until the parse eventually dies
  from what caused this, or some other error.

--
Karl Williamson

[p5pRT]

@khwilliamson - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'