Assertion failure in Perl_av_push/S_handle_regex_sets

[p5pRT]

Migrated from rt.perl.org#133988 (status was 'resolved')

Searchable as RT133988$

[p5pRT]

From @dur-randir

Created by @dur-randir

While fuzzing perl v5.29.9-63-g2496d8f3f7 built with afl and run
under libdislocator, I found the following

m,(?[\N{U+0.0},

to cause an assertion failure

perl​: av.c​:611​: void Perl_av_push(AV *, SV *)​: Assertion `val' failed.

GDB stack trace is following

#0 __GI_raise (sig=sig@​entry=6) at ../sysdeps/unix/sysv/linux/raise.c​:50
#1 0x00007ffff7c25535 in __GI_abort () at abort.c​:79
#2 0x00007ffff7c2540f in __assert_fail_base (fmt=0x7ffff7d87ee0
"%s%s%s​:%u​: %s%sAssertion `%s' failed.\n%n", assertion=0x555555aa7c1b
"val",
  file=0x555555aa79b6 "av.c", line=611, function=<optimized out>) at
assert.c​:92
#3 0x00007ffff7c330f2 in __GI___assert_fail (assertion=0x555555aa7c1b
"val", file=0x555555aa79b6 "av.c", line=611,
  function=0x555555aa7d38 <__PRETTY_FUNCTION__.18093>
"Perl_av_push") at assert.c​:101
#4 0x0000555555756ca4 in Perl_av_push (av=0x555555b72f00, val=0x0) at av.c​:611
#5 0x00005555556d4f84 in S_handle_regex_sets
(pRExC_state=0x7fffffffd660, return_invlist=0x0, flagp=0x7fffffffca98,
depth=6,
  oregcomp_parse=0x555555b796b1 "?[\\N{U+0.0}") at regcomp.c​:16261
#6 0x00005555556b84c8 in S_reg (pRExC_state=0x7fffffffd660, paren=91,
flagp=0x7fffffffca98, depth=5) at regcomp.c​:11975
#7 0x00005555556c3431 in S_regatom (pRExC_state=0x7fffffffd660,
flagp=0x7fffffffcd24, depth=4) at regcomp.c​:13354
#8 0x00005555556bc7e4 in S_regpiece (pRExC_state=0x7fffffffd660,
flagp=0x7fffffffce40, depth=3) at regcomp.c​:12457
#9 0x00005555556bc0e4 in S_regbranch (pRExC_state=0x7fffffffd660,
flagp=0x7fffffffcee8, first=1, depth=2) at regcomp.c​:12377
#10 0x00005555556b9906 in S_reg (pRExC_state=0x7fffffffd660, paren=0,
flagp=0x7fffffffd398, depth=1) at regcomp.c​:12088
#11 0x000055555569d08c in Perl_re_op_compile (patternp=0x0,
pat_count=1, expr=0x555555b795a8, eng=0x555555b41d20
<PL_core_reg_engine>, old_re=0x0,
  is_bare_re=0x0, orig_rx_flags=0, pm_flags=0) at regcomp.c​:7705
#12 0x00005555555ba159 in Perl_pmruntime (o=0x555555b795e8,
expr=0x555555b795a8, repl=0x0, flags=1, floor=0) at op.c​:7127
#13 0x000055555566ffde in Perl_yyparse (gramtype=258) at perly.y​:1228
#14 0x00005555555ec726 in S_parse_body (env=0x0, xsinit=0x5555555a11f8
<xs_init>) at perl.c​:2531
#15 0x00005555555ea9f8 in perl_parse (my_perl=0x555555b4c260,
xsinit=0x5555555a11f8 <xs_init>, argc=2, argv=0x7fffffffe1c8, env=0x0)
at perl.c​:1822
#16 0x00005555555a113b in main (argc=2, argv=0x7fffffffe1c8,
env=0x7fffffffe1e0) at perlmain.c​:126

This is a regression in blead, and while being very similar to
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133889 bisect points to a
different commit​:

commit 7c932d0
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Oct 19 09​:48​:34 2018 -0600

  Remove sizing pass from regular expression compiler

  This commit removes the sizing pass for regular expression compilation.
  It attempts to be the minimum required to do this. Future patches are
  in the works that improve it,, and there is certainly lots more that
  could be done.

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.29.9:

Configured by dur-randir at Wed Feb 27 14:51:01 MSK 2019.

Summary of my perl5 (revision 5 version 29 subversion 9) configuration:
  Commit id: c1e47bad34ce1d9c84ed57c9b8978bcbd5a02e98
  Platform:
    osname=darwin
    osvers=13.4.0
    archname=darwin-thread-multi-2level
    uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0:
mon jan 11 18:17:34 pst 2016; root:xnu-2422.115.15~1release_x86_64
x86_64 '
    config_args='-de -Dusedevel -DDEBUGGING -Dusethreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -DPERL_USE_SAFE_PUTENV'
    optimize='-O3 -g'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=10.9 -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
    libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined
dynamic_lookup -L/usr/local/lib -fstack-protector'



@INC for perl 5.29.9:
    lib
    /usr/local/lib/perl5/site_perl/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/site_perl/5.29.9
    /usr/local/lib/perl5/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/5.29.9


Environment for perl 5.29.9:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/dur-randir
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin
    PERLBREW_HOME=/Users/dur-randir/.perlbrew
    PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/Users/dur-randir/perlbrew
    PERLBREW_SHELLRC_VERSION=0.84
    PERLBREW_VERSION=0.84
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

[p5pRT]

@khwilliamson - Status changed from 'new' to 'open'

[p5pRT]

From @khwilliamson

Fixed by
commit bc035ee
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Apr 5 16​:34​:21 2019 -0600

  PATCH​: [perl #133988], Assertion failure
 
  This was due to not checking that a \N{} expanded to a single code point
  in contexts where only that is legal. This never could have worked
  properly, though the assertion failure is new.
 
  The diagnostic needs to be reworded to accommodate this new case, but
  its too late in the 5.29 cycle to do that, so I'm changing just the
  description, and will reword in 5.31, [perl #133996]

--
Karl Williamson

[p5pRT]

@khwilliamson - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'