heap-use-after-free in S_free_codeblocks (regcomp.c:6136)

[p5pRT]

Migrated from rt.perl.org#130650 (status was 'resolved')

Searchable as RT130650$

[p5pRT]

From @dur-randir

Created by @dur-randir

While fuzzing perl v5.25.9-35-g32207c637b built with afl and run
under libdislocator, I found the following program

qr'(?{})\6'

to perform an access outside of an allocated memory slot. ASAN diagnostics are​:

Reference to nonexistent group in regex; marked by <-- HERE in
m/(?{})\6 <-- HERE / at 0043 line 1.

==30807==ERROR​: AddressSanitizer​: heap-use-after-free on address
0x60200000dd50 at pc 0x0000007b47cc bp 0x7fffb07c9f00 sp
0x7fffb07c9ef8
READ of size 1 at 0x60200000dd50 thread T0
  #0 0x7b47cb in S_free_codeblocks /home/afl/afl-asan/regcomp.c​:6136​:14
  #1 0xaaa2dc in Perl_leave_scope /home/afl/afl-asan/scope.c​:1256​:6
  #2 0x5f7627 in S_my_exit_jump /home/afl/afl-asan/perl.c​:5200​:5
  #3 0x605e6e in Perl_my_failure_exit /home/afl/afl-asan/perl.c​:5184​:5
  #4 0xae1d24 in Perl_die_unwind /home/afl/afl-asan/pp_ctl.c​:1771​:5
  #5 0x85b256 in Perl_vcroak /home/afl/afl-asan/util.c​:1808​:5
  #6 0x8513a3 in Perl_croak /home/afl/afl-asan/util.c​:1853​:5
  #7 0x8006b4 in S_regatom /home/afl/afl-asan/regcomp.c​:12582​:2
  #8 0x7db579 in S_regpiece /home/afl/afl-asan/regcomp.c​:11655​:11
  #9 0x7db579 in S_regbranch /home/afl/afl-asan/regcomp.c​:11580
  #10 0x74d446 in S_reg /home/afl/afl-asan/regcomp.c​:11318​:10
  #11 0x72dbac in Perl_re_op_compile /home/afl/afl-asan/regcomp.c​:7304​:9
  #12 0x548e33 in Perl_pmruntime /home/afl/afl-asan/op.c​:5882​:19
  #13 0x6fca80 in Perl_yyparse /home/afl/afl-asan/perly.y​:1204​:23
  #14 0x5ec465 in S_parse_body /home/afl/afl-asan/perl.c​:2376​:9
  #15 0x5e2a6f in perl_parse /home/afl/afl-asan/perl.c​:1691​:2
  #16 0x5224a6 in main /home/afl/afl-asan/perlmain.c​:121​:18
  #17 0x7f7d5bd1c2b0 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
  #18 0x43adb9 in _start (/home/afl/afl-asan/perl+0x43adb9)

0x60200000dd50 is located 0 bytes inside of 16-byte region
[0x60200000dd50,0x60200000dd60)
freed by thread T0 here​:
  #0 0x4ea510 in __interceptor_cfree.localalias.1
(/home/afl/afl-asan/perl+0x4ea510)
  #1 0x851dfa in Perl_safesysfree /home/afl/afl-asan/util.c​:388​:2
  #2 0x7218c4 in S_free_codeblocks /home/afl/afl-asan/regcomp.c​:6141​:5
  #3 0x7218c4 in Perl_regfree_internal /home/afl/afl-asan/regcomp.c​:19526
  #4 0x7b1c1d in Perl_pregfree2 /home/afl/afl-asan/regcomp.c​:19379​:9
  #5 0x9bde08 in Perl_sv_clear /home/afl/afl-asan/sv.c​:6607​:6
  #6 0x9c4b4b in Perl_sv_free2 /home/afl/afl-asan/sv.c​:7061​:9
  #7 0xaad123 in S_SvREFCNT_dec /home/afl/afl-asan/./inline.h​:189​:6
  #8 0xaad123 in Perl_leave_scope /home/afl/afl-asan/scope.c​:917
  #9 0x5f7627 in S_my_exit_jump /home/afl/afl-asan/perl.c​:5200​:5
  #10 0x605e6e in Perl_my_failure_exit /home/afl/afl-asan/perl.c​:5184​:5
  #11 0xae1d24 in Perl_die_unwind /home/afl/afl-asan/pp_ctl.c​:1771​:5
  #12 0x85b256 in Perl_vcroak /home/afl/afl-asan/util.c​:1808​:5
  #13 0x8513a3 in Perl_croak /home/afl/afl-asan/util.c​:1853​:5
  #14 0x8006b4 in S_regatom /home/afl/afl-asan/regcomp.c​:12582​:2
  #15 0x7db579 in S_regpiece /home/afl/afl-asan/regcomp.c​:11655​:11
  #16 0x7db579 in S_regbranch /home/afl/afl-asan/regcomp.c​:11580
  #17 0x74d446 in S_reg /home/afl/afl-asan/regcomp.c​:11318​:10
  #18 0x72dbac in Perl_re_op_compile /home/afl/afl-asan/regcomp.c​:7304​:9
  #19 0x548e33 in Perl_pmruntime /home/afl/afl-asan/op.c​:5882​:19
  #20 0x6fca80 in Perl_yyparse /home/afl/afl-asan/perly.y​:1204​:23
  #21 0x5ec465 in S_parse_body /home/afl/afl-asan/perl.c​:2376​:9
  #22 0x5e2a6f in perl_parse /home/afl/afl-asan/perl.c​:1691​:2
  #23 0x5224a6 in main /home/afl/afl-asan/perlmain.c​:121​:18
  #24 0x7f7d5bd1c2b0 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x202b0)

previously allocated by thread T0 here​:
  #0 0x4ea6c8 in malloc (/home/afl/afl-asan/perl+0x4ea6c8)
  #1 0x850ffe in Perl_safesysmalloc /home/afl/afl-asan/util.c​:153​:21
  #2 0x72597d in S_alloc_code_blocks /home/afl/afl-asan/regcomp.c​:6149​:5
  #3 0x72597d in Perl_re_op_compile /home/afl/afl-asan/regcomp.c​:6871
  #4 0x548e33 in Perl_pmruntime /home/afl/afl-asan/op.c​:5882​:19
  #5 0x6fca80 in Perl_yyparse /home/afl/afl-asan/perly.y​:1204​:23
  #6 0x5ec465 in S_parse_body /home/afl/afl-asan/perl.c​:2376​:9
  #7 0x5e2a6f in perl_parse /home/afl/afl-asan/perl.c​:1691​:2
  #8 0x5224a6 in main /home/afl/afl-asan/perlmain.c​:121​:18
  #9 0x7f7d5bd1c2b0 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x202b0)

This is a recent regression in blead, bisect points to

1acab4c is the first bad commit
commit 1acab4c
Author​: David Mitchell <davem@​iabyn.com>
Date​: Sun Oct 30 12​:15​:03 2016 +0000

  better handle freeing of code blocks in /(?{...})/

  [perl #129140] attempting double-free

  Thus fixes some leaks and double frees in regexes which contain code
  blocks.

  During compilation, an array of struct reg_code_block's is malloced.
  Initially this is just attached to the RExC_state_t struct local var in
  Perl_re_op_compile(). Later it may be attached to a pattern. The difficulty
  is ensuring that the array is free()d (and the ref counts contained within
  decremented) should compilation croak early, while avoiding double frees
  once the array has been attached to a regex.

  The current mechanism of making the array the PVX of an SV is a bit flaky,
  as the array can be realloced(), and code can be re-entered when utf8 is
  detected mid-compilation.

  This commit changes the array into separately malloced head and body.
  The body contains the actual array, and can be realloced. The head
  contains a pointer to the array, plus size and an 'attached' boolean.
  This indicates whether the struct has been attached to a regex, and is
  effectively a 1-bit ref count.

  Whenever a head is allocated, SAVEDESTRUCTOR_X() is used to call
  S_free_codeblocks() to free the head and body on scope exit. This function
  skips the freeing if 'attached' is true, and this flag is set only at the
  point where the head gets attached to the regex.

  In one way this complicates the code, since the num_code_blocks field is now
  not always available (it's only there is a head has been allocated), but
  mainly its simplifies, since all the book-keeping is now done in the two
  new static functions S_alloc_code_blocks() and S_free_codeblocks()

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.25.9:

Configured by root at Sat Jan 14 02:25:05 MSK 2017.

Summary of my perl5 (revision 5 version 25 subversion 9) configuration:
  Commit id: cbe2fc5001aa59cdc73e04cc35e097a2ecfbeec0
  Platform:
    osname=linux
    osvers=3.16.0-4-amd64
    archname=x86_64-linux
    uname='linux dorothy 3.16.0-4-amd64 #1 smp debian 3.16.36-1+deb8u2
(2016-10-19) x86_64 gnulinux '
    config_args='-des -Dusedevel -DDEBUGGING -Dcc=afl-clang-fast
-Doptimize=-O0 -g -ggdb3'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    bincompat5005=undef
  Compiler:
    cc='afl-clang-fast'
    ccflags ='-DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O0 -g -ggdb3'
    cppflags='-DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Clang 3.9.1 (tags/RELEASE_391/rc2)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='afl-clang-fast'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/lib
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu
/lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.24.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.24'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O0 -g -ggdb3 -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.25.9:
    lib
    /usr/local/lib/perl5/site_perl/5.25.9/x86_64-linux
    /usr/local/lib/perl5/site_perl/5.25.9
    /usr/local/lib/perl5/5.25.9/x86_64-linux
    /usr/local/lib/perl5/5.25.9


Environment for perl 5.25.9:
    HOME=/home/afl
    LANG=en_US.UTF-8
    LANGUAGE=en_US:en
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/afl/perlbrew/bin:/home/afl/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
    PERLBREW_BASHRC_VERSION=0.78
    PERLBREW_HOME=/home/afl/.perlbrew
    PERLBREW_MANPATH=/home/afl/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/home/afl/perlbrew/bin:/home/afl/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/home/afl/perlbrew
    PERLBREW_VERSION=0.78
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

[p5pRT]

From @tonycoz

On Thu, 26 Jan 2017 05​:01​:17 -0800, randir wrote​:

While fuzzing perl v5.25.9-35-g32207c637b built with afl and run
under libdislocator, I found the following program

qr'(?{})\6'

to perform an access outside of an allocated memory slot. ASAN
diagnostics are​:

Reference to nonexistent group in regex; marked by <-- HERE in
m/(?{})\6 <-- HERE / at 0043 line 1.

==30807==ERROR​: AddressSanitizer​: heap-use-after-free on address
0x60200000dd50 at pc 0x0000007b47cc bp 0x7fffb07c9f00 sp
0x7fffb07c9ef8

This requires feeding code to the parser (the (?{}} block) which is
normally protected by C< use re 'eval'; >.

This isn't a security issue, so I've moved it to the public queue.

I suspect regfree_internal() simply shouldn't free the code blocks.

Tony

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @iabyn

On Tue, Jan 31, 2017 at 06​:50​:15PM -0800, Tony Cook via RT wrote​:

On Thu, 26 Jan 2017 05​:01​:17 -0800, randir wrote​:

While fuzzing perl v5.25.9-35-g32207c637b built with afl and run
under libdislocator, I found the following program

qr'(?{})\6'

to perform an access outside of an allocated memory slot. ASAN
diagnostics are​:

Reference to nonexistent group in regex; marked by <-- HERE in
m/(?{})\6 <-- HERE / at 0043 line 1.

==30807==ERROR​: AddressSanitizer​: heap-use-after-free on address
0x60200000dd50 at pc 0x0000007b47cc bp 0x7fffb07c9f00 sp
0x7fffb07c9ef8

This requires feeding code to the parser (the (?{}} block) which is
normally protected by C< use re 'eval'; >.

This isn't a security issue, so I've moved it to the public queue.

I suspect regfree_internal() simply shouldn't free the code blocks.

Fixed with the following​:

  commit f8def6c
  Author​: David Mitchell <davem@​iabyn.com>
  AuthorDate​: Wed Feb 1 15​:50​:14 2017 +0000
  Commit​: David Mitchell <davem@​iabyn.com>
  CommitDate​: Wed Feb 1 16​:09​:07 2017 +0000

  avoid double-freeing regex code blocks
 
  RT #130650 heap-use-after-free in S_free_codeblocks
 
  When compiling qr/(?{...})/, a reg_code_blocks structure is allocated
  and various SVs are attached to it. Initially this is set to be freed
  via a destructor on the savestack, in case of early dying. Later the
  structure is attached to the compiling regex, and a boolean flag in the
  structure, 'attached', is set to true to show that the destructor no
  longer needs to free the struct.
 
  However, it is possible to get three orders of destruction​:
 
  1) allocate, push destructor, die early
  2) allocate, push destructor, attach to regex, die
  2) allocate, push destructor, attach to regex, succeed
 
  In 2, the regex is freed (via the savestack) before the destructor is
  called. In 3, the destructor is called, then later the regex is freed.
 
  It turns out perl can't currently handle case 2​:
 
  qr'(?{})\6'
 
  Fix this by turning the 'attached' boolean field into an integer refcount,
  then keep a count of whether the struct is referenced from the savestack
  and/or the regex. Since it normally has a value of 1 or 2, it's similar
  to a boolean flag, but crucially it no longer just indicates that the
  regex has a pointer to it ('attached'), but that at least one of the
  savestack and regex have a pointer to it. So order of freeing no longer
  matters.
 
  I also updated S_free_codeblocks() so that it nulls out SV pointers in
  the reg_code_blocks struct before freeing them. This is is generally good
  practice to avoid double frees, although is probably not needed at the
  moment.

--
But Pity stayed his hand. "It's a pity I've run out of bullets",
he thought. -- "Bored of the Rings"

[p5pRT]

@iabyn - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0, this and 210 other issues have been
resolved.

Perl 5.26.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'