Storable null pointer deref on truncated data #9822

p5pRT · 2009-08-09T13:56:30Z

Migrated from rt.perl.org#68348 (status was 'resolved')

Searchable as RT68348$

p5pRT · 2009-08-09T13:56:32Z

From @dgl

Created by @dgl

When deserialising truncated storable data where the truncation is within a
coderef there seems to be a null pointer dereference:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at Storable.xs:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! */

text is 0x0.

Code to reproduce is below, rather oddly if you uncomment the use v5.10 line it
doesn't seem to segfault.

# Uncommenting the line below appears to stop the segfault
#use v5.10;
use Storable qw(freeze thaw);

$Storable::Eval = 1;
$Storable::Deparse = 1;
for(4000..5000) {
print "$_\n";

my $s = {
i => sub { "foo" },
x => "y" x $_,
y => sub { "foo" },
};

thaw(substr(freeze($s), 0, 4096));
}

It seems to vary slightly on different machines but this segfaults after
27-31 iterations for me.

Perl Info


Flags:
    category=library
    severity=medium

Site configuration information for perl 5.10.0:

Configured by Debian Project at Fri Jun 26 18:43:11 UTC 2009.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.6.24-23-server, archname=i486-linux-gnu-thread-multi
    uname='linux rothera 2.6.24-23-server #1 smp wed apr 1 22:22:14 utc 2009 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.0 -Dsitearch=/usr/local/lib/perl/5.10.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.0 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.3.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/lib64
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.9.so, so=so, useshrplib=true, libperl=libperl.so.5.10.0
    gnulibc_version='2.9'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib'

Locally applied patches:
    


@INC for perl 5.10.0:
    /etc/perl
    /usr/local/lib/perl/5.10.0
    /usr/local/share/perl/5.10.0
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.10
    /usr/share/perl/5.10
    /usr/local/lib/site_perl
    .


Environment for perl 5.10.0:
    HOME=/home/dgl
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/lib/jvm/java-1.5.0-sun/bin:/home/dgl/bin:/sbin:/usr/sbin:/usr/local/sbin:/sw/bin:/sw/sbin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/zsh

p5pRT · 2009-08-10T15:14:34Z

From [email protected]

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at Storable.xs:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! */

text is 0x0.

Code to reproduce is below, rather oddly if you uncomment the use v5.10 line it
doesn't seem to segfault.

# Uncommenting the line below appears to stop the segfault
#use v5.10;
use Storable qw(freeze thaw);

$Storable::Eval = 1;
$Storable::Deparse = 1;
for(4000..5000) {
print "$_\n";

my $s = {
i => sub { "foo" },
x => "y" x $_,
y => sub { "foo" },
};

thaw(substr(freeze($s), 0, 4096));
}

It seems to vary slightly on different machines but this segfaults after
27-31 iterations for me.

Thanks for this report.

I can reproduce it down to perl 5.8.1, either with threads enabled or
not (I have no 5.8.0 to test against), so it's not a regression.
On 5.10.0, valgrind says :

$ valgrind perl5.10.0-dbg x.pl
==4168== Memcheck, a memory error detector.
==4168== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==4168== Using LibVEX rev 1906, a library for dynamic binary translation.
==4168== Copyright (C) 2004-2009, and GNU GPL'd, by OpenWorks LLP.
==4168== Using valgrind-3.5.0.SVN, a dynamic binary instrumentation
framework.
==4168== Copyright (C) 2000-2009, and GNU GPL'd, by Julian Seward et al.
==4168== For more details, rerun with: -v
==4168==
4000
4001
4002
4003
4004
4005
4006
4007
4008
4009
4010
4011
4012
4013
4014
4015
4016
4017
4018
4019
4020
4021
4022
4023
4024
4025
4026
==4168== Invalid read of size 4
==4168== at 0x47CE9F5: retrieve_code (Storable.xs:5438)
==4168== by 0x47D26B0: retrieve (Storable.xs:5967)
==4168== by 0x47C9079: retrieve_ref (Storable.xs:4483)
==4168== by 0x47D26B0: retrieve (Storable.xs:5967)
==4168== by 0x47CDC52: retrieve_hash (Storable.xs:5214)
==4168== by 0x47D26B0: retrieve (Storable.xs:5967)
==4168== by 0x47D3725: do_retrieve (Storable.xs:6148)
==4168== by 0x47D3BE2: mretrieve (Storable.xs:6264)
==4168== by 0x47D4D19: XS_Storable_mretrieve (Storable.xs:6460)
==4168== by 0x811F702: Perl_pp_entersub (pp_hot.c:2847)
==4168== by 0x80C5E30: Perl_runops_debug (dump.c:1931)
==4168== by 0x80FE980: S_run_body (perl.c:2384)
==4168== Address 0x8 is not stack'd, malloc'd or (recently) free'd
==4168==
==4168== Process terminating with default action of signal 11 (SIGSEGV)
==4168== Access not within mapped region at address 0x8
==4168== at 0x47CE9F5: retrieve_code (Storable.xs:5438)
==4168== by 0x47D26B0: retrieve (Storable.xs:5967)
==4168== by 0x47C9079: retrieve_ref (Storable.xs:4483)
==4168== by 0x47D26B0: retrieve (Storable.xs:5967)
==4168== by 0x47CDC52: retrieve_hash (Storable.xs:5214)
==4168== by 0x47D26B0: retrieve (Storable.xs:5967)
==4168== by 0x47D3725: do_retrieve (Storable.xs:6148)
==4168== by 0x47D3BE2: mretrieve (Storable.xs:6264)
==4168== by 0x47D4D19: XS_Storable_mretrieve (Storable.xs:6460)
==4168== by 0x811F702: Perl_pp_entersub (pp_hot.c:2847)
==4168== by 0x80C5E30: Perl_runops_debug (dump.c:1931)
==4168== by 0x80FE980: S_run_body (perl.c:2384)
==4168== If you believe this happened as a result of a stack
==4168== overflow in your program's main thread (unlikely but
==4168== possible), you can try to increase the size of the
==4168== main thread stack using the --main-stacksize= flag.
==4168== The main thread stack size used in this run was 8388608.
==4168==
==4168== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 17 from 2)
==4168== malloc/free: in use at exit: 2,450,124 bytes in 52,272 blocks.
==4168== malloc/free: 92,241 allocs, 39,969 frees, 5,604,415 bytes
allocated.
==4168== For counts of detected errors, rerun with: -v
==4168== searching for pointers to 52,272 not-freed blocks.
==4168== checked 2,550,944 bytes.
==4168==
==4168== LEAK SUMMARY:
==4168== definitely lost: 176 bytes in 1 blocks.
==4168== indirectly lost: 3,393 bytes in 42 blocks.
==4168== possibly lost: 1,679,240 bytes in 42,509 blocks.
==4168== still reachable: 767,315 bytes in 9,720 blocks.
==4168== suppressed: 0 bytes in 0 blocks.
==4168== Rerun with --leak-check=full to see details of leaked memory.

Indeed, if you "use 5.010", the error disappears - but not if you "use
5.008" with 5.8.x.
It seems that filling the hints hash with something prevents the bug to
happen, as

BEGIN { $^H{foo} = 1 }
use Storable qw(freeze thaw);
...

also doesn't segfault and is valgrind-clean.

Vincent.

p5pRT · 2009-08-10T15:14:36Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2009-08-10T15:28:39Z

From [email protected]

Indeed, if you "use 5.010", the error disappears - but not if you "use
5.008" with 5.8.x.
It seems that filling the hints hash with something prevents the bug to
happen, as
BEGIN \{ $^H\{foo\} = 1 \}
use Storable qw$freeze thaw$;
\.\.\.
also doesn't segfault and is valgrind-clean.

More precisely, setting the hint must happen before initializing $s, as
this does not produce an error :

BEGIN { $^H{foo} = 1 }
my $s = {
i => sub { "foo" },
x => "y" x $_,
y => sub { "foo" },
};

while this does :

my $s = {
i => sub { "foo" },
x => "y" x $_,
y => sub { "foo" },
};
BEGIN { $^H{foo} = 1 }

Vincent.

p5pRT · 2016-11-21T18:27:26Z

From [email protected]

On Sun, 09 Aug 2009 06:56:32 -0700, dgl wrote:

When deserialising truncated storable data where the truncation is
within a
coderef there seems to be a null pointer dereference:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at
Storable.xs:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv!
*/

text is 0x0.

The patch attached to https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130098 fixes this segfault.

p5pRT · 2016-12-25T02:51:53Z

From @jkeenan

On Mon, 21 Nov 2016 18:27:26 GMT, lightsey@cpan.org wrote:

On Sun, 09 Aug 2009 06:56:32 -0700, dgl wrote:

When deserialising truncated storable data where the truncation is
within a
coderef there seems to be a null pointer dereference:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at
Storable.xs:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv!
*/

text is 0x0.

The patch attached to
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130098 fixes this segfault.

Please review the smoke-me/jkeenan/130098-storable branch. Can we add a regression test to t/store.t in that branch that specifically addresses the problem reported in this ticket?

Thank you very much.

--
James E Keenan (jkeenan@cpan.org)

p5pRT · 2017-01-01T15:19:12Z

From @jkeenan

On Mon, 21 Nov 2016 18:27:26 GMT, lightsey@cpan.org wrote:

On Sun, 09 Aug 2009 06:56:32 -0700, dgl wrote:

When deserialising truncated storable data where the truncation is
within a
coderef there seems to be a null pointer dereference:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at
Storable.xs:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv!
*/

text is 0x0.

The patch attached to
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130098 fixes this segfault.

Can you confirm that commit adf9095 to blead remedies this situation?

Thank you very much.

--
James E Keenan (jkeenan@cpan.org)

p5pRT · 2017-01-07T02:13:41Z

From @jkeenan

On Sun, 01 Jan 2017 15:19:12 GMT, jkeenan wrote:

On Mon, 21 Nov 2016 18:27:26 GMT, lightsey@cpan.org wrote:

On Sun, 09 Aug 2009 06:56:32 -0700, dgl wrote:

When deserialising truncated storable data where the truncation is
within a
coderef there seems to be a null pointer dereference:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at
Storable.xs:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no
sv_catsv!
*/

text is 0x0.

The patch attached to
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130098 fixes this
segfault.

Can you confirm that commit adf9095
to blead remedies this situation?

Thank you very much.

I don't think that commit fixes the problem.

When I build a threaded perl at blead (see attachment) and then run the program in the original post in this ticket, I get output like this:

#####
$ ./bin/perl -Ilib ~/learn/perl/p5p/68348-storable.pl
4000
4001
4002
4003
4004
4005
4006
4007
Unable to retrieve code, at /home/jkeenan/learn/perl/p5p/68348-storable.pl line 15.
#####

Any ideas?

Thank you very much.
--
James E Keenan (jkeenan@cpan.org)

p5pRT · 2017-01-07T02:13:41Z

From @jkeenan

Summary of my perl5 (revision 5 version 25 subversion 9) configuration:
Commit id: 9065925
Platform:
osname=linux
osvers=4.4.0-57-generic
archname=x86_64-linux-thread-multi
uname='linux zareason 4.4.0-57-generic #78-ubuntu smp fri dec 9 23:50:32 utc 2016 x86_64 x86_64 x86_64 gnulinux '
config_args='-des -Dusedevel -Dusethreads -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
bincompat5005=undef
Compiler:
cc='cc'
ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
optimize='-O2'
cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='5.4.0 20160609'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.23.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.23'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
PERL_USE_DEVEL
USE_64_BIT_ALL
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
Built under linux
Compiled at Jan 6 2017 21:04:04
%ENV:
PERLBREW_BASHRC_VERSION="0.67"
PERLBREW_HOME="/home/jkeenan/.perlbrew"
PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/man"
PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin:/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/bin"
PERLBREW_PERL="perl-5.24.0"
PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"
PERLBREW_VERSION="0.67"
PERL_WORKDIR="/home/jkeenan/gitwork/perl"
@INC:
lib
/home/jkeenan/testing/blead/lib/perl5/site_perl/5.25.9/x86_64-linux-thread-multi
/home/jkeenan/testing/blead/lib/perl5/site_perl/5.25.9
/home/jkeenan/testing/blead/lib/perl5/5.25.9/x86_64-linux-thread-multi
/home/jkeenan/testing/blead/lib/perl5/5.25.9
.

p5pRT · 2017-01-07T02:13:41Z

From @jkeenan

68348-storable.pl

p5pRT · 2017-01-10T15:04:29Z

From @lightsey

On Fri, 2017-01-06 at 18:13 -0800, James E Keenan via RT wrote:

On Sun, 01 Jan 2017 15:19:12 GMT, jkeenan wrote:

Can you confirm that commit adf9095
to blead remedies this situation?
Â
Thank you very much.

I don't think that commit fixes the problem.

When I build a threaded perl at blead (see attachment) and then run the
program in the original post in this ticket, I get output like this:

#####
$ ./bin/perl -Ilib ~/learn/perl/p5p/68348-storable.plÂ
4000
4001
4002
4003
4004
4005
4006
4007
Unable to retrieve code, at /home/jkeenan/learn/perl/p5p/68348-storable.pl
line 15.
#####

Any ideas?

Your results show the problem is fixed. Storable is dieing instead of
segfaulting now.

The data is corrupted by the substr() on the 8th iteration, so it can't be
loaded. The segfault was the bug.

p5pRT · 2017-01-11T12:29:55Z

@iabyn - Status changed from 'open' to 'resolved'

p5pRT closed this as completed Jan 11, 2017

p5pRT added Severity Medium distro-Linux hasCoreDump type-library labels Oct 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Storable null pointer deref on truncated data #9822

Storable null pointer deref on truncated data #9822

p5pRT commented Aug 9, 2009

p5pRT commented Aug 9, 2009

p5pRT commented Aug 10, 2009

p5pRT commented Aug 10, 2009

p5pRT commented Aug 10, 2009

p5pRT commented Nov 21, 2016

p5pRT commented Dec 25, 2016

p5pRT commented Jan 1, 2017

p5pRT commented Jan 7, 2017

p5pRT commented Jan 7, 2017

p5pRT commented Jan 7, 2017

p5pRT commented Jan 10, 2017

p5pRT commented Jan 11, 2017

Storable null pointer deref on truncated data #9822

Storable null pointer deref on truncated data #9822

Comments

p5pRT commented Aug 9, 2009

p5pRT commented Aug 9, 2009

From @dgl

Created by @dgl

p5pRT commented Aug 10, 2009

From [email protected]

p5pRT commented Aug 10, 2009

p5pRT commented Aug 10, 2009

From [email protected]

p5pRT commented Nov 21, 2016

From [email protected]

p5pRT commented Dec 25, 2016

From @jkeenan

p5pRT commented Jan 1, 2017

From @jkeenan

p5pRT commented Jan 7, 2017

From @jkeenan

p5pRT commented Jan 7, 2017

From @jkeenan

p5pRT commented Jan 7, 2017

From @jkeenan

p5pRT commented Jan 10, 2017

From @lightsey

p5pRT commented Jan 11, 2017