FFI-Platypus
FFI-Platypus copied to clipboard
Published
20 hours ago •
PerlFFI
PerlFFI
Memory leak for simple callback
I have this simple test library in C lib/mylib.c:
#include <stdio.h>
void mylib_func (void (*callback)()) {
printf( "Inside func..\n");
(*callback)();
}
I compiled it with
gcc -c -fpic mylib.c
gcc -shared -o libmylib.so mylib.o
Then I created a test Perl script p.pl:
#! /usr/bin/env perl
use feature qw(say);
use strict;
use warnings;
use FFI::Platypus;
my $ffi = FFI::Platypus->new();
$ffi->lib( 'lib/libmylib.so' );
$ffi->type('()->void' => 'callback_t');
$ffi->attach( mylib_func => [ 'callback_t' ] => 'void' );
my $callback = $ffi->closure(
sub { say "Perl callback()" }
);
mylib_func( $callback );
and running the script seems to work fine:
$ p.pl
Inside func..
Perl callback()
Then I wanted to run it through valgrind to check that everything worked.
First I created a debugging version of perl:
$ perlbrew install perl-5.30.0 --as=5.30.0-D3L -DDEBUGGING \
-Doptimize=-g3 -Accflags="-DDEBUG_LEAKING_SCALARS"
$ perlbrew use 5.30.0-D3L
$ cpanm FFI::Platypus
Then I ran it with valgrind setting PERL_DESTRUCT_LEVEL=2 as recommended in perlhacktips:
$ PERL_DESTRUCT_LEVEL=2 valgrind --leak-check=yes perl p.pl
==18483== Memcheck, a memory error detector
==18483== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==18483== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==18483== Command: perl p.pl
==18483==
Inside func..
Perl callback()
==18483==
==18483== HEAP SUMMARY:
==18483== in use at exit: 12,514 bytes in 65 blocks
==18483== total heap usage: 174,637 allocs, 174,572 frees, 22,909,636 bytes allocated
==18483==
==18483== 1 bytes in 1 blocks are possibly lost in loss record 1 of 18
==18483== at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483== by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483== by 0x7511BF6: XS_FFI__Platypus__TypeParser_create_type_closure (TypeParser.xs:216)
==18483== by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483== by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483== by 0x1A17F9: S_run_body (perl.c:2716)
==18483== by 0x1A0D77: perl_run (perl.c:2639)
==18483== by 0x15514D: main (perlmain.c:127)
==18483==
==18483== 8 bytes in 1 blocks are possibly lost in loss record 2 of 18
==18483== at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483== by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483== by 0x751EF14: ffi_pl_type_new (meta.c:269)
==18483== by 0x750FC74: XS_FFI__Platypus__TypeParser_create_type_basic (TypeParser.xs:37)
==18483== by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483== by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483== by 0x1A2FD9: Perl_call_sv (perl.c:3043)
==18483== by 0x1ACEE3: Perl_call_list (perl.c:5084)
==18483== by 0x181233: S_process_special_blocks (op.c:10471)
==18483== by 0x180989: Perl_newATTRSUB_x (op.c:10397)
==18483== by 0x16F560: Perl_utilize (op.c:7592)
==18483== by 0x221027: Perl_yyparse (perly.y:335)
==18483==
==18483== 56 bytes in 1 blocks are possibly lost in loss record 8 of 18
==18483== at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483== by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483== by 0x751EF14: ffi_pl_type_new (meta.c:269)
==18483== by 0x7511C15: XS_FFI__Platypus__TypeParser_create_type_closure (TypeParser.xs:217)
==18483== by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483== by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483== by 0x1A17F9: S_run_body (perl.c:2716)
==18483== by 0x1A0D77: perl_run (perl.c:2639)
==18483== by 0x15514D: main (perlmain.c:127)
==18483==
==18483== 72 (64 direct, 8 indirect) bytes in 1 blocks are definitely lost in loss record 10 of 18
==18483== at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483== by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483== by 0x74FA472: XS_FFI__Platypus__Function__Function_new (Function.xs:53)
==18483== by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483== by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483== by 0x1A17F9: S_run_body (perl.c:2716)
==18483== by 0x1A0D77: perl_run (perl.c:2639)
==18483== by 0x15514D: main (perlmain.c:127)
==18483==
==18483== 104 bytes in 13 blocks are definitely lost in loss record 11 of 18
==18483== at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483== by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483== by 0x751EF14: ffi_pl_type_new (meta.c:269)
==18483== by 0x750FC74: XS_FFI__Platypus__TypeParser_create_type_basic (TypeParser.xs:37)
==18483== by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483== by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483== by 0x1A2FD9: Perl_call_sv (perl.c:3043)
==18483== by 0x1ACEE3: Perl_call_list (perl.c:5084)
==18483== by 0x181233: S_process_special_blocks (op.c:10471)
==18483== by 0x180989: Perl_newATTRSUB_x (op.c:10397)
==18483== by 0x16F560: Perl_utilize (op.c:7592)
==18483== by 0x221027: Perl_yyparse (perly.y:335)
==18483==
==18483== 112 bytes in 14 blocks are definitely lost in loss record 12 of 18
==18483== at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483== by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483== by 0x751EF14: ffi_pl_type_new (meta.c:269)
==18483== by 0x7510F10: XS_FFI__Platypus__TypeParser_create_type_pointer (TypeParser.xs:127)
==18483== by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483== by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483== by 0x1A2FD9: Perl_call_sv (perl.c:3043)
==18483== by 0x1ACEE3: Perl_call_list (perl.c:5084)
==18483== by 0x181233: S_process_special_blocks (op.c:10471)
==18483== by 0x180989: Perl_newATTRSUB_x (op.c:10397)
==18483== by 0x16F560: Perl_utilize (op.c:7592)
==18483== by 0x221027: Perl_yyparse (perly.y:335)
==18483==
==18483== LEAK SUMMARY:
==18483== definitely lost: 280 bytes in 28 blocks
==18483== indirectly lost: 8 bytes in 1 blocks
==18483== possibly lost: 65 bytes in 3 blocks
==18483== still reachable: 12,161 bytes in 33 blocks
==18483== suppressed: 0 bytes in 0 blocks
==18483== Reachable blocks (those to which a pointer was found) are not shown.
==18483== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==18483==
==18483== For counts of detected and suppressed errors, rerun with: -v
==18483== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)
So it seems there are some memory leaks here? Any idea what the problem is?
Attaching a function does lock some data structures into memory, which does appear as a memory leak. There is a trade off between memory usage and the increased speed of using an attached XSUB. If I do not attach the same script and use a function object like this:
my $mylib_func = $ffi->function( mylib_func => [ 'callback_t' ] => 'void' );
my $callback = $ffi->closure(
sub { say "Perl callback()" }
);
$mylib_func->call( $callback );
then I see only two leaks:
root@f004414c4c0d:/work/examples/leak# PERL_DESTRUCT_LEVEL=2 valgrind --leak-check=yes perl p.pl
==8423== Memcheck, a memory error detector
==8423== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==8423== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==8423== Command: perl p.pl
==8423==
Inside func..
Perl callback()
==8423==
==8423== HEAP SUMMARY:
==8423== in use at exit: 6,142 bytes in 46 blocks
==8423== total heap usage: 43,911 allocs, 43,865 frees, 6,989,634 bytes allocated
==8423==
==8423== 112 bytes in 14 blocks are definitely lost in loss record 5 of 11
==8423== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==8423== by 0x2C48BB: Perl_safesysmalloc (util.c:155)
==8423== by 0x679AD84: ffi_pl_type_new (meta.c:269)
==8423== by 0x678A740: XS_FFI__Platypus__TypeParser_create_type_basic (TypeParser.xs:37)
==8423== by 0x331885: Perl_pp_entersub (pp_hot.c:5237)
==8423== by 0x2C2F8D: Perl_runops_debug (dump.c:2537)
==8423== by 0x1A2D44: Perl_call_sv (perl.c:3043)
==8423== by 0x1ACD39: Perl_call_list (perl.c:5084)
==8423== by 0x180FC7: S_process_special_blocks (op.c:10471)
==8423== by 0x180720: Perl_newATTRSUB_x (op.c:10396)
==8423== by 0x16F284: Perl_utilize (op.c:7592)
==8423== by 0x221678: Perl_yyparse (perly.y:335)
==8423==
==8423== 112 bytes in 14 blocks are definitely lost in loss record 6 of 11
==8423== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==8423== by 0x2C48BB: Perl_safesysmalloc (util.c:155)
==8423== by 0x679AD84: ffi_pl_type_new (meta.c:269)
==8423== by 0x678BE7A: XS_FFI__Platypus__TypeParser_create_type_pointer (TypeParser.xs:147)
==8423== by 0x331885: Perl_pp_entersub (pp_hot.c:5237)
==8423== by 0x2C2F8D: Perl_runops_debug (dump.c:2537)
==8423== by 0x1A2D44: Perl_call_sv (perl.c:3043)
==8423== by 0x1ACD39: Perl_call_list (perl.c:5084)
==8423== by 0x180FC7: S_process_special_blocks (op.c:10471)
==8423== by 0x180720: Perl_newATTRSUB_x (op.c:10396)
==8423== by 0x16F284: Perl_utilize (op.c:7592)
==8423== by 0x221678: Perl_yyparse (perly.y:335)
==8423==
==8423== LEAK SUMMARY:
==8423== definitely lost: 224 bytes in 28 blocks
==8423== indirectly lost: 0 bytes in 0 blocks
==8423== possibly lost: 0 bytes in 0 blocks
==8423== still reachable: 5,918 bytes in 18 blocks
==8423== suppressed: 0 bytes in 0 blocks
==8423== Reachable blocks (those to which a pointer was found) are not shown.
==8423== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==8423==
==8423== For counts of detected and suppressed errors, rerun with: -v
==8423== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
I think this has to do with the way it caches the basic types and global destruction. This could probably be improved, however.
