OpenDKIM icon indicating copy to clipboard operation
OpenDKIM copied to clipboard

from Sourceforge: #273 v2.11.1 coredump, 'signal 6', BT provided

Open ghost opened this issue 7 years ago • 22 comments

On,

    lsb_release -rd
        Description:    openSUSE Leap 15.0
        Release:        15.0
    uname -rm
        4.18.12-lp150.3.ga880bd8-default x86_64

I've built & run,

    /usr/local/opendkim/sbin/opendkim -V
        opendkim: OpenDKIM Filter v2.11.0
                Compiled with OpenSSL 1.1.1  11 Sep 2018
                SMFI_VERSION 0x1000001
                libmilter version 1.0.1
                Supported signing algorithms:
                        rsa-sha1
                        rsa-sha256
                        ed25519-sha256
                Supported canonicalization algorithms:
                        relaxed
                        simple
                Active code options:
                        USE_MDB
                        USE_UNBOUND
                        _FFR_DIFFHEADERS
                        _FFR_STATS
                        _FFR_VBR
                libopendkim 2.11.0: diffheaders debug

Once launched, it runs for a fair while, accepting/verifying submitted msgs, then intermittenly terminates with signal 6,

    journalctl -f | grep -i opendkim
        ...
        Oct 08 11:54:03 mail.loc systemd[1]: Starting OpenDKIM...
        Oct 08 11:54:03 mail.loc systemd[1]: Started OpenDKIM.
        Oct 08 11:54:03 mail.loc opendkim[44544]: OpenDKIM Filter: Opening listen socket on conn local:/var/run/opendkim/opendkim.sock
        Oct 08 11:54:03 mail.loc opendkim[44544]: OpenDKIM Filter v2.11.0 starting (args: -l -x /usr/local/etc/opendkim/opendkim.conf -u opendkim)
        ...
        Oct 08 13:39:16 mail.loc opendkim[44576]: 42TXK023bYz10hm: DKIM-Signature field added (s=dkim-7f99436557929e03, d=example.com)
        Oct 08 13:39:16 mail.loc opendkim[44543]: terminated with signal 6, restarting
        Oct 08 13:39:16 mail.loc opendkim[49996]: OpenDKIM Filter: Opening listen socket on conn local:/var/run/opendkim/opendkim.sock
        Oct 08 13:39:16 mail.loc opendkim[49996]: OpenDKIM Filter v2.11.0 starting (args: -l -x /usr/local/etc/opendkim/opendkim.conf -u opendkim)
        ...

Core dumps

    ls -al /var/lib/systemd/coredump/core*
        -rw------- 1 opendkim postfix-milter 39M Oct  8 13:39 /var/lib/systemd/coredump/core.44576

and a backtrace reports,

    gdb /usr/local/opendkim/sbin/opendkim /var/lib/systemd/coredump/core.44576
        GNU gdb (GDB; openSUSE Leap 15.0) 8.1
        Copyright (C) 2018 Free Software Foundation, Inc.
        License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
        This is free software: you are free to change and redistribute it.
        There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
        and "show warranty" for details.
        This GDB was configured as "x86_64-suse-linux".
        Type "show configuration" for configuration details.
        For bug reporting instructions, please see:
        <http://bugs.opensuse.org/>.
        Find the GDB manual and other documentation resources online at:
        <http://www.gnu.org/software/gdb/documentation/>.
        For help, type "help".
        Type "apropos word" to search for commands related to "word"...
        Reading symbols from /usr/local/opendkim/sbin/opendkim...done.
        [New LWP 49990]
        [New LWP 44576]
        [New LWP 44577]
        [New LWP 48292]
        [New LWP 44578]
        [Thread debugging using libthread_db enabled]
        Using host libthread_db library "/lib64/libthread_db.so.1".
        Core was generated by `/usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf -'.
        Program terminated with signal SIGABRT, Aborted.
        #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
        51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
        [Current thread is 1 (Thread 0x7f6c460a3700 (LWP 49990))]
        (gdb) print p
        No symbol "p" in current context.
        (gdb) bt
            #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
            #1  0x00007f6c492336c1 in __GI_abort () at abort.c:79
            #2  0x00007f6c49275427 in __libc_message (action=action@entry=(do_abort | do_backtrace),
                fmt=fmt@entry=0x7f6c4937e230 "*** Error in `%s': %s: 0x%s ***\n")
                at ../sysdeps/posix/libc_fatal.c:181
            #3  0x00007f6c4927bc43 in malloc_printerr (action=<optimized out>,
                str=0x7f6c4937e5b8 "double free or corruption (out)", ptr=<optimized out>,
                ar_ptr=<optimized out>) at malloc.c:5428
            #4  0x00007f6c4927d549 in _int_free (av=0x7f6c495b0c20 <main_arena>, p=<optimized out>,
                have_lock=<optimized out>) at malloc.c:4170
            #5  0x00007f6c4936260b in tcache_thread_freeres () at malloc.c:2999
            #6  0x00007f6c49362662 in __libc_thread_freeres () at thread-freeres.c:29
            #7  0x00007f6c495bd57e in start_thread (arg=0x7f6c460a3700) at pthread_create.c:478
            #8  0x00007f6c492f482f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

ghost avatar Oct 15 '18 00:10 ghost

The BT shows that something in the past was suboptimal. Showing the BTs of all threads might or might not show sufficient information. If the crash appears with particular message, share the message. Make sure also to apply all fixes referenced in GH to make sure that thks problem is not already fixed. Use the develop branch.

dilyanpalauzov avatar Oct 31 '18 19:10 dilyanpalauzov

What "fixes referenced in GH" are you specifically referring to?

This build is from latest 'develop'

git branch -a | head -n 1
	* develop
git log | head
	commit b180e905e1597f72f77cc5a691db815ecd690e35
	Author: Murray S. Kucherawy <[email protected]>
	Date:   Sun May 13 00:12:59 2018 -0700

	    Add "a" and "s" tags to Authentication-Results

	commit 531ae37e23045aa4984a8dfb6e74356775058f82
	Author: Murray S. Kucherawy <[email protected]>
	Date:   Sun Feb 25 02:15:49 2018 -0800

If there's particular debug output you're looking for, pls specify. Happy to provide.

ghost avatar Oct 31 '18 20:10 ghost

You don’t state whether a specific message causes the crash. You also have not provided backtrace of all threads. With fixes referenced in GH I mean e.g.

https://github.com/trusteddomainproject/OpenDKIM/issues/22 or

https://github.com/trusteddomainproject/OpenDKIM/issues/8 .

dilyanpalauzov avatar Oct 31 '18 21:10 dilyanpalauzov

You don’t state whether a specific message causes the crash

it's not specific to any particular message. as stated, it happens intermittently

You also have not provided backtrace of all threads

pls state specifically what you want, and how you want it generated

With fixes referenced in GH I mean e.g.

neither one of those have been tested/committed to any branch here, have they?

ghost avatar Oct 31 '18 22:10 ghost

The fixes are not commmited, that is why you have to apply them manually. If you use libunbound one more fix is necessary. In gdb entering 'thread apply all bt f' prints the detailed backtrace of all threads. Ideal will be if gdb has the debug symbols from opendkim.

dilyanpalauzov avatar Oct 31 '18 23:10 dilyanpalauzov

@dilyanpalauzov

thanks for the specific request

re-building opendkim git/develop, with debug symbols on (" --enable-debug --enable-allsymbols")

launched daemon

after ~20 minutes, segfault

ls -al /var/lib/systemd/coredump/*
	-rw------- 1 opendkim postfix-milter 26M Oct 31 18:46 /var/lib/systemd/coredump/core.5512
gdb /usr/local/opendkim/sbin/opendkim /var/lib/systemd/coredump/core.5512
	...
	GNU gdb (GDB; openSUSE Leap 15.0) 8.2
	Copyright (C) 2018 Free Software Foundation, Inc.
	License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
	This is free software: you are free to change and redistribute it.
	There is NO WARRANTY, to the extent permitted by law.
	Type "show copying" and "show warranty" for details.
	This GDB was configured as "x86_64-suse-linux".
	Type "show configuration" for configuration details.
	For bug reporting instructions, please see:
	<http://bugs.opensuse.org/>.
	Find the GDB manual and other documentation resources online at:
	    <http://www.gnu.org/software/gdb/documentation/>.

	For help, type "help".
	Type "apropos word" to search for commands related to "word"...
	Reading symbols from /usr/local/opendkim/sbin/opendkim...done.
	[New LWP 5963]
	[New LWP 5512]
	[New LWP 5515]
	[New LWP 5514]
	[Thread debugging using libthread_db enabled]
	Using host libthread_db library "/lib64/libthread_db.so.1".
	Core was generated by `/usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf -'.
	Program terminated with signal SIGABRT, Aborted.
	#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
	51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
	--Type <RET> for more, q to quit, c to continue without paging--c
	[Current thread is 1 (Thread 0x7f8a21c40700 (LWP 5963))]
	(gdb) thread apply all bt f

	Thread 4 (Thread 0x7f8a22c42700 (LWP 5514)):
	#0  do_sigwait (set=<optimized out>, set@entry=0x7f8a22c41e70, sig=sig@entry=0x7f8a22c41efc)
	    at ../sysdeps/unix/sysv/linux/sigwait.c:64
	        ret = <optimized out>
	        tmpset = {__val = {0 <repeats 16 times>}}
	        __mask = <optimized out>
	        __word = <optimized out>
	        __mask = <optimized out>
	        __word = <optimized out>
	#1  0x00007f8a23bc6e6d in __sigwait (set=0x7f8a22c41e70, sig=0x7f8a22c41efc)
	    at ../sysdeps/unix/sysv/linux/sigwait.c:96
	        oldtype = 0
	        result = <optimized out>
	        sig = 0x7f8a22c41efc
	        set = 0x7f8a22c41e70
	        oldtype = <optimized out>
	        result = <optimized out>
	        oldtype = <optimized out>
	--Type <RET> for more, q to quit, c to continue without paging--c
	        result = <optimized out>
	#2  0x00000000004083e4 in dkimf_reloader ()
	No symbol table info available.
	#3  0x00007f8a23bbc559 in start_thread (arg=0x7f8a22c42700) at pthread_create.c:465
	        pd = 0x7f8a22c42700
	        now = <optimized out>
	        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140231265494784, -2341276610847858710, 140731773459966, 140731773459967, 0, 140731773459968, 2348161955612303338, 2348163367245160426}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	        not_first_call = <optimized out>
	        pagesize_m1 = <optimized out>
	        sp = <optimized out>
	        freesize = <optimized out>
	        __PRETTY_FUNCTION__ = "start_thread"
	        frame = <optimized out>
	#4  0x00007f8a238f381f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
	No locals.

	Thread 3 (Thread 0x7f8a22441700 (LWP 5515)):
	#0  do_sigwait (set=<optimized out>, set@entry=0x7f8a22440e70, sig=sig@entry=0x7f8a22440e6c) at ../sysdeps/unix/sysv/linux/sigwait.c:64
	        ret = <optimized out>
	        tmpset = {__val = {0 <repeats 16 times>}}
	        __mask = <optimized out>
	        __word = <optimized out>
	        __mask = <optimized out>
	        __word = <optimized out>
	#1  0x00007f8a23bc6e6d in __sigwait (set=0x7f8a22440e70, sig=0x7f8a22440e6c) at ../sysdeps/unix/sysv/linux/sigwait.c:96
	        oldtype = 0
	        result = <optimized out>
	        sig = 0x7f8a22440e6c
	        set = 0x7f8a22440e70
	        oldtype = <optimized out>
	        result = <optimized out>
	        oldtype = <optimized out>
	        result = <optimized out>
	#2  0x00000000004281ac in mi_signal_thread ()
	No symbol table info available.
	#3  0x00007f8a23bbc559 in start_thread (arg=0x7f8a22441700) at pthread_create.c:465
	        pd = 0x7f8a22441700
	        now = <optimized out>
	        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140231257102080, -2341276610847858710, 140731773459790, 140731773459791, 20592432, 140731773459792, 2348163055660802026, 2348163367245160426}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	        not_first_call = <optimized out>
	        pagesize_m1 = <optimized out>
	        sp = <optimized out>
	        freesize = <optimized out>
	        __PRETTY_FUNCTION__ = "start_thread"
	        frame = <optimized out>
	#4  0x00007f8a238f381f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
	No locals.

	Thread 2 (Thread 0x7f8a26323740 (LWP 5512)):
	#0  0x00007f8a238eaef3 in __GI___select (nfds=5, readfds=0x7ffeab5d9cb0, writefds=0x0, exceptfds=0x7ffeab5d9d30, timeout=0x7ffeab5d9ca0) at ../sysdeps/unix/sysv/linux/select.c:41
	        resultvar = 18446744073709551102
	        sc_cancel_oldtype = 0
	        sc_ret = <optimized out>
	#1  0x000000000042683f in mi_listener ()
	No symbol table info available.
	#2  0x0000000000425c7e in smfi_main ()
	No symbol table info available.
	#3  0x000000000041aa1b in main ()
	No symbol table info available.

	Thread 1 (Thread 0x7f8a21c40700 (LWP 5963)):
	#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
	        set = {__val = {16899, 140231316553824, 0, 3, 4294967294, 0, 140231248706944, 140231321043571, 5, 0, 0, 140231277855752, 140231248706880, 140231321073642, 140231152159552, 0}}
	        pid = <optimized out>
	        tid = <optimized out>
	        ret = <optimized out>
	#1  0x00007f8a238326c1 in __GI_abort () at abort.c:79
	        save_stage = 1
	        act = {__sigaction_handler = {sa_handler = 0x64203b646578616c, sa_sigaction = 0x64203b646578616c}, sa_mask = {__val = {8032016439431360573, 3760898644827779385, 4051325634811748657, 3762248639015107960, 7064188739919623473, 8675461341612877096, 44965372020, 0, 0, 2, 1, 0, 0, 4, 140231248706464, 140231248707072}}, sa_flags = 566492064, sa_restorer = 0x1000}
	        sigs = {__val = {32, 0 <repeats 15 times>}}
	#2  0x00007f8a23874427 in __libc_message (action=action@entry=(do_abort | do_backtrace), fmt=fmt@entry=0x7f8a2397d210 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:181
	        ap = {{gp_offset = 40, fp_offset = 32650, overflow_arg_area = 0x7f8a21c3fe10, reg_save_area = 0x7f8a21c3fda0}}
	        fd = <optimized out>
	        list = <optimized out>
	        nlist = <optimized out>
	        cp = <optimized out>
	        written = <optimized out>
	#3  0x00007f8a2387ac43 in malloc_printerr (action=<optimized out>, str=0x7f8a2397d598 "double free or corruption (out)", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5428
	        buf = "00007f8a1c02cc40"
	        cp = <optimized out>
	        ar_ptr = <optimized out>
	        ptr = <optimized out>
	        str = 0x7f8a2397d598 "double free or corruption (out)"
	        action = <optimized out>
	        buf = <optimized out>
	        cp = <optimized out>
	#4  0x00007f8a2387c549 in _int_free (av=0x7f8a23bafc20 <main_arena>, p=<optimized out>, have_lock=<optimized out>) at malloc.c:4170
	        size = <optimized out>
	        fb = <optimized out>
	        nextchunk = <optimized out>
	        nextsize = <optimized out>
	        nextinuse = <optimized out>
	        prevsize = <optimized out>
	        bck = <optimized out>
	        fwd = <optimized out>
	        errstr = <optimized out>
	        locked = <optimized out>
	        __func__ = "_int_free"
	#5  0x00007f8a239615fb in tcache_thread_freeres () at malloc.c:2999
	        e = <optimized out>
	        i = <optimized out>
	        tcache_tmp = 0x7f8a1c0008c0
	#6  0x00007f8a23961652 in __libc_thread_freeres () at thread-freeres.c:29
	        ptr = 0x7f8a23bab8b8 <__elf_set___libc_thread_subfreeres_element_tcache_thread_freeres__>
	#7  0x00007f8a23bbc57e in start_thread (arg=0x7f8a21c40700) at pthread_create.c:478
	        pd = 0x7f8a21c40700
	        now = <optimized out>
	        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140231248709376, -2341276610847858710, 140731773459438, 140731773459439, 20734944, 140731773459440, 2348168521543557098, 2348163367245160426}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	        not_first_call = <optimized out>
	        pagesize_m1 = <optimized out>
	        sp = <optimized out>
	        freesize = <optimized out>
	        __PRETTY_FUNCTION__ = "start_thread"
	        frame = <optimized out>
	#8  0x00007f8a238f381f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
	No locals.

ghost avatar Nov 01 '18 01:11 ghost

+patch

checkout

	opendkim git/develop

apply patch

	cat ../opendkim-test.patch
		--- opendkim.ORIG/opendkim/opendkim.c	2018-10-31 19:33:55.657887787 -0700
		+++ opendkim/opendkim/opendkim.c	2018-10-31 19:36:02.346814583 -0700
		@@ -4732,10 +4732,9 @@
			if (myuid == (uid_t) -1)
				myuid = pw->pw_uid;

		-	pthread_mutex_unlock(&pwdb_lock);
		-
		 #ifdef HAVE_REALPATH
			strlcpy(myname, pw->pw_name, sizeof myname);
		+	pthread_mutex_unlock(&pwdb_lock);

			p = realpath(path, real);
			if (p == NULL)
		@@ -4774,6 +4773,7 @@

			return 1;
		 #else /* HAVE_REALPATH */
		+	pthread_mutex_unlock(&pwdb_lock);
			struct stat s;

			status = stat(path, &s);
		@@ -5905,7 +5905,6 @@
		 dkimf_config_free(struct dkimf_config *conf)
		 {
			assert(conf != NULL);
		-	assert(conf->conf_refcnt == 0);

			dkimf_zapkey(conf);

	patch -p1 < ../opendkim-test.patch
		patching file opendkim/opendkim.c

launch daemon

after ~25 minutes, segfault

ls -al /var/lib/systemd/coredump/*
	-rw------- 1 opendkim postfix-milter 26M Oct 31 19:44 /var/lib/systemd/coredump/core.3557
gdb /usr/local/opendkim/sbin/opendkim /var/lib/systemd/coredump/core.3557
	...
	[Thread debugging using libthread_db enabled]
	Using host libthread_db library "/lib64/libthread_db.so.1".
	Core was generated by `/usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf -'.
	Program terminated with signal SIGABRT, Aborted.
	#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
	51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
	--Type <RET> for more, q to quit, c to continue without paging--c
	[Current thread is 1 (Thread 0x7fe2deafe700 (LWP 3685))]
	(gdb)  thread apply all bt f

	Thread 4 (Thread 0x7fe2dfb00700 (LWP 3558)):
	#0  do_sigwait (set=<optimized out>, set@entry=0x7fe2dfaffe70, sig=sig@entry=0x7fe2dfaffefc)
	    at ../sysdeps/unix/sysv/linux/sigwait.c:64
	        ret = <optimized out>
	        tmpset = {__val = {0 <repeats 16 times>}}
	        __mask = <optimized out>
	        __word = <optimized out>
	        __mask = <optimized out>
	        __word = <optimized out>
	#1  0x00007fe2e0a84e6d in __sigwait (set=0x7fe2dfaffe70, sig=0x7fe2dfaffefc)
	    at ../sysdeps/unix/sysv/linux/sigwait.c:96
	        oldtype = 0
	        result = <optimized out>
	        sig = 0x7fe2dfaffefc
	        set = 0x7fe2dfaffe70
	        oldtype = <optimized out>
	        result = <optimized out>
	        oldtype = <optimized out>
	--Type <RET> for more, q to quit, c to continue without paging--c
	        result = <optimized out>
	#2  0x00000000004083e4 in dkimf_reloader ()
	No symbol table info available.
	#3  0x00007fe2e0a7a559 in start_thread (arg=0x7fe2dfb00700) at pthread_create.c:465
	        pd = 0x7fe2dfb00700
	        now = <optimized out>
	        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140612392191744, 481145786195812320, 140723795460766, 140723795460767, 0, 140723795460768, -474882555636572192, -474955194474389536}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	        not_first_call = <optimized out>
	        pagesize_m1 = <optimized out>
	        sp = <optimized out>
	        freesize = <optimized out>
	        __PRETTY_FUNCTION__ = "start_thread"
	        frame = <optimized out>
	#4  0x00007fe2e07b181f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
	No locals.

	Thread 3 (Thread 0x7fe2df2ff700 (LWP 3559)):
	#0  do_sigwait (set=<optimized out>, set@entry=0x7fe2df2fee70, sig=sig@entry=0x7fe2df2fee6c) at ../sysdeps/unix/sysv/linux/sigwait.c:64
	        ret = <optimized out>
	        tmpset = {__val = {0 <repeats 16 times>}}
	        __mask = <optimized out>
	        __word = <optimized out>
	        __mask = <optimized out>
	        __word = <optimized out>
	#1  0x00007fe2e0a84e6d in __sigwait (set=0x7fe2df2fee70, sig=0x7fe2df2fee6c) at ../sysdeps/unix/sysv/linux/sigwait.c:96
	        oldtype = 0
	        result = <optimized out>
	        sig = 0x7fe2df2fee6c
	        set = 0x7fe2df2fee70
	        oldtype = <optimized out>
	        result = <optimized out>
	        oldtype = <optimized out>
	        result = <optimized out>
	#2  0x000000000042817c in mi_signal_thread ()
	No symbol table info available.
	#3  0x00007fe2e0a7a559 in start_thread (arg=0x7fe2df2ff700) at pthread_create.c:465
	        pd = 0x7fe2df2ff700
	        now = <optimized out>
	        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140612383799040, 481145786195812320, 140723795460590, 140723795460591, 34314032, 140723795460592, -474881456661815328, -474955194474389536}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	        not_first_call = <optimized out>
	        pagesize_m1 = <optimized out>
	        sp = <optimized out>
	        freesize = <optimized out>
	        __PRETTY_FUNCTION__ = "start_thread"
	        frame = <optimized out>
	#4  0x00007fe2e07b181f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
	No locals.

	Thread 2 (Thread 0x7fe2e31e1740 (LWP 3557)):
	#0  0x00007fe2e07a8ef3 in __GI___select (nfds=5, readfds=0x7ffccfd70150, writefds=0x0, exceptfds=0x7ffccfd701d0, timeout=0x7ffccfd70140) at ../sysdeps/unix/sysv/linux/select.c:41
	        resultvar = 18446744073709551102
	        sc_cancel_oldtype = 0
	        sc_ret = <optimized out>
	#1  0x000000000042680f in mi_listener ()
	No symbol table info available.
	#2  0x0000000000425c4e in smfi_main ()
	No symbol table info available.
	#3  0x000000000041a9f1 in main ()
	No symbol table info available.

	Thread 1 (Thread 0x7fe2deafe700 (LWP 3685)):
	#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
	        set = {__val = {16899, 140612443250784, 0, 3, 4294967294, 0, 140612375403904, 140612447740531, 5, 0, 0, 140612404552712, 140612375403840, 140612447770602, 140612263398208, 0}}
	        pid = <optimized out>
	        tid = <optimized out>
	        ret = <optimized out>
	#1  0x00007fe2e06f06c1 in __GI_abort () at abort.c:79
	        save_stage = 1
	        act = {__sigaction_handler = {sa_handler = 0x64203b646578616c, sa_sigaction = 0x64203b646578616c}, sa_mask = {__val = {8032016439431360573, 3760898644827779385, 4051325634811748657, 3762248639015107960, 7064188739919425589, 8675461341612877096, 44965372020, 0, 0, 2, 2, 1, 0, 4, 140612375403424, 140612375404032}}, sa_flags = -558900320, sa_restorer = 0x1000}
	        sigs = {__val = {32, 0 <repeats 15 times>}}
	#2  0x00007fe2e0732427 in __libc_message (action=action@entry=(do_abort | do_backtrace), fmt=fmt@entry=0x7fe2e083b210 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:181
	        ap = {{gp_offset = 40, fp_offset = 32738, overflow_arg_area = 0x7fe2deafde10, reg_save_area = 0x7fe2deafdda0}}
	        fd = <optimized out>
	        list = <optimized out>
	        nlist = <optimized out>
	        cp = <optimized out>
	        written = <optimized out>
	#3  0x00007fe2e0738c43 in malloc_printerr (action=<optimized out>, str=0x7fe2e083b598 "double free or corruption (out)", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5428
	        buf = "00007fe2d802cc40"
	        cp = <optimized out>
	        ar_ptr = <optimized out>
	        ptr = <optimized out>
	        str = 0x7fe2e083b598 "double free or corruption (out)"
	        action = <optimized out>
	        buf = <optimized out>
	        cp = <optimized out>
	#4  0x00007fe2e073a549 in _int_free (av=0x7fe2e0a6dc20 <main_arena>, p=<optimized out>, have_lock=<optimized out>) at malloc.c:4170
	        size = <optimized out>
	        fb = <optimized out>
	        nextchunk = <optimized out>
	        nextsize = <optimized out>
	        nextinuse = <optimized out>
	        prevsize = <optimized out>
	        bck = <optimized out>
	        fwd = <optimized out>
	        errstr = <optimized out>
	        locked = <optimized out>
	        __func__ = "_int_free"
	#5  0x00007fe2e081f5fb in tcache_thread_freeres () at malloc.c:2999
	        e = <optimized out>
	        i = <optimized out>
	        tcache_tmp = 0x7fe2d80008c0
	#6  0x00007fe2e081f652 in __libc_thread_freeres () at thread-freeres.c:29
	        ptr = 0x7fe2e0a698b8 <__elf_set___libc_thread_subfreeres_element_tcache_thread_freeres__>
	#7  0x00007fe2e0a7a57e in start_thread (arg=0x7fe2deafe700) at pthread_create.c:478
	        pd = 0x7fe2deafe700
	        now = <optimized out>
	        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140612375406336, 481145786195812320, 140723795460238, 140723795460239, 34456544, 140723795460240, -474880357687058464, -474955194474389536}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
	        not_first_call = <optimized out>
	        pagesize_m1 = <optimized out>
	        sp = <optimized out>
	        freesize = <optimized out>
	        __PRETTY_FUNCTION__ = "start_thread"
	        frame = <optimized out>
	#8  0x00007fe2e07b181f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
	No locals

ghost avatar Nov 01 '18 02:11 ghost

The backtraces don’t show the debug symbols in opendkim. They have anyway a limited usage, if the memory was misused a while ago.

Consider running opendkim in foreground mode under valgrind. Messages like ”use after free“ or ”invalid free()“ emitted by valgrind indicate the right place, where things go wrong.

dilyanpalauzov avatar Nov 01 '18 10:11 dilyanpalauzov

@dilyanpalauzov

I've finally stumbled on a reproducible way to trigger a crash. Only on outbound signing, not (yet) on inbound verification (which does still intermittently happen; trying to find a reproducible trigger).

To trigger that^ crash, sending this from the same machine:

	sendmail -i -f [email protected] -t <<EOF
	From: [email protected]
	To:  [email protected].
	Subject: test
	
	test
	EOF

immediately causes the crash.

But ... only in one scenario :-/

If opendkim is launched as its usual systemd service,

	...
	[Service]
	User=opendkim
	Group=postfix-milter
	Type=forking
	ExecStart=/usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf
	...

on 'send', it crashes immediately.

However, if I launch instead as

	/usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf

directly from shell, opendkim works -- and does NOT crash on 'send'.

With valgrind as well, either as systemd service

	...
	[Service]
	User=opendkim
	Group=postfix-milter
	Type=forking
	ExecStart=/usr/bin/valgrind --tool=memcheck --leak-check=yes --show-leak-kinds=all --log-file=/var/log/valgrind/opendkim.log /usr/local/opendkim/sbin/opendkim -l  -x /usr/local/etc/opendkim/opendkim.conf
	...

or directly from shell as,

	/usr/bin/valgrind --tool=memcheck --leak-check=yes --show-leak-kinds=all --log-file=/var/log/valgrind/opendkim.log /usr/local/opendkim/sbin/opendkim -l  -x /usr/local/etc/opendkim/opendkim.conf

again, opendkim works, does NOT crash on 'send', and no trace in valgrind log of either ”use after free“ or ”invalid free()“

I can't seem to manage to get the gdb or valgrind info needed out of this ...

ghost avatar Nov 01 '18 16:11 ghost

It makes no sense to crash only when started from systemd - the software does not care and know, whether it is started from systemd.

The sample message has no Date: header. What value have the RequiredHeaders and IgnoreMalformedMail configuration options?

Change the type to simple in the systemd unit file, add -f to opendkim and run under valgrind with --log-file= and systemd. At the place it misbehaves, valgrind/memcheck should print what the problem is, instead of crashing. This might not be exactly what I typed, valgrind/memcheck can find also other problems.

Why do you put a fullstop at the end of the To: line?

dilyanpalauzov avatar Nov 01 '18 17:11 dilyanpalauzov

Why do you put a fullstop at the end of the To: line?

It's just a typo, here only

for sent message

sendmail -i -f [email protected] -t <<EOF
From: [email protected]
To:  [email protected]
Subject: test

test
EOF

it's received just fine as

Return-Path: <[email protected]>
...
From: [email protected]
To:  [email protected]
Subject: test
Message-Id: <[email protected]>
Date: Thu,  1 Nov 2018 10:39:05 -0700 (PDT)

test

And, it's not the only thing that causes these crashes.

This is just a reproducible way of generating a crash.

Change the type to simple in the systemd unit file, add -f to opendkim and run under valgrind with --log-file= and systemd. At the place it misbehaves, valgrind/memcheck should print what the problem is, instead of crashing. This might not be exactly what I typed, valgrind/memcheck can find also other problems.

here's the valgrind log output:

for systemd start
	==60312== Memcheck, a memory error detector
	==60312== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
	==60312== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
	==60312== Command: /usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf -f
	==60312== Parent PID: 1
	==60312== 
	==60312== Conditional jump or move depends on uninitialised value(s)
	==60312==    at 0x4E57AA2: dkim_dns_config (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x41F6EA: dkimf_dns_config (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x40C3CA: dkimf_dns_init (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x40C580: dkimf_config_setlib (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x41A151: main (in /usr/local/opendkim/sbin/opendkim)
	==60312==  Uninitialised value was created by a heap allocation
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x4E4FFF6: dkim_init (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40C451: dkimf_config_setlib (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x41A151: main (in /usr/local/opendkim/sbin/opendkim)
	==60312==
on subsequent send
	==60312== Thread 4:
	==60312== Conditional jump or move depends on uninitialised value(s)
	==60312==    at 0x4E57AA2: dkim_dns_config (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x41F6EA: dkimf_dns_config (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x40C3CA: dkimf_dns_init (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x41106A: mlfi_connect (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x429816: st_connectinfo (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Uninitialised value was created by a heap allocation
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x4E4FFF6: dkim_init (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40C451: dkimf_config_setlib (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x41A151: main (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 4
	==60312==    at 0x6B0E9AC: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5bf0 is 80 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 8
	==60312==    at 0x6B0E9C1: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5ba8 is 8 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 8
	==60312==    at 0x6B0E9C8: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5bb0 is 16 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 8
	==60312==    at 0x6B0E9F4: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5ba0 is 0 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 4
	==60312==    at 0x6B14679: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0EA09: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5bc4 is 36 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 4
	==60312==    at 0x6B14685: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0EA09: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5bc0 is 32 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 8
	==60312==    at 0x6B14692: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0EA09: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5bd8 is 56 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 8
	==60312==    at 0x6BF2425: CRYPTO_free_ex_data (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0EA1A: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5c08 is 104 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid write of size 8
	==60312==    at 0x6BF242E: CRYPTO_free_ex_data (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0EA1A: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5c08 is 104 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid read of size 8
	==60312==    at 0x6B0EA1B: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5c10 is 112 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid free() / delete / delete[] / realloc()
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA23: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5c60 is 0 bytes inside a block of size 56 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA23: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6C64036: CRYPTO_THREAD_lock_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E899: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312== 
	==60312== Invalid free() / delete / delete[] / realloc()
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x40D79A: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416F9A: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A574: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Address 0x82c5ba0 is 0 bytes inside a block of size 120 free'd
	==60312==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6B0EA37: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x425CA8: mi_thread_handle_wrapper (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x719B558: start_thread (pthread_create.c:465)
	==60312==    by 0x74AA81E: clone (clone.S:95)
	==60312==  Block was alloc'd at
	==60312==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==60312==    by 0x6BF7D18: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B0E85C: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x6B14EB9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==60312==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==60312==    by 0x407AA7: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x416835: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x4291A1: st_bodyend (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x42A666: mi_engine (in /usr/local/opendkim/sbin/opendkim)
	==60312==    by 0x426F97: mi_handle_session (in /usr/local/opendkim/sbin/opendkim)
	==60312==

ghost avatar Nov 01 '18 17:11 ghost

Compile opendkim with debug information, so that gdb and valgrind show the line numbers.

dilyanpalauzov avatar Nov 01 '18 18:11 dilyanpalauzov

If valgrind --logfile= produces considerably different output, when started outside of systemd, and opendkim is again called with -f, then you should find what makes the start under systemd different.

dilyanpalauzov avatar Nov 01 '18 18:11 dilyanpalauzov

Compile opendkim with debug information, so that gdb and valgrind show the line numbers

it has been compiled with:

./configure  --enable-debug --enable-allsymbols ...

Is there something else required?

ghost avatar Nov 01 '18 18:11 ghost

Many factors have influence, e.g. a config.site file containing LDFLAGS="-Wl,-s" or an old valgrind with a new ld.bfd. It is up to you to find out how to get the debug information.

dilyanpalauzov avatar Nov 01 '18 21:11 dilyanpalauzov

Core was generated by `/usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf -'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
[Current thread is 1 (Thread 0x7f74e52dd700 (LWP 43015))]
(gdb) thread apply all bt f

Thread 6 (Thread 0x7f74ea99a740 (LWP 43010)):
#0  0x00007f74e7d7f07b in __GI___poll (fds=0x7ffd04f5edf0, nfds=1, timeout=5000)
    at ../sysdeps/unix/sysv/linux/poll.c:29
        resultvar = 18446744073709551100
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f74ea1a9ef1 in mi_listener () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#2  0x00007f74ea1a77a7 in smfi_main () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#3  0x000000000041a8fb in main ()
No symbol table info available.

Thread 5 (Thread 0x7f74e5ade700 (LWP 43014)):
#0  0x00007f74e8058c56 in futex_abstimed_wait_cancelable (private=<optimized out>, 
    abstime=0x7f74e5addec0, expected=0, futex_word=0x7f74ea3b1484)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
        __ret = -516
        oldtype = 0
        err = <optimized out>
        oldtype = <optimized out>
        err = <optimized out>
        __ret = <optimized out>
        resultvar = <optimized out>
        __arg6 = <optimized out>
        __arg5 = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a6 = <optimized out>
        _a5 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x7f74e5addec0, mutex=0x7f74ea3b1430, 
    cond=0x7f74ea3b1458) at pthread_cond_wait.c:539
        spin = 0
        buffer = {__routine = 0x7f74e8058610 <__condvar_cleanup_waiting>, 
--Type <RET> for more, q to quit, c to continue without paging--c
          __arg = 0x7f74e5adde50, __canceltype = 6, __prev = 0x0}
        cbuffer = {wseq = 43, cond = 0x7f74ea3b1458, mutex = 0x7f74ea3b1430, private = 0}
        err = <optimized out>
        g = 1
        flags = 8
        g1_start = <optimized out>
        maxspin = 0
        signals = <optimized out>
        result = 0
        wseq = <optimized out>
        seq = 21
        private = <optimized out>
        maxspin = <optimized out>
        err = <optimized out>
        result = <optimized out>
        wseq = <optimized out>
        g = <optimized out>
        seq = <optimized out>
        flags = <optimized out>
        private = <optimized out>
        signals = <optimized out>
        g1_start = <optimized out>
        spin = <optimized out>
        buffer = <optimized out>
        cbuffer = <optimized out>
        rt = <optimized out>
        s = <optimized out>
#2  __pthread_cond_timedwait (cond=0x7f74ea3b1458, mutex=0x7f74ea3b1430, abstime=0x7f74e5addec0) at pthread_cond_wait.c:667
No locals.
#3  0x00007f74ea1aaff9 in ?? () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#4  0x00007f74e8052559 in start_thread (arg=0x7f74e5ade700) at pthread_create.c:465
        pd = 0x7f74e5ade700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140140046313216, 2398737420674280814, 140724686679310, 140724686679311, 0, 140724686679312, -2331794085798859410, -2331783839275668114}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#5  0x00007f74e7d8981f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 4 (Thread 0x7f74e62df700 (LWP 43013)):
#0  0x00007f74e7d7f07b in __GI___poll (fds=0x7f74e0000b10, nfds=1, timeout=10000) at ../sysdeps/unix/sysv/linux/poll.c:29
        resultvar = 18446744073709551100
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f74ea1aa8df in ?? () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#2  0x00007f74e8052559 in start_thread (arg=0x7f74e62df700) at pthread_create.c:465
        pd = 0x7f74e62df700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140140054705920, 2398737420674280814, 140724686679310, 140724686679311, 0, 140724686679312, -2331797383796871826, -2331783839275668114}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#3  0x00007f74e7d8981f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 3 (Thread 0x7f74e6ae0700 (LWP 43012)):
#0  do_sigwait (set=<optimized out>, set@entry=0x7f74e6adfe60, sig=sig@entry=0x7f74e6adfe5c) at ../sysdeps/unix/sysv/linux/sigwait.c:64
        ret = <optimized out>
        tmpset = {__val = {0 <repeats 16 times>}}
        __mask = <optimized out>
        __word = <optimized out>
        __mask = <optimized out>
        __word = <optimized out>
#1  0x00007f74e805ce6d in __sigwait (set=0x7f74e6adfe60, sig=0x7f74e6adfe5c) at ../sysdeps/unix/sysv/linux/sigwait.c:96
        oldtype = 0
        result = <optimized out>
        sig = 0x7f74e6adfe5c
        set = 0x7f74e6adfe60
        oldtype = <optimized out>
        result = <optimized out>
        oldtype = <optimized out>
        result = <optimized out>
#2  0x00007f74ea1ac9ef in ?? () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#3  0x00007f74e8052559 in start_thread (arg=0x7f74e6ae0700) at pthread_create.c:465
        pd = 0x7f74e6ae0700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140140063098624, 2398737420674280814, 140724686679438, 140724686679439, 28821440, 140724686679440, -2331796283748373138, -2331783839275668114}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#4  0x00007f74e7d8981f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 2 (Thread 0x7f74e72e1700 (LWP 43011)):
#0  do_sigwait (set=<optimized out>, set@entry=0x7f74e72e0e70, sig=sig@entry=0x7f74e72e0efc) at ../sysdeps/unix/sysv/linux/sigwait.c:64
        ret = <optimized out>
        tmpset = {__val = {0 <repeats 16 times>}}
        __mask = <optimized out>
        __word = <optimized out>
        __mask = <optimized out>
        __word = <optimized out>
#1  0x00007f74e805ce6d in __sigwait (set=0x7f74e72e0e70, sig=0x7f74e72e0efc) at ../sysdeps/unix/sysv/linux/sigwait.c:96
        oldtype = 0
        result = <optimized out>
        sig = 0x7f74e72e0efc
        set = 0x7f74e72e0e70
        oldtype = <optimized out>
        result = <optimized out>
        oldtype = <optimized out>
        result = <optimized out>
#2  0x00000000004082c4 in dkimf_reloader ()
No symbol table info available.
#3  0x00007f74e8052559 in start_thread (arg=0x7f74e72e1700) at pthread_create.c:465
        pd = 0x7f74e72e1700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140140071491328, 2398737420674280814, 140724686679630, 140724686679631, 0, 140724686679632, -2331799568861483666, -2331783839275668114}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#4  0x00007f74e7d8981f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 1 (Thread 0x7f74e52dd700 (LWP 43015)):
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
        set = {__val = {16899, 1, 140140085629584, 140140085536544, 140136192933888, 0, 140140129353728, 206158430224, 140140037918160, 140140037917952, 140140129353728, 140140037916968, 140140037916964, 140140124866808, 140136192933888, 140140124866808}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1  0x00007f74e7cc86c1 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0xffffffff, sa_sigaction = 0xffffffff}, sa_mask = {__val = {91, 140140085558056, 1, 140140128904144, 140140127152532, 91, 140140085567296, 140140128904144, 140140037917368, 140140037917364, 140140127151822, 0, 140140085567296, 140140085559064, 140140037917344, 140140037917952}}, sa_flags = -449983840, sa_restorer = 0x1000}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007f74e7d0a427 in __libc_message (action=action@entry=(do_abort | do_backtrace), fmt=fmt@entry=0x7f74e7e13210 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:181
        ap = {{gp_offset = 40, fp_offset = 0, overflow_arg_area = 0x7f74e52dcd10, reg_save_area = 0x7f74e52dcca0}}
        fd = <optimized out>
        list = <optimized out>
        nlist = <optimized out>
        cp = <optimized out>
        written = <optimized out>
#3  0x00007f74e7d10c43 in malloc_printerr (action=<optimized out>, str=0x7f74e7e0fbda "free(): invalid pointer", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5428
        buf = "00007f74d8041690"
        cp = <optimized out>
        ar_ptr = <optimized out>
        ptr = <optimized out>
        str = 0x7f74e7e0fbda "free(): invalid pointer"
        action = <optimized out>
        buf = <optimized out>
        cp = <optimized out>
#4  0x00007f74e7d12549 in _int_free (av=0x7f74e8045c20 <main_arena>, p=<optimized out>, have_lock=<optimized out>) at malloc.c:4170
        size = <optimized out>
        fb = <optimized out>
        nextchunk = <optimized out>
        nextsize = <optimized out>
        nextinuse = <optimized out>
        prevsize = <optimized out>
        bck = <optimized out>
        fwd = <optimized out>
        errstr = <optimized out>
        locked = <optimized out>
        __func__ = "_int_free"
#5  0x00007f74ea1a86c4 in mi_engine () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#6  0x00007f74ea1aadb6 in ?? () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#7  0x00007f74e8052559 in start_thread (arg=0x7f74e52dd700) at pthread_create.c:465
        pd = 0x7f74e52dd700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140140037920512, 2398737420674280814, 140724686679310, 140724686679311, 0, 140724686679312, -2331795185847358098, -2331783839275668114}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#8  0x00007f74e7d8981f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.
(gdb)

ghost avatar Nov 02 '18 22:11 ghost

Thread 2 frame 2 shows: 0x00000000004082c4 in dkimf_reloader () No symbol table info available. Once you get the debug information in opendkim, run the whole again under systemd and valgrind in foreground mode, so that in the following output from valgrind contains line numbers:

==60312== Conditional jump or move depends on uninitialised value(s)
==60312==    at 0x4E57AA2: dkim_dns_config (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)

Do you literally use the example.com and example1.com domains in the tests?

Do you use libunbound for DNS as I do?

Opendkim can crash every 20 minutes for four days, if a particular incoming message causes problems and the sending end retries sending the message every 20 minutes. You can try to isolate that problematic message somehow. Perhaps the reason is that some involved domain, like example.com or example1.com does not exist. I will try this particular scenario in the next days, when I have time.

dilyanpalauzov avatar Nov 03 '18 09:11 dilyanpalauzov

Once you get the debug information in opendkim

no success so far. no such issues with any other app I have. it's something unique to this app build.

Do you literally use the example.com and example1.com domains in the tests?

no. real domains. same problems occur whether on a production postfix server, or a local-delivery-only, test instance

Do you use libunbound for DNS as I do?

no. config'd with "--without-unbound" . local DNS provided by a local bind9 instance.

Opendkim can crash every 20 minutes for four days

that it does crash in 'normal usage' is a problem. no luck isolating a specific message, inbound. only reproducibility so far is with these 'outbound, signing tests.

ghost avatar Nov 03 '18 15:11 ghost

Unitl if & when the opendkim devs (https://github.com/trusteddomainproject/OpenDKIM/graphs/contributors) start to participate in this project again, and at least provide clear, specific instructions how to get all symbols generated for end-users to actually provide the information they need -- including for, apparently, that dkimf_reloader() you've mentioned -- it's not happening here. Done spinning my wheels; it's a waste of my, and -- importantly -- your, time.

So, @dilyanpalauzov , thanks for your efforts.

And, one last attempt:

With

which ld
	/usr/bin/ld
ls -al /usr/bin/ld /etc/alternatives/ld
	lrwxrwxrwx 1 root root 15 Jun  7 10:16 /etc/alternatives/ld -> /usr/bin/ld.bfd*
	lrwxrwxrwx 1 root root 20 Oct 31 07:15 /usr/bin/ld -> /etc/alternatives/ld*
ld -v
	GNU ld () 2.31.1.20180828-lp150.329
gcc --version
	gcc (SUSE Linux) 8.2.1 20181025 [gcc-8-branch revision 265488]
valgrind --version
	valgrind-3.13.0

and

cat /etc/systemd/system/opendkim.service
	...
	Type=simple
	ExecStart=/usr/bin/valgrind --tool=memcheck --leak-check=yes --show-leak-kinds=all --track-origins=yes --log-file=/var/log/valgrind/opendkim.log /usr/local/opendkim/sbin/opendkim -l  -x /usr/local/etc/opendkim/opendkim.conf -u opendkim -f
	...

On

systemctl restart opendkim-custom

logs

journalctl -f
	Nov 03 10:00:49 dev01.example.net opendkim[46580]: OpenDKIM Filter: Opening listen socket on conn local:/var/run/opendkim/opendkim.sock
	Nov 03 10:00:49 dev01.example.net opendkim[46580]: OpenDKIM Filter v2.11.0 starting (args: -l -x /usr/local/etc/opendkim/opendkim.conf -u opendkim -f)

tail -f /var/log/valgrind/opendkim.log
	==46580== Memcheck, a memory error detector
	==46580== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
	==46580== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
	==46580== Command: /usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf -u opendkim -f
	==46580== Parent PID: 1
	==46580==
	==46580== Conditional jump or move depends on uninitialised value(s)
	==46580==    at 0x4E57AA2: dkim_dns_config (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x41F5F4: dkimf_dns_config (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x40C2D4: dkimf_dns_init (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x40C48A: dkimf_config_setlib (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41A05B: main (in /usr/local/opendkim/sbin/opendkim)
	==46580==  Uninitialised value was created by a heap allocation
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x4E4FFF6: dkim_init (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40C35B: dkimf_config_setlib (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41A05B: main (in /usr/local/opendkim/sbin/opendkim)
	==46580==

On 'test send', via Postfix's sendmail

sendmail -i -f root -t <<EOF
From: [email protected]
To: root@localhost
Subject: test

testing
EOF

logs

journalctl -f | egrep -i "valgrind|dkim"
	Nov 03 10:04:16 dev01.example.net opendkim[46580]: 5EFC338: DKIM-Signature field added (s=dkim-abc5dc3bb4e1debf, d=example.net)
tail -f /var/log/valgrind/opendkim.log
	==46580== Thread 5:
	==46580== Conditional jump or move depends on uninitialised value(s)
	==46580==    at 0x4E57AA2: dkim_dns_config (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x41F5F4: dkimf_dns_config (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x40C2D4: dkimf_dns_init (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x410F74: mlfi_connect (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x5279066: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Uninitialised value was created by a heap allocation
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x4E4FFF6: dkim_init (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40C35B: dkimf_config_setlib (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41A05B: main (in /usr/local/opendkim/sbin/opendkim)
	==46580==
	==46580== Thread 6:
	==46580== Invalid read of size 4
	==46580==    at 0x6D2842E: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x85049d0 is 80 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid read of size 8
	==46580==    at 0x6D28449: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x8504988 is 8 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid read of size 8
	==46580==    at 0x6D28450: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x8504990 is 16 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid read of size 8
	==46580==    at 0x6D2847C: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x8504980 is 0 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid read of size 4
	==46580==    at 0x6D2E6E9: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D28491: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x85049a4 is 36 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid read of size 4
	==46580==    at 0x6D2E6F5: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D28491: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x85049a0 is 32 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid read of size 8
	==46580==    at 0x6D2E702: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D28491: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x85049b8 is 56 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid read of size 8
	==46580==    at 0x6E123FE: CRYPTO_free_ex_data (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D284A2: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x85049e8 is 104 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid write of size 8
	==46580==    at 0x6E12407: CRYPTO_free_ex_data (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D284A2: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x85049e8 is 104 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid read of size 8
	==46580==    at 0x6D284A3: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x85049f0 is 112 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid free() / delete / delete[] / realloc()
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284AB: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x8504a40 is 0 bytes inside a block of size 56 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284AB: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6E88D96: CRYPTO_THREAD_lock_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D28319: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==
	==46580== Invalid free() / delete / delete[] / realloc()
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E515B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x527854F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Address 0x8504980 is 0 bytes inside a block of size 120 free'd
	==46580==    at 0x4C2F24B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6D284BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4F6AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x73C5558: start_thread (pthread_create.c:465)
	==46580==    by 0x76D481E: clone (clone.S:95)
	==46580==  Block was alloc'd at
	==46580==    at 0x4C2E01F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
	==46580==    by 0x6E18428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D282DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x6D2EFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
	==46580==    by 0x4E4A0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E4EEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x4E54581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
	==46580==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
	==46580==    by 0x52780B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x52784AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
	==46580==    by 0x527ADB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
	==46580==

fwiw, the test message IS delivered, signed

mail
? 3
      1 Message  3:
      2 From root@localhost  Sat Nov  3 10:04:16 2018
      3 X-Original-To: root@localhost
      4 Delivered-To: root@localhost
      5 DKIM-Filter: OpenDKIM Filter v2.11.0 localhost 5EFC338
      6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.net;
      7         s=dkim-abc5dc3bb4e1debf; t=1541264656; x=1541869456;
      8         bh=uhPj+w5pFrAQn4C3cOGi+psh6iIBCeHKJgB6bvYS7Ss=;
      9         h=From:To:Subject:Message-Id:Date:From:Subject:To;
     10         z=From:[email protected]|To:=20root@localhost|Subject:=20t
     11          est|Message-Id:=20<20181103170415.5EFC338@localhost>|Date:=20Sat,=
     12          20=203=20Nov=202018=2010:04:15=20-0700=20(PDT);
     13         b=laF...xqQ==
     19 From: [email protected]
     20 To: root@localhost
     21 Subject: test
     22 Date: Sat,  3 Nov 2018 10:04:15 -0700 (PDT)
     23
     24 testing
     25

switching to

cat /etc/systemd/system/opendkim.service
	...
	Type=forking
	ExecStart=/usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf -u opendkim
	...

On

systemctl restart opendkim-custom

logs

journalctl -f
	Nov 03 10:18:46 dev01.example.net opendkim[47562]: OpenDKIM Filter: mi_stop=1
	Nov 03 10:18:46 dev01.example.net opendkim[47562]: OpenDKIM Filter v2.11.0 terminating with status 0, errno = 0
	Nov 03 10:18:46 dev01.example.net opendkim[47918]: OpenDKIM Filter: Opening listen socket on conn local:/var/run/opendkim/opendkim.sock
	Nov 03 10:18:46 dev01.example.net opendkim[47918]: OpenDKIM Filter v2.11.0 starting (args: -l -x /usr/local/etc/opendkim/opendkim.conf -u opendkim)

on send,

postfix log

tail -f /var/log/postfix/postfix.log
	Nov  3 10:20:04 dev01 postfix/pickup[45473]: 8EE6838: uid=0 from=<root>
	Nov  3 10:20:04 dev01 postfix/cleanup[48075]: 8EE6838: message-id=<20181103172004.8EE6838@localhost>
	Nov  3 10:20:04 dev01 postfix/cleanup[48075]: warning: milter unix:/var/run/opendkim/opendkim.sock: can't read SMFIC_HEADER reply packet header: Success
	Nov  3 10:20:04 dev01 postfix/cleanup[48075]: 8EE6838: milter-reject: END-OF-MESSAGE from localhost[127.0.0.1]: 4.7.1 Service unavailable - try again later; from=<root@localhost> to=<root@localhost>

journalctl -f
	Nov 03 10:20:04 dev01.example.net postfix/cleanup[48075]: warning: milter unix:/var/run/opendkim/opendkim.sock: can't read SMFIC_HEADER reply packet header: Success
	Nov 03 10:20:04 dev01.example.net systemd[1]: opendkim-custom.service: Main process exited, code=killed, status=6/ABRT
	Nov 03 10:20:04 dev01.example.net systemd[1]: opendkim-custom.service: Unit entered failed state.
	Nov 03 10:20:04 dev01.example.net systemd[1]: opendkim-custom.service: Failed with result 'signal'.
	Nov 03 10:20:20 dev01.example.net postfix/cleanup[48075]: warning: connect to Milter service unix:/var/run/opendkim/opendkim.sock: Connection refused

ls -altr /var/lib/systemd/coredump/
	-rw------- 1 opendkim postfix-milter  43M Nov  3 10:22 core.48234

and

gdb /usr/local/opendkim/sbin/opendkim /var/lib/systemd/coredump/core.48234

GNU gdb (GDB; openSUSE Leap 15.0) 8.2
...
Core was generated by `/usr/local/opendkim/sbin/opendkim -l -x /usr/local/etc/opendkim/opendkim.conf -'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
[Current thread is 1 (Thread 0x7f08244dc700 (LWP 48238))]
(gdb) thread apply all bt f

Thread 6 (Thread 0x7f0829398740 (LWP 48234)):
#0  0x00007f082677d07b in __GI___poll (fds=0x7ffe1a2d6500, nfds=1, timeout=5000) at ../sysdeps/unix/sysv/linux/poll.c:29
        resultvar = 18446744073709551100
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f0828ba7ef1 in mi_listener () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#2  0x00007f0828ba57a7 in smfi_main () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#3  0x000000000041a8fb in main ()
No symbol table info available.
--Type <RET> for more, q to quit, c to continue without paging--c

Thread 5 (Thread 0x7f0823cdb700 (LWP 48239)):
#0  0x00007f0826a56c56 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f0823cdaec0, expected=0, futex_word=0x7f0828daf484) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
        __ret = -516
        oldtype = 0
        err = <optimized out>
        oldtype = <optimized out>
        err = <optimized out>
        __ret = <optimized out>
        resultvar = <optimized out>
        __arg6 = <optimized out>
        __arg5 = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a6 = <optimized out>
        _a5 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x7f0823cdaec0, mutex=0x7f0828daf430, cond=0x7f0828daf458) at pthread_cond_wait.c:539
        spin = 0
        buffer = {__routine = 0x7f0826a56610 <__condvar_cleanup_waiting>, __arg = 0x7f0823cdae50, __canceltype = 6, __prev = 0x0}
        cbuffer = {wseq = 59, cond = 0x7f0828daf458, mutex = 0x7f0828daf430, private = 0}
        err = <optimized out>
        g = 1
        flags = 8
        g1_start = <optimized out>
        maxspin = 0
        signals = <optimized out>
        result = 0
        wseq = <optimized out>
        seq = 29
        private = <optimized out>
        maxspin = <optimized out>
        err = <optimized out>
        result = <optimized out>
        wseq = <optimized out>
        g = <optimized out>
        seq = <optimized out>
        flags = <optimized out>
        private = <optimized out>
        signals = <optimized out>
        g1_start = <optimized out>
        spin = <optimized out>
        buffer = <optimized out>
        cbuffer = <optimized out>
        rt = <optimized out>
        s = <optimized out>
#2  __pthread_cond_timedwait (cond=0x7f0828daf458, mutex=0x7f0828daf430, abstime=0x7f0823cdaec0) at pthread_cond_wait.c:667
No locals.
#3  0x00007f0828ba8ff9 in ?? () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#4  0x00007f0826a50559 in start_thread (arg=0x7f0823cdb700) at pthread_create.c:465
        pd = 0x7f0823cdb700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139672937150208, 1266886485488127309, 140729337603102, 140729337603103, 0, 140729337603104, -1187642911359808179, -1187631154765542067}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#5  0x00007f082678781f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 4 (Thread 0x7f0824cdd700 (LWP 48237)):
#0  0x00007f082677d07b in __GI___poll (fds=0x7f081c000b10, nfds=1, timeout=10000) at ../sysdeps/unix/sysv/linux/poll.c:29
        resultvar = 18446744073709551100
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f0828ba88df in ?? () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#2  0x00007f0826a50559 in start_thread (arg=0x7f0824cdd700) at pthread_create.c:465
        pd = 0x7f0824cdd700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139672953935616, 1266886485488127309, 140729337603102, 140729337603103, 0, 140729337603104, -1187636315363783347, -1187631154765542067}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#3  0x00007f082678781f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 3 (Thread 0x7f08254de700 (LWP 48236)):
#0  do_sigwait (set=<optimized out>, set@entry=0x7f08254dde60, sig=sig@entry=0x7f08254dde5c) at ../sysdeps/unix/sysv/linux/sigwait.c:64
        ret = <optimized out>
        tmpset = {__val = {0 <repeats 16 times>}}
        __mask = <optimized out>
        __word = <optimized out>
        __mask = <optimized out>
        __word = <optimized out>
#1  0x00007f0826a5ae6d in __sigwait (set=0x7f08254dde60, sig=0x7f08254dde5c) at ../sysdeps/unix/sysv/linux/sigwait.c:96
        oldtype = 0
        result = <optimized out>
        sig = 0x7f08254dde5c
        set = 0x7f08254dde60
        oldtype = <optimized out>
        result = <optimized out>
        oldtype = <optimized out>
        result = <optimized out>
#2  0x00007f0828baa9ef in ?? () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#3  0x00007f0826a50559 in start_thread (arg=0x7f08254de700) at pthread_create.c:465
        pd = 0x7f08254de700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139672962328320, 1266886485488127309, 140729337603230, 140729337603231, 21284800, 140729337603232, -1187639613361795763, -1187631154765542067}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#4  0x00007f082678781f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 2 (Thread 0x7f0825cdf700 (LWP 48235)):
#0  do_sigwait (set=<optimized out>, set@entry=0x7f0825cdee70, sig=sig@entry=0x7f0825cdeefc) at ../sysdeps/unix/sysv/linux/sigwait.c:64
        ret = <optimized out>
        tmpset = {__val = {0 <repeats 16 times>}}
        __mask = <optimized out>
        __word = <optimized out>
        __mask = <optimized out>
        __word = <optimized out>
#1  0x00007f0826a5ae6d in __sigwait (set=0x7f0825cdee70, sig=0x7f0825cdeefc) at ../sysdeps/unix/sysv/linux/sigwait.c:96
        oldtype = 0
        result = <optimized out>
        sig = 0x7f0825cdeefc
        set = 0x7f0825cdee70
        oldtype = <optimized out>
        result = <optimized out>
        oldtype = <optimized out>
        result = <optimized out>
#2  0x00000000004082c4 in dkimf_reloader ()
No symbol table info available.
#3  0x00007f0826a50559 in start_thread (arg=0x7f0825cdf700) at pthread_create.c:465
        pd = 0x7f0825cdf700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139672970721024, 1266886485488127309, 140729337603422, 140729337603423, 0, 140729337603424, -1187638515460780723, -1187631154765542067}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#4  0x00007f082678781f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 1 (Thread 0x7f08244dc700 (LWP 48238)):
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
        set = {__val = {16899, 1, 139672984859280, 139672984766240, 139672336465920, 0, 139673028583424, 206158430224, 139672945540560, 139672945540352, 139673028583424, 139672945539368, 139672945539364, 139673024096504, 139672336465920, 139673024096504}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1  0x00007f08266c66c1 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0xffffffff, sa_sigaction = 0xffffffff}, sa_mask = {__val = {91, 139672984787752, 1, 139673028133840, 139673026382228, 91, 139672984796992, 139673028133840, 139672945539768, 139672945539764, 139673026381518, 0, 139672984796992, 139672984788760, 139672945539744, 139672945540352}}, sa_flags = 609073824, sa_restorer = 0x1000}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007f0826708427 in __libc_message (action=action@entry=(do_abort | do_backtrace), fmt=fmt@entry=0x7f0826811210 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:181
        ap = {{gp_offset = 40, fp_offset = 0, overflow_arg_area = 0x7f08244dbd10, reg_save_area = 0x7f08244dbca0}}
        fd = <optimized out>
        list = <optimized out>
        nlist = <optimized out>
        cp = <optimized out>
        written = <optimized out>
#3  0x00007f082670ec43 in malloc_printerr (action=<optimized out>, str=0x7f082680dbda "free(): invalid pointer", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5428
        buf = "00007f0814041690"
        cp = <optimized out>
        ar_ptr = <optimized out>
        ptr = <optimized out>
        str = 0x7f082680dbda "free(): invalid pointer"
        action = <optimized out>
        buf = <optimized out>
        cp = <optimized out>
#4  0x00007f0826710549 in _int_free (av=0x7f0826a43c20 <main_arena>, p=<optimized out>, have_lock=<optimized out>) at malloc.c:4170
        size = <optimized out>
        fb = <optimized out>
        nextchunk = <optimized out>
        nextsize = <optimized out>
        nextinuse = <optimized out>
        prevsize = <optimized out>
        bck = <optimized out>
        fwd = <optimized out>
        errstr = <optimized out>
        locked = <optimized out>
        __func__ = "_int_free"
#5  0x00007f0828ba66c4 in mi_engine () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#6  0x00007f0828ba8db6 in ?? () from /usr/lib64/libmilter.so.1.0
No symbol table info available.
#7  0x00007f0826a50559 in start_thread (arg=0x7f08244dc700) at pthread_create.c:465
        pd = 0x7f08244dc700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139672945542912, 1266886485488127309, 140729337603102, 140729337603103, 0, 140729337603104, -1187637413264798387, -1187631154765542067}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
        frame = <optimized out>
#8  0x00007f082678781f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.
(gdb)

ghost avatar Nov 03 '18 17:11 ghost

As I wrote earlier new ld.bfd (after 27 Feb 2018) with old valgrind (before 16 Juli 2018) do not work together, when it comes to debug information, see https://bugs.kde.org/show_bug.cgi?id=395682 .

dilyanpalauzov avatar Nov 03 '18 18:11 dilyanpalauzov

upgrading to upstream, latest release

valgrind --version
  valgrind-3.14.0
tail -f /var/log/valgrind/opendkim.log
==12328== Thread 5:
==12328== Conditional jump or move depends on uninitialised value(s)
==12328==    at 0x4E58AA2: dkim_dns_config (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x41F5F4: dkimf_dns_config (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x40C2D4: dkimf_dns_init (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x410F74: mlfi_connect (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527A066: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Uninitialised value was created by a heap allocation
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x4E50FF6: dkim_init (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40C35B: dkimf_config_setlib (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41A05B: main (in /usr/local/opendkim/sbin/opendkim)
==12328== 
==12328== Thread 6:
==12328== Invalid read of size 4
==12328==    at 0x6D2942E: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x85059d0 is 80 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid read of size 8
==12328==    at 0x6D29449: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x8505988 is 8 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid read of size 8
==12328==    at 0x6D29450: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x8505990 is 16 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid read of size 8
==12328==    at 0x6D2947C: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x8505980 is 0 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid read of size 4
==12328==    at 0x6D2F6E9: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D29491: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x85059a4 is 36 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid read of size 4
==12328==    at 0x6D2F6F5: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D29491: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x85059a0 is 32 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid read of size 8
==12328==    at 0x6D2F702: mem_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D29491: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x85059b8 is 56 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid read of size 8
==12328==    at 0x6E133FE: CRYPTO_free_ex_data (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D294A2: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x85059e8 is 104 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid write of size 8
==12328==    at 0x6E13407: CRYPTO_free_ex_data (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D294A2: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x85059e8 is 104 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid read of size 8
==12328==    at 0x6D294A3: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x85059f0 is 112 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid free() / delete / delete[] / realloc()
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294AB: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x8505a40 is 0 bytes inside a block of size 56 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294AB: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6E89D96: CRYPTO_THREAD_lock_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D29319: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328== 
==12328== Invalid free() / delete / delete[] / realloc()
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E525B9: dkim_free (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x40D6A4: dkimf_cleanup (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x416EA4: mlfi_close (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x527954F: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Address 0x8505980 is 0 bytes inside a block of size 120 free'd
==12328==    at 0x4C2F49B: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6D294BF: BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E506AD: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x73C6558: start_thread (pthread_create.c:465)
==12328==    by 0x76D581E: clone (clone.S:95)
==12328==  Block was alloc'd at
==12328==    at 0x4C2E26F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==12328==    by 0x6E19428: CRYPTO_zalloc (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D292DC: BIO_new (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x6D2FFC9: BIO_new_mem_buf (in /usr/local/openssl11/lib64/libcrypto.so.1.1)
==12328==    by 0x4E4B0CE: dkim_privkey_load (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E4FEC9: dkim_eom_sign (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x4E55581: dkim_eom (in /usr/local/opendkim/lib64/libopendkim.so.11.0.0)
==12328==    by 0x407987: dkimf_msr_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x41673F: mlfi_eom (in /usr/local/opendkim/sbin/opendkim)
==12328==    by 0x52790B4: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x52794AE: mi_engine (in /usr/lib64/libmilter.so.1.0.1)
==12328==    by 0x527BDB5: ??? (in /usr/lib64/libmilter.so.1.0.1)
==12328==

ghost avatar Nov 04 '18 03:11 ghost

BIO_free (in /usr/local/openssl11/lib64/libcrypto.so.1.1) implies OpenSSL 1.1 or LibreSSL (I do not know). On my system I use GnuTLS 3.6.6 with the code use from https://mail.aegee.org/cgit/OpenDKIM/ and OpenDKIM does not crash.

https://github.com/trusteddomainproject/OpenDKIM/issues/32 contains a patch for OpenDKIM when LibreSSL 2.8.2 is used.

dilyanpalauzov avatar Feb 15 '19 21:02 dilyanpalauzov