gerbera icon indicating copy to clipboard operation
gerbera copied to clipboard

memory leaks with libebml

Open KarlStraussberger opened this issue 3 years ago • 21 comments

Latest release leaks memory with libebml:

=================================================================
==1500777==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 340928 byte(s) in 5327 object(s) allocated from:
    #0 0x7f7d41a40087 in operator new(unsigned long) (/lib64/libasan.so.6+0xb0087)
    #1 0x7f7d401dd584 in libebml::EbmlVoid::Create() (/lib64/libebml.so.5+0x16584)

Direct leak of 3264 byte(s) in 1 object(s) allocated from:
    #0 0x7f7d41a3ec98 in __interceptor_realloc (/lib64/libasan.so.6+0xaec98)
    #1 0x10938b3 in ixml_membuf_insert (/home/mangix/devstuff/gerbera/upnp/gerbera+0x10938b3)
    #2 0x10927b5 in ixmlPrintDomTreeRecursive (/home/mangix/devstuff/gerbera/upnp/gerbera+0x10927b5)
    #3 0x1092dc0 in ixmlPrintDocument (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1092dc0)
    #4 0x106a83a in configure_urlbase (/home/mangix/devstuff/gerbera/upnp/gerbera+0x106a83a)
    #5 0x1035aa3 in GetDescDocumentAndURL.constprop.0 (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1035aa3)
    #6 0x1036013 in UpnpRegisterRootDevice2 (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1036013)
    #7 0xa7ea3e in Server::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa7ea3e)
    #8 0x5cf15b in main (/home/mangix/devstuff/gerbera/upnp/gerbera+0x5cf15b)
    #9 0x7f7d3f63055f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)

Direct leak of 1512 byte(s) in 21 object(s) allocated from:
    #0 0x7f7d41a40087 in operator new(unsigned long) (/lib64/libasan.so.6+0xb0087)
    #1 0x7f7d401d8744 in libebml::EbmlCrc32::Create() (/lib64/libebml.so.5+0x11744)

Direct leak of 17 byte(s) in 1 object(s) allocated from:
    #0 0x7f7d41a3ec98 in __interceptor_realloc (/lib64/libasan.so.6+0xaec98)
    #1 0x1061af3 in membuffer_set_size (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1061af3)
    #2 0x1061e89 in membuffer_insert (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1061e89)
    #3 0x10573bc in web_server_set_alias (/home/mangix/devstuff/gerbera/upnp/gerbera+0x10573bc)
    #4 0x106a8d0 in configure_urlbase (/home/mangix/devstuff/gerbera/upnp/gerbera+0x106a8d0)
    #5 0x1035aa3 in GetDescDocumentAndURL.constprop.0 (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1035aa3)
    #6 0x1036013 in UpnpRegisterRootDevice2 (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1036013)
    #7 0xa7ea3e in Server::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa7ea3e)
    #8 0x5cf15b in main (/home/mangix/devstuff/gerbera/upnp/gerbera+0x5cf15b)
    #9 0x7f7d3f63055f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)

Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x7f7d41a3e91f in __interceptor_malloc (/lib64/libasan.so.6+0xae91f)
    #1 0x10573d1 in web_server_set_alias (/home/mangix/devstuff/gerbera/upnp/gerbera+0x10573d1)
    #2 0x106a8d0 in configure_urlbase (/home/mangix/devstuff/gerbera/upnp/gerbera+0x106a8d0)
    #3 0x1035aa3 in GetDescDocumentAndURL.constprop.0 (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1035aa3)
    #4 0x1036013 in UpnpRegisterRootDevice2 (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1036013)
    #5 0xa7ea3e in Server::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa7ea3e)
    #6 0x5cf15b in main (/home/mangix/devstuff/gerbera/upnp/gerbera+0x5cf15b)
    #7 0x7f7d3f63055f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)

SUMMARY: AddressSanitizer: 345725 byte(s) leaked in 5351 allocation(s).

Fixed by: https://github.com/gerbera/gerbera/pull/2280

Originally posted by @neheb in https://github.com/gerbera/gerbera/issues/2456#issuecomment-1050542477

KarlStraussberger avatar Feb 25 '22 07:02 KarlStraussberger

@neheb This is more likely to be a ebml issue. Can you try patching your libebml with


EbmlElement * EbmlElement::SkipData(EbmlStream & DataStream, const EbmlSemanticContext & Context, EbmlElement * TestReadElt, bool AllowDummyElt)
{
  EbmlElement * Result = nullptr;
  if (bSizeIsFinite) {
    assert(TestReadElt == nullptr);
    assert(ElementPosition < SizePosition);
    DataStream.I_O().setFilePointer(SizePosition + CodedSizeLength(Size, SizeLength, bSizeIsFinite) + Size, seek_beginning);
    //    DataStream.I_O().setFilePointer(Size, seek_current);
  } else {
    /////////////////////////////////////////////////
    // read elements until an upper element is found
    /////////////////////////////////////////////////
    bool bEndFound = false;
    while (!bEndFound && Result == nullptr) {
      // read an element
      /// \todo 0xFF... and true should be configurable
      //      EbmlElement * NewElt;
      if (TestReadElt == nullptr) {
        int bUpperElement = 0; // trick to call FindNextID correctly
        Result = DataStream.FindNextElement(Context, bUpperElement, 0xFFFFFFFFL, AllowDummyElt);
      } else {
        Result = TestReadElt;
        TestReadElt = nullptr;
      }

      if (Result != nullptr) {
        unsigned int EltIndex;
        // data known in this Master's context
        for (EltIndex = 0; EltIndex < EBML_CTX_SIZE(Context); EltIndex++) {
          if (EbmlId(*Result) == EBML_CTX_IDX_ID(Context,EltIndex)) {
            // skip the data with its own context
            EbmlElement * Result2 = Result;
            Result = Result2->SkipData(DataStream, EBML_SEM_CONTEXT(EBML_CTX_IDX(Context,EltIndex)), nullptr);
            if (Result2 != Result) delete Result2;
            break; // let's go to the next ID
          }
        }

        if (EltIndex >= EBML_CTX_SIZE(Context)) {
          if (EBML_CTX_PARENT(Context) != nullptr) {
            EbmlElement * Result2 = Result;
            Result = SkipData(DataStream, *EBML_CTX_PARENT(Context), Result);
            if (Result2 != Result) delete Result2;
          } else {
            assert(Context.GetGlobalContext != nullptr);
            if (Context != Context.GetGlobalContext()) {
              EbmlElement * Result2 = Result;
              Result = SkipData(DataStream, Context.GetGlobalContext(), Result);
              if (Result2 != Result) delete Result2;
            } else {
              bEndFound = true;
            }
          }
        }
      } else {
        bEndFound = true;
      }
    }
  }
  return Result;
}

KarlStraussberger avatar Feb 25 '22 07:02 KarlStraussberger

That just results in instant crash

=================================================================
==1723157==ERROR: AddressSanitizer: heap-use-after-free on address 0x6060000ba5c0 at pc 0x000000ff511a bp 0x7f262a5141b0 sp 0x7f262a5141a8
READ of size 8 at 0x6060000ba5c0 thread T5
    #0 0xff5119 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias] (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5119)
    #1 0xa85d79 in MatroskaHandler::parseMKV(std::shared_ptr<CdsItem> const&, std::unique_ptr<MemIOHandler, std::default_delete<MemIOHandler> >*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa85d79)
    #2 0xa863bb in MatroskaHandler::fillMetadata(std::shared_ptr<CdsObject> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa863bb)
    #3 0xa11f47 in MetadataHandler::extractMetaData(std::shared_ptr<Context> const&, std::shared_ptr<CdsItem> const&, std::filesystem::__cxx11::directory_entry const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa11f47)
    #4 0x8ea546 in ContentManager::createObjectFromFile(std::filesystem::__cxx11::directory_entry const&, bool, bool) const (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ea546)
    #5 0x8ec55c in ContentManager::createSingleItem(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, bool, bool, bool, bool, std::shared_ptr<CMAddFileTask>) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ec55c)
    #6 0x8f1c02 in ContentManager::_addFile(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path, AutoScanSetting&, std::shared_ptr<CMAddFileTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f1c02)
    #7 0x8f2eba in ContentManager::addFileInternal(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, AutoScanSetting&, bool, bool, unsigned int, bool) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f2eba)
    #8 0x8ff960 in ContentManager::_rescanDirectory(std::shared_ptr<AutoscanDirectory> const&, int, std::shared_ptr<GenericTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ff960)
    #9 0x9028b8 in CMRescanDirectoryTask::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9028b8)
    #10 0x907868 in ContentManager::threadProc() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x907868)
    #11 0x9236b2 in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::_FUN(void*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9236b2)
    #12 0x7f2647ac5b19 in start_thread (/lib64/libc.so.6+0x8db19)
    #13 0x7f2647b4a64f in __GI___clone3 (/lib64/libc.so.6+0x11264f)

0x6060000ba5c0 is located 0 bytes inside of 64-byte region [0x6060000ba5c0,0x6060000ba600)
freed by thread T5 here:
    #0 0x7f2649da6f07 in operator delete(void*, unsigned long) (/lib64/libasan.so.6+0xb0f07)
    #1 0xff4cb4 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias] (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff4cb4)
    #2 0xff4ec7 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias] (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff4ec7)
    #3 0xa85d79 in MatroskaHandler::parseMKV(std::shared_ptr<CdsItem> const&, std::unique_ptr<MemIOHandler, std::default_delete<MemIOHandler> >*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa85d79)
    #4 0xa863bb in MatroskaHandler::fillMetadata(std::shared_ptr<CdsObject> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa863bb)
    #5 0xa11f47 in MetadataHandler::extractMetaData(std::shared_ptr<Context> const&, std::shared_ptr<CdsItem> const&, std::filesystem::__cxx11::directory_entry const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa11f47)
    #6 0x8ea546 in ContentManager::createObjectFromFile(std::filesystem::__cxx11::directory_entry const&, bool, bool) const (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ea546)
    #7 0x8ec55c in ContentManager::createSingleItem(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, bool, bool, bool, bool, std::shared_ptr<CMAddFileTask>) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ec55c)
    #8 0x8f1c02 in ContentManager::_addFile(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path, AutoScanSetting&, std::shared_ptr<CMAddFileTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f1c02)
    #9 0x8f2eba in ContentManager::addFileInternal(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, AutoScanSetting&, bool, bool, unsigned int, bool) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f2eba)
    #10 0x8ff960 in ContentManager::_rescanDirectory(std::shared_ptr<AutoscanDirectory> const&, int, std::shared_ptr<GenericTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ff960)
    #11 0x9028b8 in CMRescanDirectoryTask::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9028b8)
    #12 0x907868 in ContentManager::threadProc() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x907868)
    #13 0x9236b2 in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::_FUN(void*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9236b2)
    #14 0x7f2647ac5b19 in start_thread (/lib64/libc.so.6+0x8db19)

previously allocated by thread T5 here:
    #0 0x7f2649da6087 in operator new(unsigned long) (/lib64/libasan.so.6+0xb0087)
    #1 0x10155b1 in libebml::EbmlVoid::Create() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x10155b1)
    #2 0xff5453 in libebml::EbmlElement::CreateElementUsingContext(libebml::EbmlId const&, libebml::EbmlSemanticContext const&, int&, bool, bool, unsigned int) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5453)
    #3 0xff5c69 in libebml::EbmlElement::FindNextElement(libebml::IOCallback&, libebml::EbmlSemanticContext const&, int&, unsigned long, bool, unsigned int) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5c69)
    #4 0xff4d31 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias] (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff4d31)
    #5 0xa85d79 in MatroskaHandler::parseMKV(std::shared_ptr<CdsItem> const&, std::unique_ptr<MemIOHandler, std::default_delete<MemIOHandler> >*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa85d79)
    #6 0xa863bb in MatroskaHandler::fillMetadata(std::shared_ptr<CdsObject> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa863bb)
    #7 0xa11f47 in MetadataHandler::extractMetaData(std::shared_ptr<Context> const&, std::shared_ptr<CdsItem> const&, std::filesystem::__cxx11::directory_entry const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa11f47)
    #8 0x8ea546 in ContentManager::createObjectFromFile(std::filesystem::__cxx11::directory_entry const&, bool, bool) const (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ea546)
    #9 0x8ec55c in ContentManager::createSingleItem(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, bool, bool, bool, bool, std::shared_ptr<CMAddFileTask>) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ec55c)
    #10 0x8f1c02 in ContentManager::_addFile(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path, AutoScanSetting&, std::shared_ptr<CMAddFileTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f1c02)
    #11 0x8f2eba in ContentManager::addFileInternal(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, AutoScanSetting&, bool, bool, unsigned int, bool) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f2eba)
    #12 0x8ff960 in ContentManager::_rescanDirectory(std::shared_ptr<AutoscanDirectory> const&, int, std::shared_ptr<GenericTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ff960)
    #13 0x9028b8 in CMRescanDirectoryTask::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9028b8)
    #14 0x907868 in ContentManager::threadProc() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x907868)
    #15 0x9236b2 in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::_FUN(void*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9236b2)
    #16 0x7f2647ac5b19 in start_thread (/lib64/libc.so.6+0x8db19)

Thread T5 created by T0 here:
    #0 0x7f2649d4c866 in pthread_create (/lib64/libasan.so.6+0x56866)
    #1 0x90ad03 in ContentManager::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x90ad03)
    #2 0xa9dfd5 in Server::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa9dfd5)
    #3 0x5d0bbb in main (/home/mangix/devstuff/gerbera/upnp/gerbera+0x5d0bbb)
    #4 0x7f2647a6555f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)

SUMMARY: AddressSanitizer: heap-use-after-free (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5119) in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias]
Shadow bytes around the buggy address:
  0x0c0c8000f460: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x0c0c8000f470: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
  0x0c0c8000f480: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c0c8000f490: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x0c0c8000f4a0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
=>0x0c0c8000f4b0: fd fd fd fd fa fa fa fa[fd]fd fd fd fd fd fd fd
  0x0c0c8000f4c0: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x0c0c8000f4d0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
  0x0c0c8000f4e0: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c0c8000f4f0: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x0c0c8000f500: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==1723157==ABORTING

neheb avatar Feb 25 '22 08:02 neheb

Fuller picture of the memory leaks (without that fix): https://gist.github.com/neheb/9b6d067f1bba94af59ce5a3350a74acc

What's wrong with my 2 line patch?

neheb avatar Feb 25 '22 08:02 neheb

The patch does not address the issue, it fixes its symptoms.

KarlStraussberger avatar Feb 25 '22 08:02 KarlStraussberger

sure but much better than hunting for a libebml bug, which we don't have control over.

neheb avatar Feb 25 '22 08:02 neheb

The spirit of open source: When you discover an issue, report or fix it. I'm about to locate it before reporting.

KarlStraussberger avatar Feb 25 '22 08:02 KarlStraussberger

That just results in instant crash

=================================================================
==1723157==ERROR: AddressSanitizer: heap-use-after-free on address 0x6060000ba5c0 at pc 0x000000ff511a bp 0x7f262a5141b0 sp 0x7f262a5141a8
READ of size 8 at 0x6060000ba5c0 thread T5
    #0 0xff5119 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias] (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5119)
    #1 0xa85d79 in MatroskaHandler::parseMKV(std::shared_ptr<CdsItem> const&, std::unique_ptr<MemIOHandler, std::default_delete<MemIOHandler> >*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa85d79)
    #2 0xa863bb in MatroskaHandler::fillMetadata(std::shared_ptr<CdsObject> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa863bb)
    #3 0xa11f47 in MetadataHandler::extractMetaData(std::shared_ptr<Context> const&, std::shared_ptr<CdsItem> const&, std::filesystem::__cxx11::directory_entry const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa11f47)
    #4 0x8ea546 in ContentManager::createObjectFromFile(std::filesystem::__cxx11::directory_entry const&, bool, bool) const (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ea546)
    #5 0x8ec55c in ContentManager::createSingleItem(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, bool, bool, bool, bool, std::shared_ptr<CMAddFileTask>) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ec55c)
    #6 0x8f1c02 in ContentManager::_addFile(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path, AutoScanSetting&, std::shared_ptr<CMAddFileTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f1c02)
    #7 0x8f2eba in ContentManager::addFileInternal(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, AutoScanSetting&, bool, bool, unsigned int, bool) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f2eba)
    #8 0x8ff960 in ContentManager::_rescanDirectory(std::shared_ptr<AutoscanDirectory> const&, int, std::shared_ptr<GenericTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ff960)
    #9 0x9028b8 in CMRescanDirectoryTask::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9028b8)
    #10 0x907868 in ContentManager::threadProc() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x907868)
    #11 0x9236b2 in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::_FUN(void*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9236b2)
    #12 0x7f2647ac5b19 in start_thread (/lib64/libc.so.6+0x8db19)
    #13 0x7f2647b4a64f in __GI___clone3 (/lib64/libc.so.6+0x11264f)

0x6060000ba5c0 is located 0 bytes inside of 64-byte region [0x6060000ba5c0,0x6060000ba600)
freed by thread T5 here:
    #0 0x7f2649da6f07 in operator delete(void*, unsigned long) (/lib64/libasan.so.6+0xb0f07)
    #1 0xff4cb4 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias] (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff4cb4)
    #2 0xff4ec7 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias] (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff4ec7)
    #3 0xa85d79 in MatroskaHandler::parseMKV(std::shared_ptr<CdsItem> const&, std::unique_ptr<MemIOHandler, std::default_delete<MemIOHandler> >*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa85d79)
    #4 0xa863bb in MatroskaHandler::fillMetadata(std::shared_ptr<CdsObject> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa863bb)
    #5 0xa11f47 in MetadataHandler::extractMetaData(std::shared_ptr<Context> const&, std::shared_ptr<CdsItem> const&, std::filesystem::__cxx11::directory_entry const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa11f47)
    #6 0x8ea546 in ContentManager::createObjectFromFile(std::filesystem::__cxx11::directory_entry const&, bool, bool) const (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ea546)
    #7 0x8ec55c in ContentManager::createSingleItem(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, bool, bool, bool, bool, std::shared_ptr<CMAddFileTask>) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ec55c)
    #8 0x8f1c02 in ContentManager::_addFile(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path, AutoScanSetting&, std::shared_ptr<CMAddFileTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f1c02)
    #9 0x8f2eba in ContentManager::addFileInternal(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, AutoScanSetting&, bool, bool, unsigned int, bool) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f2eba)
    #10 0x8ff960 in ContentManager::_rescanDirectory(std::shared_ptr<AutoscanDirectory> const&, int, std::shared_ptr<GenericTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ff960)
    #11 0x9028b8 in CMRescanDirectoryTask::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9028b8)
    #12 0x907868 in ContentManager::threadProc() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x907868)
    #13 0x9236b2 in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::_FUN(void*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9236b2)
    #14 0x7f2647ac5b19 in start_thread (/lib64/libc.so.6+0x8db19)

previously allocated by thread T5 here:
    #0 0x7f2649da6087 in operator new(unsigned long) (/lib64/libasan.so.6+0xb0087)
    #1 0x10155b1 in libebml::EbmlVoid::Create() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x10155b1)
    #2 0xff5453 in libebml::EbmlElement::CreateElementUsingContext(libebml::EbmlId const&, libebml::EbmlSemanticContext const&, int&, bool, bool, unsigned int) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5453)
    #3 0xff5c69 in libebml::EbmlElement::FindNextElement(libebml::IOCallback&, libebml::EbmlSemanticContext const&, int&, unsigned long, bool, unsigned int) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5c69)
    #4 0xff4d31 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias] (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff4d31)
    #5 0xa85d79 in MatroskaHandler::parseMKV(std::shared_ptr<CdsItem> const&, std::unique_ptr<MemIOHandler, std::default_delete<MemIOHandler> >*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa85d79)
    #6 0xa863bb in MatroskaHandler::fillMetadata(std::shared_ptr<CdsObject> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa863bb)
    #7 0xa11f47 in MetadataHandler::extractMetaData(std::shared_ptr<Context> const&, std::shared_ptr<CdsItem> const&, std::filesystem::__cxx11::directory_entry const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa11f47)
    #8 0x8ea546 in ContentManager::createObjectFromFile(std::filesystem::__cxx11::directory_entry const&, bool, bool) const (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ea546)
    #9 0x8ec55c in ContentManager::createSingleItem(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, bool, bool, bool, bool, std::shared_ptr<CMAddFileTask>) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ec55c)
    #10 0x8f1c02 in ContentManager::_addFile(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path, AutoScanSetting&, std::shared_ptr<CMAddFileTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f1c02)
    #11 0x8f2eba in ContentManager::addFileInternal(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, AutoScanSetting&, bool, bool, unsigned int, bool) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8f2eba)
    #12 0x8ff960 in ContentManager::_rescanDirectory(std::shared_ptr<AutoscanDirectory> const&, int, std::shared_ptr<GenericTask> const&) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x8ff960)
    #13 0x9028b8 in CMRescanDirectoryTask::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9028b8)
    #14 0x907868 in ContentManager::threadProc() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x907868)
    #15 0x9236b2 in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::_FUN(void*) (/home/mangix/devstuff/gerbera/upnp/gerbera+0x9236b2)
    #16 0x7f2647ac5b19 in start_thread (/lib64/libc.so.6+0x8db19)

Thread T5 created by T0 here:
    #0 0x7f2649d4c866 in pthread_create (/lib64/libasan.so.6+0x56866)
    #1 0x90ad03 in ContentManager::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x90ad03)
    #2 0xa9dfd5 in Server::run() (/home/mangix/devstuff/gerbera/upnp/gerbera+0xa9dfd5)
    #3 0x5d0bbb in main (/home/mangix/devstuff/gerbera/upnp/gerbera+0x5d0bbb)
    #4 0x7f2647a6555f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)

SUMMARY: AddressSanitizer: heap-use-after-free (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5119) in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool) [clone .localalias]
Shadow bytes around the buggy address:
  0x0c0c8000f460: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x0c0c8000f470: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
  0x0c0c8000f480: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c0c8000f490: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x0c0c8000f4a0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
=>0x0c0c8000f4b0: fd fd fd fd fa fa fa fa[fd]fd fd fd fd fd fd fd
  0x0c0c8000f4c0: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x0c0c8000f4d0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
  0x0c0c8000f4e0: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c0c8000f4f0: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x0c0c8000f500: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==1723157==ABORTING

Can you provide more details on the line where it crashes.

KarlStraussberger avatar Feb 25 '22 08:02 KarlStraussberger

#0  0x00000000007b6e0d in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool)
    (this=0x7fffbc016ce0, DataStream=..., Context=..., TestReadElt=0x0, AllowDummyElt=false) at ../subprojects/libebml-release-1.4.2/src/EbmlElement.cpp:530
#1  0x00000000005a3770 in MatroskaHandler::parseMKV(std::shared_ptr<CdsItem> const&, std::unique_ptr<MemIOHandler, std::default_delete<MemIOHandler> >*)
    (this=0x7fffd77fb7e0, item=std::shared_ptr<CdsItem> (use count 4, weak count 0) = {...}, pIoHandler=0x0) at ../src/metadata/matroska_handler.cc:144
#2  0x00000000005a34bc in MatroskaHandler::fillMetadata(std::shared_ptr<CdsObject> const&)
    (this=0x7fffd77fb7e0, obj=std::shared_ptr<CdsObject> (use count 4, weak count 0) = {...}) at ../src/metadata/matroska_handler.cc:101
#3  0x000000000057dd41 in MetadataHandler::extractMetaData(std::shared_ptr<Context> const&, std::shared_ptr<CdsItem> const&, std::filesystem::__cxx11::directory_entry const&)
    (context=std::shared_ptr<Context> (use count 2, weak count 0) = {...}, item=std::shared_ptr<CdsItem> (use count 4, weak count 0) = {...}, dirEnt=...)
    at ../src/metadata/metadata_handler.cc:112
#4  0x000000000051a7f2 in ContentManager::createObjectFromFile(std::filesystem::__cxx11::directory_entry const&, bool, bool) const
    (this=0xaf5af0, dirEnt=..., followSymlinks=true, allowFifo=false) at ../src/content/content_manager.cc:1230
#5  0x00000000005107b2 in ContentManager::createSingleItem(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, bool, bool, bool, bool, std::shared_ptr<CMAddFileTask>)
    (this=0xaf5af0, dirEnt=..., rootPath=filesystem::path "/home/mangix/Videos" = {...}, followSymlinks=true, checkDatabase=true, processExisting=false, firstChild=false, task=std::shared_ptr<CMAddFileTask> (empty) = {...}) at ../src/content/content_manager.cc:395
#6  0x00000000005110a8 in ContentManager::_addFile(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path, AutoScanSetting&, std::shared_ptr<CMAddFileTask> const&) (this=0xaf5af0, dirEnt=..., rootPath=filesystem::path "/home/mangix/Videos" = {...}, asSetting=..., task=std::shared_ptr<CMAddFileTask> (empty) = {...})
    at ../src/content/content_manager.cc:447
#7  0x000000000051bd36 in ContentManager::addFileInternal(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, AutoScanSetting&, bool, bool, unsigned int, bool)
    (this=0xaf5af0, dirEnt=..., rootpath=filesystem::path "/home/mangix/Videos" = {...}, asSetting=..., async=false, lowPriority=false, parentTaskID=0, cancellable=true)
    at ../src/content/content_manager.cc:1400
#8  0x00000000005139ab in ContentManager::_rescanDirectory(std::shared_ptr<AutoscanDirectory> const&, int, std::shared_ptr<GenericTask> const&)
    (this=0xaf5af0, adir=std::shared_ptr<AutoscanDirectory> (use count 39, weak count 0) = {...}, containerID=23, task=std::shared_ptr<GenericTask> (use count 3, weak count 1) = {...}) at ../src/content/content_manager.cc:705
#9  0x0000000000520050 in CMRescanDirectoryTask::run() (this=0x7fffc400e180) at ../src/content/content_manager.cc:1839
#10 0x000000000051b642 in ContentManager::threadProc() (this=0xaf5af0) at ../src/content/content_manager.cc:1347
#11 0x000000000050dd54 in operator()(void*) const (__closure=0xb0dc48, arg=0xaf5af0) at ../src/content/content_manager.cc:102
#12 0x000000000052135f in std::__invoke_impl<void*, ContentManager::run()::<lambda(void*)>&, void*>(std::__invoke_other, struct {...} &) (__f=...)
    at /usr/include/c++/11/bits/invoke.h:61
#13 0x000000000052139c in std::__invoke_r<void, ContentManager::run()::<lambda(void*)>&, void*>(struct {...} &) (__fn=...) at /usr/include/c++/11/bits/invoke.h:111
#14 0x0000000000520ca6 in std::_Function_handler<void(void*), ContentManager::run()::<lambda(void*)> >::_M_invoke(const std::_Any_data &, void *&&)
    (__functor=..., __args#0=@0x7fffd77fd0e0: 0xaf5af0) at /usr/include/c++/11/bits/std_function.h:290
#15 0x0000000000535f09 in std::function<void (void*)>::operator()(void*) const (this=0xb0dc48, __args#0=0xaf5af0) at /usr/include/c++/11/bits/std_function.h:590
#16 0x0000000000532003 in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::operator()(void*) const
    (__closure=0x0, arg=0xb0dba0) at ../src/util/thread_runner.h:197
#17 0x000000000053202a in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::_FUN(void*) ()
    at ../src/util/thread_runner.h:195
#18 0x00007ffff5d79b1a in start_thread () at /lib64/libc.so.6
#19 0x00007ffff5dfe650 in clone3 () at /lib64/libc.so.6

neheb avatar Feb 25 '22 08:02 neheb

unless I'm reading this wrong, pIoHandler is NULL...

neheb avatar Feb 25 '22 08:02 neheb

pIoHandler is not passed into libebml. Another try

EbmlElement * EbmlElement::SkipData(EbmlStream & DataStream, const EbmlSemanticContext & Context, EbmlElement * TestReadElt, bool AllowDummyElt)
{
  EbmlElement * Result = nullptr;
  if (bSizeIsFinite) {
    assert(TestReadElt == nullptr);
    assert(ElementPosition < SizePosition);
    DataStream.I_O().setFilePointer(SizePosition + CodedSizeLength(Size, SizeLength, bSizeIsFinite) + Size, seek_beginning);
    //    DataStream.I_O().setFilePointer(Size, seek_current);
  } else {
    /////////////////////////////////////////////////
    // read elements until an upper element is found
    /////////////////////////////////////////////////
    bool bEndFound = false;
    while (!bEndFound && Result == nullptr) {
      // read an element
      /// \todo 0xFF... and true should be configurable
      //      EbmlElement * NewElt;
      if (TestReadElt == nullptr) {
        int bUpperElement = 0; // trick to call FindNextID correctly
        Result = DataStream.FindNextElement(Context, bUpperElement, 0xFFFFFFFFL, AllowDummyElt);
      } else {
        Result = TestReadElt;
        //TestReadElt = nullptr;
      }

      if (Result != nullptr) {
        unsigned int EltIndex;
        // data known in this Master's context
        for (EltIndex = 0; EltIndex < EBML_CTX_SIZE(Context); EltIndex++) {
          if (EbmlId(*Result) == EBML_CTX_IDX_ID(Context,EltIndex)) {
            // skip the data with its own context
            EbmlElement * Result2 = Result;
            Result = Result2->SkipData(DataStream, EBML_SEM_CONTEXT(EBML_CTX_IDX(Context,EltIndex)), nullptr);
            if (Result2 != Result && TestReadElt != nullptr) delete Result2;
            break; // let's go to the next ID
          }
        }

        if (EltIndex >= EBML_CTX_SIZE(Context)) {
          if (EBML_CTX_PARENT(Context) != nullptr) {
            EbmlElement * Result2 = Result;
            Result = SkipData(DataStream, *EBML_CTX_PARENT(Context), Result);
            if (Result2 != Result && TestReadElt != nullptr) delete Result2;
          } else {
            assert(Context.GetGlobalContext != nullptr);
            if (Context != Context.GetGlobalContext()) {
              EbmlElement * Result2 = Result;
              Result = SkipData(DataStream, Context.GetGlobalContext(), Result);
              if (Result2 != Result && TestReadElt != nullptr) delete Result2;
            } else {
              bEndFound = true;
            }
          }
        }
      } else {
        bEndFound = true;
      }
    }
  }
  return Result;
}

KarlStraussberger avatar Feb 25 '22 09:02 KarlStraussberger

#0  0x00000000007b6bda in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool)
    (this=0x7fffb8016cc0, DataStream=..., Context=..., TestReadElt=0x7fffb8017d80, AllowDummyElt=false) at ../subprojects/libebml-release-1.4.2/src/EbmlElement.cpp:511
#1  0x00000000007b6dc0 in libebml::EbmlElement::SkipData(libebml::EbmlStream&, libebml::EbmlSemanticContext const&, libebml::EbmlElement*, bool)
    (this=0x7fffb8016cc0, DataStream=..., Context=..., TestReadElt=0x0, AllowDummyElt=false) at ../subprojects/libebml-release-1.4.2/src/EbmlElement.cpp:529
#2  0x00000000005a375c in MatroskaHandler::parseMKV(std::shared_ptr<CdsItem> const&, std::unique_ptr<MemIOHandler, std::default_delete<MemIOHandler> >*)
    (this=0x7fffd77fb7e0, item=std::shared_ptr<CdsItem> (use count 4, weak count 0) = {...}, pIoHandler=0x0) at ../src/metadata/matroska_handler.cc:144
#3  0x00000000005a34bc in MatroskaHandler::fillMetadata(std::shared_ptr<CdsObject> const&)
    (this=0x7fffd77fb7e0, obj=std::shared_ptr<CdsObject> (use count 4, weak count 0) = {...}) at ../src/metadata/matroska_handler.cc:101
#4  0x000000000057dd41 in MetadataHandler::extractMetaData(std::shared_ptr<Context> const&, std::shared_ptr<CdsItem> const&, std::filesystem::__cxx11::directory_entry const&)
    (context=std::shared_ptr<Context> (use count 2, weak count 0) = {...}, item=std::shared_ptr<CdsItem> (use count 4, weak count 0) = {...}, dirEnt=...)
    at ../src/metadata/metadata_handler.cc:112
#5  0x000000000051a7f2 in ContentManager::createObjectFromFile(std::filesystem::__cxx11::directory_entry const&, bool, bool) const
    (this=0xaf5af0, dirEnt=..., followSymlinks=true, allowFifo=false) at ../src/content/content_manager.cc:1230
#6  0x00000000005107b2 in ContentManager::createSingleItem(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, bool, bool, bool, bool, std::shared_ptr<CMAddFileTask>)
    (this=0xaf5af0, dirEnt=..., rootPath=filesystem::path "/home/mangix/Videos" = {...}, followSymlinks=true, checkDatabase=true, processExisting=false, firstChild=false, task=std::shared_ptr<CMAddFileTask> (empty) = {...}) at ../src/content/content_manager.cc:395
#7  0x00000000005110a8 in ContentManager::_addFile(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path, AutoScanSetting&, std::shared_ptr<CMAddFileTask> const&) (this=0xaf5af0, dirEnt=..., rootPath=filesystem::path "/home/mangix/Videos" = {...}, asSetting=..., task=std::shared_ptr<CMAddFileTask> (empty) = {...})
    at ../src/content/content_manager.cc:447
#8  0x000000000051bd36 in ContentManager::addFileInternal(std::filesystem::__cxx11::directory_entry const&, std::filesystem::__cxx11::path const&, AutoScanSetting&, bool, bool, unsigned int, bool)
    (this=0xaf5af0, dirEnt=..., rootpath=filesystem::path "/home/mangix/Videos" = {...}, asSetting=..., async=false, lowPriority=false, parentTaskID=0, cancellable=true)
    at ../src/content/content_manager.cc:1400
#9  0x00000000005139ab in ContentManager::_rescanDirectory(std::shared_ptr<AutoscanDirectory> const&, int, std::shared_ptr<GenericTask> const&)
    (this=0xaf5af0, adir=std::shared_ptr<AutoscanDirectory> (use count 39, weak count 0) = {...}, containerID=23, task=std::shared_ptr<GenericTask> (use count 3, weak count 1) = {...}) at ../src/content/content_manager.cc:705
#10 0x0000000000520050 in CMRescanDirectoryTask::run() (this=0x7fff8c00e110) at ../src/content/content_manager.cc:1839
#11 0x000000000051b642 in ContentManager::threadProc() (this=0xaf5af0) at ../src/content/content_manager.cc:1347
#12 0x000000000050dd54 in operator()(void*) const (__closure=0xb0dc48, arg=0xaf5af0) at ../src/content/content_manager.cc:102
#13 0x000000000052135f in std::__invoke_impl<void*, ContentManager::run()::<lambda(void*)>&, void*>(std::__invoke_other, struct {...} &) (__f=...)
    at /usr/include/c++/11/bits/invoke.h:61
#14 0x000000000052139c in std::__invoke_r<void, ContentManager::run()::<lambda(void*)>&, void*>(struct {...} &) (__fn=...) at /usr/include/c++/11/bits/invoke.h:111
#15 0x0000000000520ca6 in std::_Function_handler<void(void*), ContentManager::run()::<lambda(void*)> >::_M_invoke(const std::_Any_data &, void *&&)
    (__functor=..., __args#0=@0x7fffd77fd0e0: 0xaf5af0) at /usr/include/c++/11/bits/std_function.h:290
#16 0x0000000000535f09 in std::function<void (void*)>::operator()(void*) const (this=0xb0dc48, __args#0=0xaf5af0) at /usr/include/c++/11/bits/std_function.h:590
#17 0x0000000000532003 in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::operator()(void*) const
    (__closure=0x0, arg=0xb0dba0) at ../src/util/thread_runner.h:197
#18 0x000000000053202a in ThreadRunner<std::_V2::condition_variable_any, std::recursive_mutex>::startThread()::{lambda(void*)#1}::_FUN(void*) ()
    at ../src/util/thread_runner.h:195
#19 0x00007ffff5d79b1a in start_thread () at /lib64/libc.so.6
#20 0x00007ffff5dfe650 in clone3 () at /lib64/libc.so.6

neheb avatar Feb 25 '22 09:02 neheb

Seems to be a lot more sophisticated

EbmlElement * EbmlElement::SkipData(EbmlStream & DataStream, const EbmlSemanticContext & Context, EbmlElement * TestReadElt, bool AllowDummyElt)
{
  EbmlElement * Result = nullptr;
  EbmlElement * Result2 = nullptr;
  if (bSizeIsFinite) {
    assert(TestReadElt == nullptr);
    assert(ElementPosition < SizePosition);
    DataStream.I_O().setFilePointer(SizePosition + CodedSizeLength(Size, SizeLength, bSizeIsFinite) + Size, seek_beginning);
    //    DataStream.I_O().setFilePointer(Size, seek_current);
  } else {
    /////////////////////////////////////////////////
    // read elements until an upper element is found
    /////////////////////////////////////////////////
    bool bEndFound = false;
    while (!bEndFound && Result == nullptr) {
      // read an element
      /// \todo 0xFF... and true should be configurable
      //      EbmlElement * NewElt;
      if (TestReadElt == nullptr) {
        int bUpperElement = 0; // trick to call FindNextID correctly
        Result = DataStream.FindNextElement(Context, bUpperElement, 0xFFFFFFFFL, AllowDummyElt);
        Result2 = Result;
      } else {
        Result = TestReadElt;
        TestReadElt = nullptr;
      }

      if (Result != nullptr) {
        unsigned int EltIndex;
        // data known in this Master's context
        for (EltIndex = 0; EltIndex < EBML_CTX_SIZE(Context); EltIndex++) {
          if (EbmlId(*Result) == EBML_CTX_IDX_ID(Context,EltIndex)) {
            // skip the data with its own context
            Result = Result->SkipData(DataStream, EBML_SEM_CONTEXT(EBML_CTX_IDX(Context,EltIndex)), nullptr);
            break; // let's go to the next ID
          }
        }

        if (EltIndex >= EBML_CTX_SIZE(Context)) {
          if (EBML_CTX_PARENT(Context) != nullptr) {
            EbmlElement * Result2 = Result;
            Result = SkipData(DataStream, *EBML_CTX_PARENT(Context), Result);
          } else {
            assert(Context.GetGlobalContext != nullptr);
            if (Context != Context.GetGlobalContext()) {
              Result = SkipData(DataStream, Context.GetGlobalContext(), Result);
            } else {
              bEndFound = true;
            }
          }
        }
      } else {
        bEndFound = true;
      }
    }
  }
  // Result created but overwritten.
  if (Result2 != nullptr && Result2 != Result) delete Result2;
  return Result;
}

KarlStraussberger avatar Feb 25 '22 09:02 KarlStraussberger

No longer crashes. But does not fix the memory leaks either. Output is the same as before.

neheb avatar Feb 25 '22 09:02 neheb

What's your libebml version? Is it actually linked? In current Head 1.4.2 I don't see a direct call to

    #1 0x1015461 in libebml::EbmlVoid::Create() (/home/mangix/devstuff/gerbera/upnp/gerbera+0x1015461)
    #2 0xff5b19 in libebml::EbmlElement::FindNextElement(libebml::IOCallback&, libebml::EbmlSemanticContext const&, int&, unsigned long, bool, unsigned int) (/home/mangix/devstuff/gerbera/upnp/gerbera+0xff5b19)

FindNextElement calls CreateElementUsingContext

KarlStraussberger avatar Feb 25 '22 11:02 KarlStraussberger

release-1.4.2 is what's used. I'm modifying it based on your patches.

neheb avatar Feb 25 '22 14:02 neheb

Sure it get's loaded when running AddressSanitizer?

KarlStraussberger avatar Feb 25 '22 14:02 KarlStraussberger

Absolutely. I have the OS libebml uninstalled.

neheb avatar Feb 25 '22 14:02 neheb

🤔

KarlStraussberger avatar Feb 25 '22 14:02 KarlStraussberger

Create() is probably https://github.com/Matroska-Org/libebml/blob/release-1.4.2/src/EbmlElement.cpp#L215

neheb avatar Feb 25 '22 14:02 neheb

It says libebml::EbmlVoid::Create()

KarlStraussberger avatar Feb 25 '22 14:02 KarlStraussberger

ping @robUx4

There's a memory leak in libebml that is fixed with https://github.com/gerbera/gerbera/pull/2280 . @KarlStraussberger is convinced the actual bug is in libebml. I can't make heads or tails of it. Can you take a look?

neheb avatar Feb 25 '22 23:02 neheb

Update in LibEBML

KarlStraussberger avatar Oct 25 '22 11:10 KarlStraussberger

@KarlStraussberger An alternative would be to use Exiv2 to read matroska files. The current 0.27 version has support for that.

neheb avatar Nov 27 '22 01:11 neheb

There are also newer versions of ebml and matroska which also say they fixed leaks.

KarlStraussberger avatar Nov 29 '22 09:11 KarlStraussberger