redefinition of ‘struct extent_hooks_s’

[ftiasch]

Building on Archlinux gives me the following error:

Building CXX object CMakeFiles/folly_base.dir/folly/experimental/JemallocHugePageAllocator.cpp.o
/home/ftiasch/Documents/folly/src/folly-2018.09.24.00/folly/experimental/JemallocHugePageAllocator.cpp:52:8: error: redefinition of ‘struct extent_hooks_s’
 struct extent_hooks_s {
        ^~~~~~~~~~~~~~
In file included from /home/ftiasch/Documents/folly/src/folly-2018.09.24.00/folly/memory/Malloc.h:32,
                 from /home/ftiasch/Documents/folly/src/folly-2018.09.24.00/folly/experimental/JemallocHugePageAllocator.h:22,
                 from /home/ftiasch/Documents/folly/src/folly-2018.09.24.00/folly/experimental/JemallocHugePageAllocator.cpp:17:
/usr/include/jemalloc/jemalloc.h:339:8: note: previous definition of ‘struct extent_hooks_s’
 struct extent_hooks_s {
        ^~~~~~~~~~~~~~

Is the jemalloc installed too new?

[detective7th]

I have exactly the same problem too. os: Manjaro (Archlinux) jemalloc version: 5.1

[gdankel]

The intention is that for jemalloc version > 5, that definition should not be compiled (see folly/experimental/JemallocHugePageAllocator.cpp:27). Can you check what JEMALLOC_VERSION_MAJOR reports?

[ftiasch]

@gdankel my JEMALLOC_VERSION_MAJOR is 5. Checked with the following code:

#include <cstdio>
#include <jemalloc/jemalloc.h>

int main()
{
    printf("%d\n", JEMALLOC_VERSION_MAJOR);
}

It prints "5".

[gdankel]

@ftiasch Thanks! Could you also please try commenting out line 26 and see if anything changes:

//#include <jemalloc/jemalloc.h>

It's likely the case that jemalloc.h is getting included via one of the other headers.

[ftiasch]

@gdankel it still presents the same error.

[gdankel]

OK, you can try this patch to JemallocHugePageAllocator.cpp:

@@ -40,6 +40,8 @@
 #define MALLOCX_ARENA(x) 0
 #define MALLOCX_TCACHE_NONE 0
 #define MADV_HUGEPAGE 0
+
+#ifndef JEMALLOC_VERSION_MAJOR
 typedef struct extent_hooks_s extent_hooks_t;
 typedef void*(extent_alloc_t)(
     extent_hooks_t*,
@@ -52,6 +54,8 @@
 struct extent_hooks_s {
   extent_alloc_t* alloc;
 };
+#endif // JEMALLOC_VERSION_MAJOR
+
 bool folly::JemallocHugePageAllocator::hugePagesSupported{false};
 #endif // FOLLY_JEMALLOC_HUGE_PAGE_ALLOCATOR_SUPPORTED

[ftiasch]

it works!

[gdankel]

I'll submit a proper patch - there might be an issue with jemalloc version < 5 as well if the header is included via another header.

By the way, just to make it clear (if you're trying to use this allocator) - with this part of the code compiled if won't actually back the memory by huge pages.

[gdankel]

There seems to be an issue with the FOLLY_HAVE_LIBJEMALLOC as well. You can try to replace that with FOLLY_USE_JEMALLOC (in the same file). This is probably the better fix in your case.

[Orvid]

We're working on this internally :)

[mechanyx]

Building folly in an Ubuntu 18.04 docker container with jemalloc from head is giving me this same error. I was able to resolve it with the patch here on this issue. Is there a reason the patch can't be merged? Thanks

[gdankel]

We did merge a different patch that seems to have worked for most folks. I will also merge the above.

[mechanyx]

Great, thank you. I'm currently applying the patch after checking out folly in our build script. I imagine there are others who will also benefit from having this merged. Thanks!

[mechanyx]

Sorry, I meant to follow up here. It appears that part of the problem is that jemalloc isn't setting its version when building from source. The configure script checks for a file called VERSION in the root of its checkout and, in my instance, that file was not there. I needed to create such a file with an appropriate version number inside to get this to work. Thanks

[dutor]

This happens if jemalloc was built from source without the --with-version option set properly.