separate lzx header

[tansy]

Would it be too much to ask for separate lzx header that would allow to use lzx compressor outside of wim container? Right now, it is deeply embedded in the libwim and there is no good way to use it by itself.

So it would look, something like this:

$ cat lzx1.c

#include "lzx_compress.h"

int main()
    {
    char input[] = "Which witch is which? Which witch is which?";
    char in_buf[100] = {0}, out_buf[100] = {0};
    int in_size, out_size, result, level = 6;

    struct lzx_compressor *compressor;

    compressor = lzx_alloc_compressor(level);

    result = lzx_compress(compressor, in_buf, in_size, out_buf, out_size);

    lzx_free_compressor(compressor);
    return result;
    }

$ cc -Ilibwim/include -o lzx1 lzx1.c -lwim
$ ./lzx1

It would still require libwim but it was 'separate', and somewhat independent. This easy user could use the compressor on their own, like in libdeflate.

[ebiggers]

Don't the compression functions in libwim already do this? This is what they are for. See: https://wimlib.net/apidoc/group__G__compression.html

These functions are already used by wimlib internally when appropriate for reading and writing WIM archives. But they are exported and documented so that they can be used in other applications or libraries for general-purpose lossless data compression. They are implemented in highly optimized C code, using state-of-the-art compression techniques. The main limitation is the lack of sliding window support; this has, however, allowed the algorithms to be optimized for block-based compression.

[tansy]

Don't the compression functions in libwim already do this?

Do they?

$ cat lzxa2.c
#include <stdio.h>
#include <string.h>
#include "wimlib.h"

int main()
    {
    char input[] = "Which witch is which? Which witch is which?";
    char in_buf[100] = {0}, out_buf[100] = {0};
    int in_size, out_size, result, level = 6;

    struct lzx_compressor *compressor;

    result = lzx_create_compressor(sizeof(out_buf), level, 0, compressor);

    result = lzx_compress(compressor, in_buf, in_size, out_buf, out_size);

    lzx_free_compressor(compressor);
    return result;
    }

$ gcc -O2 -I../include -I../include/wimlib -c -o lzxa2.o lzxa2.c
lzxa2.c: In function ‘main’:
lzxa2.c:13:14: warning: implicit declaration of function ‘lzx_create_compressor’ [-Wimplicit-function-declaration]
     result = lzx_create_compressor(sizeof(out_buf), level, 0, compressor);
              ^~~~~~~~~~~~~~~~~~~~~
lzxa2.c:15:14: warning: implicit declaration of function ‘lzx_compress’ [-Wimplicit-function-declaration]
     result = lzx_compress(compressor, in_buf, in_size, out_buf, out_size);
              ^~~~~~~~~~~~
lzxa2.c:17:5: warning: implicit declaration of function ‘lzx_free_compressor’ [-Wimplicit-function-declaration]
     lzx_free_compressor(compressor);
     ^~~~~~~~~~~~~~~~~~~

$ gcc  -o lzxa2 lzxa2.o -L. -lwim
lzxa2.o: In function `main':
lzxa2.c:(.text.startup+0x39): undefined reference to `lzx_create_compressor'
lzxa2.c:(.text.startup+0x53): undefined reference to `lzx_compress'
lzxa2.c:(.text.startup+0x5e): undefined reference to `lzx_free_compressor'
collect2: error: ld returned 1 exit status
make: *** [lzxa2] Error 1

these function are not in any header

$ grep -r -nH lzx_compress include
include/wimlib/compressor_ops.h:32:extern const struct compressor_ops lzx_compressor_ops;

nor they are accessible in library, even though they are compiled in library.

$ nm libwim.a | grep lzx_create_compressor
00006710 t lzx_create_compressor
$ nm libwim.a | grep lzx_compress
         U lzx_compressor_ops
libwim_la-lzx_compress.o:
00006940 t lzx_compress
00004420 t lzx_compress_lazy_16
00005540 t lzx_compress_lazy_32
00001680 t lzx_compress_near_optimal_16
00002d30 t lzx_compress_near_optimal_32
00000000 R lzx_compressor_ops
$ nm libwim.a | grep lzx_free_compressor
00000550 t lzx_free_compressor

It is explained in first sentence of your quote:

These functions are already used by wimlib internally

There is no access to them.

[ebiggers]

You need to use the wimlib_* API functions, not internal functions that you found in the source. Please see the API documentation I linked to -- this is what it is for.

[tansy]

Got to the point where it compiles

$ cat lzxa3.c

#include <stdio.h>
#include <string.h>
#include "wimlib.h"

int main()
    {
    char input[] = "Which witch is which? Which witch is which?";
    char in_buf[100] = {0}, out_buf[100] = {0};
    int in_size, out_size, result, level = 6;

    struct wimlib_compressor *compressor;

    result = wimlib_create_compressor(WIMLIB_COMPRESSION_TYPE_LZX, sizeof(out_buf), level, &compressor);

    result = wimlib_compress(input, strlen(input), out_buf, sizeof(out_size), compressor);
    printf("result=%ld\n", result);

    wimlib_free_compressor(compressor);
    return result;
    }

$ cc -O2 -I../include -I../include/wimlib -c -o lzxa3.o lzxa3.c
$ cc  -o lzxa3 lzxa3.o -L. -lwim -lpthread

but cannot link it due to bunch of undefined references to `ntfs-3g_capture.c' (y0v9Cd7m). What it has to do with compression?

[ebiggers]

Well, it is still part of libwim, so you have to use the correct flags for linking to libwim. For static linking which you seem to want, use pkg-config --libs --static wimlib to get the flags. You can also reduce the dependencies of libwim by using : ./configure --without-fuse --without-ntfs-3g when building it.

[tansy]

./configure (...) --without-ntfs-3

That's the one I was missing.

Still now get 0 result.