Upgrade guide from v3.0 to v4.0

It seems that v4.0 is now part of master, which breaks my repository that uses this library as a submodule for those people that do a fresh checkout with up-to-date submodules.

I've been trying to fix this, and replaced deprecated functions but I'm failing to include libktx using cmake the way it worked with v3.0.

Is there an a guide somewhere that shows the biggest changes from v3.0 to v4.0?

There is a v3.0.1 tag for the last v3 release. Can you point the submodule at that?

No there is no guide yet. We are in the middle of migrating the build system to cmake. Until that is finished we aren't ready to update BUILDING.md. The main differences when linking is that OpenGL {,ES} is no longer a dependency of libktx and on Linux, you need to -lpthread and -ldl. The equivalent functions on macOS & Windows don't need explicit link libraries.

I also have not yet had time to write a guide to the API changes. The Introduction to the libktx reference manual gives a concise summary of using the API. For reading KTX version 1 files the API is substantially unchanged. For writing you have to be aware of the ktxTexture1 class. There are many new API functions for KTX version 2 files including zstd deflation & inflation and transcoding of UASTC and BasisLZ/ETC1S universal texture formats.

If you have specific questions, I'm happy to answer them.

Thanks for the quick reply :)

There is a v3.0.1 tag for the last v3 release. Can you point the submodule at that?

Just did that to make sure people can do a clean checkout and are able to compile my samples.

My biggest problem with v4.0 is that I'm getting several compiler warning for inconsistent DLL bindings (MSVC) and an unreferenced symbol for ktxTexture_CreateFromNamedFile which worked fine with 3.0. Will wait until migration is finished and try to do a proper update to v4.0 then.

Any news on this. We want to update to 4.0 for the Khronos Vulkan Samples repo, and I'm having a hard time updating from 3.x to 4.x. The linker can't find any of the ktxTexture functions, even though I have included texture1. I already spent dozens of hours with no success, and I'm not sure if it's even possible to include 4.x via cmake the way we've been doing it with 3.x.

An easy guide on how to update projects that include KTX-Software via cmake from 3.x to 4.x would be very much appreciated.

I think the inconsistent DLL bindings are because some of the pre-compiled libraries used by the loadtests are release only. These warnings come when you compile the debug version. However I've just noticed they do not appear in the CI builds only when I build locally. I will investigate further.

Are you using the latest ktx.h header file? Many, but not all, of the ktxTexture_* function are now macros that call through function pointers to simulate virtual functions. However ktxTexture_CreateFromNamedFile is not one of those. I do not know why you are getting an unreferenced symbol. What do you mean "I have included texture1?" If you mean you included texture1.h You should not. Most of the functions in that file are not exported to the dll as they are the targets of the function pointers I just mentioned. You should include only ktx.h.

Thanks for your response. I have updated the KTX-Software submodule locally to the latest 4.0.0 release, and the ktx.h matches the one from this repo (copyright 2010-2018).

Adding texture1.h to CMAKE was just a test because I ran out of ideas. I have reverted that but I still get undefined references.

This is how we include KTX-Software into our repo: https://github.com/KhronosGroup/Vulkan-Samples/blob/master/third_party/CMakeLists.txt#L79

It works fine for 3.x, but I haven't been able to make this work with 4.x. After updating the submodule I get unresolved external symbols for e.g. __imp_ktxTexture_CreateFromMemory.

I guess we need to add different files, but I haven't found out which ones.

A documentation on how to include 4.x with CMAKE in a project would be very much appreciated :)

It works fine for 3.x, but I haven't been able to make this work with 4.x. After updating the submodule I get unresolved external symbols for e.g. __imp_ktxTexture_CreateFromMemory

Is that the only unresolved symbol? Do you by any chance somehow have KHRONOS_STATIC defined when compiling libktx or do you have KTX_FEATURE_STATIC_LIBRARY defined in your CMake configuration. Either of these would prevent __declspec(dllexport) from being defined for any of the library functions and hence any import stubs (__imp*) being defined in the .lib. That looks to be the case here but then I would expect there to be more than one unresolved symbol unless ktxTexture_CreateFromMemory is the only one you are using.

A documentation on how to include 4.x with CMAKE in a project would be very much appreciated :)

I've asked for help with this over in issue #326.

Is that the only unresolved symbol?

I get unresolved symbols for all ktxTexture_ functions we are using.

Do you by any chance somehow have KHRONOS_STATIC defined when compiling libktx or do you have KTX_FEATURE_STATIC_LIBRARY defined in your CMake configuration.

We don't have any KTX CMAKE flags defined in our setup. Looking at the code, none of those static defines are set:

This is how we include KTX-Software into our repo: https://github.com/KhronosGroup/Vulkan-Samples/blob/master/third_party/CMakeLists.txt#L79

This is missing most of the library source files. I suggest you either use the pre-compiled library from our Releases section or the KTX CMakeLists.txt to configure the build. I don't know CMake well enough to know for sure but I think that using the KTX CMakeLists is what issue #326 is about. I hope we get a reply soon.

When we get this figured out, you might prefer to use the ktx_read target rather than the ktx target. The former supports only the read and GPU upload functions, no writers.

Tried porting over the CMakeLists.txt to our own but to no avail. Same errors :(

The pre-compiled library may be an option, but it looks like there is no pre-compiled library for android, which is an important target for our samples.

One thing I noticed though: When compiling only the KTX part of our third_party library, I'm getting lots of messages about inconsistent dll linkage:

2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\lib\texture.c(41,52): warning C4273: 'ktxTexture_GetDataSize': inconsistent dll linkage
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\include\ktx.h(805,1): message : see previous definition of 'ktxTexture_GetDataSize'
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\lib\texture.c(367,1): warning C4273: 'ktxTexture_CreateFromStdioStream': inconsistent dll linkage
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\include\ktx.h(768,1): message : see previous definition of 'ktxTexture_CreateFromStdioStream'
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\lib\texture.c(415,1): warning C4273: 'ktxTexture_CreateFromNamedFile': inconsistent dll linkage
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\include\ktx.h(773,1): message : see previous definition of 'ktxTexture_CreateFromNamedFile'
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\lib\texture.c(468,1): warning C4273: 'ktxTexture_CreateFromMemory': inconsistent dll linkage
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\include\ktx.h(778,1): message : see previous definition of 'ktxTexture_CreateFromMemory'
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\lib\texture.c(491,1): warning C4273: 'ktxTexture_GetData': inconsistent dll linkage
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\include\ktx.h(786,1): message : see previous definition of 'ktxTexture_GetData'
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\lib\texture.c(525,1): warning C4273: 'ktxTexture_GetElementSize': inconsistent dll linkage
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\include\ktx.h(799,1): message : see previous definition of 'ktxTexture_GetElementSize'
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\lib\texture.c(619,1): warning C4273: 'ktxTexture_IterateLevelFaces': inconsistent dll linkage
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\include\ktx.h(816,1): message : see previous definition of 'ktxTexture_IterateLevelFaces'
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\lib\texture.c(871,2): warning C4273: 'ktxTexture_GetRowPitch': inconsistent dll linkage
2>V:\Vulkan-KHR-Samples\Vulkan-Samples-Fork\third_party\ktx\include\ktx.h(793,1): message : see previous definition of 'ktxTexture_GetRowPitch'
...

Those sound like they could be the cause for the problem. Can these be caused by a missing CMake setting? I'll try to investigate later.

PR #335 will add Android build support. The person behind it is on paternity leave right now so I do not know when it will be completed.

What VC++ warning level are you using? We used to have /Wall with the GYP-based build but I've just discovered that CMake-based build is only setting /W1. I know I had to reduce the level at one point due to issues compiling the Basis code but not down to /W1! With /W1 I'm not seeing these warnings.

The declarations in ktx.h all have macros that by default add __declspec(dllimport) while the definitions in the texture.c do not. I'm pretty sure this is standard practice but could be wrong.

Even with /W4 I do not get the inconsistent dll linkage warnings when building the tools and tests included in KTX-Software using its CMake build. I am mystified as to what is going on at your end.

I have realized what is going on. When you compile the library you need to define KTX_API=__declspec(dllexport). If that is not specified KTX_API is defined as __declspec(dllimport) so nothing is exported from the dll. It also causes the inconsistent linkage warnings because the function definitions in texture.c are inconsistent with the dllimport declarations in ktx.h.

You can build a static library instead of a dll by defining KHRONOS_STATIC both when compiling the library and when compiling the applications.

There are a few other cpp macros you'll need to define as well. If you want the library to include writing:

KTX_FEATURE_WRITE=1
BASISD_SUPPORT_KTX2_ZSTD=0
BASISD_SUPPORT_KTX2=1
BASISD_SUPPORT_FXT1=0

If you want the Basis Universal encoder to use SSE instructions for better performance also define

BASISU_SUPPORT_SSE=1

There are more source files you'll need to include too.

If you are building the read-only library then:

KTX_FEATURE_WRITE=0
BASISD_SUPPORT_KTX2_ZSTD=0
BASISD_SUPPORT_KTX2=0
BASISD_SUPPORT_FXT1=0

Did KTX_API=__declspec(dllexport) fix your problem?

Gave this another spin, and after fiddling with some pre-processor defines and the list of source files to include I finally got it working. Loading KTX1 and KTX2 files works now :)

I'll do some clean-up and try to get the transcoding working too and will write something up on how to include this into one's project via CMAKE.

I'll do some clean-up and try to get the transcoding working too and will write something up on how to include this into one's project via CMAKE.

Sounds great.

By the way KTX v2 has tightened up the spec for cubemaps. We define a cubemap coordinate system compatible with the OpenGL & Vulkan cube samplers. Faces must be laid out according to that coordinate system. There's an explanation in an appendix of the spec. I mention this because the cubemaps in some (all?) of your samples would not be compliant if converted to ktx2 via ktx2ktx2. It is not possible for the tool to know that you have¹ yflipped all the faces and swapped the +Y and -Y faces, which is what I saw in one of your examples.

I plan to mention this in the v3.0 to v4.0 guide though it is a spec. change not an API change.

¹ Somebody has

I'm now at a point where:

• It's properly included into the CMake build for the Khronos samples
• It builds on all platforms (Windows, Linux, Android, MacOS) with different compilers
• Loading ktx1, ktx2 and transcoding from basis universal works fine

To get it working on all platforms with different compilers I had to further fiddle around with the pre processor defines (e.g. having to define BASISU_NO_ITERATOR_DEBUG_LEVEL for using the transcoder with msvc and our repo).

So if you're okay with that, I'd add a page to the wiki with a small guide on how to include libktx into a project via CMake and what has to be defined in order to use the different components like the transcoder. Or do you have a different preferred way of documenting something like this?

@SaschaWillems I am very happy you've got this working. Sorry for the delay to our Android build. The PR is working now but landing it is backed up behind another PR.

I've been debating whether to document this in BUILDING.md or the wiki. I think the wiki is better because BUILDING.md is already quite long and its focus is building of the repo with the repo's CMake files. Why did you choose not to use our CMake files? It would be could to explain the reasons in the wiki too.

Why did you choose not to use our CMake files? It would be could to explain the reasons in the wiki too.

tbh there is no real reason. We just put all third party libraries in a single CMakeLists.txt to keep things simple. I'll take a look if it's possible to use the CMakeLists.txt provided by this repo instead.