Build error with VS 2022, and zlib compression enabled

[jzakrzewski]

Describe the bug

Build failed

Expected Behavior

Successful build

Current Behavior

FAILED: _deps/aws-sdk-build/src/aws-cpp-sdk-core/CMakeFiles/aws-cpp-sdk-core.dir/ub_core.cpp.obj 

C:\PROGRA~1\MIB055~1\2022\ENTERP~1\VC\Tools\MSVC\1435~1.322\bin\Hostx64\x64\cl.exe  /nologo /TP -DAWS_AUTH_USE_IMPORT_EXPORT -DAWS_CAL_USE_IMPORT_EXPORT -DAWS_CHECKSUMS_USE_IMPORT_EXPORT -DAWS_COMMON_USE_IMPORT_EXPORT -DAWS_COMPRESSION_USE_IMPORT_EXPORT -DAWS_CORE_EXPORTS=1 -DAWS_CRT_CPP_USE_IMPORT_EXPORT -DAWS_EVENT_STREAM_USE_IMPORT_EXPORT -DAWS_HTTP_USE_IMPORT_EXPORT -DAWS_IO_USE_IMPORT_EXPORT -DAWS_MQTT_USE_IMPORT_EXPORT -DAWS_MQTT_WITH_WEBSOCKETS -DAWS_S3_USE_IMPORT_EXPORT -DAWS_SDKUTILS_USE_IMPORT_EXPORT -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=11 -DAWS_SDK_VERSION_PATCH=48 -DAWS_TEST_REGION=US_EAST_1 -DAWS_USE_IO_COMPLETION_PORTS -DBOOST_ALL_NO_LIB -DCONSTEXPR=constexpr -DENABLED_REQUEST_COMPRESSION -DENABLED_ZLIB_REQUEST_COMPRESSION -DENABLE_BCRYPT_ENCRYPTION -DENABLE_CURL_LOGGING -DENABLE_WINDOWS_CLIENT -DFORCE_USE_RESULT="" -DJAVA_SCRIPT_DEBUG -DNOEXCEPT=noexcept -DPLATFORM_WINDOWS -DUSE_IMPORT_EXPORT=1 -DUSE_WINDOWS_DLL_SEMANTICS -DWINHTTP_HAS_H2 -DWININET_HAS_H2 -D_CRTDBG_MAP_ALLOC -D_HAS_AUTO_PTR_ETC=1 -Daws_cpp_sdk_core_EXPORTS -IC:\build -IC:\build\external\include -IC:\build\_deps\aws-sdk-src\src\aws-cpp-sdk-core\include\aws\core\platform\refs -IC:\build\_deps\aws-sdk-src\src\aws-cpp-sdk-core\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-http\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-io\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-common\include -IC:\build\_deps\aws-sdk-build\crt\aws-crt-cpp\crt\aws-c-common\generated\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-cal\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-compression\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-mqtt\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-auth\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-sdkutils\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-checksums\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-event-stream\include -IC:\build\_deps\aws-sdk-src\crt\aws-crt-cpp\crt\aws-c-s3\include -external:W0 /DWIN32 /D_WINDOWS /EHsc /MP /bigobj /utf-8 /WX /W4 /Zi /Ob0 /Od /RTC1 -MDd -FS -std:c++20 /showIncludes /Fo_deps\aws-sdk-build\src\aws-cpp-sdk-core\CMakeFiles\aws-cpp-sdk-core.dir\ub_core.cpp.obj /Fd_deps\aws-sdk-build\src\aws-cpp-sdk-core\CMakeFiles\aws-cpp-sdk-core.dir\ /FS -c C:\build\_deps\aws-sdk-build\src\aws-cpp-sdk-core\ub_core.cpp

C:/build/_deps/aws-sdk-src/src/aws-cpp-sdk-core/source/client/RequestCompression.cpp(159): error C2220: the following warning is treated as an error

C:/build/_deps/aws-sdk-src/src/aws-cpp-sdk-core/source/client/RequestCompression.cpp(159): warning C4267: '=': conversion from 'size_t' to 'uInt', possible loss of data

C:/build/_deps/aws-sdk-src/src/aws-cpp-sdk-core/source/client/RequestCompression.cpp(280): warning C4267: '=': conversion from 'size_t' to 'uInt', possible loss of data

Reproduction Steps

Build with Visual Studio 2022, and zlib compression enabled.

Possible Solution

In src/aws-cpp-sdk-core/source/client/RequestCompression.cpp:

line 159: strm.avail_in = static_cast<uInt>((flush == Z_FINISH)?toRead-1:toRead); //skip EOF if included

line 208: strm.avail_in = static_cast<uInt>(toRead);

Additional Information/Context

No response

AWS CPP SDK version used

1.11.48

Compiler and Version used

VS 2022

Operating System and version

Windows 10

[jmklix]

What build commands are you using to repro this?

[jzakrzewski]

Originally I used it as a submodule (via FetchContent), but I have just reproduced it stand-alone. This is the command line:

cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCPP_STANDARD=20 -DBUILD_ONLY=s3 -DENABLE_TESTING=OFF -DBUILD_TESTING=OFF -DAUTORUN_UNIT_TESTS=OFF -DENABLE_UNITY_BUILD=ON -DFORCE_CURL=ON -DUSE_OPENSSL=ON -DENABLE_ZLIB_REQUEST_COMPRESSION=ON -DCMAKE_PREFIX_PATH:PATH=C:\tmp\libs\debug -DCMAKE_INSTALL_PREFIX:PATH=C:\tmp\aws-install C:\tmp\aws-sdk-cpp-1.11.48

and the build:

c:\tmp\aws-build>ninja
[293/314] Building CXX object src\aws-cpp-sdk-core\CMakeFiles\aws-cpp-sdk-core.dir\ub_core.cpp.obj
FAILED: src/aws-cpp-sdk-core/CMakeFiles/aws-cpp-sdk-core.dir/ub_core.cpp.obj
C:\PROGRA~1\MIB055~1\2022\ENTERP~1\VC\Tools\MSVC\1435~1.322\bin\Hostx64\x64\cl.exe /nologo /TP -DAWS_AUTH_USE_IMPORT_EXPORT -DAWS_CAL_USE_IMPORT_EXPORT -DAWS_CHECKSUMS_USE_IMPORT_EXPORT -DAWS_COMMON_USE_IMPORT_EXPORT -DAWS_COMPRESSION_USE_IMPORT_EXPORT -DAWS_CORE_EXPORTS=1 -DAWS_CRT_CPP_USE_IMPORT_EXPORT -DAWS_EVENT_STREAM_USE_IMPORT_EXPORT -DAWS_HTTP_USE_IMPORT_EXPORT -DAWS_IO_USE_IMPORT_EXPORT -DAWS_MQTT_USE_IMPORT_EXPORT -DAWS_MQTT_WITH_WEBSOCKETS -DAWS_S3_USE_IMPORT_EXPORT -DAWS_SDKUTILS_USE_IMPORT_EXPORT -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=11 -DAWS_SDK_VERSION_PATCH=48 -DAWS_TEST_REGION=US_EAST_1 -DAWS_USE_IO_COMPLETION_PORTS -DENABLED_REQUEST_COMPRESSION -DENABLED_ZLIB_REQUEST_COMPRESSION -DENABLE_BCRYPT_ENCRYPTION -DENABLE_CURL_CLIENT -DENABLE_CURL_LOGGING -DPLATFORM_WINDOWS -DUSE_IMPORT_EXPORT=1 -DUSE_WINDOWS_DLL_SEMANTICS -Daws_cpp_sdk_core_EXPORTS -IC:\tmp\aws-sdk-cpp-1.11.48\src\aws-cpp-sdk-core\include\aws\core\platform\refs -IC:\tmp\aws-sdk-cpp-1.11.48\src\aws-cpp-sdk-core\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-http\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-io\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-common\include -IC:\tmp\aws-build\crt\aws-crt-cpp\crt\aws-c-common\generated\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-cal\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-compression\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-mqtt\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-auth\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-sdkutils\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-checksums\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-event-stream\include -IC:\tmp\aws-sdk-cpp-1.11.48\crt\aws-crt-cpp\crt\aws-c-s3\include -external:IC:\tmp\libs\debug\include -external:W0 /DWIN32 /D_WINDOWS /W4 /GR /EHsc /MP /bigobj /utf-8 /WX /MDd /Zi /Ob0 /Od /RTC1 /showIncludes /Fosrc\aws-cpp-sdk-core\CMakeFiles\aws-cpp-sdk-core.dir\ub_core.cpp.obj /Fdsrc\aws-cpp-sdk-core\CMakeFiles\aws-cpp-sdk-core.dir\ /FS -c C:\tmp\aws-build\src\aws-cpp-sdk-core\ub_core.cpp
C:/tmp/aws-sdk-cpp-1.11.48/src/aws-cpp-sdk-core/source/client/RequestCompression.cpp(159): error C2220: the following warning is treated as an error
C:/tmp/aws-sdk-cpp-1.11.48/src/aws-cpp-sdk-core/source/client/RequestCompression.cpp(159): warning C4267: '=': conversion from 'size_t' to 'uInt', possible loss of data
C:/tmp/aws-sdk-cpp-1.11.48/src/aws-cpp-sdk-core/source/client/RequestCompression.cpp(280): warning C4267: '=': conversion from 'size_t' to 'uInt', possible loss of data
[310/314] Building CXX object generated\src\aws-cpp-sdk-s3\CMakeFiles\aws-cpp-sdk-s3.dir\ub_S3.cpp.obj
ninja: build stopped: subcommand failed.

[jzakrzewski]

Any news?

[radwaverf]

I reproduced this error as well, but the fix may also require changing

static const size_t ZLIB_CHUNK=263144;

to

static const size_t ZLIB_CHUNK=65536;

because uInt could only be 16 bits on some systems. My system has 32 bit unsigned int though, so I don't know how to test that case. Any ideas?

[jzakrzewski]

Hmmm - why does ZLIB_CHUNK have to be a size_t anyway? Maybe it should just be uInt? Maybe even initialized like: static const auto ZLIB_CHUNK=std::numeric_limits<uInt>::max();?

[radwaverf]

This is a bit beyond my knowledge of zlib, but maybe there are performance implications with ZLIB_CHUNK sizes? It seems like the only reason it's size_t is to match the type of streamSize, which could exceed 4 GB. So whether ZLIB_CHUNK is uInt or size_t, I think static_cast would be needed.

Also, 263144 isn't actually 256k. 256k is 262144. Is this a typo? https://github.com/aws/aws-sdk-cpp/blob/main/src/aws-cpp-sdk-core/source/client/RequestCompression.cpp#L22-L23 Or is that intentional so that when it rolls over with 16 bit systems it still a non-zero value?

[AndyHUI711]

I got the same issue, but how to slove this

[jzakrzewski]

For now, I just patch the source (see the "Possible solution" section in issue description). I guess one could also build without compression support (for me it's a no-go).

Those are only workarounds. The actual solution must be provided by the authors. It makes me wonder, how they build it...

[RochaStratovan]

Almost a year old? Any official fixes for this yet?