aws-sdk-cpp [Bug]: TransferManager S3 Upload - "Http request to retrieve credentials failed"

[Bug]: TransferManager S3 Upload - "Http request to retrieve credentials failed"

Open HyunWinter opened this issue 3 years ago • 0 comments

Describe the bug

I'm on Windows 11 and Qt6 with cmake SDK build. My goal is to upload large media files (10~15 GB) to an AWS S3 bucket. Currently trying two approaches using PutObjectAsync and TransferManager. This issue causes TransferManager fail to create an instance and gives the following errors:

Http request to retrieve credentials failed
Failed to add HTTP request headers

I am using the same client configs and AWS credentials for both approaches, and PutObjectAsync works fine. I found another ticket with the same error, but it was closed due to insufficient information. Thank you so much ahead.

Expected Behavior

The upload handler from the TransferManager creates a correct connection to a S3 bucket and passes a file.

Current Behavior

TransferManager fails to create an instance then the app crashes. Here is the trace log:

[INFO] 2022-10-14 23:46:48.739 Aws_Init_Cleanup [337940] Initiate AWS SDK for C++ with Version:1.9.362
[TRACE] 2022-10-14 23:46:48.740 FileSystem [337940] Checking USERPROFILE for the home directory.
[DEBUG] 2022-10-14 23:46:48.740 FileSystem [337940] Environment value for variable USERPROFILE is C:\Users\<username>
[INFO] 2022-10-14 23:46:48.740 Aws::Config::AWSConfigFileProfileConfigLoader [337940] Initializing config loader against fileName C:\Users\<username>\.aws\credentials and using profilePrefix = 0
[TRACE] 2022-10-14 23:46:48.740 FileSystem [337940] Checking USERPROFILE for the home directory.
[DEBUG] 2022-10-14 23:46:48.740 FileSystem [337940] Environment value for variable USERPROFILE is C:\Users\<username>
[INFO] 2022-10-14 23:46:48.740 Aws::Config::AWSConfigFileProfileConfigLoader [337940] Initializing config loader against fileName C:\Users\<username>\.aws\config and using profilePrefix = 1
[TRACE] 2022-10-14 23:46:48.740 FileSystem [337940] Checking USERPROFILE for the home directory.
[DEBUG] 2022-10-14 23:46:48.740 FileSystem [337940] Environment value for variable USERPROFILE is C:\Users\<username>
[INFO] 2022-10-14 23:46:48.741 Aws::Config::AWSConfigFileProfileConfigLoader [337940] Unable to open config file C:\Users\<username>\.aws\credentials for reading.
[INFO] 2022-10-14 23:46:48.741 Aws::Config::AWSProfileConfigLoaderBase [337940] Failed to reload configuration.
[TRACE] 2022-10-14 23:46:48.741 FileSystem [337940] Checking USERPROFILE for the home directory.
[DEBUG] 2022-10-14 23:46:48.741 FileSystem [337940] Environment value for variable USERPROFILE is C:\Users\<username>
[INFO] 2022-10-14 23:46:48.741 Aws::Config::AWSConfigFileProfileConfigLoader [337940] Unable to open config file C:\Users\<username>\.aws\config for reading.
[INFO] 2022-10-14 23:46:48.741 Aws::Config::AWSProfileConfigLoaderBase [337940] Failed to reload configuration.
[INFO] 2022-10-14 23:46:48.741 event-loop [337940] id=0000018FCE15F1E0: Initializing IO Completion Port
[INFO] 2022-10-14 23:46:48.741 event-loop [337940] id=0000018FCE15F1E0: Starting event-loop thread.
[INFO] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BAA9F8: cpu affinity of cpu_id 0 was specified, attempting to honor the value.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BAA9F8: computed mask 1 on group 0.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BAA9F8: SetThreadGroupAffinity() result 1.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BAA9F8: SetThreadIdealProcessorEx() result 1.
[INFO] 2022-10-14 23:46:48.741 event-loop [337940] id=0000018FCE15F320: Initializing IO Completion Port
[INFO] 2022-10-14 23:46:48.741 event-loop [337940] id=0000018FCE15F320: Starting event-loop thread.
[INFO] 2022-10-14 23:46:48.741 event-loop [331680] id=0000018FCE15F1E0: main loop started
[INFO] 2022-10-14 23:46:48.741 event-loop [331680] id=0000018FCE15F1E0: default timeout 100000
[INFO] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BA8D38: cpu affinity of cpu_id 0 was specified, attempting to honor the value.
[TRACE] 2022-10-14 23:46:48.741 event-loop [331680] id=0000018FCE15F1E0: waiting for a maximum of 100000 ms
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BA8D38: computed mask 1 on group 0.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BA8D38: SetThreadGroupAffinity() result 1.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BA8D38: SetThreadIdealProcessorEx() result 1.
[INFO] 2022-10-14 23:46:48.741 event-loop [337940] id=0000018FCE160040: Initializing IO Completion Port
[INFO] 2022-10-14 23:46:48.741 event-loop [337940] id=0000018FCE160040: Starting event-loop thread.
[INFO] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BA8E78: cpu affinity of cpu_id 0 was specified, attempting to honor the value.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BA8E78: computed mask 1 on group 0.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BA8E78: SetThreadGroupAffinity() result 1.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BA8E78: SetThreadIdealProcessorEx() result 1.
[INFO] 2022-10-14 23:46:48.741 event-loop [337940] id=0000018FCE15F7D0: Initializing IO Completion Port
[INFO] 2022-10-14 23:46:48.741 event-loop [337940] id=0000018FCE15F7D0: Starting event-loop thread.
[INFO] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BB6498: cpu affinity of cpu_id 0 was specified, attempting to honor the value.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BB6498: computed mask 1 on group 0.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BB6498: SetThreadGroupAffinity() result 1.
[DEBUG] 2022-10-14 23:46:48.741 thread [337940] id=0000018FC9BB6498: SetThreadIdealProcessorEx() result 1.
[INFO] 2022-10-14 23:46:48.741 event-loop [329104] id=0000018FCE15F320: main loop started
[INFO] 2022-10-14 23:46:48.741 event-loop [329104] id=0000018FCE15F320: default timeout 100000
[TRACE] 2022-10-14 23:46:48.741 event-loop [329104] id=0000018FCE15F320: waiting for a maximum of 100000 ms
[INFO] 2022-10-14 23:46:48.741 dns [337940] id=0000018FCE0BC390: Initializing default host resolver with 8 max host entries.
[INFO] 2022-10-14 23:46:48.741 channel-bootstrap [337940] id=0000018FCE1C8360: Initializing client bootstrap with event-loop group 0000018FCE1B7040
[INFO] 2022-10-14 23:46:48.741 event-loop [336100] id=0000018FCE160040: main loop started
[INFO] 2022-10-14 23:46:48.741 event-loop [336100] id=0000018FCE160040: default timeout 100000
[TRACE] 2022-10-14 23:46:48.741 event-loop [336100] id=0000018FCE160040: waiting for a maximum of 100000 ms
[INFO] 2022-10-14 23:46:48.741 event-loop [337484] id=0000018FCE15F7D0: main loop started
[INFO] 2022-10-14 23:46:48.741 event-loop [337484] id=0000018FCE15F7D0: default timeout 100000
[TRACE] 2022-10-14 23:46:48.741 event-loop [337484] id=0000018FCE15F7D0: waiting for a maximum of 100000 ms
[INFO] 2022-10-14 23:46:48.742 EC2MetadataClient [337940] Using IMDS endpoint: http://169.254.169.254
[DEBUG] 2022-10-14 23:46:48.743 ClientConfiguration [337940] ClientConfiguration will use SDK Auto Resolved profile: [default] if not specified by users.
[WARN] 2022-10-14 23:46:48.743 ClientConfiguration [337940] Retry Strategy will use the default max attempts.
[INFO] 2022-10-14 23:46:48.743 EC2MetadataClient [337940] Creating AWSHttpResourceClient with max connections 2 and scheme http
[INFO] 2022-10-14 23:46:48.743 WinHttpSyncHttpClient [337940] Creating http client with user agent  with max connections 2 request timeout 1000,and connect timeout 1000
[DEBUG] 2022-10-14 23:46:48.743 WinHttpHttp2 [337940] HTTP/2 enabled on WinHttp handle: 0000018FCDF098E0.
[DEBUG] 2022-10-14 23:46:48.743 WinHttpSyncHttpClient [337940] API handle 0000018FCDF098E0
[INFO] 2022-10-14 23:46:48.743 ConnectionPoolMgr [337940] Creating connection pool mgr with handle 0000018FCDF098E0, and max connections per host 2, request timeout 1000 ms, and connect timeout in 1000 ms, enabling TCP keep-alive.
[DEBUG] 2022-10-14 23:46:48.743 ClientConfiguration [337940] ClientConfiguration will use SDK Auto Resolved profile: [default] if not specified by users.
[WARN] 2022-10-14 23:46:48.743 ClientConfiguration [337940] Retry Strategy will use the default max attempts.
[TRACE] 2022-10-14 23:46:48.743 EC2MetadataClient [337940] Getting current region for ec2 instance
[TRACE] 2022-10-14 23:46:48.744 EC2MetadataClient [337940] Retrieving credentials from http://169.254.169.254/latest/meta-data/placement/availability-zone
[TRACE] 2022-10-14 23:46:48.744 WinHttpSyncHttpClient [337940] Making GET request to uri http://169.254.169.254/latest/meta-data/placement/availability-zone
[INFO] 2022-10-14 23:46:48.744 WinHttpConnectionPoolMgr [337940] Attempting to acquire connection for 169.254.169.254:80
[DEBUG] 2022-10-14 23:46:48.744 WinHttpConnectionPoolMgr [337940] Pool doesn't exist for endpoint, creating...
[DEBUG] 2022-10-14 23:46:48.744 WinHttpConnectionPoolMgr [337940] Pool has no available existing connections for endpoint, attempting to grow pool.
[INFO] 2022-10-14 23:46:48.744 WinConnectionContainer [337940] Pool grown by 2
[INFO] 2022-10-14 23:46:48.744 WinHttpConnectionPoolMgr [337940] Connection now available, continuing.
[DEBUG] 2022-10-14 23:46:48.744 WinHttpConnectionPoolMgr [337940] Returning connection handle 0000018FCE0F3650
[DEBUG] 2022-10-14 23:46:48.744 WinHttpSyncHttpClient [337940] Acquired connection 0000018FCE0F3650
[WARN] 2022-10-14 23:46:48.744 WinHttpSyncHttpClient [337940] Failed setting TCP keep-alive interval with error code: 12018
[DEBUG] 2022-10-14 23:46:48.744 WinHttpHttp2 [337940] HTTP/2 enabled on WinHttp handle: 0000018FCE1CB0D0.
[DEBUG] 2022-10-14 23:46:48.744 WinHttpSyncHttpClient [337940] AllocateWindowsHttpRequest returned handle 0000018FCE1CB0D0
[DEBUG] 2022-10-14 23:46:48.744 WinHttpSyncHttpClient [337940] with headers:
[DEBUG] 2022-10-14 23:46:48.744 WinHttpSyncHttpClient [337940] host: 169.254.169.254

user-agent: aws-sdk-cpp/1.9.362 Windows/10.0.22000.708 AMD64 MSVC/1916

x-aws-ec2-metadata-token: 


[ERROR] 2022-10-14 23:46:48.744 WinHttpSyncHttpClient [337940] Failed to add HTTP request headers: host: 169.254.169.254

user-agent: aws-sdk-cpp/1.9.362 Windows/10.0.22000.708 AMD64 MSVC/1916

x-aws-ec2-metadata-token: 

, with error code: 12150
[WARN] 2022-10-14 23:46:48.753 WinHttpSyncHttpClient [337940] Send request failed: A connection with the server could not be established


[DEBUG] 2022-10-14 23:46:48.753 WinHttpSyncHttpClient [337940] Closing http request handle 0000018FCE1CB0D0
[DEBUG] 2022-10-14 23:46:48.753 WinHttpSyncHttpClient [337940] Releasing connection handle 0000018FCE0F3650
[DEBUG] 2022-10-14 23:46:48.753 WinHttpConnectionPoolMgr [337940] Releasing connection to endpoint 169.254.169.254:80
[ERROR] 2022-10-14 23:46:48.753 EC2MetadataClient [337940] Http request to retrieve credentials failed
[WARN] 2022-10-14 23:46:48.753 EC2MetadataClient [337940] Request failed, now waiting 0 ms before attempting again.
[TRACE] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] Making GET request to uri http://169.254.169.254/latest/meta-data/placement/availability-zone
[INFO] 2022-10-14 23:46:48.754 WinHttpConnectionPoolMgr [337940] Attempting to acquire connection for 169.254.169.254:80
[DEBUG] 2022-10-14 23:46:48.754 WinHttpConnectionPoolMgr [337940] Pool found, reusing
[INFO] 2022-10-14 23:46:48.754 WinHttpConnectionPoolMgr [337940] Connection now available, continuing.
[DEBUG] 2022-10-14 23:46:48.754 WinHttpConnectionPoolMgr [337940] Returning connection handle 0000018FCE0F3650
[DEBUG] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] Acquired connection 0000018FCE0F3650
[WARN] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] Failed setting TCP keep-alive interval with error code: 12018
[DEBUG] 2022-10-14 23:46:48.754 WinHttpHttp2 [337940] HTTP/2 enabled on WinHttp handle: 0000018FCE23C600.
[DEBUG] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] AllocateWindowsHttpRequest returned handle 0000018FCE23C600
[DEBUG] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] with headers:
[DEBUG] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] host: 169.254.169.254

user-agent: aws-sdk-cpp/1.9.362 Windows/10.0.22000.708 AMD64 MSVC/1916

x-aws-ec2-metadata-token: 


[ERROR] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] Failed to add HTTP request headers: host: 169.254.169.254

user-agent: aws-sdk-cpp/1.9.362 Windows/10.0.22000.708 AMD64 MSVC/1916

x-aws-ec2-metadata-token: 

, with error code: 12150
[WARN] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] Send request failed: A connection with the server could not be established


[DEBUG] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] Closing http request handle 0000018FCE23C600
[DEBUG] 2022-10-14 23:46:48.754 WinHttpSyncHttpClient [337940] Releasing connection handle 0000018FCE0F3650
[DEBUG] 2022-10-14 23:46:48.754 WinHttpConnectionPoolMgr [337940] Releasing connection to endpoint 169.254.169.254:80
[ERROR] 2022-10-14 23:46:48.754 EC2MetadataClient [337940] Http request to retrieve credentials failed
[ERROR] 2022-10-14 23:46:48.754 EC2MetadataClient [337940] Can not retrieve resource from http://169.254.169.254/latest/meta-data/placement/availability-zone
[INFO] 2022-10-14 23:46:48.754 EC2MetadataClient [337940] Unable to pull region from instance metadata service

Reproduction Steps

I don't think this is a build issue, but this is how I built the SDK and added to Qt.

cd C:\dev\sdk_build
cmake.exe "C:/dev/aws-sdk-cpp" -DCMAKE_BUILD_TYPE=Debug -DBUILD_ONLY="core;s3;transfer"
MSBuild.exe ALL_BUILD.vcxproj -p:Configuration=Debug
MSBuild.exe INSTALL.vcxproj -p:Configuration=Debug

INCLUDEPATH += $$PWD/aws-cpp-sdk-all/include
DEPENDPATH += $$PWD/aws-cpp-sdk-all/include

LIBS += \
    -L$$PWD/aws-cpp-sdk-all/bin/ -laws-cpp-sdk-core \
    -L$$PWD/aws-cpp-sdk-all/bin/ -laws-cpp-sdk-s3 \
    -L$$PWD/aws-cpp-sdk-all/bin/ -laws-cpp-sdk-transfer

DLLPATH_WIN = $$PWD\aws-cpp-sdk-all\bin\*.dll

Following the example code from the docs, this is how I'm calling TransferManager:

#ifndef AWSAPI_H
#define AWSAPI_H
#define USE_IMPORT_EXPORT // Required

#include <chrono>
#include <condition_variable>
#include <fstream>
#include <iostream>
#include <mutex>
#include <sys/stat.h>

#include <aws/core/auth/AWSCredentials.h>
#include <aws/core/Aws.h>
#include <aws/core/utils/threading/Executor.h>
#include <aws/transfer/TransferManager.h>
#include <aws/transfer/TransferHandle.h>
#include <aws/s3/S3Client.h>
#include <aws/core/utils/memory/AWSMemory.h>
#include <aws/core/utils/memory/stl/AWSStreamFwd.h>
#include <aws/core/utils/stream/PreallocatedStreamBuf.h>
#include <aws/core/utils/StringUtils.h>

static void post(QString filePath)
{
    Aws::SDKOptions options;
    options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
    Aws::InitAPI(options);
    {
        // File information
        QFileInfo videoFileInfo(filePath);
        const Aws::String BUCKET = "<bucket-name>";
        const Aws::String FILEPATH = videoFileInfo.filePath().toStdString();
        const Aws::String FILENAME = videoFileInfo.fileName().toStdString();
        const Aws::String ACCESS_KEY = "<access-key>";
        const Aws::String SECRET_KEY = "<secret-access-key>";

        // Config & Credentials
        Aws::Client::ClientConfiguration clientConfig;
        clientConfig.region = "<region-name>"; // "us-east-1";

        Aws::Auth::AWSCredentials credentials;
        credentials.SetAWSAccessKeyId(ACCESS_KEY);
        credentials.SetAWSSecretKey(SECRET_KEY);

        // Upload
        auto s3_client = Aws::MakeShared<Aws::S3::S3Client>("S3Client", credentials, clientConfig);
        auto executor = Aws::MakeShared<Aws::Utils::Threading::PooledThreadExecutor>("executor", 25);
        Aws::Transfer::TransferManagerConfiguration transfer_config(executor.get());
        transfer_config.s3Client = s3_client;

        auto transfer_manager = Aws::Transfer::TransferManager::Create(transfer_config);

        auto uploadHandle = transfer_manager->UploadFile(
            FILEPATH,
            BUCKET,
            FILENAME,
            "text/plain",
            Aws::Map<Aws::String, Aws::String>()
        );
        uploadHandle->WaitUntilFinished();
        bool success = uploadHandle->GetStatus() == Aws::Transfer::TransferStatus::COMPLETED;

        // Results
        if (!success)
        {
            // auto err = uploadHandle->GetLastError();
            qCCritical(models).noquote() <<  "File upload failed: "; // << err.GetMessage().c_str();
        }
        else
        {
            qCInfo(models).noquote() << "File upload succeeded"; 
        }
    }
    Aws::ShutdownAPI(options);
};

#endif // AWSAPI_H

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

v1.9.362

Compiler and Version used

Microsoft Visual Studio C++ Compiler 16.11.32106.194 (amd64)

Operating System and version

Windows 11, v21H2

Oct 14 '22 23:10 HyunWinter

aws-sdk-cpp aws-sdk-cpp copied to clipboard

[Bug]: TransferManager S3 Upload - "Http request to retrieve credentials failed"

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

AWS CPP SDK version used

Compiler and Version used

Operating System and version

aws-sdk-cpp
aws-sdk-cpp copied to clipboard