IXmlHttpRequest2HttpClient - Error while making request with code: Security certificate required to access this resource is invalid.

[franccan]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [X] I've gone though Developer Guide and API reference
• [X] I've searched for previous similar issues and didn't find any solution

Describe the bug Unable to get a thing shadow state. The connection times out and the trace suggest that there is a security problem.

Error received when using IXmlHttpRequest2: IXmlHttpRequest2HttpClient - Error while making request with code: Security certificate required to access this resource is invalid.

Error received when using WinHTTP: WinHttpSyncHttpClient - Send request failed: A security error occurred

SDK version number aws-sdk-cpp/1.8.42/IoT Data Plane/Windows/10.0.17763.1339 AMD64 MSVC/1927

Platform/OS/Hardware/Device Windows Server 2019

To Reproduce (observed behavior)

Building using IXML_HTTP_REQUEST_2

cd build
cmake ../ -G "Visual Studio 16 2019" -A x64 -DCMAKE_BUILD_TYPE=Debug -DBUILD_ONLY=”core;iot-data;s3”  -DUSE_IXML_HTTP_REQUEST_2=ON
msbuild ALL_BUILD.vcxproj /p:Configuration=Debug

Build using WinHTTP:

cd build
cmake ../ -G "Visual Studio 16 2019" -A x64 -DCMAKE_BUILD_TYPE=Debug -DBUILD_ONLY=”core;iot-data;s3”  -DUSE_IXML_HTTP_REQUEST_2=OFF
msbuild ALL_BUILD.vcxproj /p:Configuration=Debug

Minimal code used to reproduce the problem:

int main()
{
	Aws::SDKOptions options;
	options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
	options.loggingOptions.logger_create_fn = [] 
	{ 
		return std::make_shared<Aws::Utils::Logging::ConsoleLogSystem>(Aws::Utils::Logging::LogLevel::Trace); 
	};
	Aws::InitAPI(options);
	{
		Aws::IoTDataPlane::IoTDataPlaneClient iot_client;	
		Aws::IoTDataPlane::Model::GetThingShadowRequest thing_request;
		thing_request.WithThingName("myThing");
		auto outcome = iot_client.GetThingShadow(thing_request);
	}
	Aws::ShutdownAPI(options);
}

Expected behavior I was expecting a response from the server, but instead received a security related error from the SDK.

Logs/output using IXmlHttpRequest2HttpClient

[TRACE] 2020-09-08 10:06:41.168 WinHttpSyncHttpClient [3012] Response content-length header: 10
[TRACE] 2020-09-08 10:06:41.171 WinHttpSyncHttpClient [3012] Response body length: 10
[DEBUG] 2020-09-08 10:06:41.171 WinHttpSyncHttpClient [3012] Closing http request handle 0000021FCE8860C0
[DEBUG] 2020-09-08 10:06:41.172 WinHttpSyncHttpClient [3012] Releasing connection handle 0000021FCE818590
[DEBUG] 2020-09-08 10:06:41.172 WinHttpConnectionPoolMgr [3012] Releasing connection to endpoint 169.254.169.254:80
[DEBUG] 2020-09-08 10:06:41.173 EC2MetadataClient [3012] Calling EC2MetadataService resource /latest/meta-data/placement/availability-zone , returned credential string eu-west-1b
[INFO] 2020-09-08 10:06:41.173 EC2MetadataClient [3012] Detected current region as eu-west-1
[INFO] 2020-09-08 10:06:41.173 Aws::Config::AWSProfileConfigLoader [3012] Successfully reloaded configuration.
[TRACE] 2020-09-08 10:06:41.174 Aws::Config::AWSProfileConfigLoader [3012] reloaded config at 2020-09-08T10:06:41Z
[INFO] 2020-09-08 10:06:41.175 HttpClientFactory [3012] Creating IXMLHttpRequest http client.
[INFO] 2020-09-08 10:06:41.175 IXmlHttpRequest2HttpClient [3012] Initializing client with pool size of 25
[TRACE] 2020-09-08 10:06:41.181 AWSClient [3012] No content body, content-length headers
[ERROR] 2020-09-08 10:06:41.181 ProcessCredentialsProvider [3012] Failed to find credential process's profile: default
[DEBUG] 2020-09-08 10:06:41.182 InstanceProfileCredentialsProvider [3012] Checking if latest credential pull has expired.
[DEBUG] 2020-09-08 10:06:41.182 AWSAuthV4Signer [3012] Using cached empty string sha256 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 because payload is empty.
[DEBUG] 2020-09-08 10:06:41.184 AWSAuthV4Signer [3012] Canonical Header String: amz-sdk-invocation-id:1EF65FD3-690C-48F8-9776-C5F73D091E2A
amz-sdk-request:attempt=1
content-type:application/x-amz-json-1.1
host:data.iot.eu-west-1.amazonaws.com
x-amz-api-version:2015-05-28
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20200908T100641Z
x-amz-security-token:AAABBBBDEMARCATED==
[DEBUG] 2020-09-08 10:06:41.185 AWSAuthV4Signer [3012] Signed Headers value:amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token
[DEBUG] 2020-09-08 10:06:41.186 AWSAuthV4Signer [3012] Canonical Request String: GET
/things/myThing/shadow

amz-sdk-invocation-id:1EF65FD3-690C-48F8-9776-C5F73D091E2A
amz-sdk-request:attempt=1
content-type:application/x-amz-json-1.1
host:data.iot.eu-west-1.amazonaws.com
x-amz-api-version:2015-05-28
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20200908T100641Z
x-amz-security-token:AAABBBBDEMARCATED==
amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token
AAABBBBDEMARCATED==
[DEBUG] 2020-09-08 10:06:41.187 AWSAuthV4Signer [3012] Final String to sign: AWS4-HMAC-SHA256
20200908T100641Z
20200908/eu-west-1/iotdata/aws4_request
572de8b897705fa60510f1b79a88d5e92ff7941cc893ba4f546dfbb8f75b22fc
[DEBUG] 2020-09-08 10:06:41.188 AWSAuthV4Signer [3012] Final computed signing hash: bc65c456bb9937e8aa849ed659f4c9aa16d0da6dbe634ca4f3d4e2a9c8bcafab
[DEBUG] 2020-09-08 10:06:41.192 AWSAuthV4Signer [3012] Signing request with: AWS4-HMAC-SHA256 Credential=ASIAYAAAA/20200908/eu-west-1/iotdata/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=bc65c456bb9937e8aa849ed659f4c9aa16d0da6dbe634ca4f3d4e2a9c8bcafab
[DEBUG] 2020-09-08 10:06:41.193 AWSClient [3012] Request Successfully signed
[DEBUG] 2020-09-08 10:06:41.194 IXmlHttpRequest2HttpClient [3012] Making GET request to url: https://data.iot.eu-west-1.amazonaws.com/things/myThing/shadow
[TRACE] 2020-09-08 10:06:41.209 IXmlHttpRequest2HttpClient [3012] Setting up request handle with verifySSL = 1, follow redirects = 1 and timeout = 4000
[TRACE] 2020-09-08 10:06:41.209 IXmlHttpRequest2HttpClient [3012] Setting http headers:
[TRACE] 2020-09-08 10:06:41.210 IXmlHttpRequest2HttpClient [3012] amz-sdk-invocation-id: 1EF65FD3-690C-48F8-9776-C5F73D091E2A
[TRACE] 2020-09-08 10:06:41.211 IXmlHttpRequest2HttpClient [3012] amz-sdk-request: attempt=1
[TRACE] 2020-09-08 10:06:41.211 IXmlHttpRequest2HttpClient [3012] authorization: AWS4-HMAC-SHA256 Credential=ASIAYAAAA/20200908/eu-west-1/iotdata/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=bc65c456bb9937e8aa849ed659f4c9aa16d0da6dbe634ca4f3d4e2a9c8bcafab
[TRACE] 2020-09-08 10:06:41.212 IXmlHttpRequest2HttpClient [3012] content-type: application/x-amz-json-1.1
[TRACE] 2020-09-08 10:06:41.212 IXmlHttpRequest2HttpClient [3012] host: data.iot.eu-west-1.amazonaws.com
[TRACE] 2020-09-08 10:06:41.213 IXmlHttpRequest2HttpClient [3012] user-agent: aws-sdk-cpp/1.8.42/IoT Data Plane/Windows/10.0.17763.1339 AMD64 MSVC/1927
[TRACE] 2020-09-08 10:06:41.213 IXmlHttpRequest2HttpClient [3012] x-amz-api-version: 2015-05-28
[TRACE] 2020-09-08 10:06:41.214 IXmlHttpRequest2HttpClient [3012] x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
[TRACE] 2020-09-08 10:06:41.214 IXmlHttpRequest2HttpClient [3012] x-amz-date: 20200908T100641Z
[TRACE] 2020-09-08 10:06:41.215 IXmlHttpRequest2HttpClient [3012] x-amz-security-token: AAABBBBDEMARCATED==
[DEBUG] 2020-09-08 10:06:41.229 IXmlHttpRequest2HttpClient [3012] Waiting for request to finish.
- [ERROR] 2020-09-08 10:06:41.281 IXmlHttpRequest2HttpClient [1252] Error while making request with code: Security certificate required to access this resource is invalid.
[DEBUG] 2020-09-08 10:06:41.282 IXmlHttpRequest2HttpClient [3012] Request completed, continueing thread.
[ERROR] 2020-09-08 10:06:41.283 IXmlHttpRequest2HttpClient [3012] Request finished with response code: -1
[DEBUG] 2020-09-08 10:06:41.283 AWSClient [3012] Request returned error. Attempting to generate appropriate error codes from response
[ERROR] 2020-09-08 10:06:41.284 AWSClient [3012] HTTP response code: -1
Resolved remote host IP address:
Request ID:
Exception name:
Error message: Request finished with response code: -1
0 response headers:
[WARN] 2020-09-08 10:06:41.285 AWSClient [3012] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
[DEBUG] 2020-09-08 10:06:41.285 AWSClient [3012] Date header was not found in the response, can't attempt to detect clock skew
[WARN] 2020-09-08 10:06:41.286 AWSClient [3012] Request failed, now waiting 0 ms before attempting again.
[TRACE] 2020-09-08 10:06:41.287 AWSClient [3012] No content body, content-length headers

Logs/output using WinHttpSyncHttpClient

[TRACE] 2020-09-08 10:17:55.716 WinHttpSyncHttpClient [7864] Response content-length header: 10
[TRACE] 2020-09-08 10:17:55.717 WinHttpSyncHttpClient [7864] Response body length: 10
[DEBUG] 2020-09-08 10:17:55.717 WinHttpSyncHttpClient [7864] Closing http request handle 00000131E05D9FC0
[DEBUG] 2020-09-08 10:17:55.718 WinHttpSyncHttpClient [7864] Releasing connection handle 00000131E0584440
[DEBUG] 2020-09-08 10:17:55.718 WinHttpConnectionPoolMgr [7864] Releasing connection to endpoint 169.254.169.254:80
[DEBUG] 2020-09-08 10:17:55.719 EC2MetadataClient [7864] Calling EC2MetadataService resource /latest/meta-data/placement/availability-zone , returned credential string eu-west-1b
[INFO] 2020-09-08 10:17:55.719 EC2MetadataClient [7864] Detected current region as eu-west-1
[INFO] 2020-09-08 10:17:55.720 Aws::Config::AWSProfileConfigLoader [7864] Successfully reloaded configuration.
[TRACE] 2020-09-08 10:17:55.723 Aws::Config::AWSProfileConfigLoader [7864] reloaded config at 2020-09-08T10:17:55Z
[INFO] 2020-09-08 10:17:55.724 WinHttpSyncHttpClient [7864] Creating http client with user agent  with max connections 25 request timeout 3000,and connect timeout 1000
[DEBUG] 2020-09-08 10:17:55.724 WinHttpHttp2 [7864] HTTP/2 enabled on WinHttp handle: 00000131E05EBA80.
[DEBUG] 2020-09-08 10:17:55.725 WinHttpSyncHttpClient [7864] API handle 00000131E05EBA80
[INFO] 2020-09-08 10:17:55.725 ConnectionPoolMgr [7864] Creating connection pool mgr with handle 00000131E05EBA80, and max connections per host 25, request timeout 3000 ms, and connect timeout in 1000 ms, enabling TCP keep-alive.
[TRACE] 2020-09-08 10:17:55.731 AWSClient [7864] No content body, content-length headers
[ERROR] 2020-09-08 10:17:55.733 ProcessCredentialsProvider [7864] Failed to find credential process's profile: default
[DEBUG] 2020-09-08 10:17:55.735 InstanceProfileCredentialsProvider [7864] Checking if latest credential pull has expired.
[DEBUG] 2020-09-08 10:17:55.736 AWSAuthV4Signer [7864] Using cached empty string sha256 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 because payload is empty.
[DEBUG] 2020-09-08 10:17:55.741 AWSAuthV4Signer [7864] Canonical Header String: amz-sdk-invocation-id:4F43BCC0-7F3F-4445-8BFA-888AB4B665AD
amz-sdk-request:attempt=1
content-type:application/x-amz-json-1.1
host:data.iot.eu-west-1.amazonaws.com
x-amz-api-version:2015-05-28
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20200908T101755Z
x-amz-security-token:AAABBBBDEMARCATED==

[DEBUG] 2020-09-08 10:17:55.747 AWSAuthV4Signer [7864] Signed Headers value:amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token
[DEBUG] 2020-09-08 10:17:55.749 AWSAuthV4Signer [7864] Canonical Request String: GET
/things/myThing/shadow

amz-sdk-invocation-id:4F43BCC0-7F3F-4445-8BFA-888AB4B665AD
amz-sdk-request:attempt=1
content-type:application/x-amz-json-1.1
host:data.iot.eu-west-1.amazonaws.com
x-amz-api-version:2015-05-28
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20200908T101755Z
x-amz-security-token:AAABBBBDEMARCATED==

amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
[DEBUG] 2020-09-08 10:17:55.752 AWSAuthV4Signer [7864] Final String to sign: AWS4-HMAC-SHA256
20200908T101755Z
20200908/eu-west-1/iotdata/aws4_request
39baf6087ac6f2af0431f21a288074a6105a3310824296c54e8db97da4e02aed
[DEBUG] 2020-09-08 10:17:55.753 AWSAuthV4Signer [7864] Final computed signing hash: 151404790aec0cfea82f1c725ec4c4d5a48afe18c67e1e5cebe8db60ff264c59
[DEBUG] 2020-09-08 10:17:55.754 AWSAuthV4Signer [7864] Signing request with: AWS4-HMAC-SHA256 Credential=ASIAYAAA/20200908/eu-west-1/iotdata/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=151404790aec0cfea82f1c725ec4c4d5a48afe18c67e1e5cebe8db60ff264c59
[DEBUG] 2020-09-08 10:17:55.756 AWSClient [7864] Request Successfully signed
[TRACE] 2020-09-08 10:17:55.758 WinHttpSyncHttpClient [7864] Making GET request to uri https://data.iot.eu-west-1.amazonaws.com/things/myThing/shadow
[INFO] 2020-09-08 10:17:55.759 WinHttpConnectionPoolMgr [7864] Attempting to acquire connection for data.iot.eu-west-1.amazonaws.com:443
[DEBUG] 2020-09-08 10:17:55.760 WinHttpConnectionPoolMgr [7864] Pool doesn't exist for endpoint, creating...
[DEBUG] 2020-09-08 10:17:55.761 WinHttpConnectionPoolMgr [7864] Pool has no available existing connections for endpoint, attempting to grow pool.
[INFO] 2020-09-08 10:17:55.762 WinConnectionContainer [7864] Pool grown by 2
[INFO] 2020-09-08 10:17:55.763 WinHttpConnectionPoolMgr [7864] Connection now available, continuing.
[DEBUG] 2020-09-08 10:17:55.764 WinHttpConnectionPoolMgr [7864] Returning connection handle 00000131E05FD370
[DEBUG] 2020-09-08 10:17:55.764 WinHttpSyncHttpClient [7864] Acquired connection 00000131E05FD370
[WARN] 2020-09-08 10:17:55.765 WinHttpSyncHttpClient [7864] Failed setting TCP keep-alive interval with error code: 12018
[DEBUG] 2020-09-08 10:17:55.765 WinHttpHttp2 [7864] HTTP/2 enabled on WinHttp handle: 00000131E05FB6D0.
[DEBUG] 2020-09-08 10:17:55.768 WinHttpSyncHttpClient [7864] AllocateWindowsHttpRequest returned handle 00000131E05FB6D0
[DEBUG] 2020-09-08 10:17:55.769 WinHttpSyncHttpClient [7864] with headers:
[DEBUG] 2020-09-08 10:17:55.769 WinHttpSyncHttpClient [7864] amz-sdk-invocation-id: 4F43BCC0-7F3F-4445-8BFA-888AB4B665AD
amz-sdk-request: attempt=1
authorization: AWS4-HMAC-SHA256 Credential=ASIAAAAA/20200908/eu-west-1/iotdata/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=151404790aec0cfea82f1c725ec4c4d5a48afe18c67e1e5cebe8db60ff264c59
content-type: application/x-amz-json-1.1
host: data.iot.eu-west-1.amazonaws.com
user-agent: aws-sdk-cpp/1.8.42/IoT Data Plane/Windows/10.0.17763.1339 AMD64 MSVC/1927
x-amz-api-version: 2015-05-28
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20200908T101755Z
x-amz-security-token: AAABBBBDEMARCATED==

- [WARN] 2020-09-08 10:17:55.865 WinHttpSyncHttpClient [7864] Send request failed: A security error occurred

[DEBUG] 2020-09-08 10:17:55.872 WinHttpSyncHttpClient [7864] Closing http request handle 00000131E05FB6D0
[DEBUG] 2020-09-08 10:17:55.888 WinHttpSyncHttpClient [7864] Releasing connection handle 00000131E05FD370
[DEBUG] 2020-09-08 10:17:55.889 WinHttpConnectionPoolMgr [7864] Releasing connection to endpoint data.iot.eu-west-1.amazonaws.com:443
[DEBUG] 2020-09-08 10:17:55.891 AWSClient [7864] Request returned error. Attempting to generate appropriate error codes from response
[ERROR] 2020-09-08 10:17:55.896 AWSClient [7864] HTTP response code: -1
Resolved remote host IP address:
Request ID:
Exception name:
Error message: Encountered network error when sending http request
0 response headers:
[WARN] 2020-09-08 10:17:55.903 AWSClient [7864] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
[DEBUG] 2020-09-08 10:17:55.911 AWSClient [7864] Date header was not found in the response, can't attempt to detect clock skew
[WARN] 2020-09-08 10:17:55.936 AWSClient [7864] Request failed, now waiting 0 ms before attempting again.
[TRACE] 2020-09-08 10:17:55.955 AWSClient [7864] No content body, content-length headers

[KaibaLopez]

Hi @franccan , This is not a bug, but rather a usage question. And before I go digging on this can you tell me more about what you've tried? Any guide, samples or documentation that you followed, just so I can get a better idea on the use case. As a side note, I am able to run this sample on a thing with a shadow, so I think the problem is most likely related to the thing's policies.

Also if you're only trying to get the state of a thing's shadow, it may be better to try the aws-iot-device-sdk-cpp-v2.

[franccan]

Hi KaibaLopez,

Thanks for your quick response!

I did not follow any guide or sample because there isn’t one; I couldn’t find a single example of using the ‘iot-data’ api with AWS C++ SDK. However, the request is quite simple and should work.

I also thought this could be a permission’s issue until I tried the same call using the Windows CLI and it worked. Both, the CPP code and the CLI use the EC2 instance role to authenticate.

When you said that you are able to get the shadow from a thing. Do you mean using the sample C++ code I posted from a Windows Server host?

I suspect that this is related to how Windows negotiates the TLS with the API. As seen on: https://github.com/aws/aws-sdk-cpp/issues/281, https://github.com/aws/aws-sdk-cpp/issues/382, https://github.com/aws/aws-sdk-cpp/issues/671

PS C:\> date
Wednesday, September 9, 2020 10:19:03 AM
PS C:\> aws iot-data get-thing-shadow --thing-name myThing outfile
PS C:\> ls outfile
Directory: C:\
Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----         9/9/2020  10:19 AM            273 outfile
PS C:\> cat outfile
{"state":{"desired":{"welcome":"aws-iot"},"reported":{"welcome":"aws-iot","moisture":"okay"}},"metadata":{"desired":{"welcome":{"timestamp":1599500883}},"reported":{"welcome":{"timestamp":1599500883},"moisture":{"timestamp":1599504930}}},"version":2,"timestamp":1599643145}
PS C:\>

[franccan]

Hi,

Is there any update regarding this issue?

[KaibaLopez]

Hi @franccan , Sorry for the lack of response here, but yea you were right it is not working on windows server currently trying to see what exactly is going on, will update again should I find a solution or workaround.

[jmklix]

Can you try using curl for windows and see if that works for you?

Also using the IoT sdk might be a better option for you?

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.