aws-sdk-js
aws-sdk-js copied to clipboard
aws
Include resource-explorer service
Is your feature request related to a problem? Please describe. I want to be able to list all resources within an AWS account, as part of closing down an AWS account to verify that there are no surprises. The AWS web console has a way of doing this in Resource Groups -> Tagging -> Tag Editor. But I can find no way of replicating this using code in the SDK so we have a process that is not reliant on a human remembering to check the right options in the AWS web console. In the browser dev-console network tab, I can see requests that look like:
- Resources list:
- URL:
https://resource-explorer.eu-west-2.amazonaws.com/resources-list - JSON Payload:
{"ResourceType":"AWS::EC2::Subnet"} - Response: a list of ARNs
- URL:
- Tags list:
- URL:
https://resource-explorer.eu-west-2.amazonaws.com/tags-list - JSON Payload: the list of ARNs returned by the above
- Response: a list of tags for each of those resources
- URL:
This service is doing exactly what I need.
Further, in the dev-console if I evaluate some JS:
> resExp = new AWS.ResourceExplorer();
Object { config: {…}, isGlobalEndpoint: false, endpoint: {…}, _clientId: 104, … }
> resExp.listResources
function t(e, n)
> resExp.listTags
function t(e, n)
So it really is a service, and it exists in some form of the AWS JS SDK.
AWS.VERSION in the browser when at https://eu-west-2.console.aws.amazon.com/resource-groups/tag-editor/find-resources?region=eu-west-2 says 2.145.0, which is old, but still matches a version published on NPM.
But:
$ curl --silent https://sdk.amazonaws.com/js/aws-sdk-2.145.0.js | grep -i 'resource\-\?explorer'
finds nothing. So I think maybe this is a slightly different version than the one bundled and used within the AWS web console.
Describe the solution you'd like To be able to use the ResourceExplorer service from the JS SDK just like the AWS web console does.
Describe alternatives you've considered I've looked at the other services in the SDK, but the closest I could find was https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/ResourceGroupsTaggingAPI.html#getResources-property which doesn't cut it for my uses, as it doesn't find things that have no tags.
Additional context I suspect the fix here is to add something into https://github.com/aws/aws-sdk-js/tree/master/apis. I could probably attempt to locate the JSON for the Resource Explorer in the code that is served from the AWS web console, but is there a (public?) repo somewhere that contains those JSON definitions of the services?
I think I've been able to hack the shape of the .normal.json file from what I was served from the AWS web console, with the following monstrosity:
$ curl --silent https://d30hge9gncwdge.cloudfront.net/1591978612186/bundle.js | grep 'name:"ResourceExplorer"' | sed 's/.*name:"ResourceExplorer"}}},function(e,t){e.exports=//' | sed 's/},function.*//' | node -e 'fs = require("fs"); eval("data = " + fs.readFileSync(0, "utf8")); console.log(JSON.stringify(data, null, 2))'
- Get the bundle I was served in the AWS web console
- Find the bit that defines the ResourceExplorer service
- Chop out the bit I want
- Pass to node and evaluate
- Dump as JSON
I've then put into the SDK, and wired it up: https://github.com/aws/aws-sdk-js/compare/master...plumdog:issue-3369-resource-explorer-service
And it seems to work as I'd expect:
$ node -e 'AWS = require("."); re = new AWS.ResourceExplorer(); re.listResources({ResourceType: "AWS::EC2::Subnet"}).promise().then(console.log)'
{
NextToken: null,
ResourceArns: [
'arn:aws:ec2:eu-west-2:xxxx:subnet/subnet-xxxx',
...
]
}
So...
Now I really do believe it is a real AWS service, that would have real value being added to the SDK.
I'm not seriously suggesting that my branch be merged, but rather that the config for the ResourceExplorer service be included in the service definitions that are copied in from somewhere by awstools, because I think it would add real value to the SDK, as it allows something that cannot be done another way.
TLDR
How do I request that awstools include the ResourceExplorer service in the definitions copied into apis/?
Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.
It's frustrating seeing this in the GUI but not being able to access it programmatically. The functionality this API provides is the white whale of so many abandoned GitHub repos.
Yes, 100% yes! It's quite poor that AWS have this feature in the web ui but not in the CLI - they love telling you that everything in the Web UI is an API call just like the CLI/SDK's but they don't actually share the same features :(
@plumdog - I've referenced this in new Issues in the boto3 and CLI repos - hope you don't mind :)
For what it's worth, I've asked the AWS Support about this and got this (expected) answer back (emphasis mine):
I understand that you have observed that the results obtained using Resource Groups Tagging API 'GetResources' or CLI command 'get-resources' differ from the results obtained via the Tag Editor console. Upon digging deeper you also noticed that the console seems to be using an undocumented API endpoint which provides the results as expected and you have a couple of queries regarding the same.
Please note that, the public 'GetResources' API only reports currently or previously tagged resources, but does not return resources which were never tagged. This is a known limitation with 'GetResources' API. The same is mentioned in the API reference document [1] as below: "Returns all the tagged or previously tagged resources that are located in the specified Region for the AWS account."
That being said, we have an existing feature request present with the Resource Groups Tagging API service team to update the 'GetResources' API, such that it can also list resources even if they have never been tagged. This will allow to list all resources and then determine ones with no tag. But I am unable to share any rough ETA for delivering this feature as inclusion of any new feature needs rigorous testing. Also, this shall be prioritized along with the team's additional workload and it all depends on our product team discretion.
Now regarding the undocumented API endpoint (Resource Explorer) that you observed, I checked through our internal communications and I see that the respective internal team has mentioned the following: "Thanks for bringing the issue to our notice. We have tracked the feature request in our backlog. We will evaluate externalizing the APIs for Tag Editor (Resource Explorer) as part of our roadmap."
Thus, the Resource Explorer service API is not currently available to be used via public release of any AWS SDK. The service owners have marked this as a feature request and they will evaluate the possibility of externalizing the APIs, so that they are available in future public AWS SDK releases. They have added this request to their backlog, but I cannot comment on any ETA of when it will be actioned.
Hence to answer your queries, using the API endpoint (https://resource-explorer.us-east-1.amazonaws.com/resources-list) is not recommended, since it is not officially documented and is subject to change without any prior notification. Also, as mentioned above the internal team will evaluate externalizing the APIs as part of the roadmap. That being said, for now a workaround to get all resources would be to use service-specific List/Describe/Get APIs.
https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md#212500 👀
feature: ResourceExplorer2: This is the initial SDK release for AWS Resource Explorer. AWS Resource Explorer lets your users search for and discover your AWS resources across the AWS Regions in your account.
And https://aws.amazon.com/blogs/aws/introducing-aws-resource-explorer-quickly-find-resources-in-your-aws-account/
https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md#212500 👀
feature: ResourceExplorer2: This is the initial SDK release for AWS Resource Explorer. AWS Resource Explorer lets your users search for and discover your AWS resources across the AWS Regions in your account.
And https://aws.amazon.com/blogs/aws/introducing-aws-resource-explorer-quickly-find-resources-in-your-aws-account/
I can confirm this is working (testing in boto3 as well). So it looks like this issue can be closed. THANK YOU!