boto3 icon indicating copy to clipboard operation
boto3 copied to clipboard

[doc] Glacier.Client.list_vaults() example: limit='' and marker='' arg values not supported

Open SCalwas opened this issue 6 years ago • 3 comments

The Glacier.Client.list_vaults() method Examples section lists the code statement:

response = client.list_vaults( accountId='-', limit='', marker='', )

Passing an empty string with limit= raises an exception. The Glacier REST API for the argument states:

Constraints: Minimum integer value of 1. Maximum integer value of 10.

Passing an empty string with marker= also raises an exception. The Glacier REST API states:

Specifying an empty value ("") for the marker returns a list of vaults starting from the first vault.

However, with Boto3, the marker= argument must specify a valid marker value, i.e., a vault ARN, returned from a previous call to list_vaults(). Any other value raises an exception.

SCalwas avatar Nov 13 '18 01:11 SCalwas

I can confirm that copying this example does not function. I think the limit and marker should just be dropped from the example as they're not required.

joguSD avatar Nov 20 '18 21:11 joguSD

Best solution would also accept the passing of an empty string for the marker= argument as the underlying Glacier service supports. Doing so simplifies the logic in wrapping functions, such as demonstrated in this similar EKS list_clusters() function.

def list_clusters(max_clusters=10, iter_marker=''):
    eks = boto3.client('eks')

    clusters = eks.list_clusters(maxResults=max_clusters, nextToken=iter_marker)
    marker = clusters.get('nextToken')       # None if no more clusters to retrieve
    return clusters['clusters'], marker

Because the Boto3 Glacier list_vaults() method raises an exception when marker is any value other than a valid marker returned by a previous call to list_vaults(), its wrapping function is a bit awkward.

def list_vaults(max_vaults=10, iter_marker=None):
   glacier = boto3.client('glacier')

    if iter_marker is None:
        vaults = glacier.list_vaults(limit=str(max_vaults))
    else:
        vaults = glacier.list_vaults(limit=str(max_vaults), marker=iter_marker)
    marker = vaults.get('Marker')       # None if no more vaults to retrieve
    return vaults['VaultList'], marker

SCalwas avatar Nov 20 '18 22:11 SCalwas

Linking to Glacier docs referenced earlier: https://docs.aws.amazon.com/amazonglacier/latest/dev/api-vaults-get.html#api-vaults-get-requests

tim-finnigan avatar Apr 06 '22 19:04 tim-finnigan

To expand more on the documentation linked above, this section in particular is relevant:

By default, this operation returns up to 10 items per request. If there are more vaults to list, the marker field in the response body contains the vault Amazon Resource Name (ARN) at which to continue the list with a new List Vaults request; otherwise, the marker field is null. In your next List Vaults request you set the marker parameter to the value Amazon S3 Glacier (S3 Glacier) returned in the responses to your previous List Vaults request. You can also limit the number of vaults returned in the response by specifying the limit parameter in the request.

I think this command is somewhat of an outlier as most APIs now use something like NextToken or NextMarker to indicate that there are additional results to retrieve. If you tried passing a NextToken in other API calls with an empty string then that would fail the parameter validation. Similarly - the solution here is just to avoid passing marker in the initial call of this command. But the example likely won't change, as it's there to demonstrate how to pass the marker for subsequent API calls as described by the documentation.

But regardless - the Glacier team owns their service API documentation so there's nothing the boto3 team can do here aside from forwarding feedback, which anyone can also do directly by clicking the Provide feedback button at the bottom of the API documentation page.

I hope that helps - please let us know if there's anything regarding this that you'd like to discuss further, other I think this issue should be closed.

tim-finnigan avatar Nov 17 '22 23:11 tim-finnigan