action_service.list_values no limit or offset support?

[fdrab]

SUMMARY

action_service.list_values not working properly?

STACKSTORM VERSION

3.7.0

OS, environment, install method

OS install on RHEL

Steps to reproduce the problem

As per the contrib/runners/python_runner/python_runner/python_action_wrapper.py the action service allows for listing of datastore values. The relevant method I think is this:

def list_values(self, local=True, prefix=None):
    return self.datastore_service.list_values(local=local, prefix=prefix)

However, the above is calling below method without the "limit" parameter: st2common/st2common/services/datastore.py line 71

def list_values(self, local=True, prefix=None, limit=None, offset=0):
    """
    Retrieve all the datastores items.

    :param local: List values from a namespace local to this pack/class. Defaults to True.
    :type: local: ``bool``

    :param prefix: Optional key name prefix / startswith filter.
    :type prefix: ``str``

    :param limit: Number of keys to get. Defaults to the configuration set at 'api.max_page_size'.
    :type limit: ``integer``

    :param offset: Number of keys to offset. Defaults to 0.
    :type offset: ``integer``

    :rtype: ``list`` of :class:`KeyValuePair`
    """
    client = self.get_api_client()
    self._logger.debug("Retrieving all the values from the datastore")

    limit = limit or cfg.CONF.api.max_page_size
    key_prefix = self._get_full_key_prefix(local=local, prefix=prefix)
    kvps = client.keys.get_all(prefix=key_prefix, limit=limit, offset=offset)
    return kvps

Using action_service.list_values I'm not able to provide the limit parameter and thus the action will fail with: oslo_config.cfg.NoSuchOptError: no such option in group api: max_page_size

even though I did put max_page_size = 100 into st2.conf and restarted st2.

Is there a reason why I can't provide the limit to the action_service.list_values method?

(another missing param is "offset", which would also be useful in certain situations)

Expected Results

a list of keyvalue pairs

Actual Results

st2.actions.python.SetAccountsObject: DEBUG Creating new Client object. st2.actions.python.SetAccountsObject: DEBUG Retrieving all the values from the datastore Traceback (most recent call last): File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2262, in _get return self.__cache[key] KeyError: ('api', 'max_page_size')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 395, in obj.run() File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 214, in run output = action.run(**self._parameters) File "/opt/stackstorm/internal_packs/preprocessing_shared_scripts/actions/set_accounts_obj.py", line 8, in run accts_objects = self.get_overrides(capability_name) File "/opt/stackstorm/internal_packs/preprocessing_shared_scripts/actions/set_accounts_obj.py", line 12, in get_overrides return self.action_service.list_values(local=False, prefix=f'{capability}accounts') File "/opt/stackstorm/st2/lib/python3.8/site-packages/python_runner/python_action_wrapper.py", line 136, in list_values return self.datastore_service.list_values(local=local, prefix=prefix) File "/opt/stackstorm/st2/lib/python3.8/site-packages/st2common/services/datastore.py", line 92, in list_values limit = limit or cfg.CONF.api.max_page_size File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2547, in getattr return self._conf._get(name, self._group) File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2264, in _get value = self._do_get(name, group, namespace) File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2282, in _do_get info = self._get_opt_info(name, group) File "/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/cfg.py", line 2415, in _get_opt_info raise NoSuchOptError(opt_name, group) oslo_config.cfg.NoSuchOptError: no such option in group api: max_page_size

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!