databricks
databricks jobs list-runs limit does not work
Describe the issue
I want to get just the latest run, but I seem to fail.
The --help describes limit as follows:
--limit int The number of runs to return.
The api (https://docs.databricks.com/api/workspace/jobs/listruns#limit) mentions limit as follows:
The number of runs to return. This value must be greater than 0 and less than 25. The default value is 20. If a request specifies a limit of 0, the service instead uses the maximum limit.
I wonder why I get more than 20 or even 25 results.
Steps to reproduce the behavior
Please list the steps required to reproduce the issue, for example:
databricks jobs list-runs --job-id xxx --limit 1
Expected Behavior
I want to get just the latest job-run when I provide --limit 1
Actual Behavior
Clear and concise description of what actually happened
OS and CLI version
Databricks CLI v0.220.0 on Windows using WSL
Is this a regression?
I'm not sure.
Debug Logs
16:49:35 DEBUG GET /api/2.1/jobs/runs/list?job_id=xxx&limit=1 16:49:35 DEBUG GET /api/2.1/jobs/runs/list?job_id=xxx&limit=1&page_token=CAEQ97q7kfwxIL7F6NfD-wQ= 16:49:35 DEBUG GET /api/2.1/jobs/runs/list?job_id=xxx&limit=1&page_token=CAEQxevk9_sxII6r2p_NIg== [...]
Not sure if it helps but I've noticed similar on the cluster events call - limit seems to be used to control how many results are sent in each batch, not to control the number of results total. In the cluster events example it actually only serves to make the command slower. Just never got around to opening an issue for it.
Same issue with jobs list on CLI v0.230.0. I'm running on Mac OS. I only want to list, say, 5 jobs but no matter what I specify for --limit I get all the jobs. I have also observed the behavior that the --limit flag seems to be setting the batch / page size for the results. The higher I set it, the faster the jobs list command completes.
Agreed that the documentation on the "limit" keyword is not correct and should be updated to clarify that it is the limit used under the hood for pagination but does not affect the # of returned results.
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
The documentation needs to be updated - I am relatively certain this limit, in newer versions of the lib, is not a total limit but a limit per page that it uses under the hood. This api will return the total list of jobs, regardless of page size.
Requesting confirmation from the databricks team and an update to the documentation if warranted.
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.
