fix: add pagination to 'submit all' to process all datasets

[dsanchezmatilla]

This PR adds a loop to paginate through all datasets in batches of 1000 using package_search, so that all datasets are submitted to the xloader, not just the first 1000.

[dsanchezmatilla]

@duttonw and @ThrawnCA , thanks for the advice about possible memory management issues with my code when working with large numbers of datasets. I'm currently working on a solid fix for the problem.