ckanext-xloader rq child process logging broke burst click logs and unit test cli ckan jobs worker --burst logging

ckan (master) has moved from rq==1.16.2 to rq==2.3.3

There has been subtle changes under the hood where the worker thread is now spawned as a child and may not synchronise log back to click cli routing. This causes many tests looking for assert "Express Load completed" in stdout to fail even though the job passed on what occurred on inside ckan processing.

Sep 20 '25 06:09 duttonw

Since we have added the --sync flag to enable synchronous XLoader processing, we should be able to adapt our tests. We can split them to have one test that just confirms the queued job ran, then another that synchronously runs XLoader and verifies the right logs are output.

Sep 26 '25 06:09 ThrawnCA

Encountering difficulties around changes made to the index_package function and how it interacts with custom package schemas.

Sep 29 '25 05:09 ThrawnCA

It turns out that some corner-cutting for the sake of efficiency (skipping API layer to go straight to the model) in https://github.com/ckan/ckan/issues/3245 interacts badly with recent changes made for https://github.com/ckan/ckan/issues/8395

Apparently resource_patch is more efficient than it used to be, and so the API layer can now be used correctly.

Oct 01 '25 22:10 ThrawnCA