girder_worker icon indicating copy to clipboard operation
girder_worker copied to clipboard

Using kwargs in chained tasks fails to create Job object

Open Xarthisius opened this issue 5 years ago • 0 comments

Girder worker mishandles keyword arguments in chained tasks.

Steps to reproduce

Consider the following task:

@girder_job(title='Test Kwargs')
@app.task(bind=True)
def kwargs_task(self, **kwargs):
    print('Got {}'.format(kwargs))
    return kwargs

when executed from within girder shell as an authenticated user:

from girder.models.user import User
from girder.models.token import Token
from girder.plugins.jobs.constants import REST_CREATE_JOB_TOKEN_SCOPE
from gwvolman.tasks import kwargs_task

user = User().list().next()
token = Token().createToken(user=user)
Token().addScope(token, scope=REST_CREATE_JOB_TOKEN_SCOPE)
task1 = kwargs_task.signature(kwargs={'a': 1, 'b':2, 'girder_client_token': str(token['_id'])}, immutable=True)
task2 = kwargs_task.signature(kwargs={'c': [1,2,3], 'd': dict(a=4), 'girder_client_token': str(token['_id'])}, immutable=True)
task2.delay()  # works fine
(task1 | task2).delay()  # fails

the last command yields:

[2019-05-29 17:56:56,814: INFO/MainProcess] Received task: gwvolman.tasks.kwargs_task[33482416-4807-4787-92cb-0adfe50b2ada]  
[2019-05-29 17:56:56,817: WARNING/ForkPoolWorker-2] [2019-05-29 17:56:56,817] WARNING: No jobInfoSpec. Setting job_manager to None.
[2019-05-29 17:56:56,817: WARNING/ForkPoolWorker-2] No jobInfoSpec. Setting job_manager to None.
[2019-05-29 17:56:56,817: WARNING/ForkPoolWorker-2] Got {'b': 2, 'a': 1}
[2019-05-29 17:56:56,820: ERROR/ForkPoolWorker-2] Task gwvolman.tasks.kwargs_task[33482416-4807-4787-92cb-0adfe50b2ada] raised unexpected: AttributeError("'list' object has no attribute 'apply_async'",)
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: '_apply_async'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 439, in trace_task
    parent_id=uuid, root_id=root_id,
  File "/usr/local/lib/python3.5/dist-packages/celery/canvas.py", line 207, in apply_async
    _apply = self._apply_async
  File "/usr/local/lib/python3.5/dist-packages/kombu/utils/objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/usr/local/lib/python3.5/dist-packages/celery/canvas.py", line 487, in _apply_async
    return self.type.apply_async
AttributeError: 'list' object has no attribute 'apply_async'

Xarthisius avatar May 29 '19 17:05 Xarthisius