cylc-uiserver icon indicating copy to clipboard operation
cylc-uiserver copied to clipboard

Lack of info when Clean fails

Open MetRonnie opened this issue 3 years ago • 0 comments

If the Clean command fails, the user is presented with very little info in the UI:

image

The info does get logged in the UIServer log, however: (I've highlighted the precise cause of the failure in this case by arrow - just due to incorrect Cylc setup for HPC)

Click to show log
[I CylcUIServer] Cleaning ~user/my-workflow
ERROR:cylc:Could not clean my-workflow on install target: xy
platform: xyz - clean up did not complete
COMMAND:
    ssh -oBatchMode=yes -oConnectTimeout=8 \ 
        -oStrictHostKeyChecking=no xyzl01 env \ 
        CYLC_VERSION=8.0.1.dev bash --login -c 'exec "$0" "$@"' \ 
        timeout 600 cylc clean --local-only my-workflow --rm .service:log
RETURN CODE:
    1
STDOUT:
STDERR:
      WARNING: This computer is provided for the processing of official...
      
    ERROR: cylc-8.0.1.dev not found in /data/whatever/    <-------------------------------------

WARNING:cylc:Remote clean failed for my-workflow

[E CylcUIServer] CylcError: Clean failed: my-workflow
    concurrent.futures.process._RemoteTraceback: 
    """
    Traceback (most recent call last):
      File "~/.conda/envs/cylc8/lib/python3.9/concurrent/futures/process.py", line 246, in _process_worker
        r = call_item.fn(*call_item.args, **call_item.kwargs)
      File "~/github/cylc-uiserver/cylc/uiserver/resolvers.py", line 188, in _clean
        return loop.run_until_complete(
      File "~/.conda/envs/cylc8/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
        return future.result()
      File "~/github/cylc-flow/cylc/flow/scripts/clean.py", line 204, in run
        raise CylcError(f"Clean failed: {', '.join(failed)}")
    cylc.flow.exceptions.CylcError: Clean failed: my-workflow
    """
    
    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "~/github/cylc-uiserver/cylc/uiserver/resolvers.py", line 233, in clean
        await asyncio.get_event_loop().run_in_executor(
    cylc.flow.exceptions.CylcError: Clean failed: my-workflow

The clean command runs in a ProcessPoolExecutor. I am not sure how/if it is possible to capture the stdout/stderr of that command to display in the UI.

Another option may be to attach the info to the CylcError exception object that gets raised, in which case this needs to be done in cylc-flow

Pull requests welcome! This is an Open Source project - please consider contributing a bug fix yourself (please read CONTRIBUTING.md before starting any work though).

MetRonnie avatar Aug 22 '22 15:08 MetRonnie