campaign.recollate() produces an error; is there a more elegant solution for delayed collation (or recollation) than the hack used below?

[DavidPCoster]

Using campaign.recollate() produces the error

'Campaign' object has no attribute 'collate'

as shown in

Indeed collate now seems to be part of actions/action_statuses.py,

grep -R "def collate" easyvvuq/
easyvvuq//actions/action_statuses.py:    def collate(self, progress_bar=False):

Is there any clean way of recollating (or collating at a later stage)?

I am currently using the hack

decoder = uq.decoders.SimpleCSV(
    target_filename="output.csv",
    output_columns=["dpar", "te", "ti", "ne", "na_d0", "na_d1", "na_n0", "na_n1", "na_n2", "na_n3", "na_n4", "na_n5", "na_n6", "na_n7"]
)

for R in campaign.list_runs():
    RESULT = {'result': decoder.parse_sim_output(R[1]), 'run_info': 0, 'rundir': R[1]['run_dir']}
    campaign.campaign_db.store_result(R[0], RESULT, change_status=True)
campaign.campaign_db.session.commit()

results_df = campaign.get_collation_result()

which works but is, as I said above, a hack.

Is there a more elegant solution?

[orbitfold]

What is the use case here?

[DavidPCoster]

I first encountered the problem when analysing a crash somebody was experiencing, which was related to the first element of the vector (for every sample) being identical, and thus having a variance of zero. The ultimate fix was to prevent the divide by zero destroying the calculation of the remaining good quantities, but as a temporary fix I tried to edit the output CSV files and then tried to force a recollate -- resulting in the pasted error message.

I now have a use case where running the cases takes significant time and user intervention so I essentially have a split workflow:

1. EasyVVUQ[1]: prepare the input directories and a file containing the run parameters
2. SOLPS:setup the SOLPS directories, instantiate the parameters, run the cases, check the cases, continue the cases, prepare the output CSV file
3. EasuVVUQ[2]: decode, collate and analyse the output

I might also choose to analyse a different subset of the output data from SOLPS, producing a new output CSV file in step 2, and then needing to redo step 3.

The most suggestive option in the campaign description seemed to be the .recollate() method, which no longer works. I have found a work-around but it is not clean.

[orbitfold]

Is it not possible to do 2. as part of an EasyVVUQ workflow?

[DavidPCoster]

In principle, yes. In practise, no.

While parts could be automated with a script do all of 1, 2 & 3, the decision whether to continue runs is still made manually.

It is also useful to re-use the existing runs, just changing the QOIs which involves running a very small analysis code that transforms the large amount of data output from SOLPS into a different output.csv file.

[DavidPCoster]

In the particular case it takes about 1 weeks to run the SOLPS cases. In other cases it might take months -- which would require that the EasuVVUQ driver needs to be hanging around for that length of time (which cannot be guaranteed given the systems people wanting to install security patches periodically)

[orbitfold]

The basic idea would be as follows - create an actions chain that does the first part and a separate one that does the third part. Set the first part as active_app, run execute. Then set the third part as active_app and run execute again. The first chain will do the set-up. The second chain decoding, etc. In between do your thing. There might be some finer details I am missing but I've tested this in the past and it works. Best if you try and show me a partial solution along those lines then we can figure out what else needs to be done. Even if it takes months you can always resume from the database. All the actions chains are stored in the database as well (as serialized objects).

[DavidPCoster]

I tried that:

actions = uq.actions.Actions(
    uq.actions.CreateRunDirectory(root=campaign_work_dir, flatten=True),
    uq.actions.Encode(encoder)
)
...
my_campaign.execute().collate()

and I then tried a second block (new program)

actions = uq.actions.Actions(uq.actions.Decode(decoder))
...
my_campaign.execute().collate()

but was not able to get this to work ...

[orbitfold]

well it needs to work, so we need to figure out why that is and how to prevent it

[orbitfold]

What exactly doesn't work? Looking at the code again I see that I've added changes precisely for this case (for example the run status will only be set to collated after the Decode action is executed).

[DavidPCoster]

My recollection is that it wasn't clear how to tell the system that the actions had to be replaced with a new set of actions ...

[orbitfold]

Simplest way is to use the "replace_actions" method of the Campaign. It is by itself hacky but simple.

https://github.com/UCL-CCS/EasyVVUQ/blob/efa979589c58e844853c3d8dcafbc1f53f29f28c/easyvvuq/campaign.py#L270

[DavidPCoster]

I would have sworn that I tried that and it didn't work -- but I have just tried it again and it worked!

Thanks!