Keeping diagnostic going after all sources are completed?

[xiangzhi]

I have a program that reads a PsiStore and does calculation on it. The program is pretty slow and end up taking way longer (~180 seconds) than the actual PsiStore stream length (~30 seconds). The PsiStore Importer finish publishing messages after the 30 second mark and the diagnostic stop around 50 second mark. The last message won't be created until the 300 second mark. Is there a way to keep diagnostic running till the end instead of just stopping at 50 sec?

Here's a screenshot of PsiStudio illustrating my problem. The first row is the messages from diagnostic. I made some changes to Visualization to make the time go beyond originating time interval. The changes can be seen here: https://github.com/CMU-TBD/psi/commit/389298d04ff5f8624949119e1130530299cfdceb

[AshleyF]

Hi Zhi, I can repro this behavior. It seems that it's the DiagnosticsSampler source component being stopped during regular pipeline shutdown. To unblock you, please try a workaround like the below to prevent the pipeline from shutting down:

pipeline.ProposeReplayTime(TimeInterval.LeftBounded(DateTime.UtcNow));
Generators.Repeat(pipeline, true, 2, TimeSpan.FromSeconds(500));

I've opened a work item to investigate a real solution (e.g. ensuring that DiagnosticsSampler is the very last component to be finalized).

Thanks for reporting this!

[xiangzhi]

Thanks Ashley, the workaround works! Hopefully we don't have add an additional generator in the future.

I'll keep the issue open until a complete solution is done.