HPCC-31649 New StSizePeakEphemeralStorage and StSizePeakSubgraphTemp look ahead

[shamser]

Type of change:

• [ ] This change is a bug fix (non-breaking change which fixes an issue).
• [x] This change is a new feature (non-breaking change which adds functionality).
• [ ] This change improves the code (refactor or other change that does not change the functionality)
• [ ] This change fixes warnings (the fix does not alter the functionality or the generated code)
• [ ] This change is a breaking change (fix or feature that will cause existing behavior to change).
• [ ] This change alters the query API (existing queries will have to be recompiled)

Checklist:

• [x] My code follows the code style of this project.
  • [x] My code does not create any new warnings from compiler, build system, or lint.
• [x] The commit message is properly formatted and free of typos.
  • [x] The commit message title makes sense in a changelog, by itself.
  • [x] The commit is signed.
• [ ] My change requires a change to the documentation.
  • [ ] I have updated the documentation accordingly, or...
  • [ ] I have created a JIRA ticket to update the documentation.
  • [ ] Any new interfaces or exported functions are appropriately commented.
• [ ] I have read the CONTRIBUTORS document.
• [ ] The change has been fully tested:
  • [ ] I have added tests to cover my changes.
  • [ ] All new and existing tests passed.
  • [ ] I have checked that this change does not introduce memory leaks.
  • [ ] I have used Valgrind or similar tools to check for potential issues.
• [ ] I have given due consideration to all of the following potential concerns:
  • [ ] Scalability
  • [ ] Performance
  • [ ] Security
  • [ ] Thread-safety
  • [ ] Cloud-compatibility
  • [ ] Premature optimization
  • [ ] Existing deployed queries will not be broken
  • [ ] This change fixes the problem, not just the symptom
  • [ ] The target branch of this pull request is appropriate for such a change.
• [ ] There are no similar instances of the same problem that should be addressed
  • [ ] I have addressed them here
  • [ ] I have raised JIRA issues to address them separately
• [ ] This is a user interface / front-end modification
  • [ ] I have tested my changes in multiple modern browsers
  • [ ] The component(s) render as expected

Smoketest:

• [ ] Send notifications about my Pull Request position in Smoketest queue.
• [ ] Test my draft Pull Request.

Testing:

[github-actions[bot]]

Jira Issue: https://hpccsystems.atlassian.net//browse/HPCC-31649

Jirabot Action Result: Workflow Transition: Merge Pending Updated PR

[jakesmith]

@shamser - infact this also covers a hashdistribute spilling, which is also used as part of hash dedup (so https://hpccsystems.atlassian.net/browse/HPCC-28757 alone won't give a full picture of a hash dedup;s spilling).

Can you update this JIRA and it's PR title to mention that it covers hash distribute spilling as well?

[shamser]

@shamser - please see comments.

Some changes made in https://github.com/hpcc-systems/HPCC-Platform/pull/18573 will mean that this PR will need to be updated. I'll update this PR once 18573 is merged. @jakesmith

[jakesmith]

@shamser - there's a clash now, can you rebase