Don't do a DeepCopy in the remote state backend to avoid double copying

[alexott]

The updateStateHook performs DeepCopy to create a state object that is then pushed to the StateMgr.WriteState. If we use remote state backend, then it's doing DeepCopy again leading to increased pressure on the garbage collection in case of big states (couple of hundreds megabytes).

Fixes #35113

Target Release

1.8.x

Draft CHANGELOG entry

NEW FEATURES | UPGRADE NOTES | ENHANCEMENTS | BUG FIXES | EXPERIMENTS

• Bug fix: improve performance of remote state backend on huge plans

[alexott]

@jbardin I thought about this, but it's a really big change - theoretically, we'll need to isolate the state under the interface, and implement copy-on-write for it and objects inside the state.

[alexott]

@jbardin would it help if we put this change under the feature flag?

[jbardin]

@alexott,

Honestly I don't know right now. The simple change of removing the copy looks suspect, because that method can be called from many locations, not just the update hook. If we can prove there is no race or unexpected modification of cached state in all cases then it's something we can consider, but until then we can't be sure this doesn't have the possibility of damaging a user's state. In all likelihood if there is a solution to remove an extra copy we will adopt that.

[jbardin]

Hi @alexott, I have an alternative proposal in #35164. This would effectively remove the extra copy for all implementations, while retaining the existing copy semantics of WriteState.

[jbardin]

superseded by #35164

[github-actions[bot]]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.