RFC-0025 Improving incremental builds

[peterbell10]

This RFC proposes changes to ATen that will allow more granular header dependencies and tools to enforce their usage across the codebase which should greatly improve incremental and cached build performance.

Note that a lot of this has already been implemented and merged but this RFC should provide a complete picture of the motivation and how these PRs fit together.

• The stack starting with https://github.com/pytorch/pytorch/pull/63612 introduces TensorBase and makes various CUDA files TORCH_ASSERT_NO_OPERATORS compliant, as does https://github.com/pytorch/pytorch/pull/66620.
• The stack starting with https://github.com/pytorch/pytorch/pull/68246 generates the per-operator headers and makes various code-generated files TORCH_ASSERT_ONLY_METHOD_OPERATORS compliant.
• The unmerged stack starting with https://github.com/pytorch/pytorch/pull/68462 converts most of ATen to use per-operator headers.

Rendered version: https://github.com/pytorch/rfcs/blob/rfc-improve-build-times/RFC-0025-improving-incremental-builds.md

[albanD]

The CUDA builds are not mentioned in this doc. Does it need any special casing?

[peterbell10]

CUDA builds don't require any additional work for the main changes here. .cu files can use the per-operator headers as they would any other header and TensorBase was used first and foremost for cuda files.

One consideration is that include-what-you-use doesn't support cuda files, so using per-operator headers takes a little more manual work.