storage: upgrade giovanni SDK and support AAD auth

[manicminer]

Storage Data Plane upgrades and support for AAD authentication

• Use latest tombuildsstuff/giovanni which has switched to the go-azure-sdk base layer
• Support for sdk-level retries for known eventually consistent scenarios
• Support for AAD authentication for all data plane APIs (where the API supports it)
• Refactor data plane client helpers to be operation-aware, so that supported authentication methods can be determined not just by endpoint but also by the capabilities of the operation (enables more granular support for preferred authentication methods).
• Adopt data plane resource IDs from tombuildsstuff/giovanni- Some tidying of provider package - move helper functions into own source file
• Some tidying of services/storage/client package - move data plane client helpers into own source file and add support for parsing out a storage account endpoint for constructing a data plane ID
• internal/common: adopt client.BaseClient interface from go-azure-sdk to support more SDK base clients
• internal/provider: move helper functions into own source file

CHANGELOG

• dependencies: updating to v0.20240227.1200200 of github.com/hashicorp/go-azure-sdk
• dependencies: updating to v0.25.0 of github.com/tombuildsstuff/giovanni
• data.azurerm_storage_table_entities - support for AAD authentication
• data.azurerm_storage_table_entity - support for AAD authentication
• azurerm_storage_share_file - support for AAD authentication
• azurerm_storage_share_directory - support for AAD authentication, deprecate share_name and storage_account_name in favor of storage_share_id
• azurerm_storage_table_entity - support for AAD authentication, deprecate share_name and storage_account_name in favor of storage_table_id
• azurerm_storage_table - support for AAD authentication
• azurerm_storage_table_entity - support for AAD authentication

Depends on:

• [x] https://github.com/hashicorp/go-azure-sdk/pull/876
• [x] https://github.com/hashicorp/go-azure-sdk/pull/890
• [x] https://github.com/tombuildsstuff/giovanni/pull/101
• [x] https://github.com/tombuildsstuff/giovanni/pull/104
• [x] https://github.com/hashicorp/go-azure-sdk/pull/902
• [x] https://github.com/hashicorp/go-azure-sdk/pull/906
• [x] https://github.com/tombuildsstuff/giovanni/pull/106
• [x] https://github.com/hashicorp/terraform-provider-azurerm/pull/25055

Note: this updates the go-azure-sdk version to pull in the SharedKeyAuthorizer fix, but can be squashed out if this gets added in main before this PR is merged

[katbyte]

does this close #24213? and unblock #22583?

[manicminer]

@katbyte Yes and yes 👍

[manicminer]

~Still having some issues with tables, possible state migration needed for Table IDs~

^ this now resolved, PR is ready for review

[manicminer]

Test results

(Legacy VM failures unrelated to this change)

[dkuzmenok]

@manicminer Would it somehow fix this famous issue - https://github.com/hashicorp/terraform-provider-azurerm/issues/2977 ?

[manicminer]

@dkuzmenok Unfortunately not, the storage resources still use the storage data plane APIs as we will only be in a position to switch when there is feature parity in the resource manager API.

[manicminer]

Latest test results

[github-actions[bot]]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.