azure-sdk-for-cpp icon indicating copy to clipboard operation
azure-sdk-for-cpp copied to clipboard

[Core] Update azure-test to use test-proxy

Open vhvb1989 opened this issue 2 years ago • 2 comments

The test framework is currently implementing recording and playback for unit tests. The implementation includes pipeline policies to create the recordings and to consume them for playback.

The recording policy does not provide sanitizer features (automatically removing sensitive information like id, password, etc). The reason for this is that it is easier to remove the Azure resources used to create the recording data. The policy has the capability to only convert dynamic inputs (like UUID, dates, etc) into fixed constants.

The C++ test framework needs to be updated to use the test-proxy from Azure SDK tools (https://github.com/Azure/azure-sdk-tools/tree/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy), following what all other languages are doing.

The recording policy should redirect the requests to the test-proxy, and set the path for storing the recordings. The playback policy should redirect to the request as well to test-proxy and use the response from it.

The performance-framework for C++ can already use the test-proxy (without creating recording data). The code can be re-used between the performance-framework and the test-framework to interact with the test-proxy.

vhvb1989 avatar May 27 '22 20:05 vhvb1989

@vhvb1989 is the new test-proxy more powerful than the current one? Or they just provide the same functionalities with different interface? Like can I use random container/blob names, and does it support requests from multiple threads?

Jinming-Hu avatar May 27 '22 23:05 Jinming-Hu

@vhvb1989 is the new test-proxy more powerful than the current one? Or they just provide the same functionalities with different interface? Like can I use random container/blob names, and does it support requests from multiple threads?

I am not 100% aware of all the features. I know this is a cross-language solution, and it is out-of-process (the record and playback comes from an actual HTTP server, instead of being generated by the test-process).

I know it brings Sanitizers, matchers and transformations, You might want to take a look to those features.

And there's a backlog of features to be added to it. You can also create requests or questions for it.

pd. It might bring some extra steps at the beginning, as we will need to launch the test-proxy before running the tests (either with docker or using dotnet), but we should be able to ask CMake to launch it if -DBUILD_TESTING=ON (something to evaluate in the future)

vhvb1989 avatar May 28 '22 04:05 vhvb1989