Reduce the execution time of Remap ACL AuTest

[masaori335]

We have ton of test cases in reamp_acl.test.py (AuTest), each of test cases starts and stop ATS and origin server. It takes long long time (over 5 min).

https://github.com/apache/trafficserver/blob/3f949f717b7fe8ef1d6aabd70aa18d340f0da342/tests/gold_tests/remap/remap_acl.test.py#L475-L495

This is just a random idea, but probably, we can have few big remap.configs that has many remap rules that represents combinations and keep ATS running during the test.

[a-shaikh12]

@masaori335 I am Interested in Contributing to Issue #11917

[masaori335]

@a-shaikh12 You're welcome! If you need help, let's chat on slack or somewhere :)

[ysinghc]

@masaori335 I am not new to open source but a little new to contributing on big repos. I do have some experience with python, not too extensive with writing python tests but I would love to learn through this issue would that be ok?

[masaori335]

@ysinghc Contribution is welcome. Please go through how to build ATS and run AuTest in this page. The first step will be running the "remap_acl" AuTest against ATS. https://github.com/apache/trafficserver/tree/master/tests

I thought this is "good first issue", but seems like required knowledge of working this issue is a bit high. So far, at least two people tried (thanks!), but unfortunately no success yet.

[ysinghc]

@masaori335 no worries, I chose this cause it seems something whose code i can understand and probably fix. I will go through the building process of ATS and run AuTest. I just need someone to review my steps through the process and correct me before i raise a PR that might break the system.

[ysinghc]

@masaori335 I have installed the dependencies and build the ATS. i also ran the remap_acl test on the ats-dev/bin and it has passed. What should i do next?

[masaori335]

@ysinghc cool! then, you observed running remap_acl test took long time. the goal is shorten it.

One of my guess is when Test_remap_acl is defined it starts and stops ATS every time, that's the overhead. An idea is combining tests and keep ATS running.

For example we have 2 tests:

test_ip_allow_optional_methods = Test_remap_acl(
    "Verify add_allow adds an allowed method.",
    replay_file='remap_acl_get_post_allowed.replay.yaml',
    ip_allow_content=IP_ALLOW_CONTENT,
    deactivate_ip_allow=False,
    acl_behavior_policy=1,
    acl_configuration='@action=add_allow @src_ip=127.0.0.1 @method=POST',
    named_acls=[],
    expected_responses=[200, 200, 403, 403, 403],
    proxy_protocol=False)

test_ip_allow_optional_methods = Test_remap_acl(
    "Verify add_allow adds allowed methods.",
    replay_file='remap_acl_get_post_allowed.replay.yaml',
    ip_allow_content=IP_ALLOW_CONTENT,
    deactivate_ip_allow=False,
    acl_behavior_policy=1,
    acl_configuration='@action=add_allow @src_ip=127.0.0.1 @method=GET @method=POST',
    named_acls=[],
    expected_responses=[200, 200, 403, 403, 403],
    proxy_protocol=False)

https://github.com/apache/trafficserver/blob/447a2a01961c4c1896f7a2b6c66eb4aefad07496/tests/gold_tests/remap/remap_acl.test.py#L258C1-L278C26

The difference is acl_configuration (and expected_responses), so if we can make it list like this, I expect we can reduce the overhead.

test_ip_allow_optional_methods = Test_remap_acl(
    "Verify add_allow adds allowed methods.",
    replay_file='remap_acl_get_post_allowed.replay.yaml',
    ip_allow_content=IP_ALLOW_CONTENT,
    deactivate_ip_allow=False,
    acl_behavior_policy=1,
    test_cases=[
        ['@action=add_allow @src_ip=127.0.0.1 @method=POST', [200, 200, 403, 403, 403]],
        ['@action=add_allow @src_ip=127.0.0.1 @method=GET @method=POST'],  [200, 200, 403, 403, 403]], 
    named_acls=[],
    proxy_protocol=False)

This change requires

• generate remap rules per list like

map /test_1/ http://127.0.0.1:8080/ @action=add_allow @src_ip=127.0.0.1 @method=POST
map /test_2/ http://127.0.0.1:8080/ @action=add_allow @src_ip=127.0.0.1 @method=GET @method=POST

• remap_acl_get_post_allowed.replay.yaml needs to make more requests against each remap rules.

[ysinghc]

@masaori335 I will start working on it

[ysinghc]

@masaori335 I have a very stupid question to ask I have used this command to run the test inside the tests folder in the root directory

./autest.sh -D gold_tests --filter remap_acl --ats-bin ~/ats-dev/bin

is this the correct command or is there a different way to actually test it using the autest suite

[masaori335]

NP. That's the command when I want to run specific test.