Programatically adding host header causes headers to be reused accross tests

[snake66]

Observed behaviour

Request header values are reused between tests when the headers for one stage have been modified by the test function.

Expected behaviour

Request headers should be reinitialized with correct values for each stage.

Details

I'm trying to add a default Host header to tests using the following code in my test funtion:

def test_default(ruleset, test, destaddr, port, protocol):
    runner = testrunner.TestRunner()
    try:
        for stage in test.stages:
            print("\n[+] stage.input.headers:", id(stage.input.headers), "{}".format(stage.input.headers))
            if 'Host' not in stage.input.headers:
                stage.input.headers = {'Host': 'example.com'} | stage.input.headers

            print("[+] stage.input.headers:", id(stage.input.headers), "{}".format(stage.input.headers))
            runner.run_stage(stage, None)

This works fine for the first test, but all the remaining tests fail, seemingly because they will reuse the exact same headers as the first test. I.e. Content-Length, Content-Type etc are not recalculated because they are already set before the test function is invoked.

There's no headers specified in the test declaration:

-
    test_title: sample-test-1
    stages:
      -
        stage:
          input:
            method: POST
            uri: /wp-admin/admin-ajax.php
            data: "some_param=some_value&another_param=another_value"
          output:
            status: 200
-
    test_title: sample-test-2
    stages:
      -
        stage:
          input:
            method: POST
            uri: /wp-admin/admin-ajax.php
            data: "some_param=some_longer_value&another_param=another_longer_value"
          output:
            status: 200
-
    test_title: sample-test-3
    stages:
      -
        stage:
          input:
            method: GET
            uri: /
          output:
            status: 200

Running the above tests give the following:

(.ftwenv) % ./.ftwenv/bin/py.test --destaddr localhost --port 1234 --ruledir ./test/yaml --capture=tee-sys --tb=no
=================================================================== test session starts ===================================================================
platform linux -- Python 3.10.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/user/src/test/ftwtest
plugins: ftw-1.3.0
collected 3 items                                                                                                                                         

test/test_default.py 
[+] stage.input.headers: 139685386732480 {'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 49}
[+] stage.input.headers: 139685371491520 {'Host': 'example.com', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 49}
.
[+] stage.input.headers: 139685386732480 {'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 49}
[+] stage.input.headers: 139685371824064 {'Host': 'example.com', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 49}
F
[+] stage.input.headers: 139685386732480 {'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 49}
[+] stage.input.headers: 139685371835456 {'Host': 'example.com', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 49}
F                                                                                                                            [100%]

================================================================= short test summary info =================================================================
FAILED test/test_default.py::test_default[ruleset1-sample-tests -- sample-test-2] - Failed: {'Host': 'example.com', 'Content-Type': 'application/x-www-f...
FAILED test/test_default.py::test_default[ruleset2-sample-tests -- sample-test-3] - Failed: {'Host': 'example.com', 'Content-Type': 'application/x-www-f...
=============================================================== 2 failed, 1 passed in 5.14s ===============================================================

Notice that the id of the headers dictionary before adding the Host header is the same in all stages, and that the content length is not recalculated after the first stage has run.

Adding a host header to the tests input sections, works as expected:

(.ftwenv) % ./.ftwenv/bin/py.test --destaddr localhost --port 1234 --ruledir ./test/yaml --capture=tee-sys --tb=no
=================================================================== test session starts ===================================================================
platform linux -- Python 3.10.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/user/src/test/ftwtest
plugins: ftw-1.3.0
collected 3 items                                                                                                                                         

test/test_default.py 
[+] stage.input.headers: 139927179816064 {'Host': 'example.com', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 49}
[+] stage.input.headers: 139927179816064 {'Host': 'example.com', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 49}
.
[+] stage.input.headers: 139927179817472 {'Host': 'example.com', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 63}
[+] stage.input.headers: 139927179817472 {'Host': 'example.com', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': 63}
.
[+] stage.input.headers: 139927179815488 {'Host': 'example.com'}
[+] stage.input.headers: 139927179815488 {'Host': 'example.com'}
.                                                                                                                            [100%]

==================================================================== 3 passed in 0.08s ====================================================================

In this case the headers dictionary for each stage has a different id, and the contents are calculated correctly.