caching fixtures for easier debugging

[jhutar]

I'm creating a test which depends on a setup (via @pytest.fixture(scope='session')) which takes, say, 15 minutes to run. That is quite expensive to learn that I have made another error or I have not used some object correctly. And on each try, one more organization and location and domain and ... is created in my satellite.

Yes, I could use debugger and breakpoint or some interactive console and maybe everybody is happy with that when creating new test (?). I have created this decorator:

+import os
+from functools import wraps
+import pickle
+
+def debug_caching(func):
+
+    @wraps(func)
+    def wrapper(*args, **kwargs):
+        import os
+        # If debug caching is not enabled, simply just run the wrapped function and return
+        if 'DEBUG_CACHING' not in os.environ:
+            LOGGER.debug('Debug caching not enabled, just running %s' % func.__name__)
+            return func(*args, **kwargs)
+
+        # If debug caching is enabled
+        try:
+            with open('/tmp/debug_caching_data.pickle', 'rb') as fp:
+                debug_caching_data = pickle.load(fp)
+        except FileNotFoundError:
+            debug_caching_data = {}
+        LOGGER.debug('Loaded debug cache with %d items' % len(debug_caching_data))
+
+        if func.__name__ in debug_caching_data:
+            LOGGER.debug('Returning object %s from debug cache' % func.__name__)
+            return debug_caching_data[func.__name__]
+        else:
+            LOGGER.debug('Running %s and saving to debug cache' % func.__name__)
+            result = func(*args, **kwargs)
+            debug_caching_data[func.__name__] = result
+            with open('/tmp/debug_caching_data.pickle', 'wb') as fp:
+                pickle.dump(debug_caching_data, fp)
+            return result
+
+    return wrapper

and added this to all the fixtures I'm using:

 @pytest.fixture(scope='session')
+@debug_caching
 def user_credentials():
     # Create a new user with admin permissions
     login = gen_string('alphanumeric')
@@ -64,6 +102,7 @@ def user_credentials():
 
 
 @pytest.fixture(scope='session')
+@debug_caching
 def org(user_credentials):
     org = entities.Organization(user_credentials).create()
     with manifests.clone() as manifest:
@@ -72,6 +111,7 @@ def org(user_credentials):
 
 
 @pytest.fixture(scope='session')
+@debug_caching
 def loc(user_credentials, org):
     loc = entities.Location(user_credentials).create()
     loc.organization = [org]

It runs the fixture and saves its return value to a pickle file on a first run. Then, on every subsequent run it only reads that object from the pickle and runs mine test with that, so I can run the test in about second.

Does anybody else thinks it could be useful? Shall I/we invest more in exploring this path, or is this a bad practice with simple and better workaround?

[sghai]

I see value in it if it saves time on test execution and avoid creating new Org/Location entities.

[swadeley]

Hello, I can see that not creating new random orgs all the time is useful when running tests.

This would have saved me time in the past when working on debugging UI sync plan tests, every time I ran the test I had to grep for the newly created org.

Thank you

[jhutar]

It saves time (15 min -> 1sec in my case) only when debugging/developing a test - when running it again and again and when fixtures are not to blame. Actually I'm bit supprised it works :)

[rplevka]

Looks great - will you raise PR for it?

[vijay8451]

Overall idea looks great and time saving however having below query on it:

1. What if same fixture called by mutiple tests ? in that case it always deliver the same pickle object of first test (If I understood the above mentions example correctly)

[JacobCallahan]

Just a heads up, your example code uses %s instead of .format for string formatting. You'll need to change that for the actual PR.