httpx icon indicating copy to clipboard operation
httpx copied to clipboard

perf: using None checking cookies instead of bool(#2875)

Open Tunglies opened this issue 2 years ago • 0 comments
trafficstars

Summary

Performance improve. Related #2875 Before

         16002280 function calls (14002219 primitive calls) in 10.641 seconds

   Ordered by: internal time
   List reduced from 225 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  2000002    2.431    0.000    2.898    0.000 cookiejar.py:1679(set_cookie)
  1000001    2.243    0.000    6.808    0.000 _models.py:1057(set)
  1000001    1.289    0.000    1.599    0.000 copy.py:66(copy)
  1000001    1.219    0.000    2.914    0.000 cookiejar.py:761(__init__)
3000009/1000003    1.089    0.000    1.089    0.000 cookiejar.py:1227(deepvalues)
        1    0.532    0.532    0.532    0.532 perf_test.py:11(<dictcomp>)
        1    0.386    0.386   10.056   10.056 _models.py:1147(update)
        3    0.381    0.127    7.189    2.396 _models.py:1024(__init__)
  2000003    0.277    0.000    0.277    0.000 {method 'acquire' of '_thread.RLock' objects}
  1000001    0.207    0.000    0.207    0.000 {method 'copy' of 'dict' objects}

Now

         2245 function calls (2190 primitive calls) in 0.539 seconds

   Ordered by: internal time
   List reduced from 221 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.505    0.505    0.505    0.505 perf_test.py:11(<dictcomp>)
        3    0.029    0.010    0.029    0.010 {method 'load_verify_locations' of '_ssl._SSLContext' objects}
       14    0.001    0.000    0.001    0.000 {built-in method __new__ of type object at 0x00007FF9B50B8F90}
        3    0.001    0.000    0.001    0.000 {built-in method nt.stat}
        3    0.000    0.000    0.000    0.000 {method 'set_ciphers' of '_ssl._SSLContext' objects}
        6    0.000    0.000    0.000    0.000 enum.py:1374(_missing_)
        1    0.000    0.000    0.000    0.000 {built-in method winreg.OpenKey}
        5    0.000    0.000    0.000    0.000 _urlparse.py:149(urlparse)
      126    0.000    0.000    0.000    0.000 <frozen os>:674(__getitem__)
        3    0.000    0.000    0.032    0.011 default.py:119(__init__)

Benchmark

import io
import pstats
import cProfile
import httpx

merge_cookies_times = 1_000_000

pr = cProfile.Profile()
pr.enable()

httpx.Client(cookies={"default":"none"})._merge_cookies(cookies={f"{k}":f"{v}" for k,v in zip(range(merge_cookies_times), range(merge_cookies_times))})

pr.disable()
s = io.StringIO()
ps = (pstats.Stats(pr, stream=s)
      .strip_dirs()
      .sort_stats("tottime"))
ps.print_stats(10)
print(s.getvalue())

Github Action Workflow test all passed.

Checklist

  • [x] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
  • [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • [ ] I've updated the documentation accordingly.

Tunglies avatar Oct 14 '23 11:10 Tunglies