python performance degrades over time

[mosherayman]

I have an application which reads very large pst files. I noticed a degradation in performance over time. To demonstrate this, i took the test.py file and deleted the print statements, and added the following lines to print out a message every 500 "items":

count += 1
if count % 500 == 0:
      logging.warn("count = %d" % count)

The output starts off "reasonable" :

2017-07-25 12:15:49,746 count = 500 2017-07-25 12:15:49,910 count = 1000 2017-07-25 12:15:50,143 count = 1500 2017-07-25 12:15:50,432 count = 2000 2017-07-25 12:15:50,744 count = 2500 2017-07-25 12:15:51,138 count = 3000 2017-07-25 12:15:51,553 count = 3500 2017-07-25 12:15:51,998 count = 4000 2017-07-25 12:15:52,686 count = 4500

but then deteriorates:

2017-07-25 12:20:26,865 count = 47000 2017-07-25 12:20:32,875 count = 47500 2017-07-25 12:20:39,700 count = 48000 2017-07-25 12:20:45,640 count = 48500 2017-07-25 12:20:51,255 count = 49000 2017-07-25 12:21:00,353 count = 49500 2017-07-25 12:21:11,214 count = 50000 2017-07-25 12:21:20,816 count = 50500 2017-07-25 12:21:30,561 count = 51000

Is goes from taking less than 1/2 second per 500 messages to over 9 seconds.

Do we know why this is happening?

thanks

[thinrunner]

Hi @mosherayman have you ever found a solution for this issue? Seems like I have the same issue.

Thanks.