micropython-lib icon indicating copy to clipboard operation
micropython-lib copied to clipboard

Possible memory leak in urquests.get and others.

Open emacstheviking opened this issue 1 year ago • 11 comments

Hi, I just answered a Reddit issue where somebody has some very trivial code in a loop, and it runs out of memory after a while, the reddit page is here:

https://www.reddit.com/r/raspberrypipico/comments/12v8k0a/oserror_errno_12_enomem/?sort=new

I walked through the code and on line 93 of urequests.py, there is an allocation taking place in wrap_socket(), assigned to s and then not seemingly released after s.close() has been called. I have only looked at the code for aobut half an hour so it's entirely possible the deallocation is done somewhere else but wrap_socket returns a handle to the memory, that's s, and there seems to be no corresponding call to free it.

https://github.com/micropython/micropython-lib/blob/master/python-ecosys/urequests/urequests.py

emacstheviking avatar Apr 22 '23 15:04 emacstheviking

looking at the thread why do you assume this is a https connection wrap_socket would only be used if we were using https

the example code posted uses example.com, without a protocol specified

GM-Script-Writer-62850 avatar Apr 22 '23 16:04 GM-Script-Writer-62850

I had failed to notice that. It's not my code that has the problem, but on closer inspection that's correct. In which case I apologise for wasting your time with it. Where does that memory get freed then ?

It is of course entirely possible that the code posted has other mismtached allocate/deallocate pairings the author is unaware of.

Thanks for your time, as an intermittent user of micropython myself, I greatly appreciate the amount of effort put into it.

emacstheviking avatar Apr 22 '23 16:04 emacstheviking

i found this from the same reddit thread, not saying this is not a issue, but it may not be the issue you are looking for, seems like something worth checking

GM-Script-Writer-62850 avatar Apr 22 '23 16:04 GM-Script-Writer-62850

For many years I was an embedded systems designer / developer and memory allocation is one of those things that can bite you eventually in a long running process! All the best with it.

emacstheviking avatar Apr 22 '23 16:04 emacstheviking

turns out the OP was using https

GM-Script-Writer-62850 avatar Apr 22 '23 16:04 GM-Script-Writer-62850

Wow, so I might have been right after all?! There's always a first time for everything. Can I get that in writing to show the wife next time we have an argument please.

emacstheviking avatar Apr 22 '23 21:04 emacstheviking

if you pay the postage ;) BTW it will look bad if you pay for international shipping just to prove to your wife you MAY have been right, i would suggest just letting her win it makes things easier

GM-Script-Writer-62850 avatar Apr 22 '23 21:04 GM-Script-Writer-62850

if you pay the postage ;) BTW it will look bad if you pay for international shipping just to prove to your wife you MAY have been right, i would suggest just letting her win it makes things easier

Yeah but, being a software developer means I deal in logic and truth and false being recognised...sigh...yeah, after 30 years you'd think I'd have learned. :D

emacstheviking avatar Apr 23 '23 08:04 emacstheviking

So why close the report? we know OP used https now

could email you a pdf

here is your boolean logic:

if correct or wife_said_so and not finical_ruin: 
  print("OK")
elif wife_said_so:
  print("I see your point that this is a bad idea cause of",X)
else:
  print("your wrong")

GM-Script-Writer-62850 avatar Apr 23 '23 12:04 GM-Script-Writer-62850

So why close the report? we know OP used https now

could email you a pdf

here is your boolean logic:

if correct or wife_said_so and not finical_ruin: 
  print("OK")
elif wife_said_so:
  print("I see your point that this is a bad idea cause of",X)
else:
  print("your wrong")

Didn't realise I'd closed it, my bad, I mean why destroy valuable ammo for the future ?

emacstheviking avatar Apr 23 '23 12:04 emacstheviking

The SSL socket's allocated memory is freed after the stream is closed (via an ioctl), then the ioctl is passed to the wrapped socket, which also frees its allocated memory.

See modssl_axtls.c, modssl_mbedtls.c and modsocket.c for details.

ned-pcs avatar Jun 12 '23 14:06 ned-pcs