Memory Issues

[benedictjones]

Environment (OS, Python version, PySpice version, simulator)

Windows 10, Python 3.7.6, PySpice 1.4.2, ngspice-shared

Issue

I have set up a function to create a circuit and execute a dc or transient sweep on it. This allows me to feed in a list or parameters I wish to test for.

Expected Behaviour

The function is called and executes causing a small uptick in memory. This memory is then released.

Actual Behaviour

The memory seems to non get released after each sequential simulation (iterating over the input list of paramaters), causing it to ramp up over time. Instead, the memory is returned when the program finishes entirely. For a very large list of input parameters, the memory keeps increasing until it reaches it's maximum and then python throws a memory error.

If I call my function using multiprocessing Pool, allowing it to execute the function for the different input parameters, I find that the memory will tick up upon creation of the process, then be reclaimed once it is destroyed (saw-toothed memory usage seen below). While this works, it can lead to slower execution (especially if the simulation sweep is small) due to pool start up overhead.

Initially, I thought it was my code that had a memory leak. But switching to Windows Subsystem for Linux (WSL) and running using ngspice-subprocess leads to there being no gradual memory increase. I also tested running the program in WSL using ngspice-shared and again there was no gradual increase in memory usage.

This still might be down to my code, but it appears to be a problem with windows? Is there any way to completely exit/quit the shared library between sequential executions so memory is released? Any thoughts would be appreciated!

[benedictjones]

A similar issue (and solution) can be seen here but for ngspyce: https://github.com/ignamv/ngspyce/issues/11

[parhamk75]

@benedictjones I have such a problem with my code both when I run it in Windows, and Raspberry Pi OS; So it seems not to be a problem just in Windows :(

[benedictjones]

@benedictjones I cannot reproduce your crash and I suspect you made a mistake transposing my patch to your local installation.

That said, I believe that #246 is not a manifestation of this issue. You cannot produce a perceptible memory leak by just storing a hundred strings. The problem this PR fixes only comes into play when you execute several millions of commands without restart.

It sounds more like you are forgetting to call destroy() after you have run a simulation. Ngspice keeps all data from a simulation in memory unless you tell it with destroy() that you definitely do not need it any longer. If you simulate in a loop you should probably have a destroy() in the loop as well.

If your simulation produce a significant amount of data it is very easy to fill your memory by running just a few of them without deleting your previous results.

@ARF1 in reference to your response on #260 (copied above) while what you say makes complete sense, I can't seem to find a way to call destroy. With a basic circuit, such as:

import PySpice.Logging.Logging as Logging
from PySpice.Spice.Netlist import Circuit
from PySpice.Unit import *

# # create the circuit
circuit = Circuit('Voltage Divider')

# # add components to the circuit
circuit.V('input', 'in', circuit.gnd, 10@u_V) # @u_kΩ is a unit of Volts
circuit.R(1, 'in', 'out', 9@u_kΩ)  # @u_kΩ is a unit of kOhms
circuit.R(2, 'out', circuit.gnd, 1@u_kΩ)

# # create a simulator 
simulator = circuit.simulator(temperature=25, nominal_temperature=25)

# # Run an operating point simulation
analysis = simulator.operating_point()

do you know how destroy() might be called? Or do i need to build, execute & destroy my circuit using the PySpice.Spice.NgSpice.Shared.NgSpiceShared class directly? Any help would be very appreciated! Thanks

[ARF1]

@benedictjones To be honest, I have never used PySpice with circuits. That said, it should not make a difference, as NgSpice Shared will be used under the hood.

Try this:

Somewhere before your analysis loop:

from PySpice.Spice.NgSpice.Shared import NgSpiceShared
ngspice = NgSpiceShared.new_instance()

Then call destroy() in the loop in which you are running your analysis:

ngspice.destroy()

---

Example:

import PySpice.Logging.Logging as Logging
from PySpice.Spice.Netlist import Circuit
from PySpice.Unit import *
from PySpice.Spice.NgSpice.Shared import NgSpiceShared

# # create the circuit
circuit = Circuit('Voltage Divider')

# # add components to the circuit
circuit.V('input', 'in', circuit.gnd, 10@u_V) # @u_kΩ is a unit of Volts
circuit.R(1, 'in', 'out', 9@u_kΩ)  # @u_kΩ is a unit of kOhms
circuit.R(2, 'out', circuit.gnd, 1@u_kΩ)

# # create a simulator 
simulator = circuit.simulator(temperature=25, nominal_temperature=25)

ngspice = NgSpiceShared.new_instance()

my_simulation_parameters = list(range(1000))

# # Analysis loop
for param in my_simulation_parameters:
    # # Alter simulation parameters

    # # Run some simulation
    analysis = simulator.operating_point()

    # do something with your analysis results...

    # # Remove simulation results from memory
    ngspice.destroy()

I did not run the above example to check that it works, but it should at least give you an idea how to use destroy().

[benedictjones]

Thank you very much for your response and help!

I have tried what you suggested, but unfortunately it didn't seem to fix the issue. I'll stick to Linux for now when doing large simulations in a loop.

[GleidsonLeite]

Hi, i'm working with an optimization problem.

I saw this issue because i had the same problem with many iterations using simulations. Reading the documentation and doing some debugs i found a solution that work for me.

If you instantiate a simulator, you'll have:

simulator = self.circuit.simulator(
            temperature=temperature, nominal_temperature=nominal_temperature
        )

To call the destroy method, you should get the appropriate instance through the factory method. You can get the right ngspice instance like the following code

simulator = self.circuit.simulator(
            temperature=temperature, nominal_temperature=nominal_temperature
        )
analysis = simulator.ac(
    start_frequency=start_frequency,
    stop_frequency=stop_frequency,
    number_of_points=number_of_points,
    variation=variation,
)
ngspice = simulator.factory(self.circuit).ngspice
ngspice.remove_circuit() # i think you can remove this line
# destroying the ngspice instance
ngspice.destroy()

[benedictjones]

Thanks @GleidsonLeite! I can confirm that this seems to stop the memory leak problem on windows, and means you can execute long loops without the memory constantly increasing (and maybe causing an error when you max out).

I find the "ngspice.remove_circuit()" line is needed for this to work, where as "ngspice.destroy()" might not be.

I would suggest it's use like this:

while i < 1000:  # a long loop

    # Make a circuit
    circuit = Circuit('my_test')
    etc. 

    # make simulator 
    simulator = circuit.simulator(temperature=25, nominal_temperature=25)

    # perform some sort of simulation
    analysis = simulator.dc(Vin=slice(0,5,0.1))

    # extract data 
    print(np.array(analysis['out']))

    # If you are on a windows system
    if sys.platform == "win32":
        ngspice = simulator.factory(circuit).ngspice
        ngspice.remove_circuit()  # I find this is necessary!!!
        ngspice.destroy()  # this might not be needed

However, as I have found on my PySpice journey, in general I would probably suggest using linux.