ironpython2 icon indicating copy to clipboard operation
ironpython2 copied to clipboard

Import not thread safe

Open tinyg opened this issue 5 years ago • 2 comments

Description

The import mechanism does not appear to be thread safe under heavy load.

Running the code:

var code = 
@"def method1():
    from System.Diagnostics import Stopwatch;
    from collections import Iterable";

var _engine = IronPython.Hosting.Python.CreateEngine();

var searchPaths = _engine.GetSearchPaths();
searchPaths.Add(@"C:\TFS\SDKS\ironpython\2.7.9\IronPython.StdLib.2.7.9.zip");
_engine.SetSearchPaths(searchPaths);

var runtime = _engine.Runtime;
var scope = _engine.CreateScope();
var source = _engine.CreateScriptSourceFromString(code, SourceCodeKind.AutoDetect);
source.Execute(scope);
var method = scope.GetVariable("method1");

Parallel.ForEach(Enumerable.Range(1, 1000), new ParallelOptions { MaxDegreeOfParallelism = 1000 }, (i) => method());

produces errors along the lines of "failed to import ...." or "error importing..." at least once in 1000 concurrent runs.

Steps to Reproduce

  1. Use LinqPad or add code into the main method of a console app
  2. Run
  3. Error is generated

Expected behavior:

Code runs to completion successfully

Actual behavior:

Error is thrown

Versions

IronPython 2.7.9 2.7.9.0 on .NET 4.0.30319.42000

tinyg avatar Sep 23 '19 22:09 tinyg

Workaround is to run this code (only once!)

import clr

def _():
    from System.Threading import Monitor

    import __builtin__

    __lock = object()
    base_import = __builtin__.__import__

    def new_import(name, globals={}, locals={}, fromlist=[], level=-1):
        Monitor.Enter(__lock)
        try:
            return base_import(name, globals, locals, fromlist, level)
        finally:
            Monitor.Exit(__lock)

    __builtin__.__import__ = new_import
_()
del _

def method1():
    from System.Diagnostics import Stopwatch
    from collections import Iterable

tinyg avatar Sep 23 '19 22:09 tinyg

Note that this code causes issues with using the clrtypes metaclass.

Suggested fix is to add a lock around the contents of import in Src\IronPython\Modules\Builtin.cs

tinyg avatar Oct 08 '19 12:10 tinyg