interactive icon indicating copy to clipboard operation
interactive copied to clipboard

getting stuck

Open superichmann opened this issue 2 years ago • 16 comments
trafficstars

need to run initialize cell 3 times before it hits notebooks will run infinity, need to restart vscode

can you fix?

superichmann avatar Oct 20 '23 08:10 superichmann

Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension ---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'. at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.OnLoadAsync(Kernel kernel) at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117 --- End of inner exception stack trace ---

superichmann avatar Oct 20 '23 09:10 superichmann

oh and clicking stop does not actually stop the execution

superichmann avatar Oct 20 '23 09:10 superichmann

Error: Microsoft.DotNet.Interactive.NoSuitableKernelException: No kernel found for Microsoft.DotNet.Interactive.Commands.RequestHoverText with target kernel '.NET'. at Microsoft.DotNet.Interactive.Commands.KernelCommand.InvokeAsync(KernelInvocationContext context) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\Commands\KernelCommand.cs:line 184 at Microsoft.DotNet.Interactive.Kernel.HandleAsync(KernelCommand command, KernelInvocationContext context) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\Kernel.cs:line 323 at Microsoft.DotNet.Interactive.CompositeKernel.HandleAsync(KernelCommand command, KernelInvocationContext context) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\CompositeKernel.cs:line 183 at Microsoft.DotNet.Interactive.KernelCommandPipeline.<BuildPipeline>b__6_0(KernelCommand command, KernelInvocationContext context, KernelPipelineContinuation _) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 60 at Microsoft.DotNet.Interactive.KernelCommandPipeline.<>c__DisplayClass6_1.<<BuildPipeline>b__3>d.MoveNext() in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 75 --- End of stack trace from previous location --- at Microsoft.DotNet.Interactive.App.KernelExtensions.<>c__DisplayClass3_0.<<UseTelemetrySender>b__0>d.MoveNext() in D:\a_work\1\s\src\dotnet-interactive\KernelExtensions.cs:line 124 --- End of stack trace from previous location --- at Microsoft.DotNet.Interactive.KernelCommandPipeline.<>c__DisplayClass6_1.<<BuildPipeline>b__3>d.MoveNext() in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 75 --- End of stack trace from previous location --- at Microsoft.DotNet.Interactive.App.KernelExtensionLoader.<>c__DisplayClass0_0.<<UseNuGetExtensions>b__0>d.MoveNext() in D:\a_work\1\s\src\dotnet-interactive\KernelExtensionLoader.cs:line 21 --- End of stack trace from previous location --- at Microsoft.DotNet.Interactive.KernelCommandPipeline.<>c__DisplayClass6_0.<<BuildPipeline>g__Combine|2>d.MoveNext() in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 73 --- End of stack trace from previous location --- at Microsoft.DotNet.Interactive.KernelCommandPipeline.<>c__DisplayClass6_0.<<BuildPipeline>g__Combine|2>d.MoveNext() in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 73 --- End of stack trace from previous location --- at Microsoft.DotNet.Interactive.KernelCommandPipeline.SendAsync(KernelCommand command, KernelInvocationContext context) in D:\a_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 41 Loading extensions from C:\Users\x\.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactiv

superichmann avatar Oct 20 '23 09:10 superichmann

so many bugs...

superichmann avatar Oct 20 '23 09:10 superichmann

Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension ---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'. at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.OnLoadAsync(Kernel kernel) at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117 --- End of inner exception stack trace ---

Looks like this may be happening because you are using an older version of the AutoML package against latest version of polyglot notebooks extension.

Could you try updating to the latest version from nuget.org? This version which was released last week may fix the error https://www.nuget.org/packages/Microsoft.ML.AutoML/0.21.0-preview.23511.1https://www.nuget.org/packages/Microsoft.ML.AutoML/0.21.0-preview.23511.1.

shyamnamboodiripad avatar Oct 20 '23 19:10 shyamnamboodiripad

Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension ---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'. at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.OnLoadAsync(Kernel kernel) at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117 --- End of inner exception stack trace ---

This is the same issue as #3244 and #2366. We are currently investigating a fix for this.

shyamnamboodiripad avatar Oct 20 '23 19:10 shyamnamboodiripad

@shyamnamboodiripad is there already a branch created for this fix in another fork?

cartercanedy avatar Oct 21 '23 05:10 cartercanedy

Error: Failed to load kernel extension "AutoMLMonitorKernelExtension" from assembly C:\Users\x.nuget\packages\microsoft.ml.automl\0.21.0-preview.23406.1\interactive-extensions\dotnet\Microsoft.ML.AutoML.Interactive.dll Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension ---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'. at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel) at Microsoft.ML.AutoML.AutoMLMonitorKernelExtension.OnLoadAsync(Kernel kernel) at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117 --- End of inner exception stack trace ---

Looks like this may be happening because you are using an older version of the AutoML package against latest version of polyglot notebooks extension.

Could you try updating to the latest version from nuget.org? This version which was released last week may fix the error https://www.nuget.org/packages/Microsoft.ML.AutoML/0.21.0-preview.23511.1https://www.nuget.org/packages/Microsoft.ML.AutoML/0.21.0-preview.23511.1.

Just saw this, nevermind

cartercanedy avatar Oct 21 '23 15:10 cartercanedy

I have noticed that the hanging is still persistent, though

cartercanedy avatar Oct 21 '23 15:10 cartercanedy

Could you please clarify the exact repro steps for the hang? Am I correct in assuming that the hang only happens when the AutoML extension is in play? Is there a particular notebook that you are able to repro the hang with?

shyamnamboodiripad avatar Oct 22 '23 01:10 shyamnamboodiripad

image

That code submission should've only taken 10 secs (per awaiting that sleeping promise), but it only completes on the first submission, any subsequent submissions of the unaltered code hang.

cartercanedy avatar Oct 22 '23 01:10 cartercanedy

Thanks for sharing the repro @cartercanedy. I am also able to repro this with the code you shared below. Sometimes I am able to cancel execution, other times it hangs and yet other times, the cell completes.

sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
await sleep(10000);
const someVar = "Hello, world";

We will take a look - more than half of our team is away this week so it may be next week before we can get to this.

(cc @colombod FYI looks like there are some more problems beyond what we fixed in #3268)

shyamnamboodiripad avatar Oct 23 '23 17:10 shyamnamboodiripad

@superichmann Is the hang you were running into above similar to the one @cartercanedy reported above? Are you also running into the hang in JavaScript? Please share a repro if you think it is different.

shyamnamboodiripad avatar Oct 23 '23 18:10 shyamnamboodiripad

TLDR: FIXED, changed Microsoft Data analysis with version 0.21.0-preview.23511.1

I am keeping this open in case you want to change the infinite run behavior and if you would want to reproduce. feel free to close.

This is exactly what is happening on my side:

  1. I am creating new notebook
  2. I save the notebook as a file (ipynb)
  3. I add this to the first cell:
Console.WriteLine("Initializing...");
#r "nuget: Microsoft.ML,3.0.0-preview.23511.1"
#r "nuget: Microsoft.ML.TimeSeries,3.0.0-preview.23511.1"
#r "nuget: Microsoft.Data.Analysis"
#r "nuget: Microsoft.ML.AutoML,0.21.0-preview.23511.1"
#r "nuget: Microsoft.ML.OneDal,0.21.0-preview.23511.1"
#r "nuget: npgsql"
#r "nuget: Telegram.Bot"
#r "nuget: MathNet.Numerics"
  1. I run the first cell, get this output after 4 seconds:
Installed Packages
MathNet.Numerics, 5.0.0
Microsoft.Data.Analysis, 0.20.1
Microsoft.ML, 3.0.0-preview.23511.1
Microsoft.ML.AutoML, 0.21.0-preview.23511.1
Microsoft.ML.OneDal, 0.21.0-preview.23511.1
Microsoft.ML.TimeSeries, 3.0.0-preview.23511.1
npgsql, 7.0.6
Telegram.Bot, 19.0.0
Loading extensions from `C:\Users\X\.nuget\packages\microsoft.data.analysis\0.20.1\interactive-extensions\dotnet\Microsoft.Data.Analysis.Interactive.dll`
Error: Failed to load kernel extension "DataFrameKernelExtension" from assembly C:\Users\X\.nuget\packages\microsoft.data.analysis\0.20.1\interactive-extensions\dotnet\Microsoft.Data.Analysis.Interactive.dll
Error: Microsoft.DotNet.Interactive.KernelExtensionLoadException: Failure loading Kernel Extension
---> System.MissingMethodException: Method not found: 'Void Microsoft.DotNet.Interactive.Commands.SubmitCode..ctor(System.String, System.String, Microsoft.DotNet.Interactive.Commands.SubmissionType)'.
at Microsoft.Data.Analysis.Interactive.DataFrameKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.Data.Analysis.Interactive.DataFrameKernelExtension.LoadExtensionApiAsync(Kernel cSharpKernel)
at Microsoft.Data.Analysis.Interactive.DataFrameKernelExtension.OnLoadAsync(Kernel kernel)
at Microsoft.DotNet.Interactive.App.PackageDirectoryExtensionLoader.LoadFromAssemblyFile(FileInfo assemblyFile, Kernel kernel, KernelInvocationContext context) in D:\a\_work\1\s\src\dotnet-interactive\PackageDirectoryExtensionLoader.cs:line 117
--- End of inner exception stack trace ---
  1. I run the cell again, it never ends running - continuous infinitely

I have tried restarting vs code and it didn't help

more info:

Version: 1.84.1 (user setup)
Commit: 2b35e1e6d88f1ce073683991d1eff5284a32690f
Date: 2023-11-06T12:35:59.022Z
Electron: 25.9.2
ElectronBuildId: 24603566
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Windows_NT x64 10.0.19045

Polyglot Notebooks v1.0.4552011 Pre-Release

dotnet --list-sdks
3.1.426 [C:\Program Files\dotnet\sdk]
6.0.202 [C:\Program Files\dotnet\sdk]
5.0.214 [C:\Program Files\dotnet\sdk]
5.0.416 [C:\Program Files\dotnet\sdk]
7.0.416 [C:\Program Files\dotnet\sdk]
8.0.203 [C:\Program Files\dotnet\sdk]
7.0.310 [C:\Program Files\dotnet\sdk]
7.0.403 [C:\Program Files\dotnet\sdk]

if further information is needed from my side please tell me how to obtain it and I will add it here

superichmann avatar Nov 07 '23 11:11 superichmann

Hi, I encountered this also, when i tried running the following code while (true) Console.WriteLine("Please stop!");

clicking the Stop execution button doesn't stop the execution and I have to restart the .NET interactive kernel for it to stop

princeanire avatar Dec 18 '23 14:12 princeanire

@princeanire, this is unrelated. The cancel button will not cancel infinite loops, since .NET Core / .NET 5 / .NET 6 did not support an equivalent of .NET Framework's Thread.Abort. We are working on a potential fix using the ControlledExecution API introduced in .NET 7.

Related: #3049.

jonsequitur avatar Dec 18 '23 17:12 jonsequitur