chia-blockchain icon indicating copy to clipboard operation
chia-blockchain copied to clipboard

[Bug] Slow Bluebox Timelord - Child Process Terminated

Open thesemaphoreslim opened this issue 1 year ago • 5 comments

What happened?

When attempting to run a slow bluebox timelord in Windows 11, the debug.log outputs repeat "timelord chia.timelord.timelord : ERROR Exception manage discriminant queue: A child process terminated abruptly, the process pool is not usable anymore" errors when receiving uncompacted proofs from the full node. Attached to this ticket you will find the debug.log (DEBUG logging enabled) and my config.yaml. This occurs when starting the services with "chia start timelord-only" as well as "chia start timelord-launcher-only timelord-only".
debug.zip

I've confirmed the bug does NOT exist on Linux distributions (specifically Ubuntu server 22.04).

Version

2.4.3

What platform are you using?

Windows

What ui mode are you using?

CLI

Relevant log output

Logs attached.

thesemaphoreslim avatar Aug 22 '24 11:08 thesemaphoreslim

I have this issue also

Motophan avatar Aug 23 '24 03:08 Motophan

Update: This issue appears to only occur when using the installer for Windows available from chia.net/downloads. When installing from source on Windows, the slow bluebox timelord functions as expected. Perhaps a dependency or other component is missing from the installer.

thesemaphoreslim avatar Aug 27 '24 12:08 thesemaphoreslim

What CPU are you running? Can we get the crash log. Thinking it might be an illegal instruction.

wjblanke avatar Aug 28 '24 18:08 wjblanke

Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz 2.59 GHz

thesemaphoreslim avatar Aug 28 '24 19:08 thesemaphoreslim

What CPU are you running? Can we get the crash log. Thinking it might be an illegal instruction.

DumpStack.log

I've attached the crash log. I understand you guys are finding it difficult to duplicate the problem so here is some additional information.

I'm launching the "slow" bluebox timelord on Windows devices with the following command: chia start timelord-only

I've already provided the config.yaml and it is the same one I use for installations from source (which result in a fully working "slow" bluebox timelord) and installations using the packaged installer (which result in errors).

I've duplicated the problem across 3 bare metal devices as well as several VMs. Here are the CPU specs and relevant OS information for all:

Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz 2.59 GHz Edition: Windows 11 Pro Version: 23H2 Build: 22631.4037

Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz 3.90 GHz Edition: Windows 10 Pro Version: 22H2 Build: 19045.4780

AMD Ryzen Threadripper 2950X 16-Core Processor 3.50 GHz Edition: Windows 10 Pro Version: 22H2 Build: 19045.4780

If I can provide any additional information don't hesitate to ask.

thesemaphoreslim avatar Aug 28 '24 20:08 thesemaphoreslim

Yes I can reproduce it here now. Hmmm

wjblanke avatar Aug 30 '24 05:08 wjblanke

https://github.com/Chia-Network/chia-blockchain/pull/18544

wjblanke avatar Sep 01 '24 01:09 wjblanke

https://github.com/Chia-Network/chia-blockchain/pull/18546

better

wjblanke avatar Sep 03 '24 14:09 wjblanke

assigning to earle to look at killing the entire process tree

wjblanke avatar Sep 11 '24 16:09 wjblanke

https://stackoverflow.com/questions/1230669/subprocess-deleting-child-processes-in-windows

wjblanke avatar Sep 17 '24 16:09 wjblanke

https://github.com/Chia-Network/chia-blockchain/pull/18642

wjblanke avatar Oct 16 '24 14:10 wjblanke

This should be fixed in main, including proper shutdown behavior. release will come eventually but can be used now. Thanks!

wjblanke avatar Oct 17 '24 13:10 wjblanke