gPad is not consistent in pyROOT

[linev]

Check duplicate issues.

• [X] Checked for duplicates

Description

If configuring any web display, gPad is not properly set. As a result, h.Draw creates new canvas. Code works properly when gROOT.SetWebDisplay("off") invoked.

Reproducer

Here is reproducer:

import ROOT

ROOT.gROOT.SetWebDisplay("")

c = ROOT.TCanvas()

if ROOT.gPad:
   print("gPad name", ROOT.gPad.GetName(), "editable", ROOT.gPad.IsEditable())
else:
   print("gPad is not there - why?")

h = ROOT.TH1D("","",64, -4,4)
h.FillRandom("gaus")

# this is a problem - histogram do not "see" gPad and creates new canvas
h.Draw()

input("Press Enter to exit")

ROOT version

master branch

Python 3.11.9

Installation method

build from source

Operating system

OpenSUSE linux

Additional context

No response

[vepadulano]

Just to confirm, this only happens when running ROOT from Python right? An equivalent C++ code would work?

[linev]

Yes, similar C++ code works as expected:

{
   gROOT->SetWebDisplay("");

   auto c = new TCanvas();

   if (gPad)
      printf("gPad name %s editable %d\n", gPad->GetName(), gPad->IsEditable());
   else
      printf("gPad is not there - why?\n");

   auto h = new TH1D("","",64, -4,4);
   h->FillRandom("gaus");

   // this is a problem - histogram do not "see" gPad and creates new canvas
   h->Draw();
}

[linev]

I just not understand how gPad can differ in h.Draw. Are there any special pythonization for it? Or is there other thread? gPad is thread-local variable.

[dpiparo]

@couet we need perhaps help

[linev]

I do not think Olivier can help here.

Something wired with pythonization here. At the moment when h.Draw() is called gPad is not consistent.

[vepadulano]

Starting to investigate, just for reference, this slightly modified repro

import ROOT

ROOT.gROOT.SetWebDisplay("")

c = ROOT.TCanvas()

if ROOT.gPad:
   print("gPad name", ROOT.gPad.GetName(), "editable", ROOT.gPad.IsEditable())
else:
   print("gPad is not there - why?")

h = ROOT.TH1D("","",64, -4,4)
h.FillRandom("gaus")

# this is a problem - histogram do not "see" gPad and creates new canvas
h.Draw()

if ROOT.gPad:
   print("gPad name", ROOT.gPad.GetName(), "editable", ROOT.gPad.IsEditable())
else:
   print("gPad is not there - why?")

input("Press Enter to exit")

Prints correctly

Info in <THttpEngine::Create>: Starting HTTP server on port 127.0.0.1:9581
gPad name c1 editable True
Info in <TCanvas::MakeDefCanvas>:  created default TCanvas with name c1_n2
gPad name c1 editable True

So gPad appears to be there even after the call to TH1::Draw. It's somehow within that call that it is not recognised/found.

[linev]

It's somehow within that call that it is not recognised/found.

Yes, histogram draw do not "see" gPad. Can it be that TH1::Draw() method called from other thread?

[vepadulano]

There are 13 threads running at the moment TH1::Draw is reached

  Id   Target Id                                             Frame 
* 1    Thread 0x7ffff7ec4740 (LWP 2762011) "python"          TH1::Draw (this=0x55555c45d3e0, option=0x7fffdfd1d000 "")
    at /home/vpadulan/Programs/rootproject/rootsrc/hist/hist/src/TH1.cxx:3070
  2    Thread 0x7fffcee006c0 (LWP 2762119) "python"          0x00007ffff76f7163 in clock_nanosleep@GLIBC_2.2.5 () from /lib64/libc.so.6
  3    Thread 0x7fffcce006c0 (LWP 2762143) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  4    Thread 0x7fffc7e006c0 (LWP 2762144) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  5    Thread 0x7fffc74006c0 (LWP 2762145) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  6    Thread 0x7fffc6a006c0 (LWP 2762146) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  7    Thread 0x7fffc60006c0 (LWP 2762147) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  8    Thread 0x7fffc56006c0 (LWP 2762148) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  9    Thread 0x7fffc4c006c0 (LWP 2762149) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  10   Thread 0x7fffbfe006c0 (LWP 2762150) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  11   Thread 0x7fffbf4006c0 (LWP 2762151) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  12   Thread 0x7fffbea006c0 (LWP 2762152) "civetweb-worker" 0x00007ffff76a9169 in __futex_abstimed_wait_common () from /lib64/libc.so.6
  13   Thread 0x7fffbe0006c0 (LWP 2762153) "civetweb-master" 0x00007ffff7725e3d in poll () from /lib64/libc.so.6

[linev]

Question if it is same thread as when TCanvas is created and gPad is set. gPad is thread-local variable

[vepadulano]

It does not seem to be the case, which makes the issue even weirder

Thread 1 "python" hit Breakpoint 1.1, TCanvas::TCanvas (this=0x55555b6ce330, build=true)
    at /home/vpadulan/Programs/rootproject/rootsrc/graf2d/gpad/src/TCanvas.cxx:147
147	TCanvas::TCanvas(Bool_t build) : TPad(), fDoubleBuffer(0)
(gdb) p $_thread
$1 = 1
(gdb) c
Continuing.
Info in <THttpEngine::Create>: Starting HTTP server on port 127.0.0.1:9375
[New Thread 0x7fffcce006c0 (LWP 2795576)]
[New Thread 0x7fffc7e006c0 (LWP 2795577)]
[New Thread 0x7fffc74006c0 (LWP 2795578)]
[New Thread 0x7fffc6a006c0 (LWP 2795579)]
[New Thread 0x7fffc60006c0 (LWP 2795580)]
[New Thread 0x7fffc56006c0 (LWP 2795581)]
[New Thread 0x7fffc4c006c0 (LWP 2795582)]
[New Thread 0x7fffbfe006c0 (LWP 2795583)]
[New Thread 0x7fffbf4006c0 (LWP 2795584)]
[New Thread 0x7fffbea006c0 (LWP 2795585)]
[New Thread 0x7fffbe0006c0 (LWP 2795586)]
[Detaching after vfork from child process 2795587]
gPad name c1 editable True

Thread 1 "python" hit Breakpoint 2, TH1::Draw (this=0x55555b695c10, option=0x7fffdfd1d000 "")
    at /home/vpadulan/Programs/rootproject/rootsrc/hist/hist/src/TH1.cxx:3070
3070	   TString opt1 = option; opt1.ToLower();
(gdb) p $_thread
$2 = 1
(gdb) c
Continuing.
Thread 1 "python" hit Breakpoint 1.3, TCanvas::TCanvas (this=0x55555c52e8c0, name=0x55555c63c110 "c1_n2", title=0x55555c63c110 "c1_n2", form=1)
    at /home/vpadulan/Programs/rootproject/rootsrc/graf2d/gpad/src/TCanvas.cxx:269
269	TCanvas::TCanvas(const char *name, const char *title, Int_t form) : TPad(), fDoubleBuffer(0)
(gdb) p $_thread
$3 = 1
(gdb) n
272	   fUseGL = gStyle->GetCanvasPreferGL();
(gdb) 
186	   Bool_t           GetCanvasPreferGL() const {return fCanvasPreferGL;}
(gdb) 
272	   fUseGL = gStyle->GetCanvasPreferGL();
(gdb) n
274	   Constructor(name, title, form);
(gdb) 
[Detaching after vfork from child process 2802535]
TCanvas::MakeDefCanvas () at /home/vpadulan/Programs/rootproject/rootsrc/graf2d/gpad/src/TCanvas.cxx:1523
1523	   ::Info("TCanvas::MakeDefCanvas"," created default TCanvas with name %s", cdef.Data());
(gdb) p $_thread
$4 = 1

[vepadulano]

The apparent reason is that at this line https://github.com/root-project/root/blob/2ee1281e6dfa7bb82a650608442e75f8dfb4c6a7/core/thread/src/TThread.cxx#L896

TThread::SelfId() returns 140737352845120 fgMainId returns 140736664176320

So somehow they are different even though gdb says it's the same thread

[linev]

Looks like TThread is not able to correctly detect python thread id?

[linev]

Or python executes different script lines from different threads?

[vepadulano]

Looks like TThread is not able to correctly detect python thread id?

I don't know. TThread is called by ROOT internals anyway, so it's not like it should "detect" an external thread id of sorts. I tried stepping through gdb in all occurences of usage of TThread but it's too much noise.

[pcanal]

Or python executes different script lines from different threads?

It smells like this is the problem. (But oddly on the surface the example at https://github.com/root-project/root/issues/15498#issue-2293017347 seems to not turn on ROOT thread safety)>

[vepadulano]

(But oddly on the surface the example at https://github.com/root-project/root/issues/15498#issue-2293017347 seems to not turn on ROOT thread safety)>

Interesting comment! In fact, just writing ROOT.EnableThreadSafety() at the beginning of the program makes the example run properly on my machine (i.e. only one canvas shows up). @linev can you confirm?

[linev]

Yes, after ROOT.EnableThreadSafety() script works normally and only single canvas is created

[linev]

~~Now I see similar problem with slightly modified ROOT macro from DF tutorials -df027_SQliteDependencyOverVersion.C ~~

~~When web graphics enabled - gPad is not that it should be. All histograms drawn on main canvas instead of sub-pads.~~

Edited - this problem has different nature and not connected with gPad

[linev]

@vepadulano

Actually, it is enough to call TThread::SelfId() method to make macro working.

Without it libThread will be initialized from other thread - and internal TThread::fgMainId will be set wrongly. Looks like it is main reason why gPad will not work properly after that.

Probably I can make workaround for TWebCanvas - but may be one can always load and initialize threads library already with import ROOT?

[linev]

Resolved by #15825 and #15830 in master and 6.32