Rust print can't displayed in notebook environment

[sun-rs]

trafficstars

I tried the official example- https://pyo3.rs/v0.16.2/trait_bounds.html. It works well when the code is in .py file, I can see the print from python and rust. But when I use it in jupyter notebook(vscode), the rust print can not display.

[birkenfeld]

Because notebook likely only redirects the Python streams while Rust prints to the OS stdout. Not much we can do about it I think...

[sun-rs]

Because notebook likely only redirects the Python streams while Rust prints to the OS stdout. Not much we can do about it I think...

Thanks for reply, I was wondering if Rust println! not work, would some other Rust logging module may work in notebook?

[birkenfeld]

I don't know if anyone has written a Rust-log-to-Python-logging adapter yet.

We could provide println! etc macros in pyo3 that goes to sys.stdout - what do others think about that?

[adamreichold]

I think #1208 tracks integration between Rust logging and Python logging. The guide also used to mention https://github.com/vorner/pyo3-log and there is also tracing integration project https://github.com/danielschemmel/tracing-for-pyo3-logging

[davidhewitt]

We could provide println! etc macros in pyo3 that goes to sys.stdout - what do others think about that?

Potentially interesting. Are there C APIs that allow us to get access to the Python output handles (if they've been modified)?

The guide also used to mention https://github.com/vorner/pyo3-log

https://pyo3.rs/v0.16.2/ecosystem/logging.html

[birkenfeld]

There is PySys_WriteStdout and ...Stderr.

[aganders3]

The wurlitzer Python package may help in the shorter term. It can be loaded as an IPython extension.

[sun-rs]

It seems we can share rust log info to print in notebook in 0.19?

[sun-rs]

The wurlitzer Python package may help in the shorter term. It can be loaded as an IPython extension.

wurlitzer Python包可能在短期内有所帮助。它可以作为IPython扩展加载。

Unfortunately wurlitzer can only work with linux not windows

[kngwyu]

I don't think println will work as is, but maybe we can provide some convenient wrapper to print something to the stdout that PyThreadState holds. I'm gonna investigate that direction a bit.

[sun-rs]

Is there some alternative method to log or print rust info in jupyter env especially for windows？ Right now I must use windows terminal to run a py file and see what is going on rather than jupyter which makes me blind in jupyter env.

I have a method learn from polars which is set a trigger to make all info panic instantly.

[davidhewitt]

I think we might be able to have py.stdout() and py.stderr() functions to give structures which use PySys_WriteStdout and PySys_WriteStderr ? Just need someone who's got time to play with an implementation.

[goulart-paul]

I think we might be able to have py.stdout() and py.stderr() functions to give structures which use PySys_WriteStdout and PySys_WriteStderr ? Just need someone who's got time to play with an implementation.

I gave this a try and got it to work. See here.

It's unclear to me if my method is thread safe though. Is it necessary to acquire the GIL when calling pyo3 ffi functions, specifically PyObject_CallMethod, PySys_GetObject, PySys_WriteStderr, PySys_WriteStdout? The first two of those are used only to call sys.stdout.flush().

[davidhewitt]

Nice!

Yes those functions will need to be protected by the GIL. One straightforward option would be to carry the 'py lifetime on your types, so that they can only exist as long as the GIL is held.

Would you be interested in opening this as a PR against PyO3? I think these types would be welcome to exist as part of this library.

[goulart-paul]

Yes those functions will need to be protected by the GIL. One straightforward option would be to carry the 'py lifetime on your types, so that they can only exist as long as the GIL is held.

OK, I thought that might be the case and have rewritten it accordingly. I have not tried to carry the lifetime around though, since for our use we create a py.stdout() type object and then write to it over an extended time. It seems better to acquire the GIL only momentarily as needed within the Write implementation of that object. The stdout object is wrapped inside a LineWriter buffer, so hopefully not acquiring the GIL too frequently.

Would you be interested in opening this as a PR against PyO3? I think these types would be welcome to exist as part of this library.

Yes, I can do that.

Note that this doesn't really address the issue of println! calls printing to the wrong place. It just allows you to replace them with writeln!(pyo3::io::stdout(),...) calls. Not a big deal for us, but likely a problem for someone with a really big codebase.

[sun-rs]

I have test macos/ubuntu/win10, only win10 couldn't print rust info in notebook