UDF concurrency > 1 suppresses logs output

[ion-elgreco]

Describe the bug

When you use UDFs with concurrency, it suppresses the logs that are emitted from the processes. If I use 1 concurrency docling udf, I can beter follow which document is parsed.

To Reproduce

No response

Expected behavior

No response

Component(s)

Expressions

Additional context

No response

[universalmind303]

I believe @srilman has been working logging for udfs. @srilman could you take a look into this?

[srilman]

Are you using the Ray runner or the native runner? Also, are you looking for Python logs or stdout / stderr? If the former, I believe we recently pushed a fix for this: https://github.com/Eventual-Inc/Daft/pull/5168

[ion-elgreco]

Are you using the Ray runner or the native runner? Also, are you looking for Python logs or stdout / stderr? If the former, I believe we recently pushed a fix for this: https://github.com/Eventual-Inc/Daft/pull/5168

Native runner now, but happens with both.

Docling is logging lots of things to a python logger, so in stdout/stderr indeed

[srilman]

@ion-elgreco Are you still seeing this issue on the latest version of Daft with the merge fixed?

[ion-elgreco]

@srilman which version was it introduced? The last time I checked it was still the case