Eventual-Inc
process name need be defined as udfname in native engine
Is your feature request related to a problem?
Current process name is daft.execution.udf_worker， can we define it like 'udf_name'
Describe the solution you'd like
None
Describe alternatives you've considered
No response
Additional Context
No response
Would you like to implement a fix?
No
Unfortunately, there is now way to change the name of the process itself from daft.execution.udf_worker to something else, because the name comes from the Python script that we use to start up the process (aka daft/execution/udf_worker.py). We need to use an in-between script for communication between the main Daft process and UDF worker process.
We could temporarily add the name of the UDF as a CLI argument, so it would look like:
/opt/conda/bin/python -m daft.execution.udf_worker my_udf_name /tmp/... ...
But this wouldn't be a permanent change because we plan to soon reuse UDF worker processes for multiple different UDFs.
Can I ask why you're looking for the UDF name in top / htop? Are you trying to monitor CPU or memory utilization in the UDF?
@srilman yes, i want to know cpu/mem util, if native engine will reuse worker process, i think i will ignore this point.
Closing this because it's likely not possible, as per @srilman 's points, i.e the name comes from the script file name, and we will eventually reuse processes for performance.
