dl-on-flink icon indicating copy to clipboard operation
dl-on-flink copied to clipboard

运行Linear.java python报dl_on_flink_framework依赖不存在

Open I-am-DJ opened this issue 1 year ago • 0 comments

我执行的步骤

按照初始化步骤对dl-on-flink进行打包 新建了一个服务用来跑flink任务 服务中引入dl-on-flink-dist

          <dependency>
            <groupId>org.flinkextended</groupId>
            <artifactId>dl-on-flink-dist</artifactId>
            <version>0.6.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-python_2.12</artifactId>
            <version>${flink.version}</version>
        </dependency>

复制dl-on-flink-examples-tensorflow项目下的Linear.java到新服务中运行 对应报错信息

[08-29 17:52:28] [INFO] [org.mortbay.log:67] Start logging process worker:0 with org.flinkextended.flink.ml.cluster.node.runner.python.log.StdOutErrorProcessOutputConsumer@610546a2
[08-29 17:52:28] [INFO] [org.mortbay.log:67] Start logging process worker:1 with org.flinkextended.flink.ml.cluster.node.runner.python.log.StdOutErrorProcessOutputConsumer@4ea7c222
Traceback (most recent call last):
  File "/Users/lsz/code/AppAIRealTimeDataProvider/startup.py", line 22, in <module>
Traceback (most recent call last):
  File "/Users/lsz/code/AppAIRealTimeDataProvider/startup.py", line 22, in <module>
    from dl_on_flink_framework import context
ModuleNotFoundError: No module named 'dl_on_flink_framework'
    from dl_on_flink_framework import context
ModuleNotFoundError: No module named 'dl_on_flink_framework'
[08-29 17:52:29] [ERROR] [org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner:262] Got exception during python running
org.flinkextended.flink.ml.util.MLException: worker:0 python process exited with code 1
	at org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner.runProcess(ProcessPythonRunner.java:145)
	at org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner.runScript(ProcessPythonRunner.java:111)
	at org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner.runScript(CommonMLRunner.java:276)
	at org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner.run(CommonMLRunner.java:252)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
[08-29 17:52:29] [ERROR] [org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner:262] Got exception during python running
org.flinkextended.flink.ml.util.MLException: worker:1 python process exited with code 1
	at org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner.runProcess(ProcessPythonRunner.java:145)
	at org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner.runScript(ProcessPythonRunner.java:111)
	at org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner.runScript(CommonMLRunner.java:276)
	at org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner.run(CommonMLRunner.java:252)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.AbstractScriptRunner:41] worker:0 mark input queue finished.
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.AbstractScriptRunner:41] worker:1 mark input queue finished.
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner:248] Python scriptRunner for worker:1 closed
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner:248] Python scriptRunner for worker:0 closed
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.NodeHeartBeatRunner:86] worker:1 heartbeat thread interrupted
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.NodeHeartBeatRunner:86] worker:0 heartbeat thread interrupted

I-am-DJ avatar Aug 29 '23 12:08 I-am-DJ