incubator-heron
incubator-heron copied to clipboard
apache
can not submit standalone topology
After I start standalone cluster well, I just submit a example topology, but error is below:
heron submit standalone heron-streamlet-examples.jar org.apache.heron.examples.streamlet.WindowedWordCountTopology WindowedWordCountTopology
[2020-03-12 14:24:15 +0800] [INFO]: Using cluster definition in /Users/zz/.heron/conf/standalone
三月 12, 2020 2:24:16 下午 org.apache.heron.streamlet.impl.StreamletBaseImpl defaultNameCalculator
信息: Calculated stage Name as consumer1
[2020-03-12 14:24:16 +0800] [INFO]: Launching topology: 'WindowedWordCountTopology'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/zz/.heron/lib/uploader/heron-dlog-uploader.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/zz/.heron/lib/statemgr/heron-zookeeper-statemgr.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
[2020-03-12 14:24:17 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: 127.0.0.1:2181
[2020-03-12 14:24:17 +0800] [信息] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting
[2020-03-12 14:24:17 +0800] [信息] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED
[2020-03-12 14:24:17 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.
[2020-03-12 14:24:17 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking existence of path: /heron/topologies/WindowedWordCountTopology
[2020-03-12 14:24:17 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Initalizing RoundRobinPacking. CPU default: 1.000000, RAM default: ByteAmount{1.0 GB (1073741824 bytes)}, DISK default: ByteAmount{1.0 GB (1073741824 bytes)}, RAM padding: ByteAmount{2.0 GB (2147483648 bytes)}.
[2020-03-12 14:24:17 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal: container CPU hint: -1.000, RAM hint: ByteAmount{-1 bytes}, disk hint: ByteAmount{-1 bytes}.
[2020-03-12 14:24:17 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: container#1 CPU: 3.000000, RAM: ByteAmount{4.0 GB (4294967296 bytes)}, disk: ByteAmount{14.0 GB (15032385536 bytes)}.
[2020-03-12 14:24:17 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: container#2 CPU: 3.000000, RAM: ByteAmount{4.0 GB (4294967296 bytes)}, disk: ByteAmount{14.0 GB (15032385536 bytes)}.
[2020-03-12 14:24:18 +0800] [严重] org.apache.heron.uploader.http.HttpUploader: Error uploading package to location: /var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc80000gp/T/tmpwXznPX/topology.tar.gz
java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)
at org.apache.http.entity.mime.content.FileBody.writeTo(FileBody.java:121)
at org.apache.http.entity.mime.AbstractMultipartForm.doWriteTo(AbstractMultipartForm.java:134)
at org.apache.http.entity.mime.AbstractMultipartForm.writeTo(AbstractMultipartForm.java:157)
at org.apache.http.entity.mime.MultipartFormEntity.writeTo(MultipartFormEntity.java:107)
at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.heron.uploader.http.HttpUploader.execute(HttpUploader.java:107)
at org.apache.heron.uploader.http.HttpUploader.uploadPackageAndGetURI(HttpUploader.java:98)
at org.apache.heron.uploader.http.HttpUploader.uploadPackage(HttpUploader.java:79)
at org.apache.heron.scheduler.SubmitterMain.uploadPackage(SubmitterMain.java:549)
at org.apache.heron.scheduler.SubmitterMain.submitTopology(SubmitterMain.java:452)
at org.apache.heron.scheduler.SubmitterMain.main(SubmitterMain.java:334)
[2020-03-12 14:24:18 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the CuratorClient to: 127.0.0.1:2181
[2020-03-12 14:24:18 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the tunnel processes
[2020-03-12 14:24:18 +0800] [严重] org.apache.heron.scheduler.SubmitterMain: Exception when submitting topology
org.apache.heron.spi.uploader.UploaderException: Error uploading package to location: /var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc80000gp/T/tmpwXznPX/topology.tar.gz
at org.apache.heron.uploader.http.HttpUploader.uploadPackage(HttpUploader.java:83)
at org.apache.heron.scheduler.SubmitterMain.uploadPackage(SubmitterMain.java:549)
at org.apache.heron.scheduler.SubmitterMain.submitTopology(SubmitterMain.java:452)
at org.apache.heron.scheduler.SubmitterMain.main(SubmitterMain.java:334)
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)
at org.apache.http.entity.mime.content.FileBody.writeTo(FileBody.java:121)
at org.apache.http.entity.mime.AbstractMultipartForm.doWriteTo(AbstractMultipartForm.java:134)
at org.apache.http.entity.mime.AbstractMultipartForm.writeTo(AbstractMultipartForm.java:157)
at org.apache.http.entity.mime.MultipartFormEntity.writeTo(MultipartFormEntity.java:107)
at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.heron.uploader.http.HttpUploader.execute(HttpUploader.java:107)
at org.apache.heron.uploader.http.HttpUploader.uploadPackageAndGetURI(HttpUploader.java:98)
at org.apache.heron.uploader.http.HttpUploader.uploadPackage(HttpUploader.java:79)
... 3 more
[2020-03-12 14:24:18 +0800] [ERROR]: Error uploading package to location: /var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc80000gp/T/tmpwXznPX/topology.tar.gz [2020-03-12 14:24:18 +0800] [ERROR]: Failed to launch topology 'WindowedWordCountTopology' macdeMBP-2:java zz$
@dttlgotv
org.apache.heron.uploader.http.HttpUploader: Error uploading package to location: /var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc80000gp/T/tmpwXznPX/topology.tar.gz
HttpUploader Only http://...... It must be a URL.
@dttlgotv
org.apache.heron.uploader.http.HttpUploader: Error uploading package to location: /var/folders/zr/dgwn7zgn3fbcjmv25t_ybsc80000gp/T/tmpwXznPX/topology.tar.gzHttpUploader Only http://...... It must be a URL.
Anyway I change to use local file to update， but it can not run well too. It seems that Prometheus metrics must be used in this mode though I have delete it in metrics_sinks.yaml.
heron submit standalone heron-streamlet-examples.jar org.apache.heron.examples.streamlet.FormattedOutputTopology FormattedOutputTopology
[2020-03-16 19:41:17 +0800] [INFO]: Using cluster definition in /Users/guoxinghua/.heron/conf/standalone
三月 16, 2020 7:41:17 下午 org.apache.heron.streamlet.impl.StreamletBaseImpl defaultNameCalculator
信息: Calculated stage Name as supplier1
三月 16, 2020 7:41:18 下午 org.apache.heron.streamlet.impl.StreamletBaseImpl defaultNameCalculator
信息: Calculated stage Name as filter1
三月 16, 2020 7:41:18 下午 org.apache.heron.streamlet.impl.StreamletBaseImpl defaultNameCalculator
信息: Calculated stage Name as consumer1
[2020-03-16 19:41:18 +0800] [INFO]: Launching topology: 'FormattedOutputTopology'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/guoxinghua/.heron/lib/uploader/heron-dlog-uploader.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/guoxinghua/.heron/lib/statemgr/heron-zookeeper-statemgr.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Starting Curator client connecting to: 127.0.0.1:2181
[2020-03-16 19:41:19 +0800] [信息] org.apache.curator.framework.imps.CuratorFrameworkImpl: Starting
[2020-03-16 19:41:19 +0800] [信息] org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Directory tree initialized.
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Checking existence of path: /heron/topologies/FormattedOutputTopology
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Initalizing RoundRobinPacking. CPU default: 1.000000, RAM default: ByteAmount{1.0 GB (1073741824 bytes)}, DISK default: ByteAmount{1.0 GB (1073741824 bytes)}, RAM padding: ByteAmount{2.0 GB (2147483648 bytes)}.
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal: container CPU hint: -1.000, RAM hint: ByteAmount{-1 bytes}, disk hint: ByteAmount{-1 bytes}.
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: container#1 CPU: 4.000000, RAM: ByteAmount{5.0 GB (5368709120 bytes)}, disk: ByteAmount{15.0 GB (16106127360 bytes)}.
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Initalizing RoundRobinPacking. CPU default: 1.000000, RAM default: ByteAmount{1.0 GB (1073741824 bytes)}, DISK default: ByteAmount{1.0 GB (1073741824 bytes)}, RAM padding: ByteAmount{2.0 GB (2147483648 bytes)}.
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal: container CPU hint: -1.000, RAM hint: ByteAmount{-1 bytes}, disk hint: ByteAmount{-1 bytes}.
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.packing.roundrobin.RoundRobinPacking: Pack internal finalized: container#1 CPU: 4.000000, RAM: ByteAmount{5.0 GB (5368709120 bytes)}, disk: ByteAmount{15.0 GB (16106127360 bytes)}.
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/topologies/FormattedOutputTopology
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/packingplans/FormattedOutputTopology
[2020-03-16 19:41:19 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/executionstate/FormattedOutputTopology
[2020-03-16 19:41:19 +0800] [警告] org.apache.heron.common.config.ConfigReader: Config file name cannot be null
[2020-03-16 19:41:20 +0800] [信息] org.apache.heron.scheduler.utils.SchedulerUtils: Updating scheduled-resource in packing plan: FormattedOutputTopology
[2020-03-16 19:41:20 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Deleted node for path: /heron/packingplans/FormattedOutputTopology
[2020-03-16 19:41:20 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/packingplans/FormattedOutputTopology
[2020-03-16 19:41:20 +0800] [严重] org.apache.heron.scheduler.nomad.NomadScheduler: Failed to find port file for Prometheus metrics. Please check metrics sinks configurations
[2020-03-16 19:41:20 +0800] [严重] org.apache.heron.scheduler.nomad.NomadScheduler: Failed to find port file for Prometheus metrics. Please check metrics sinks configurations
[2020-03-16 19:41:20 +0800] [信息] org.apache.heron.scheduler.utils.SchedulerUtils: Setting Scheduler locations: topology_name: "FormattedOutputTopology"
http_endpoint: "scheduler_as_lib_no_endpoint"
job_page_link: "http://127.0.0.1:4646/ui/jobs"
[2020-03-16 19:41:20 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Created node for path: /heron/schedulers/FormattedOutputTopology
[2020-03-16 19:41:20 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the CuratorClient to: 127.0.0.1:2181
[2020-03-16 19:41:20 +0800] [信息] org.apache.heron.statemgr.zookeeper.curator.CuratorStateManager: Closing the tunnel processes
[2020-03-16 19:41:20 +0800] [INFO]: Successfully launched topology 'FormattedOutputTopology'
macdeMBP-2:java guoxinghua$
- ~/.heron/bin/heron-tracker
- ~/.heron/bin/heron-ui
- heron submit local ~/.heron/examples/heron-streamlet-examples.jar org.apache.heron.examples.streamlet.FormattedOutputTopology FormattedOutputTopology
- http://0.0.0.0:8889/topologies/local/default/FormattedOutputTopology
I succeeded in doing this in the locale. Can I?
no problem in local mode on my side too.
发自我的iPhone
------------------ Original ------------------ From: Roger Pack <[email protected]> Date: Mon,Mar 16,2020 10:15 PM To: apache/incubator-heron <[email protected]> Cc: dttlgotv <[email protected]>, Mention <[email protected]> Subject: Re: [apache/incubator-heron] can not submit standalone topology (#3489)
~/.heron/bin/heron-tracker
~/.heron/bin/heron-ui
heron submit local ~/.heron/examples/heron-streamlet-examples.jar org.apache.heron.examples.streamlet.FormattedOutputTopology FormattedOutputTopology
I succeeded in doing this in the locale. Can I?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
- ~/.heron/bin/heron-tracker
- ~/.heron/bin/heron-ui
- heron submit local ~/.heron/examples/heron-streamlet-examples.jar org.apache.heron.examples.streamlet.FormattedOutputTopology FormattedOutputTopology
- http://0.0.0.0:8889/topologies/local/default/FormattedOutputTopology
I succeeded in doing this in the locale. Can I?
Could you run "heron submit standalone ....."
I can't because I don't have local nomads.
# The URI of Nomad API
heron.nomad.scheduler.uri: http://127.0.0.1:4646