incubator-streampark icon indicating copy to clipboard operation
incubator-streampark copied to clipboard

[Bug] When streamx stops a task, start savepoint, but the savepoint path is not required

Open muchunjin opened this issue 2 years ago • 9 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

When streamx stops a task, start savepoint, but the savepoint path is not required. If you do not specify a savepoint path to stop, the task will fail to stop and cannot be stopped again. You can only go to yarn to stop

StreamX Version

1.2.3

Java Version

jdk1.8

Flink Version

1.14.3

Scala Version of Flink

2.11

Error Exception

Background prompt savepoint path is not specified.

Screenshots

No response

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

muchunjin avatar Jun 29 '22 01:06 muchunjin

same problem

chuangehh avatar Jul 01 '22 06:07 chuangehh

@wolfboys When savepoint is on, custom savepoint is set as required in the front end. Is it OK?

muchunjin avatar Jul 01 '22 07:07 muchunjin

When stopping, if enable savepoint policy as follows:

  1. Specifying the savepoint path will keep the savepoint to the path specified by the user
  2. If the savepoint path is not specified, it will be kept to the default path, and the configuration will be read (state.savepoints.dir)

停止时如果用户开启savepoint 策略如下:

  1. 指定了savepoint path 会将 savepoint 保持到用户指定的路径下
  2. 未指定 savepoint path 则会保持到 默认的路径下, 会读取配置(state.savepoints.dir)

wolfboys avatar Jul 01 '22 08:07 wolfboys

@wolfboys ok

muchunjin avatar Jul 01 '22 08:07 muchunjin

@wolfboys Through the test, when state.savepoints.dir is set in the configuration file flink-conf.yaml. After dir, this bug no longer appears, the job stops normally, and savepiont is normally guaranteed in state savepoints. In the path set by the dir parameter, I think this bug can be closed

muchunjin avatar Jul 03 '22 13:07 muchunjin

@wolfboys 通过测试,当在配置文件flink-conf.yaml中设置state.savepoints.dir后,此bug不在出现,作业正常停止,savepiont正常保障在state.savepoints.dir参数设置的路径中,那么此bug我认为可以关闭了

muchunjin avatar Jul 03 '22 13:07 muchunjin

@wolfboys 当没有指定Custom SavePoint又没有在Flink的配置文件中设置state.savepoints.dir时,可以提示必须指定Custom SavePoint或者设置state.savepoints.dir,这样可以防止停止任务出现失败的情况,是否可以?

When custom savepoint is not specified and state.savepoints.dir is not set in Flink's configuration, you can prompt that you must specify custom savepoint or set state.savepoints.dir, which can prevent the failure of stopping the task. Is it OK?

muchunjin avatar Jul 05 '22 03:07 muchunjin

Yes, it's better to do so. see Applicationserviceimpl.java 912 to 941 lines for more detail.


可以, 这样做比较好, 具体代码可以看看 ApplicationServiceImpl.java 912 到 941 行

wolfboys avatar Jul 05 '22 04:07 wolfboys

Yes, it's better to do so. see Applicationserviceimpl.java 912 to 941 lines for more detail.

可以, 这样做比较好, 具体代码可以看看 ApplicationServiceImpl.java 912 到 941 行

ok

muchunjin avatar Jul 05 '22 06:07 muchunjin