[Bug] [jdbc write] the job did not end, running every minutes

[jeanleen]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

first i delete the data from mysql ,then write data ,so i set two jdbc configuration in sink module. when write data to mysql, the job did not end ,run every minute.

SeaTunnel Version

2.3.3

SeaTunnel Config

# Defining the runtime environment
env {
  parallelism = 2
#   spark.master = "yarn"
#   spark.deploy-mode= "cluster"
  job.name = ""
#   spark.executor.instances = 2
#   spark.executor.cores = 2
#   spark.executor.memory = "4g"
  job.mode = "BATCH"
}
source{
# Jdbc {
#        
#         
#         query = """
#                  select * from test.users
#
#         """
# #         query = """SELECT  PRO_ID,PRO_BOUGHT,CATE_1,TO_CHAR(SYNC_DATETIME, 'YYYY-MM-DD')  as SYNC_DATETIME FROM ( SELECT PRO_ID,PRO_BOUGHT,CATE_1,SYNC_DATETIME FROM SMT_GET_PRODUCT ORDER BY UPDATE_TIME ) WHERE ROWNUM <= 100 """
# #           query ="SELECT  PRO_ID,PRO_BOUGHT,CATE_1,TO_CHAR(SYNC_DATETIME, 'YYYY-MM-DD') as SYNC_DATETIME FROM SMT_GET_PRODUCT where TO_CHAR(SYNC_DATETIME, 'YYYY-MM-DD')='2024-01-16'"
#     }


  HdfsFile {
#   schema {
#     fields {
#       name = string
#       address = string
#     }
#   }
  path = "/user/hive/warehouse/test.db/users/part_ymd=jian"
  file_format_type = "orc"
  fs.defaultFS = "hdfs://hadoop121"
  parse_partition_from_path = false
  read_columns =[name,address]

  result_table_name = "users"

  # If you would like to get more information about how to configure seatunnel and see full list of source plugins,
  # please go to https://seatunnel.apache.org/docs/category/source-v2
}


# #
#   Hive {
#     table_name = "test.users"
#     metastore_uri = "thrift://hadoop121:9083"
#   }

#     Jdbc {
#         
#         query = """
#                    SELECT ID, CATEGORY_ID, PRODUCT_ID, RANK, AVG_RANK_7, AVG_RANK_7_UP,
#                            AVG_RANK_7_INCREASE, CREATE_USER_NAME,TO_CHAR(CREATE_DATETIME, 'YYYY-MM-DD')as CREATE_DATETIME,
#                            UPDATE_USER_NAME, UPDATE_DATETIME
#                    FROM product_selection_aliexpress_rank
#
#         """
# #         query = """SELECT  PRO_ID,PRO_BOUGHT,CATE_1,TO_CHAR(SYNC_DATETIME, 'YYYY-MM-DD')  as SYNC_DATETIME FROM ( SELECT PRO_ID,PRO_BOUGHT,CATE_1,SYNC_DATETIME FROM SMT_GET_PRODUCT ORDER BY UPDATE_TIME ) WHERE ROWNUM <= 100 """
# #           query ="SELECT  PRO_ID,PRO_BOUGHT,CATE_1,TO_CHAR(SYNC_DATETIME, 'YYYY-MM-DD') as SYNC_DATETIME FROM SMT_GET_PRODUCT where TO_CHAR(SYNC_DATETIME, 'YYYY-MM-DD')='2024-01-16'"
#     }
}

transform{

    Filter {
        source_table_name = "users"
        fields = [name]
        result_table_name = "fake_name"
    }
    Filter {
        source_table_name = "users"
        fields = [name,address]
        result_table_name = "users_all"
    }



}




sink {

# 测试是否支持事务，初步测试不支持 20240120
#    jdbc {
#       
#         auto_commit= false
#         query = """
#           start transaction ;
#           delete from test.users where name = 'jian' ;
#           insert into test.users(name,address) values(?,?) ;
#           COMMIT;
#            """
#
#         }

# 分两步执行，第一步删除，第二步插入
    jdbc {
           
            
            query = """
              delete from test.users where name = 'jian'
               """
            }
#       console {
#         source_table_name = "users_all"
#       }
#
#         console {
#         source_table_name = "fake_name"
#       }
    jdbc {
          

            query = """
              insert into test.users(name,address) values(?,?)
               """
            }



    }



# create table test.sink_user
# (  name string,age int)
# comment '测试seatunnel hive'
# ROW FORMAT DELIMITED FIELDS TERMINATED BY '#'
# STORED AS orc ;

Running Command

/usr/local/apache-seatunnel-2.3.3/bin/seatunnel.sh --config ods_pdc_product_selection_aliexpress_rank.conf

Error Exception

2024-01-20 15:25:23,458 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Submit job finished, job id: 800989361205149698, job name: SeaTunnel
2024-01-20 15:25:23,466 WARN  org.apache.seatunnel.engine.client.job.JobMetricsRunner - Failed to get job metrics summary, it maybe first-run
2024-01-20 15:26:23,624 INFO  org.apache.seatunnel.engine.client.job.JobMetricsRunner - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    :  800989361205149698
Read Count So Far         :                   1
Write Count So Far        :                   1
Average Read Count        :                 0/s
Average Write Count       :                 0/s
Last Statistic Time       : 2024-01-20 15:25:23
Current Statistic Time    : 2024-01-20 15:26:23
***********************************************

2024-01-20 15:27:23,593 INFO  org.apache.seatunnel.engine.client.job.JobMetricsRunner - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    :  800989361205149698
Read Count So Far         :                   2
Write Count So Far        :                   2
Average Read Count        :                 0/s
Average Write Count       :                 0/s
Last Statistic Time       : 2024-01-20 15:26:23
Current Statistic Time    : 2024-01-20 15:27:23
***********************************************

2024-01-20 15:28:23,560 INFO  org.apache.seatunnel.engine.client.job.JobMetricsRunner - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    :  800989361205149698
Read Count So Far         :                   2
Write Count So Far        :                   2
Average Read Count        :                 0/s
Average Write Count       :                 0/s
Last Statistic Time       : 2024-01-20 15:27:23
Current Statistic Time    : 2024-01-20 15:28:23
***********************************************

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[liugddx]

set parallelism = 1 and try again.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.