[improve][plugin][DRDS] The task of DRDS-Reader Big data fails

[HsbcJone]

[improve][plugin][DRDS] The task of DRDS-Reader Big data fails with the following error 【net_write_timeout】 #1838

Check list

• [x] Code changed are covered with tests, or it does not need tests for reason:
• [x] If any new Jar binary package adding in your PR, please add License Notice according
• [x] If necessary, please update the documentation to describe the new feature.
• [x] If you are contributing the connector code, please check that the following files are updated:
  1. Update change log that in connector document.
  2. Update the pom file of datax

[HsbcJone]

1

[HsbcJone]

thanks. Is there anyone else who can review it. @hf200012 @iuhoay @penglin358 @dingxiaobo

[dingxiaobo]

Some suggestion.

1. Make NetworkTimeout configurable.
2. Seems thread pool never close.
3. Let logger Print stack trace.

[HsbcJone]

Collaborator

Some suggestion.

1. Make NetworkTimeout configurable.
2. Seems thread pool never close.
3. Let logger Print stack trace.

[HsbcJone]

Some suggestion.

1. Make NetworkTimeout configurable.
2. Seems thread pool never close.
3. Let logger Print stack trace.

1.Make NetworkTimeout configurable =>done 2.Let logger Print stack trace. =>done 3.update DRDS docs. =>done

Regarding the second point, the thread was not closed:

The threads here are monitoring threads and will be automatically managed. The time of the networktimeout thread depends on any timeout set, and it will close when any condition is met. @dingxiaobo

[HsbcJone]

Some suggestion.

1. Make NetworkTimeout configurable.
2. Seems thread pool never close.
3. Let logger Print stack trace.

1.Make NetworkTimeout configurable =>done 2.Let logger Print stack trace. =>done 3.update DRDS docs. =>done

Regarding the second point, the thread was not closed:

The threads here are monitoring threads and will be automatically managed. The time of the networktimeout thread depends on any timeout set, and it will close when any condition is met. @dingxiaobo

@dingxiaobo can review again

[dingxiaobo]

Problem 2 still not change.

[dingxiaobo]

I've checked MySQL JDBC driver, seems socketTimeout has same effect. setNetworkTimeout is not needed.

[HsbcJone]

Problem 2 still not change.

re-fixed and submitted

[HsbcJone]

1

[HsbcJone]

I've checked MySQL JDBC driver, seems socketTimeout has same effect. setNetworkTimeout is not needed.

The following is the difference between the two parameters

In MySQL, the socketTimeout and networkTimeout parameters serve different purposes in managing timeout behavior for database connections.

socketTimeout: The socketTimeout parameter is set at the database connection level and specifies the maximum time allowed for data transmission after the connection has been established. If no data is transmitted within the specified time, the connection is considered to have timed out. This timeout pertains to the data transfer process after the connection to the MySQL server is established. If data transfer takes longer than the time set by this parameter, a timeout exception will be thrown.

networkTimeout: The networkTimeout parameter is also set at the connection level and determines the maximum time allowed for establishing a connection. It primarily controls the timeout for network operations during the connection establishment process. If a connection cannot be established within the specified time, a timeout exception will be raised.

Differences and Distinctions:

socketTimeout focuses on potential timeouts during the data transmission process after the connection is established. networkTimeout is concerned with the connection establishment process itself and the timeout of network-related activities during that process. In practical usage, the configuration of these parameters would depend on your specific requirements. For instance, if you are more concerned about data transfer timeouts, you might adjust the socketTimeout. Conversely, if your primary concern is the timeout during connection establishment, you could adjust the networkTimeout.

It's important to note that the specific parameter names, meanings, and usage might vary based on different MySQL client libraries, versions, and configuration environments. It is advisable to refer to relevant documentation or official resources to understand the precise implications and usage of these parameters in the MySQL version you are working with.