No reconnect after error

[thenovacreator]

Hello Folks,

i have got a problem that a have i huge logging file of 250Mb (please do not ask) that i want to import to a mysql database. The entries itself are small enought to get inserted one by one into the database. But if i want to import the whole file i first got the error "Mysql Server has gone away". It seems to me that there was a 250Mb insert statement generated that sure enought failed cause of the size. Fluentd logging says "Got deterministic error. Fallback to one-by-one import". So this is catched and the plugin tries to do it one by one. But there seems no reconnect is going on for the database. So inserts are tried on a connection that is not alive anymore. You might see this at the following logging.

fluentd    | 2020-01-14 08:40:56 +0100 [warn]: #0 Failed to import a record: retry number = 1 error_class=Mysql2::Error error="MySQL client is not connected"
fluentd    | 2020-01-14 08:40:57 +0100 [warn]: #0 Failed to import a record: retry number = 2 error_class=Mysql2::Error error="MySQL client is not connected"
fluentd    | 2020-01-14 08:40:57 +0100 [warn]: #0 Failed to import a record: retry number = 3 error_class=Mysql2::Error error="MySQL client is not connected"
fluentd    | 2020-01-14 08:40:58 +0100 [warn]: #0 Failed to import a record: retry number = 4 error_class=Mysql2::Error error="MySQL client is not connected"
fluentd    | 2020-01-14 08:40:58 +0100 [warn]: #0 Failed to import a record: retry number = 5 error_class=Mysql2::Error error="MySQL client is not connected"
fluentd    | 2020-01-14 08:40:59 +0100 [error]: #0 Can't recover undeterministic error. Dump a record error_class=Mysql2::Error error="MySQL client is not connected" record=#<Fluent::Plugin::SQLOutput::BaseModel_1539762876::Log id: nil, time: "2020-01-07 00:00:00", source: "Igetnow.Api.Logic.PusherManager", level: "Error", message: "Pusher error while checking online status of chann...", exception_name: "System.Net.Http.HttpRequestException", exception_message: "Cannot assign requested address ---> System.Net.So...", exception_stacktrace: "System.Net.Http.HttpRequestException: Cannot assig...">

That might be an issue to look into, For us there were a lot of errors gone witch we needed to prevent bigger damage.

And in addition to that. Is it possible to split the bucket size? So that there is not 250Mb Statement generated?