Exception when sending post data to Log Analytics

[dcalap]

When using fluentd v1.11-1 and your last plugin version (0.7.0) we are facing an issue when sending data to Log Analytics to be ingested.

Basically, we are sending a JSON like this one:

{
   "anonymous_uid":"45645645645645",
   "event_date":"2021-05-11T04:00:00",
   "country_code":"ZZ",
   "country_name":"Unknown",
   "web_privacy":"PRIVATE",
   "info_level":"PERSONAL",
   "timezone":"america/new_york"
}

And we are getting this error from this line https://github.com/yokawasa/fluent-plugin-azure-loganalytics/blob/master/lib/fluent/plugin/out_azure-loganalytics.rb#L98:

2021-05-11 08:39:32 +0000 [fatal]: #3 [loganalyticis_custom_endpoint] Exception occured in posting to DataCollector API: 'undefined method `response' for #<SocketError:0x000055b025a4ee28>
Did you mean?  respond_to?', data=> [{"anonymous_uid":"45645645645645", "event_date":"2021-05-11T04:00:00", "country_code":"ZZ", "country_name":"Unknown", "web_privacy":"PRIVATE", "info_level":"PERSONAL", "timezone":"america/new_york"}]

This error happens from time to time, not on every request.

We are also seeing that when an http status code 503 from Log Analytics occurs, instead of going through line 100 (https://github.com/yokawasa/fluent-plugin-azure-loganalytics/blob/master/lib/fluent/plugin/out_azure-loganalytics.rb#L100) is catching the general exception in line 104 (https://github.com/yokawasa/fluent-plugin-azure-loganalytics/blob/master/lib/fluent/plugin/out_azure-loganalytics.rb#L104) as follows:

Exception occured in posting to DataCollector API: '503 Service Unavailable'

Could you take a look?

Thanks!

[tspearconquest]

It looks like its expecting an array of objects. The error indicates this by showing square brackets around the data value in your output.

v here Did you mean? respond_to?', data=> [{"anonymous_uid":"45645645645645", "event_date":"2021-05-11T04:00:00", "country_code":"ZZ", "country_name":"Unknown", "web_privacy":"PRIVATE", "info_level":"PERSONAL", "timezone":"america/new_york"}] ^ and here

As a workaround, if you're able to use the record transformer plugin, you could add the missing characters to the data so that it will be properly formatted. Not sure what the root cause is, whether this is something in the plugin or in your data set, but this might help those of you encountering this type of error.

Transform the record to add a [ before the opening { and a ] after the closing } only if those characters are missing.

[legion20]

Is it resolved? Still facing the same issue.