error happend when view daemon errors pages

[comicfans]

hello, I'm running fluentd-ui v1.1.0, fluentd 1.2.6, when accessing daemon errors page,

browser shows

We're sorry, but something went wrong. If you are the application owner check the logs for more information.

and fluentd ui log shows following information:

I, [2018-10-13T18:40:43.458550 #9920] INFO -- : [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] Started GET "/daemon/errors" for 127.0.0.1 at 2018-10-13 18:40:43 +0800 I, [2018-10-13T18:40:43.464120 #9920] INFO -- : [86c2bd27-7048-47ea-8f66-1277919082dd] Completed 200 OK in 56ms (Views: 0.6ms) I, [2018-10-13T18:40:43.467729 #9920] INFO -- : [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] Processing by FluentdController#errors as HTML I, [2018-10-13T18:40:43.536376 #9920] INFO -- : [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] Completed 500 Internal Server Error in 68ms F, [2018-10-13T18:40:43.537201 #9920] FATAL -- : [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] F, [2018-10-13T18:40:43.537254 #9920] FATAL -- : [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] NoMethodError (undefined method []' for nil:NilClass): F, [2018-10-13T18:40:43.537283 #9920] FATAL -- : [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] F, [2018-10-13T18:40:43.537321 #9920] FATAL -- : [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] app/models/fluentd_log.rb:115:in block in split_error_lines_to_error_units' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] app/models/fluentd_log.rb:113:in each' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] app/models/fluentd_log.rb:113:in each_with_index' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] app/models/fluentd_log.rb:113:in split_error_lines_to_error_units' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] app/models/fluentd_log.rb:75:in block in logged_errors' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] lib/file_reverse_reader.rb:74:in block in split_each_line' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] lib/file_reverse_reader.rb:73:in each' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] lib/file_reverse_reader.rb:73:in split_each_line' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] lib/file_reverse_reader.rb:48:in read_rest' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] lib/file_reverse_reader.rb:15:in block in each_line' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] lib/file_reverse_reader.rb:13:in loop' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] lib/file_reverse_reader.rb:13:in each_line' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] app/models/fluentd_log.rb:58:in logged_errors' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] app/models/fluentd_log.rb:16:in errors_since' [a8c60824-9bc3-4a19-bd35-8d42f6f9e20e] app/controllers/fluentd_controller.rb:44:in errors'

seems it happend after I've sent some utf8 log to fluentd (I'm not sure about that), and this problem can't recovery by restart fluentd-ui . any suggestion ?

[okkez]

Could you show me reproducible small fluent.log?

[comicfans]

seems that nothing special happened in fluentd.log, it just runs as normal

2018-10-16 15:45:05 +0800 [info]: #0 Oj is not installed, and failing back to Yajl for json parser 2018-10-16 15:45:05 +0800 [info]: adding source type="monitor_agent" 2018-10-16 15:45:05 +0800 [info]: adding source type="debug_agent" 2018-10-16 15:45:05 +0800 [warn]: section is not used in of buffered_slack plugin 2018-10-16 15:45:05 +0800 [warn]: section is not used in of buffered_slack plugin 2018-10-16 15:45:05 +0800 [info]: #0 starting fluentd worker pid=1555 ppid=1550 worker=0 2018-10-16 15:45:05 +0800 [info]: #0 listening dRuby uri="druby://0.0.0.0:24230" object="Fluent::Engine" worker=0 2018-10-16 15:45:05 +0800 [info]: #0 listening port port=24224 bind="0.0.0.0" 2018-10-16 15:45:05 +0800 [info]: #0 fluentd worker is now running worker=0 2018-10-16 15:45:05.514385539 +0800 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}

[comicfans]

fluentd log size 150GB, maybe it's too big ?

[okkez]

It's too big, I think. Could you reduce the size to about 10-100MB using binary search?

BTW, you should rotate fluentd log.

[comicfans]

truncate it down to 300MB still not work, clear it to zero resolve this.