fluent-plugin-bigquery icon indicating copy to clipboard operation
fluent-plugin-bigquery copied to clipboard

Published 20 hours ago • verified publisher icon fluent-plugins-nursery

lexical error: invalid bytes in UTF8 string

Open mac2000 opened this issue 8 years ago • 1 comments

periodically I do catch following error:

2017-07-29 11:46:43 +0300 [warn]: #0 plugin/output.rb:1096:block in update_retry_state: failed to flush the buffer. retry_time=0 next_retry_seconds=2017-07-29 11:46:43 +0300 chunk="55570d4efaf3ca91e7b235dffef577db" error_class=MultiJson::ParseError error="lexical error: invalid bytes in UTF8 string.\n          w=1&hl=ru&ie=windows-1251&q=\xF0\xEE\xE1\xEE\xF2\xE0 ua\",\"host\":\"rabota.ua\",\"s\n                     (right here) ------^\n"

not sure is it bigquery or fluentd or ruby issue

after looking around I have added .scrub.strip to all fields which may cause this issue in record_transformer filter but still with no luck

mac2000 avatar Jul 29 '17 10:07 mac2000

I can confirm this issues is happening to me too. It appears tob ean underlying library, but I'm not 100% sure yet.

GLStephen avatar Jun 18 '18 23:06 GLStephen