Application log plugin weird symbols

[ksemaev]

When I use Application log plugin it outputs some meaningless symbols in log file like:

_�� o� � [<�m�I"� �`�87�"q���{"txid":"0xc4ff0271227f37389a601d9f200e00b72249896da23c5b000a00141fe40aea6f","executions":

What is that? can we use some ASCI characters in log please?

[ksemaev]

The .ldb files (which I assume is Microsoft Access files) are not readable on linux at all. They look like:

p � � � � � �

,>Pbt�������(:L^p������ 0BTfx������� -?Q������ 1Cs������.@Rdv�����$6HZl�����,>Pbt�������(:j|����$6HZl~������� 2DVhz�����(Xj|���$6HZl~����DVhz����"4FXj|�����<N`r�������'9K]o��������/ A S e w � � � � � !!%!7!g!y!�!�!�!�!�!�!�! "-"?"Q"c"u"�"�"�"�"�"�"###S#e#w#�#�#�#�#�#$$%$7$g$y$�$�$�$�$�$%%'%W%i%�%�%�%�%�%&#&S&�&�&�&�&�&�&�&'O'a's'�'�'�'�' (-(?(Q(c(**O*�����*)q)�)�)�+9+K+]+o+�+�+�+�+�+�+ ,,/,A,S,e,w,�,�,�,�,�,�,�,%-7-I-y-�-�-�-�-.!.3.E.W.i.{.�.�.�0=0O0a0s0�0�0�0�0�0�01!131E1W1i1{1�1�1�1�1�1�1�1 22M2}2�2�2�2�2�233+3=3O3a3s3�3�3�3�3�3�34!434E4W4i4{4cg�^

[shargon]

ldb is leveldb files, you need to open it with a leveldb reader

[ksemaev]

TY @shargon But what about 00000062.log etc files? They require leveldb reader as well? they also include [<�m�I"� �`�87�"q���{"txid":"0xc4ff0271227f37389a601d9f200e00b72249896da23c5b000a00141fe40aea6f","executions":

[shargon]

is required by leveldb

[lock9]

@shargon shouldn't it save only valid json? How can we guide users to read the log files? The file should either follow a standard (json) or at least have proper instructions to read it.

[shargon]

Currently is stored in the same engine, so in levelDB format.

[lock9]

@shargon where can we find instructions about the format of the notification? I had the same problem in the past, it is not intuitive. It saves the notifications but it doesn't seem to be possible to read them (it is, but it lacks instructions)

[ksemaev]

@lock9 I must say that instructions are not solving the case. Production environment has clusters with tones of apps, logs are collected and parsed, then stored in elasticsearch, where they can be searched/filtered/investogated. The only proper solution is readable json.

[lock9]

@ksemaev I agree with you. Also, leveldb locks the file, meaning we cannot read with external tools. I think the issue of having one json per notification is the number of files it will create. We need some kind of "json" adapter, to save it in json and also merge multiple notifications into one larger file. Maybe group notifications from 100 blocks at a time? 1000? We need a plugin for the plugin 😅

Do you have any solution design you think suits your needs @ksemaev ? I know that we can't create one file per notification, otherwise it will break in ubuntu servers.

[ksemaev]

@lock9 TY for asking :) I personally use kubernetes, so for me the best option is send logs to stdout in json format: https://github.com/neo-project/neo-plugins/issues/106

[lock9]

@ksemaev that is great, if that suits your needs, I think we can go for that. It is very easy to implement (I think). I will remove the low priority tag from that issue

[shargon]

Take a look to this plugin https://github.com/neo-project/neo-plugins/blob/master/SystemLog/Logger.cs

[lock9]

@shargon will it print notifications in json format into the output?

[shargon]

No, but it could be adjusted. currently only print the log (not the same as Log/notification of ApplicationEngine)

[hal0x2328]

Also, leveldb locks the file, meaning we cannot read with external tools.

RocksDB doesn't have this issue - you can have additional read-only connections to the db from external code. (Another point in its favor I didn't see mentioned in https://github.com/neo-project/neo/issues/966)