asksven
New 'combined' data export is not valid json
The new 'combined' data export when sharing (with json and plain text in one file) that was introduced in the latest version of BBS is not valid json and trying to parse it (with python or javascript, for example) throws errors even though the plain text is 'commented' out - sadly, json doesn't support comments in any form.
I see how this behaviour is a cleaner solution than what BBS did before (with the dialog showing every time) but now the json files are no longer usable without (pre)processing them before parsing.
Maybe having an option in settings to select what the export should contain (like a checklist) would be nice? That way the old popup isn't needed but it still gives us some control.
Holycrap I just tried a bunch of preg_replacers (php) to remove the plain text dump but that doesn't work at all. It seems as if the text is between comment-brackets (/* .. */) but there are entries that contain those characters INSIDE that block, like for example "*sync*/com.google.android.gms.fitness/com.google/[email protected]",
So the dumpfile is probably not even de-comment-able by python libs like jsmin or javascript JSON.minify
I know, that this solution is not ideal because Json does not support comments. As I can not control what is inside of the comments I put a hack on the line prior to the closing comment: --- human readable part ends here
I know it's a hack but in lack of a better solution it should allow for a two-line Regex for finding the 'right' closing element. If you have any suggestion for a better implementation please feel free to submit it here or as a pull request
Heh I completely overlooked that line (figures, there are a lot of lines :P). Well that does help. I'm currently doing an indexOf('*/\n{') and then substr it but this helps. I'll give it some thought. Including the dump in the json probably isn't really a good solution because that would increase the memory usage a lot..
(by including i mean making it part of the json, as in
"dump":"....lotsoflines...."
I recognised that the current implementation is not the most elegant. The reason why I combined both into one file is that I want to have a way to be able to import all dumpfiles provided by mail or in the xda thread:
- for stats purposes, to be able to do data mining/analytics in a future web-app
- for providing a means to import a dumpfile back into the app for interactive analysis (a feature that I plan to implement)
The trade-off in that case is about making sure that every human-readable file also contains the machine readable data as I don't want to write a parser.
Just out of curiosity: what kind of precession of the JSON are you doing/planning?
I have a custom built webapp I use toview the data and compare it with previous dumps. It's far from perfect but it does the job. I just put the file on Google Drive and the webapp (which runs on a local apache server on my computer) just reads the folder and builds a dropdown so I can select the dumpfile I want to review.
Who knows, I might revisit that project one day and see if I can build it in a proper way :P