Utf8Json
Utf8Json copied to clipboard
neuecc
Started a fork/v2 at https://github.com/vilinet/Utf8Json-v2
I know it is not the nicest way to introduce a fork of a repo but i had no other choice. I asked the author but my messages were ignored.
The v2 can be found at https://github.com/vilinet/Utf8Json-v2/ I cleaned up the code a bit, added new frameworks. And already fixex some issues but there are many left.
I hope we can make this stunning library even better :) Its alive!
@vilinski @RamType-0 @st-gwerner @SaAnnka @neon-izm @Kiryushin-Andrey @AndrewGumenyuk @ pCYSl5EDgo
do you really have a time and energy to maintain the fork? are you sure @neuecc has really stepped down? I mean I would appreciate to see this lib evolving and bugfixed. But... not if it results in another "padleft" hell.
@neuecc said that he gonna work with this repo after he release UniTask v2
@neuecc said that he gonna work with this repo after he release UniTask v2
That is great news!
It does not matter to me which version just need to keep this stuff alive.
I made my repo private :) Any idea how long it will take?
I have seen he is is active on many other projects.
I'm relieved to hear he may start giving this attention, but I would certainly be happier if there was more than one maintainer. Even if it means that it requires unanimous consensus on all pull requests, that'd be better than the feeling that things were abandoned for a couple years. There are a few fixes that we all can look at and agree would be good, but without him personally approving them, the issues linger, and that results in way too many forks with the same fix.
I would welcome a well maintained fork. The goal here is to have a low allocation UTF8 aware serializer for JSON in .NET.
The point in the MIT license in to foster innovation. If @neuecc (who has done a great job here) does get the time to come back to maintain this library, then all we will have is 2 great libraries to choose from. Maybe there will be some differences that will make each more suitable for different applications, but that also sounds great to me.
I think calling it the same thing is probably unfair though and could create confusion, you should probably come up with a new name, and then maintain recognition for the original base.
Would be much better if everyone could concentrate on one lib instead decentralizing manpower into 2 separated libs.
Best option really would be to share ownership with someone who could review and approve PR. Or to have a "bugfix branch" that we can take care of. We could also release some bugfix packages in prelease status or just simply increasing the build version.. Whatever.
There are fundamental very basic and simple bugs and they have not been fixed in the last 2 years. And it blocks many people from using it. (like its not supporting json escape sequences).
I know many people who look at the repo and they just dont use it as they dont thrust in an abandoned codebase that have these basic issues untouched.
And the best is how the owner does not even write a single comment ...
I don't see it such essential to justify a fork. Json serializing is a not complex operation. You can always take another format if it is not fast enough. And there is already decent System.Text.Json. And if you not aware there is also https://github.com/Dogwei/Swifter.Json it claims to be even better than anything else, but you probably need google translate to read the docs ))
@vilinski, hello do you think you can share your private link? I'd like to see if we can use it while waiting for the author.
@vilinski, hello do you think you can share your private link?
@markmadlangbayan you surely mixed me up with @vilinet ? Don't know why he even has my nick :D
@vilinet hi we are encountering a critical & security issue in this library: https://github.com/neuecc/Utf8Json/issues/224 - this \u0003 doesn't serialeze properly causing the front end to crash. I hope you could help us and for the community as well, is there a fix or workaround for this?
Drop me a mail to the vilinet9@ the mail domain that google uses dottt com :)
Issue #224 sounds a lot like something several of us have fixed, including a PR I submitted #217.
@st-gwerner it works! Thank you so much.
Hey guys @vilinet, i think while the main author is not yet around and looks like this repo is not active anymore, it would be better to create a V2 Fork of this repo and include all the current pending Pull Requests especially those who are bug fixes.
@st-gwerner Yeah definitely a fix. But also would make string writing much slower because of that condition check on every single character and also copying. Also that array access is not the fastest either.
I would do something like this: https://pastebin.com/xfYC7f3z No additional if, no array access, conversion to hex is faster than the convert one. sure the common stuff could be "aggressivly" inlined though c# compiler most of the times does not care about that either :D
@firephantomassasin Yeah but also there are many PR that would slow it down serialization. Of course sometimes there is now way to avoid it but many of them can be improved.
@vilinet All the more reason that your idea of a single repo would be amazing, managed by multiple heads who can discuss and evaluate the best ways to do it. My focus was getting correct behavior with the same relative performance (in my testing of 40-50MB files, the check didn't slow performance much at all), but I would love it if we had a central repo where we can bat things around, do additional testing, etc.
I also agree with you also that not all "fixes" are good.
@st-gwerner Yeah :) Also if the original author decides to continue he can use the actual up-to-date code as a base. Maybe it should not be called v2 but something like utf8json-patches / fixes. Dont know.
Also i checked that Swifter.Json that was linked above and i need to stay that is much more faster then this lib(by 35% with my test data), but also has less feature. But this code base is much more "contributor" friendly and there are things we could use from that lib as well.
@vilinet just wonder, which features you miss in Swifter.Json compared to Utf8Json? For me no support for F# types makes it currently useles, but author already works on it.
@vilinski No simple way for camel case output for example. You can use the callback but perfermance drops heavily. Or you rename your properties to use camel case.
No attribute support to ignore/rename properties.
@vilinet I opened an issue #209 a short while ago. Being able to easily define your own naming formats would be very handy in any fork, and not hard to expose without slowing anything down. We have to consume a lot of weird APIs at our shop, and having to put attributes on everything is annoying.
@vilinet not yet tried myself, but there are RWField and RWFielAccess attributes and also other JsonFormatterOptions
@st-gwerner @vilinet have you checked this another security issue in this library? https://github.com/neuecc/Utf8Json/issues/127
@vilinet If you think that you have time to continue this project, please go on with the fork. It is badly needed.
Okay guys. So i made this https://github.com/vilinet/Utf8Json-v2/ public again. I already released a 1.5 nuget package with some crucial issue fix but raised a ticket also to have a discussion how things should go on(like versioning).
So please have a look :) We can also decide to stop it and we will switch to another library. (There are some nice candidates)
@vilinet
Do you really think someone will work on this new repo if you deleted the history and just copied all the source code in the initial commit!? https://github.com/vilinet/Utf8Json-v2/commit/fdf4515d7cac2bef8e4b700706ca98aad0a08301
- It's not discoverable as a fork
- It's very hard to understand what's going on in complex places without history
- It's hard to see what and where you added/removed from original
Not related to this library, but recently I forked a 5-years-old near 1k-stars library and I spent almost a day just to clean and keep Git history, otherwise I could hardly navigate and understand what was going on in this 3rd party codebase. This is important.
In my fork of this library I try to force-push to have as few commits as possible and view them as patches rather than full-blown continuation of upstream development (I do not have skills and resources for that, do you?). There are multiple "natural" forks of this library and it's easy to cherry-pick bug-fixes from there (or from PRs) and keep the changes to upstream visible as isolated patches. It would be great if there is one place that accumulates all useful PRs and fork commits in one place, but keeps the development anchored to the upstream.
My fork is incompatible with the upstream (no explicit Unity support and I do not know if it works there, changed internals so it works with native memory and not only with byte[], serializes tuples as JS arrays and other changes; but faster), so it's not even a candidate for such a place. But I do periodically review if there are worthy changes and cherry-pick them. Your "fork" will be completely invisible to me.
So, ideally and in my opinion, a new centralized fork should have all worthy PRs and commits from other forks that are cherry-picked from their branches (or merged with their authors kept in git history). Anyone should be able to cherry-pick from this fork or rebase their fork on the new centralized fork. This also makes it possible to backport all community contributions to the upstream in a couple of clicks whenever it is active again. Without this any "artificial"/"forced" fork adds more hassle than value.
@vilinet what are those another library (There are some nice candidates) ? So far i've tried other libraries and this one gives the fastest performance.