Is there any way to make this dependency any lighter?

[bernardosulzbach]

I am talking about disk usage. Yes, 2.7 MB is getting... troublesome in one scenario I have. Can you give me any hints on an easy way to cut this down (the more the better)?

I only need FSTObjectInput and FSTObjectOutput as drop-in replacements for Java's ObjectInputStream and ObjectOutputStream. Should run under OpenJDK 6 (or newer) and Oracle JDK 6 (or newer). I see that some dependencies are there because of JSON, Android, etc. Maybe I can make my own build without them?

I tried to pretty print this so that it can be used as a reference question in the future.

Your help is much appreciated.

[RuedigerMoeller]

yes. for pure serialization to work (no android, no json, no structs) there is logically zero dependency. However you need jdk 1.7. There probably will be runtime errors due to static init code and stuff. Needs to be converted into Class.forName'y style code then ..

for 1.6 you need the fst 1.x branch, however this is unsupported and many bug fixes have not been backported.

[bernardosulzbach]

Yeah, supposedly. I went check it before I replied. I've been using 2.33 successfully with OpenJDK6 for a while now. I am not making this library with Java 6, I just USE it with Java 6.

java -version

    java version "1.6.0_36"
    OpenJDK Runtime Environment (IcedTea6 1.13.8) (6b36-1.13.8-0ubuntu1~14.04)
    OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)

in pom.xml

    <dependency>
        <groupId>de.ruedigermoeller</groupId>
        <artifactId>fst</artifactId>
        <version>2.33</version>
    </dependency>

But that's what I thought, I would need to make some corrections myself to get it to build before using it without its dependencies. You wouldn't know of a similar open source library out there, would you?

[RuedigerMoeller]

there is kryo, however its not compatible and sometimes requires some extra work to make classes serializable. I don't know of a similar project to fst (that's why I created it :) ).

You can do a branch of 2.4x and get it to run on 1.6 (1.7 would be better), I might merge the diff.

Even better would be a pull request achieving what you want without breaking json, structs + offheap.

[bernardosulzbach]

Oh, you are missing the point. I already have a solid, magic piece of software running with amazing results under Java 6. Thanks you for starting it (+1). I just need the dependency to be lighter. I need exactly to break JSON, structs, etc. For my project. I might make it open source in the end.

[RuedigerMoeller]

Hm .. I am confused now. Serialization does not need the dependencies, so all to be done is to modify the code it does not throw an (unnecessary) "ClassNOtFound" during init. This way one could simply drop the dependencies as long only fst serialization is used. If you have already done that, is there a github repo with that ?

[bernardosulzbach]

Will look into doing it. No need to be confused, that's exactly what I was looking for.

[RuedigerMoeller]

;)

[bernardosulzbach]

You are mad FAST replying. God.

[RuedigerMoeller]

always on :)

[korniltsev]

Any news on the issue? I'm going to strip the deps on my fork for my internal needs like this (jackson removed) https://github.com/korniltsev/fast-serialization/commit/dda3312f9239f439b828e62fbcb8ccfea24c4a3d I have not looked into code preciesely but it looks like I can do the same with javasist ( I am not really sure javassist works well on android ). @RuedigerMoeller do you have plans on making dependencies optional? I can try to work on PR if you let me.

[RuedigerMoeller]

I am accepting pull request, however stripping dependencies requires some careful work, I currently have not the need for that feature and no time/motivation to work for other peoples needs ;).

• its probably not a good idea to pay a performance price.
• if using FST to connect to a backend from within an android client, its very likely to use JSON encoding because of versioning safety

[korniltsev]

I am an android engineer and the reason I want fst to be smaller(modularized) is 65k dex method limit. In short: good android applications can't have more than 65k methods and fst is huge and brings 9k of them.

There are certainly usecases when you need only some part of the lib, for example default/android serialization and don't need json,kson,minbin,structs. In fact we use fst at ok.ru and when we run release build, proguard removes json/kson,minbin silently because it is not used. The problem is javasist&structs. We don't use it and afaik we can't use it on android. However since the FstStruct.class and some other classes are used in default configurations, proguard can't shrink it.

So I have an Idea of splitting the lib in modules with no performance price paid at all. Here is my plan:

1. Fix tests. https://github.com/RuedigerMoeller/fast-serialization/pull/162
2. Create a branch 'modularization' and create all next prs agains it. This way we do not pollute master untill we are sure it works/ready.
3. PR#1 Create module 'core' and move all the sources & tests to core module.
4. PR#2 Create separate module 'structs' and move all the structs, offheap, onheap, javasist stuff to this module. I should start with structs first, because it is the hardest part and there are some hardcoded stuff in FSTClazzNameRegistry#classForName which makes this move not straightforward. We can probably reuse LastResortResolver to move this stuff("_Struct", "_ActorProxy") aside.
5. PR#3 Create separate modules for json,kson,minbin. It is pretty easy and straightforward.
6. Write migration guide since API will definetly change(slightly)
7. Merge modularization' into master

This way I wil use fst-core on android and if I need I can add fst-json and others.

One more thing. I would like to know the reason behind desicion to keep both gradle and maven in the repo. It is tedious and error-prone to keep both of them. Especially if we agree to modularize fst. Here is an example to convince you leave only one build tool: there is already jackson dependency mismatch. In pom it is 2.8.1 and in gradle it is 2.5.3.

So @RuedigerMoeller please let me know what you think about it. Please let me know if I can start work on it.

PS: If anyone interested, I've already forked & removed all but default/android configurations for my internal needs here https://github.com/korniltsev/fast-serialization/tree/ok_2.47 .

[simonrob]

Another vote for this - it would be great to just be able to import the Android-related components.

@korniltsev - do you have a ProGuard configuration for FST that you could share?

[korniltsev]

@simonrob No I don't have one. Json and minbin parts are removed by proguard automatically(if you don't use them) and AFAIK javasist and structs parts can't be removed by proguard since they are used in FstStruct, default configuration classes. We've been using our fork mentioned in the previous message for a while, but now we don't and I do not recommend to use the fork.

[RuedigerMoeller]

I'll split+cleanup fst with version 3.0, however it might take some time until then.

[korniltsev]

@RuedigerMoeller please don't! its probably not a good idea to pay a performance price.

[RuedigerMoeller]

There won't be any performance price.

• javassist is required only for fst.structs which is a very specialized hack and could be separated in a library.
• minbin can be removed completely, its used rarely, in addition JSon can be used instead of minbin
• jackson+json serialization can be moved to a separate extension jar.
• objenesis is required, but this one is smallish