Adding Performance Tests to ODFDOM

[svanteschubert]

Our existing performance (esp. of ODFDOM) should become more transparent and we should become aware of updates that influence the performance.

Therefore should basic functionality of ODFDOM (the library which is used by the ODF Validator and the Simple API depends on - partly forked) bet tested by a performance regression test. We might want to start with something simple:

• Loading and saving a document other functionality as for instance table actions might follow.

The latest JDK 12 has updated the Java Micro Benchmarks to facilitate Java performace tests: https://openjdk.java.net/jeps/230

Anybody interested in taking a look into it?

Thanks in advance, Svante

[kv-gh]

If you have some sample documents, I could give this a stab.

[svanteschubert]

Cool, you may find sample documents in the folder: https://github.com/tdf/odftoolkit/tree/master/odfdom/src/test/resources/performance

Thanks in advance, Svante

[kv-gh]

Silly question: Is there a branch or fork of the project using JDK12? It looks the project uses 1.8...

I can bring in JMH via a dependency in the meantime:

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>1.19</version>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>1.19</version>
</dependency>

[kv-gh]

I've pushed the following to my fork of the repo...Please let me know if this was something in line with what you were looking for:

https://github.com/derrg/odftoolkit-1/commit/ec0f94fa550a46c4d1b1a367c86534b65eefb544

Thank you!

[svanteschubert]

That was quick! :-) Thank you! 👍 How do we trigger the performance test and where might be the best place to document its existence?

PS: Yes, it is still JDK 8 and my suggestion is to keep it simple, do quickly our first Maven release on TDF and update the JDK to be up to date with existing updates and technology features.

[kv-gh]

The test has a runner class (https://github.com/derrg/odftoolkit-1/commit/ec0f94fa550a46c4d1b1a367c86534b65eefb544#diff-b0a9e2ac591d88bd05137b54b02f453c) which will pick up any benchmarks annotated with @Benchmark.

As far as documentation, I can write up a how-to-use where the other testing documentation lives.

[kv-gh]

I don't know where that documentation lives, so any insight would be appreciated.

Thanks,

[svanteschubert]

Hi Ken, I was a little distracted from a different opensource project (synergies hopefully come later), but the apache documentation is under reconstruction at the moment on this branch: https://github.com/tdf/odftoolkit/tree/website The branch is basically copied from my work at https://github.com/svanteschubert/odftoolkit_website, where the /docs directory is being mapped to: https://svanteschubert.github.io/odftoolkit_website/ and in the last URL you can view the current state of documentation.. For instance, for ODFDOM this is the right place: https://svanteschubert.github.io/odftoolkit_website/odfdom/index.html and now I found a broken link: https://svanteschubert.github.io/odftoolkit_website/odfdom/Development.html If you (or others) like to assist, just ping me I am located in Berlin (UTC+2) and can give you some introduction if desired..

[kv-gh]

Hey Svante,

Quick update for you...I've added a benchmark to test saving ODF documents as well. I'm going to pull odftoolkit_website into my fork and make the appropriate updates there. I'll @ you when I have the some documentation up for review.

Wrt getting these to run via maven, it looks like I could set up the surefire plugin to run the benchmark runner. I'm more familiar with Gradle, so I'll probably ask for input and/or guidance when I get to that.

Thanks,

[svanteschubert]

Many thanks for your assistance, Ken! My current guess is that the performance test is quite related to integration tests. Those are running after the JAR is being build and not based on the classes in the directories, but from the JAR.

I looked them up, the configuration for integration tests should be one of the code snipplets from: https://stackoverflow.com/questions/1399240/how-do-i-get-my-maven-integration-tests-to-run We have already an integration test, which requires the JAR. It calls the JAR by command line and tests if it provides the correct version, name and build date as output. Otherwise, it would be pretty hard to realize what version someone is using. All integration tests should end in their names with "IT.java" and are currently within the project located at: tdf-odftoolkit/odfdom/src/test/java/org/odftoolkit/odfdom/integrationtest

The more visionary, upcoming goal would be to integrate such a performance test via Travis. But as the performance of the machine is not stable (some machine in the cloud), I can only imagine to run a reference JAR beforehand, where we know the performance from and use this as base line.

Finally, the most visionary goal: if we like to archive quickest build times (esp. test times), we should only run those tests that are influenced by the code being edited - when in doubt run & test everything(someone edited the pom.xml).

I can only imagine that this could work by code coverage & dependency graph, which seems "a little bit" sophisticated ;-)

But one step after the other, the visions are for later.. Thanks again for your help, Ken! :-) Svante

[kv-gh]

Hey Sven,

Is there a CI tool that we're looking at using? I can start to get the ITs set up to run on commit & PR if we do.

[svanteschubert]

Hey Ken,

Currently, we started using Travis, but if there is any good reason we might switch to anything else. Preferable runnable within GitHub and free for OpenSource. :-)

Good luck! And thanks in advance, Svante

Am Do., 25. Juli 2019 um 00:55 Uhr schrieb Ken V [email protected]:

Hey Sven,

Is there a CI tool that we're looking at using? I can start to get the ITs set up to run on commit & PR if we do.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tdf/odftoolkit/issues/10?email_source=notifications&email_token=AAGJNW44HWX6BKYXMKUUJB3QBDMUPA5CNFSM4HAE5LA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2X2V7Y#issuecomment-514829055, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGJNWYETOUVCANMG4ZWDOTQBDMUPANCNFSM4HAE5LAQ .