OOM while compiling the tests in HTTP module

[TharmiganK]

Description:

Compiling ballerina tests after updating the lang version and the dependent versions compatible with latest lang gives a OOM

Steps to reproduce:

Use the update-lang-version branch of this HTTP repo

Delete the Dependancies.toml files and run the gradle build with tests.

The following logs were printed :

Compiling source
        ballerina/http_tests:2.4.0
java.lang.OutOfMemoryError: Java heap space
Dumping heap to $(pwd) ...
Heap dump file created [1766375770 bytes in 17.196 secs]
Exception in thread "OkHttp ConnectionPool" java.lang.OutOfMemoryError: Java heap space
        at java.base/java.util.ArrayDeque.iterator(ArrayDeque.java:671)
        at okhttp3.internal.connection.RealConnectionPool.cleanup(RealConnectionPool.java:173)
        at okhttp3.internal.connection.RealConnectionPool.lambda$new$0(RealConnectionPool.java:55)
        at okhttp3.internal.connection.RealConnectionPool$$Lambda$141/0x0000000800264440.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
ballerina: Oh no, something really went wrong. Bad. Sad.

We appreciate it if you can report the code that broke Ballerina in
https://github.com/ballerina-platform/ballerina-lang/issues with the
log you get below and your sample code.

We thank you for helping make us better.

[2022-07-20 08:37:12,186] SEVERE {b7a.log.crash} - Java heap space 
java.lang.OutOfMemoryError: Java heap space
        at java.base/java.util.HashMap.resize(HashMap.java:700)
        at java.base/java.util.HashMap.putVal(HashMap.java:659)
        at java.base/java.util.HashMap.put(HashMap.java:608)
        at org.wso2.ballerinalang.compiler.semantics.model.SymbolEnv.lambda$cloneSymbolEnvForClosure$1(SymbolEnv.java:382)
        at org.wso2.ballerinalang.compiler.semantics.model.SymbolEnv$$Lambda$246/0x00000008003dd840.accept(Unknown Source)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at org.wso2.ballerinalang.compiler.semantics.model.SymbolEnv.cloneSymbolEnvForClosure(SymbolEnv.java:382)
        at org.wso2.ballerinalang.compiler.semantics.model.SymbolEnv.createClone(SymbolEnv.java:361)
        at org.wso2.ballerinalang.compiler.desugar.AnnotationDesugar.addReturnAndDefineLambda(AnnotationDesugar.java:806)
        at org.wso2.ballerinalang.compiler.desugar.AnnotationDesugar.defineAnnotations(AnnotationDesugar.java:680)
        at org.wso2.ballerinalang.compiler.desugar.AnnotationDesugar.defineFunctionAnnotations(AnnotationDesugar.java:491)
        at org.wso2.ballerinalang.compiler.desugar.AnnotationDesugar.rewritePackageAnnotations(AnnotationDesugar.java:173)
        at org.wso2.ballerinalang.compiler.desugar.Desugar.desugarAnnotations(Desugar.java:816)
        at org.wso2.ballerinalang.compiler.desugar.Desugar.visit(Desugar.java:773)
        at org.wso2.ballerinalang.compiler.tree.BLangPackage.accept(BLangPackage.java:167)
        at org.wso2.ballerinalang.compiler.desugar.Desugar.rewrite(Desugar.java:8552)
        at org.wso2.ballerinalang.compiler.desugar.Desugar.visit(Desugar.java:807)
        at org.wso2.ballerinalang.compiler.tree.BLangPackage.accept(BLangPackage.java:167)
        at org.wso2.ballerinalang.compiler.desugar.Desugar.rewrite(Desugar.java:8552)
        at org.wso2.ballerinalang.compiler.desugar.Desugar.perform(Desugar.java:455)
        at io.ballerina.projects.internal.CompilerPhaseRunner.desugar(CompilerPhaseRunner.java:216)
        at io.ballerina.projects.internal.CompilerPhaseRunner.performBirGenPhases(CompilerPhaseRunner.java:152)
        at io.ballerina.projects.ModuleContext.generateCodeInternal(ModuleContext.java:447)
 
ERROR [http_tests:(1:1,1:1)] Compilation failed due to: Java heap space

Heap dump can be found here : https://drive.google.com/file/d/1GXyMRpA4pSTjEEWpkcHyT4aQgsnjAMdE/view?usp=sharing

[gimantha]

https://github.com/TharmiganK/module-ballerina-http/blob/update-lang-version/ballerina-tests/Dependencies.toml Is this the file you are referring to delete? @TharmiganK

[TharmiganK]

Yes, both of these Dependencies files : https://github.com/TharmiganK/module-ballerina-http/blob/update-lang-version/ballerina/Dependencies.toml https://github.com/TharmiganK/module-ballerina-http/blob/update-lang-version/ballerina-tests/Dependencies.toml

Just remove all the content(no need to delete) before building the module.

[TharmiganK]

This was happened when we update the lang version to RC1 and other standard library modules to the latest timestamped version which is also updated with RC1.

If I only update the lang version to RC1, I did not see any OOM and tests are running fine. And then I tried to update other standard library versions level by level. So when I update the IO version, still there is no OOM issue in running tests. But when I update the Log version then the tests compilation take a long time and finally ends up in OOM

[gayalkuruppu]

https://github.com/ballerina-platform/module-ballerina-c2c/pull/565 PR fails with OOM

[KavinduZoysa]

We started to debug the lang while http tests are running. What we have noticed is that test files are hung on this loop. So we have commented the body of the loop and run the tests again. Tests run successfully. After doing the above change, we have run tests several times. Still, there is not an OOM. @pramodya1994 could you please check this?

[TharmiganK]

I have also tested the HTTP module by removing all the usage of log and then the tests are running fine without OOM. So adding @MadhukaHarith92 also to this issue, since this may be related to the code modifier usage in log module.

[MadhukaHarith92]

There is a similar code modifier usage in the log module[1] to the ballerina-lang test case @KavinduZoysa had mentioned.

[1] https://github.com/ballerina-platform/module-ballerina-log/blob/master/compiler-plugin/src/main/java/io.ballerina.stdlib.log.plugin/LogCodeModifier.java#L66

[gimantha]

Please re-assign to Kavindu if this needs to be addressed from FE side.

[daneshk]

@pramodya1994 Can you check and let us know whether we need to make any changes in the code modifier to avoid this

[pramodya1994]

@daneshk We are checking this from our side, I will update you soon.

[pramodya1994]

As per @KavinduZoysa's comment and some offline discussions we did an improvement to the modifyPackage method to modify module for all source documents only once. However, we are still getting the OOM error. I have taken 2 dumps before the improvement and after the improvement. They are almost the same.

Before improvement

After improvement