ozone icon indicating copy to clipboard operation
ozone copied to clipboard

HDDS-7316. Print stacktrace to identify the location of RocksObject leaks.

Open sadanand48 opened this issue 2 years ago • 1 comments

What changes were proposed in this pull request?

Add logging to identify location/trace of unclosed RocksObjects .

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-7316

How was this patch tested?

Tested on docker with debug logs enabled , identified an instance and fixed in this PR

[34mrecon_1     |[0m 2022-10-12 13:56:51 WARN  ManagedRocksObjectUtils:49 - RocksIterator is not closed properly
[34mrecon_1     |[0m 2022-10-12 13:56:51 DEBUG ManagedRocksObjectUtils:52 - StackTrace for unclosed instance: org.apache.hadoop.hdds.utils.db.managed.ManagedObject.<init>(ManagedObject.java:34)
[34mrecon_1     |[0m org.apache.hadoop.hdds.utils.db.managed.ManagedRocksIterator.<init>(ManagedRocksIterator.java:28)
[34mrecon_1     |[0m org.apache.hadoop.hdds.utils.db.managed.ManagedRocksIterator.managed(ManagedRocksIterator.java:32)
[34mrecon_1     |[0m org.apache.hadoop.hdds.utils.db.RocksDatabase.newIterator(RocksDatabase.java:492)
[34mrecon_1     |[0m org.apache.hadoop.hdds.utils.db.RDBTable.iterator(RDBTable.java:161)
[34mrecon_1     |[0m org.apache.hadoop.hdds.utils.db.TypedTable.iterator(TypedTable.java:280)
[34mrecon_1     |[0m org.apache.hadoop.ozone.recon.spi.impl.ReconContainerMetadataManagerImpl.initializeKeyContainerTable(ReconContainerMetadataManagerImpl.java:617)
[34mrecon_1     |[0m org.apache.hadoop.ozone.recon.spi.impl.ReconContainerMetadataManagerImpl.initializeTables(ReconContainerMetadataManagerImpl.java:135)
[34mrecon_1     |[0m org.apache.hadoop.ozone.recon.spi.impl.ReconContainerMetadataManagerImpl.<init>(ReconContainerMetadataManagerImpl.java:88)
[34mrecon_1     |[0m org.apache.hadoop.ozone.recon.spi.impl.ReconContainerMetadataManagerImpl$$FastClassByGuice$$7c269485.newInstance(<generated>)
[34mrecon_1     |[0m com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
[34mrecon_1     |[0m com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:61)
[34mrecon_1     |[0m com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:105)
[34mrecon_1     |[0m com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
[34mrecon_1     |[0m com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:267)```

sadanand48 avatar Oct 12 '22 14:10 sadanand48

@duongkame please review

adoroszlai avatar Oct 13 '22 18:10 adoroszlai

Thanks @sadanand48 for the patch, @duongkame, @kerneltime for the review.

adoroszlai avatar Oct 27 '22 15:10 adoroszlai