NNAnalytics
NNAnalytics copied to clipboard
paypal
NameNodeAnalytics is a self-help utility for scouting and maintaining the namespace of an HDFS instance.
Hadoop cluster version is 3.0.0. After the platform starts normally, the data cannot be updated. I keep refreshing /info, but Current TxID: X don't change.  Could you give me...
It occurred to me that the testing I did with AirConcurrentMap could have been better validated if I had had a benchmark test against the SuggestionEngine. This is relatively straightforward...
It would be very useful to see metrics around deletions available in the API. Some useful metrics: - Count of files deleted grouped by owner - Count of files deleted...
Assuming a valid hbase-site.xml exists, we can use it to find the HBase parent directory in HDFS and then perform directory analysis on it. Some early possibilities if we are...
Assuming a valid hive-site.xml, it will be possible to determine the active hive warehouse HDFS directory and HiveServer2 and Metastore URIs. From there we should be able to perform a...
Just a reminder issue to provide some examples around how to use the SQL page. The following are some example calls: Some documentation already exists [here](https://nnanalytics.readthedocs.io/en/latest/REST_Endpoints/SQL/), but no examples. Example...
Descirbe the problem: First I install NNAnalytics on hadoop3.0 according to official configuration, then copy hdfs-site.xml and core-site.xml to NNAnalytics config. when start the nna, errors met.the nna has stated,...
Keeping in line with getting to Apache adoption, the next major refactoring stretch is to remove the old SparkJava webserver and replace it with the Jetty-based NameNodeAnalyticsHttpServer (which is just...
In NNA today, particularly if you look around here: https://github.com/paypal/NNAnalytics/blob/b17e8e6d91fd853b23f67a0b3ed0c5c95c2d8788/src/main/java/org/apache/hadoop/hdfs/server/namenode/cache/SuggestionsEngine.java#L161-L165 You will see that NNA uses a hardcoded cut off of 128 Megabyte block sizes to distinguish between "Medium Files"...
Assuming we have access to a valid yarn-site.xml, it should be trivial to determine if log aggregation is enabled and where in HDFS those logs are being stored. We can...