Lihadoop 39635.2

[edwinalu]

Add new configuration parameters heuristic, which will list the current values for configuration parameters, and also recommended new values. To determine new values, it will check for:

• execution memory spill: this will slow down the application, so try to prevent this by increasing partitions, increasing executor memory, or decreasing cores.
• long tasks: this will slow down the application, so try to prevent this by increasing the number of partitions.
• task skew: this will slow down the application, so add recommendations for making partitions more even.
• OOM or GC: increase memory, increase partitions, or decrease cores, to try to avoid the error.
• container killed by YARN errors: increase overhead memory.
• unused executor memory, if this is much higher than max JVM used memory; either increase cores or decrease memory.
• driver configuration parameters (memory and cores).