shark icon indicating copy to clipboard operation
shark copied to clipboard

This patch addresses two cache issues

Open sundeepn opened this issue 11 years ago • 7 comments
trafficstars

  1. 'cache tablename' directive overwrites existing memory tables and can cause memory leaks when the cache command is used on an already cached table. caching new partition also causes a memory leak if the parition already exists.
  2. using 'cache' directive on a partitioned table causes table to be corrupt. Only the last cached partition is accessable with the rest being leaked.

sundeepn avatar Feb 26 '14 01:02 sundeepn

@rxin - Please review.

sundeepn avatar Feb 26 '14 01:02 sundeepn

Merged build triggered.

AmplabJenkins avatar Feb 26 '14 02:02 AmplabJenkins

Merged build started.

AmplabJenkins avatar Feb 26 '14 02:02 AmplabJenkins

#2 is actually a bug in SparkLoadTask#loadPartitionedMemoryTable(), I'll submit a fix in a bit

harveyfeng avatar Feb 26 '14 02:02 harveyfeng

This should address it: https://github.com/amplab/shark/pull/296

harveyfeng avatar Feb 26 '14 03:02 harveyfeng

Merged build finished.

AmplabJenkins avatar Feb 26 '14 03:02 AmplabJenkins

All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/Shark-Pull-Request-Builder/12151/

AmplabJenkins avatar Feb 26 '14 03:02 AmplabJenkins