Terry Kim

Results 101 comments of Terry Kim

> For source data files, we need to use Parquet's `mergeSchema` as the data could have columns not listed as indexed or included columns. What if source data files are...

Per @sezruby: I think we can utilize or refer TreeNode.toJSON to create a JSON string for a logical plan. A possible problem is that it serializes LogicalRelation (and maybe other...

> * Option1) Not support refresh function across different versions of Spark > > * Keep the spark version at index creation > * index summary string fix If we...

> It might not just be moved, but could be removed. So I think class based deserialization is barely possible; we might have to reconstruct the plan with high level...

Whether we vacuum log files or not, we should update `getLatestStableLog` to handle the last vacuum operation correctly?

> * can work on higher level joins if one side of the join can be replaced by index and can eliminate shuffle. Could you update this with a concrete...

I need to check the secure way to provide the sonatype credential.

Related thread: https://github.com/microsoft/hyperspace/pull/268#discussion_r529071399

> Can we just rename create() to apply() and remove uses of .create()? I believe this would cause stack overflow (infinite recursion) since they have the same parameters. You can...