Plan for materializing tables

[chanwutk]

Materializing table transformations helps improving runtime performance if the transformations are to be execute multiple times. Some transformations on a table also change table schema based on content in the table.

When to materialize

• When user observe the table: We already compute all the transformations that builds up the table. We should materialize it, unless the user only observe a part of the table. Ex: table.print({limit: 2})
• When user say so: We should allow users to control when a table can materialized on demand.
• When transforming with a verb that requires the knowledge of the data countent: For example, pivot needs to know all the possible values of the pivoting key column before calculating the schema of the output table.