SPARKC-403: Add CLUSTERING ORDER in cql statement

[skp33]

I added code for Clustering Order support for table creation. Please review the code.

[RussellSpitzer]

I think this is a good addition but I have two major requests.

1. We need a Jira for tracking
2. I don't like just having a String "option". I think we are slowly approaching fully redoing the Java Driver TableMetadata so I think we should have it be a copy of that object. We don't have to necessarily expose all the parameters right now but I'd feel more comfortable with TableDef just having a list of ClusteringOrders.

https://docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/TableMetadata.html

ClusteringOrder is represented by a list of Orderings so very close to what you currently have.

[skp33]

@RussellSpitzer based on your suggestion i changed the code. Do i need to add & change anything else?

[RussellSpitzer]

Make a jira https://datastax-oss.atlassian.net/projects/SPARKC/ For tracking (this is how we catch everything up in release notes and track versioning)

[skp33]

@RussellSpitzer , Jira is already there, this is the Link.

According to discussion, we need to have list of ClusteringOrders inside TableDef, correct me if i miss understood.

And for property part, it depends on use case, i don't have better project knowledge, so let me know what should be there?

[RussellSpitzer]

It looks to me like the information about ClusteringOrders is in two places within the TableDef.

1. Available as an indexedSequence clusteringOrder: Option[Seq[ClusteringOrder]] = None,
2. Available within the clusteringColumn objects themselves + clusteringOrder: ClusteringOrder = ClusteringOrder.ASC) extends FieldDef {

I think only 2 is neccessary? Having both 1 and 2 gives us the possibility of having them not matching which seems like a bit of a hole in the implementation. I think it would be find to have a function on tableDef

def clusterOrder: Seq[ClusteringOrder] = clusteringColumns.map(_.clusteringOrder

If we want it exposed at that level.

[skp33]

I changed according to you, Please verify. And do we need to add some functionality for this method also DataFrameFunctions.createCassandraTable?