jonathanmaw

I have not been able to figure out a way to do the same with window functions.

In Spark ObjectExpressions, the above is: ``` Aggregate "department" [ ["aggregated", "1"] , Function "count" [ Function "when" [ BinaryOperation ">=" Column "salary" Literal (StringLiteral 2001) , Column "salary" ]...

In src/Morphir/Spark/AST.elm, (List.filter predicate sourceRelation) is transformed into a (Filter fieldExpression source). i.e. expressionFromValue is responsible for transforming a filter function into a Spark Expression. Filter functions are expected to...

#833 is an issue that involves reducing the amount of effort in writing a new test, which would likely be useful to this issue.

This appears to be a combination of List.map functionality and aggregation support. Taking the christmas bonanza example and reducing it to its simplest is: ``` source |> List.map .priceValue |>...

An equivalent in Spark would be: ``` source.select( org.apache.spark.sql.functions.min( org.apache.spark.sql.functions.col("ageOfItem") ).alias("foo") ) ``` Which in Spark IR would be ``` Select [ ( Name "min" ),( Function "min" [Column "ageOfItem"]...

I think objectExpressionFromValue would need to match against a Lambda being called on an application of List.map, which all gets presented as a Select, to implement this.

I wonder if considering a general aggregation case might be instructive, e.g. As Elm: ``` source = [{cost=1, weight=2}, {cost=3, weight=4}] testMultiple source = [{ lightest = source |> List.map...

After a lot of headscratching, I think I've gotten closer to a working implementation. I need to match against `Apply _ (Lambda _ _ (List _ [Record _ fields])) sourceRelation`,...

I have the pull request https://github.com/finos/morphir-elm/pull/804 for this