Matthew Powers
Matthew Powers
I think Poetry dependency groups should work well for this task.
@michael72 - Yea, it might make sense to just create another repo. The Spark core devs are definitely excited about a Spark Connect + Scala 3 project. You're right that...
@brayanjuls - blog post looks amazing. I am going to post it on LinkedIn now. Would be good to add the import statement for `DeltaHelpers`. Also, just noticed that the...
@brayanjuls - updated the list by crossing out the post that was made. Will try to get through all of these over the coming weeks.
@puneetsharma04 @ilyasse05 - can you please put code in pull requests? The code should be checked with unit tests. That will make it easier for other people to run the...
This seems like a useful addition. @brayanjuls - are you cool with this? I think the sorting should be optional though and we should make sure it only happens if...
@ilyasse05 - this is to remove from directly the table itself ;)
@Jegan7 - I just chatted with @brayanjuls on this one. He's going to think about this and ping you with a suggested implementation. After he pings you, it should be...
Seems like a reasonable suggestion, but perhaps we'd need a new name for that functionality. When I copy "something", I expect the source and destination to have the same data....
How does this deal with the Delta Lake / PySpark versions that are incompatible? I don't think mack supports Delta 1.0. Here are the version combinations I think we should...