connectors
connectors copied to clipboard
delta-io
Hive: Add `AS OF` syntax support
It would be very helpful for the Hive connector to support the AS OF syntax - for example TIMESTAMP AS OF and VERSION AS OF.
The Spark SQL interface is going through a PR now 8) (https://github.com/apache/spark/pull/34497)
Currently I have Spark batch processes to create end of day views, this would enable Hive users to do this directly.
@spmp thanks for raising this. We would need to push the syntax to Hive so that the connector can use it, similar to the Spark PR. Could you raise a feature request in https://issues.apache.org/jira/browse/HIVE and see what the hive community replies?
@zsxwing,can you give me any pointers as to what may need to be done to try/POC this feature with the latest Hive seeing as the changes mentioned above have been in mainline for a while.
(Sorry I erroneously thought Hive 4.0.0 was released... my bad, but there are increasing noises of this happening https://lists.apache.org/thread/xyvttddcjhk9ffg242wn1wkggsghpc5c)
@spmp I haven't looked at what is done by https://issues.apache.org/jira/browse/HIVE-25344. I'd suggest you to look at what interface added by Hive and see if Delta can plugin its code to this interface.
This repo has been deprecated and the code is moved under connectors module in https://github.com/delta-io/delta repository. Please create the issue in repository https://github.com/delta-io/delta. See delta-io/connectors#556 for details.
