paimon icon indicating copy to clipboard operation
paimon copied to clipboard
verified publisher icon apache

[Feature] HiveCatalog to commit to tables can we use environmentContext instead of hive locks to provide transactional commits?

Open chenwyi2 opened this issue 1 year ago • 4 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Motivation

It can be big pressure when lots of table use hive locks to provide transaction, so can we use environmentContext instead of hive locks?

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

  • [ ] I'm willing to submit a PR!

chenwyi2 avatar Feb 29 '24 08:02 chenwyi2

I don't know what is environmentContext lock, can you explain more? Or provide a example.

JingsongLi avatar Mar 01 '24 05:03 JingsongLi

https://github.com/apache/hive/pull/3888 changes HMS API , we can use "hive_metastoreConstants.EXPECTED_PARAMETER_KEY" or "hive_metastoreConstants.EXPECTED_PARAMETER_VALUE" to commit instead of 4 HMS calls

chenwyi2 avatar Mar 01 '24 06:03 chenwyi2

It seems there are some problem for this feature, see the discussion in: https://issues.apache.org/jira/browse/HIVE-26882

Let's keep an eye on it.

JingsongLi avatar Mar 06 '24 08:03 JingsongLi

https://issues.apache.org/jira/browse/HIVE-26882 has been resolved, there will be a plan to make some improvement on hivelock?

chenwyi2 avatar Feb 10 '25 06:02 chenwyi2