hudi icon indicating copy to clipboard operation
hudi copied to clipboard

Published 20 hours ago • verified publisher icon apache

[MINOR] Fixes to make unit tests work on m1

Open jonvex opened this issue 3 years ago • 1 comments

Change Logs

an older version of snappy that is incompatible with m1 was used in by a few dependencies. To fix we make spark version to be 2.4.8 and we add some exclusions so that the dependencies don't pull in the old version of snappy and use the newer one instead.

Impact

Unit tests should now work on m1

*Risk level: low

Contributor's checklist

  • [x] Read through contributor's guide
  • [x] Change Logs and Impact were stated clearly
  • [x] Adequate tests were added if applicable
  • [x] CI passed

jonvex avatar Sep 22 '22 20:09 jonvex

have cancelled CI run for now. investigating CI flakiness. will trigger azure CI run once we fix the flakiness.

nsivabalan avatar Sep 23 '22 19:09 nsivabalan

CI report:

  • c7a1d373796e8bfce040bd79a07f68ef6b7ffc59 UNKNOWN
  • 287c52c6da5eb75093f3c9f7bfd5bfaf0eeb9ac0 Azure: FAILURE
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Sep 30 '22 02:09 hudi-bot

After revisiting this PR, I feel that this introduces the risk of building release artifacts with different versions of dependencies on M1 Macbook (note that the release manager needs to build the artifacts locally and push them to the staging area using the release script). We should directly upgrade the versions without keeping two variants based on the OS/Arch. HUDI-5388 for followup.

cc @nsivabalan @alexeykudinkin @xushiyan

yihua avatar Dec 14 '22 06:12 yihua