delta-rs icon indicating copy to clipboard operation
delta-rs copied to clipboard

Published 20 hours ago • verified publisher icon delta-io

Handle PyArrow CVE-2023-47248

Open roeap opened this issue 2 years ago • 6 comments

Environment

Delta-rs version: *

Binding: python

Environment:

  • Cloud provider: n.a.
  • OS: n.a.
  • Other:

Bug

There is a critical CVE out for all pyarrow versions between >= 0.14 < 14.0.1. Not sure if we can force all downstream consumers to upgrade to > 14. There is a hotfix package available as well for other versions that we could conditionally install and load.

What you expected to happen:

How to reproduce it:

More details:

roeap avatar Nov 10 '23 06:11 roeap

Current dependency is "pyarrow>=8" so it will install the latest (14.0.1) but won't touch 14.0.0 unless the dependency is changed. I'm happy to give it a try if you think this could be changed to "pyarrow>=14.0.1". I can also add a runtime version check to prevent prevent running if 14.0.0 is installed but I don't know if that's a good idea if the user may have installed the patch

r3stl355 avatar Nov 11 '23 20:11 r3stl355

but won't touch 14.0.0 unless the dependency is changed

It's not just 14.0.0. Basically every release before that also has the same bug.

I don't think the relevant codepath is used in by our library, so I don't think we have strong reason to take any action. I think it's on our users to upgrade to a fixed pyarrow version. I do wish that Arrow project would consider releasing bugfix releases going back to 8.0.0, but looks like they found that too difficult.

wjones127 avatar Nov 11 '23 20:11 wjones127

It's not just 14.0.0. Basically every release before that also has the same bug.

I didn't realise it also holds for previous versions, CVE description is misleading as it says "versions 0.14.0 to 14.0.0". EDIT - I don't think I'm really good at Spot the Difference game 🤦‍♂️

r3stl355 avatar Nov 11 '23 20:11 r3stl355

I don't think the relevant codepath is used in by our library, so I don't think we have strong reason to take any action.

I guess from my work critical CVEs are just triggering 😆 - thinking about it you are right though .. as we are not directly affected, downstream consumers should handle that...

roeap avatar Nov 11 '23 20:11 roeap

I didn't realise it also holds for previous versions, CVE description is misleading as it says "versions 0.14.0 to 14.0.0"

What do you mean? "versions 0.14.0 to 14.0.0" clearly includes 1.0.0, 8.0.0, etc.

wjones127 avatar Nov 11 '23 21:11 wjones127

🤦 that 14 in both, I clearly could not spot a difference 🤦

r3stl355 avatar Nov 11 '23 22:11 r3stl355

this is handled.

roeap avatar Jan 28 '24 09:01 roeap