Cut 0.5 Rust crate

[rtyler]

Need to create a new release branch from 0.4 and then try to merge the latest main into it. We cannot release with Azure SDK for Rust yet.

Arrow 6.0.0 should be updated in the Cargo.toml first

This issue should be closed once we have published a 0.5.0 release to crates.io

[rtyler]

As discussed in our open development meeting, I'll lean on @houqp and @mosyp for some Changelog help

[mpetri]

Not sure if this is the best way to report this but currently when I try to use the deltalake crate with datafusion:

datafusion = "7.0.0"
deltalake = { version = "0.4.0", features = ["s3", "datafusion-ext"] }

I get the following error:

error: failed to select a version for `zstd-sys`.
    ... required by package `zstd-safe v3.1.0+zstd.1.4.9`
    ... which satisfies dependency `zstd-safe = "=3.1.0"` of package `zstd v0.7.0+zstd.1.4.9`
    ... which satisfies dependency `zstd = "^0.7"` of package `parquet v4.0.0`
    ... which satisfies dependency `parquet = "^4"` of package `deltalake v0.4.0`
    ... which satisfies dependency `deltalake = "^0.4.0"` of package `datafusion-s3-test v0.1.0 (/home/ubuntu/dev/datafusion-s3-test)`
versions that meet the requirements `=1.5.0` are: 1.5.0+zstd.1.4.9

the package `zstd-sys` links to the native library `zstd`, but it conflicts with a previous package which links to `zstd` as well:
package `zstd-sys v1.6.3+zstd.1.5.2`
    ... which satisfies dependency `zstd-sys = "=1.6.3"` of package `zstd-safe v4.1.4+zstd.1.5.2`
    ... which satisfies dependency `zstd-safe = "=4.1.4"` of package `zstd v0.10.0+zstd.1.5.2`
    ... which satisfies dependency `zstd = "^0.10"` of package `parquet v9.0.2`
    ... which satisfies dependency `parquet = "^9.0.0"` of package `datafusion v7.0.0`
    ... which satisfies dependency `datafusion = "^7.0.0"` of package `datafusion-s3-test v0.1.0 (/home/ubuntu/dev/datafusion-s3-test)`
Only one package in the dependency graph may specify the same links value. This helps ensure that only one copy of a native library is linked in the final binary. Try to adjust your dependencies so that only one package uses the links ='zstd-sys' value. For more information, see https://doc.rust-lang.org/cargo/reference/resolver.html#links.

I saw deps were recently updated but crates.io still only contains version 0.4 of the deltalake crate

[roeap]

A lot has happened in delta-rs and I think it might be a good idea to pick up on this again 😆. There are some things we need to clean up first (like #781), but i guess we need some prep here as well?

Unfortunately I am so far not familiar with the release process, but am happy to support in any way I can.

cc @houqp @wjones127 @fvaleye

[houqp]

Thanks @roeap for taking care of the release! We really need to get one out :D

The release process is really straight forward. Similar to the python release, all we need to do is to push a rust-v* tag, see https://github.com/delta-io/delta-rs/blob/4902d5beacd7f586d878960efba0d1471355d789/.github/workflows/rust_release.yml#L5.

[mpetri]

@roeap any progress on this? I'm still facing similar issues as I posted about in this thread in March. Is there any way to help?

[mpetri]

@houqp Will new versions of the rust crate automatically be published to crates.io? I don't see the new version yet.

[houqp]

@mpetri ran into some issues with the cargo publish. Hopefully https://github.com/delta-io/delta-rs/pull/968 is the last blocking fix.