vector chore: Bump OpenDAL to v0.50.0

This PR will bump opendal to latest v0.50.0 for up to date deps and performance improvement.

Oct 12 '24 06:10 Xuanwo

It looks like there were a few compilation errors when you get time.

Oct 12 '24 16:10 jszwedko

Thank you @jszwedko for the quick review, should have been fixed.

Oct 14 '24 02:10 Xuanwo

Hi,

Neither DATADOG_API_KEY nor DD_API_KEY is in your environment.
Internal Error: API key is missing
    at UploadJUnitXMLCommand.getApiHelper (/usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:197:19)
    at UploadJUnitXMLCommand.<anonymous> (/usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:138:30)
    at Generator.next (<anonymous>)
    at /usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:31:71
    at new Promise (<anonymous>)
    at __awaiter (/usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:27:12)
    at UploadJUnitXMLCommand.execute (/usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:118:16)
    at UploadJUnitXMLCommand.validateAndExecute (/usr/local/lib/node_modules/@datadog/datadog-ci/node_modules/clipanion/lib/advanced/Command.js:73:37)
    at /usr/local/lib/node_modules/@datadog/datadog-ci/node_modules/clipanion/lib/advanced/Cli.js:223:53
    at noopCaptureActivator (/usr/local/lib/node_modules/@datadog/datadog-ci/node_modules/clipanion/lib/advanced/Cli.js:449:12)
+ echo 'Failed to upload results'
Failed to upload results

Seems not related to this PRs change?

Oct 16 '24 03:10 Xuanwo

Hi,

Neither DATADOG_API_KEY nor DD_API_KEY is in your environment.
Internal Error: API key is missing
    at UploadJUnitXMLCommand.getApiHelper (/usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:197:19)
    at UploadJUnitXMLCommand.<anonymous> (/usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:138:30)
    at Generator.next (<anonymous>)
    at /usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:31:71
    at new Promise (<anonymous>)
    at __awaiter (/usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:27:12)
    at UploadJUnitXMLCommand.execute (/usr/local/lib/node_modules/@datadog/datadog-ci/dist/commands/junit/upload.js:118:16)
    at UploadJUnitXMLCommand.validateAndExecute (/usr/local/lib/node_modules/@datadog/datadog-ci/node_modules/clipanion/lib/advanced/Command.js:73:37)
    at /usr/local/lib/node_modules/@datadog/datadog-ci/node_modules/clipanion/lib/advanced/Cli.js:223:53
    at noopCaptureActivator (/usr/local/lib/node_modules/@datadog/datadog-ci/node_modules/clipanion/lib/advanced/Cli.js:449:12)
+ echo 'Failed to upload results'
Failed to upload results

Seems not related to this PRs change?

Where do you see that? The failure I see is for the webhdfs sink:

2024-10-15T15:24:14.5662119Z thread 'sinks::webhdfs::integration_tests::hdfs_rotate_files_after_the_buffer_size_is_reached' panicked at src/sinks/webhdfs/integration_tests.rs:86:5:
2024-10-15T15:24:14.5663096Z assertion failed: `(left == right)`'
2024-10-15T15:24:14.5663457Z   left: `"4"`
2024-10-15T15:24:14.5663720Z  right: `"3"`
2024-10-15T15:24:14.5663866Z 
2024-10-15T15:24:14.5664026Z Differences (-left|+right):
2024-10-15T15:24:14.5664346Z -4
2024-10-15T15:24:14.5664548Z +3

I know that sink uses OpenDAL so it seems plausibly related.

Oct 16 '24 13:10 jszwedko

Oh, I see the error you found too. That will just prohibit it from uploading the CI results to Datadog, but isn't actually a CI failure. I think the webhdfs failure is the one that is failing the suite.

Oct 16 '24 13:10 jszwedko

Oh, I see. I will take a look. Thank you for pointing out.

Oct 16 '24 13:10 Xuanwo

Small bump on this when you have some time @Xuanwo 🙏

Dec 05 '24 01:12 jszwedko

Small bump on this when you have some time @Xuanwo 🙏

Thank you, I will check it out.

Dec 05 '24 06:12 Xuanwo

Hi,@jszwedko, sorry for the late. I finally get some time working on this.

Mar 05 '25 08:03 Xuanwo

Thanks @Xuanwo !

Mar 05 '25 16:03 jszwedko

Hi, @jszwedko, the new commit should have addressed all CI issues.

Mar 07 '25 05:03 Xuanwo

There are other issues here, I will take a look.

Mar 07 '25 16:03 Xuanwo

Hi @pront, sorry for the extra work. Would you like to approve the CI for this one?

Mar 10 '25 07:03 Xuanwo

@Xuanwo Hello, do you have any update? As you may know, since OpenDal now supports webhdfs authentication as of 0.51.2, so #21347 could be solved after this change is merged. Thank you for your contribution!

Apr 11 '25 04:04 ghost

cc @pront, please enable the test for this PR. Thank you!

Apr 11 '25 05:04 Xuanwo