Bump dependency of aws-creds to fix instance metadata
Check List
- [ ] Tests has been run in packages where changes made if available
- [ ] Linter has been run for changed code
- [ ] Tests for the changes have been added if not covered yet
- [ ] Docs have been added / updated if required
Issue Reference this PR resolves
None
Description of Changes Made (if issue reference is not provided)
I want to bring this update of the aws-creds into cube:
https://github.com/durch/rust-s3/pull/139/files
I was trying to use the instance metadata to mount the remote storage from s3 using the instance metadata of the pod running in the eks cluster.
I was facing the error:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: CubeError { message: "No credentials provided as arguments, in the environment or in the profile file. \n Json Error: expected value at line 1 column 1", backtrace: "", cause: User }', /build/cubestore/cubestore/src/config/mod.rs:848:86
stack backtrace:
0: rust_begin_unwind
at /rustc/71b8742bbcbed2cd908dbc031d6552d8b528c037/library/std/src/panicking.rs:515:5
1: core::panicking::panic_fmt
at /rustc/71b8742bbcbed2cd908dbc031d6552d8b528c037/library/core/src/panicking.rs:92:14
2: core::result::unwrap_failed
at /rustc/71b8742bbcbed2cd908dbc031d6552d8b528c037/library/core/src/result.rs:1355:5
3: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
4: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
6: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
7: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
8: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
9: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
10: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
11: cubestored::main::{{closure}}
12: tokio::park::thread::CachedParkThread::block_on
13: tokio::runtime::thread_pool::ThreadPool::block_on
14: tokio::runtime::Runtime::block_on
15: cubestored::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
I was using the kube2iam for assuming roles.
Running the command:
curl http://169.254.169.254/latest/meta-data/iam/info
I got the wrong profile name (not the one assigned by the pod annotations). The subsequent call for the details returns a non JSON response which causes the above error.
When I use the endpoint provided in a fix the I get the correct profile name
curl http://169.254.169.254/latest/meta-data/iam/security-credentials
@bartlomiejolma Thanks for contributing it! However, this one won't be that easy. We'd need to update Rust in Cube Store first which will be quite a change. cc @waralex @cristipp
@bartlomiejolma Rust version has been updated. Could you please rebase your branch so we can test your changes on CI? Thanks!
Hello, is 0.28.0 latest that we can use with cubestore? Will this PR be merged? @bartlomiejolma could you rerun tests and merge that?
It seems that without the deps upgrade, this doesn't work?
Cube Store can retrieve security credentials from instance metadata automatically. This means you can skip defining the CUBESTORE_AWS_ACCESS_KEY_ID and CUBESTORE_AWS_SECRET_ACCESS_KEY environment variables.
https://cube.dev/docs/caching/running-in-production#storage-aws
Are there any updates on this? Is it just a matter of updating the library, or do we need to do something else?
I could try to solve this with some directions.
I am going to close this PR in favor of https://github.com/cube-js/cube/pull/6019, which was merged.
Thanks