feat: opendal should support gcs `authorized_user` credential

[kingwingfly]

Describe the bug

GCS not work, and Operator::check returns an IncompleteMessage error.

Steps to Reproduce

pub fn create_operator(bucket: impl AsRef<str>) -> Result<Operator> {
    let builder = services::Gcs::default().bucket(bucket.as_ref());
    Ok(Operator::new(builder)?.finish())
}

#[tokio::test]
async fn basic_probe() -> Result<()> {
    let op = create_operator("xxx-datasets").unwrap();
    dbg!(op.check().await);
    Ok(())
}

Just block for 10 seconds or so,

[test/probe.rs:18:5] op.check().await = Err(
    Error {
        kind: Unexpected,
        message: "loading credential to sign http request",
        status: Temporary,
        operation: "list",
        context: [
            (
                "called",
                "reqsign::LoadCredential",
            ),
            (
                "service",
                "gcs",
            ),
            (
                "path",
                "/",
            ),
            (
                "listed",
                "0",
            ),
        ],
        source: Some(
            reqwest::Error {
                kind: Request,
                source: hyper_util::client::legacy::Error(
                    SendRequest,
                    hyper::Error(
                        IncompleteMessage,
                    ),
                ),
            },
        ),
    },
)

Expected Behavior

The operator of GCS can work correctly.

Additional Context

Add trace_subscriber and got:

running 1 test
2025-06-12T05:56:50.245403Z DEBUG ThreadId(02) opendal::services::gcs::backend: 241: backend build started: GcsBuilder { config: GcsConfig { root: None, bucket: "xvu-datasets", endpoint: None, scope: None, .. }, .. }    
2025-06-12T05:56:50.245487Z DEBUG ThreadId(02) opendal::services::gcs::backend: 244: backend use root /    
2025-06-12T05:56:50.245499Z DEBUG ThreadId(02) opendal::services::gcs::backend: 263: backend use endpoint: https://storage.googleapis.com    
2025-06-12T05:56:50.264007Z DEBUG ThreadId(02) reqwest::connect: 788: starting new connection: http://metadata.google.internal/    
2025-06-12T05:56:52.004969Z DEBUG ThreadId(02) reqwest::connect: 788: starting new connection: http://metadata.google.internal/    
2025-06-12T05:56:55.422559Z DEBUG ThreadId(02) reqwest::connect: 788: starting new connection: http://metadata.google.internal/    
2025-06-12T05:57:00.092319Z DEBUG ThreadId(02) reqwest::connect: 788: starting new connection: http://metadata.google.internal/    
[test/probe.rs:18:5] op.check().await = Err(
    Error {
        kind: Unexpected,
        message: "loading credential to sign http request",
        status: Temporary,
...

It looks like failing to connect a strange address http://metadata.google.internal/

Are you willing to submit a PR to fix this bug?

• [ ] Yes, I would like to submit a PR.

[dosubot[bot]]

Hi @kingwingfly! I'm Dosu and I’m helping the opendal team.

The repeated attempts to connect to http://metadata.google.internal/ are due to OpenDAL's GCS backend trying to load credentials from the Google Cloud metadata server by default. This is expected if you haven't provided credentials explicitly and are not running on GCP. The resulting IncompleteMessage error is a symptom of failing to connect to that endpoint and not receiving a valid response, which blocks credential loading and thus all GCS operations.

To fix this, provide valid GCS credentials using either the credential string, credential file path, or token methods in your GcsBuilder. If you do not want OpenDAL to attempt loading credentials from the metadata server, call .disable_vm_metadata() on your builder. For example:

let builder = services::Gcs::default()
    .bucket("your-bucket")
    .credential_path("path/to/your/credentials.json")
    .disable_vm_metadata();
let op = Operator::new(builder)?.finish();

This will prevent OpenDAL from trying to reach the metadata server and should resolve the error if your credentials are valid. See the relevant builder methods and config options for more details on credential loading and disabling metadata server access
reference
reference.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[kingwingfly]

After disable_vm_matadata, I got with credencial_path set to "/home/me/.config/gcloud/application_default_credentials.json"

[test/probe.rs:18:5] op.check().await = Err(
    Error {
        kind: ConfigInvalid,
        message: "no valid credential found",
        status: Permanent,
        operation: "list",
        context: [
            (
                "service",
                "gcs",
            ),
            (
                "path",
                "/",
            ),
            (
                "listed",
                "0",
            ),
        ],
        source: None,
    },
)

However, gcloud storage objects list gs://xxx-datasets/ works. I'll keep trying and if I succeed, I'll close this issue.

[Xuanwo]

One possible issue is that opendal may not support the credential type specified in your local JSON file. What is the value of the type field in your application_default_credentials.json?

We support the following type:

• service_account
• external_account
• impersonated_service_account

[kingwingfly]

The type field is authorized_user (◐‿◑)

[Xuanwo]

The type field is authorized_user (◐‿◑)

Interesting. Let me take a look on this type.

[kingwingfly]

It's produced by gcloud init; gcloud auth application-default login;

[jokester]

More on this key type: https://cloud.google.com/docs/authentication/application-default-credentials#personal

CredentialFile::AuthorizedUser in https://github.com/Xuanwo/reqsign/blob/main/services/google/src/credential.rs seems to be a match. But the tree, maybe reqsign:0.17.0, is quite different from reqsign:0.16.3 used by recent Opendal. I couldn't simply use it in [replace] to test 😕

[dentiny]

+1 for authorized user credential support, it's prob the easiest setup to play with GCS for personal users. :(

[venim]

bumping this for posterity. updating opendal to use reqsign:0.17.0 is not trivial, but it will unblock the default path (and most secure) path for humans who want to use opendal to interact with GCS.