out_azure_blob: add path templating support comparable to s3

[nberlee]

fixes #7515

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing Before we can approve your change; please submit the following in a comment:

• [x] Example configuration file for the change

[SERVICE]
    flush     1
    log_level info

[INPUT]
    name      dummy
    dummy     {"name": "Fluent Bit", "year": 2020}
    samples   1
    tag       app.log


[OUTPUT]
    name                  azure_blob
    match                 *
    account_name          devstoreaccount1
    shared_key            Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
    path                  logs/$TAG/%Y/%m/%d/$UUID
    container_name        devstoreaccount1
    auto_create_container on
    tls                   off
    emulator_mode         on
    endpoint              http://localhost:10000

• [x] Debug log output from testing the change see valgrind log

• [x] Attached Valgrind output that shows no leaks or memory corruption was found

valgrind.log

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Run local packaging test showing all targets (including any new ones) build.
• [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• [ ] Documentation required for this feature

Backporting

• [ ] Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

• New Features
  • Path templating for Azure Blob output: dynamic blob paths with UUID substitution, time token/strftime expansion, and tag-based segments; path prefixes are resolved and honored when creating blobs and building URIs.
• Bug Fixes
  • Improved lifecycle and cleanup for templated paths, reducing error/retry edge cases.
• Tests
  • Added unit tests validating templating behavior, UUID/time/token expansion and URI formation.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[nberlee]

@coderabbitai review

[coderabbitai[bot]]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai[bot]]

[!NOTE]

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Adds path-templating and resolution to the Azure Blob output: implements token expansion (UUID, time tokens, strftime, record-accessors) via azb_resolve_path(), threads a resolved path_prefix through DB records and URI/append/block/commit flows, updates DB schema to store path_prefix, and adds unit tests for path resolution.

Changes

Cohort / File(s)	Summary
Core path templating & resolver plugins/out_azure_blob/azure_blob.c	Implements token-replacement helpers (azb_replace_uuid, azb_simple_replace, azb_apply_time_tokens, azb_apply_strftime, azb_trim_slashes), RNG utilities, and azb_resolve_path(); threads path_prefix through create/send/commit/delete flows with error-path cleanup.
Public context & config plugins/out_azure_blob/azure_blob.h, plugins/out_azure_blob/azure_blob_conf.c	Adds path_templating_enabled to struct flb_azure_blob; declares azb_resolve_path() and azb_commit_prefix_with_fallback(); moves/sanitizes ctx->path handling and manages templating flag lifecycle.
URI helpers & effective path plugins/out_azure_blob/azure_blob_uri.c, plugins/out_azure_blob/azure_blob_uri.h	Adds azb_effective_path(); changes azb_uri_create_blob() to accept path_prefix and use effective path when building URIs; adds NULL/error checks for sds operations and SAS appends.
Append-blob URI updates plugins/out_azure_blob/azure_blob_appendblob.c, plugins/out_azure_blob/azure_blob_appendblob.h	Extends azb_append_blob_uri() to accept path_prefix, computes effective_path, includes it in append-block URIs, and tightens error handling.
Block-blob URI & commit updates plugins/out_azure_blob/azure_blob_blockblob.c, plugins/out_azure_blob/azure_blob_blockblob.h	Adds path_prefix parameter to blocklist/uri/commit/block functions; uses effective_path in URI assembly and propagates path_prefix through commit/block APIs and call sites.
Database schema & accessors plugins/out_azure_blob/azure_blob_db.c, plugins/out_azure_blob/azure_blob_db.h	Adds path_prefix column to out_azure_blob_files, migration helper ensure_path_prefix_column(), extends INSERT/SELECT SQL and DB API signatures to store and return path_prefix, and propagates path_prefix through DB retrievals and file-part handling.
Configuration & lifecycle plugins/out_azure_blob/azure_blob_conf.c	Sanitizes ctx->path after remote overrides, enables path_templating_enabled when path present, and clears flag on destroy.
Tests & build tests/internal/azure_blob_path.c, tests/internal/CMakeLists.txt	Adds unit tests covering tag/time/UUID/strftime templating and URI behavior; conditionally includes and links tests when FLB_OUT_AZURE_BLOB is enabled.
Minor integrations plugins/out_azure_blob/* (multiple files)	Updates numerous call sites to accept and pass path_prefix through URI creation, append/block/commit, file-part handling, ingestion/flush flows, and ensures path_prefix memory is freed on error paths.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Ingest as Ingest/Flush
    participant Resolver as azb_resolve_path()
    participant Expander as Token Expander
    participant DB as SQLite (azb_db_*)
    participant URI as URI builders (azb_uri_*/azb_effective_path)
    participant Creator as create/commit routines

    Ingest->>Resolver: resolve path_prefix(tag, timestamp, record)
    activate Resolver
    Resolver->>Expander: expand tokens (%L, %3N, %9N), strftime, $UUID, record-accessors
    Expander-->>Resolver: resolved, trimmed path_prefix
    Resolver-->>Ingest: return resolved path_prefix
    deactivate Resolver

    Ingest->>DB: insert file(..., path_prefix)
    DB-->>Ingest: rowid / ack

    Ingest->>URI: build blob URI(path_prefix, name)
    activate URI
    URI->>URI: effective_path = azb_effective_path(ctx, path_prefix)
    URI-->>Ingest: blob URI (with effective_path + SAS if present)
    deactivate URI

    Ingest->>Creator: create/append/commit blob(path_prefix, blob_name, parts)
    activate Creator
    Creator->>DB: update/commit metadata (with path_prefix)
    Creator-->>Ingest: success / retry / error
    deactivate Creator

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

• Areas needing extra attention:
  • plugins/out_azure_blob/azure_blob.c — token expansion correctness, RNG/UUID replacement, strftime/time-token handling, memory ownership and trimming edge cases.
  • Cross-file signature changes (azure_blob_uri.*, azure_blob_appendblob.*, azure_blob_blockblob.*, callers) — ensure all call sites updated and const-correctness maintained.
  • DB migration and SQL binding (ensure_path_prefix_column(), updated INSERT/SELECT) — NULL handling, backward compatibility, and statement preparation.
  • Unit tests (tests/internal/azure_blob_path.c) — determinism for UUID/randomness and coverage for empty/error cases.

Suggested reviewers

• edsiper
• cosmo0920
• fujimotos

Poem

🐇 I hopped through tokens, chased a $UUID seed,
I stitched time into paths and trimmed each weed,
Resolved prefixes tidy, no slashes stray,
A rabbit's small cheer for every mapped away,
Hop, commit, and store — templating saved the day!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.92% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically summarizes the main change: adding path templating support to Azure Blob output to match S3 functionality, directly addressing the PR's core objective.
Linked Issues check	✅ Passed	The PR fully addresses issue #7515 by implementing S3-like path templating with support for date/time format directives (strftime), $TAG, and $UUID tokens in Azure Blob path configuration, as verified by the comprehensive changes across files.
Out of Scope Changes check	✅ Passed	All changes are directly related to implementing path templating: new functions (azb_resolve_path, azb_effective_path), database schema updates for path_prefix tracking, URI construction modifications, and comprehensive test coverage. No unrelated changes detected.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

• [ ] 📝 Generate docstrings

🧪 Generate unit tests (beta)

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[edsiper]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[nberlee]

@codex review

[chatgpt-codex-connector[bot]]

Codex Review: Didn't find any major issues. More of your lovely PRs please.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[nberlee]

@codex review

[chatgpt-codex-connector[bot]]

Codex Review: Didn't find any major issues. Bravo.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[nberlee]

@edsiper I fixed all codex and coderabbit findings