feat: add !store.getkey YAML function for arbitrary key retrieval

[jamengual]

trafficstars

feat: add !store.getkey YAML function for arbitrary key retrieval

Summary

This PR introduces the !store.getkey YAML function, enabling retrieval of arbitrary keys from any supported store (Azure Key Vault, AWS SSM, Redis, Google Secret Manager, Artifactory). Unlike the existing !store function, !store.getkey does not require keys to follow the Atmos stack/component/key naming pattern. Users can retrieve any key by specifying its exact name or path.

Key Features

• New Store Interface Method:
  Adds GetKey(key string) (interface{}, error) to the Store interface for arbitrary key retrieval.
• Store Implementations:
  Implements GetKey in all supported stores, handling prefixes and store-specific key/path conventions.
• YAML Function Processor:
  Adds the !store.getkey YAML function and its processor, allowing direct key/path access in YAML.
• Testing:
  Comprehensive, table-driven unit tests for the new function, including happy paths and error conditions.
  Adds a RedisClient() method to the Redis store for direct test setup of arbitrary keys.
• Documentation:
  Updates website documentation with a new section for !store.getkey, including usage, arguments, and clear examples.
  Explicitly documents that this function does not follow the stack/component/key pattern.

Usage Example

# Retrieve a key from Redis by its exact name
my_config: !store.getkey redis global-config

# Retrieve a secret from Azure Key Vault by its name
my_secret: !store.getkey azure-keyvault my-arbitrary-secret

# Retrieve a parameter from AWS SSM by its full path
ssm_value: !store.getkey aws-ssm-parameter-store /custom/path/to/parameter

Notable Differences from !store

• !store.getkey does not construct keys using stack/component/key; it expects the full key or path.
• Useful for retrieving values stored outside of Atmos or not following the standard naming convention.

Compliance

• Follows Atmos project rules for code structure, testing, linting, and documentation.
• All tests pass and code is lint-free.

Summary by CodeRabbit

• New Features
  • Introduced the !store.getkey YAML function for direct key-based value retrieval from supported stores, bypassing Atmos naming conventions.
  • Added GetKey methods to all supported store types (Artifactory, AWS SSM, Azure Key Vault, Google Secret Manager, Redis) enabling single-key access.
• Documentation
  • Added detailed documentation for the !store.getkey function, including usage, arguments, examples, and differences from the existing !store function.
• Tests
  • Added comprehensive tests validating the new key-based retrieval functionality and error handling across supported stores.

[coderabbitai[bot]]

📝 Walkthrough

Walkthrough

Adds a new YAML function tag !store.get with parsing and handler, extends the Store interface with GetKey, implements GetKey across multiple store backends, adds tests and docs for !store.get, adjusts some error/cleanup handling, and bumps two dependencies.

Changes

Cohort / File(s)	Summary of changes
YAML function plumbing internal/exec/yaml_func_utils.go, internal/exec/yaml_func_store_get.go, internal/exec/yaml_func_errors.go, internal/exec/yaml_func_store.go, pkg/utils/yaml_utils.go	Register new tag !store.get, add processTagStoreGet handler with pipe parameter parsing (default/query), introduce new error variables, and a minor logging tweak.
YAML function tests internal/exec/yaml_func_store_get_test.go	Unit tests for !store.get against a Redis-backed store covering success, JSON/yq extraction, default handling, and invalid parameter cases.
Store interface & core types pkg/store/store.go, pkg/store/errors.go	Change Store API to use any, add GetKey(key string) (any, error) to Store, update StoreFactory type, add internal error-format constants, and remove two shared error vars.
Redis store pkg/store/redis_store.go, pkg/store/redis_store_test.go	Add GetKey implementation (JSON-unmarshal fallback), expose RedisClient(), remove local errFormat, and add table-driven tests for GetKey (duplicate test blocks present).
AWS SSM store pkg/store/aws_ssm_param_store.go, pkg/store/aws_ssm_param_store_test.go	Add GetKey to fetch a parameter by full key name, change Set/Get signatures to use any, JSON-unmarshal fallback to raw string, and add extensive table-driven tests including read-role (STS) flows.
Azure Key Vault store pkg/store/azure_keyvault_store.go, pkg/store/azure_keyvault_store_test.go	Add GetKey using normalized secret names, handle 404/403 explicitly, JSON-unmarshal fallback, and add table-driven tests for direct key retrieval and error cases.
Google Secret Manager store pkg/store/google_secret_manager_store.go, pkg/store/google_secret_manager_store_test.go	Add GetKey to access latest secret version by key, add timeout/context handling and client Close() cleanup on init failure, plus tests for direct key retrieval.
Artifactory store pkg/store/artifactory_store.go, pkg/store/artifactory_store_test.go	Add GetKey to download file by key path, replace some defer cleanup with wrappers that swallow errors, switch to errWrapFormat usage, and test updates/renames.
Docs website/docs/core-concepts/stacks/yaml-functions/store.mdx, website/docs/core-concepts/stacks/yaml-functions/store.get.mdx, website/docs/.../*.mdx	Clarify !store doc, add comprehensive !store.get doc and examples, and update sidebar positions for several YAML-function docs.
Misc / repo .cursor/.cursor, go.mod	Add empty .cursor file; bump github.com/posthog/posthog-go and github.com/redis/go-redis/v9 patch versions.

Sequence Diagram(s)

sequenceDiagram
  participant UserYAML as User YAML
  participant Parser
  participant Exec as processCustomTags
  participant Handler as processTagStoreGet
  participant Registry as StoreRegistry
  participant Store as Store.GetKey
  participant Backend as Backend Provider

  UserYAML->>Parser: contains !store.get <store> <key> [| default | query]
  Parser->>Exec: detect custom tag
  Exec->>Handler: dispatch !store.get payload
  Handler->>Registry: lookup <store> by name
  Registry-->>Handler: store instance
  Handler->>Store: GetKey(key)
  Store->>Backend: fetch value
  Backend-->>Store: raw value or error
  Store-->>Handler: value (decoded JSON or raw) or error
  alt query provided
    Handler->>Handler: evaluate yq expression on value
  end
  Handler-->>Parser: resolved value or default/error
  Parser-->>UserYAML: final resolved YAML value

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~35 minutes

Possibly related PRs

• cloudposse/atmos#865 — Closely related changes to store subsystems and YAML function processing; directly relevant to adding GetKey and YAML handler wiring.
• cloudposse/atmos#810 — Prior YAML functions work; related to tag registration and exec plumbing used by !store.get.
• cloudposse/atmos#1017 — Related Artifactory store key/path handling changes that intersect with the new GetKey implementation.

Suggested labels

major

Suggested reviewers

• mcalhoun
• osterman

[!TIP]

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

---

📜 Recent review details

Configuration used: .coderabbit.yaml Review profile: CHILL Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 097c39cab71cede80e6ae8940ee710f97ef56f6a and b07118b5c44d2e3db400fe036aa8a142fd7bcb1b.

📒 Files selected for processing (1)

• pkg/store/artifactory_store_test.go (4 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

• pkg/store/artifactory_store_test.go

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (22)

• GitHub Check: [mock-macos] examples/demo-atlantis
• GitHub Check: [mock-macos] examples/demo-context
• GitHub Check: [mock-macos] examples/demo-vendoring
• GitHub Check: [mock-macos] tests/fixtures/scenarios/complete
• GitHub Check: [mock-windows] examples/demo-component-versions
• GitHub Check: [mock-macos] examples/demo-component-versions
• GitHub Check: [mock-windows] tests/fixtures/scenarios/complete
• GitHub Check: [mock-linux] examples/demo-component-versions
• GitHub Check: [mock-windows] examples/demo-atlantis
• GitHub Check: [mock-windows] examples/demo-context
• GitHub Check: [mock-linux] examples/demo-context
• GitHub Check: [mock-windows] examples/demo-vendoring
• GitHub Check: [mock-linux] examples/demo-vendoring
• GitHub Check: [mock-linux] tests/fixtures/scenarios/complete
• GitHub Check: [lint] demo-context
• GitHub Check: Acceptance Tests (ubuntu-latest, linux)
• GitHub Check: Acceptance Tests (macos-latest, macos)
• GitHub Check: Acceptance Tests (windows-latest, windows)
• GitHub Check: [k3s] demo-helmfile
• GitHub Check: [lint] Dockerfile
• GitHub Check: [localstack] demo-localstack
• GitHub Check: Summary

✨ Finishing Touches

• [ ] 📝 Generate Docstrings

🧪 Generate unit tests

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment
• [ ] Commit unit tests in branch implement_store_getkey

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary or @auto-summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai or @auto-title anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

• Visit our Status Page to check the current availability of CodeRabbit.
• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[codecov[bot]]

Codecov Report

:x: Patch coverage is 62.98932% with 104 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 55.28%. Comparing base (9179bd8) to head (b07118b). :warning: Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/store/artifactory_store.go	14.92%	57 Missing :warning:
internal/exec/yaml_func_store_get.go	75.32%	16 Missing and 3 partials :warning:
pkg/store/google_secret_manager_store.go	70.58%	7 Missing and 3 partials :warning:
pkg/store/azure_keyvault_store.go	72.00%	5 Missing and 2 partials :warning:
pkg/store/aws_ssm_param_store.go	86.66%	4 Missing and 2 partials :warning:
pkg/store/redis_store.go	90.00%	2 Missing and 1 partial :warning:
internal/exec/yaml_func_utils.go	0.00%	2 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1352      +/-   ##
==========================================
+ Coverage   55.16%   55.28%   +0.11%     
==========================================
  Files         271      272       +1     
  Lines       28185    28444     +259     
==========================================
+ Hits        15548    15725     +177     
- Misses      10874    10943      +69     
- Partials     1763     1776      +13     

Flag	Coverage Δ
unittests	55.28% <62.98%> (+0.11%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[osterman]

Test coverage is too low.

[mergify[bot]]

[!WARNING]

This PR exceeds the recommended limit of 1,000 lines.

Large PRs are difficult to review and may be rejected due to their size.

Please verify that this PR does not address multiple issues. Consider refactoring it into smaller, more focused PRs to facilitate a smoother review process.

[github-actions[bot]]

These changes were released in v1.187.0.