feat: As a user, I want to be added in default log-format the `consumer_group_id` variable

[felpasl]

Description

I need to include a consumer_group_id in the default log variables or have an option to add extra variables to an existing log.

I understand we have log_format, but it is going to take too long, and I need the full log information. Creating this will result in a huge template and a complex plugin configuration.

[Baoyuantop]

but it is going to take too long, and I need the full log information.

Can you describe this in more detail?

[felpasl]

Hi all,

Thanks for the interest in this feature!

To provide more context and clarify my request:

Current Situation

• Apache APISIX logging plugins such as file-logger, http-logger, and syslog support configuring the log format via the log_format option, which allows users to define key-value pairs using APISIX or Nginx variables prefixed with $.
• However, the default log format does not currently include the consumer_group_id variable, which is important for tracking and analyzing requests by consumer groups.
• While it is possible to customize the log_format to add this field, doing so often results in very large and complex templates, making configuration harder to maintain and prone to errors.

Why consumer_group_id?

• The consumer_group_id is a key attribute when using consumer groups for API access control and rate limiting.
• Including it in logs by default would greatly simplify observability and troubleshooting, especially in multi-tenant or grouped consumer scenarios.
• It helps correlate logs with consumer group policies and monitor group-level metrics.

Proposal for Enhancement

• Add consumer_group_id as a default field in the standard log format for all logging plugins.
• Alternatively, provide a mechanism to extend the default log format with additional variables (like consumer_group_id) without redefining the entire format.
• This would keep the default configuration simple while allowing flexibility for users who need extra fields.

Additional Notes

• The variable consumer_group_id should be exposed as an APISIX variable, similar to other request or consumer-related variables, so it can be referenced easily in log formats.
• Performance impact should be minimal since it’s just an additional field, but it’s worth verifying.
• This feature aligns with the goal of improving logging and observability in APISIX.

Thanks again for considering this!

[Baoyuantop]

doing so often results in very large and complex templates, making configuration harder to maintain and prone to errors.

Can you describe this in more detail?