tyk [TT-10909]: fix issue with missing upstream headers in graphql proxy only

User description

Description

This PR fixes an issue where the requests from the client were not sent upstream. This was due to an edge case cause by the open telemetry context modification

TT-10909

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

[ ] I ensured that the documentation is up to date
[ ] I explained why this PR updates go.mod in detail with reasoning why it's required
[ ] I would like a code coverage CI quality gate exception and have explained why

Type

Bug fix

Description

Refactored the GraphQLProxyOnlyContext to use context values instead of embedding context. This change was made in the gateway/mw_graphql_transport.go file.
Added a function WithOtelEnabled to enable OpenTelemetry in the gateway/mw_graphql_transport.go file.
Updated the handleProxyOnly and setProxyOnlyHeaders functions to use the new context values in the gateway/mw_graphql_transport.go file.
Updated tests in the gateway/mw_graphql_transport_test.go file to use the new context value functions.
Updated the handoverRequestToGraphQLExecutionEngine and returnErrorsFromUpstream functions in the gateway/reverse_proxy.go file to use the new context value functions.
Enabled OpenTelemetry in the gateway/reverse_proxy.go file if it's configured.
Added a test for passing headers with OpenTelemetry enabled in the gateway/reverse_proxy_test.go file.

Changes walkthrough

Relevant files

Enhancement

mw_graphql_transport.go `Refactor GraphQL proxy context and enable OpenTelemetry` gateway/mw_graphql_transport.go Refactored `GraphQLProxyOnlyContext` to use context values instead of embedding context. Added `WithOtelEnabled` function to enable OpenTelemetry. Updated `handleProxyOnly` and `setProxyOnlyHeaders` functions to use the new context values.	+31/-16
reverse_proxy.go `Update reverse proxy for GraphQL proxy context refactor and OpenTelemetry` gateway/reverse_proxy.go Updated `handoverRequestToGraphQLExecutionEngine` and `returnErrorsFromUpstream` functions to use the new context value functions. Enabled OpenTelemetry if it's configured.	+6/-3

Tests

mw_graphql_transport_test.go `Update tests for GraphQL proxy context refactor` gateway/mw_graphql_transport_test.go Updated tests to use the new context value functions.	+2/-2
reverse_proxy_test.go `Add test for OpenTelemetry enabled headers` gateway/reverse_proxy_test.go Added a test for passing headers with OpenTelemetry enabled.	+38/-0

✨ Usage guide:

Overview: The describe tool scans the PR code changes, and generates a description for the PR - title, type, summary, walkthrough and labels. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.

When commenting, to edit configurations related to the describe tool (pr_description section), use the following template:

/describe --pr_description.some_config1=... --pr_description.some_config2=...

With a configuration file, use the following template:

[pr_description]
some_config1=...
some_config2=...

Enabling\disabling automation

When you first install the app, the default mode for the describe tool is:

pr_commands = ["/describe --pr_description.add_original_user_description=true" 
                         "--pr_description.keep_original_user_title=true", ...]

meaning the describe tool will run automatically on every PR, will keep the original title, and will add the original user description above the generated description.

Markers are an alternative way to control the generated description, to give maximal control to the user. If you set:

pr_commands = ["/describe --pr_description.use_description_markers=true", ...]

the tool will replace every marker of the form pr_agent:marker_name in the PR description with the relevant content, where marker_name is one of the following:

type: the PR type.
summary: the PR summary.
walkthrough: the PR walkthrough.

Note that when markers are enabled, if the original PR description does not contain any markers, the tool will not alter the description at all.

Custom labels

The default labels of the describe tool are quite generic: [Bug fix, Tests, Enhancement, Documentation, Other].

If you specify custom labels in the repo's labels page or via configuration file, you can get tailored labels for your use cases. Examples for custom labels:

Main topic:performance - pr_agent:The main topic of this PR is performance
New endpoint - pr_agent:A new endpoint was added in this PR
SQL query - pr_agent:A new SQL query was added in this PR
Dockerfile changes - pr_agent:The PR contains changes in the Dockerfile
...

The list above is eclectic, and aims to give an idea of different possibilities. Define custom labels that are relevant for your repo and use cases. Note that Labels are not mutually exclusive, so you can add multiple label categories. Make sure to provide proper title, and a detailed and well-phrased description for each label, so the tool will know when to suggest it.

Inline File Walkthrough 💎

For enhanced user experience, the describe tool can add file summaries directly to the "Files changed" tab in the PR page. This will enable you to quickly understand the changes in each file, while reviewing the code changes (diffs).

To enable inline file summary, set pr_description.inline_file_summary in the configuration file, possible values are:

'table': File changes walkthrough table will be displayed on the top of the "Files changed" tab, in addition to the "Conversation" tab.
true: A collapsable file comment with changes title and a changes summary for each file in the PR.
false (default): File changes walkthrough will be added only to the "Conversation" tab.

Utilizing extra instructions

The describe tool can be configured with extra instructions, to guide the model to a feedback tailored to the needs of your project.

Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Notice that the general structure of the description is fixed, and cannot be changed. Extra instructions can change the content or style of each sub-section of the PR description.

Examples for extra instructions:

[pr_description] 
extra_instructions="""
- The PR title should be in the format: '<PR type>: <title>'
- The title should be short and concise (up to 10 words)
- ...
"""

Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

More PR-Agent commands

To invoke the PR-Agent, add a comment using one of the following commands:

/review: Request a review of your Pull Request.

/describe: Update the PR title and description based on the contents of the PR.

/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.

/ask <QUESTION>: Ask a question about the PR.

/update_changelog: Update the changelog based on the PR's contents.

/add_docs 💎: Generate docstring for new components introduced in the PR.

/generate_labels 💎: Generate labels for the PR based on the PR's contents.

/analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the describe usage page for a comprehensive guide on using this tool.

Feb 02 '24 14:02 kofoworola

API Changes

--- prev.txt	2024-02-07 09:53:36.073632589 +0000
+++ current.txt	2024-02-07 09:53:33.381638668 +0000
@@ -6980,6 +6980,7 @@
 
 func RevokeAllTokens(storage ExtendedOsinStorageInterface, clientId, clientSecret string) (int, []string, error)
 func RevokeToken(storage ExtendedOsinStorageInterface, token, tokenTypeHint string)
+func SetProxyOnlyContextValue(ctx context.Context, req *http.Request) context.Context
 func Start()
 func TestReq(t testing.TB, method, urlStr string, body interface{}) *http.Request
 func TestReqBody(t testing.TB, body interface{}) io.Reader
@@ -7969,15 +7970,11 @@
 
 func (m *GraphQLMiddleware) ProcessRequest(w http.ResponseWriter, r *http.Request, _ interface{}) (error, int)
 
-type GraphQLProxyOnlyContext struct {
-	context.Context
-
+type GraphQLProxyOnlyContextValues struct {
 	// Has unexported fields.
 }
 
-func NewGraphQLProxyOnlyContext(ctx context.Context, forwardedRequest *http.Request) *GraphQLProxyOnlyContext
-
-func (g *GraphQLProxyOnlyContext) Response() *http.Response
+func GetProxyOnlyContextValue(ctx context.Context) *GraphQLProxyOnlyContextValues
 
 type GraphQLRequest struct {
 	Query     string          `json:"query"`
@@ -7994,6 +7991,8 @@
 
 type GraphqlEngineTransportOption func(transport *GraphQLEngineTransport)
 
+func WithOtelEnabled() GraphqlEngineTransportOption
+
 type GraphqlGranularAccessChecker struct{}
 
 func (g *GraphqlGranularAccessChecker) CheckGraphqlRequestFieldAllowance(gqlRequest *graphql.Request, accessDef *user.AccessDefinition, schema *graphql.Schema) GraphqlGranularAccessResult

Feb 02 '24 14:02 github-actions[bot]

PR Description updated to latest commit (https://github.com/TykTechnologies/tyk/commit/0a678ee282391ad743d9f1b4f3d0e665778bab04)

Feb 02 '24 14:02 github-actions[bot]

PR Analysis

🎯 Main theme: Fixing issue with missing upstream headers in GraphQL proxy
📝 PR summary: This PR addresses an issue where client requests were not sent upstream due to an edge case caused by the open telemetry context modification. The changes mainly involve refactoring the GraphQLProxyOnlyContext struct and related functions to use context values instead of embedding the context directly.
📌 Type of PR: Bug fix
🧪 Relevant tests added: Yes
⏱️ Estimated effort to review [1-5]: 3, because the PR involves changes in multiple files and the logic is moderately complex. The changes are well-structured and follow good practices, which makes the review easier.
🔒 Security concerns: No security concerns found

PR Feedback

💡 General suggestions: The PR is well-structured and addresses the issue effectively. The changes are appropriately tested. However, it would be beneficial to add more comments in the code to explain the logic, especially for complex parts. This would make the code easier to understand for other developers.

🤖 Code feedback:

relevant file	gateway/mw_graphql_transport.go
suggestion	Consider using constants for repeated string literals like "date", "content-type", and "content-length". This can help avoid potential errors due to typos and make the code more maintainable. [medium]
relevant line	http.CanonicalHeaderKey("date"): true,

relevant file	gateway/mw_graphql_transport.go
suggestion	It's a good practice to handle the error returned by `context.WithValue()`. Even though it's unlikely to return an error in this case, handling it can prevent potential issues in the future. [medium]
relevant line	return context.WithValue(ctx, graphqlProxyContextInfo, value)

relevant file	gateway/mw_graphql_transport.go
suggestion	Consider adding a default case in the switch statement for `g.transportType`. This can help catch unexpected values and handle them appropriately. [medium]
relevant line	switch g.transportType {

relevant file	gateway/reverse_proxy.go
suggestion	It's a good practice to handle the error returned by `body.Seek()`. Even though it's unlikely to return an error in this case, handling it can prevent potential issues in the future. [medium]
relevant line	_, err := body.Seek(0, io.SeekStart)

✨ Usage guide:

Overview: The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR. When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

Utilizing extra instructions

The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project.

Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize.

Examples for extra instructions:

[pr_reviewer] # /review #
extra_instructions="""
In the 'general suggestions' section, emphasize the following:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""

Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

How to enable\disable automation

When you first install PR-Agent app, the default mode for the review tool is:

pr_commands = ["/review", ...]

meaning the review tool will run automatically on every PR, with the default configuration. Edit this field to enable/disable the tool, or to change the used configurations

Auto-labels

The review tool can auto-generate two specific types of labels for a PR:

a possible security issue label, that detects possible security issues (enable_review_labels_security flag)
a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)

Extra sub-tools

The review tool provides a collection of possible feedbacks about a PR. It is recommended to review the possible options, and choose the ones relevant for your use case. Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example: require_score_review, require_soc2_ticket, and more.

More PR-Agent commands

To invoke the PR-Agent, add a comment using one of the following commands:

/review: Request a review of your Pull Request.

/describe: Update the PR title and description based on the contents of the PR.

/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.

/ask <QUESTION>: Ask a question about the PR.

/update_changelog: Update the changelog based on the PR's contents.

/add_docs 💎: Generate docstring for new components introduced in the PR.

/generate_labels 💎: Generate labels for the PR based on the PR's contents.

/analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.

Feb 02 '24 14:02 github-actions[bot]

PR Code Suggestions

Suggestions

best practice

Use a more unique context key type to avoid potential collisions.

Consider using a well-defined key type for context values to avoid potential collisions.
The current approach of using a private struct may not be unique enough, especially in
larger applications or libraries. A more robust approach could involve defining a context
key type or using a package-level private variable.

gateway/mw_graphql_transport.go [30-32]

-type contextKey struct{}
-var graphqlProxyContextInfo = contextKey{}
+type graphqlProxyContextKey string
+var graphqlProxyContextInfo = graphqlProxyContextKey("graphqlProxyContextInfo")

Use a custom type for context keys and consider the implications of storing pointers in context.

It's recommended to avoid using context.WithValue with basic types as keys due to the
risk of collisions. Instead, use a custom type for the key. Additionally, storing a
pointer to a struct in the context can lead to unexpected mutations if the struct is
modified after being stored. Consider if this is the desired behavior or if a defensive
copy should be made.

gateway/mw_graphql_transport.go [32-50]

-var graphqlProxyContextInfo = contextKey{}
-return context.WithValue(ctx, graphqlProxyContextInfo, value)
+type graphqlProxyContextKeyType struct{}
+var graphqlProxyContextKey = graphqlProxyContextKeyType{}
+return context.WithValue(ctx, graphqlProxyContextKey, value)

Consider the implications of directly modifying the context in SetProxyOnlyContextValue.

The SetProxyOnlyContextValue function directly modifies the incoming context with a new
value. This approach can lead to side effects if the original context is expected to
remain unchanged elsewhere. Consider returning a new context derived from the original or
clearly documenting this behavior.

gateway/mw_graphql_transport.go [40-50]

-func SetProxyOnlyContextValue(ctx context.Context, req *http.Request) context.Context {
-  value := &GraphQLProxyOnlyContextValues{
-    forwardedRequest: req,
-    ignoreForwardedHeaders: map[string]bool{
-      http.CanonicalHeaderKey("date"):           true,
-      http.CanonicalHeaderKey("content-type"):   true,
-      http.CanonicalHeaderKey("content-length"): true,
-    },
-  }
-  return context.WithValue(ctx, graphqlProxyContextInfo, value)
-}
+// Consider documenting or revising the approach to modifying the context
+// Documentation example:
+// SetProxyOnlyContextValue returns a new context derived from the original with the proxy-only value set.

maintainability

Move the initialization of ignoreForwardedHeaders to a separate function.

To improve the maintainability and readability of the code, consider moving the
initialization of ignoreForwardedHeaders to a separate function or package-level
variable. This will make the SetProxyOnlyContextValue function cleaner and separate
concerns more clearly.

gateway/mw_graphql_transport.go [43-47]

-ignoreForwardedHeaders: map[string]bool{
-  http.CanonicalHeaderKey("date"):           true,
-  http.CanonicalHeaderKey("content-type"):   true,
-  http.CanonicalHeaderKey("content-length"): true,
-},
+ignoreForwardedHeaders: getDefaultIgnoreForwardedHeaders()

enhancement

Ensure the otelEnabled field is utilized in the transport's logic.

The otelEnabled field in GraphQLEngineTransport is being set through a functional
option, which is a good practice. However, it's not being utilized in the transport's
logic. Ensure that the otelEnabled field is checked and used to conditionally enable
OpenTelemetry tracing or other related functionality within the transport methods.

gateway/mw_graphql_transport.go [64]

-otelEnabled       bool
+// Example usage within a method
+if g.otelEnabled {
+  // Enable OpenTelemetry tracing or other related functionality
+}

✨ Usage guide:

Overview: The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR. When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

With a configuration file, use the following template:

[pr_code_suggestions]
some_config1=...
some_config2=...

Enabling\disabling automation

When you first install the app, the default mode for the improve tool is:

pr_commands = ["/improve --pr_code_suggestions.summarize=true", ...]

meaning the improve tool will run automatically on every PR, with summarization enabled. Delete this line to disable the tool from running automatically.

Utilizing extra instructions

Extra instructions are very important for the improve tool, since they enable to guide the model to suggestions that are more relevant to the specific needs of the project.

Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on.

Examples for extra instructions:

[pr_code_suggestions] # /improve #
extra_instructions="""
Emphasize the following aspects:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""

Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

A note on code suggestions quality

While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically.
Suggestions are not meant to be simplistic. Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base.
Recommended to use the 'extra_instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project, or use the custom suggestions :gem: tool
With large PRs, best quality will be obtained by using 'improve --extended' mode.

More PR-Agent commands

To invoke the PR-Agent, add a comment using one of the following commands:

/review: Request a review of your Pull Request.

/describe: Update the PR title and description based on the contents of the PR.

/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.

/ask <QUESTION>: Ask a question about the PR.

/update_changelog: Update the changelog based on the PR's contents.

/add_docs 💎: Generate docstring for new components introduced in the PR.

/generate_labels 💎: Generate labels for the PR based on the PR's contents.

/analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the improve usage page for a more comprehensive guide on using this tool.

Feb 02 '24 14:02 github-actions[bot]

API tests result - postgres15-sha256 env: success :white_check_mark: Branch used: refs/pull/6024/merge Commit: 1584cd456575d70fc2c398c39cd4310eb02c84aa Triggered by: pull_request (@kofoworola) Execution page

Feb 02 '24 14:02 buger

API tests result - mongo44-sha256 env: success :white_check_mark: Branch used: refs/pull/6024/merge Commit: 1584cd456575d70fc2c398c39cd4310eb02c84aa Triggered by: pull_request (@kofoworola) Execution page

Feb 02 '24 14:02 buger