http.send: force_cache doesn't seem to work when Date header missing from response

[bartandacc]

Short description

I have a case when need to obtain user information from Keycloak (token) during policy evaluation. I have created policy with http.send method as per OPA documentation . The only difference is I am using “password” OAuth flow, not “client_credentials”. Example code below:

package newdata

import future.keywords.in

default allow := false

tok := t {
    response := http.send({
        "url": "http://address/realms/realm/protocol/openid-connect/token",
        "method": "POST",
        "headers": {
            "Content-Type": "application/x-www-form-urlencoded",
            "Authorization": concat(" ", [
                "Basic",
                base64.encode(sprintf("%v:%v", ["yyy", "xxx"]))
            ]),
        },
        "raw_body": sprintf("username=%v&grant_type=password&scope=openid", [input.user]),
        "force_cache": true,
        "force_cache_duration_seconds": 3600
    })
    response.status_code == 200

    t := response.body.access_token
}

token := payload {
	[_, payload, _] := io.jwt.decode(tok)
}

allow {
	user_is_in_group
}

# Allow user when is in a given group
user_is_in_group {
	resource := data.resources[_]
    resource.resource_id == input.resource
    token.groups[_] in resource.groups
}

force_cache parameter is set. Cache duration for 3600 secs. Unfortunately after every request I can see a new session on KC side, so it looks like given token (http.send invocation) is not cached.

• Testing the flow on standalone OPA (opa run -s –bundle {bundle_name})
• OPA version: v.0.42.1 (Windows, Linux), 0.43.0 (Windows).

Expected behavior

A single call to the Keycloak service every 3600 seconds is expected.

Additional context

Some investigation has already been done with another servers (not Keycloak). For some, cache functionality works correctly, for some not. The only difference in servers response I noticed is "Date" header. When set up with proper time, OPA cache seems to work. When not present or not set correctly it does not, even when force_cache is set to true.

[srenatus]

Force cache overrides the cache headers, not the general logic: do you send the same headers with every request? Is there perhaps some authorization header that isn't the same for every request?

[anderseknert]

@srenatus longer discussion previous to this ticket getting filed: https://openpolicyagent.slack.com/archives/CBR63TK2A/p1659336607542569

definitely not ruling out the different header theory, but not ruling out other options either

[anderseknert]

Looks like I'm able to reproduce this. The Go http server apparently adds the Date header automatically, so this will need to be overridden in order for this to "work":

server

package main

import (
	"fmt"
	"net/http"
)

func main() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		fmt.Println("request received")
		w.Header()["Date"] = nil
		_, _ = fmt.Fprintf(w, "ok")
	})
	
	_ = http.ListenAndServe(":7777", nil)
}

client

package policy

response := http.send({
    "method": "GET",
    "url": "http://localhost:7777/",
    "force_cache": true,
    "force_cache_duration_seconds": 60,
})

Without w.Header()["Date"] = nil the http.send calls to the server are cached, and "request received" printed only once. Adding w.Header()["Date"] = nil and by doing so removing the Date header from the response, and now no caching is done, as made evident by "request received" printed per each request.

[stale[bot]]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[ppapryczka]

Just to provide more information - issue was resolved in version 0.56. For details see this discussion: https://github.com/orgs/open-policy-agent/discussions/479