AppEngine Context and middleware

[clanstyles]

I keep getting the same error over and over.

appengine: NewContext passed an unknown http.Request

I'm trying to daisy chain my middleware (auth, logging) with Gorilla Mux. Each middleware is written as:

func HandlerFunc(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		ctx := NewContext(r.Context(), lang)
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

I always return r.WithContext(ctx). The problem is that the next middleware isn't aware that it's an appengine Context. I tried creating a middleware that would init the request context as appengine's context.

func AppEngine(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		ctx := appengine.WithContext(r.Context(), r)
		r = r.WithContext(ctx)
		next.ServeHTTP(w, r)
	})
}

What's the best way to handle this?

[s-mang]

Hello. Which is the line generating the error?

I think the issue is probably that appengine.NewContext returns an golang.org/x/net/context.Context, while gorilla is expecting a context.Context. But it is hard to say until i know which line you are referring to. You can do a shallow copy if this is the case. The two types are easy to convert between e.o.

But if it is in fact a gorilla/context.Context that you are passing around, that will be more challenging. Note that gorilla's mux will default to using gorilla/context.Context over context.Context only if Go version < 1.7.

ctx := NewContext(r.Context(), lang) <== what is the body and signature of NewContext here? Or did you mean to type appengine.NewContext?

[clanstyles]

This is the new context

// NewContext returns a new Context that carries value u.
func NewContext(ctx context.Context, l Language) context.Context {
	return context.WithValue(ctx, languageKey, l)
}

// FromContext returns the User value stored in ctx, if any.
func FromContext(ctx context.Context) (Language, bool) {
	u, ok := ctx.Value(languageKey).(Language)
	return u, ok
}

I have the routes running through an app handler

type appHandler struct {
	H func(context.Context, http.ResponseWriter, *http.Request) (int, error)
}

// Our ServeHTTP method is mostly the same, and also has the ability to
// access our *appContext's fields (templates, loggers, etc.) as well.
func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	ctx := appengine.WithContext(r.Context(), r)

	// Updated to pass ah.appContext as a parameter to our handler type.
	status, err := ah.H(ctx, w, r)
	if err != nil {
		log.Infof(ctx, "HTTP %d: %q", status, err)

		switch status {
		case http.StatusNotFound:
			NotFoundHandler(w, r)
			return
		case http.StatusInternalServerError:
			http.Error(w, http.StatusText(status), status)

		default:
			http.Error(w, http.StatusText(status), status)
		}
	}
}

Router outside

	middleware = []func(http.Handler) http.Handler{
		AppEngine,
		i18n.HandlerFunc,
	}

func use(h appHandler, middlewares ...func(http.Handler) http.Handler) http.Handler {
	var res http.Handler = h

	for _, m := range middlewares {
		res = m(res)
	}

	return res
}

	router = router.PathPrefix("/{lang}").Subrouter()
	router.Handle("/", use(appHandler{HomeHandler}, middleware...))

The line that triggers the error is always next.ServeHTTP(w, r)

[clanstyles]

So digging into this more, it seems like the question is about how to store context data within an AppEngine context in middleware.

If you resolve a new context as follows:

ctx := appengine.NewContext(r)
ctx = context.WithValue(ctx, "foo", "bar")
r.WithContext(ctx)

It ends up losing it's AppEngine context.

ctx := r.Context()
ctx = context.WithValue(ctx, "foo", "bar")
ctx = appengine.WithContext(ctx, r)
r.WithContext(ctx)

Both don't seem to work though

[delphinus]

r.WithContext() creates a new(Request) and copies the original. But appengine's Context is managed by ctxs.m (here) the map[*http.Request]*Context, so the newly created Request does not exist in this map. This causes the error appengine: NewContext passed an unknown http.Request.

So I prefer to prepare my own manager of context outside middlewares. gorilla/context does it well.

[clanstyles]

@delphinus that makes a lot more sense. Let me toy around with this. I find it weird that they're creating new instances of request instead of appending the data to the existing request? What's the purpose of that?

[delphinus]

I think one of the reasons is for concurrent goroutine procedures. If it overwrites the existent *Request, the change affects other goroutines unintentionally.

[mvpmvh]

is it known if this is actively being looked into/worked on?

[CaptainCodeman]

r.WithContext() clones the request which messes with the ability to use it with AppEngine API calls.

But as long as you create the AppEngine context from the original request (which can be done in middleware) and save that context for future use then it works OK. Yes, it's a little strange storing a context in the context but it works although you have to specifically request the AppEngine context, not just use the context off the request.

Some code may make it clearer:

package api

import (
	"context"

	"net/http"

	"google.golang.org/appengine"
)

// contextKey is a value for use with context.WithValue. It's used as
// for defining context keys was copied from Go 1.7's new use of context in net/http.
type contextKey struct {
	name string
}

var contextKeyContext = &contextKey{"context"}

func appEngineContextMiddleware(next http.Handler) http.Handler {
	fn := func(w http.ResponseWriter, r *http.Request) {
		ctx := appengine.NewContext(r)
		next.ServeHTTP(w, requestWithAppengineContext(r, ctx))
	}

	return http.HandlerFunc(fn)
}

func requestWithAppengineContext(r *http.Request, c context.Context) *http.Request {
	ctx := context.WithValue(r.Context(), contextKeyContext, c)
	return r.WithContext(ctx)
}

func appengineContextFromRequest(r *http.Request) context.Context {
	return r.Context().Value(contextKeyContext).(context.Context)
}

Add the middleware to whatever mux you're using, e.g.:

router.Use(appEngineContextMiddleware)

Within the handler, get the AppEngine context:

func someHandler(w http.ResponseWriter, r *http.Request) {
	ctx := appengineContextFromRequest(r)

	// datastore calls, etc... using ctx
}

[CaptainCodeman]

In case anyone finds this useful, I made a little package for this: https://github.com/CaptainCodeman/appengine-context