Course assets should be served by a view rather than a middleware

[timmc-edx]

Acceptance criteria:

• [x] StaticContentServer is converted from a middleware to a view
• [x] The cutover is managed with a Waffle flag or similar to allow careful rollout
• [x] Documentation added to next Open edX named-release notes
• [ ] Optional: Additional cleanup
  • Remove temporary IMPL variable and StaticContentServer class by lifting methods to module scope
  • Separate views for different URL patterns

Course asset URLs like /asset-v1:... are served from a middleware at openedx.core.djangoapps.contentserver.middleware.StaticContentServer rather than from a view. This means that Django starts handling the request as a Not Found (because there is no matching urlpattern), but then the middleware intercepts the request and returns a response of its own. There wasn't any good reason for doing this instead of using a view, as far as I can tell. It causes some problems for telemetry: When the code-owner middleware asks Django what view handled the request, it does so by looking at the result of the resolve utility, but these URLs get a Resolver404 (because there's no registered urlpattern).

Implementation suggestion:

1. Register equivalent urlpatterns and a view that calls into the same code the middleware currently contains.
2. Add a Waffle flag that causes the middleware to fall through to the view.
3. Try turning the flag on.
4. Remove the middleware so that the view is always used.
5. [Beyond this ticket:] Bring the new view's code up to standards (better use of urlpatterns, etc.)

PRs:

• Refactor to add urlpatterns and allow use of a view via a flag: https://github.com/openedx/edx-platform/pull/34703
• A fix for that PR: https://github.com/openedx/edx-platform/pull/35247
• Switch permanently to view: https://github.com/openedx/edx-platform/pull/35420

[kdmccormick]

+1. This seriously confused Connor and I today!

[timmc-edx]

Private 2U dashboard for monitoring cutover: https://app.datadoghq.com/dashboard/vjs-j3j-czr

[robrap]

We don't actually own this, and it is no longer a high priority, but we might be able to help with rollout.

[jristau1984]

@timmc-edx is this work complete, or is there anything left to do?

[timmc-edx]

We're still in the early stage of this work. I had been hoping to try the cutover once the Datadog traces were fixed, but that's not happening any time soon, so maybe we should do that sooner instead.

[robrap]

Nice to have, but close to done, so we just want to close this out.

[timmc-edx]

Looks like I enabled this for stage.edx.org on June 10. Haven't seen any complaints, and the graphs look fine.

[timmc-edx]

Enabled on edge.edx.org but had to disable again as I discovered that it was causing almost all c4x URLs to return 404. Not sure why yet.

EDIT: Example for testing: https://courses.stage.edx.org/c4x/edX/Open_DemoX/asset/images_course_image.jpg -- probably just messed up the urlpattern path.

[robrap]

@timmc-edx: This is a P4 that we made a P3 because we were hoping to simply ship an implemented solution. Now that a bug was found, we should probably timebox a resolution, rather than just plowing through this no matter how long it takes.

[timmc-edx]

Agreed, although I think the fix will be pretty straightforward.

[timmc-edx]

edx.org has been converted over successfully. We'll let this bake for a week or two in production and then we can move the implementation properly into views.py and remove the old middleware.

[robrap]

@timmc-edx: FYI: I moving this out of blocked because it has sufficiently baked based on your last comment.

[timmc-edx]

This is largely complete, and now just wants a bit of optional cleanup. Acceptance criteria updated.

[timmc-edx]

Closing out -- last bit of cleanup would involve turning this into three views, and probably isn't worth the effort right now.