Robots.Txt: Static file does not respect DebugSettings.DisableRobotsTxt

[HeyJoel]

If we add a robots.txt static file to the wwwroot content folder, it is always served even if DebugSettings.DisableRobotsTxt is set to true. This is because static files are served before the mvc middleware is invoked.

We'd need to adapt the StaticResourceFileProvider so it could be configured to filter out certain files like robots.txt similar to FilteredEmbeddedFileProvider.