Probot app config validator

[jwsloan]

What if there was either a config yml validator site, or possibly a specific app that watches for changes to config files, and comments on PRs involving them? Or puts in Issues for ones that exist already when the app is installed.