Pull project data from repos

[daveaglick]

As suggested by @khalidabuhakmeh - consider searching all of GitHub for a file like discoverdotnet.yml and use the ones we find to supplement the project data in this project. Some considerations:

• Security will be a concern - when all the data is here it has to go through review, not so if pulling data. Will need to make sure all content that gets output is properly escaped.
• Need to handle malformed files. Again, because data here is reviewed it’s not a problem, but could have bad data from external files.
• Consider quality of projects - how to make sure folks don’t just add hundreds of “Hello World” projects? Some minimum bar to include them?
• Tag taxonomy might be a problem, consider implementing a tag white list and only use tags from that. How to warn if a tag is used that isn’t part of the whitelist?
• If a data file exists both in here and at the originating repo, which one takes precedence? First thought is that the external one is always used first but any values from it are overwritten by equivalent properties from the internal one. That way we can override bad data here, but supplemental data like chat, donations, etc. and get pulled from the external file even if we define the project here.

[khalidabuhakmeh]

I remembered I contributed to this a while back. It is GitHub Topics. It allows you to create a community topic that any project can opt into.

https://github.com/github/explore

If you wanted, you could create and submit a new topic called discover .net that you could use to opt people into scanning. It would reduce the breadth of your search.

[daveaglick]

The more I process incoming PRs with new data, the more I wonder if this is doable. Having to hand-review each one does add a little overhead, but it also ensures the data is correct and in the correct format. There have been a ton of little mistakes.

Going to leave this open for now and will continue to think about it, but leaning towards not implementing.