Policy for outdated, non-official docs

[ericholscher]

trafficstars

The goal here is to have a way to remove forks of common projects in search results. We had some private conversations about this in our old, private roadmap board. Copying here:

Tasks

• [ ] Author guidelines for dealing with forked projects (as part of our abandoned project policy?)
• [ ] Add way to disable search indexing (robots.txt) for forks or outdated project docs on request of the project maintainer .
• [ ] Add way to validate ownership or "official"ness of docs on RTD? Not 100% sure if this is really valuable, since it would have to be visible on doc pages to be meaningful, and hard for us to integrate.

---

Initial thinking

A couple questions:

• How do we validate "official" docs vs. unofficial?
• How do we educate users on what is official, and what that means?

Twitter's verification is a textbook example of how hard "verification" is. Are we just saying "The repo in this project is linking to this site" in some kind of automated fashion? If so, educating users is still going to be the hardest step.

I think it's worth doing in some fashion, but it's definitely not a magic bullet, but it's a good incremental step forward.

---

I would say that we need a bit more than a verification badge alone, though it still might be part of a solution or our at least our UI.

The larger UX issue is surfacing these projects for indexing at Google, as this is the primary place these projects are found. Alone, a verification badge probably doesn't address this issue -- users will still land on the offending docs and we can't guarantee the reader user is familiar with Read the Docs or our verification system.

The most immediate problem seems to be that we don't have a way to stop forked projects from being indexed, and subsequently shown in search results. Right now, the best I can do is tell the user that we'll follow our abandoned project police and usually wait 6 weeks to remove the project. It would be much easier to visually confirm the project is a fork and flag it in the admin as a forked project, and this would be less destructive than deleting or changing maintainers.

The benefit behind hiding this feature from users is that we don't need to invest time in educating about verified projects, and we can be selective about what projects get our attention. We probably mostly only help large/popular projects with this work.

Perhaps in the future there would be a separate project to identify these forked projects in an automated or semi-automated way. However, we'd need to look at data first, and see what forked projects are doing.

[agjohnson]

Is it okay to put this on the backlog or push this one out to revisit? It seems useful still, but I guess we also only hit this infrequently -- though to be fair when we do hit this issue, it's with a high profile project.

We could kick this back to a discussion if we have unanswered questions here still too.

[ericholscher]

I think this is still worth doing, but I've been unable to get to it. If someone else wants to draft a version, that would be a good next step, but I'm fine moving it to the backlog for now, given my inability to do it 🙃

I went ahead and pushed it back a few sprints, since I do still want to get it done.

[humitos]

Twitter's verification is a textbook example of how hard "verification" is. Are we just saying "The repo in this project is linking to this site" in some kind of automated fashion? If so, educating users is still going to be the hardest step.

Mastodon has a method to verify you are the owner of the links you use in your profile:

I'm not sure if this process applies to verify a project itself, but it may bring some other ideas for this.

[agjohnson]

This pattern, or some similar schema/metadata, could be a good proactive solution for high profile projects on Read the Docs. We still need a decision tree in our policy of course, as we can't expect many projects to follow this type of pattern -- especially projects not on Read the Docs or following Read the Docs. But this at least gives an explicit mechanism for sites to tell us what is official and what is not.