Create a dashboard on the website

[KhudaDad414]

Reason/Context

AsyncAPI Initiative has 55 repositories as of writing this issue. It is hard to have a clear picture of what is going on in the initiative and where. the Shape Up was a great attempt to create a clear picture of what is currently happening and what should happen which unfortunately only went on for three cycles. I think a dashboard on the website would be an amazing tool to have.

Why do we need a dashboard?

Well, A dashboard would help new contributors to select and start working on a good-first-issue and will help maintainers to have a clear picture of what is going on in the Initiative.

What are the main features of the dashboard?

We can add more and more features in the future but now we can go with two.

1. A list of Good First Issues that can be filtered by the repo, type and language maybe?
2. A list of currently hot issues in all repositories.

Description

what should be the location of the dashboard on the website?

It can be separate from the website somewhere like dashboard.asyncapi.com or similar.

How is it going to be implemented?

this feature can be implemented in three steps.

1. Collecting data from repositories using Github Actions and Github Graphql APIs.
2. Storing those data in Projects (beta).
3. Redeploy the website on-demand or in time intervals maybe?

Scenario 1

• an issue is tagged as "good-first-issue".
• this would trigger an action that does two things: a) add the issue to the projects (beta) list of the Good First Issues. b) trigger a deployment of the website with the latest updates.

Scenario 2

• someone comments on an issue.
• this would trigger an action that would do two things: a) update the "hotness" of that issue on the projects (beta). (I am not sure how we could calculate it though. maybe comments+reactions/unit-of-time?) b) trigger a deployment that queries the top hot issues from projects (beta) and updates the website.

[AceTheCreator]

@KhudaDad414 That's a very lovely idea you've got there. IMHO I think this would make it really easy for new contributors to have a quick overview of what repo they can start contributing to, also based on the issue label.

[derberg]

A list of Good First Issues that can be filtered by the repo, type and language maybe?

It is also very important to have information on the difficulty level. Last year I interviewed every single person that contributed to the project through Hacktoberfest and they said that the best stuff was our single list like this where they could not only see what language is involved but also difficulty level.

It can be separate from the website somewhere like dashboard.asyncapi.com or similar.

why do we need a subdomain, why not just asyncapi.com/community ?

Scenario 1

yup, definitely, GH Actions workflow in every repo that reacts on label added event

trigger a deployment of the website with the latest updates

I suggest we don't do it, dashboard will not be visited by 100s of people every day and doesn't have to be refreshed with current state "real-time". Netlify is free but not unlimited, we should always carefully care about the build minutes limits. Better have a cron-job that runs website rebuilt one a day or once a week. Definitely something to consider 🤔

I am not sure how we could calculate it though. maybe comments+reactions/unit-of-time?

I think it is all that you mentioned, we just need to agree on an algorithm. Like if the issue has more than 3 reactions added in a day or more than 3 comments added in a day - or something like that. Then, when you save such a "hot" issue/PR, you save information about amount of reactions and comments, and we use these to sort and pick the hotest.

[KhudaDad414]

@AceTheCreator It was @fmvilas idea, but still, thank you. 😄

[fmvilas]

why do we need a subdomain, why not just asyncapi.com/community ?

Yeah, I think we can have it on the website. No "need" for a separate domain.

I suggest we don't do it, dashboard will not be visited by 100s of people every day and doesn't have to be refreshed with current state "real-time". Netlify is free but not unlimited, we should always carefully care about the build minutes limits. Better have a cron-job that runs website rebuilt one a day or once a week. Definitely something to consider 🤔

That would be sufficient for now, yeah. Maybe once a day and we can increase it if we think we need it. @KhudaDad414 if we end up doing it with this approach, you would not need to store information anywhere. Just a few API calls during the deployment to grab all the good-first-issues, hot issues, etc. No problem with API rate limits as it's only once a day (the website is not deployed so frequently either).

[KhudaDad414]

@KhudaDad414 if we end up doing it with this approach, you would not need to store information anywhere.

this is a great idea. IMO we should introduce extra tags for good-first issues like Lukasz said. e.g. language that is involved and the difficulty level.

[derberg]

Just a few API calls during the deployment to grab all the good-first-issues, hot issues, etc. No problem with API rate limits as it's only once a day (the website is not deployed so frequently either)

I'm always less optimistic here. Highly recommend doing a test from https://docs.github.com/en/graphql/overview/explorer with rate limit check to check a cost of such a call. We are doing a lot of calls to API already on PRs, and different workflows, like checking stale issues or other stuff. I recommend this presentation from Rea where she shares a hidden costs of API calls on GraphQL.

Just saying that eventually your initial idea to react on event and aggregate data is really the best one imho

language that is involved and the difficulty level.

just suggest labels names and colors. We definitely need prefixes for them. So maybe lang/javascript and complex/easy or something like this

[KhudaDad414]

@derberg @fmvilas experimented with Graphql API and IMO we should go with a hybrid approach. for each feature, we can calculate the cost of the queries. if it's feasible to query at deploy time we can do that and if it is not feasible we can go with the data collection approach. I calculated the cost for the two main features and here is the result:

Good First Issues

to query all of the good first issues along with their labels it will cost 12 points which is not a lot so we can query them at deploy time.

Hot Issues

Since we have to query all of the comments and reactions as well for all issues and PRs, it is simply not possible to query at deploy time. I don't have an exact amount since we have to go FIVE levels deep which would require pagination in multiple levels. but it is definitely in thousands. unless someone comes up with a clever approach, we should go with a data collection approach for this one.

[derberg]

so I guess that to not overcomplicate, we should just do data collection for both really, right?

[KhudaDad414]

Update:

The data collection part is done (maybe). you can play around with it here and see the collected data Here and Here.

An small issue with hot Issues/PRs:

Something that I haven't anticipated is that we can't trigger GitHub Actions by reactions to comments and issues. we are considering each reaction as a point but recalculation of scores is relying on comments being added. ( we can add other actions to trigger the recalculation of scores like, ready for review, labelling etc.)

API call costs

every action costs 2 but recalculation of scores for each issue/PR costs 3.

[derberg]

I see this but no labels here that would indicate code or complexity. I guess I need to look somewhere else, right? or was it manually added?

Nevertheless, it looks neat! For sure we need to clearly document what is the calculation algorithm behind Hot Issues and PRs

Something that I haven't anticipated is that we can't trigger GitHub Actions by reactions to comments and issues

Actually, I think it better if the calculation would not react to the event but on a cron schedule. Like once a day, we do not need real-time data "refreshment" there I think

[KhudaDad414]

I see this but no labels here that would indicate code or complexity. I guess I need to look somewhere else, right? or was it manually added?

I added them manually in this repo.

we need to clearly document what is the calculation algorithm behind Hot Issues and PRs

Well, IMO we can use the Hacker News algorithm (It's simple and sweet 🍬 ). p / (t + 2)^1.8 p = comments + reactions + reviews ... t = age of issue (Hacker News puts the hours here, maybe we can go by day?)

I think it better if the calculation would not react to the event but on a cron schedule. Like once a day, we do not need real-time data "refreshment" there I think.

Since we have around 600 issues and PRs open right now, it would cost around 1800 points to refresh the scores at the same hour... which is a lot. I am experimenting with Github search API and I think I can come up with a custom solution to pull everything from Github with less than 200 points. If that was successful, then we can ditch all of the complications and pull everything once a day.

[derberg]

I'm not sure we should take age into account, HackerNews has a different scope, age plays an important role there, but here, where things are always super slow in open source, I think age should probably be ignored. Maybe we need a label that we can set on issue or PR to ignore it for HOT calculation? not sure

[KhudaDad414]

@derberg I see your point there. But if we take out the age factor then "hot" has no meaning here. maybe we can call them "All-time popular open Issues/PR" or something like that. The idea behind hot issues/PRs is that we should be able to see where the community is focused right now or what is "hot" in the whole community. Hacker News posts age every hour, issues can age every day, every week, or maybe every month. we can find a sweet spot. Why not have both though 🤔 ? popular issues (no ageing) and hot issues (ageing)?

[derberg]

you have a valid point mate. Maybe let us have a look at some examples, and the best is https://github.com/asyncapi/spec/issues/618

• do we count age from issue creation or last comment?
• month should be good but only from last comment I think

[mcturco]

It is also very important to have information on the difficulty level. Last year I interviewed every single person that contributed to the project through Hacktoberfest and they said that the best stuff was our single list like this where they could not only see what language is involved but also difficulty level.

This is a great idea @derberg, and forgive me if I missed it while reading through all your comments above, but is there a way for us to rate an issue within github? I'm actually wondering if we can label the issues with a prefix of some sort and then pull that in as a filter somehow. Unless there already is an option for this in GH and I am just not aware of it!

[mcturco]

In addition, @KhudaDad414 reached out to me to put together a design for this. I initially put together a low-fidelity wireframe so we can work out the functionality together without any aesthetic distractions.

Here is a baseline wireframe, and if we see more opportunities to add options/filters I am open to suggestions! And depending if it is possible, would love to work in the "difficulty" attribute for the tasks: https://www.figma.com/proto/WKOxyb4VQ9D1QcRd7y7OfU/Wireframes?page-id=1%3A17&node-id=1%3A18&viewport=241%2C48%2C1&scaling=min-zoom&starting-point-node-id=1%3A18

[KhudaDad414]

@mcturco Loved the design. ❤️ just a few things that I think needs addressing.

1. about the difficulty level, I don't think there is a way to do it so we just agreed to use a variation of these labels complexity/easy complexity/medium and complexity/high.
2. About the filters, should we be able to filter the Hot issues by any metrics? currently, I am just taking the top 10 and showing them without any filter options(this is how it looks like), definitely want to know others opinions. as an extension to this, IMO we should have localized filters for each column. for example, good first issues can be filtered by area and complexity, hot issues can have no filters and another column of data will have other matrics to be filtered by?
3. IMO Language or as we call it area filter should be applied only to the good first issues since we can't expect maintainers to provide this information on all issues.
4. hot issues also includes PRs, so I was thinking about a more general name, like Hot discussions? so do we need to show the type of discussion on the Dashboard?

[mcturco]

@KhudaDad414 thank you for the feedback! Read my responses below:

1. about the difficulty level, I don't think there is a way to do it so we just agreed to use a variation of these labels complexity/easy complexity/medium and complexity/high.

Okay! I am wondering if we can then use those tags and string replace the complexity/ part out and use these as filters in the design? Just a suggestion to make it easier for the user to find the issue they want to work on from the dashboard. But let me know if that's not possible!

2. About the filters, should we be able to filter the Hot issues by any metrics?

I think the Hot issues/PRs (or whatever we decide to name it) should be top 10 issues/PRs that have the most activity. I also think similar to how GitHub lists out repos based on how recently someone committed, we should put the issues/PRs in order from Last activity as you have it in your current link. For example, in your working link, the PR: feat: iterate through all the schemas and handle circular references would be at the top of the list. (This order would only update daily because of our Netlify limitations, so I am also wondering if we should even list Last activity?

Good first issues can have its own set of filters, maybe as a fly out window instead of how I currently have it on top of all 3 columns. These filters would only have area and complexity as you were saying above, no problem!

4. hot issues also includes PRs, so I was thinking about a more general name, like Hot discussions?

I wonder if we should keep it simple and call it Hot with the icon? I think that Hot discussions might also be misconstrued as a column of ONLY discussion boards. What do you think? Perhaps Hot topics could work better?

Now that I know there are issues and PRs that will be listed on this dashboard, I think I am going to use the issue and PR icons for each item to label them accordingly. I wonder if we can add a baseline filter to each column to filter between Issues and PRs, just a thought.

Going to take this feedback and make another iteration of the wireframe!! :)

[mcturco]

Some more thoughts...

1. Would Good First Issues ONLY include issues? No pull requests right? And would we list that Good First Issue here if there is already a PR attached to it? I think no, because the purpose for Good First Issues would be to get people to contribute to issues that we need help with, right?
2. Would someone looking for a Good First Issue want to filter them by the hardest complexity? Most of the time when someone is contributing for the first time they are looking for easier tasks right? So is it possible that we should only include easy tasks in the Good First Issues column? Maybe on the dashboard we can have another column that has All Issues or something along those lines and they can filter through the complexity there? But that would only work if everyone got into a routine of labeling the complexity of each issue. Just a thought!

[KhudaDad414]

@mcturco

I am also wondering if we should even list Last activity?

Agreed. since it is not real-time then the Last activity can be omitted.

Would Good First Issues ONLY include issues? No pull requests right?

Yes, only issues.

And would we list that Good First Issue here if there is already a PR attached to it? I think no, because the purpose for Good First Issues would be to get people to contribute to issues that we need help with, right?

Yes, exactly. I think we can include the is assigned field.

I am wondering if we can then use those tags and string replace the complexity/ part out and use these as filters in the design?

Definitely, I am currently doing this on the draft.

I wonder if we should keep it simple and call it Hot with the icon? I think that Hot discussions might also be misconstrued as a column of ONLY discussion boards. What do you think? Perhaps Hot topics could work better?

I think both works. Hot or Hot topics. as the saying goes "I have no strong feelings, one way or another." 😆

Would someone looking for a Good First Issue want to filter them by the hardest complexity?

Lukasz answered this question here. If I add to that I would say we need to choose words that imply the intent of complexity, not hardship. Hard is a strong word maybe we can go with challenging ?

We can also take inspiration from https://codetribute.mozilla.org/ . I like the I’m Feeling Adventurous button.

[derberg]

Lukasz answered this question here. If I add to that I would say we need to choose words that imply the intent of complexity, not hardship. Hard is a strong word maybe we can go with challenging ?

maybe we should just have 1-5, like complexity/1? any word we pick can cause confusion trivial might be depressing for folks that have problems with doing the task and therefore think that can't even handle something that is trivial. Numbers seem pretty neutral, and for us advantage is that they are easy to remember

[mcturco]

@derberg that is a great idea! Do you think a scale of 1-5 is too complex (no pun intended) or should we go with 1-3? That way we can associate the numbers with easy medium hard without labelling it as such

[derberg]

@mcturco yup, I agree 1-3 seems really enough

[mcturco]

@KhudaDad414 @derberg Okay so taking into account the discussions we had about the dashboard and how we need it to function, I am proposing this layout/functionality that you can see in this low-fidelity prototype: https://www.figma.com/proto/WKOxyb4VQ9D1QcRd7y7OfU/Wireframes?page-id=1%3A17&node-id=43%3A165&viewport=241%2C48%2C0.4&scaling=min-zoom&starting-point-node-id=43%3A165&show-proto-sidebar=1

(Please note that this is not the design including UI, that step will come after we finalize the UX)

Intro Section This section at the top will be a short description of how people can use the dashboard and some quick links to join Slack and Github

Hot Topics I decided to call this column Hot Topics as I feel it is the best label to be inclusive to both Issues and PRs (possibly even discussions?) In my opinion, in this column PRs and Issues should render first according to recent activity. For example, if someone comments on an issue, that one will render to the top until another issue/PR gets activity then that one will render to the top (I hope that makes sense, kind of like a timeline?) In each issue/PR card, you will be able to see the issue/PR number, the repo it belongs to, title, labels, comment count, and assignee.

Good First Issues These will be only issues that are tagged with good-first-issue. Next to the title of this column, you can hover over the question icon that will explain to you what this column means (we can update this verbiage to whatever we think will work best there). At the right of the column title, you will see three dots. If you click on that it will toggle a filter modal where you can filter by a repo, by complexity, and code area. I have prototyped some of the actions so you can get a feel of how I am imagining it to work. In each issue card, you will see the issue number, repo, title, label(s), comment count, complexity scale (shown visually), and assignee.

I think it might be cool to add some more "columns" with different sorting, such as "Stale Issues" or something where we can list out issues that have been sitting for a while and need attention. Curious to know what everyone thinks would be useful to them in this tool.

[doc-jones]

Two things occurred to me after first looking at the prototype and then reading your description.

1. I wrongly thought that Hot Topics referred to the most popular issues, rather than, the newest. Perhaps it would be more intuitive to say that these are the Most Recent Issues ?
2. Would it be possible to allow people to select a tag for display? You might highlight good-first-issue as an example of what is available in the list to ensure that new contributors are drawn to that select and view.

I really like the layout and presentation of the boxes. I know it's not possible, but I would love to see it stay in grayscale. heh

[mcturco]

Thank you for your feedback, @doc-jones !

I wrongly thought that Hot Topics referred to the most popular issues, rather than, the newest.

I think I was trying to figure out how the "algorithm" for that column would be. I agree that it should be the most popular, but I guess I was trying to think of how we can make sure that issues don't go stale. Maybe it could be sorted by amount of actions (comments, mentions, etc)? Just wondering how that would actually look when in the application. Something that we can definitely discuss and resolve as a group!

Would it be possible to allow people to select a tag for display? You might highlight good-first-issue as an example of what is available

So the idea of the Good First Issues column is that it will ONLY display issues that are labelled with good-first-issue consider it a pre-filter 😄 However, we can add a "filter by label" filter in the filter modal if we all think that would be useful? My opinion is that its not necessary because most first-timers aren't going to understand the meaning of each of the labels that we currently have.

[doc-jones]

@mcturco I'm not advocating for one approach or the other when considering what should be displayed under Hot Topics. My only point was the the label lead me to think it was something that it wasn't. I'm not sure that most recent isn't the more valuable. I haven't thought through that part or looked into the goal enough to know.

When I saw the list separated by tag, I thought, Hey! that's really useful. I would love to be able generate a list with other labels. If you say, it should only be for new people then okay, but I still like the idea of generating a list for each label as a non first time open source contributor. :-)

[KhudaDad414]

@mcturco @doc-jones For Hot Topics algorithm, Lukasz and I discussed it here. It would be great to have your input on that.

[derberg]

@doc-jones

I would love to be able generate a list with other labels. If you say, it should only be for new people then okay, but I still like the idea of generating a list for each label as a non first time open source contributor

could you share what would be the added value for you? Good first issues has a use case that it is for first-time contributors that do not want to jump to each repo one by one to figure out where such issues are located. How about bug or enhancement? They usually require more work and therefore the use case a see is that you look for them only for specific repo and not entire org.

risk to consider - we do not enforce "fixed" labels structure. We only add some that we consider useful for entire org, like good first issue or the others needed for the dashboard. So a label filter can contain lots of noise, like labels that are available only in one specific repo.

[KhudaDad414]

@mcturco what do you think we move forward to the next step on this issue? After receiving the design I can open a PR then we can refine things there maybe?