Preview Environment for PRs on Answer

[ShrutiC-git]

Description

This PR is adding support for preview environments for PR events (opened, sync'ed, reopened, etc) powered by Uffizzi.

This PR is using GHA workflows to build and provision the preview environments. Utilizing docker-compose examples from answer's repo, the PR uses docker-compose.uffizzi.yml to build answer from source - this makes sure the latest changes are reflected in the preview. The docker-compose also includes a MySQL instance to persist data.

Once a PR event is triggered — PR opened, PR synced, review requested — the build workflow will build the image, and the preview workflow will either spin up a new preview or update an existing preview. The previews are ephemeral — so they live for only as long as needed.

A comment will be posted on the PR containing the URL of the preview environment, allowing contributors and maintainers to easily navigate to the preview.

Fixes - https://github.com/answerdev/answer/issues/115

Motivation and Context

The motivation behind this PR is to allow maintainers and contributors to approve contributions in half the time with live previews for every pull request. The idea behind preview environments by Uffizzi is to eliminate friction while testing and approving changes. Since answer has multiple components and each component can be interacted with in various ways (QnA → and several features related to QnAs, like upvoting, categorizing, etc), these environments will prove extremely helpful in testing if changes in the UI/API are working as they are supposed to.

An in-depth study on preview environments can be found here. Here is another easy read on where can you use Uffizzi inside answer's environment.

How Has This Been Tested?

In order to test this, I have created a PR against the main branch of my personal fork of answer. This triggered the workflow to create a new preview environment - here and this comment, posted on the PR, takes me to the preview environment.

To log into the above preview environment, use the following credentials:

• username: [email protected]
• password: password

Once you are logged in, you will be able to do all you would on a production instance of Answer; create users, confirm users, create tags, etc.

To test notifications, I started an SMTP server and through the admin console of Answer, passed the server name, username, and password of the server. Upon saving these changes, test emails were sent successfully to the test email and subsequent notifications as well.

Navigating Around a New Preview Environment

Once this PR is merged, post that, here is how you could use the new preview environment — when you visit the link in the PR, visit /install and choose your desired configuration. This preview comes with MySQL, available at -

• username: mysql
• password: mysql
• database: answer
• connection string: localhost:3306

After successful installation, you will be able to log into the preview environment using the admin credentials you provided during installation. After setting up the admin, you can also set up other users.

In case you get a 50x, make sure the app is actually installed by visiting /install. This needs to be done every time a new preview environment is spun, which happens when either a PR is closed, a new PR opens or an environment has been marked stale past its expiration time.

---

Looking forward to powering preview environments for Answer, and thereby, making reviewing, testing, releasing easy. Please let me know if there are any questions or if there's anything we can change to make Uffizzi better align with Answer!

cc @joyqi @waveywaves

[ShrutiC-git]

hey, there @joyqi, let me know your thoughts on this PR! If this needs any changes, let me those, too.

[joyqi]

Hi @ShrutiC-git , we appreciate your hard work. I've researched your product in the past few days, and I think it's interesting and very useful. Come to this pull request, I suggest you can combine these steps into a single GitHub action, so that we can use it more conveniently. There will be more and more third party services in the future, we need to manage them in the same way.

[ShrutiC-git]

hi @joyqi, thank you for your feedback 🙏🏼

Re combining the steps into a single GHA — that's a very valid suggestion, we definitely have encountered questions along the same line of thought previously. We use the 2-stage GHA workflow for open-source projects to allow them to have previews for contributions from forks, too.

• The uffizzi-build.yml action runs in the context of the contributor's head branch, from where the PR is coming.
• The uffizzi-preview.yml action runs in the context of the target base branch.

We need this to allow a "privileged-unprivileged" model, where the first action (uffizzi-build.yml), running in the context of the contributor's head branch, does not need to have access to the OIDC tokens, which are required to verify the identity of the requester (needed by Uffizzi to authenticate the request and create a new preview). So this action runs in unprivileged mode.

The uffizzi-preview action runs in privileged mode. This action needs to have access to these OIDC tokens to help Uffizzi verify where the requests are coming from. This action must run in the context of the base branch for which the PR is generated.

We have published a really thorough read on Why 2-stage workflows → this is a great read and a deeper dive into this topic.

Let me know if this flow works for Answer's community. This flow is a recommendation, of course, to allow PRs from forks. Depending on what works best for your team, we'll make the amends.

[joyqi]

@ShrutiC-git thanks for your introduction. I can understand the purpose of your use of two separate GitHub actions. Will you provide an official packaged composite action? It will be good for arranging our CI/CD code.

[ShrutiC-git]

@joyqi, just to confirm, you are looking for one single action which will run both the uffizz-build and the uffizzi-preview step, rather than two separate workflows. Is that what you meant by official packaged composite action?

[joyqi]

@ShrutiC-git Just like

- name: Preview with UffizziCloud
  uses: UffizziCloud/preview-action@v1
  with:
    file: Dockerfile
    ...balah balah

It will be nice if you have a general GitHub action with some parameters to handle these requests.

[ShrutiC-git]

I see what you mean, @joyqi

We do have a reusable workflow published on GHA marketplace - here.

There are a couple of ways this reusable workflow can be used — this will also check the single composite action mark. I can switch from the 2-stage workflow (what is on this PR now) to a single action flow so the Answer community can get the most benefit out of preview environments. With the caveat that contributions from forks might not create previews.

If you'd like, we can set up a call to discuss the use case and how we could set Uffizzi up to fit into Answer's current architecture.

[ShrutiC-git]

hey there, @joyqi~

Hope you were able to check out the reusable workflow action I linked to (here). Let me know if this flow would be more helpful!

[joyqi]

Hi @ShrutiC-git , sorry for the late reply. Our team had discussed this PR a lot. For us, the main concern is commercial use. As an open-source project, we're still determining if to integrate a third-party service.

[jpthurman]

Hi @joyqi I'm one of the founders at Uffizzi - I wanted to see if there a particular concern you have that I might be able to address?

There's several other open source projects we are supporting that I could point you to if a reference would help the situation.

Let me know how I can help

[joyqi]

Hi @jpthurman , Here are my questions.

1. What are Uffizzi's limitations on the use of open-source projects?
2. Are there some potential commercial requirements? e.g., show Uffizzi's logo on our website.
3. So if we decide to use it, is it time for us to have a Uffizzi account?

Thanks!

[jpthurman]

Hi @joyqi, happy to answer those.

1. Open Source Projects are automatically on our "Pro" Tier. This is the highest tier we offer. Unlimited Preview Environments and Members

2. There's no requirement for you to promote Uffizzi on your repo or website. We would plan on listing your project as one of the several open source projects we support - list is here.

3. Yes, there's no way to have the environments without an account but the set-up is embedded in this PR. If you merge this PR, an account will automatically be provisioned. Preview Environments will then run automatically based on the GHA workflow triggers that are embedded in this PR. You can read more about the set-up here.

Logs for your environments are available in our UI and anyone who opens a PR to your repo will have access to those logs by logging in to Uffizzi with their Github Username.

[joyqi]

Hi @jpthurman, thanks for your explanation. We're glad to see there's such a superb service provided for an open-source project. We'll merge this PR after the next release and do some environment set-up work during this time.

[jpthurman]

Sounds good - let us know how we can help

[ShrutiC-git]

hi, @joyqi, were you able to go through the setup work you mentioned?

Eager to see how the answer community will use Uffizzi and also collect your feedback! If there's anything that's holding you back, feel free to share your concerns and we'll make sure we address those.

[joyqi]

Hi @ShrutiC-git , happy Chinese new year! We're on vacation now but will do some set-up work after we get back.

[ShrutiC-git]

Happy Chinese New Years, @joyqi! thanks for the update. No worries enjoy the holidays :)

[joyqi]

Hi @ShrutiC-git , we've merged this PR. There seems to be something wrong here: https://github.com/answerdev/answer/actions/runs/4099990252

[ShrutiC-git]

hey, @joyqi Thanks for raising the issue.

You're seeing this error because when you closed the PR, this triggers our Delete Deployment for that PR step in the Deploy Preview action. However, since there was no active deployment for my PR (PR #118), the action failed.

We're working on a release of our action where for such conditions the action is skipped, so it does not fail - the failed action is definitely confusing.

I checked and also saw some recent Deploy Preview actions failing. These are because of what we have identified as race conditions.

1. PR #186 was opened → this triggered the Build PR Image action (and as that was completed, it triggered the Deploy Preview action.
2. Before the previous action successfully deployed the preview, PR #186 was synced within a second of it being opened → which triggered the Build PR action again. When the Deploy Preview action was triggered, the sync event should ideally update the preview for that PR or deploy a new one if no deployment is found. In this case, the previous action had not been completed, so no active deployment was found. So this step also started creating a new deployment (hence the two comments on the PR, each deployed by these two actions running in parallel).
3. When PR #186 was updated again by new commits → there were two deployments found. One deployed by Step-1 and another by Step-2. So this action had two deployments (https://github.com/answerdev/answer/actions/runs/4100286337/jobs/7070975460#step:9:13).

We have identified that this happens when a PR is synced within seconds of it deploying a preview environment.

For this particular PR, I suggest you delete one of the previews from the console. For subsequent PRs, I'll keep monitoring if the same behavior persists.

We have also marked and will be releasing a fix for this so these race conditions and also the first error you were seeing are better handled!

[ShrutiC-git]

I'm also seeing a container state issue with the preview — I'm taking a look into why that error popped. I will post an update here