reuse built frontend in ci, merge compose files

[AndrewChubatiuk]

What type of PR is this?

• [x] Refactor
• [ ] Feature
• [ ] Bug Fix
• [ ] New Query Runner (Data Source)
• [ ] New Alert Destination
• [ ] Other

Description

• renamed docker-compose.yml to compose.yml as it's supported by docker compose v2 by default
• merged all compose files to simplify local backend and e2e tests execution. added profiles:
  • local - spins up all services in a setup, that previously was in docker-compose.yml
  • e2e - spins up all services in a setup, that previously was in .ci/docker-compose.cypress.yml
  • default(no profile set) - spins up all services in a setup, that previously was in .ci/docker-compose.yml
• added ability to build frontend inside docker or reuse prebuilt frontend locally for CI purposes to speed up build

How is this tested?

• [ ] Unit tests (pytest, jest)
• [ ] E2E Tests (Cypress)
• [x] Manually - executed e2e and unit tests locally against all three compose profiles
• [ ] N/A

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

[AndrewChubatiuk]

@justinclift could you please review this MR?

[justinclift]

@AndrewChubatiuk It's unlikely I'll have much free time to review Redash stuff today. :frowning:

Maybe one of the others can help instead? :smile:

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 63.37%. Comparing base (af0773c) to head (15b94b9). Report is 2 commits behind head on master.

:exclamation: Current head 15b94b9 differs from pull request most recent head 1619b0b. Consider uploading reports for the commit 1619b0b to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6674      +/-   ##
==========================================
- Coverage   63.82%   63.37%   -0.45%     
==========================================
  Files         161      162       +1     
  Lines       13060    13170     +110     
  Branches     1803     1819      +16     
==========================================
+ Hits         8335     8347      +12     
- Misses       4425     4532     +107     
+ Partials      300      291       -9     

see 21 files with indirect coverage changes

[justinclift]

Thinking this bit over:

renamed docker-compose.yml to compose.yml as it's supported by docker compose v2 by default

I'm unsure if we want that rename. My thinking is that docker-compose.yml is widely known and recognised, while compose.yml isn't.

[AndrewChubatiuk]

@justinclift could you please review it again?

[AndrewChubatiuk]

@guidopetri @justinclift @eradman could you please review this MR? With pull_request_target event type (which redash was recently switched to) Github Actions uses a pipeline from a target(master) branch, so this MR will never be green

[eradman]

When I run the build there are a number of new warnings:

$ make
docker compose build
WARN[0000] The "COMMIT_INFO_SHA" variable is not set. Defaulting to a blank string.
WARN[0000] The "COMMIT_INFO_REMOTE" variable is not set. Defaulting to a blank string.
WARN[0000] The "CYPRESS_PROJECT_ID" variable is not set. Defaulting to a blank string.
WARN[0000] The "PERCY_TOKEN" variable is not set. Defaulting to a blank string.
WARN[0000] The "PERCY_BRANCH" variable is not set. Defaulting to a blank string.
WARN[0000] The "COMMIT_INFO_AUTHOR" variable is not set. Defaulting to a blank string.
WARN[0000] The "COMMIT_INFO_BRANCH" variable is not set. Defaulting to a blank string.
WARN[0000] The "COMMIT_INFO_MESSAGE" variable is not set. Defaulting to a blank string.
WARN[0000] The "CYPRESS_RECORD_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "PERCY_COMMIT" variable is not set. Defaulting to a blank string.
WARN[0000] The "PERCY_PULL_REQUEST" variable is not set. Defaulting to a blank string.
WARN[0000] The "REDASH_PRODUCTION" variable is not set. Defaulting to a blank string.
WARN[0000] The "REDASH_PRODUCTION" variable is not set. Defaulting to a blank string.
WARN[0000] The "REDASH_PRODUCTION" variable is not set. Defaulting to a blank string.

The build works, but these messages make it look like I am missing a step. I'd say we should only emit a warning if the developer has an incomplete configuration.

[AndrewChubatiuk]

@eradman made these variable overridable and empty by default to remove warnings

[justinclift]

Hmmm, that doesn't seem happy:

stat /home/runner/work/redash/redash/.ci/compose.ci.yaml: no such file or directory

[AndrewChubatiuk]

Hmmm, that doesn't seem happy:

stat /home/runner/work/redash/redash/.ci/compose.ci.yaml: no such file or directory

this is expected behaviour as CI relies on configuration in master. that's why I've tested it in fork to prove it's working properly

[justinclift]

Ahhh, yeah. We might as well include the lint change too. All good. :smile: