Fix failing tests when `HIDE_THINGS` is true

[gazayas]

There are some tests that fail when HIDE_THINGS is set to true, so I took care of most of those here. There are a lot of diffs here, but all I did was wrap the tests that used Creative Concepts and such with scaffolded_things_disabled?.

The only test that isn't covered is the Teams endpoint test. I took care of it in this PR, so we'll need to merge that in too to finish up.

Besides that, I didn't want to just depend on CircleCI tests without the presence of HIDE_THINGS, so I set things up to run the tests twice, once without the presence of HIDE_THINGS, and one more time when it's set to true. This might be overkill, but I think it's worth having if developers are running into issues like the one mentioned in bullet_train-api PR 10.

I'll create the PR to see if CircleCI is set up properly.

[gazayas]

This is going to seem silly, but by the time we run the same tests again in the same job (which is what I've been trying to do with these commits), knapsack pro is just saying all the tests are done so it won't bother to run them again even if we want to... That's why all the tests are passing even when HIDE_THINGS is in the environment variables.

If you look at Job 1282, the HIDE_THINGS portion of the tests ends in 2 seconds flat, and yields the following message:

#!/bin/bash -eo pipefail
HIDE_THINGS="true"
export RAILS_ENV=test
bundle exec rails "knapsack_pro:queue:minitest[--verbose]"

D, [2022-05-20T10:37:46.554191 #1954] DEBUG -- : [knapsack_pro] POST https://api.knapsackpro.com/v1/queues/queue
D, [2022-05-20T10:37:46.554251 #1954] DEBUG -- : [knapsack_pro] API request UUID: 872a4cb0-54f4-408e-a7df-a412e2f0f558
D, [2022-05-20T10:37:46.554272 #1954] DEBUG -- : [knapsack_pro] API response:
D, [2022-05-20T10:37:46.554301 #1954] DEBUG -- : [knapsack_pro] {"queue_name"=>"2234:febb88c702ea3d5910c2968273bca295", "build_subset_id"=>nil, "test_files"=>[]}
W, [2022-05-20T10:37:46.554390 #1954]  WARN -- : [knapsack_pro] No test files were executed on this CI node.
D, [2022-05-20T10:37:46.554405 #1954] DEBUG -- : [knapsack_pro] This CI node likely started work late after the test files were already executed by other CI nodes consuming the queue.
D, [2022-05-20T10:37:46.790186 #1954] DEBUG -- : [knapsack_pro] POST https://api.knapsackpro.com/v1/build_subsets
D, [2022-05-20T10:37:46.790237 #1954] DEBUG -- : [knapsack_pro] API request UUID: 7f1eb7d2-1b8d-43c4-a698-8416cccc8f20
D, [2022-05-20T10:37:46.790260 #1954] DEBUG -- : [knapsack_pro] API response:
D, [2022-05-20T10:37:46.790276 #1954] DEBUG -- : [knapsack_pro] 
D, [2022-05-20T10:37:46.790301 #1954] DEBUG -- : [knapsack_pro] Saved time execution report on Knapsack Pro API server.
CircleCI received exit code 0

To fix this, I just created a new job.

You can see that the most recent commit fails with the Teams endpoint test as expected. This fails similarly to when running tests linearly, which I checked in https://github.com/bullet-train-co/bullet_train/pull/172/commits/c0ece29c0fe606b1b0179e4117b72ec51f53e474.

[gazayas]

There was some talk about tests failing when HIDE_THINGS is set to true, so I looked over this PR again and added the joint PR branch to show that this is passing when it’s included.

It might not be the best way to test things since we’re simply skipping the code, but if we have this in conjunction with the normal tests and make sure the tests there run with HIDE_THINGS=false, then we can cover all of our bases so to speak.

I’ll go ahead and remove the PR branch from the Gemfile.

[andrewculver]

@gazayas Can you resolve the conflicts on this one?

[gazayas]

@andrewculver Merged this one :+1: Just to reiterate, we'll need this PR too to make the unit tests pass:

• https://github.com/bullet-train-co/bullet_train-api/pull/10

Here's a oneliner to include it in the Gemfile:

gem "bullet_train-api", git: "[email protected]:bullet-train-co/bullet_train-api.git", branch: "fixes/hide-things-in-api"

[gazayas]

I revisited this after it was mentioned recently in Discord: https://discord.com/channels/836637622432170028/836637623048601633/threads/1075767218433962074

Again, the diffs look kind of crazy, but I simply wrapped the tests in unless scaffolding_things_disabled? ... end blocks so they're skipped with HIDE_THINGS is set to "true".