[Archive Migration] x-pack-banners/multispace

[wayneseymour]

Summary

I've the before() fn loading the new kbn archive containing one config, for the default space. Then, the fn creates a new space and changes the ui settings of this new space, w/o using an archive.

Helps with: https://github.com/elastic/kibana/issues/102552

Bugs Filed: https://github.com/elastic/kibana/issues/140307 https://github.com/elastic/kibana/issues/140309

[elasticmachine]

Pinging @elastic/kibana-qa (Team:QA)

[LeeDr]

@wayneseymour I don't think that kbn_archive really does anything for this test. It only contains a config saved object and Kibana would have already created a config object when it starts (and will ignore the one you load).

[LeeDr]

I'm trying it locally without the kbn_archive

[LeeDr]

Interesting that this test won't pass for me locally. It fails here because the save button doesn't appear on the page after changing the banner text in advanced settings;

await PageObjects.settings.setAdvancedSettingsTextArea(
        'banners:textContent',
        'default space banner text'
      );

[LeeDr]

I'm also getting tons of this error in the server log while running the test. I don't see any reason for this test to change the default route for the space. It doesn't seem to have anything to do with the test.

 proc [kibana] [2022-07-07T14:09:55.579-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:09:55.579-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:09:55.847-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:10:00.326-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 info [o.e.x.s.a.r.TransportPutRoleAction] [ftr] updated role [system_indices_superuser]
 info [o.e.x.s.a.u.TransportPutUserAction] [ftr] updated user [system_indices_superuser]
 info [o.e.x.s.a.u.TransportPutUserAction] [ftr] added user [test_user]
 proc [kibana] [2022-07-07T14:11:38.164-05:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic)
 proc [kibana] [2022-07-07T14:11:38.837-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:38.889-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:40.261-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:40.262-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:40.262-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:40.263-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:50.436-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:50.458-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..

[LeeDr]

I'm not sure how the test passes in CI, but locally it doesn't seem to work as the tests describe. The difference is mainly on the last test 'displays the global banner on the login page' where instead of displaying the global banner, it actually displays the default space banner.

Steps to reproduce:

1. Start Elasticsearch and Kibana with node scripts/functional_tests_server.js --config x-pack/test/banners_functional/config.ts (the config file sets '--xpack.banners.placement=top', '--xpack.banners.textContent="global banner text" )
2. log in to Kibana as elastic user. I see the global banner text as expected. I'm not really why I see the space selector since there's only the default space at this time.
3. click on the default space
4. click on the D and Manage spaces and create a new space space1
5. go to Advanced Settings (still in the default space)
6. change banners:textContent to default space banner, save changes, reload page, see the new banner text
7. switch to space1, see global banner text
8. logout and log back in
9. on the space selector page, the banner is default space banner not global banner text

Either this is a bug, or the test is incorrect?

[LeeDr]

It appears that these tests are not running in buildkite. If you go to the passing buildkite job and look at

Pick Test Group Run Order. .buildkite/scripts/steps/test/pick_test_group_run_order.sh

and then at ftr_run_order.json

and then look for the test pathx-path/test/banners_functional it's not there.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[wayneseymour]

buildkite doesn't seem to be running these tests. There's some ftr config somewhere that it's probably missing from.

It's in the section here labeled # Configs that exist but weren't running in CI when this file was introduced

https://github.com/elastic/kibana/blob/main/.buildkite/ftr_configs.yml#L52

Yeah, now I see it is defined within the disabled stanza of ftr_configs.yml

[LeeDr]

Looks like @pgayvallet added these tests in https://github.com/elastic/kibana/pull/94449 a little over a year ago. Things have change since then quite a bit from Jenkins to Buildkite and how we run tests. But I think there still would have been some config file change to add the banners_functional tests to CI that I don't see in that PR. So I think these never ran in CI. The settings_page method setAdvancedSettingsTextArea was also added in that PR and doesn't appear to be used anywhere else. I had problems with it locally but I think I have a fix for it (sending the return key to it after entering the text). And then there's the actual functionality in the product doesn't seem to match the test descriptions. It could be a bug.

[pgayvallet]

@LeeDr sorry, I see you added our team's label to the PR. What exactly is expected from us here?

And then there's the actual functionality in the product doesn't seem to match the test descriptions. It could be a bug.

Mind elaborating?

[LeeDr]

@pgayvallet It looks like you wrote these tests but didn't get them added to CI so they've never been running. And meanwhile things have changed enough that they don't pass. Your team still owns these tests. Maybe the functionality is covered by other tests already?

The other issue with the tests I described in this earlier comment https://github.com/elastic/kibana/pull/135783#issuecomment-1178178075

[wayneseymour]

@elasticmachine merge upstream

[LeeDr]

@pgayvallet would you mind pulling this PR locally and running it to see if it passes for you?

[LeeDr]

You can see that these tests in this PR and not run in CI in 2 ways.

1. Go to the FTR Configs link about to get to the Buildkite job, then to the Pick Test Group Run Order pipeline step, open ftr_run_order.json and search for x-pack/test/banners_functional. It's not there which means it isn't going to be run.
2. go to https://github.com/elastic/kibana/blob/main/.buildkite/ftr_configs.yml#L52 (the line number may have changed) but you'll find it's excluded. But if you add it to the enabled list, I think it will fail.

[pgayvallet]

go to https://github.com/elastic/kibana/blob/main/.buildkite/ftr_configs.yml#L52 (the line number may have changed) but you'll find it's excluded. But if you add it to the enabled list, I think it will fail.

Yea, you're right, it's in the disabled list.

@wayneseymour mind moving x-pack/test/banners_functional/config.ts from https://github.com/elastic/kibana/blob/main/.buildkite/ftr_configs.yml#L53 to the enabled list? We'll see that way if it passes properly on CI.

[wayneseymour]

@wayneseymour mind moving x-pack/test/banners_functional/config.ts from https://github.com/elastic/kibana/blob/main/.buildkite/ftr_configs.yml#L53 to the enabled list? We'll see that way if it passes properly on CI.

No problem at all! :)

[wayneseymour]

This should be reverted once https://github.com/elastic/kibana/pull/140688 get merged

I could have sworn this test was a duplicate. Am I crazy? lol

[pgayvallet]

Can you change back from global_banner_text to global banner text as was originally in the test?

Nope. See https://github.com/elastic/kibana/pull/140688#discussion_r971632190

[LeeDr]

Can you change back from global_banner_text to global banner text as was originally in the test?

Nope. See #140688 (comment)

Couldn't this PR just add the double-quotes here and include in this PR? https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/bin/scripts/kibana#L29

Or we want to do that in a separate PR?

[pgayvallet]

Couldn't this PR just add the double-quotes here and include in this PR?

We can, yes (even if I'm not sure to see the issue using a string without whitespaces in our FTR tests)

[kibana-ci]

:green_heart: Build Succeeded

• Buildkite Build
• Commit: 896249c4e7765f9b05394e6cab5080087309ba8e

Metrics [docs]

✅ unchanged

History

• :green_heart: Build #75539 succeeded 77da79d2655edf7c4df2ea21ccb26b656efb0c9f
• :green_heart: Build #75478 succeeded 31c211a5e3be996f65535dbf0be2e3d7999c993e
• :broken_heart: Build #75350 failed 4a689f361db27caa12e45464192be42cfd24bd67
• :broken_heart: Build #75238 failed 010c0a5dc3efd3f8e32655c3c5212d0aefd53a64
• :green_heart: Build #74799 succeeded ca19e27544679efd4460bf7b156369234a850de5
• :green_heart: Build #74350 succeeded b55c3c6f48f9d105f649261025f40973c12109ed

To update your PR or re-run it, just comment with: @elasticmachine merge upstream

cc @wayneseymour