Support generating `ios_uitests` without a `test_host`

[benjaminRomano]

Background Currently, we have a TestBundle created through ios_uitest which hosts our core user flow performance tests (e.g. performance test a page load, startup times, etc.). The current workflow is that whenever a change is made to the app, the TestBundle must also be recreated as the TestBundle hosts both the tests and the app itself (as defined by test_host).

For an app of our size, re-running performance tests ends up being extremely slow as we need to re-build the TestBundle every time (~1 minute) plus re-install the app (~2 minutes) just to run a performance test.

The ideal workflow, which would emulate how performance testing is done on Android with UI Automator, is that we'd ideally have an isolated TestBundle that only contains the performance tests themselves. Then the test itself hardcodes which bundle identifier to target or takes it in as a parameter. This can be done by instantiating XCUIApplication which a custom bundleIdentifier.

Feature Request

Presently, it is not possible to use ios_ui_test rule without test_host being set. Looking through the code for that rule it appears that there'd be some work required in how the xctestrun template is generating and some associated steps for that rule.

Workaround In the interim, I was able to achieve what I want here by creating effectively a no-op app ios_application target with that just renders an empty view and specifying that as the test_host, then using the approach mentioned above of constructing XCUIApplication instances with an arbitrary bundle identifier.

Embedding a no-op app doesn't add that much additional overhead so this workaround is reasonable for the time being.

[keith]

Can you clarify the specific changes you're thinking about? It's not possible to run XCUI tests without a test host, so as far as I understand the request I think the only potential solution would be to do exactly what you are doing, and providing an empty default test_host app, but in the rules instead of requiring you to create it. Is that what you're requesting?

[benjaminRomano]

Ah I was thinking it was valid to have a test bundle without a test host. If that's not the case, then I'd say just manually performing the workaround is good enough.

It'd be neat to have an effectively noop test hot generated supported out of the box but given that its not that difficult to implement externally it's probably not worth the dev time

[keith]

You can have a ios_unit_test bundle without a test_host, but not if you're using XCUIApplication, so maybe that could work for your use case?

[benjaminRomano]

I'm using XCUIApplication(:bundleIdentifier) to target arbitrary apps.

I could try manually stripping out the test host after the fact and modify xctestrun to see if that approach still ends up working out