[Infra UI] Host Anomaly Detection job creation always shows incorrect error message when there are no hosts

[roshan-elastic]

Description

When creating an host anomaly detection job in serverless, the job will always say that it has failed regardless:

User experience when creating job

API response when 'error' occurs

When trying to create again, it shows it actually works (so it didn't fail)

Video

https://github.com/elastic/kibana/assets/117740680/47f28176-f36e-432a-8cd9-88b5157eca36

Expectation

It is expected that when the job is enabled, the user should be presented with information indicating the true state (i.e. the job wasn't able to be immediately created but it is being queued to be setup) - much like what you see if you try again:

Example message when you try to recreate

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[roshan-elastic]

@smith - here is the bug. This feels like it should be a high priority in serverless to me.

WDYT?

[roshan-elastic]

@vinaychandrasekhar FYI (cc @grabowskit)

[roshan-elastic]

@chrisdistasio FYI this is something users are likely to see if they try and create anomaly detection jobs via the Infra UI in serverless...

Note : this is pretty related to the not-so-good error handling in the anomaly detection job creation workflow present in stateful too:

• [ ] https://github.com/elastic/kibana/issues/170500

It's close to top of the backlog here but realistically I can't see this being resolved in serverless before users experience it.

cc @smith @grabowskit

[neptunian]

@roshan-elastic Are you creating an ml job on an empty cluster (no metrics data)? Because you will get that error whether you are in serverless or not, if so. Not great error handling for sure but probably not a common case to be creating them when you have no data. Did you try creating in on a cluster with data?

[roshan-elastic]

Hey @neptunian - good catch! I didn't even realise that I was using a project with no hosts.

I just tried on a project with hosts and it worked fine first time - it's only when there are no hosts does it give the incorrect dialogue (and if I refresh it shows the jobs have started to be created).

I wonder if this is actually serverless specific or whether it is a generic issue? I just tried a new cloud deployment with no hosts and I think I see a similar behaviour:

Creating host anomaly detection job on an empty stateful cloud deployment

If this is the same problem, it looks like it's a been a problem for a long time for customers already so it's not something new that users haven't been experiencing before (so may affect the priority).

[neptunian]

@roshan-elastic It's not serverless specific. The error is saying it doesn't have mappings because there is no data.

[roshan-elastic]

Thanks for confirming @neptunian.

@smith - I think we can lower the priority on this given that (a) this only happens when you have no metrics data so unlikely to happen very often (b) this is not something that serverless users are likely to encounter more than anyone else.

I've moved these out of the backlog as I don't think there's an urgency to resolve these and I figure these can be prioritised once there is a PM to support you with the backlog (although in the interim, I'm happy to help)

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[roshan-elastic]

Closing - the error message will now pass through the error returned from the API