[Inference] Implement NL-to-ESQL task

[dgieselaar]

Implements the NL-to-ESQL task and migrates the Observability AI Assistant to use the new task. Most of the files are simply generated documentation. I've also included two scripts: one to generate the documentation, and another to evaluate the task against a real LLM.

TBD: run evaluation framework in Observability AI Assistant to ensure there are no performance regressions.

[obltmachine]

:robot: GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[dgieselaar]

@elasticmachine merge upstream

[dgieselaar]

@pgayvallet is currently working on adding gemini/bedrock adapters, I'll put this back in draft until that work has completed.

[pgayvallet]

/ci

[pgayvallet]

@dgieselaar I'm done rebasing your PR, you should be good to go

[pgayvallet]

/ci

[dgieselaar]

As @pgayvallet has concluded his work, I'm pulling this out of draft again.

[kibana-ci]

:green_heart: Build Succeeded

• Buildkite Build
• Commit: fe3596ee99fd65b03c94fa1ff66e643c37c57b6f
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-190433-fe3596ee99fd

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
inference	14	15	+1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
inference	14	39	+25

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
inference	11	13	+2

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
inference	5.2KB	5.6KB	+390.0B

Unknown metric groups

API count

id	before	after	diff
inference	16	41	+25

ESLint disabled line counts

id	before	after	diff
inference	0	1	+1

Total ESLint disabled count

id	before	after	diff
inference	2	3	+1

History

• :yellow_heart: Build #231924 was flaky 4cb642cde244da930dacd76478717630e0cd2391
• :yellow_heart: Build #231894 was flaky e71fe3b843596d88e966c3db73d54d4f27b98dc1
• :yellow_heart: Build #231847 was flaky c8864195bf918ff26e21c9c3dc435bb018283c6a
• :green_heart: Build #231803 succeeded 1b57f13ccb7d5756992458436686456a184fa4bc
• :green_heart: Build #231789 succeeded 47afdbedbb9c2831267b4da600e717c650582915

To update your PR or re-run it, just comment with: @elasticmachine merge upstream