[Feature Request]: Text to DSL (Domain specific language of Elastic Search)

[bkakadiya]

Feature Description

Request to have Text to Elastic search DSL feature. This is similar to Text to SQL functionality. Additionally combine it with readers/elasticsearch.py to get search result.

• DSL Details: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html
• Possible Default prompt : https://github.com/langchain-ai/langchain/blob/v0.0.303/libs/langchain/langchain/chains/elasticsearch_database/prompts.py

This is what I am thinking:

• Ask for elastic search URL and Index name
• Ask for elastic search api key having permission on that index
• Fetch index property mapping
• Generate prompt using index property mapping asking for JSON ES DSL as output
• Use user input along with prompt and hit chat completion API
• Check the flag dsl_only, -- if its true return DSL -- if its false (default) hit elastic search and get response as well.

Reason

I was not able to find this feature. Let me know if its there.

Note that this is different from

• Using Elastic Search as a vector store.
• This is just to request DSL and hit that DSL to get ES documents (similar to text to sql

Value of Feature

• Can be useful for quick DSL query generation and getting quicker response
• Actual data will not be posted to LLM, only object index schema/property mapping will be posted to LLM

[dosubot[bot]]

🤖

Hello @bkakadiya!

I'm Dosu, a bot designed to help with issues, answer questions, and assist contributors. I'm here to help while we wait for a human maintainer to address your issue. Just consider me your friendly digital assistant in this coding journey.

Don't hesitate to ask if you have any more questions or need further assistance!

Thank you for your feature request. As per the current version of LlamaIndex (v0.9.30), there is no functionality that converts text to a Domain Specific Language (DSL). However, the LlamaIndex codebase does interact with Elastic Search, as seen in the test_elasticsearch.py file and the ElasticsearchStore class.

Your feature request for a Text to Elastic Search DSL functionality is quite detailed and would indeed be a valuable addition to the LlamaIndex repository. This feature would allow for quick DSL query generation and faster responses, as you've mentioned.

However, as a Dosu, I am unable to create pull requests or issues. I would recommend you to create a new issue detailing this feature request on the LlamaIndex repository. This way, the maintainers and contributors of the repository can review your request and potentially work on implementing this feature.

Please include as much detail as possible in your issue, including the links you've provided and the potential benefits of this feature. This will help the maintainers and contributors understand your request better and expedite the process of potentially adding this feature to LlamaIndex.

Thank you for your contribution to improving LlamaIndex. Your input is greatly appreciated.

Sources

• llama_index/tools/tool_spec/load_and_search/base.py
• tests/vector_stores/test_elasticsearch.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[bkakadiya]

This is what I am thinking:

1. Add method in ElasticsearchReader (readers/elasticsearch.py) to get elastic search index metadata fields

2. Create dsl_query.py in indices/struct_store

•   It should have NLDSLQueryEngine class 
  

•   NLDSLQueryEngine should expect ElasticsearchReader as an argument 
    	during construction it should fetch index metadata fields (as defined in #1) and keep it cached
  

•   It should have DEFAULT_DSL_TEMPLATE  similar to https://github.com/langchain-ai/langchain/blob/v0.0.303/libs/langchain/langchain/chains/elasticsearch_database/prompts.py
  

•   It should have query(user_input:str) method 
  

•   It should have clear index metadata method 
  

3. Once user invokes query

•   Fetch index metadata if its not there 
  

•   Invoke llm chat to get output 
  

[dosubot[bot]]

Hi, @bkakadiya,

I'm helping the LlamaIndex team manage their backlog and am marking this issue as stale. The issue you raised requested a feature for converting text to Elastic Search DSL, similar to the Text to SQL functionality. It seems that the feature has been resolved with a detailed response from me, who suggested creating a new issue detailing the feature request on the LlamaIndex repository. Additionally, you outlined a plan for implementing the feature, and user DennyMedirecords expressed support for the proposed plan.

Could you please confirm if this issue is still relevant to the latest version of the LlamaIndex repository? If it is, please let the LlamaIndex team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days.

Thank you for your understanding and contributions to the LlamaIndex repository.

I (Dosu)

[jaiminee]

Hi, @bkakadiya

I am looking for the exact feature of Text-to-ElasticSearch DSL. Could you please share some details on how you approached this issue?

Any reference documentation and code will be really helpful.

Thank You

[raviteja-cfx]

Hi, @bkakadiya @jaiminee

I am looking for the same feature of Text-to-ElasticSearch DSL. any update on this ? is it available now ?

Any reference documentation and code will be really helpful.

Thank You