elasticsearch-labs

elasticsearch-labs copied to clipboard

Iclude code to blog using-nvidia-nim-with-elasticsearch-vector-store

[]

Addressed supporting blog for https://github.com/elastic/search-labs-elastic-co/pull/643

salgado

Chunking Ingestion Notebooks: update colab links

[]

update the colab links to work.

joemcelroy

Adds Cohere & Elasticsearch tutorial notebook

[]

## Overview Related to https://github.com/elastic/search-docs-team/issues/103. This PR adds a Jupyter notebook that contains an end-to-end example of all the operations that currently Cohere and Elasticsearch are capable of together. ###...

szabosteve

Added jupyter notebook for Geospatial Blog

[{"_id":"6710e659eedbdc0cb3101766","body":"Hi there,\r\n\r\nTheres an issue rendering the notebook (https:\/\/github.com\/elastic\/elasticsearch-labs\/blob\/c9d54687840cac94f94dffa5ac4e4e76a9c7f2ce\/notebooks\/search\/09-geospatial-search.ipynb) - looks like you have images in the notebook that doesn't exist.\r\n\r\nAlso as this is for content for blog, could you add it in the supporting-blog-content folder instead please?\r\n","issue_id":1715320166243,"origin_id":2092792732,"user_origin_id":49480,"create_time":1714734600,"update_time":1714734600,"id":1729160793924,"updated_at":"2024-10-17T10:26:33.923000Z","created_at":"2024-10-17T10:26:33.923000Z"},{"_id":"6710e659eedbdc0cb3101767","body":"@philippkahr I think you've got a formatting issue still on the notebook as it's still not building:\r\n\r\n> <unknown>:11: SyntaxWarning: invalid escape sequence '\\.'\r\n<unknown>:11: SyntaxWarning: invalid escape sequence '\\.'\r\n<unknown>:11: SyntaxWarning: invalid escape sequence '\\.'\r\nerror: cannot format supporting-blog-content\/geospatial-llm\/09-geospatial-search.ipynb: Cannot parse: 37:46: print(f\"Distance between AirBnB and {hit[\"_source\"][\"name\"]}: {hit[\"sort\"][0]} km\")","issue_id":1715320166243,"origin_id":2137683364,"user_origin_id":74931905,"create_time":1716996167,"update_time":1716996167,"id":1729160793927,"updated_at":"2024-10-17T10:26:33.927000Z","created_at":"2024-10-17T10:26:33.927000Z"}]

Hi, this should add the geospatial Jupyter Notebook for the accompanying blog post.

philippkahr

Changing examples to use hosts instead of Cloud ID

[]

Most of the examples in this repo use Could ID instead of hosts, while chatting with @miguelgrinberg and @joemcelroy we noticed that we should probably update them. What is in...

JessicaGarson

Unable to run chatbot-rag-app using Azure OpenAI model

[{"_id":"6710e667ad3c4128c711bb7a","body":"The error that you get is this:\r\n\r\n```\r\nopenai.error.InvalidRequestError: Resource not found\r\n```\r\n\r\nThis would suggest that the configuration values that you provided are incorrect. I have just re-tested Azure to confirm that it works. These are my settings:\r\n\r\n```\r\nLLM_TYPE=azure\r\nOPENAI_VERSION=2023-03-15-preview\r\nOPENAI_BASE_URL=https:\/\/ent-search-demo.openai.azure.com\/\r\nOPENAI_ENGINE=ent-search-gpt4\r\nOPENAI_API_KEY=<redacted>\r\n```\r\n\r\nDo you want to try with these? I'm not sure if the version and URL are current or out of date, but you can play with the settings after you get it to work.","issue_id":1715320166308,"origin_id":2009689948,"user_origin_id":2715854,"create_time":1710944442,"update_time":1710944442,"id":1729160807023,"updated_at":"2024-10-17T10:26:47.022000Z","created_at":"2024-10-17T10:26:47.022000Z"},{"_id":"6710e667ad3c4128c711bb7b","body":"closing this for now. feel free to re-open if you still unable to resolve it :)","issue_id":1715320166308,"origin_id":2035442449,"user_origin_id":49480,"create_time":1712173460,"update_time":1712173460,"id":1729160807026,"updated_at":"2024-10-17T10:26:47.026000Z","created_at":"2024-10-17T10:26:47.026000Z"},{"_id":"6710e667ad3c4128c711bb7c","body":"Ok, looking to reopen this:\r\n\r\nI've had some time to re-attempt this and I'm still unable to get the azure openai mode to work\r\n\r\nhere's my .env LLM env vars:\r\n\r\nLLM_TYPE=azure\r\nOPENAI_VERSION=0613\r\nOPENAI_BASE_URL=https:\/\/sa-openai.openai.azure.com\/\r\nOPENAI_API_KEY=<redacted>\r\nOPENAI_ENGINE=gpt-4\r\n\r\nHere's the model resource details from the azure-cli:\r\n {\r\n \"kind\": \"OpenAI\",\r\n \"model\": {\r\n \"baseModel\": null,\r\n \"callRateLimit\": null,\r\n \"capabilities\": {\r\n \"FineTuneTokensMaxValue\": \"2000000000\",\r\n \"FineTuneTokensMaxValuePerExample\": \"8192\",\r\n \"chatCompletion\": \"true\",\r\n \"scaleType\": \"Standard\"\r\n },\r\n \"deprecation\": {\r\n \"fineTune\": \"2024-09-30T00:00:00Z\",\r\n \"inference\": \"2024-09-30T00:00:00Z\"\r\n },\r\n \"finetuneCapabilities\": null,\r\n \"format\": \"OpenAI\",\r\n \"isDefaultVersion\": true,\r\n \"lifecycleStatus\": \"GenerallyAvailable\",\r\n \"maxCapacity\": 3,\r\n \"name\": \"gpt-4\",\r\n \"skus\": [\r\n {\r\n \"capacity\": {\r\n \"default\": 10,\r\n \"maximum\": 10000,\r\n \"minimum\": null,\r\n \"step\": null\r\n },\r\n \"deprecationDate\": \"2024-09-30T00:00:00+00:00\",\r\n \"name\": \"Standard\",\r\n \"rateLimits\": [\r\n {\r\n \"count\": 1.0,\r\n \"key\": \"request\",\r\n \"renewalPeriod\": 10.0,\r\n \"rules\": null\r\n },\r\n {\r\n \"count\": 1000.0,\r\n \"key\": \"token\",\r\n \"renewalPeriod\": 60.0,\r\n \"rules\": null\r\n }\r\n ],\r\n \"usageName\": \"OpenAI.Standard.gpt-4\"\r\n },\r\n {\r\n \"capacity\": {\r\n \"default\": null,\r\n \"maximum\": 30000,\r\n \"minimum\": 100,\r\n \"step\": 100\r\n },\r\n \"deprecationDate\": \"2024-09-30T00:00:00+00:00\",\r\n \"name\": \"ProvisionedManaged\",\r\n \"rateLimits\": [\r\n {\r\n \"count\": 20.0,\r\n \"key\": \"request\",\r\n \"renewalPeriod\": 10.0,\r\n \"rules\": null\r\n }\r\n ],\r\n \"usageName\": \"OpenAI.ProvisionedManaged.gpt-4\"\r\n }\r\n ],\r\n \"source\": null,\r\n \"systemData\": {\r\n \"createdAt\": \"2023-06-19T00:00:00+00:00\",\r\n \"createdBy\": \"Microsoft\",\r\n \"createdByType\": \"Application\",\r\n \"lastModifiedAt\": \"2023-06-19T00:00:00+00:00\",\r\n \"lastModifiedBy\": \"Microsoft\",\r\n \"lastModifiedByType\": \"Application\"\r\n },\r\n \"version\": \"0613\"\r\n },\r\n \"skuName\": \"S0\"\r\n }\r\n","issue_id":1715320166308,"origin_id":2083514602,"user_origin_id":93660701,"create_time":1714419287,"update_time":1714419287,"id":1729160807030,"updated_at":"2024-10-17T10:26:47.030000Z","created_at":"2024-10-17T10:26:47.030000Z"},{"_id":"6710e667ad3c4128c711bb7d","body":"Hey\r\nAre u sure about enigne-name='gpt-4'?\r\nFor Azure `OPENAI_ENGINE ` means deployment name in Azure not a basic openAI model.\r\nAnd `OPENAI_VERSION` is supposed to be like`The API version to use for this operation. This follows the YYYY-MM-DD format` according to Azure API\r\nCould you check those values?\r\n[Link to Azure Open AI authentication](https:\/\/learn.microsoft.com\/en-us\/azure\/ai-services\/openai\/reference#authentication)","issue_id":1715320166308,"origin_id":2086295390,"user_origin_id":17390745,"create_time":1714499935,"update_time":1714499967,"id":1729160807037,"updated_at":"2024-10-17T10:26:47.037000Z","created_at":"2024-10-17T10:26:47.037000Z"}]

I attempted to run the chatbot-rag-app using an Azure OpenAI model, and found that when I entered a question in the prompt box in the GUI, the service would never...

thaddeuselastic

Guides and examples for 2nd phase retrieval

[{"_id":"6710e65334eadd792810ff78","body":"Great suggestion.\r\n\r\nSimilar to how embedding pipelines can be added: https:\/\/www.elastic.co\/guide\/en\/machine-learning\/current\/ml-nlp-text-emb-vector-search-example.html\r\n\r\nor here:\r\n\r\nhttps:\/\/www.elastic.co\/search-labs\/how-to-deploy-nlp-text-embeddings-and-vector-search\r\n\r\nIt would be cool to utilize the machine learning instance for re-ranking. Similarly, you could bring your own model via hugging face by installing it through the Eland Docker agent (as in the later link).","issue_id":1715320166333,"origin_id":1766256957,"user_origin_id":55871537,"create_time":1697543370,"update_time":1697543370,"id":1729160787408,"updated_at":"2024-10-17T10:26:27.408000Z","created_at":"2024-10-17T10:26:27.408000Z"},{"_id":"6710e65334eadd792810ff79","body":"cc @joshdevins @leemthompo related to our conversation around guides and documentation for reranking ","issue_id":1715320166333,"origin_id":2021577847,"user_origin_id":13574083,"create_time":1711492101,"update_time":1711492101,"id":1729160787413,"updated_at":"2024-10-17T10:26:27.413000Z","created_at":"2024-10-17T10:26:27.413000Z"},{"_id":"6710e65334eadd792810ff7a","body":"Yeah we should coordinate on this. We have some plans for documentation around this as well so let's make sure they are all talking about the same things.","issue_id":1715320166333,"origin_id":2023094377,"user_origin_id":181622,"create_time":1711554085,"update_time":1711554085,"id":1729160787418,"updated_at":"2024-10-17T10:26:27.418000Z","created_at":"2024-10-17T10:26:27.418000Z"}]

Providing examples on how to re-rank using: - bi-encoders and cross-encoders - Learn to rank via feature extractors - boosting on attributes like dates, popularity

joemcelroy

Create a pointer to code for dev genai workshops

[{"_id":"6710e660ad3c4128c711bb78","body":"@ElishevaStern FYI ","issue_id":1715320166361,"origin_id":2021576614,"user_origin_id":13574083,"create_time":1711492028,"update_time":1711492028,"id":1729160800543,"updated_at":"2024-10-17T10:26:40.543000Z","created_at":"2024-10-17T10:26:40.543000Z"},{"_id":"6710e660ad3c4128c711bb79","body":"I have yet to turn my attention back to this. It's coming","issue_id":1715320166361,"origin_id":2021633830,"user_origin_id":945150,"create_time":1711495584,"update_time":1711495584,"id":1729160800546,"updated_at":"2024-10-17T10:26:40.546000Z","created_at":"2024-10-17T10:26:40.546000Z"}]

in the spirit of opennes

derickson

Wrong fields used in the filter and aggregation

[{"_id":"6710e64f34eadd792810ff74","body":"@jeffvestal Will you be looking at this? The corresponding blog article needs to be fixed as well, it appears.","issue_id":1715320166381,"origin_id":1829618777,"user_origin_id":2715854,"create_time":1701170297,"update_time":1701170297,"id":1729160783905,"updated_at":"2024-10-17T10:26:23.904000Z","created_at":"2024-10-17T10:26:23.904000Z"},{"_id":"6710e64f34eadd792810ff75","body":"@jeffvestal just a ping on this one","issue_id":1715320166381,"origin_id":2021577254,"user_origin_id":13574083,"create_time":1711492067,"update_time":1711492067,"id":1729160783913,"updated_at":"2024-10-17T10:26:23.912000Z","created_at":"2024-10-17T10:26:23.912000Z"},{"_id":"6710e64f34eadd792810ff76","body":"Hey 11 months later\nSorry, this slipped way by me\n\nfixed in https:\/\/github.com\/elastic\/elasticsearch-labs\/pull\/339","issue_id":1715320166381,"origin_id":2403300314,"user_origin_id":53237856,"create_time":1728503042,"update_time":1728503042,"id":1729160783919,"updated_at":"2024-10-17T10:26:23.919000Z","created_at":"2024-10-17T10:26:23.919000Z"}]

For the notebook at https://github.com/elastic/elasticsearch-labs/blob/main/supporting-blog-content/vector-search-implementation-guide-api/vector_search_implementation_guide_api.ipynb, "my_metadata" is a text field, and it should not be used in the term filter, and it is the same for the aggregation. The correct...

liu-xiao-guo

chatbot-rag-app does not work for self-signed (local development)

[{"_id":"6710e65028465553fb1048b4","body":"Thanks! For now we have focused on two scenarios: cloud + local instance with security turned off. \r\n\r\nAn incremental improvement will be to add a heading for when you need to use the self-signed cert and allow the developer to customise the code to accommodate it. Happy for you to add that heading to the readme in a PR.\r\n\r\nWill keep this issue open in case there are more developers who want this support. If you do, please react \/ comment on the issue!","issue_id":1715320166395,"origin_id":1851587514,"user_origin_id":49480,"create_time":1702372302,"update_time":1702372302,"id":1729160784124,"updated_at":"2024-10-17T10:26:24.124000Z","created_at":"2024-10-17T10:26:24.124000Z"}]

For self-signed environment, the code does not work. we need to set the certificate in order to establish a connection with Elasticsearch. There are two places to change: 1) index_data.py...

liu-xiao-guo