manticoresearch icon indicating copy to clipboard operation
manticoresearch copied to clipboard

Add the use of local models

Open nepridumalnik opened this issue 3 months ago • 0 comments

Proposal:

Add the ability to use local neural network models (for example, in ONNX, PyTorch, TensorFlow, or other formats) to expand the capabilities of semantic search, classification, embeddings, and other NLP tasks.

Use cases:

  • Semantic search:search documents by meaning, not just by keywords.
  • Text classification: automatic document categorization.
  • Embedding generation: vectorization of text for subsequent vector search.
  • Multimodal search is possible in the future: integration with models that process images, audio, and other data.

Possible Implementation Ideas:

  • Support for loading models via a configuration file or API.
  • Caching of inference results.
  • Support for popular formats (ONNX, TorchScript, etc.).
  • The ability to run models on the CPU / GPU.
  • Expandable interface for connecting new types of models.

Checklist:

To be completed by the assignee. Check off tasks that have been completed or are not applicable.

  • [ ] Implementation completed
  • [ ] Tests developed
  • [ ] Documentation updated
  • [ ] Documentation reviewed
  • [x] OpenAPI YAML updated and issue created to rebuild clients

nepridumalnik avatar Sep 23 '25 04:09 nepridumalnik