llama_index icon indicating copy to clipboard operation
llama_index copied to clipboard

Object index improvements

Open logan-markewich opened this issue 10 months ago β€’ 3 comments

This PR does a few things

  • Makes creating an object index easier (no need to think about mappings in most cases)
  • Adds postprocessors to object index retriever (since it is hard to use post processors with agents)
  • Makes saving an object index to a backend (qdrant, chroma, etc.) much easier
  • Makes object index nodes have static IDs (helps with persistence)
  • Updates docs/notebooks

logan-markewich avatar Mar 28 '24 22:03 logan-markewich

Check out this pull request onΒ  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@jerryjliu we can add an option for that, but that really only makes sense for objects you cam define on the fly (I.e. a query engine tool). This wouldn't really work for arbitrary function tools

The main advantage of backing by a vector store is so that re-creating the index is a no-op πŸ‘πŸ»

logan-markewich avatar Mar 29 '24 03:03 logan-markewich

@jerryjliu we can add an option for that, but that really only makes sense for objects you cam define on the fly (I.e. a query engine tool). This wouldn't really work for arbitrary function tools

The main advantage of backing by a vector store is so that re-creating the index is a no-op πŸ‘πŸ»

yeah that's fair. at small scales it doesn't really matter. it does somewhat negate the value of vector store support

doesn't have to be in this PR! to unblock

jerryjliu avatar Mar 29 '24 03:03 jerryjliu

@jerryjliu Added FnNodeMapping and an example + test πŸ‘πŸ» Should be good to go

logan-markewich avatar Mar 31 '24 17:03 logan-markewich