cdp-backend
cdp-backend copied to clipboard
Better method for finding/creating context span of an IndexedEventGram (in event index pipeline)
Describe the Bug
IndexedEventGram's context_span
provides a surrounding context of the gram. The current method to find/create the context span is not ideal. Sometimes context span doesn't entirely contain the gram.
Expected Behavior
The context span should contain the gram in its entirety.
In the intermediary step of creating n grams, keep track of the gram's start index (and maybe end index) in the original sentence so that a context span could be created from these indices.