Index document is not returned in search as is

[kernel164]

Document gets modified when it is returned from search.

E.g.

Indexed Document

ID: doc1
Properties: map[string]interface {}{"body":"foo bar", "date":1606746899, "title":"Hello world2"}
Payload: []byte(___encoded_payload___)

Search Result Document

ID: doc1
Properties: map[string]interface {}{"__payload":"\x01\x001\xff\x81\x03\x01\x01\aMyIndex\x01\xff\x82\x00\x01\x03\x01\x04Date\x01\x04\x00\x01\x05Title\x01\f\x00\x01\x04Body\x01\f\x00\x00\x00 \xff\x82\x01\xfc\xbf\x8a\x06&\x01\fHello world2\x01\afoo bar\x00", "body":"foo bar", "date":"1606746899", "title":"Hello world2"}
Payload: nil

Two issues:

1. result document properties does't return Payload instead there is a "__payload" key in properties but the value is also not byte array, its string.
2. result document properties value type lost, all values are string. date which was integer while indexing, is string in search result.

[filipecosta90]

hi there @kernel164 can you please provide the following:

• redisearch-go version
• RediSearch module version
• If possible an an snippet of the code you use to ingest/query?

[kernel164]

version:

github.com/RediSearch/redisearch-go v1.0.1
github.com/gomodule/redigo v1.8.3

sample code:

func main() {
	c := redisearch.NewClient("localhost:6379", "myIndex")
	defer c.Drop() // cleanup
	sc := redisearch.NewSchema(redisearch.DefaultOptions).
		AddField(redisearch.NewTextField("body")).
		AddField(redisearch.NewTextFieldOptions("title", redisearch.TextFieldOptions{Weight: 5.0, Sortable: true})).
		AddField(redisearch.NewNumericField("date"))
	if err := c.CreateIndex(sc); err != nil {
		log.Fatal(err)
	}
	doc := redisearch.NewDocument("doc1", 1.0)
	doc.Set("title", "Hello world").
		Set("body", "foo bar").
		Set("date", time.Now().Unix())
	// set payload
	doc1 := &doc
	doc1.SetPayload([]byte{1, 2, 3, 4, 5, 6, 7, 8, 9})
	fmt.Printf("Index Document: %#v\n", doc)
	if err := c.IndexOptions(redisearch.DefaultIndexingOptions, doc); err != nil {
		log.Fatal(err)
	}
	docs, _, err := c.Search(redisearch.NewQuery("hello world"))
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Search Results: %#v\n", docs[0])
}

[kernel164]

https://github.com/RediSearch/redisearch-go/blob/v1.0.1/redisearch/document.go#L129

[kernel164]

Will this be fixed? We are evaluating redis search and we are stuck with this issue.

[filipecosta90]

Will this be fixed? We are evaluating redis search and we are stuck with this issue.

Hi there @kernel164 , I will take a deep look at this until Monday morning and come up with an explanation/fix if required :) Thank you for bringing this up. PS: when you say evaluating RediSearch you mean v1.6 or v2.0 ( important for my analysis )?

[kernel164]

Hi @filipecosta90, When I tried, 2.0 was not released. I will try 2.0 this week, but a quick look at the code for both version have same issue.

[filipecosta90]

Hi @filipecosta90, When I tried, 2.0 was not released. I will try 2.0 this week, but a quick look at the code for both version have same issue.

Hi there @kernel164 , when I say 1.6 and 2.0 I mean https://github.com/RediSearch/RediSearch/releases/tag/v2.0.0 and not the module version. Are we refering to the same thing?

[filipecosta90]

Will this be fixed? We are evaluating redis search and we are stuck with this issue.

Hi there @kernel164 , with regards to the payload it can be fixed by including the QueryWithPayloads flag. Here's the code snipet. Change:

	docs, _, err := c.Search(redisearch.NewQuery("hello world"))

to

	docs, _, err := c.Search(redisearch.NewQuery("hello world").SetFlags(redisearch.QueryWithPayloads))

With regards to losing the property value types ( for example numeric ) let me check how we're doing it in other clients ( if we're explicitly converting ), but bottom line we can ( and should do it ) via the func (d *Document) loadFields(lst []interface{}) *Document function and based uppon the index definition. WDYT about it @MeirShpilraien ?

PS: In the meantime and since the last reply @kernel164, did you you guys encountered other blockers or just this one? If you guys needs further examples on the documentation have that kind of feedback is very precious. ( Will add examples with payload fields to the docs in the meantime )

[kernel164]

Hi @filipecosta90, When I tried, 2.0 was not released. I will try 2.0 this week, but a quick look at the code for both version have same issue.

Hi there @kernel164 , when I say 1.6 and 2.0 I mean https://github.com/RediSearch/RediSearch/releases/tag/v2.0.0 and not the module version. Are we refering to the same thing?

Yes I'm trying to check RediSearch 2.0 after looking at this blog => https://redislabs.com/blog/search-benchmarking-redisearch-vs-elasticsearch/

I didn't proceed further after this basic blocker. I will check the payload flag that you mentioned and try other things next week. Thanks.

[filipecosta90]

Thank you for the feedback @kernel164 . In the meantime I'll pursue the 2nd issue that you mentioned, that is losing property value types. cc @emmanuelkeller