Initial Go support

[MarkReedZ]

Added a few functions for go. If we merge this I'll finish it up.

$ go run scripts/bench.go
Contains
   38.798µs 	strings.Contains
   37.872µs 	sz.Contains
Index
   26.29µs 	strings.Index
   37.712µs 	sz.Index
IndexAny
   588.567µs 	strings.IndexAny
   45.911µs 	sz.IndexAny

[ashvardanian]

Wow, I am surprised the latency is comparable. How long is the input?

I believe for Go, it would be great to transpile our C code.

[MarkReedZ]

I'll look at transpiling. I don't expect there to be a significant improvement on these benchmarks - its probably likely to slow things down considerably from what I've read.

The strings are 1mb. Below are 10kb strings. Short strings show the same performance except for IndexAny where the go time eventually matches stringzilla.

$ go run bench.go
Contains
   607ns 	strings.Contains
   804ns 	sz.Contains
Index
   324ns 	strings.Index
   492ns 	sz.Index
IndexAny
   5.917µs 	strings.IndexAny
   733ns 	sz.IndexAny

[ashvardanian]

How about super short inputs in the range of 10-100 bytes?

[MarkReedZ]

None of the transpilers work on the stringzilla code. I don't see much information on them.

Build flags are the following by the way. -O3 slows us down.

// #cgo CFLAGS: -g -mavx2
// #include <stdlib.h>
// #include <../../include/stringzilla/stringzilla.h>
import "C"

100b

Contains
   245ns 	strings.Contains
   425ns 	sz.Contains
Index
   139ns 	strings.Index
   137ns 	sz.Index
IndexAny
   181ns 	strings.IndexAny
   275ns 	sz.IndexAny

16b

Contains
   138ns 	strings.Contains
   452ns 	sz.Contains
Index
   56ns 	strings.Index
   198ns 	sz.Index
IndexAny
   205ns 	strings.IndexAny
   262ns 	sz.IndexAny

[MarkReedZ]

Others benchmarking cgo state its 40ns per C call, but I don't see it being that slow. I'll get more of the funcs in as I'm using this in some of my go projects. We can also try manually writing a func or two in go to see how that goes.

[MarkReedZ]

@ashvardanian We do have an issue with repeated c calls. Count is faster until we match too many times. It is ~50ns per call to sz_find. We can?

A) Write Count in C so its one function call to C instead of N calls B) Write sz_find in go

Let me know and I'll go ahead. The below numbers are for a 9 byte substring to match - strings.Count is faster if the substring to search for is 4 bytes or less.

Count 1  
   55.06µs 	strings.Count
   40.692µs 	sz.Count
Count 100,000 
   1.344558ms 	strings.Count
   6.815231ms 	sz.Count

[ashvardanian]

How about we implement the count in the cGo header?