mafsa
mafsa copied to clipboard
Impossible to use utf
It seems something wrong with encoding or decoding of utf. Consider this test:
import (
"github.com/smartystreets/mafsa"
"github.com/stretchr/testify/require"
"testing"
)
func TestRussian(t *testing.T) {
a := mafsa.New()
a.Insert("я") // I'm in russian
a.Finish()
a.Save("test")
require.True(t, a.Contains("я")) // Fine
b, err := mafsa.Load("test")
require.NoError(t, err)
require.True(t, b.Contains("я")) // Failure!!
}
func TestSpanish(t *testing.T) {
a := mafsa.New()
a.Insert("Gracías")
a.Finish()
a.Save("test")
require.True(t, a.Contains("Gracías")) // Thank you in spanish
b, err := mafsa.Load("test")
require.NoError(t, err)
require.True(t, b.Contains("Gracías")) // Failure!!
}
@mholt - Does mafsa support non-ascii characters?
@mholt Not sure but I did find any alphabet limitations here. It works with non-ascii but it seems an error during load/save process. Look at the test it works fine with buildTree and fails only after I load it from file and start to use minTree
It seems merging this PR will solve this issue https://github.com/smartystreets/mafsa/pull/8