featurebase
featurebase copied to clipboard
Allowing arbitrary data in PQL strings
Description
Currently PQL strings don't allow arbitrary data/text. For example backslash (\) and carriage return (\n) are not allowed., e.g., the following test fails (based on the test at line 77 of parser_test.go:
t.Run("ArgumentsOnly", func(t *testing.T) {
q, err := pql.ParseString("MyCall( key= value, foo='bar', age = 12 , bool0=true, bool1=false, x=null, escape='\\escape\n\\\"\"' )")
if err != nil {
t.Fatal(err)
} else if !reflect.DeepEqual(q.Calls[0],
&pql.Call{
Name: "MyCall",
Args: map[string]interface{}{
"key": "value",
"foo": "bar",
"age": int64(12),
"bool0": true,
"bool1": false,
"x": nil,
"escape": "\\escape\n",
},
},
) {
t.Fatalf("unexpected call: %#v", q.Calls[0])
}
})
The following PR makes that pass by slightly changing the PQL grammar: https://github.com/pilosa/pilosa/pull/1713
An alternative approach is base64 encoding strings on the client side before posting them to the server. They can be decoded back to the original strings whenever necessary . This approach has the added benefit of being able to store arbitrary binary or text data in the server.
Success criteria (What criteria will consider this ticket closeable?)
- Decide what characters are allowed in a PQL string
- If we decide to allow arbitrary strings, how do we do that?