featurebase icon indicating copy to clipboard operation
featurebase copied to clipboard

Allowing arbitrary data in PQL strings

Open yuce opened this issue 7 years ago • 0 comments

Description

Currently PQL strings don't allow arbitrary data/text. For example backslash (\) and carriage return (\n) are not allowed., e.g., the following test fails (based on the test at line 77 of parser_test.go:

	t.Run("ArgumentsOnly", func(t *testing.T) {
		q, err := pql.ParseString("MyCall( key= value, foo='bar', age = 12 , bool0=true, bool1=false, x=null, escape='\\escape\n\\\"\"'  )")
		if err != nil {
			t.Fatal(err)
		} else if !reflect.DeepEqual(q.Calls[0],
			&pql.Call{
				Name: "MyCall",
				Args: map[string]interface{}{
					"key":    "value",
					"foo":    "bar",
					"age":    int64(12),
					"bool0":  true,
					"bool1":  false,
					"x":      nil,
					"escape": "\\escape\n",
				},
			},
		) {
			t.Fatalf("unexpected call: %#v", q.Calls[0])
		}
	})

The following PR makes that pass by slightly changing the PQL grammar: https://github.com/pilosa/pilosa/pull/1713

An alternative approach is base64 encoding strings on the client side before posting them to the server. They can be decoded back to the original strings whenever necessary . This approach has the added benefit of being able to store arbitrary binary or text data in the server.

Success criteria (What criteria will consider this ticket closeable?)

  • Decide what characters are allowed in a PQL string
  • If we decide to allow arbitrary strings, how do we do that?

yuce avatar Nov 01 '18 19:11 yuce