Add support for bit strings and hex strings

[crericha]

Work in progress. Submitting as a PR for feedback. Comments/questions will be inline.

[alippai]

Can we separate bit strings from binary values (number) and hex strings from hex numbers resp? There are dialects (eg. Vertica) where formatting the query results in type change now (from hex number to bit string)

[crericha]

Can we separate bit strings from binary values (number) and hex strings from hex numbers resp? There are dialects (eg. Vertica) where formatting the query results in type change now (from hex number to bit string)

To verify my understanding, are you saying 0xFF (hex number) and X'FF' (hex string) should be handled differently?

[alippai]

Can we separate bit strings from binary values (number) and hex strings from hex numbers resp? There are dialects (eg. Vertica) where formatting the query results in type change now (from hex number to bit string)

To verify my understanding, are you saying 0xFF (hex number) and X'FF' (hex string) should be handled differently?

Yes, exactly. String printed as string, number as number, but not doing an opinionated conversion on parse level (as far as I understand the code, later in the generator I won't know if it was a string or number)

Edit: I know some dialects won't differentiate between the two and using only one type of output is actually useful

[crericha]

Can we separate bit strings from binary values (number) and hex strings from hex numbers resp? There are dialects (eg. Vertica) where formatting the query results in type change now (from hex number to bit string)

To verify my understanding, are you saying 0xFF (hex number) and X'FF' (hex string) should be handled differently?

Yes, exactly. String printed as string, number as number, but not doing an opinionated conversion on parse level (as far as I understand the code, later in the generator I won't know if it was a string or number)

Edit: I know some dialects won't differentiate between the two and using only one type of output is actually useful

I spoke with Toby and the summary is that since Vertica isn't a supported dialect (at least currently), I'm not going add complexity to distinguish between string and value. If it is needed in the future, it will be pretty straight-forward to add since the tokenizer can already distinguish.