jitsu
jitsu copied to clipboard
Support Array types in Destinations
Problem
At present Jitsu saves fields of array type as strings into destinations. Some destinations have JSON column types (e.g. Postgres has jsonb). So Jitsu can support typing.ARRAY format and map it to jsonb in Postgres case and other JSON types in other destinations. It helps users to use such columns in SQL queries in a more convenient way.
Solution
- Add
typing.ARRAYto Jitsu types tree (typing/datatype.goandtyping/converter.go). - Add parsing rules in
schema/flattener.goandtyping/datatype.go TypeFromValue(). - Enrich all destination adapters with typing.ARRAY -> destination type mappings (e.g.
adapters/postgres.go SchemaToPostgres) - Add tests
Clickhouse also has array type columns and can be very convenient to store part of fields/subfields in an array column type.