Support nested fields

[smackysnacks]

We have some nested fields that we want to use within column_mapping, such as column_mapping 'user.id:user_id,user.name:name', but this code is preventing it from working:

https://github.com/fluent/fluent-plugin-sql/blob/daca62a41d1a0a725c54f997d3af7bfca7bfc46f/lib/fluent/plugin/out_sql.rb#L56-L67

Specifically record[k]. We patched this with record.dig(*k.split('.')) to support these nested fields, however this relies on Ruby >=2.3. Would you consider supporting our use case? If so, I'd be glad to help.

[repeatedly]

Since v1, we can use record_accessor helper for nest field access. So if column_mapping has nested access syntax, use it.

[AdamVB]

As a workaround you can also add a record transformer filter to "de-nest" the keys:

   <filter kubernetes.**>
      @type record_transformer
      enable_ruby
      <record>
        pod_name ${record["kubernetes"]["pod_name"]}        
      </record>
      <record>      
        container_image_id ${record["kubernetes"]["container_image_id"]}
      </record>
    </filter>

And do the column mapping without nesting: column_mapping 'time:created_at,pod_name:dbcol1,container_image_id:dbcol2,log:dbcol3

[billzhuang]

hi @repeatedly is there any sample for record_accessor use in colum_mapping?

I tried column_mapping '$.data.category:category,$.data.subCategory:subcategory,$.data.params:eventdata,$.utcLogTime:eventtimeatutc'

unforturnetely, when i checked the pg db, all field is null.