Sadayuki Furuhashi

Results 27 comments of Sadayuki Furuhashi

@sonots I chose `json` because many storage systems or SaaS services use JSON as the primary format of schema-less values. Such as postgresql, mysql, treasure data, etc. It's good for...

> Nowadays, there are several storages which accept json for schema-full values such as bigquery, vertica. Embulk's assumption is that you use schema-full columns instead of a single `json` column....

Using `.%03d.%02d` seems like the way to go. I fix LocalFileInputPlugin. I should say sorry about the confusing changes.

It's good point that each source files want to use different type of filters. In my idea above, each jobs will use global configuration + job-specific configuration. Job-specific configurations are...

#306 should be implemented first.

0xc1 is reserved to not be used anywhere. Some implementations depend on the fact that 0xc1 is never used. On the other hand, ext format family is reserved for extension.

Ext type could be used in following manner: fixarray 2-element ext-8 [a type tag that indicates that this is a sequence of N-D array’s buffer chunks] [dims...] array-16/32 [number of...

An example use of 0xc1: https://github.com/msgpack/msgpack-ruby/blob/1e35fb8a771339fc51a9a9c96e77046dfc086954/ext/msgpack/unpacker.c#L53-L61 One thing we need to think is how existent applications and tools should deal with N-D array. They could (A) deal with them as...

> 64bit machines and compilers allow to allocate dynamic or static arrays of length defined via size_t, on a 64bit machine, is SIZE_MAX=2^64-1 bytes Does it mean applications in real...

I think N-D array (or generally array of fixed-length elements) is a good idea. I have some other use cases in mind that can reduce overhead of type tag with...