v6d icon indicating copy to clipboard operation
v6d copied to clipboard

Published 20 hours ago • verified publisher icon v6d-io

HDF5 driver to interact with HDF5 format with data structures supported by vineyard

Open sighingnow opened this issue 2 years ago • 9 comments

Describe your problem

TBF

Additional context

Add any other context about the problem here.

sighingnow avatar May 20 '22 04:05 sighingnow

Hi @sighingnow, is the issue still available. I would like to work on it.

2022H1030042G avatar Sep 09 '22 08:09 2022H1030042G

Hi @AdwitiyaMishra,

This feature is still on our todo list and is a missing part of our I/O drivers. I would be great if you could take some effort on this ticket and free to comment here if you need any further information or guidance!

sighingnow avatar Sep 09 '22 10:09 sighingnow

Thanks for reverting @sighingnow. Can you give some guidance on:

  1. How do I get started working on this issue and what it's all about? Give some background knowledge.
  2. What are the prerequisites I need to know, like technologies in order to solve it?
  3. Lastly it'd be great if you could assign it to me.

2022H1030042G avatar Sep 09 '22 11:09 2022H1030042G

@sighingnow, I'm going to start working on it, so please do not assign it to anyone else. Thanks.

2022H1030042G avatar Sep 12 '22 12:09 2022H1030042G

Hi @2022H1030042G,

You could find some description about the design space of IO drivers in vineyard in https://v6d.io/notes/io-drivers.html

The existing implementation for supported IO rontines located in https://github.com/v6d-io/v6d/tree/main/modules/io and you may found some existing adaptors (e.g., adaptors for ORC and CSV) in https://github.com/v6d-io/v6d/tree/main/modules/io/python/drivers/io/adaptors.

sighingnow avatar Sep 13 '22 01:09 sighingnow

@sighingnow Basically, I need to perform file operations on HDF5 file formats? and this is where : https://github.com/v6d-io/v6d/tree/main/modules/io/python/drivers/io/adaptors , I need to perform these changes ?

2022H1030042G avatar Sep 26 '22 15:09 2022H1030042G

@sighingnow is the issue resolved? and if it isn't then pls assign this issue to me. Thanks

Nageshbansal avatar Nov 16 '22 13:11 Nageshbansal

@sighingnow @Nageshbansal I'm working on it.

2022H1030042G avatar Nov 16 '22 13:11 2022H1030042G

Hi @2022H1030042G,

How's it going since your last comments? We recently add the parquet and ORC support to the I/O modules and I think that would be a good reference for adding HDF5 support: https://github.com/v6d-io/v6d/commit/69108db75657bb37b2f6d1cf02fdebc7de781db9

Just feel free to ping us under this issue or in our slack channel if you have any further detailed questions!

sighingnow avatar Nov 17 '22 01:11 sighingnow