onyx icon indicating copy to clipboard operation
onyx copied to clipboard

Plugin request: HDFS

Open MichaelDrogalis opened this issue 10 years ago • 3 comments

This plugin should be capable of reading off the Hadoop file system and writing segments back to it. The point of input and partitioning should be a single file, and the partitioning will happen over the byte sequence representing the file distributed over blocks in the cluster.

MichaelDrogalis avatar Aug 09 '14 03:08 MichaelDrogalis

@MichaelDrogalis - Is there any effort in this area that you are aware off?

anujsrc avatar Apr 24 '17 11:04 anujsrc

Not that I know of, but with the new plugin interface now stable and final, this should be really easy to implement. Should look almost exactly the same as the Flink HDFS connector.

On Monday, April 24, 2017, Anuj Kumar [email protected] wrote:

@MichaelDrogalis https://github.com/MichaelDrogalis - Is there any effort in this area that you are aware off?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/onyx-platform/onyx/issues/7#issuecomment-296629548, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4FSQqcb9HQg2Nl9kbtTLJsAR1xj8JUks5rzImqgaJpZM4CVuki .

MichaelDrogalis avatar Apr 24 '17 14:04 MichaelDrogalis

Ok, got it.

anujsrc avatar Apr 24 '17 14:04 anujsrc