io icon indicating copy to clipboard operation
io copied to clipboard

Support Hadoop Compatible Filesystem interface using config instead of hard-code

Open zuston opened this issue 4 years ago • 4 comments

Why

In our internal env, we develop a new filesystem implementing HCFS. To serve for users better, we hope users could use tf dataset api to read new filesystem storage like before. So it's necessary to support HCFS in tensorflow io.

zuston avatar Oct 14 '21 03:10 zuston

@yongtang @burgerkingeater @oliverhu Any ideas on it? If this issue is OK, i can finish it.

zuston avatar Oct 14 '21 03:10 zuston

@zuston I think this can be very helpful to the community. Contributions are welcomed 👍

yongtang avatar Oct 14 '21 13:10 yongtang

We are having similar issues to support federated clusters with a different schema! Sounds great, please create a mini design before you start?

oliverhu avatar Oct 14 '21 14:10 oliverhu

@oliverhu Got it. Maybe PR will be submitted later after mini design attached here.

zuston avatar Oct 15 '21 06:10 zuston