Boris Lublinsky comments

Results 88 comments of


                                            Boris Lublinsky

[Feature] Enable transform() to terminate all processing of documents across all instances

Done

initial pass at a pipelining transform

These issues just magnify why this approach is not starter (in my opinion)

initial pass at a pipelining transform

This is breaking cli

[Feature] Add functionality to organise files by repository structure

I am sorry, I am missing something here. What exactly are we trying to produce here?

[Feature] Add functionality to organise files by repository structure

So basically it creates a single arrow table per repository, right? And how does it know the repository name? is it a separate column? Final question. Should it be part...

[Feature] Enable composable/pipelined transforms in the python API

I am not convinced that this is the right thing to do. It will only work if all transforms in the sequence have comparable dependencies, which is not a save...

[Feature] Enable composable/pipelined transforms in the python API

I would rather continue with the launcher, etc. 1. We do not need anything special to support configuration. Transforms already have unique prefixes, so there is no need for nesting,...

[Bug] Cannot connect to an external Ray Cluster

> So the current problems in dpk are that: > > 1. It is not possible to connect to a remote ray cluster to execute some transform (a feature which...

[Bug] Cannot connect to an external Ray Cluster

can we, please, close this

[Feature] Extend doc_id to enable the doc id has to be computed from a list of columns instead of a single content column

Is there a new requirement? Currently it is implemented the way it was required